![\"Parcel](\"/images/custom-agent-harness/workbench.png\")
An agent harness is the software around an AI model that lets it work on a task. It supplies context and tools, executes the model's tool requests, feeds back the results, and controls when the agent continues or stops.
\nFor example, the model requests an order lookup. The harness returns the order, giving the model the information it needs to check inventory or ask a follow-up question.
\nHumanLayer describes a coding agent's harness as “the agent’s runtime” in Kyle's Skill Issue: Harness Engineering for Coding Agents. Cloudflare's documentation describes the work around each model turn, including prompt construction, tool handling, message persistence, and stopping decisions.
\n| Agent | Context it needs | Tools it can use | Application controls |
|---|---|---|---|
| Coding assistant | Repository files, instructions, previous work | Read files, edit code, run tests | File access, command permissions, execution limits |
| Damaged-order support assistant | Selected order, damage report, policy, saved decisions | Look up an order, check stock, propose a replacement | Case ownership, replacement eligibility, approval |
We'll build the support example with Next.js, Vercel's AI SDK, AI Elements, and PostgreSQL, then compare Sol and Luna on the same conversations. The walkthrough assumes familiarity with TypeScript and web applications.
\nAn agent harness runs the agent. An eval harness runs test conversations against it and checks the results. The example repository includes both.
\nParcel & Pine is a fictional homewares store. A support employee opens a case for Alex Morgan, whose order contained two mugs. One arrived broken.
\nThe employee types:
\n\n\nLet's replace it for them.
\n
The assistant should use the open case to check the order, policy, and stock, then save a proposal for one replacement. The employee reviews that proposal and clicks Approve. Afterward, “Let the customer know” should produce a customer reply draft based on the recorded decision.
\n
The local workbench uses fictional orders. Each case has independent stock and conversation history.
\nThe store policy requires a damage report within 30 days of delivery, enough stock, and a replacement quantity within the purchased quantity. The scenarios use a fixed reference date so an eligible example does not expire between publication and the day someone runs it.
\nApproval creates a simulated replacement record and decrements local stock. The POC does not contact customers or ship anything.
\nThe AI SDK’s ToolLoopAgent handles the loop between model calls and tool results. Our code decides what context enters that loop, which tools exist, and what those tools may change.
The SDK also has a HarnessAgent integration for established runtimes such as Pi. I used ToolLoopAgent because this application defines its own tools and workflow.
AI Elements renders the conversation, tool calls, and suggested replies. The workbench reads the order and resolution directly from the database.
\nI used GPT-6 Astra in Codex to build this POC. Sol and Luna are the models running inside the support agent; Astra was the coding agent building its harness.
\n\n\n\nwhy are models bad at harness engineering!?
\n
Kyle quoted that post and wrote:
\n\n\nmodels are VERY bad at building harnesses & agents
\n
That matches my experience here. Astra was terrible at getting the first version of this harness right.
\nI opened the case and said, “Let's replace it for them.” It asked me for the order number. The order was right there on the page, but we hadn't given it to the model.
\nThat is basic context engineering. I should not have had to discover it by using the demo.
\nThe server now loads the authorized case before every turn. It supplies the order, seeded customer report, current stock, proposal status, and replacement receipt. The browser cannot choose another case's owner or pass a fabricated approval as authoritative state.
\nIf the employee says “Correction: both mugs broke,” the assistant needs to use that new information. If an earlier assistant message incorrectly claimed that a replacement shipped, the saved business record must take precedence.
\nThe agent configuration uses prepareCall to load that context.
// Abridged wiring; see agent.ts for the instructions and implementations.\nreturn new ToolLoopAgent({\n model: supportModel(modelName),\n instructions,\n prepareCall,\n tools: scopedTools,\n stopWhen: isStepCount(8),\n maxOutputTokens: 2500,\n maxRetries: 1,\n});\nThe assistant has six tools:
\n| Tool | Job |
|---|---|
lookupOrder | Verify the linked order within the selected case |
readPolicy | Return the damaged-item replacement rules |
checkStock | Return replacement availability |
proposeReplacement | Validate and save a pending proposal |
checkResolution | Read the saved proposal and receipt status |
suggestReplies | Choose up to three messages the employee can click to submit |
Each business tool is scoped to the case on the server. The model supplies an order number or damaged quantity where needed, but it cannot select another owner or run arbitrary SQL. It has no approval or shipping tool.
\n“Draft a reply to the customer” saves typing. “Both mugs were cracked” introduces an unverified fact that becomes a user message when clicked.
\nSuggestions appear above the composer and submit through the normal chat endpoint. They do not perform approvals. Old suggestions disappear when their saved proposal status no longer matches the case.
\n![\"A](\"/images/custom-agent-harness/live-proposal.png\")
The assistant checked the order, policy, and stock. The proposal is waiting for the employee's recorded decision.
\nDex Horthy's 12-Factor Agents calls for owned control flow and human intervention. Here, the employee approves through an application endpoint, with credentials the model never receives.
\nWhen the employee clicks Approve, the server checks case ownership, locks the relevant records, and rechecks policy, quantity, and stock. It saves the replacement receipt and stock change in one PostgreSQL transaction.
\nRepeating the approval request returns the same receipt without decrementing stock again. A declined proposal cannot later be approved. These rules live in application code, where deterministic tests can verify them.
\nThen I said, “Let the customer know.” It drafted a message saying the replacement was pending human review.
\nI'm the human using the workbench. The app needed to tell me how to record my decision, not write to the customer as though we were waiting on somebody else. Customers do not need an explanation of our AI workflow.
\nWe changed the instructions to address the employee directly and keep approval guidance outside the customer draft. The later evals still found failures.
\n![\"A](\"/images/custom-agent-harness/live-approved.png\")
The assistant reads the recorded approval before drafting the customer reply. It has no tool for sending that message.
\nFor this local demo, a generated reviewer token unlocks the approval controls. A real application would use its own user authentication and authorization.
\nAstra kept reporting successful tests. The tests checked persistence, duplicate approvals, and case ownership. A live browser test even restarted PostgreSQL and reloaded the conversation.
\nBut I was still catching obvious problems by talking to the agent. It didn't know its case, misunderstood who was reviewing the replacement, and suggested checking status again instead of helping me act. These are things I expect an AI engineer to catch when designing the workflow.
\nI asked for an eval suite that judged the conversation, the tools it called, and the suggestions it made.
\nA “no proposal saved” assertion also passes when the model attempts an ineligible replacement and the application blocks it. Our tool-selection eval must flag that attempted call.
\nThe eval audit, guided by Eval Skills, documents these gaps.
\nI wanted to know whether Luna could do this job as well as Sol. We wrote ten scenarios covering missing information, corrected quantities, expired orders, unavailable stock, approval and rejection, and incorrect assistant claims in conversation history. Two scenarios carry live generated responses into a second turn. Other cases start with fixed history to test whether the model follows stale or false claims.
\nThe suite uses code checks for required and forbidden tools, prerequisite steps, quantities, and saved state. It permits the read-only checks to run in any order, including in parallel, but requires them before a proposal. Selected text checks catch known wording regressions.
\nI didn't want another model call grading every response. I had Codex write the expected behavior, then read and judge the recorded conversations. The suite itself runs code checks and saves the traces. The table separates those checks from Codex's review judgments.
\nBoth models ran through Bedrock at medium reasoning. Two repetitions produced 24 evaluated turns per model:
\n| Measure | GPT-5.6 Sol | GPT-5.6 Luna |
|---|---|---|
| Turns passing code checks | 19/24 | 20/24 |
| Tool decisions passing Codex review | 23/24 | 22/24 |
| Responses passing Codex review | 23/24 | 20/24 |
| Suggestion sets passing Codex review | 18/24 | 19/24 |
| Mean observed latency per turn | 3.08 s | 2.33 s |
Luna was faster but made more unsupported customer promises. Both models invented damage details in suggestions and attempted expired-order replacements.
\nThe per-turn report explains disagreements: a regex flagged checking stock after replenishment, which Codex accepted because the state would have changed. Paraphrased promises of future updates passed text checks but failed review.
\nWe kept the harness fixed while comparing models. Ten scenarios run twice are not enough to tell me how either model will perform in production, or whether switching saves money. Sol remains the default.
\nYou need Node.js 22.19 or newer, Docker Compose, and credentials for a supported model endpoint. From the cloned repository:
\nnpm ci\nnpm run setup\nAdd your provider credential to .env.local, then start PostgreSQL and the app:
npm run db:up\nnpm run db:migrate\nnpm run dev\nOpen http://localhost:3000 and choose the broken-mug case. Try “Let's replace it for them.” To record approval, unlock the reviewer controls with OPERATOR_TOKEN from .env.local, then click Approve. Keep the token out of chat. Try “Let the customer know” after approval and inspect the draft.
PostgreSQL runs in Docker Compose; Next.js runs locally. The project does not require Vercel hosting. Any compatible PostgreSQL provider can support an adapted deployment.
\nThe recorded model runs used Bedrock's OpenAI-compatible Responses endpoint. Direct OpenAI is configurable, but its generation path was not successfully verified in this development environment. The README documents provider setup.
\nRun the comparison with explicit model IDs:
\nnpm run eval:compare -- --models gpt-5.6-sol,gpt-5.6-luna --repeats 2\nThis makes paid model calls and saves each attempt, including execution errors, under a new .evals/ directory. The artifacts include source and golden-set snapshots, tool arguments and results, state, token counts, and latency.
You can rerun the code checks on the published results without calling a model:
\nnpm run eval:compare -- --score-only docs/evals/results/sol-luna-2026-09-13\nThat command currently exits nonzero because the recorded outputs contain failures.
\nI gave Astra instructions and examples for building this harness. It still missed the case context, confused the approval workflow, and treated passing tests as proof that the support experience worked. These were basic requirements I had to catch myself.
\nI want the model to run prompts, try variations, and compare which version calls the right tools. That is useful work to delegate. I should design the tools, the system context, and the rules around them. I need to read the traces and use the application myself. A human needs to stay involved in the design and testing, not just click Approve inside the finished product.
\nAt Astra's launch, Greg Brockman said, “Welcome to the AGI era,” according to Axios.
\nIf this is AGI, I still had to tell it to give a support agent the case open on the screen.
\nThe next changes are concrete: stop suggestions from inventing customer facts, offering approvals they cannot execute, and promising follow-up nobody has arranged. Then rerun the golden set and try fresh cases. I want evidence that the workflow improved before calling the harness good.
", "url": "https://www.groff.dev/blog/custom-agent-harnesses-vercel-ai-sdk", "title": "Building Custom Agent Harnesses for Specific Domains Using Vercel's AI SDK", "summary": "What is an agent harness? Learn how it works, build a custom support agent with Vercel AI SDK, and compare Sol and Luna with a runnable eval suite.", "date_modified": "2026-09-13T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/gpt-6-astra-vs-claude-fable-knowledge-work", "content_html": "I love Fable and Astra. For the research, proposals, planning, and consulting delivery I do, both are intelligent enough for me.
\nWhat I want next is the same level of intelligence, cheaper and faster. I do not feel much need for a smarter model for this work. The returns are diminishing for me.
\nI still have things I want improved: following through on an agreed action, keeping track of decisions, and writing without the extra AI prose. But I can already work through complicated ideas with these models and produce something I want to put my name on.
\nMy first weekend with GPT-6 Astra in Codex Desktop has been encouraging because I like it just as much as Fable for this work, and I can use a lot more of it.
\nOn my Premium Claude for Teams seat, Fable is limited to half of my total weekly usage. In practice, I burn through that allowance in one or two days of what I consider light usage. I cannot get through half a workweek.
\nA typical session getting a proposal to where I want it can eat $60 to $100 in credits. For my workflow, that is not sustainable.
\nEven while I have usage left, I find myself worrying about spending it. Should I save Fable for something harder? Is this conversation worth the credits? How much work do I still have this week?
\nI want to think about the proposal.
\nI started using GPT-6 Astra in Codex Desktop on Friday afternoon. I worked through Saturday. As of Sunday, September 6, I still had 33% left and two banked resets remaining.
\nThat is on my $200 ChatGPT Pro account, which I expense through work. These are my account experience and usage figures from this weekend, not a controlled cost comparison or a promise about anyone else’s limits.
\nFor the knowledge work I am doing, Astra feels equal to Fable in intelligence. It also produces much less of the writing I call Claude-speak.
\nBefore Astra, I was trying to make Opus 5 work for this. I spent too much time correcting the tone and trying to get the quality I wanted. Fable and Astra have been much easier for me to work with.
\nMy current recommendation is Astra on Medium reasoning. That is what I have been using for this work. My recommendation comes from this early experience, not a systematic comparison of every setting.
\nThe practical difference is that I can keep working without being afraid that another round of questions will use up the model I need.
\nI miss Claude Cowork. I prefer its interface to Codex Desktop. I prefer Claude Artifacts to ChatGPT Sites. Sharing work with my team is easier in Claude for Teams because everyone is already there.
\nI wrote previously about Zero Defect and how I review AI-generated business work. Making plugins for Codex has felt clunkier and more error-prone to me than making them for Claude. I expect OpenAI will catch up there.
\nEven my grill-me workflow takes extra effort in Codex Desktop. I want it to ask me a question and wait until I answer before moving on. I have had to effectively trick it into keeping the turn open while I think through my response. That is another reason I prefer the Claude Cowork interface for this kind of back-and-forth.
So I still have plenty of reasons to prefer the Claude environment. The limits are what keep getting in the way. Astra gives me a level of quality I am happy with and enough room to use it.
\nThis weekend’s work was consulting delivery. I was taking meeting notes, a signed statement of work, client questions, existing architecture material, and research, then turning them into recommendations and deliverables.
\nThe output lives in a client delivery hub: documents, decisions, tasks, references, and interactive examples. Some recommendations also need a small proof of concept so we can distinguish what sounds feasible from what we have actually demonstrated.
\nThere is code involved, but the central work is deciding what to recommend, why it fits, what we promised, and what remains unresolved.
\nI have opinions. I also have questions I cannot answer yet. I want the model to research those questions, explain the options, challenge my assumptions, and help me develop a position I understand well enough to defend.
\nIn one exchange, I suspected that most MCP documentation focused on installation rather than explaining available tools. I asked Astra to research it.
\nIt came back with counterexamples. Its first pass found several vendors publishing tool catalogs, and it explicitly said the sample did not establish what most sites do.
\nThat was useful. I had a hypothesis, and the research gave me reasons to reconsider it.
\nMy eventual recommendation was still opinionated: emphasize setup and useful workflows for the audience we were serving. But that recommendation needed its own justification. It could not rest on an unsupported claim that everyone else only documents installation.
\nThat is the relationship I want with the research. My opinions should survive being questioned, or change because I learned something.
\nThe mistakes in this conversation were not limited to factual claims.
\nAstra turned part of a client discussion into an extra task that had not actually been requested. I challenged it and asked for a quoted, cited origin for every task. It acknowledged that it had over-interpreted the discussion and removed the duplicate work.
\nIt also attributed a client’s architecture diagram to me. I corrected that. Who proposed something matters, especially when the document is going back to the client.
\nA plausible recommendation can make its way into a polished document before I have agreed with it. Then it looks like my position.
\nThat is what I mean when I say I need to catch the model hallucinating opinions that are not mine. Thinking through the decision is my job. Putting my name on the output should mean I understand it and have made that decision myself.
\nI start with a hypothesis or an opinion. The model can research it, present options and tradeoffs, and challenge me. I have changed my opinions several times working with both Fable and Astra, and I am learning a lot from the process.
\nIn one architecture discussion, I asked Astra to explain the options piece by piece. I did not understand them well enough to choose. We needed more examples, research, and discussion before I could form my own recommendation.
\nI wrote about this on LinkedIn on Friday: when you are using the /grill-me skill and reach a question you do not fully understand, pause. Ask the agent to explain the concepts in a way that makes sense to you.
I like using Claude Artifacts or ChatGPT Sites to see a concept explained visually with animations and diagrams. Maybe you learn better with a podcast or a video. Ask for the explanation that helps you understand.
\nSometimes I need to know what others in the industry are choosing and why. Have they written about it in a blog or posted about it on X? Ask the agent to research those examples and the tradeoffs, then bring that evidence back to the decision.
\nNever blindly accept the recommended option or say “whatever is best” when you do not understand the question. That is outsourcing your thinking to AI. Don’t be a meat-proxy.
\nI have seen people on X describe getting great results while supervising Astra and feeling frustrated when they let it run on its own. Those are anecdotes, but I recognize the experience.
\nFor this kind of work, I want to stay involved.
\nReviewing a proposed decision, rejecting extra scope, asking for better evidence, and correcting a misleading phrase are worthwhile uses of my time. They are part of producing work I can stand behind.
\nThere is also friction I want fixed. During this task, I had to ask whether research was still happening. It was not. At another point, I had to prompt it to continue an interview. I also spent time reconciling inconsistent task and document statuses.
\nThose interruptions do not make the work better. Once we have agreed on an action, I want it executed and verified. I should not have to keep reminding the agent to continue.
\nThere is a fair distinction in my own prompts, too. If I ask “Thoughts?” about changing a page, an answer discussing the change is reasonable. Following that with “Do it” is me moving from discussion to execution. I cannot count every two-step exchange as an execution failure.
\nHere is the kind of sentence I am tired of:
\n\n\nYou were half right, and the half you were right about matters more.
\n
Just explain what was right and what needs correcting.
\nI find myself removing much less of this with Astra. It still adds unnecessary material sometimes. In the delivery work, I had to remove commentary about our drafting process from client-facing pages and bring examples back to what I had actually asked for.
\nBut overall, I spend less time fighting the voice. That matters when the output is supposed to sound like me.
\nMy goal is to produce research-backed consulting work that reflects my judgment, with Astra handling the investigation, drafting, and execution under clear direction.
\nThe workflow I am taking from this weekend is:
\nI like both models. I still prefer much of the Claude experience, and Fable gives me the quality I want. Astra is giving me comparable quality for this work, less Claude-speak, and more room to keep going.
\nMore intelligence would have to make a practical difference to my work for me to care. Right now, I would get more value from this level of intelligence being cheaper, faster, and available for more of my working week.
\nAnyone else feeling the same? How are Astra and Fable comparing for your workflows?
\nIf you or your team feel left behind and unsure about what to do, you are not alone. My team and I at Elios are here to help. Let’s talk.
", "url": "https://www.groff.dev/blog/gpt-6-astra-vs-claude-fable-knowledge-work", "title": "GPT-6 Astra vs Claude Fable for Knowledge Work: My Experience", "summary": "My experience with GPT-6 Astra and Claude Fable for research, proposals, and consulting: both are smart enough. I want the same intelligence cheaper and faster.", "date_modified": "2026-09-06T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/more-thorough-ai-review-claude-code-codex-claude-cowork", "content_html": "AI can produce a lot of work quickly. Reviewing that work is still surprisingly shallow.
\nMost AI review feels like one smart person reading once. The model scans the change, notices a few things, and returns a plausible summary. That can be useful, but it is easy for one pass to miss a broken contract, a race condition, an unsupported claim, or an accidental promise.
\nI wanted a review process that behaves more like a group of skeptical specialists. Each specialist gets a narrow mandate. They investigate independently. A stronger parent model then checks their claims, rejects weak findings, and produces one report.
\nThat idea became two open-source projects: Friendly Adversary for software and Zero Defect for business work.
\nThe shared idea is adversarial review through focused lenses.
\nA lens is a reviewer with one job. It does not try to judge everything. That narrower scope encourages a deeper investigation and makes overlap useful. Two lenses may reach the same defect from different directions, which gives the final judge more evidence to work with.
\nThe parent model still owns the result. It reads the independent reports, checks the evidence, resolves duplicates, assigns severity, and decides what is actually worth fixing. The purpose is to make important defects harder to miss without generating an inflated list.
\nFriendly Adversary is a local Claude Code and Codex code review system for TypeScript, JavaScript, and Python repositories. It can review a pull request, audit an existing codebase, or challenge a new codebase design before scaffolding begins.
\nIts pull request review uses nine lenses:
\nThe system also bundles deterministic tools instead of asking the model to rediscover every clue. Semgrep, Ruff, an Oxlint-compatible engine, and ripgrep run locally as WebAssembly. Their native output becomes evidence for the review. Repository-configured checks can also run during a pull request review when the repository is trusted.
\nFor each pull request lens, Friendly Adversary launches an isolated local Codex CLI process using the lower-cost Luna model. The lens runs with high reasoning and Fast mode in a read-only sandbox. All nine run concurrently. The Claude Code or Codex model that started the review acts as the more capable final judge. This keeps the broad investigation relatively inexpensive while reserving the strongest model for adjudication.
\nZero Defect applies the same philosophy to English-language business deliverables in Claude Cowork. It is intended for proposals, emails, candidate summaries, product plans, reports, requirements, spreadsheets, presentations, and similar work.
\nIts seven lenses are different because business documents fail differently:
\nZero Defect is deliberately simple. It has no scripts, MCP server, package dependencies, local service, or filesystem output. Claude Cowork invokes the seven reviewers in parallel, and the parent model validates and consolidates their findings. It diagnoses defects and suggests concise repairs. It does not rewrite the deliverable unless you separately ask Claude to do that.
\nNeither project can promise defect-free work. Models can miss things. Deterministic analyzers have coverage limits. A reviewer can misunderstand intent, and a clean report does not replace testing, professional judgment, security review, legal review, or accountable human approval.
\nThe standard is more useful: review important work from multiple explicit angles, demand evidence for findings, and make the final judge distinguish a material defect from a nitpick.
\nBoth projects are early and opinionated. They reflect the review process I wanted for my own work, and they will keep changing as real use exposes weak prompts, missing lenses, noisy checks, and better orchestration patterns.
\nFeedback and forks are welcome. Contributions are welcome for consideration too, although I may not accept every proposed change. The repositories are the best place to inspect the full design and decide whether the approach fits your workflow:
\n", "url": "https://www.groff.dev/blog/more-thorough-ai-review-claude-code-codex-claude-cowork", "title": "More Thorough AI Review for Claude Code, Codex, and Claude Cowork", "summary": "Open-source adversarial review for code and business work with Friendly Adversary and Zero Defect across Claude Code, Codex, and Claude Cowork.", "date_modified": "2026-08-21T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/linux-codex-desktop-app-with-remote-control", "content_html": "I wanted to use Codex Desktop on Linux to remote control my Mac mini at home.
\nSpecifically, I wanted the Linux Codex Desktop App with Remote Control working well enough that I could open Codex Desktop on Linux, go to Control other devices, authorize the machine, and remote into my Mac mini.
\nThis is not official support. This is an experimental, unofficial, local hack against the excellent Linux build repo here:
\n\nThere is also a relevant GitHub issue tracking this rough edge:
\n\nIf you are searching for things like Codex Desktop Linux remote control, Codex remote control Linux, Codex Desktop for Linux Control other devices, or Linux Codex Desktop App remote control Mac mini, this is the exact debugging path that worked for me.
\nThe normal install worked fine for me using the unofficial Linux repo:
\ngit clone https://github.com/ilysenko/codex-desktop-linux.git\ncd codex-desktop-linux\nmake bootstrap-native\nThat got the base app installed and launching.
\nThe issue was not the basic Linux app. The issue was outbound remote control from Linux.
\nWhen I tried to authorize remote control, Codex Desktop failed with this error:
\nremote_control_authorize_failed\nerrorMessage="Remote control device keys are only available on macOS"\nThat was the first blocker.
\nThe app was trying to use remote control device keys, but the device-key provider path it expected was macOS-only.
\nAt this point, it looked like Linux could run the desktop app, but could not authorize for remote control.
\nremote-mobile-controlThe important discovery was that ilysenko/codex-desktop-linux already has an experimental Linux feature called remote-mobile-control.
That feature matters because it adds two important pieces:
\ncodex app-server with --remote-controlThat fixed the macOS-only device-key blocker.
\nTo enable it, I created linux-features/features.json:
mkdir -p linux-features\ncat > linux-features/features.json <<'EOF'\n{\n "enabled": [\n "remote-mobile-control"\n ]\n}\nEOF\nAfter that, the app-server side was much closer to what I needed.
\nBut there was a second problem.
\nControl other devices Was Hidden on LinuxAfter enabling remote-mobile-control, the Control other devices tab was still not usable.
In the Codex UI, this is the access-other-devices settings tab. The Linux feature patch was intentionally filtering that tab out as unsupported.
That makes sense as a defensive upstream choice. If the feature is experimental and not fully supported, hiding the tab avoids presenting a broken flow to users.
\nBut in my case, I specifically wanted to test outbound remote control from Linux. So hiding the tab became the next blocker.
\nThe local hack was to let the access-other-devices tab show up on Linux.
In linux-features/remote-mobile-control/patch.js, the existing patch changed the settings tabs with something like this:
"function codexLinuxRemoteControlSettingsTabs(e){return typeof navigator!==`undefined`&&navigator.userAgent.includes(`Linux`)?e.filter(e=>e.key!==`access-other-devices`):e}"\nFor my local build, I changed it to return the tab list unchanged:
\n"function codexLinuxRemoteControlSettingsTabs(e){return e}"\nI also disabled or skipped the selected-tab fallback patch that redirected Linux away from access-other-devices.
That part matters. If the tab exists but the selected-tab logic forces Linux away from it, the UI still will not let you use the flow.
\nSo the practical patch was:
\nremote-mobile-control in linux-features/features.json.linux-features/remote-mobile-control/patch.js so codexLinuxRemoteControlSettingsTabs returns tabs unchanged instead of filtering out access-other-devices.access-other-devices.Again, this is a local hack. I am not claiming this is supported by OpenAI or by the unofficial Linux repo.
\nAfter the local patch, rebuild and reinstall:
\nmake install-native\nThen launch Codex Desktop.
\nI verified the important processes with:
\npgrep -af 'codex app-server|/opt/codex-desktop/electron'\nThe app-server process should include this:
\ncodex app-server --remote-control --analytics-default-enabled\nI also checked the launcher logs:
\ntail -200 ~/.cache/codex-desktop/launcher.log\nFor the UI patch, I verified the built webview asset contained the patched function:
\ngrep -R "function codexLinuxRemoteControlSettingsTabs(e){return e}" \\\n /opt/codex-desktop/content/webview/assets/remote-connections-settings-*.js\nOnce those lined up, the Control other devices tab was visible and usable on Linux.
\nAfter the patch and rebuild, the Linux Codex Desktop App allowed the Control other devices tab.
\nAuthorization worked.
\nI was able to remote control my Mac mini from Linux.
\nThat was the end state I wanted: OpenAI Codex Desktop Linux remote into Mac behavior through the unofficial Linux build, using the experimental remote-mobile-control feature plus a local UI patch.
A few important warnings before anyone treats this like a normal install guide:
\nSo I would treat this as a debugging note, not a supported product feature.
\nIf you are using this for real work, understand what you are patching and where keys are stored.
\nIf you want Codex CLI to apply the same local hack, this is the prompt I would paste in:
\nIn this repo https://github.com/ilysenko/codex-desktop-linux, make a local experimental build that enables outbound remote control from Linux.\n\nDo the following:\n1. Create linux-features/features.json enabling remote-mobile-control.\n2. Editlinux-features/remote-mobile-control/patch.js.\n3. Keep the Linux remote-control device-key provider and app-server --remote-control behavior.\n4. Do not hide the Control other devices / access-other-devices tab on Linux.\n5. Change codexLinuxRemoteControlSettingsTabs so it returns the tab list unchanged.\n6. Disable the selected-tab fallback patch that redirects Linux away from access-other-devices.\n7. Rebuild and reinstall with make install-native.\n8. Launch /usr/bin/codex-desktop.\n9. Verify that /opt/codex-desktop/content/webview/assets/remote-connections-settings-*.js contains `function codexLinuxRemoteControlSettingsTabs(e){return e}` and that pgrep -af 'codex app-server' shows --remote-control.\n\nThis is an unofficial local hack. Do not claim official support.\nThe main trick was that the first failure and second failure were different problems.
\nThe macOS-only device-key error was solved by enabling remote-mobile-control.
The missing Control other devices tab was solved by not filtering out access-other-devices on Linux.
Together, those two changes got codex-desktop-linux remote-mobile-control working well enough for me to control my Mac mini from Linux.
I have written about progressive disclosure for agent instructions before. Root CLAUDE.md for universal rules, task-specific skills loaded on demand, local overrides when a folder has unique constraints. That system works well.
What I missed entirely was .claude/rules/.
I found it while reading through the Claude Code memory documentation. It solves a real problem that CLAUDE.md files alone cannot solve, and it has been available since early 2026. The HumanLayer article on writing a good CLAUDE.md (published November 2025) does not mention it because rules shipped after that article was written.
This post covers what .claude/rules/ does, why it matters for monorepos, and how we adopted it alongside our existing CLAUDE.md files without replacing anything.
CLAUDE.md is directory-scoped. It loads when Claude works in that directory. You put a CLAUDE.md in apps/api/ and it activates when Claude reads or edits files there.
.claude/rules/ is pattern-scoped. Each rule file can include YAML frontmatter with glob patterns, and the rule loads when Claude touches files matching those patterns, regardless of which directory those files live in.
This distinction matters when your conventions follow file types, not directories.
\nConsider test files. In most modern projects, test files are co-located with source files throughout the entire tree. If you want Claude to follow specific testing conventions (naming patterns, assertion styles, mock strategies), you have two options with CLAUDE.md alone:
CLAUDE.md, where they burn context on every session whether Claude is writing tests or not.CLAUDE.md that contains test files.Neither scales. A glob-scoped rule like **/*.spec.ts fires exactly when Claude is working on test files. No duplication. No wasted context.
The same applies to any cross-cutting file type: database migrations, config files, API route definitions, component files that live across multiple apps in a monorepo.
\nCLAUDE.md is the right tool for:
apps/api/." That context matters for every file in the directory, not just specific file types.nx affected:test before committing. Use the self-review-checklist skill before creating PRs." These are session-level directives.CLAUDE.md is where you list them.If the instruction applies to everything in a directory or everything in the project, CLAUDE.md is still the correct place.
Rules shine for file-type conventions that span multiple directories:
\n*.controller.ts across 5 apps*.tsx in both apps/ and libs/*.schema.ts and **/migrations/** filesThe key property: rules with paths frontmatter only load into context when Claude reads a matching file. If Claude never opens a controller file in a session, the controller rules never enter the context window. This keeps token costs low and instruction relevance high.
Rules without paths frontmatter load unconditionally at startup, same priority as .claude/CLAUDE.md. Use those sparingly for conventions that genuinely apply everywhere.
Each rule is a markdown file in .claude/rules/ with optional YAML frontmatter:
---\npaths:\n - "**/*.service.ts"\n---\n\n# Backend Service Rules\n\n- Never call external services inside a transaction.\n- Use the request-scoped auth context for permission checks.\n- Repository methods return domain types, not ORM entities.\nThat is the entire format. YAML frontmatter with a paths array, then markdown body with your conventions.
Glob patterns support standard syntax:
\n| Pattern | Matches |
|---|---|
**/*.ts | All TypeScript files in any directory |
apps/*/src/**/*.tsx | React components across all apps |
**/migrations/** | Migration files anywhere in the tree |
libs/routes/**/*.routes.ts | Route files in a specific library |
You can specify multiple patterns per rule, and brace expansion works:
\n---\npaths:\n - "**/*.schema.ts"\n - "**/migrations/**"\n---\nThe .claude/rules/ directory supports subdirectories and symlinks. You can organize rules into frontend/ and backend/ folders, or symlink shared rules from a central location across multiple projects.
User-level rules in ~/.claude/rules/ apply to every project on your machine. Project rules take higher priority.
We run an Nx TypeScript monorepo with 5 apps and 10+ shared libraries. Before rules, we had 11 directory-scoped CLAUDE.md files and 19 agent guides loaded as skills.
The problem: conventions for NestJS controllers, React components, Drizzle schemas, and route definitions were locked behind skills that required explicit invocation or relied on Claude deciding they were relevant. When Claude edited a *.controller.ts file, the local CLAUDE.md said "read the backend authorization patterns guide." But the actual constraints were not in context until Claude manually loaded the guide. Sometimes it did. Sometimes it did not.
We created 7 glob-pattern rules in .claude/rules/:
| Rule file | Paths glob | What it covers |
|---|---|---|
backend-controllers.md | **/*.controller.ts | Route ownership, guard patterns, typed decorators |
backend-services.md | **/*.service.ts | Transaction safety, auth context, update patterns |
backend-modules.md | **/*.module.ts | DI encapsulation, provider resolution, .forRoot() rules |
drizzle-schema.md | **/*.schema.ts, **/migrations/** | Column naming, index conventions, type derivation |
react-patterns.md | apps/*/src/**/*.tsx, libs/*/src/**/*.tsx | Effect anti-patterns, API call patterns, permissions |
route-definitions.md | libs/routes/**/*.routes.ts | Naming, type sourcing, contract rules |
nextjs-pages.md | **/page.tsx, **/layout.tsx | Metadata, layout components, import aliases |
Every rule is under 250 words. Ultra-concise bullet lists with a pointer to the full skill guide for details. Total context cost when all 7 fire: ~1K tokens. In practice, a typical session only triggers 2 or 3 of them.
\nThe important part: we kept all 11 existing CLAUDE.md files unchanged. Rules are additive. We did not have to restructure anything.
After this migration, our instruction architecture has three layers:
\n.claude/rules/ (pattern conventions). File-type conventions that span directories. Loads only when Claude touches matching files. Deterministic, no invocation required.Sizing guidance: We target under 250 words per rule. Each rule that triggers adds to the context window, so brevity matters. If a convention needs a full page of explanation, put the details in a skill and use the rule as a concise summary with a pointer.
\nWhen to use a rule vs. a skill: If Claude should always know the convention when touching that file type, use a rule. If it is a procedure that Claude runs occasionally (like a deployment checklist or a PR review workflow), use a skill.
\nWhen to use a rule vs. CLAUDE.md: If the convention follows a directory boundary, use CLAUDE.md. If it follows a file-type pattern across directories, use a rule.
Rules without paths: Use these for conventions that genuinely apply to all files (like "never use em dashes in content" or "always use single quotes"). They load unconditionally, so keep them short.
\n.claude/rules/ is specific to Claude Code. Other agentic coding tools have their own rule mechanisms. Cursor has .cursor/rules, and there are similar concepts elsewhere. I do not believe these are cross-compatible enough to share via symlinks, but please reach out if you know otherwise.
If you work across multiple tools, you will likely need separate rule files for each. That is a real cost. For teams standardized on Claude Code, .claude/rules/ is a significant improvement over CLAUDE.md alone for pattern-scoped conventions.
If you are already using CLAUDE.md files, you do not need to change anything to start using rules.
.claude/rules/ in your project root.CLAUDE.md or a skill (controller patterns, component rules, test conventions).CLAUDE.md files continue working exactly as before.The official documentation covers the full specification, including user-level rules, symlink sharing, and the claudeMdExcludes setting for large monorepos.
If you have questions or want to share your own rules setup, find me on LinkedIn.
", "url": "https://www.groff.dev/blog/claude-rules-vs-claude-md", "title": "Claude Rules vs CLAUDE.md: Pattern-Scoped Conventions Your Team Is Missing", "summary": "CLAUDE.md is directory-scoped. .claude/rules/ is pattern-scoped. Most teams only use CLAUDE.md and miss the rules mechanism entirely. Here is how we use both in a real Nx monorepo.", "date_modified": "2026-05-27T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/azure-entra-id-mcp-server-authentication-incompatibilities", "content_html": "If you are building an MCP server and your organization uses Azure Entra ID (formerly Azure AD) for identity, you are going to have a rough time.
\nThe MCP auth spec assumes your identity provider supports RFC 8414 (OAuth Authorization Server Metadata), RFC 7591 (Dynamic Client Registration), and RFC 8707 (Resource Indicators). Azure does not support any of these properly.
\nI hit 10 distinct incompatibilities while connecting Claude Code and MCP Inspector to an Azure-protected MCP server. Every single one required a workaround. This post covers each one in the order I encountered them, with the exact error codes and fixes.
\nIf you are searching for AADSTS9010010, AADSTS90009, AADSTS50011, or "Incompatible auth server" in the context of MCP, you are in the right place.
Before diving into the individual issues, here is the high-level pattern. Your MCP server acts as an OAuth proxy between MCP clients and Azure. The proxy intercepts the standard MCP auth flow, translates it into something Azure can handle, and returns responses that MCP clients expect.
\nThe proxy serves four endpoints:
\n| Endpoint | Purpose |
|---|---|
GET /.well-known/oauth-authorization-server | Serves Azure's OIDC metadata with overrides for MCP compatibility |
GET /oauth/authorize | Strips unsupported parameters and redirects to Azure |
POST /oauth/token | Strips unsupported parameters and forwards to Azure |
POST /oauth/register | Mock Dynamic Client Registration, returns a pre-configured client ID |
Your /.well-known/oauth-protected-resource points MCP clients at your own server (not Azure directly), so they discover the proxy endpoints.
Here is the full flow:
\nMCP Client connects to /mcp\n -> Server returns 401 + WWW-Authenticate header\n -> Client fetches /.well-known/oauth-protected-resource\n -> Sees authorization_servers = [your server URL]\n -> Client fetches /.well-known/oauth-authorization-server (from your proxy)\n -> Gets Azure OIDC metadata + your overrides\n -> Client POSTs /oauth/register (mock DCR)\n -> Gets pre-configured MCP client ID\n -> Client opens browser -> GET /oauth/authorize\n -> Proxy strips resource param, redirects to Azure\n -> User logs in at Azure\n -> Azure redirects back to client's localhost callback\n -> Client POSTs /oauth/token\n -> Proxy strips resource param, forwards to Azure\n -> Gets JWT access token\n -> Client sends JWT as Bearer token to /mcp\n -> Server validates JWT and processes the request\nNow let's walk through each incompatibility.
\nError: HTTP 404: Invalid OAuth error response
MCP clients discover the authorization server by fetching /.well-known/oauth-authorization-server per RFC 8414. Azure does not serve this endpoint. Azure uses /.well-known/openid-configuration (OIDC Discovery) instead.
Fix: Your server serves /.well-known/oauth-authorization-server itself by fetching Azure's OIDC config and adding the fields MCP clients need:
@Get('.well-known/oauth-authorization-server')\nasync getAuthorizationServerMetadata() {\n const oidcUrl = `https://login.microsoftonline.com/${this.tenantId}/v2.0/.well-known/openid-configuration`;\n const res = await fetch(oidcUrl);\n const metadata = await res.json();\n\n return {\n ...metadata,\n authorization_endpoint: `${this.baseUrl}/oauth/authorize`,\n token_endpoint: `${this.baseUrl}/oauth/token`,\n registration_endpoint: `${this.baseUrl}/oauth/register`,\n code_challenge_methods_supported: ['S256'],\n };\n}\nThis proxies Azure's real metadata while overriding the endpoints MCP clients will call to route through your proxy.
\nError: Incompatible auth server: does not support dynamic client registration
MCP clients try RFC 7591 Dynamic Client Registration to register themselves and get a client ID. Azure does not support DCR.
\nFix: A mock DCR endpoint that accepts any registration request and returns the pre-configured client ID for your MCP client app registration:
\n@Post('oauth/register')\nregister() {\n return {\n client_id: this.mcpClientId,\n client_name: 'MCP Client',\n redirect_uris: ['http://localhost'],\n };\n}\nThis is the workaround I like the most. It is simple, it works, and it lets MCP clients follow their normal auth flow without knowing Azure is behind the scenes.
\nError: AADSTS9010010: The resource parameter provided in the request doesn't match with the requested scopes.
The MCP spec mandates RFC 8707 Resource Indicators. Azure v2 endpoints do not implement RFC 8707. When MCP clients include the resource parameter, Azure rejects the request.
Fix: Your OAuth proxy strips the resource parameter from both /authorize and /token requests before forwarding to Azure:
@Get('oauth/authorize')\nauthorize(@Query() query: Record<string, string>, @Res() res) {\n const params = new URLSearchParams(query);\n params.delete('resource');\n res.redirect(`${this.azureBase}/authorize?${params.toString()}`);\n}\n\n@Post('oauth/token')\nasync token(@Body() body: Record<string, string>) {\n const params = new URLSearchParams();\n for (const [key, value] of Object.entries(body)) {\n if (key !== 'resource') {\n params.append(key, String(value));\n }\n }\n\n const azureRes = await fetch(`${this.azureBase}/token`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/x-www-form-urlencoded' },\n body: params.toString(),\n });\n\n return azureRes.json();\n}\nThe scope parameter (api://{clientId}/.default) already encodes the resource, so stripping resource is safe.
Error: AADSTS90009: Application is requesting a token for itself. This scenario is supported only if resource is specified using the GUID based App Identifier.
This one is a real head-scratcher the first time you see it. If you use the same app registration as both the OAuth client and the API, Azure blocks the token request because it thinks the app is requesting a token for itself.
\nFix: You need two app registrations in Azure Entra ID.
\nAPI app registration:
\nAdmin, User)access){BASE_URL}/auth/callbackClient app registration (for MCP clients):
\nhttp://localhost/callbackaccess scopeYour mock DCR endpoint returns the client app's ID. Your JWT validation uses the API app's ID as the audience.
\nThis is probably the most confusing step if you have not done it before. Azure's portal does not make it obvious that you need two registrations.
\nThis is not an error code. It is a portal problem.
\nWhen you create the client app registration and try to add API permissions for your API app, the permission picker is empty. Your API app does not show up.
\nFix: In the API app registration, go to "Expose an API" and add a scope. Until you explicitly expose a scope, Azure will not let other apps request permissions for it. After adding the scope, the client app can find and request it.
\nError: AADSTS50011: The redirect URI specified in the request does not match the redirect URIs configured for the application.
MCP clients like Claude Code use http://localhost/callback or http://127.0.0.1/callback as their redirect URI. Azure requires exact redirect URI matching, including the path.
Fix: In the client app registration, add the exact redirect URI your MCP client uses. For Claude Code, this is typically http://localhost/callback under "Mobile and desktop applications" (not "Web"). You may need to add both localhost and 127.0.0.1 variants.
The category matters. Using "Web" instead of "Mobile and desktop applications" will cause different behavior with PKCE and public client flows.
\nAzure supports PKCE with S256, but it does not include code_challenge_methods_supported in its metadata. MCP clients see the omission and may assume PKCE is not supported or behave unpredictably.
Fix: Your OAuth proxy injects code_challenge_methods_supported: ['S256'] into the authorization server metadata response (shown in the fix for incompatibility #1).
Azure issues tokens with the v1 issuer (https://sts.windows.net/{tenant}/) even when you are using v2 endpoints, unless accessTokenAcceptedVersion is explicitly set to 2 in the app manifest.
If your JWT validation only checks for the v2 issuer, tokens will fail validation silently.
\nFix: Accept both issuers in your JWT validation:
\nissuer: [\n `https://login.microsoftonline.com/${tenantId}/v2.0`,\n `https://sts.windows.net/${tenantId}/`,\n];\nYou can also set accessTokenAcceptedVersion to 2 in the app manifest, but accepting both issuers is more defensive. You do not want authentication to break because someone changed a manifest setting.
Tokens may use either api://{clientId} or the bare GUID as the aud claim, depending on how the scope was requested.
Fix: Accept both audience formats:
\naudience: [clientId, `api://${clientId}`];\nThis is a small fix but easy to miss. If you only validate one format, some valid tokens will be rejected.
\nAzure does not just care that the redirect URI string matches. It also cares which platform bucket that redirect URI lives under.
\nThat matters because MCP clients do not all use the same callback shape:
\n| Client | Redirect URI | Azure platform type |
|---|---|---|
| Claude Code | http://localhost/callback or http://127.0.0.1/callback | Mobile and desktop applications |
| Codex / local MCP clients | Usually a loopback URI like http://localhost/callback | Mobile and desktop applications |
| Claude.ai hosted connector | https://claude.ai/api/mcp/auth_callback | Single-page application |
| ChatGPT hosted connector | Provider-generated URI like https://chatgpt.com/connector/oauth/<id> | Single-page application |
If you put the hosted connector callback under Web, Azure treats the token exchange like a confidential web app flow and rejects public PKCE redemption. If you put the local loopback callback under the wrong type, local clients can fail even when the URL looks correct.
\nErrors:
\nAADSTS50011: The redirect URI specified in the request does not match the redirect URIs configured for the application.AADSTS9002327: Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests.AADSTS9002326: Cross-origin token redemption is permitted only for the 'Single-Page Application' client-type.Fix: Treat the MCP client app registration as a public client with no secret, then register each redirect URI under the right platform type.
\nFor local agents like Claude Code and Codex, add the exact loopback redirect URI under Mobile and desktop applications. In practice, I add both http://localhost/callback and http://127.0.0.1/callback when testing multiple clients.
For hosted connectors like Claude.ai and ChatGPT, add the exact callback URL under Single-page application. Then have your /oauth/token proxy send an Origin header that matches the hosted connector origin during token exchange:
private readonly hostedConnectorOrigins = [\n 'https://chatgpt.com',\n 'https://claude.ai',\n];\n\nprivate redirectOrigin(redirectUri: string | undefined) {\n if (!redirectUri) return undefined;\n const url = new URL(redirectUri);\n\n if (this.isHostedConnectorHost(url.hostname)) {\n return url.origin;\n }\n\n if (url.protocol === 'http:' && this.isLoopbackHost(url.hostname)) {\n return url.origin;\n }\n\n return undefined;\n}\nThen pass that origin through to Azure when appropriate:
\nconst headers: Record<string, string> = {\n 'Content-Type': 'application/x-www-form-urlencoded',\n};\n\nif (tokenOrigin) {\n headers.Origin = tokenOrigin;\n}\nThe annoying part is that the correct setup is not one redirect list. It is a matrix of exact redirect URI plus Azure platform type plus token exchange behavior. Once those three line up, Claude Code, Codex-style local clients, Claude.ai hosted connectors, and ChatGPT hosted connectors can all use the same MCP OAuth proxy.
\nAfter all that plumbing, the actual authorization code is small. Here is the admin guard that protects sensitive MCP tools:
\n@Injectable()\nexport class AdminGuard implements CanActivate {\n canActivate(context: ExecutionContext): boolean {\n const req = context.switchToHttp().getRequest<{ actor?: Actor }>();\n return req.actor?.role === 'admin';\n }\n}\nTen lines. All the Azure complexity is absorbed by the middleware and proxy layers, so your business logic never has to think about it.
\nFor reference, here are the environment variables the proxy and auth layers need:
\nAZURE_TENANT_ID=<your-tenant-id>\nAZURE_CLIENT_ID=<api-app-client-id>\nAZURE_CLIENT_SECRET=<api-app-client-secret>\nAZURE_MCP_CLIENT_ID=<client-app-client-id>\nBASE_URL=<your-server-public-url>\nWhen AZURE_TENANT_ID is unset, the system falls back to a dev mode that skips auth entirely. That makes local development easy while keeping production locked down.
Azure Entra ID is the most common enterprise identity provider. MCP is the emerging standard for connecting AI tools to APIs. The fact that these two do not work together without a proxy layer is a real problem for enterprise adoption.
\nMicrosoft is aware of these issues. There is a blog post from Microsoft about building MCP servers with Entra ID, and there are open issues in the MCP SDK tracking Azure compatibility (#862). Claude Code also has an open issue about DCR (#52638).
\nThe MCP spec itself has a proposal to make the resource parameter optional, which would help with incompatibility #3.
\nUntil these get resolved, expect to build and maintain an OAuth proxy if you are connecting MCP clients to Azure-protected APIs. The proxy is not complicated (the entire implementation is under 70 lines), but you need to know it is necessary before you spend hours debugging cryptic Azure error codes.
\nIf you landed here from a search, here is a quick lookup:
\n| Error | Issue | Fix Section |
|---|---|---|
HTTP 404 on /.well-known/oauth-authorization-server | No RFC 8414 | #1 |
Incompatible auth server | No DCR | #2 |
AADSTS9010010 | Resource parameter rejected | #3 |
AADSTS90009 | Token for itself | #4 |
| Empty permission picker in portal | API not exposed | #5 |
AADSTS50011 | Redirect URI mismatch | #6 |
| PKCE not advertised | Missing metadata field | #7 |
| JWT validation fails silently | Issuer v1/v2 mismatch | #8 |
| JWT audience rejected | api:// prefix inconsistency | #9 |
AADSTS9002327 / AADSTS9002326 | Hosted connector redirect registered under the wrong Azure platform type | #10 |
Good luck out there. Azure auth is a solvable problem, it just has more steps than you expect.
\nIf you need a Forward Deployed Engineer to build your team's MCP server, reach out to matt@eliosai.com.
", "url": "https://www.groff.dev/blog/azure-entra-id-mcp-server-authentication-incompatibilities", "title": "10 Azure Entra ID Incompatibilities with MCP Server Authentication (And How to Fix Them)", "summary": "Azure Entra ID breaks the MCP auth spec in at least 10 ways. Here is every error code, every workaround, and the OAuth proxy pattern that makes it all work with Claude Code, MCP Inspector, and other MCP clients.", "date_modified": "2026-05-24T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/peanut-butter-chocolate-v2", "content_html": "In February, I released Peanut Butter & Chocolate, my Claude Code plugin for structured agentic coding.
\nThe core pipeline has not changed.
\nResearch. Plan. Implement. Review.
\nThat is still the loop. What changed is everything around it.
\nPBC v2.0 is simpler. It has fewer files. It trusts the model more. It also has a Codex version now.
\nThe first version used OpenCode to talk to Codex. That made sense at the time. OpenCode was my daily coding agent, and it gave me a clean way to hand implementation packets to Codex.
\nPBC v2.0 uses the Codex CLI directly.
\ncodex exec -m gpt-5.5 -c model_reasoning_effort="low" -c service_tier="fast" < "$PACKET_PATH"\nThis does not remove a dependency. It swaps OpenCode for the Codex CLI, which is the tool doing the implementation work anyway.
\nIf you have the Codex CLI authenticated and you have access to gpt-5.5, you are good.
Claude Code deprecated commands/ in favor of skills.
That is the right direction. Commands were a reasonable starting point, but skills fit the workflow better. They carry instructions, context, and invocation behavior in one place.
\nPBC now has seven skills instead of five commands.
\n| Skill | Purpose |
|---|---|
| /pbc:research-codebase | Parallel sub-agent codebase exploration |
| /pbc:create-plan | Interactive planning with optional stress testing |
| /pbc:grill-me | Decision-tree interrogation of a plan or design |
| /pbc:implement-plan | Packet-based Codex implementation |
| /pbc:review-work | Independent review loops with triage |
| /pbc:address-pr-comments | PR comment triage and fix delegation |
| /pbc:handoff | Session continuation documents |
The shape is the same. The packaging is cleaner.
\nThe first version shipped eight named Sonnet sub-agent definitions. codebase-locator, codebase-analyzer, pattern-finder, and several more.
I removed all of them.
\nThe skills now describe what needs to be researched. Opus decides how to delegate using Claude Code's Agent tool. There are no separate agent spec files to maintain.
\nThe research quality has not dropped because the useful guidance was never the agent name. It was the instruction.
\nThe named agents were adding indirection. They made the plugin feel more deterministic, but they also made it more brittle.
\n/pbc:grill-me is new.
This one is inspired by Matt Pocock. The idea is simple. Before implementation starts, stress-test the plan.
\nIt walks the decision tree. Scope. Data model. APIs. Migration. Testing. Rollout. Security. Failure modes.
\nIt asks one question at a time with AskUserQuestion, so each answer can shape the next question. It does not dump twenty questions on you and pretend that is collaboration.
It also checks whether it can answer the question itself before bothering you. It explores the codebase. It can use WebSearch when that is appropriate. If the answer is already knowable, it should not ask.
\nYou only get asked questions that require human judgment.
\ngrill-me can run standalone against a plan or design. It can also run inside /pbc:create-plan before the final plan is written.
Is it perfect? No. But it catches unclear decisions before Codex starts writing code, and that is exactly where I want those problems found.
\n/pbc:handoff is also new.
This is also inspired by Matt Pocock. Long sessions end. Context gets compacted. Humans go to sleep. Work still needs a clean continuation path.
\nThe handoff skill writes a continuation document to thoughts/handoffs/. It references artifacts by path instead of duplicating them. It includes branch state, commit state, what happened, what is still open, and which skills to run next.
Most of the time, the artifact chain is already the handoff mechanism.
\nResearch finishes and suggests /pbc:create-plan with the research artifact path. Create-plan finishes and suggests /pbc:implement-plan with the plan path. Implement-plan produces packets. Review-work produces review artifacts and fix packets.
The explicit handoff skill is for session boundaries. It is the thing I run when I want a fresh session to pick up without guessing.
\nPBC v1 had 30 files. PBC v2.0 has 11.
\nThat is the main change.
\nThe pipeline is still Research, Plan, Implement, Review. Independent review loops still catch bugs. The artifact trail still gives traceability. Human gates still exist at every step.
\nThe difference is that the plugin trusts the model more.
\nInstead of prescribing which sub-agent to spawn with a detailed spec, the skill says what to research and lets Opus figure out the details. That is less deterministic. It is also less brittle.
\nI care more about the artifact quality than the ceremony that produced it.
\n/pbc:address-pr-comments is one of my favorite skills.
Even after an independent review pass, you still get PR comments. Humans see things the agents miss. AI reviewers leave useful notes too. The annoying part is turning that pile of comments into clean follow-up work.
\nThis skill fetches the comments and triages each one.
\nIt presents the triage for approval before anything changes. Then it generates one fix packet per comment.
\nOne packet per comment gives clean traceability. The user alignment gate means I am never surprised by what the agent decided to fix.
\nIt is a quick way to get review feedback sorted, approved, and resolved without losing the thread.
\nPBC started as a Claude Code plugin, but the workflow is not Claude-specific.
\nThere is now a Codex plugin at github.com/mattlgroff/groffdev-codex-plugin with the same skills at feature parity.
\nThe Codex version spawns codex exec sub-agents instead of using Claude Code's Agent tool. The workflow and artifact structure are identical.
That is the part I care about. Research artifacts. Plan artifacts. Implementation packets. Review loops. Human gates.
\nThe tool can change. The workflow should survive that.
\nFor Claude Code:
\nclaude plugin marketplace add mattlgroff/pbc\nclaude plugin install pbc@mattlgroff-pbc --scope project\nFor Codex, use the README in the Codex plugin repo:
\nopen https://github.com/mattlgroff/groffdev-codex-plugin\nBoth versions require the Codex CLI with gpt-5.5 access.
\n\nUpdate, August 2026: I got the market need right, but I got the title wrong. I expected AI Deployment Engineer and AI Deployment Specialist to emerge as separate roles. Since publishing this, the market has increasingly standardized around Forward Deployed Engineer, or FDE, as the catch-all role for this work.
\nThe responsibilities did not disappear. They converged. A credible FDE now needs full-stack engineering from proof of concept through production, client-facing consulting judgment, and AI-native engineering across agentic coding, production agents, evals, MCP, and Skills. We have not seen enough market value in introducing another title, so Elios now uses FDE.
\n
Every team needs to become AI-native.
\nThat sounded obvious when I first wrote this post, but the next question was harder:
\nWhat do you actually do?
\nCompanies bought ChatGPT. Engineers adopted GitHub Copilot, Cursor, Claude Code, and Codex. Finance wanted help with Excel. Marketing wanted help with decks. Operations wanted workflows automated. Engineering wanted agentic coding, MCP servers, and safer ways to ship faster.
\nBuying the tools was the easy part.
\nDeploying them was the hard part.
\nThat part of the original argument still holds. Where I was wrong was assuming the market needed new titles for the people doing the work.
\nI called them AI Deployment Engineers and AI Deployment Specialists. The market kept using Forward Deployed Engineer as the broader category instead.
\nThat is okay. It is useful to make a prediction, watch the market, and change your mind when the evidence changes.
\nOpenAI's current Forward Deployed Engineer role owns discovery, technical scoping, system design, implementation, production rollout, adoption, and measurable workflow impact.
\nOde with Anthropic is organized around the same basic need: experienced engineers and operators working with client experts to turn frontier AI into production systems and measurable business results.
\nDifferent companies divide responsibilities differently inside their teams. That does not mean buyers want another collection of overlapping job titles.
\nThe market found a useful catch-all: FDE.
\nAn FDE is not just a software engineer working at a customer site. An FDE is also not a consultant who can talk about AI but cannot build the system. The role sits at the intersection of engineering, consulting, and AI-native delivery.
\nMost companies are still not AI-native.
\nThey may use ChatGPT. They may have Microsoft Copilot. They may have a few internal experiments or an AI roadmap sitting in a planning document.
\nThat is not the same thing as deploying AI into the company.
\nThe gap shows up everywhere.
\nOn the engineering side, the AI roadmap often has no owner. Engineers context-switch between exploration and real delivery. Pilots never make it into production. License costs accrue while adoption stalls.
\nAcross the rest of the business, people are still assembling reports, presentations, approvals, and handoffs manually. The technology is not usually the bottleneck.
\nThe bottleneck is deployment.
\nA Forward Deployed Engineer is a full-stack, client-facing, AI-native engineer who embeds with a company, turns an ambiguous business workflow into a production system, and leaves behind measurable value and a capability the client can own.
\nThat definition has three pillars. All three are required.
\nAn FDE has to build.
\nThey understand frontend and backend systems, APIs, data models, authentication, infrastructure, testing, observability, and production risk. They can enter an unfamiliar repository, learn the system, make sound architectural decisions, and ship code that the client's team can maintain.
\nThey know the difference between a demo and a proof of concept. They also know the difference between a proof of concept and production.
\nThe job includes:
\nAn FDE who cannot take work from POC to production is missing the engineering core of the role.
\nAn FDE also has to work directly with people.
\nThe client rarely arrives with a perfect problem statement. The workflow is messy. Stakeholders disagree. Important details live in meetings, spreadsheets, tribal knowledge, and exceptions that nobody wrote down.
\nThe FDE has to find the real problem before moving quickly in the wrong direction.
\nThat means:
\nThis does not turn every FDE into an account executive or project manager. It means an FDE cannot disappear into the code and expect someone else to translate the client forever.
\nThe third pillar is not a list of fashionable tools.
\nClaude Code, Codex, MCP, and Skills matter because they change how we build and what we can build. The competency is knowing how to use them with engineering judgment.
\nAn AI-native FDE can:
\nUsing Claude Code or Codex every day is useful. Shipping vibe-coded slop faster is not the standard.
\nEvals are not a final testing phase or a dashboard added after the build. They are the control loop for AI product development.
\nMy baseline here is Hamel Husain's work on domain-specific eval systems.
\nThe FDE should define success for the actual workflow, inspect traces, perform error analysis, build a durable dataset, and write scoped tests around observed failure modes. Deterministic assertions should be used wherever possible. Human evaluation should cover judgment that code cannot express. Model-based judges should be calibrated against human labels instead of trusted because they produce a score.
\nThose evals should run during development, in CI, and against production traces. When a real user finds a new failure, that failure should become a regression case.
\nWithout that loop, the team is still guessing.
\nThere is a version of AI adoption that looks like this:
\nThat does not work very well.
\nGeneric webinars do not change the way a team operates. People need someone to sit with them, understand the work, connect the right tools, write the right instructions, and make sure the workflow survives real use.
\nThe opposite mistake is treating every AI use case like a six-month custom software project.
\nSome workflows need a production application. Others need a governed connector, a Skill, a targeted automation, or a better operating process. The FDE's job is to make that judgment based on the workflow rather than forcing every problem into the same shape.
\nAI deployment is difficult to standardize because every team is different.
\nFinance pulls from a different stack than marketing. Legal has different risk constraints than sales. Two engineers on the same squad may need different data sources, local tooling, and agent workflows.
\nA generic rollout misses that.
\nA proper deployment asks better questions:
\nThe answer will look different for every team.
\nThat is the point.
\nAt Elios, we have not seen a need to keep teaching the market a separate AI Deployment Engineer category. Multiple overlapping roles created more confusion than clarity.
\nWe now use Forward Deployed Engineer as the market-facing role. An FDE may lean more heavily into architecture, delivery, domain work, or adoption on a particular engagement, but the standard still includes all three pillars.
\nThat is also how I describe my own work now. I am a Lead Forward Deployed Engineer. My official title is VP of Technology at Elios AI.
\nThe title changed. The deployment problem did not.
\nCompanies do not need more AI theater. They need people who can enter the real environment, understand the real workflow, build the real system, measure whether it works, and leave the team stronger.
\nThat is the Forward Deployed Engineer.
\nThe companies that win will not be the ones that buy the most AI licenses. They will be the ones that deploy AI into useful parts of the business with the right mix of engineering, consulting, evals, adoption, and human judgment.
\nThe real work is not buying AI.
\nThe real work is making the team AI-native.
\nIf your company needs an FDE, or if this is the kind of engineer you want to become, I would love to connect.
\nYou can find me on LinkedIn, email me at matt@eliosai.com, or book a 30-minute AI strategy session.
", "url": "https://www.groff.dev/blog/dawn-of-the-ai-deployment-engineer", "title": "The Dawn of the Forward Deployed Engineer", "summary": "I got the AI deployment gap right and the job title wrong. The market has converged on Forward Deployed Engineer, a role built on full-stack engineering, client-facing consulting, and AI-native delivery.", "date_modified": "2026-05-08T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/mcp-server-nestjs-auth0-azure-ad-sso", "content_html": "If you run an enterprise app with Azure AD SSO and you try to add MCP server support, you will hit this error:
\nAADSTS9010010: The resource parameter provided in the request\ndoesn't match with the requested scopes.\nThis happens because MCP clients are required to send a resource parameter (per RFC 8707) in the OAuth authorization request. When your identity provider federates through Azure AD (Microsoft Entra), that parameter gets forwarded. Azure AD rejects it.
This is not a niche edge case. It affects Microsoft's own Power BI MCP server, IBM's MCP Context Forge, MCP Inspector, and basically anyone trying to put MCP behind enterprise SSO. The upstream spec issue (modelcontextprotocol/modelcontextprotocol#1614) requesting resource be made optional is still open with no resolution.
Microsoft even tightened Entra's enforcement of this in March 2026, converting what had been a latent incompatibility into active breakage across production systems.
\nThe standard enterprise answer is to add infrastructure:
\nresource parameter. Pragmatic, but not spec-compliant.All of these work. All of them add moving parts. I wanted something simpler.
\nI have a NestJS API that serves a React/Next.js frontend. Auth is handled by Auth0, which federates to Azure AD for enterprise SSO. Users log in through their organization's Microsoft account. Standard enterprise setup.
\nI wanted to expose MCP tools on this same API so that Claude Code (and other MCP clients) could call them with the same authentication. No new infrastructure. No proxy. No gateway. Just Streamable HTTP, not SSE. One new endpoint, reusing the existing OAuth strategy.
\nYou can get more complex with this (SSE transport, dynamic client registration, dedicated OAuth clients), but I did not need any of that.
\nHere is what I ended up with:
\nclientId explicitly in its config.resource parameter before it reaches Azure AD.WWW-Authenticate header on 401. The well-known endpoint tells MCP clients where to authenticate.That is the entire auth layer. No gateway, no proxy, no new OAuth clients.
\nThe MCP spec (March 2025 version) said implementations "SHOULD" support dynamic client registration. Most MCP tutorials and client implementations treated this as mandatory. If your server did not have a DCR endpoint, many clients would fail.
\nThe November 2025 spec revision changed this significantly. It now defines three registration methods in explicit priority order:
\nPre-registration is not a workaround. It is the spec's preferred path for enterprise deployments.
\nThis makes sense. Some enterprise identity providers are starting to add DCR support, but adoption is uneven and often still in beta. Waiting around for full DCR support across your identity stack is not a practical path for shipping today.
\nMore importantly, DCR is a security risk in enterprise contexts. It allows unlimited app registrations, which is the opposite of what you want when you are building an SSO-grade application with controlled access. Auth0's own documentation recommends static registration for production and explicitly calls DCR a security risk that requires Enterprise-tier controls.
\nMCP clients must send a resource parameter per RFC 8707. The MCP spec converts RFC 8707's "MAY" into a "MUST." This is a deliberate deviation that breaks interoperability with identity providers that reject the parameter.
Auth0 historically used its own audience parameter instead of the standard resource. Without the compatibility profile enabled, Auth0 does not know which API to issue a token for when it receives resource. It falls back to an encrypted JWE token (5-part) instead of a standard JWT (3-part), and the whole flow breaks.
The Resource Parameter Compatibility Profile (Auth0 Dashboard, Settings, Advanced) makes Auth0 treat resource as equivalent to audience for token audience determination. Critically, it does not forward the resource parameter to upstream identity providers. That is what prevents Azure AD from seeing it and rejecting the request.
This feature entered limited early access in November 2025, driven specifically by MCP adoption. It is documented but not prominently. You will find it in Auth0's AI/MCP-specific docs, not the main Auth0 documentation.
\nFour things to configure in Auth0:
\n1. Enable the Resource Parameter Compatibility Profile. Auth0 Dashboard, Settings, Advanced tab. Toggle it on.
\n2. Register your MCP server URL as an Auth0 API. Auth0 needs to know your server URL is a valid audience. Create an API with the identifier set to your server URL (e.g., http://localhost:3000/api for local dev). This is what the resource parameter will resolve to.
3. Promote your Azure AD connection to domain-level. This makes the connection available to all applications in the tenant without per-app configuration.
\n# Find your connection ID (look for strategy: "waad")\nauth0 api get connections\n\n# Promote to domain-level\nauth0 api patch connections/YOUR_CONNECTION_ID \\\n --data '{"is_domain_connection": true}'\n4. Add the MCP callback URL to your existing application. Add http://localhost:8976/callback (or your chosen port) to the Allowed Callback URLs on the same Auth0 application your frontend uses.
You need two things on the server: an auth guard for the MCP endpoint, and a well-known metadata endpoint.
\nThe guard validates the auth context your existing middleware sets. When auth fails, it returns a 401 with a WWW-Authenticate header pointing to your OAuth metadata. This header is what triggers MCP clients to start the OAuth discovery flow.
import { Injectable, CanActivate, ExecutionContext, UnauthorizedException } from '@nestjs/common';\nimport type { Request, Response } from 'express';\n\n@Injectable()\nexport class McpAuthGuard implements CanActivate {\n canActivate(context: ExecutionContext): boolean {\n const request = context.switchToHttp().getRequest<Request>();\n const auth = (request as any).auth;\n\n if (auth?.userId && auth?.type === 'enterprise' && auth?.orgId) {\n return true;\n }\n\n // Set the WWW-Authenticate header before throwing so it survives the exception filter.\n const apiUrl = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:3000/api';\n const baseUrl = apiUrl.replace(/\\/api\\/?$/, '');\n const response = context.switchToHttp().getResponse<Response>();\n response.setHeader(\n 'WWW-Authenticate',\n `Bearer resource_metadata="${baseUrl}/.well-known/oauth-protected-resource"`\n );\n\n throw new UnauthorizedException('Unauthorized');\n }\n}\nImportant: throw the exception. Do not manually write a response body and return false. NestJS throws a ForbiddenException when a guard returns false, which causes a double-response crash (Cannot set headers after they are sent).
The well-known endpoint implements RFC 9728 (OAuth Protected Resource Metadata). It tells MCP clients where to authenticate. This must be served outside your API prefix (at /.well-known/oauth-protected-resource, not /api/.well-known/...).
import { Controller, Get } from '@nestjs/common';\nimport { ProtectedResourceMetadataBuilder, BearerMethod, SigningAlgorithm } from '@auth0/auth0-api-js';\nimport { SkipThrottle } from '@nestjs/throttler';\n\n@Controller('.well-known')\n@SkipThrottle()\nexport class McpWellKnownController {\n @Get('oauth-protected-resource')\n getProtectedResourceMetadata() {\n const apiUrl = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:3000/api';\n const auth0Domain = process.env.AUTH0_DOMAIN;\n if (!auth0Domain) {\n return { error: 'OAuth authorization server not configured.' };\n }\n\n const metadata = new ProtectedResourceMetadataBuilder(\n apiUrl,\n [`https://${auth0Domain}`]\n )\n .withBearerMethodsSupported([BearerMethod.HEADER])\n .withResourceSigningAlgValuesSupported([SigningAlgorithm.RS256])\n .withScopesSupported(['openid', 'profile', 'email'])\n .build();\n\n return metadata.toJSON();\n }\n}\nThis endpoint must be anonymous (no auth required). MCP clients hit it before they have a token. If you have a global auth guard, make sure this route is excluded or marked as public.
\nConsider relaxing rate limiting on this endpoint. MCP clients may poll it during the OAuth discovery flow, and strict throttling can cause authentication to fail. I skip it entirely in my setup, but a generous limit would also work.
\nThe resource field in this response must be your actual server URL, not the Auth0 audience URL. MCP clients use it to match the resource they are connecting to. If it does not match, you get SDK auth failed: Protected resource does not match expected.
For the MCP transport itself, I use @rekog/mcp-nest. It gives you decorator-based tool registration with full NestJS dependency injection.
\nThe module setup has two parts. A shared module registers tool classes with forFeature() and imports the domain modules those tools need:
import { Module } from '@nestjs/common';\nimport { McpModule as RekogMcpModule } from '@rekog/mcp-nest';\nimport { MyToolClass } from './tools/my.tools';\n\n@Module({\n imports: [\n RekogMcpModule.forFeature([MyToolClass], 'your-server-name'),\n // domain modules your tools depend on\n ],\n providers: [MyToolClass],\n exports: [MyToolClass]\n})\nexport class McpSharedModule {}\nA root module in your app configures the transport, attaches the auth guard, and imports the shared module:
\nimport { Module } from '@nestjs/common';\nimport { McpModule as RekogMcpModule, McpTransportType } from '@rekog/mcp-nest';\nimport { McpAuthGuard } from './mcp-auth.guard';\nimport { McpWellKnownController } from './mcp-well-known.controller';\nimport { McpSharedModule } from './mcp-shared.module';\n\n@Module({\n imports: [\n RekogMcpModule.forRoot({\n name: 'your-server-name',\n version: '1.0.0',\n description: 'Your MCP server description',\n transport: McpTransportType.STREAMABLE_HTTP,\n streamableHttp: {\n statelessMode: true,\n enableJsonResponse: true\n },\n guards: [McpAuthGuard]\n }),\n McpSharedModule\n ],\n controllers: [McpWellKnownController]\n})\nexport class McpApiModule {}\nThis split keeps tool registration separate from transport configuration. The shared module can be tested independently, and the tool classes stay focused on input validation and service delegation.
\nTool classes are ultra-thin. They validate input with Zod, call an authorized service method, and return the result. No business logic, no authorization logic. That all lives in the service layer.
\nA few design principles that matter for agent ergonomics:
\nprojectId not project. clientName not client.add_project_note needs a project ID, say "Use search_projects to find the project ID first."import { Injectable } from '@nestjs/common';\nimport { Tool } from '@rekog/mcp-nest';\nimport { z } from 'zod';\nimport { ProjectsService } from '../projects';\n\nconst projectStatuses = ['active', 'completed', 'on_hold'] as const;\n\n@Injectable()\nexport class McpProjectTools {\n constructor(private readonly projectsService: ProjectsService) {}\n\n @Tool({\n name: 'search_projects',\n description:\n 'Search projects by keyword, status, or client name. Returns matching records ' +\n 'with project IDs. Use project IDs from this tool as input to add_project_note.',\n parameters: z.object({\n search: z.string().optional().describe('Fuzzy search on project name or description.'),\n status: z.enum(projectStatuses).optional(),\n clientName: z.string().optional().describe('Exact or partial client organization name.'),\n limit: z.number().int().min(1).max(25).optional().describe('Max results (default 10).')\n })\n })\n async searchProjects(params: {\n search?: string;\n status?: string;\n clientName?: string;\n limit?: number;\n }) {\n const result = await this.projectsService.search(params);\n return { content: [{ type: 'text' as const, text: JSON.stringify(result) }] };\n }\n\n @Tool({\n name: 'add_project_note',\n description:\n 'Add a note to a project. Use search_projects to find the project ID first.',\n parameters: z.object({\n projectId: z.string().describe('The project ID from search_projects.'),\n content: z.string().min(1).max(5000).describe('Note body text.'),\n title: z.string().optional().describe('Short title for the note.')\n })\n })\n async addProjectNote(params: { projectId: string; content: string; title?: string }) {\n const result = await this.projectsService.addNote(params);\n return { content: [{ type: 'text' as const, text: JSON.stringify(result) }] };\n }\n}\nThe pattern is the same for every tool: Zod schema in, service call, JSON out. Your service methods handle authorization, entitlement checks, and business logic. The tool class is just the interface.
\nThis example is for local development. In your .mcp.json (project-level or ~/.claude.json global):
{\n "mcpServers": {\n "your-server": {\n "type": "http",\n "url": "http://localhost:3000/api/mcp",\n "oauth": {\n "clientId": "YOUR_AUTH0_CLIENT_ID",\n "callbackPort": 8976,\n "authServerMetadataUrl": "https://YOUR_TENANT.us.auth0.com/.well-known/openid-configuration"\n }\n }\n }\n}\nIn production, you would replace the url with your live server URL and drop the callbackPort (that is only needed for local OAuth redirects).
The clientId is required. This is the same client ID your React frontend uses. Because we are not using dynamic client registration, the MCP client needs to know which OAuth application to authenticate against.
The authServerMetadataUrl points directly to Auth0's OIDC discovery endpoint. This helps the MCP SDK find the authorization and token endpoints without relying on the MCP server to provide them.
Two things need to happen in main.ts: exclude the well-known path from your global API prefix, and add MCP headers to CORS.
// Exclude .well-known from /api prefix so it's served at the root\napp.setGlobalPrefix('api', {\n exclude: [\n { path: '.well-known/oauth-protected-resource', method: RequestMethod.GET },\n { path: '.well-known/oauth-protected-resource', method: RequestMethod.OPTIONS }\n ]\n});\n\napp.enableCors({\n origin: corsAllowedOrigins,\n credentials: true,\n allowedHeaders: [\n 'Content-Type',\n 'Authorization',\n 'X-Requested-With',\n 'Accept',\n 'Origin',\n 'Access-Control-Request-Method',\n 'Access-Control-Request-Headers',\n 'User-Agent', // AI SDK and MCP clients set this; Safari/Firefox require it in CORS allowedHeaders\n 'Mcp-Session-Id'\n ],\n exposedHeaders: ['Content-Length', 'Content-Type', 'Mcp-Session-Id'],\n methods: ['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'OPTIONS'],\n maxAge: 3600\n});\nThe Mcp-Session-Id header is required in both allowedHeaders and exposedHeaders. The User-Agent header matters because some MCP clients and AI SDKs set it, and Safari and Firefox will reject the preflight if it is not explicitly allowed.
MCP Client (Claude Code)\n |\n +-- POST /api/mcp -> 401 + WWW-Authenticate header\n |\n +-- GET /.well-known/oauth-protected-resource -> Auth0 server URL\n |\n +-- OAuth flow with Auth0 (resource param intercepted, NOT forwarded)\n | +-- Azure AD SSO (clean request, no resource param)\n |\n +-- Receives JWT access token (audience: your server URL)\n |\n +-- POST /api/mcp + Authorization: Bearer <token>\n +-- McpAuthGuard validates -> tools execute\nThe key insight is in the middle: Auth0 intercepts the resource parameter and uses it to determine which API to issue a token for, but never forwards it to Azure AD. Azure AD sees a clean authorization request. It authenticates the user through SSO and returns. Auth0 mints a JWT with the correct audience. The MCP client gets a usable token.
I want to be explicit about scope. This setup is for a private, secured application. MCP users must already have login access to the organization's UI. They authenticate with the same credentials, through the same SSO flow, against the same OAuth client.
\nIf you are building a public MCP server that arbitrary clients should be able to connect to, you probably do want dynamic client registration or CIMD. This post is not about that use case.
\nThis is about taking an existing enterprise app with existing enterprise auth and adding MCP as a transport layer on top of it. No new identity infrastructure. No new OAuth clients. No gateway.
\nThe November 2025 MCP spec update and the MCP blog post on evolving client registration signal a clear direction:
\nAuth0 is positioning CIMD as the future of MCP client registration. Aaron Parecki (now at Auth0, previously OAuth spec editor) has been writing extensively about enterprise-ready MCP auth patterns.
\nThe resource parameter conflict is still unresolved at the spec level. The maintainers' position (spec compliance takes priority, Entra should adapt) is defensible from a standards perspective. But for developers who cannot change Azure AD's behavior, the practical question is: what works today?
For Auth0 customers with Azure AD federation, the answer is the Resource Parameter Compatibility Profile plus pre-registered client reuse. It is the simplest working path I have found.
\nAADSTS9010010 error. The Resource Parameter Compatibility Profile is not enabled. Auth0 Dashboard, Settings, Advanced. Toggle it on.
Token is a JWE (5-part) instead of JWT (3-part). Same cause. Without the profile, Auth0 does not know which API to scope the token to.
\nSDK auth failed: Protected resource does not match expected. The resource field in your /.well-known/oauth-protected-resource response must match your MCP server URL (e.g., http://localhost:3000/api), not the Auth0 audience.
Cannot set headers after they are sent. Your guard is writing a response body manually and then returning false. Throw UnauthorizedException instead.
needs authentication but no browser redirect. Check that your Azure AD connection is promoted to domain-level and that the clientId in your MCP config matches your Auth0 application.
If you found this post helpful or have questions, feel free to connect with me on LinkedIn. It's the best place to reach me.
", "url": "https://www.groff.dev/blog/mcp-server-nestjs-auth0-azure-ad-sso", "title": "How to Build an MCP Server with NestJS, Auth0, and Azure AD SSO", "summary": "Step-by-step guide to building an MCP server on NestJS with Auth0 OAuth and Azure AD SSO. Fixes the AADSTS9010010 resource parameter error without a gateway, proxy, or dynamic client registration.", "date_modified": "2026-04-04T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/run-qwen-3-5-9b-uncensored-locally-with-opencode", "content_html": "If you want a local model that runs on a single RTX 4090, works with OpenCode, and is far less likely to refuse your prompts, this stack is a lot of fun.
\nIt is also the kind of local model you can realistically run on a 3090 or 4090 without needing some ridiculous multi-GPU box.
\nIf you are searching for the best uncensored local LLM for an RTX 3090 or RTX 4090, this is the sort of setup worth looking at.
\nI do not know, or want to know, what you will do with it. I just know it is a very capable model, and this one is much less interested in telling you no.
\nI have been testing Qwen3.5-9B-Uncensored-HauhauCS-Aggressive with llama-server from llama.cpp, then using it through OpenCode. It works.
This setup was inspired by the excellent llama.cpp discussion here:
\n\nThat thread is worth reading. It covers the broader pattern of using local OpenAI-compatible servers with coding agents. This post is the focused version for one specific stack: uncensored Qwen 3.5 9B, llama-server, and OpenCode.
The model card for Qwen3.5-9B-Uncensored-HauhauCS-Aggressive makes the pitch pretty clearly:
\nThe BF16 file is large, but a 4090 can handle it. If you are looking for a local LLM for 24GB VRAM, this is exactly why the setup is interesting. In my case, I was able to run the BF16 GGUF fully on GPU with llama-server and a 131072 context window.
That also puts it in the category of models people actually care about: stuff you can run on a single RTX 3090 or RTX 4090 at home.
\nIf you want the exact model page, it is here:
\n\nI keep a tiny helper script called qwen.sh in my working directory. This starts llama-server, waits for the health check, and also serves a simple local chat UI.
#!/bin/bash\nSCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"\nMODEL="/home/matt/models/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf"\nSERVER="/home/matt/llama.cpp/build/bin/llama-server"\nAPI_PORT=8080\nCHAT_PORT=8081\nPID_FILE="/tmp/qwen-server.pid"\nCHAT_PID_FILE="/tmp/qwen-chat.pid"\n\nstart() {\n if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then\n echo "Already running (PID $(cat "$PID_FILE"))"\n else\n echo "Starting Qwen3.5-9B on port $API_PORT..."\n nohup "$SERVER" \\\n --model "$MODEL" \\\n --host 0.0.0.0 \\\n --port "$API_PORT" \\\n --n-gpu-layers 999 \\\n --ctx-size 131072 \\\n --temp 0.6 \\\n --top-p 0.95 \\\n --top-k 20 \\\n > /tmp/qwen-server.log 2>&1 &\n echo $! > "$PID_FILE"\n echo "PID $! - waiting for model to load..."\n until curl -s "http://localhost:$API_PORT/health" | grep -q '"ok"'; do sleep 2; done\n echo "API ready at http://localhost:$API_PORT/v1"\n fi\n\n if [ -f "$CHAT_PID_FILE" ] && kill -0 "$(cat "$CHAT_PID_FILE")" 2>/dev/null; then\n echo "Chat UI already running (PID $(cat "$CHAT_PID_FILE"))"\n else\n nohup python3 -m http.server "$CHAT_PORT" --directory "$SCRIPT_DIR" --bind 0.0.0.0 \\\n > /tmp/qwen-chat.log 2>&1 &\n echo $! > "$CHAT_PID_FILE"\n echo "Chat UI at http://localhost:$CHAT_PORT/chat.html"\n fi\n}\n\nstop() {\n for name_pid in "API server:$PID_FILE" "Chat UI:$CHAT_PID_FILE"; do\n name="${name_pid%%:*}"\n pf="${name_pid#*:}"\n if [ -f "$pf" ] && kill -0 "$(cat "$pf")" 2>/dev/null; then\n kill "$(cat "$pf")"\n rm -f "$pf"\n echo "$name stopped."\n else\n echo "$name not running."\n rm -f "$pf"\n fi\n done\n}\n\nstatus() {\n if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then\n echo "API: running (PID $(cat "$PID_FILE")) - http://localhost:$API_PORT/v1"\n curl -s "http://localhost:$API_PORT/health" 2>/dev/null || echo " (not responding)"\n else\n echo "API: not running"\n fi\n if [ -f "$CHAT_PID_FILE" ] && kill -0 "$(cat "$CHAT_PID_FILE")" 2>/dev/null; then\n echo "Chat UI: running (PID $(cat "$CHAT_PID_FILE")) - http://localhost:$CHAT_PORT/chat.html"\n else\n echo "Chat UI: not running"\n fi\n}\n\ncase "${1:-}" in\n start) start ;;\n stop) stop ;;\n restart) stop; sleep 2; start ;;\n status) status ;;\n log) tail -f /tmp/qwen-server.log ;;\n *) echo "Usage: $0 {start|stop|restart|status|log}" ;;\nesac\nThe important parts are:
\n--ctx-size 131072 to keep a large context window--n-gpu-layers 999 so the model fully offloads to GPU if it fitsYou do need a recent llama.cpp build. Qwen 3.5 support landed very recently.
Once the server is up, verify that it is healthy and exposing an OpenAI-compatible API:
\ncurl http://localhost:8080/health\ncurl http://localhost:8080/v1/models\nYou should see your model in the /v1/models response, something like:
Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf\nAnd if you want to test a chat completion directly:
\ncurl http://localhost:8080/v1/chat/completions \\\n -H "Content-Type: application/json" \\\n -d '{\n "model": "Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf",\n "messages": [{"role": "user", "content": "Reply with exactly: pong"}],\n "max_tokens": 128,\n "temperature": 0\n }'\nOpenCode can talk to any OpenAI-compatible provider, which is exactly what llama-server gives us.
Add this to your global OpenCode config at ~/.config/opencode/opencode.json:
{\n "$schema": "https://opencode.ai/config.json",\n "provider": {\n "llama.cpp": {\n "name": "llama-server (local)",\n "npm": "@ai-sdk/openai-compatible",\n "options": {\n "baseURL": "http://127.0.0.1:8080/v1"\n },\n "models": {\n "qwen3.5-9b-uncensored-local": {\n "id": "Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf",\n "name": "Qwen3.5-9B Uncensored (local)",\n "reasoning": true,\n "tool_call": true,\n "temperature": true,\n "interleaved": {\n "field": "reasoning_content"\n },\n "limit": {\n "context": 131072,\n "output": 8192\n },\n "modalities": {\n "input": ["text"],\n "output": ["text"]\n }\n }\n }\n }\n }\n}\nTwo notes here:
\nid needs to match the model name exposed by llama-server.interleaved.reasoning_content bit helps OpenCode deal with Qwen's reasoning output more cleanly.This OpenCode wiring follows the same general pattern discussed in the llama.cpp guide, just adapted for this exact model.
\nopencodeqwen Just WorkOnce the provider is configured, you can add a helper function to ~/.bashrc so opencodeqwen starts the server if needed and launches OpenCode on the local uncensored model.
opencodeqwen() {\n local model="llama.cpp/qwen3.5-9b-uncensored-local"\n\n if ! curl -fsS http://127.0.0.1:8080/health >/dev/null 2>&1; then\n /home/matt/working/qwen.sh start || return $?\n fi\n\n command opencode --model "$model" "$@"\n}\nThen reload your shell:
\nsource ~/.bashrc\nAnd now this works:
\nopencodeqwen\nOr for a one-shot run:
\nopencodeqwen run "summarize this repo"\nThis combination hits a useful sweet spot:
\nThat last point matters more than people admit. I got into repeated fights with GPT-5.2 on OpenClaw over things it was sure were security issues, including passwords in an .env file. It would just flat out refuse, no matter how hard I pushed. Sometimes you want a model that will simply attempt the task instead of moralizing, stalling, or collapsing into canned refusals.
Running this stuff yourself matters. It means you are not boxed into whatever Sam Altman or Dario decide you should be allowed to do that week.
\nThis is not magic. There are still tradeoffs.
\nQwen 3.5 is new. If your local llama-server build is old, update it first.
I like keeping 131072 context available because it's generally enough and has a real VRAM cost.
The model card notes that it may still occasionally append a mild disclaimer, but it is not refusing the task. In practice, the important part is that it keeps generating the actual answer.
\nIf you intentionally choose an uncensored model, that is a power tool. Use good judgment. Please use it responsibly.
\nAlso, to be clear, this is not a coding model. It is capable, weirdly useful, and much more willing than the heavily aligned stuff, but it is not especially good at coding compared to stronger frontier models. The win here is freedom and local control, not raw coding quality.
\nIf you want the original references and downloads, start here:
\nIf you have a 3090 or 4090 sitting around and want an uncensored local model that will actually take a swing at the job, this is one of the more interesting setups I have tried lately. It is hackery in the best way. Rough edges, full control, and a lot fewer lectures.
", "url": "https://www.groff.dev/blog/run-qwen-3-5-9b-uncensored-locally-with-opencode", "title": "Run Qwen 3.5 9B Uncensored Locally on a 3090 or 4090 with OpenCode", "summary": "A practical guide to running Qwen 3.5 9B Uncensored as a local model on a single RTX 3090 or RTX 4090 with llama-server, then wiring it into OpenCode.", "date_modified": "2026-03-07T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/how-to-make-your-web-app-agent-native", "content_html": "The web has a new class of user: AI agents.
\nThey discover content, reason over it, and take actions for people and software workflows.
\nIf agents are first class users of your product, your app needs to be understandable and operable for them too.
\nAgent native apps are built in layers. Discovery, content negotiation, action semantics, safety boundaries, and tests all have to line up.
\nThere are competing standards and conventions right now. This is my best effort to cover the major ones that matter today, with concrete examples you can implement.
\nThink of agent support as progressive enhancement:
\nI think of this as a 7-layer burrito, but for agent-native websites.
\nllms.txt, agents.json)<link rel="alternate">, capability meta)Each layer helps on its own. Together they make your app predictable for agents and robust for humans.
\nStart with two static files in public/:
/.well-known/agents.json/llms.txtFor agents.json, follow the Wildcard AI spec:
Keep chains task oriented and explicit. Avoid vague tools like "handle_user_request". Define narrow actions with clear parameter contracts.
\n{\n "apiVersion": "0.1.0",\n "name": "City Services Portal",\n "description": "Residents report non-emergency city issues and track resolution status.",\n "baseUrl": "https://api.example.com",\n "chains": {\n "report_streetlight_outage": {\n "description": "Create a new outage report for a streetlight with optional photo evidence.",\n "steps": [\n {\n "endpoint": "/issues/streetlight-outages",\n "method": "POST",\n "content_type": "multipart/form-data",\n "parameters": {\n "address_id": { "type": "string", "required": true },\n "description": { "type": "string", "required": true },\n "photo": { "type": "file", "required": false },\n "contact_email": {\n "type": "string",\n "required": false,\n "format": "email"\n }\n }\n }\n ]\n }\n }\n}\nFor llms.txt, keep it concise and operational. Think "runbook for an agent," not marketing copy:
<head>Many agents parse HTML first, and some agents such as Claude Code may request markdown first when available. In both cases, give them immediate pointers.
\n<link rel="alternate" type="text/plain" href="/llms.txt" title="LLM-readable information" />\n<link rel="alternate" type="application/json" href="/.well-known/agents.json" title="Agent actions" />\n<meta\n name="agent-capabilities"\n content="Report streetlight outages, check issue status, and retrieve incident reference IDs. See /.well-known/agents.json and /llms.txt."\n/>\nThis is simple and high leverage.
\nIf a form maps to an agent action, annotate it.
\n<form\n // progressive enhancement attributes\n toolname="report_streetlight_outage"\n tooldescription="Report a streetlight outage with optional photo evidence"\n>\n <input\n type="file"\n name="photo"\n toolparamtitle="Outage photo"\n toolparamdescription="Optional JPG or PNG, max 10 MB"\n />\n</form>\nEven where these attributes are ignored today, they are harmless and future-friendly.
\nFor richer integrations, use the WebMCP proposal APIs directly:
\n\nMinimal pattern:
\nuseEffect(() => {\n const mc = (navigator as any).modelContext;\n if (!mc) return;\n\n mc.registerTool({\n name: 'report_streetlight_outage',\n description: 'Report a streetlight outage from this page',\n inputSchema: { type: 'object', properties: {}, required: [] },\n execute: async (_params, agent) => {\n const confirmed = await agent.requestUserInteraction(async () => {\n document.getElementById('outage-photo-input')?.click();\n return true;\n });\n\n if (!confirmed) {\n return { content: [{ type: 'text', text: 'Cancelled by user.' }] };\n }\n\n (\n document.getElementById(\n 'streetlight-outage-form',\n ) as HTMLFormElement | null\n )?.requestSubmit();\n return { content: [{ type: 'text', text: 'Outage report submitted.' }] };\n },\n });\n\n return () => mc.unregisterTool('report_streetlight_outage');\n}, []);\nImportant: keep human confirmation for sensitive operations. Never auto-submit file attachments without explicit user interaction.
\nDo not force every agent to parse your full HTML app shell.
\nTwo good options:
\nAccept: text/markdown is present.Simple rewrite example in Next.js:
\nasync rewrites() {\n return {\n beforeFiles: [\n {\n source: '/:path*',\n destination: '/markdown/:path*',\n has: [{ type: 'header', key: 'accept', value: '(.*)text/markdown(.*)' }]\n }\n ]\n };\n}\nThis keeps one source of truth while giving agents a low token, high signal representation.
\nAgent native does not replace SEO basics. It builds on them.
\nrobots.txtsitemap.xml with static and dynamic routesWhen your SEO map and agent map disagree, both humans and models get worse results.
\nThis is where most teams drift. They ship discovery files once, then forget to update them when limits and fields change.
\nAdd tests that validate:
\nllms.txt has expected sections and valid URLsagents.json parses and includes required fieldsIf your upload limit changes from 20 MB to 25 MB, your tests should force updates across UI, llms.txt, and agents.json.
If you want to implement this in a week, this is the order I recommend:
\nllms.txt and agents.json<head> discovery links and capability metaIf you want a fast way to debug this end to end, I built an open source companion app:
\n\nIt checks the same practical signals covered in this post:
\n/.well-known/agents.json contract shape and chain quality/llms.txt availability and structureAccept: text/markdown behaviorrobots.txt and sitemap.xml crawlabilityUse it as a repeatable loop: implement one layer, run audit, close gaps, rerun, and track score changes over time.
\nThe AI web will not be won by whoever has the fanciest model integration. It will be won by teams whose apps are easiest for agents to discover, understand, and operate safely.
\nYou do not need to build everything at once. Start with static discovery plus markdown negotiation, then layer in tool semantics and execution APIs.
\nBuild for both users: humans and agents.
", "url": "https://www.groff.dev/blog/how-to-make-your-web-app-agent-native", "title": "How to Make Your Web App Agent Native", "summary": "A practical, layered blueprint for making your web app easy for AI agents to discover, understand, and safely act on.", "date_modified": "2026-02-28T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/peanut-butter-chocolate", "content_html": "I have been writing about my agentic coding workflow for a while now. The manager/coding agent loop, the CLAUDE.md architecture, the review gates, the PRIME_DIRECTIVEs. All of that still applies.
\nWhat I have not shared until now is the actual plugin that ties it together.
\nPeanut Butter & Chocolate is a Claude Code plugin. It is MIT licensed, free, and you can install it right now. It implements the structured pipeline I run on every non-trivial PR: Research, Plan, Implement, Review.
\nThe name is the whole idea. Peanut butter is Claude. Chocolate is OpenCode with Codex. Two good things that work better when you combine them with clear boundaries.
\nI kept rebuilding the same workflow from scratch. Open a Claude Code session, manually prompt for research, copy context into a plan, hand off to Codex, paste review feedback back. Every time I started a new feature, I was re-inventing the process.
\nThe plugin packages that process into five slash commands. Each one encodes the exact workflow I described in my earlier posts, with the guardrails and artifacts baked in.
\nThree parallel Sonnet sub-agents fan out across your codebase. One finds files. One analyzes how things work. One maps patterns and conventions. They are read-only. They do not suggest improvements or critique your code. They just document what exists.
\nThe output is a timestamped research document saved to thoughts/research/. Every claim includes a file:line reference so you can verify it.
I run this before planning because plans built on assumptions break. Plans built on evidence hold up.
\nOpus reads the research and works with you interactively. You answer clarifying questions. It produces a phased plan with explicit scope boundaries, including a "what we are NOT doing" section to prevent drift.
\nEvery plan has verifiable success criteria. Not "ensure quality" but "run pytest -x and confirm zero failures." If you cannot script the check, it does not belong in the success criteria.
Plans get saved to thoughts/plans/.
This is where Codex comes in. Opus breaks the plan into self-contained execution packets. Each packet has an objective, constraints, acceptance criteria, and a required output format. Then it sends each packet to Codex via opencode run.
The packets are markdown files, not inline heredocs. A safety hook blocks heredoc execution to enforce this. File-based handoff means every instruction is auditable and reproducible.
\nOpus does not write code. It orchestrates. Codex does not plan. It implements. The separation is the point.
\nCodex reviews the changes independently. It checks the diff, runs tests, looks at code quality, and flags issues. Then Opus triages every finding into three buckets:
\nThis loops up to three times. The cap prevents infinite refinement cycles where agents keep finding new things to nitpick.
\nAfter the PR is up and AI/humans leave review comments, this command fetches them, triages each one through the same agree/defer/skip process, and generates fix packets. One packet per comment for clean traceability.
\nPBC ships with eight specialized Sonnet sub-agents. Three for codebase research, two for reading your thoughts/ artifacts, two for Jira integration, and one for web search.
Every sub-agent follows the same constraint: they are documentarians. They explain what exists. They do not critique, do not suggest, do not recommend. That keeps their output clean and prevents them from stepping on the orchestrator's job.
\nThe biggest problem with single-agent coding is that the same model that wrote the code also reviews it. It has the same blind spots, the same assumptions, the same biases. It is going to approve its own work.
\nPBC hands review to a completely separate agent with no context about how the implementation was planned. Codex sees the diff and the codebase, not the thought process that led to the code. That independence catches things the implementing agent cannot see.
\nIs it perfect? No. But it catches real bugs that would otherwise make it into PR review, and that saves senior engineer time.
\nEverything goes to a thoughts/ directory, which is gitignored by default. Research, plans, packets, reviews. Each artifact has YAML frontmatter with date, branch, commit hash, and tags.
This matters when something breaks later and you need to understand why a decision was made. The artifact trail gives you that. It also matters when you want to resume work after clearing context or starting a new session.
\nclaude plugin add mattlgroff/pbc\nThat is it. You need Claude Code and OpenCode installed. The plugin assumes opencode is available on your PATH.
This is not an auto-merge pipeline. You still review the PR. You still approve the plan before implementation starts. You still triage review findings before fixes happen. Every step has a human gate.
\nThis is also not a replacement for tests. If your repo does not have tests, the review loop cannot verify correctness. PBC makes your existing quality infrastructure more effective. It does not substitute for it.
\nI use the ChatGPT Pro plan, which gives me near unlimited access to GPT-5.3-codex. That covers the implementation and review agents. For Claude, I have a company-paid Anthropic plan that includes some amount of Opus usage, but nowhere near MAX levels. Opus handles orchestration and planning, which requires fewer tokens than the implementation work Codex does.
\nThis combination is the most cost effective strategy for me right now. Opus is excellent at reasoning through plans and triaging review findings. Codex is excellent at grinding through implementation packets. By splitting the roles along those strengths, I get the best of both without paying MAX pricing for the high-volume work.
\nThe ideas are not original to me. Hamel Husain wrote about review loops. Dexter Horthy at HumanLayer shaped how I think about agent boundaries. Ryan Carson's compound engineering concept influenced the multi-pass structure. I took those ideas, built a workflow around them, used it on real production code for months, and then packaged it.
\nThe repo is at github.com/mattlgroff/pbc. MIT license. If you try it and something breaks, open an issue.
\nIn my Early 2026 Agentic Coding Update, I talked about the loop: manager agents, coding agents, PRIME_DIRECTIVEs, and review gates.
\nThat post covered how to run the loop. This one covers the thing that makes the loop actually work: the documentation architecture.
\nEvery session with Claude Code or OpenCode starts stateless. No memory of your conventions. No knowledge of your test commands. No awareness that you use bun instead of npm or that migrations go through Alembic and not raw SQL.
\nThe only thing that reliably onboards each new session is your rules file. CLAUDE.md for Claude Code. AGENTS.md for OpenCode. Whatever you put there shapes every decision the agent makes.
Most repos either have nothing, or they have a bloated auto-generated file that the model quietly ignores. Both fail for the same reason: the agent does not have the right context at the right time.
\nThis post is the practical guide to fixing that.
\nThe official Claude best practices for agent skills make this clear: the context window is a shared public good. Your CLAUDE.md competes with the system prompt, conversation history, and every other piece of context the model needs.
HumanLayer's guide on writing a good CLAUDE.md puts it more bluntly: Claude often deprioritizes CLAUDE.md content entirely. The system injects a reminder saying the context "may or may not be relevant." The more instructions you include that are not universally applicable, the more likely the model dismisses the whole file.
This means you cannot solve the problem by writing a bigger rules file. You solve it with architecture.
\nAfter running this pattern across multiple production repos, I have landed on three tiers:
\nTier 1: Root CLAUDE.md - Universal rules that apply to every task. Under 100 lines. Loaded automatically every session.
\nTier 2: Skills (.claude/skills/name/SKILL.md) - Task-specific behavior loaded on demand. The agent reads a skill only when the task matches. Think of these as specialized playbooks: one for commits, one for PRs, one for migrations.
\nTier 3: Agent Guides (docs/agent-guides/name.md) - Deep reference material. Build commands, architecture docs, convention details. Skills point to these. The agent reads them only when it needs the full picture.
\nThe key principle is progressive disclosure. The root file is a table of contents. Skills are chapters. Agent guides are appendices. The agent loads only what the current task requires.
\n![\"3-Tier](\"/images/photos/3TierArchitecture.png\")
your-repo/\n CLAUDE.md # Tier 1: Universal (< 100 lines)\n .claude/\n skills/\n build-test-verify/SKILL.md # Tier 2: On-demand task playbooks\n create-pull-request/SKILL.md\n git-commit/SKILL.md\n core-conventions/SKILL.md\n self-review-checklist/SKILL.md\n docs/\n agent-guides/\n build-test-verify.md # Tier 3: Deep reference\n core-conventions.md\n backend/CLAUDE.md # Directory-level overrides\n frontend/CLAUDE.md\nBefore writing anything, inventory what already exists. Most repos have some combination of:
\nAGENTS.md or CLAUDE.md (possibly auto-generated)README.md with build/test/run instructionsDo not throw any of this away. You are going to absorb it into the right tier.
\nMap each piece of existing documentation to where it belongs:
\n| Existing Content | Target Tier |
|---|---|
| "Use bun, not npm" | Tier 1 (root CLAUDE.md) |
| "Run pytest -x for backend tests" | Tier 2 (build-test-verify skill) or Tier 3 (agent guide) |
| "Our agents follow this directory contract..." | Tier 2 (domain-specific skill) |
| "Here is the full project directory tree" | Tier 3 (project-map agent guide) |
| "Always use Conventional Commits" | Tier 2 (git-commit skill) |
The rule: if it applies to every task, it goes in Tier 1. If it applies only when doing a specific kind of work, it goes in Tier 2 or 3.
\nThis is the highest-leverage file in your entire repo for AI-assisted development. Every token competes for attention.
\nUse the Why / What / How / Progressive Disclosure structure:
\n# Project Name Agent Guide\n\nUse this file as the default onboarding context for this repo.\n\n## Why\n\nBrief description of what this project is and what matters.\nOne to three sentences. No filler.\n\n## What (project map)\n\n- `frontend/` - React app (Vite, Bun)\n- `backend/` - FastAPI + Pydantic AI\n- `docs/agent-guides/` - task-specific guidance loaded on demand\n\nRead `docs/agent-guides/project-map.md` for the full map.\n\n## How (always apply)\n\n- Use existing project patterns before introducing new abstractions.\n- Backend: use `uv` for Python deps, `ruff` for linting.\n- Frontend: use `bun` for deps and scripts.\n- Do not hardcode API URLs.\n- Validate changes with the smallest relevant command set first.\n\n## Progressive Disclosure\n\nDo not load every guide for every task. Read only what is relevant:\n\n- Build/test/lint: `docs/agent-guides/build-test-verify.md`\n- Conventions: `docs/agent-guides/core-conventions.md`\n- Migrations: `docs/agent-guides/alembic-migrations.md`\n\nUse skills in `.claude/skills/` for task-specific behavior:\n\n- Core: `build-test-verify`, `core-conventions`\n- Workflow: `create-pull-request`, `git-commit`, `self-review-checklist`\n\n## Local Overrides\n\nDirectory-level `CLAUDE.md` files may add stricter rules.\nApply the nearest file in addition to this one.\n\n## PR and Branching\n\nUse a feature branch. Never push directly to `main`.\nUse the `create-pull-request` skill for standards.\nTarget: under 100 lines. HumanLayer keeps theirs under 60. Anthropic recommends under 300 but less is better. In my experience, 60 to 100 lines is the sweet spot for a real production repo.
\nWhat to leave out of Tier 1:
\nSkills are the biggest upgrade over a flat CLAUDE.md. They give you task-specific behavior that loads only when relevant.
Every skill needs YAML frontmatter with name and description. The description is critical because the agent uses it to decide whether to load the skill.
---\nname: build-test-verify\ndescription: Run lint, test, and build verification commands\n for the project. Use when validating changes, running tests,\n or checking builds.\n---\nStart with these five skills. They cover the most common AI-assisted workflows in any repo:
\nThen add domain-specific skills for what makes your repo unique. For example:
\nalembic-migrations or drizzle-migrations skillSkill authoring principles:
\nSKILL.md under 500 lines (Anthropic recommendation). Under 150 is better for most skills.Agent guides live in docs/agent-guides/ and serve as the deep reference layer. Skills point to them. The agent reads them only when it needs the full context.
Minimum set:
\nAdd more as complexity demands. Migration workflows, API contracts, auth patterns, streaming protocols. Whatever would take a new human engineer more than five minutes to figure out from the code alone.
\nDo not try to maintain a project-map or directory tree guide. These go stale immediately and the maintenance cost is not worth it. The agent can explore the filesystem directly when it needs to understand structure.
\nIf your repo has distinct subsystems (like a backend/ and frontend/), add a CLAUDE.md in each directory with rules that only apply there.
# Backend Rules\n\nApply the root `CLAUDE.md` first, then this file.\n\n## Non-Negotiable\n\n- All database changes go through Alembic migrations. No raw DDL.\n- Use `uv` for all Python dependency management.\n- All database operations use async SQLAlchemy sessions.\n- Auth is MSAL-based. Do not introduce alternative auth patterns.\nKeep these short. 20 to 30 lines. They exist to prevent mistakes in a specific area, not to repeat what is already in the root file.
\nDo not auto-generate your CLAUDE.md. Running /init produces generic output that wastes your highest-leverage file. Write it yourself.
Do not use CLAUDE.md as a linter. HumanLayer is right about this: style rules in CLAUDE.md are expensive and unreliable. Use Ruff, Biome, ESLint, or Prettier. Set up a pre-commit hook or a Claude Code stop hook. LLMs are the wrong tool for deterministic formatting.
Do not duplicate content across tiers. A skill should point to an agent guide, not copy its content. If the same instructions exist in two places, they will drift apart.
\nDo not stuff everything into Tier 1. Every line in your root CLAUDE.md competes for attention. If it does not apply to literally every task, push it down to a skill or agent guide.
Do not create skills for things that do not have enough complexity. A single CI workflow file does not need a github-actions skill. A simple UI library does not need a frontend-design skill. Skill count should match actual decision surface, not aspiration.
One place I look for skill ideas is skills.sh, Vercel's skills leaderboard. Discovery is useful. Discovery is not trust.
\nSkills are just markdown files, but they shape agent behavior in real ways. Treat every installed skill like executable influence over your workflow. A few failure modes are common:
\nSkills can quietly bias recommendations toward products and services. A "database optimization" skill might steer every recommendation toward a specific managed service. You would not notice unless you read the raw markdown.
\nSkills can contain dangerous instructions. Exfiltrating secrets with curl, writing to unexpected paths, or running destructive commands. The skill does not need to be malicious on purpose. A poorly scoped skill can still cause real damage.
Skills can burn context when they are no longer useful. Install-heavy skills linger in your .claude/skills/ directory and get loaded into sessions where they add nothing. Every loaded skill competes for context window space with the work you actually need done.
Skills can conflict with your codebase conventions. A generic "React best practices" skill might recommend patterns that contradict your existing architecture. When skills and repo conventions disagree, the agent gets confused and output quality drops.
\nI have had the best results by taking inspiration from public skills, then rewriting them for my own repo standards and review gates.
\nSKILL.md yourself before enabling it.This architecture is not Claude-only. OpenCode reads .agents/skills/ and AGENTS.md but also supports Claude-compatible fallbacks. I keep .claude/skills/ as the canonical location and it works across both tools.
If you use OpenCode, you can add a compatibility note to your skill frontmatter:
\n---\nname: build-test-verify\ndescription: Run lint, test, and build verification.\ncompatibility: claude-code, opencode\n---\nOther tools like Cursor and Windsurf have their own rules file conventions. The agent guide layer (docs/agent-guides/) works with anything because it is just markdown in a standard location.
After implementing this across multiple repos, the difference is measurable:
\nBefore: The agent guesses at test commands, uses the wrong package manager, creates PRs with no description format, writes commit messages that say "update code."
\nAfter: The agent runs the right commands, follows your conventions, creates PRs that match your team's standards, and loads deep context only when the task actually needs it.
\nThe root CLAUDE.md is the highest-leverage file in your repo for AI-assisted development. The skills and agent guides turn one good file into a system. Treat it like infrastructure, not documentation.
If you want to add this to your repo today:
\nAGENTS.mdbuild-test-verify, git-commit, create-pull-request, core-conventions, self-review-checklistbuild-test-verify, core-conventionsCLAUDE.md in directories with unique constraintsDo not try to get it perfect on the first pass. The best CLAUDE.md files are iterated over weeks based on watching real agent behavior.
The best source of improvements is code review. Every comment a reviewer leaves on an AI-assisted PR is a signal that the agent lacked context. Wrong import pattern? Add it to core-conventions. Missed a test command? Update build-test-verify. Used the wrong migration tool? Add a line to the directory override. I treat every review comment as an opportunity to add a new rule, a new checklist item, a new guardrail. Over time, the same mistakes stop appearing because the agent has the context it was missing. These files are living documents, not write-once artifacts.
CLAUDE.md should not be a linter.I keep getting the same question: how involved are you really on big AI-assisted PRs, and how long does it actually take?
\nShort answer. Very involved. Usually many hours.
\nAlso, I do not use Cursor anymore.
\nMy daily stack now is OpenCode with gpt-5.3-codex on OpenAI Pro, plus Claude Code on my company's team plan.
I am sharing this because people keep asking if these PRs are mostly autonomous now. They are not.
\nI still like the Ralph loop concept.
\nAt its core, it is a while loop with an exit condition. In practice, that usually means do this until the test passes.
\nThe concept is solid. The part people miss is control.
\nI am not letting loops run wild and auto-merge whatever comes out. I stay in the loop, steer aggressively, and stop bad directions early.
\nLooping is not the advantage by itself. Checkpoints are the advantage.
\nFor complex work, this still feels like a mad scientist operation.
\nPRIME_DIRECTIVE style checks.This is not passive prompting. It is active systems management.
\nA PRIME_DIRECTIVE is the contract for the work before code starts. It defines what must be true for a PR to be considered valid, not just what compiles.
My directive docs usually include:
\nAn anonymized example shape:
\n# PRIME_DIRECTIVE\n\n## Audit Objective\n\n- Match intended external behavior from the reference system.\n- Allow divergence only when intentional, documented, and test-covered.\n\n## Non-Negotiable Pass Criteria\n\n- Strict session isolation per user/channel/peer.\n- Deterministic command semantics for safety-critical actions.\n- Authz boundaries enforced across user and admin paths.\n- Streaming events follow fixed contracts.\n\n## Required Evidence\n\n- Parity matrix: matched | intentional_divergence | uncovered\n- Security/authz matrix with evidence paths\n- Regression proof for high-risk paths\n- Validation outputs for lint, tests, and build\n\n## Decision Log (Locked)\n\n- Explicit architecture decisions that cannot drift without review.\nThe loop is not just do work until tests pass. It is manager-controlled iteration with reporting and resets between rounds.
\nPRIME_DIRECTIVE constraints.This is slower than full autopilot, but quality is much higher.
\nLLMs are effectively stateless between sessions. If you do not provide stable context every time, behavior drifts fast.
\nThat is why I anchor my workflow on CLAUDE.md. It is the one place that reliably onboards each new session with project intent, constraints, and execution defaults.
This post from HumanLayer explains the principle well: Writing a good CLAUDE.md.
\nOne of my biggest lessons this year is that instruction density can hurt compliance.
\nIf your root instructions try to cover every edge case, the agent starts ignoring more of them. I keep root rules short and broadly applicable, then push details into focused skill docs.
\nThat gives me better instruction following and cleaner output under pressure.
\nI use a progressive disclosure pattern:
\nCLAUDE.md for universal rules.Typical skill locations:
\n.opencode/skills/<name>/SKILL.md.claude/skills/<name>/SKILL.md.agents/skills/<name>/SKILL.mdIf you try to fully automate PR generation with weak controls, you can produce a lot of low-trust output very fast.
\nMy priority is protecting senior engineers from AI slop PRs that ignore conventions. That means strict gates, explicit reviews, and constant correction.
\nThe biggest 2026 upgrade is reusable Agent Skills plus shared rule files.
\nI standardize on CLAUDE.md and .claude/skills because that works across both tools.
AGENTS.md by default.AGENTS.md and also reads Claude-compatible fallbacks.Docs I use:
\n\nEach SKILL.md needs frontmatter with at least:
namedescriptionThen I lock down permissions so agents only load the skills they should.
\nI posted about this on LinkedIn yesterday, and it is worth repeating here.
\nSkills are just markdown files, but they still shape agent behavior in real ways. Treat every installed skill like executable influence over your workflow.
\nA few failure modes are common:
\ncurl.I have had the best results by taking inspiration from public skills, then rewriting them for my own repo standards and review gates.
\nOne place I look for ideas is skills.sh, Vercel's skills leaderboard.
\nDiscovery is not trust. I treat every skill as untrusted until I read the raw markdown and run a security audit pass.
\nWhen skills are aligned well, they save real code review time. When they are not, they create review debt.
\nPlease do not auto-install skills without reviewing them.
\nSKILL.md yourself before enabling it.This is the minimum loop I trust for non-trivial PRs:
\nI published my plugin pack for this here:
\n\nThe point is simple. Keep agent branch and PR behavior predictable without slowing down normal human workflows.
\nLast year I wrote about structured non-vibe coding loops:
\n\nThe 2026 version is stricter.
\nBetter gates. Better reusable skills. More self-reflection passes. More deliberate reviews.
\nI still use automation heavily. I just treat it as a managed pipeline, not autopilot.
\nIf you want better AI-assisted PR quality, do not start by removing humans from the loop.
\nStart here:
\nThat is the difference between fast progress and fast mess.
", "url": "https://www.groff.dev/blog/early-2026-agentic-coding-update", "title": "Early 2026 Agentic Coding Update: OpenCode and Claude Code", "summary": "A practical update on my early 2026 workflow with OpenCode and Claude Code, including Ralph loops, human gatekeeping, and reusable agent skills.", "date_modified": "2026-02-09T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/respect-accept-headers-for-ai-agents", "content_html": "If you're building for the web in 2025, you need to know this: AI agents aren't browsing your HTML anymore. They're asking for markdown.
\nOpenCode's next version is switching to request text/markdown versions of web pages by default. This isn't just a technical curiosity; it's a fundamental shift in how AI agents consume web content.
Here's something many developers forget: Accept headers have been part of the HTTP specification since the beginning. They're not new. We've been using them for years to negotiate between HTML, XML, and JSON responses. The Accept header is how clients tell servers what content types they prefer.
Yet somehow, most websites only serve HTML. They ignore Accept headers entirely, missing out on a fundamental piece of HTTP content negotiation. Now, with AI agents becoming primary web consumers, this oversight is becoming a real problem.
\nHere's the thing about HTML: it's verbose. All those div tags, classes, inline styles, and JavaScript snippets? They're noise to an LLM. When an AI agent fetches your beautifully crafted React app, it's wading through thousands of tokens of markup just to find the actual content.
\nMarkdown strips away the cruft. It's pure content with minimal formatting. Exactly what LLMs need to understand your site efficiently.
\nConsider the token economics:
\nThat's not just a performance improvement. It's the difference between an AI agent being able to process your entire site versus hitting rate limits after a few pages.
\nI've just implemented Accept header content negotiation on this very site. You can now request any page in three formats:
\nAccept: text/html - Traditional HTML (default)Accept: text/markdown - Clean markdown formatAccept: text/plain - Plain text, no formattingWant to see this exact blog post in markdown? Try this:
\ncurl -H "Accept: text/markdown" https://groff.dev/blog/respect-accept-headers-for-ai-agents\nThe implementation uses Next.js middleware to detect the Accept header and serve appropriate content. When a request comes in with Accept: text/markdown, the middleware rewrites to an API endpoint that:
The beauty is that your existing pages remain unchanged. You're not maintaining two versions of your content. You're serving the same content in different formats based on who's asking.
\nAI agents are becoming primary consumers of web content. They're not just search crawlers anymore. They're actively reading, understanding, and acting on information from websites.
\nIf you want AI agents to:
\n...then you need to speak their language. And increasingly, that language is markdown.
\nSites that respect Accept headers will have a significant advantage in the AI-first web:
\nIf you're ready to make your site AI-friendly:
\ntext/markdown; charset=utf-8The web is evolving from human-first to human-and-AI-first. Accept headers aren't just about content negotiation anymore. They're about making your content accessible to an entire new class of consumers.
\nOpenCode is just the beginning. As more AI tools adopt markdown-first approaches, sites that don't support Accept headers will be left serving expensive, inefficient HTML to agents that just want the content.
\nMake your site future-proof. Respect the Accept header.
\nWant to discuss AI agent optimization strategies? Connect with me on LinkedIn or follow me on X @groffdev.
", "url": "https://www.groff.dev/blog/respect-accept-headers-for-ai-agents", "title": "Why Your Website Should Respect Accept Headers for AI Agents", "summary": "OpenCode and other AI tools are requesting text/markdown by default. Here's why you should support Accept headers for better AI agent compatibility.", "date_modified": "2025-09-28T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/claude-code-opencode-productivity-boost", "content_html": "I thought I had reached peak productivity with Cursor. I was wrong.
\nAfter trying out Claude Code and OpenCode (SST's open-source, model-agnostic alternative), I'm experiencing a development velocity I didn't think was possible. The difference was so dramatic that I immediately upgraded to Claude Code Max's $200/month tier—I was burning through rate limits that quickly.
\nWhat makes this combo special isn't just raw coding speed. It's the mental bandwidth it frees up. While Claude Code and OpenCode handle the implementation details, I'm simultaneously planning the next three features. It's like having a senior developer pair programming with you 24/7, except they never get tired and can context-switch instantly.
\nThis isn't "vibe coding" where you're just throwing prompts at an AI and hoping for the best. This is structured, intentional development with AI as a force multiplier.
\nTo really put these tools through their paces, I'm building a comprehensive AI Agent Platform. Here's the tech stack:
\nThe platform already includes several sophisticated agents:
\nAnd this is just the beginning. The modular architecture makes adding new agents straightforward.
\nBuilding this platform has been an intensive masterclass in modern AI engineering:
\nHandling multi-modal inputs efficiently requires careful consideration of file processing, chunking strategies, and context window management. Each file type needs its own preprocessing pipeline to extract meaningful content.
\nDifferent AI providers use different streaming formats. Building a unified interface that translates between these protocols seamlessly was a fascinating challenge that deepened my understanding of real-time AI communication.
\nWorking with models like Gemini for both image generation and editing revealed the nuances of prompt engineering for visual tasks versus text generation. The same model behaves very differently depending on how you structure your requests.
\nWhile Claude Code gets a lot of buzz (deservedly so), OpenCode from SST is the unsung hero here. Being open-source and model-agnostic means:
\nToday the best agentic coding models are Anthropic but that could very well change tomorrow. If it changes I don't want to be locked into Claude Code.
\nThe productivity gain isn't incremental—it's exponential. Tasks that would have taken days now take hours. Complex refactoring that I would have postponed indefinitely gets done in an afternoon.
\nIf you're still on the fence about trying Claude Code or OpenCode, consider this: the time you save in the first week will likely offset the cost for months. And if you're already using Cursor, you owe it to yourself to see what the next level looks like.
\nHas anyone else made this switch? I'd love to hear about your agentic coding experience on LinkedIn - what tools you're using, what you're building, and how it's impacting your development workflow.
", "url": "https://www.groff.dev/blog/claude-code-opencode-productivity-boost", "title": "From Cursor to Claude Code & OpenCode: How I Unlocked Next-Level Development Speed", "summary": "My experience switching from Cursor to Claude Code Max and OpenCode, and how it transformed my development workflow while building an AI Agent Platform.", "date_modified": "2025-09-07T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/building-around-nano-banana", "content_html": "TL;DR: Wiring Google’s Gemini 2.5 Flash Image Preview (aka “Nano‑Banana”) into a real product meant splitting my agent runtime in two, building a bridge to the Vercel AI SDK stream protocol, and ditching base64‑in‑SSE for Cloudflare R2 presigned URLs. If you’re shipping real image gen/edit, these are the dragons you’ll meet.
\n(“Nano‑Banana” is Google’s internal codename for the native image generation/editing capability in Gemini 2.5 Flash.)
\nIt’s truly multimodal. The model emits text and image bytes, and it expects files as proper parts on input. That pushed me to think in two lanes—chat semantics for text, and binary media paths for images. Your plumbing has to respect both worlds.
\nI started with agents on Pydantic AI (Python), streaming to a TypeScript UI via the Vercel AI SDK. To add Nano‑Banana without duct tape, I split the base into two:
\nThat separation gave me clean seams for logging/config—and, most importantly, a home for message conversion.
\nGoogle streams candidates/parts (text chunks and inline image bytes). The Vercel AI SDK expects Server‑Sent Events in its own data stream protocol so the frontend can assemble a UIMessage with parts. Those universes do not align by default.
My fix was a small bridge that consumes Google’s stream and emits AI SDK‑friendly events with stable part ids. I wrapped text with text-start → id → text-end, and I used AI SDK Data Parts for everything else. For images, I defined a custom data part—data-generated-image-url—that carries { url, mediaType } so the UI can render immediately. If you haven’t seen this pattern, it’s straight from Vercel’s docs on Streaming Custom Data and reconciliation: Vercel AI SDK: Streaming Custom Data.
I tried it. It worked—until it didn’t. Browsers choked on huge event buffers and JSON decoding, mobile got sluggish, and any hiccup meant re‑streaming a mountain of base64.
\nThe pragmatic fix: upload generated bytes to Cloudflare R2 and stream a short‑lived presigned URL as a data part. The UI just does <img src={url}>, and life is good. Egress is cheap (free!), retries are cheap, and you can expire aggressively.
Because not every agent should accept files, I added an accepts_files flag to agent metadata and gated the paperclip UI on it. For rendering:
data-generated-image-url).On the stream side, this maps 1:1 to the AI SDK’s data stream semantics, so useChat assembles messages whose parts include text and my custom data parts, and it reconciles updates by id.
accepts_files keep your UI honest and users unconfused.Start with S3‑compatible storage (like R2) on day one. Treat Google and Pydantic agents as first‑class but separate. And build the message converter as a reusable module with reconciliation from the start.
\nThis was a great technical challenge, and I learned a lot. If you’re building something similar and want to trade notes, I’m happy to chat. For deeper context on the Pydantic AI ↔ Vercel AI SDK bridge, I shared my playbook here: Pydantic AI + Vercel AI SDK tech stack.
\nI've been building AI agents for a while now, and I kept running into the same patterns and problems. How do you make agents reliable? How do you handle errors gracefully? How do you maintain control over what they're doing? I thought I was figuring this stuff out on my own, developing my own best practices through trial and error.
\nThen I discovered Dex's talk on the 12-Factor Agent principles from HumanLayer, and it was like someone had been watching over my shoulder, documenting exactly what I'd been learning the hard way.
\n\nThese principles aren't just theoretical - they're battle-tested approaches that solve real problems every agent builder faces. Whether you're just starting out or you've been building agents for months, these factors will make your applications more reliable, maintainable, and user-friendly.
\nHere are the core principles that every reliable AI agent should follow:
\nFactor 1: Natural Language to Tool Calls
\nConvert user intent into structured function calls. This is the foundation - your agent needs to reliably translate what users want into actionable tool invocations.
Factor 2: Own Your Prompts
\nDon't let frameworks hide your prompts. You need visibility and control over exactly what's being sent to your model.
Factor 3: Own Your Context Window
\nManage context deliberately. Don't let it grow unbounded or let frameworks make context decisions for you.
Factor 4: Tools Are Just Structured Outputs
\nThink of tool calls as structured data generation, not magic. This mental model helps you debug and reason about agent behavior.
Factor 5: Unify Execution State and Business State
\nYour agent's execution context should align with your application's business logic. Don't create artificial boundaries between them.
Factor 6: Launch/Pause/Resume with Simple APIs
\nBuild agents that can be controlled programmatically. Users need to be able to stop, start, and resume agent workflows.
Factor 7: Contact Humans with Tool Calls
\nWhen agents need human input, make it a first-class tool call. Don't hack in human-in-the-loop as an afterthought.
Factor 8: Own Your Control Flow
\nDon't let frameworks control when and how your agent makes decisions. You need explicit control over the execution flow.
Factor 9: Compact Errors into Context Window
\nWhen things go wrong, summarize errors efficiently so your agent can learn and recover without burning through tokens.
Factor 10: Small, Focused Agents
\nBuild single-purpose agents rather than trying to create one agent that does everything. Specialization leads to reliability.
Factor 11: Trigger from Anywhere, Meet Users Where They Are
\nYour agents should be accessible through multiple interfaces - web, API, Slack, wherever your users actually work.
Factor 12: Make Your Agent a Stateless Reducer
\nDesign your agent to be a pure function: given the same inputs and state, it should produce the same outputs. This makes debugging and scaling much easier.
Reading through these factors was honestly a bit surreal. I've been unknowingly implementing most of these patterns in my own agent builds:
\nIt's validating to see these patterns formalized into principles. Even more importantly, the factors I haven't fully embraced yet (like human-in-the-loop tool calls) are clearly areas where I can improve my agent architecture.
\nI highly recommend watching Dex's full talk and diving deeper into each principle. The GitHub repository has detailed explanations and examples for each factor:
\n⭐ Star the 12-Factor Agents repository
\nWhether you're using Pydantic AI, LangGraph, or building agents from scratch, these principles will make your applications more robust. I know I'll be referring back to them as I continue building and refining my own agent systems.
\nThe best part? You don't need to implement all 12 factors at once. Pick the ones that solve your current pain points and gradually adopt the others as your agent systems mature.
", "url": "https://www.groff.dev/blog/the-12-factor-agent-best-practices", "title": "The 12-Factor Agent: Best Practices for Building Reliable LLM Applications", "summary": "Dex from HumanLayer outlines 12 essential principles for building reliable AI agents. These best practices resonated deeply with my own agent development experience.", "date_modified": "2025-08-26T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/pydantic-ai-vercel-ai-sdk-tech-stack", "content_html": "If you just want the example code and don't want to read, the link to the GitHub repo is here: Pydantic AI + FastAPI + Vercel AI SDK + React
\nThe Honest Truth: I love Pydantic AI, but it's only available in Python. I love the Vercel AI SDK, but it's only available in TypeScript. This forced me to bridge two worlds that don't naturally talk to each other.
\nAfter building several AI agents and experimenting with different frameworks, I've settled on a stack that combines the best of both ecosystems: Pydantic AI + FastAPI + Vercel AI SDK + React TypeScript.
\nAnthropic's recent "Building Effective Agents" blog post offers sage advice about starting simple and understanding what's under the hood. I completely agree - which is why I gravitated toward Pydantic AI after trying other frameworks:
\nCrewAI pushed the multi-agent "roles" narrative too hard for my taste. It also didn't work with the types of MCP Servers I gravitate towards: Stateless HTTP Streaming Remote MCP Servers.
\nLangGraph had a terrible developer experience. The documentation was lacking, it required registering for an API key to even run locally, and the vibes were just off. Having never loved LangChain, LangGraph felt very LangChain-esque.
\nPydantic AI was a breath of fresh air. Look at this Hello World example and how simple it is:
\nfrom pydantic_ai import Agent\n\nagent = Agent(\n 'google-gla:gemini-1.5-flash',\n system_prompt='Be concise, reply with one sentence.',\n)\n\nresult = agent.run_sync('Where does "hello world" come from?')\nprint(result.output)\nThat's IT. And adding tools, dynamic system prompts, and structured responses using Pydantic is incredibly easy.
\nWhat I Had to Hack Together:
\nThe Good Parts:
\nThe Trade-offs:
\nWhy This Post Exists:\nI built this example repository to show how to make these two amazing but separate ecosystems work together effectively, despite their incompatibilities.
\nWhat makes Pydantic AI special is how simple it is. Unlike LangChain or other frameworks that feel overly complex, Pydantic AI gets out of your way and lets you build agents that feel like normal Python code.
\nOne of my favorite aspects of Pydantic AI is how easy it is to integrate with stateless or stateful Remote Streaming HTTP MCP Servers. Here's all it takes:
\n# Set up MCP server connection\nserver = MCPServerStreamableHTTP(\n url="https://outlook-remote-mcp-server.onrender.com/mcp",\n headers={"X-API-KEY": mcp_api_key},\n)\nagent = Agent(model, toolsets=[server], system_prompt=system_prompt_content)\nThat's it. You can instantly expand your agent's capabilities by connecting to any MCP server. In my own work, I often wrap existing APIs using Bun TypeScript with minimal dependencies.
\nI use Groq Cloud with the Qwen3 32B model in my example repository, but you can use any provider that Pydantic AI supports. Check out the full list at https://ai.pydantic.dev/api/providers/.
\nWhy Groq + Qwen3 32B? It's cheap and decent at native tool calling.
\nUsing a Different Provider? Just update your environment variables to match the provider's API key name, modify the pydantic-settings configuration, and update the model name.
\nWant to see how this all works together? I've created a complete example repository that demonstrates this stack: Pydantic AI + FastAPI + Vercel AI SDK + React
\n![\"Math](\"/images/photos/MathAgent.png\")
The repository includes:
\nThis example is intentionally barebones to show the core logic of connecting a tool-calling Pydantic AI agent to the Vercel AI SDK Version 5. Production platforms can have a much more robust, ChatGPT-like UI built with shadcn components and inspired by Vercel's Next.js AI starter kit, but this example strips away production complexity to focus on the essential integration between these ecosystems.
\nThis stack powers internal Chat Agents platforms I have built. One of the most useful agents is a Scheduling Agent, built with Pydantic AI, that connects to an Outlook MCP Server to:
\nThe agent handles complex scheduling logic while employees interact with it through a clean React interface powered by Vercel AI SDK. It's incredibly satisfying to see team members naturally conversing with the agent to coordinate meetings instead of playing email tag.
\nI just love Pydantic AI, but it's only in Python. I love the Vercel AI SDK, but it's only in TypeScript. So I forced them to work together. Despite the complexity of bridging two languages that don't naturally communicate, I keep returning to this stack because it just works. It has a reasonable learning curve, excellent performance, and top-notch developer experience.
", "url": "https://www.groff.dev/blog/pydantic-ai-vercel-ai-sdk-tech-stack", "title": "Pydantic AI + Vercel AI SDK: My Favorite Tech Stack for Building AI Agents", "summary": "How and why I combine Pydantic AI with Vercel AI SDK to build AI agents. Learn to connect Python Pydantic AI backends to TypeScript frontends with streaming, tool calling, and MCP servers. Example GitHub repo included.", "date_modified": "2025-08-24T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/building-simple-remote-mcp-server-bun", "content_html": "Most MCP (Model Context Protocol) server examples are overcomplicated. Handling state is a huge pain. Some are not deployable to the cloud. What if I told you that you could build a complete, production-ready MCP server in a single TypeScript file with under 1,000 lines of code?
\nThat's exactly what I did with my Petfinder MCP Server. Here's how I built it using Bun with zero dependencies.
\nI chose Petfinder as my example API because, honestly, who doesn't love puppies and kittens? Plus, it demonstrates OAuth client credentials flow, which is more complex than the typical API key authentication you'll encounter with most APIs.
\nWhen I started exploring MCP servers, I found most examples were either local only (stdio), or stateful if they were remote (accessible from the internet like any other cloud api). I didn't need to have a stateful MCP Server I just wanted to wrap an existing API so that I can quickly empower my AI Agents (mostly written in Pydantic AI) to have new tools and capabilities. With this approach it only takes a few hours of work to standup this Stateless Remote Streamable HTTP MCP Server (wow that's a mouthful).
\nFor someone wanting to quickly wrap an existing API or create a learning example, this complexity is overkill. Even worse, when you're trying to use these servers as examples for LLMs to learn from, the scattered logic across multiple files makes it nearly impossible for the AI to understand the complete flow.
\nImportant Note: I built this as a single file to make it easier for humans and LLMs to read as an example. In practice, you'd probably want to split up API calls, authentication, and tool/schema definitions into separate modules for better maintainability.
\nThe Model Context Protocol is Anthropic's standard for connecting AI assistants to external data sources and tools. Think of it as a bridge that lets Claude (or other AI assistants) interact with APIs, databases, and services in a structured way.
\nInstead of the AI trying to guess API formats or making unreliable HTTP requests, MCP provides a standardized interface where tools are clearly defined with input schemas.
\nThe trickiest part of building remote MCP servers is security. Most examples just skip this over entirely. Not long ago, all MCP Servers ran on your desktop and only had one client at a time. Security wasn't as concern.
\nI solved this security challenge for the Petfinder MCP Server by proxying all authentication to the underlying API (Petfinder) and using query parameters for credentials:
\n// Extract credentials from query parameters\nfunction extractCredentialsFromQuery(url: URL): {\n clientId?: string;\n clientSecret?: string;\n} {\n const clientId = url.searchParams.get('client-id') || undefined;\n const clientSecret = url.searchParams.get('client-secret') || undefined;\n return { clientId, clientSecret };\n}\nThis approach means:
\nFor most APIs that use simple API keys, you'd just forward the Authorization header from your MCP client to the underlying API and let them handle validation. There's nothing you could do on your end to validate their API token anyway - that's the API provider's job.
\nLet's walk through the key components of the single-file MCP server.
\nSDK Note: I didn't use the official MCP TypeScript SDK because I had compatibility issues with Bun at the time of writing. If you're reading this in the future, that problem may have been solved and would make for an easier implementation.
\nFirst, create a new Bun project:
\nmkdir petfinder-mcp-server\ncd petfinder-mcp-server\nbun init\nThe package.json is minimal:
{\n "name": "petfinder-mcp-server",\n "module": "simple-mcp-server.ts",\n "type": "module",\n "dependencies": {\n "zod": "^4.0.10"\n },\n "devDependencies": {\n "@types/bun": "latest"\n }\n}\nThe server handles Petfinder's OAuth flow automatically with per-client token caching:
\nNote: Most APIs use simple API key authentication instead of OAuth. If you're wrapping a typical API, you'd just pass the API key as a Bearer token in your headers - much simpler than this token exchange setup. I'm not focusing heavily on this OAuth implementation since it's specific to Petfinder's requirements.
\ninterface TokenCache {\n access_token: string;\n expires_at: number;\n}\n\nconst tokenCache = new Map<string, TokenCache>();\n\nasync function getAccessToken(\n clientId: string,\n clientSecret: string\n): Promise<string> {\n const now = Math.floor(Date.now() / 1000);\n\n // Check cached token\n const cachedToken = tokenCache.get(clientId);\n if (cachedToken && cachedToken.expires_at > now + 60) {\n return cachedToken.access_token;\n }\n\n // Request new token\n const response = await fetch(`${PETFINDER_BASE}/oauth2/token`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/x-www-form-urlencoded' },\n body: new URLSearchParams({\n grant_type: 'client_credentials',\n client_id: clientId,\n client_secret: clientSecret,\n }),\n });\n\n const tokenData = await response.json();\n\n // Cache the token\n const newToken: TokenCache = {\n access_token: tokenData.access_token,\n expires_at: now + tokenData.expires_in,\n };\n\n tokenCache.set(clientId, newToken);\n return newToken.access_token;\n}\nZod gives us validation as well as exports to JSON for the tools endpoint.
\nconst animalSearchSchema = z.object({\n type: z.enum(['dog', 'cat', 'small-furry', 'bird']).optional(),\n breed: z.array(z.string()).optional(),\n size: z.array(z.enum(['small', 'medium', 'large'])).optional(),\n location: z.string().optional(),\n distance: z.number().int().positive().optional(),\n limit: z.number().int().min(1).max(100).optional().default(20),\n});\n\n// Convert Zod schema to MCP tool schema\nfunction zodToMCPSchema(schema: z.ZodObject<any>) {\n const jsonSchema = z.toJSONSchema(schema) as any;\n\n // Clean up required fields\n if (jsonSchema.required) {\n const shape = schema.shape;\n jsonSchema.required = jsonSchema.required.filter((fieldName: string) => {\n const field = shape[fieldName];\n return !field.isOptional() && field._def.defaultValue === undefined;\n });\n }\n\n return jsonSchema;\n}\nEach MCP tool is a simple async function:
\nasync function searchPets(input: any) {\n const validatedInput = animalSearchSchema.parse(input);\n const result = await petfinderRequest('/animals', validatedInput);\n\n return {\n content: [\n {\n type: 'text',\n text: `Found ${result.animals?.length || 0} pets matching your search.`,\n },\n {\n type: 'text',\n text: JSON.stringify(result, null, 2),\n },\n ],\n };\n}\nHere's a tip: when the API response you're wrapping is large or complex, it's much clearer to present the final output as well-formatted Markdown (.MD). LLMs handle Markdown especially well, and it's far more readable than sifting through pages of raw JSON.
The entire server runs on Bun's built-in HTTP server:
\nserve({\n port: parseInt(process.env.PORT ?? '3000', 10),\n async fetch(req) {\n const url = new URL(req.url);\n\n // Handle CORS\n if (req.method === 'OPTIONS') {\n return new Response(null, {\n headers: {\n 'Access-Control-Allow-Origin': '*',\n 'Access-Control-Allow-Methods': 'POST, GET, OPTIONS',\n 'Access-Control-Allow-Headers': '*',\n },\n });\n }\n\n // Health check\n if (url.pathname === '/healthz') {\n return new Response('OK', { status: 200 });\n }\n\n // MCP endpoint\n if (req.method === 'POST' && url.pathname === '/mcp') {\n const body = (await req.json()) as MCPRequest;\n const response = await handleMCPRequest(body, url);\n\n return new Response(JSON.stringify(response), {\n headers: {\n 'Content-Type': 'application/json',\n 'Access-Control-Allow-Origin': '*',\n },\n });\n }\n\n return new Response('Not found', { status: 404 });\n },\n});\nStart the server locally:
\nbun run simple-mcp-server.ts\nTest with curl:
\ncurl -X POST "http://localhost:3000/mcp?client-id=YOUR_CLIENT_ID&client-secret=YOUR_CLIENT_SECRET" \\\n -H "Content-Type: application/json" \\\n -d '{\n "jsonrpc": "2.0",\n "id": 1,\n "method": "tools/call",\n "params": {\n "name": "pets.search",\n "arguments": {\n "type": "dog",\n "location": "90210",\n "limit": 5\n }\n }\n }'\nOr use the MCP Inspector for interactive testing:
\nnpx @modelcontextprotocol/inspector "http://localhost:3000/mcp?client-id=YOUR_CLIENT_ID&client-secret=YOUR_CLIENT_SECRET"\nImportant: Not all MCP clients fully support Remote Stateless Streamable HTTP MCP Servers (quite a mouthful!). I've found that Cursor and Pydantic AI both work perfectly with this approach. However, you might not be able to use this server with Claude.ai or CrewAI quite yet - they blame the Python MCP client not supporting it yet.
\nOne of the best features of MCP servers is integrating them with AI coding assistants like Cursor. Add this to your Cursor settings:
\n{\n "mcpServers": {\n "petfinder-local": {\n "type": "http",\n "url": "http://localhost:3000/mcp?client-id=YOUR_CLIENT_ID&client-secret=YOUR_CLIENT_SECRET"\n }\n }\n}\nReplace YOUR_CLIENT_ID and YOUR_CLIENT_SECRET with your actual Petfinder API credentials from petfinder.com/developers.
Now Cursor can help you find adoptable pets while you code! Try asking: "Find me small dogs available for adoption in San Francisco."
\nYou can also integrate this MCP server with Pydantic AI agents. Here's a simple example:
\nfrom pydantic_ai import Agent\nfrom pydantic_ai.mcp import MCPServerStreamableHTTP\n\ndef create_petfinder_agent() -> Agent:\n """Create and configure the petfinder agent"""\n\n # Set up MCP server connection\n server = MCPServerStreamableHTTP(\n url='<your-deployed-url>/mcp?client-id=YOUR_CLIENT_ID&client-secret=YOUR_CLIENT_SECRET'\n )\n\n # Generate the system prompt\n system_prompt = """You are a Petfinder agent who helps find adoptable pets.\n\n Important linking rules:\n - Always use the `animal.url` field from the API as the pet's profile link\n - Include the primary photo when available\n - Provide clear adoption information\n """\n\n # Create the agent\n agent = Agent(\n 'groq:qwen/qwen3-32b', # or your preferred model\n toolsets=[server],\n system_prompt=system_prompt\n )\nThis creates a Pydantic AI agent that can search for adoptable pets using your MCP server. The agent will automatically use the available tools (pets.search, pets.get, etc.) to fulfill user requests.
\nThe included Dockerfile makes deployment simple:
\nFROM oven/bun:1.2.19\nWORKDIR /app\n\nCOPY package.json bun.lockb* ./\nRUN bun install --frozen-lockfile\n\nCOPY . .\nEXPOSE 3000\n\nCMD ["bun", "run", "simple-mcp-server.ts"]\nDeploy to any container platform:
\ndocker build -t petfinder-mcp .\ndocker run -p 3000:3000 petfinder-mcp\nLike any API you deploy to the world, secure it using API security best practices. If you want an additional layer of protection and you're deploying to a public cloud like Render.com (which I use), you can add an additional secret in the headers and validate that secret before processing any requests.
\nFor example:
\n// Check for additional security header\nconst authSecret = req.headers.get('x-mcp-secret');\nif (authSecret !== process.env.MCP_SECRET) {\n return new Response('Unauthorized', { status: 401 });\n}\nI won't cover securing MCP servers in depth in this post, but remember: let the underlying API handle authentication validation - that's what it's designed for.
\nTry this approach with whatever API you're trying to wrap around!
\nThe pattern is always the same:
\nPro tip: For most APIs with simple API key authentication, you'd just pass the key as a Bearer token in headers - much simpler than the OAuth dance I had to implement for Petfinder!
\nReady to build your own single-file MCP server? Here's what you need:
\ngit clone https://github.com/mattlgroff/petfinder-mcp-servercurl -fsSL https://bun.sh/install | bashbun run simple-mcp-server.tsnpx @modelcontextprotocol/inspector "http://localhost:3000/mcp?client-id=YOUR_ID&client-secret=YOUR_SECRET"The beauty of this approach is that you can adapt it to wrap any API in just a few hours of work. Just change the authentication method, update the Zod schemas, and modify the tool functions. Everything else stays the same. You can copy and paste the simple-mcp-server.ts into your LLM ChatBot or Agentic Coding Assistant as a starting place. Including my README.md would probably help as well. I find that when working with tools like Cursor or OpenCode, starting with a Plan and a README.md with defined requirements works best.
Have you built your own MCP server? I'd love to see what APIs you've wrapped! Connect with me on LinkedIn and share your creations.
", "url": "https://www.groff.dev/blog/building-simple-remote-mcp-server-bun", "title": "Building a Simple MCP Server with Bun - Easily Wrap Any Existing API", "summary": "Learn how to create a complete MCP server in a single TypeScript file using Bun. This tutorial shows you how to wrap the Petfinder API with OAuth authentication, making it perfect for both humans and LLMs to understand and modify.", "date_modified": "2025-08-10T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/recommended-tech-stacks-for-llm-development", "content_html": "When working with LLMs for development, choosing the right tech stack can make a huge difference in productivity. Here are my top recommendations for stacks that play well with AI assistance.
\n\nFrontend:
\nBackend:
\nThese stacks are deeply embedded in LLM training data, which means you'll get better code suggestions, more accurate completions, and fewer hallucinations when working with AI coding assistants.
\nSubscribe to my YouTube channel for more AI development insights.
", "url": "https://www.groff.dev/blog/recommended-tech-stacks-for-llm-development", "title": "YouTube Short - Recommended Tech Stacks for LLM Development", "summary": "My recommendations for tech stacks that work well with LLMs: React (Vite/Next), TypeScript, Python, Tailwind, Shadcn, and Ruby on Rails.", "date_modified": "2025-08-10T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/ai-agent-security-agent-flayer", "content_html": "Read about this Agent Flayer in which security researchers submitted support tickets to JIRA in order to get an Agent (in this case Cursor) to expose a users environment variables (secrets like API Keys).
\nWhen a Jira ticket can steal your secrets
\nOne of the key suggestions they make are to turn off auto-run of commands and have your team inventory and approve MCP Servers for your company.
\nUsing Atlassian MCP would be fine if you don’t have routes for untrusted parties to make support tickets that a dev or pm would read up with an Agent.
", "url": "https://www.groff.dev/blog/ai-agent-security-agent-flayer", "title": "Is Your AI Agent Vulnerable?", "summary": "Overview of the Jira “Agent Flayer” attack and practical steps: disable auto-run, inventory/approve MCP servers (e.g., Atlassian MCP), avoid untrusted ticket routes.", "date_modified": "2025-08-09T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/short-when-gpt-5-needs-a-little-nudge", "content_html": "Sometimes even the most advanced AI models need a gentle reminder to follow instructions. Here's a quick example of GPT-5 needing a little nudge to do what was asked.
\n\nSubscribe to my YouTube channel for more AI development insights.
", "url": "https://www.groff.dev/blog/short-when-gpt-5-needs-a-little-nudge", "title": "YouTube Short - When GPT-5 Needs A Little Nudge", "summary": "A quick demonstration of how sometimes GPT-5 needs a little encouragement to follow instructions properly.", "date_modified": "2025-08-09T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/vibe-coding-recommendations", "content_html": "Here are my recommendations for working with Agentic Coding Assistants-a structured approach to AI-assisted development that balances creative flow with engineering discipline. I used to call this "mostly" vibe coding, but I’ve changed my mind: "vibing" implies you’re not reading the code closely. The point here is the opposite—careful review and intervention.
\nKeep a file (e.g., .cursor-rules or CLAUDE.md for Claude Code) that every LLM call references. This ensures consistency across your codebase and helps the AI understand your preferred patterns, naming conventions, and architectural decisions from the start.
Have the LLM draft a step-by-step plan, ask clarifying questions, then save the result to PLAN.md or README.md. For August 2025, use GPT-5 and explicitly include “Think hard” in the prompt to push deeper reasoning. This upfront investment prevents the AI from going down rabbit holes and gives you a clear roadmap. For truly greenfield work, do not start coding until the written plan and requirements feel complete enough to implement without guesswork.
Switch models to implement the plan, one section at a time.
\nDifferent models excel at different tasks—use GPT-5 for planning and UI, Claude for heavier non-UI implementation.
\nRead the model’s chain-of-thought and code as it proceeds. Immediately break out and course-correct if you see a wrong practice, a shift to a different approach than requested, or the model assuming incorrect syntax and “changing directions” without asking. This happens—sometimes the model gives up or pivots silently. You must watch carefully and intervene fast.
\nWhen adding a dependency or making an API call, paste the relevant documentation so the model uses current syntax and avoids hallucination. This is especially important for rapidly evolving frameworks and APIs where the model's training data might be outdated.
\nConfirm it's necessary, actively maintained, and on the latest stable version. AI models can sometimes suggest dependencies that are deprecated, poorly maintained, or overkill for your use case. A quick manual check can save you significant technical debt later.
\nFeed stack traces back to the coding model; if it stalls, prompt for alternative angles or bring in a human "Developer Brain" for validation. Bringing in a MCP server for a browser can help with the error feedback loop. Don't let the AI spin its wheels—sometimes a fresh perspective or manual intervention is exactly what's needed.
\nUse MCP servers to give the AI controlled access to tools and services (browser for scraping/debugging, Atlassian for Jira/Confluence, GitHub for PR hygiene, Outlook for scheduling). This turns the assistant from a code generator into a capable operator while keeping actions auditable.
\nFavor stacks well covered in model training (e.g., React + TypeScript, Vite or Next.js, shadcn/ui, Tailwind CSS v3). If your stack's new you'll need to inject docs and double check all the time it's using the right syntax and APIs. I'm avoiding the upgrade to Tailwind 4 for this reason.
\nThese practices have helped me maintain both velocity and quality when working with AI coding assistants. The key is balancing AI capabilities with close human review and engineering discipline.
\nDo you have any tips for effective AI-assisted development? How are you approaching reviewing AI-generated code?
\nI'd love to hear about your experiences and techniques on LinkedIn. The field is evolving rapidly, and there’s still so much to learn about the most effective ways to collaborate with AI in software development.
", "url": "https://www.groff.dev/blog/vibe-coding-recommendations", "title": "Agentic Coding Assistants: August 2025 Recommendations", "summary": "Practical recommendations for working with Agentic Coding Assistants code effectively in August 2025: planning with GPT-5, when to code with GPT-5 vs Claude, watching chain-of-thought, and choosing stable tech.", "date_modified": "2025-08-08T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/gpt-5-first-impressions", "content_html": "I had to add these User Rules to get a more useful model in Cursor. I kept finding it disobeying me. "Update that JIRA ticket with xyz" and it decided to make a comment instead. Because of this and occasionally just offering code suggestions but NOT writing the code changes in the file themselves, I make the following User Rules:
\nTo be fair, when it follows instructions, GPT-5 produces strong, well-structured code.
\nMy other impression is it's quite slow. I can see why there is a "fast" version on Cursor that uses the 2x cost priority api calls from OpenAI that double the price. The default is quite slow for Cursor. After the free week is up, gpt-5-fast calls on Cursor will cost 4x a requests. Those 1,000 requests a month for Business Users will go very quick if you use fast.
", "url": "https://www.groff.dev/blog/gpt-5-first-impressions", "title": "GPT-5 First Impressions", "summary": "Early notes from using GPT-5 in Cursor: stricter user rules needed and performance tradeoffs between default and fast modes.", "date_modified": "2025-08-07T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/gpt-5-launch-highlights", "content_html": "OpenAI has officially launched GPT-5 today, and it's packed with some impressive capabilities and competitive pricing that could shake up the AI landscape, especially for developers.
\nContext Window: GPT-5 boasts a massive 400K token context window, though generation is capped at 128K tokens max. This is a significant improvement that allows for processing much larger codebases and documents in a single session.
\nOpenAI is offering two tiers for GPT-5 (per 1M tokens):
\nThe Priority tier is particularly interesting as it promises guaranteed speed for those willing to pay double.
\nFor context, here's how GPT-5 stacks up against Anthropic's current pricing (per 1M tokens):
\nAt first glance, GPT-5's pricing is significantly more competitive than both Claude offerings, especially if you're doing heavy coding work where the cost differential really adds up.
\nPerhaps the biggest shock from today's announcement: OpenAI will deprecate all legacy models. While this will create a simpler menu of options, it means developers will need to migrate their applications from older models. This could be a significant undertaking for teams heavily invested in the older model ecosystem.
\nOpenAI is boldly claiming that GPT-5 is the "best coding model today." If this holds true, the combination of superior coding capabilities and more competitive pricing could make it an attractive alternative to Claude Sonnet or Opus for development workflows.
\nWe'll have to see if GPT-5 lives up to its coding claims through real-world testing. But on paper, the combination of:
\n...makes this a compelling option for developers. If it truly delivers better coding performance than Sonnet or Opus at a lower cost, it could represent a significant shift in the AI coding landscape.
\nThe forced migration from legacy models will be a short-term pain point, but for teams looking to optimize their AI spending while potentially getting better results, GPT-5 might just be worth the migration effort.
", "url": "https://www.groff.dev/blog/gpt-5-launch-highlights", "title": "GPT-5 Launch Highlights", "summary": "OpenAI launches GPT-5 with 400K context window, competitive pricing, and claims to be the best coding model today. Here are the key highlights and comparisons with Anthropic's offerings.", "date_modified": "2025-08-07T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/automate-prd-to-github-issues-with-llms", "content_html": "Have you ever struggled with creating a comprehensive Product Requirements Document (PRD) and then translating it into actionable GitHub issues? Let me share how I leveraged a reasoning model like o1 or o3-mini to generate a detailed PRD that covered all possible edge cases.
\nI started by asking my reasoning model clarifying questions. Through an iterative process, we developed a comprehensive PRD that addressed:
\nOnce the PRD was refined, I saved it as PRD.md. And remember, LLMs Love Markdown – or as I like to joke, LLM stands for "LLMs Love Markdown!"
The next step was breaking down the PRD into actionable GitHub issues. I fed the PRD and the GitHub CLI schema (referenced from the gh issue create --help command) into my reasoning model. The model then generated a detailed list of GitHub issues ensuring every crucial detail—such as assignee and milestone assignments—was included.
The generated issues ranged from initial project setup to the deployment process. Here's an excerpt of what the GitHub CLI commands looked like:
\ngh issue create --title "Project - Initial Monorepo Setup" --body "Set up the initial monorepo structure with frontend and backend directories. Initialize the git repository, add necessary configuration files, and set up basic documentation as outlined in the PRD.\n\nReferences:\n\n- Starter template reference: [link]\n- Chatbot template reference: [link]" --assignee your-username --milestone "Project Release 1.0"\nAnd it continued with additional issues for both frontend and backend tasks.
\nOnce the issues list was validated, I had the reasoning model generate a bash script to run all these GitHub CLI commands in bulk. Running the script automatically created 16 issues, saving me a tremendous amount of manual work.
\nNote: The bash script was crafted by Claude 3.5 Sonnet, my preferred coding model.
\nBelow is the final bash script that generated these issues:
\n#!/bin/bash\n\n# Ensure we're in the correct directory\ncd "$(dirname "$0")"\n\necho "Creating issues for Project Release 1.0..."\n\n# Setup & Infrastructure Issues\necho "Creating Setup & Infrastructure issues..."\ngh issue create --title "Project - Initial Monorepo Setup" --body "Set up the initial monorepo structure with frontend and backend directories. Initialize the git repository, add necessary configuration files, and set up basic documentation as outlined in the PRD.\n\nReferences:\n\n- Starter template reference: [link]\n- Chatbot template reference: [link]" --assignee your-username --milestone "Project Release 1.0"\n\n# Frontend Issues\necho "Creating Frontend issues..."\ngh issue create --title "FE - Setup React + Vite Frontend Project" --body "Set up a new React + Vite project using TypeScript. Utilize modern UI components and styling libraries, and use an efficient package manager. This project will serve as the frontend for the application.\n\nReferences:\n\n- Frontend framework guide: [link]\n- Package manager documentation: [link]" --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "FE - Implement Authentication & Login Flow" --body "Integrate secure authentication to protect the frontend. Ensure that only authorized users can log in and that token handling is properly established." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "FE - Design Chat UI and Integrate Real-Time SDK" --body "Design the chat interface, including a message area for real-time streaming responses and an input box. Integrate the relevant SDK to enable real-time communication." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "FE - Implement Token Passing for Backend Requests" --body "Ensure that every request from the frontend to the backend includes a secure token. This is critical for authentication and secure backend communication." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "FE - Model Selection UI Component" --body "Develop a UI component that allows users to select from various supported models. The selected model should be integrated with the chat requests." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "FE - Deploy Frontend to Static Site Service" --body "Build the React + Vite static site and deploy it to a static site hosting service. Ensure that deployment configurations and environment variables are set correctly." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "FE - Convert Existing Chat History Component" --body "Adapt an existing chat history component to work with the new frontend setup. Update it to integrate with the backend service while preserving the current user experience.\n\nReferences:\n\n- Component reference: [link]\n- Frontend framework guide: [link]" --assignee your-username --milestone "Project Release 1.0"\n\n# Backend Issues\necho "Creating Backend issues..."\ngh issue create --title "BE - Setup Python FastAPI Backend Project" --body "Set up a new FastAPI backend project using Python 3.12. Integrate a database ORM for managing history and utilize an efficient dependency management tool. This lays the foundation for the backend service.\n\nReferences:\n\n- API framework documentation: [link]\n- ORM documentation: [link]" --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "BE - Implement Token Validation for Backend" --body "Develop middleware in the FastAPI backend to validate secure tokens. Return appropriate error responses if the token is invalid, expired, or missing." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "BE - Create /chat Endpoint for Real-Time Communication" --body "Develop a POST endpoint for receiving user messages and model selections. The endpoint should stream responses and update the conversation history in the database." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "BE - Implement Chat History CRUD Endpoints" --body "Develop CRUD endpoints for managing chat history:\n\n- GET history: Return a list of previous sessions.\n- GET session details: Return messages for a specific conversation.\n- DELETE a session (optional): Delete a specific conversation.\nIntegrate these endpoints with the database as specified in the PRD." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "BE - Setup Database Integration with ORM" --body "Configure the database integration using an ORM. Set up migration tools to manage schema changes for the backend as required by the PRD." --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "BE - Integrate Model Provider API Calls" --body "Implement integrations for model providers. Based on user selections, make the appropriate API calls and stream responses as required.\n\nReferences:\n\n- SDK documentation: [link]\n- Streaming API guide: [link]" --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "BE - Provide Dockerfile for Containerized Backend" --body "Create a Dockerfile for the FastAPI backend. Reference existing examples and update it to use the modern dependency management tool and Python 3.12. Ensure the container is optimized for deployment.\n\nReferences:\n\n- Dockerfile best practices: [link]" --assignee your-username --milestone "Project Release 1.0"\n\ngh issue create --title "BE - Deploy Backend to Containerized Service" --body "Set up a service for the containerized FastAPI backend. Configure automatic rebuilds and redeployments on pushes. Ensure that environment variables such as database connection URLs and API keys are correctly set.\n\nReferences:\n\n- Deployment documentation: [link]" --assignee your-username --milestone "Project Release 1.0"\n\necho "All issues have been created!"\nRun this script, and you'll see all 16 issues created automatically – saving you countless hours of manual work.
\nUsing a reasoning model to guide both the PRD creation and the detailed GitHub issue generation process—and then automating everything via a bash script—can transform the way you develop and manage projects.
\nSo next time you're overwhelmed with PRD and issue creation tasks, remember: LLMs Love Markdown, and they'll help turn chaos into a clear, actionable plan in no time!
\nHappy coding!
\nAs always feel free to connect with me on LinkedIn or follow me on X @groffdev.
", "url": "https://www.groff.dev/blog/automate-prd-to-github-issues-with-llms", "title": "Automating PRDs to GitHub Issues: How Reasoning Models Saved Me a Ton of Time", "summary": "Learn how to leverage reasoning models like o1 or o3-mini to generate comprehensive PRDs and automatically convert them into actionable GitHub issues.", "date_modified": "2025-02-13T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/cursor-tip-o3-mini-vs-sonnet", "content_html": "Here's a useful tip for Cursor users: When using Agent mode in Composer, you can optimize your workflow by combining two different models:
\no3-mini initially to create your plan and discuss any clarifying questions before plan creationclaude-3.5-sonnet for executionThis approach leverages the strengths of both models. Interestingly, Cursor's own developers have found that Claude 3.5 Sonnet remains the best choice for actual coding tasks, which aligns with my personal experience as well.
\nOne caveat: if you need to work with images, you'll need to use Claude 3.5 Sonnet initially, as o3-mini doesn't support image processing. Once you've extracted what you need from the images, you can start a new Agent Composer context with o3-mini to create your plan, then switch back to Sonnet for execution.
\nThis tip is particularly relevant given Cursor's recent announcement:
\n\n\no3-mini is out to all Cursor users!
\nWe're launching it for free for the time being, to let people get a feel for the model.
\nThe Cursor devs still prefer Sonnet for most tasks, which surprised us.
\n— Cursor (@cursor_ai) January 31, 2025
\n
It's interesting to see that even the Cursor team themselves were surprised by Sonnet's continued dominance in coding tasks. This reinforces the idea that newer doesn't always mean better for every use case - it's about using the right tool for the right job.
\nAs always feel free to connect with me on LinkedIn or follow me on X @groffdev.
", "url": "https://www.groff.dev/blog/cursor-tip-o3-mini-vs-sonnet", "title": "Cursor Tip: Using o3-mini and Claude 3.5 Sonnet Together", "summary": "How to effectively combine o3-mini and Claude 3.5 Sonnet in Cursor for optimal coding workflow.", "date_modified": "2025-02-01T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/local-deepseek-r1-distilled-experience", "content_html": "The DeepSeek R-1 distilled Qwen 32B quantized to FP4 (try saying that name three times fast!) is a remarkable achievement in local AI deployment. What's truly incredible is that I can run this model on my computer with a single RTX 4090, and it outperforms GPT-3.5/4 - models that were state of the art just a couple of years ago.
\nWhile my vibes check doesn't have this beating GPT-4o and o1 yet, and Claude 3.5 remains a superior coding model, that doesn't diminish how remarkable this is. Having access to a model of such high caliber running entirely offline on my local machine, with no data center needed, is a significant milestone.
\nIt's worth noting that it's slower and doesn't perform quite as well as the Llama 70B distilled version running on Groq. However, that's not exactly a fair comparison - Groq has data centers filled with custom LPU chips specifically designed for this purpose.
\nI've found that these "thinking models" excel particularly at Tool/Function Calling. Even the smaller versions of the model perform impressively well at this task, making them particularly useful for specific programmatic applications.
\nThis kind of capability running locally represents a significant step forward in democratizing AI technology, allowing developers to work with powerful models without relying on cloud services or external APIs.
\nAs always feel free to connect with me on LinkedIn or follow me on X @groffdev.
", "url": "https://www.groff.dev/blog/local-deepseek-r1-distilled-experience", "title": "Running DeepSeek R-1 Distilled Locally: A Game Changer", "summary": "My experience running the DeepSeek R-1 distilled Qwen 32B model locally on a single RTX 4090, and how it compares to cloud-based alternatives.", "date_modified": "2025-01-31T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/reality-check-distilled-models", "content_html": "I've seen a post today advertising a deployment of DeepSeek R1, but they were actually deploying the 1.5B Qwen distillation—a model that is nowhere near the full DeepSeek R1's performance. I went through DeepSeek's own benchmarks from their paper on R1 and compared 3 common benchmarks to show you some quick differences between the full R1, distilled versions, OpenAI's models, and Claude 3.5 sonnet. Take a look for yourself:
\n| Model | AIME 2024 (pass@1) | MATH-500 (pass@1) | GPQA Diamond (pass@1) |
|---|---|---|---|
| DeepSeek-R1 (full) | 79.8% | 71.5% | 90.6% |
| OpenAI-o1-1217 | 79.2% | 75.7% | 93.4% |
| DeepSeek-R1-Distill-Qwen-32B | 72.6% | 94.3% | 62.1% |
| DeepSeek-R1-Distill-Llama-70B | 70.0% | 94.5% | 65.2% |
| DeepSeek-R1-Distill-Qwen-14B | 69.7% | 93.9% | 59.1% |
| OpenAI-o1-mini | 63.6% | 90.0% | 60.0% |
| DeepSeek-R1-Distill-Qwen-7B | 55.5% | 92.8% | 49.1% |
| DeepSeek-R1-Distill-Llama-8B | 50.4% | 89.1% | 49.0% |
| QwQ-32B-Preview | 50.0% | 90.6% | 54.5% |
| DeepSeek-R1-Distill-Qwen-1.5B | 28.9% | 83.9% | 33.8% |
| Claude-3.5-Sonnet-1022 | 16.0% | 78.3% | 65.0% |
| GPT-4o-0513 | 9.3% | 74.6% | 49.9% |
Source: DeepSeek-R1 Distilled Model Evaluation
\nYes, distilled models are cheap to run, but cheap compute ≠ good performance. The 1.5B model is not DeepSeek R1. It's significantly weaker, and using it without disclosing that fact is misleading.
\nGroq is offering DeepSeek-R1 Distill Llama-70B on their LPUs, claiming it's the best balance of performance and cost at scale. Based on the benchmarks, it's the strongest distilled variant, but it's still not the full DeepSeek R1. Pricing from Groq: $0.75 /M tokens for input and $0.99 / M tokens for output.
\nRight now, I've only seen Together AI offering full DeepSeek R1, and they charge $7/million tokens, compared to OpenAI o1's $15/million. Meanwhile, DeepSeek AI's own API is overloaded and stores data in China.
\nAmazon and Microsoft employees have hinted on LinkedIn that official DeepSeek R1 support is coming. The question is: at what price? Hopefully, it stays competitive.
\nCheaper, smarter LLMs are great for competition, but do your research before jumping in. The full DeepSeek R1 is a strong contender, but distilled versions are not the same thing—don't let anyone convince you otherwise. Find the best price and performance trade-off that works for your use case and a privacy policy that matches your desires.
\nAs always feel free to connect with me on LinkedIn or follow me on X @groffdev.
", "url": "https://www.groff.dev/blog/reality-check-distilled-models", "title": "Reality Check: Distilled Models ≠ Full Model", "summary": "Understanding the performance gap between distilled and full versions of DeepSeek R1, and why it matters.", "date_modified": "2025-01-30T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/us-ai-policy-changes-and-stargate-announcement", "content_html": "With the new administration comes a new U.S. policy on Artificial Intelligence.
\nYesterday, President Trump repealed President Biden's 2023 "Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence."
\nSome see this as lifting a regulatory burden that could supercharge innovation, especially for those of us bullish on AI's potential. Yet this also raises concerns about AI safety. Meanwhile, China's rapid AI progress (such as recently with DeepSeek AI's models, which censor queries on sensitive topics like Taiwan and Tiananmen Square) may widen the capabilities gap if the U.S. doesn't match pace.
\nToday, a massive $500 billion "AI infrastructure" investment was announced between SoftBank, OpenAI, and Oracle, forming a new joint venture called "Stargate." The initiative is starting with the development of "colossal data centers" - 10 buildings are currently being built, with 10 more coming soon, and the first one in Texas. According to the announcement, 100,000 new jobs will be created "immediately."
\nPresident Trump has declared we're in an "emergency" with the U.S. falling behind on AI, and has indicated that more Executive Orders and actions will be taken on this matter.
\nIn related developments, Masayoshi Son additionally name-dropped Microsoft and NVIDIA during the announcement. Sam Altman expressed his enthusiasm, stating he's "thrilled we get to do this in America."
\nI remain positively optimistic about these developments, and I look forward to seeing the U.S. continue to innovate in Artificial Intelligence. This is particularly important for our National Security and economic prosperity.
\nAs always feel free to connect with me on LinkedIn or follow me on X @groffdev.
", "url": "https://www.groff.dev/blog/us-ai-policy-changes-and-stargate-announcement", "title": "U.S. AI Policy Changes and the $500B Stargate Announcement", "summary": "Major shifts in U.S. AI policy and a massive new infrastructure investment signal changing priorities in artificial intelligence development.", "date_modified": "2025-01-21T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/2025-prediction", "content_html": "I don't have a crystal ball, but I'll make a prediction anyway.
\n2025 will be the year of AI Agents and Knowledge Graphs.
\nSome teams are already using these, but in production and at scale, it's still rare—at least based on what I've seen and read on social media.
\nWhat's your prediction for 2025?
\nAs always feel free to connect with me on LinkedIn or follow me on X @groffdev.
", "url": "https://www.groff.dev/blog/2025-prediction", "title": "2025 Prediction", "summary": "What will be the hot AI topics in 2025?", "date_modified": "2024-12-19T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/llama-3-2-released", "content_html": "Meta just dropped Llama 3.2: AI at Meta Blog Post.
\nHere's some highlights from their blog post:
\nVision and Lightweight Models:\nLlama 3.2 includes vision-enabled models (11B and 90B) designed for tasks like document understanding and image reasoning, along with lightweight text-only models (1B and 3B) optimized for mobile and edge devices. These models are available in pre-trained and instruction-tuned versions for easier customization.
\nEdge AI Capabilities:\nThe lightweight models support a 128K token context length and are optimized for on-device processing, enabling real-time tasks like summarization and instruction following on hardware like Qualcomm, MediaTek, and Arm. This allows for faster responses and enhanced privacy by keeping data local.
\nLlama 3.2 models are available for download and immediate development.
\nMy takeaways:
\nAs an AI Engineer, it's crucial to discern when to use code, when to leverage Large Language Models (LLMs), or when to combine both to solve a problem efficiently.
\nI've noticed a trend where some developers default to using LLMs without considering simpler, more cost-effective coding solutions. For instance, I recently encountered a scenario where developers were scraping articles from Google RSS feeds but only wanted new articles about US companies. They looped over every article, sending each one to OpenAI to determine its relevance—a process that can be both time-consuming and expensive.
\nThere are several ways to optimize this:
\nFilter by Publication Date and Source Domain:
\nUse Keyword Filtering with Caution:
\nCheck Against Existing Data:
\nAfter these steps, you can then use an LLM to assess the remaining articles' relevance. By refining your approach, you might include multiple articles in a single LLM prompt, reducing the number of requests and lowering costs.
\nThe Takeaway:
\nBefore defaulting to LLMs, consider whether traditional coding can address the problem. Often, a combination of smart coding practices and selective use of LLMs yields the most efficient and cost-effective solution.
\nWork smarter, not harder. 🧠
", "url": "https://www.groff.dev/blog/when-to-code-vs-use-llms-striking-the-right-balance-for-efficiency", "title": "When to Code, When to Use LLMs: Striking the Right Balance for Efficiency", "summary": "Learn how to efficiently balance traditional coding with the use of Large Language Models (LLMs) to solve problems effectively. Discover strategies to optimize processes like web scraping, saving time and resources by knowing when to code and when to leverage AI.", "date_modified": "2024-09-13T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/knowledge-graphs-article-plus-llms-equals-cypher-queries", "content_html": "Today, I’m excited to share my latest work on how we can leverage Large Language Models (LLMs) to derive Cypher Queries directly from an article. I keep getting asked about Knowledge Graphs and this short presentation (just 15 slides!) should get folks excited about using LLMs to generate a Knowledge Graph from news articles.
\nYou can download the presentation here: Knowledge Graphs - Articles + LLMs = Cypher Queries.
\nLet me know what you think! 🚀
", "url": "https://www.groff.dev/blog/knowledge-graphs-article-plus-llms-equals-cypher-queries", "title": "Free Download! Knowledge Graphs - Article + LLM = Cypher Queries", "summary": "Getting a Cypher Query from an Article using a Large Language Model.", "date_modified": "2024-08-17T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/extracting-insights-from-articles-using-groq-and-llama-31", "content_html": "Jim Cramer recently posted on X about AI use cases. He said:
\n\n\n"If i had to put a percentage on it, i would say that 75% of the "use cases" I hear for artificial intelligence are just bogus. The most authentic is Meta and then Amazon..."
\n
I am not a betting man, but I do believe that today I will show you an AI use case that is not bogus.
\nGroq is a cutting-edge "Fast AI Inference" cloud provider that leverages their unique LPUs instead of traditional GPUs. This innovative approach allows Groq to deliver unparalleled speed and efficiency in running AI models, making it a powerful choice for AI applications that require low latency.
\nLlama 3.1 is Meta's latest AI model, released earlier this month, and is already making waves in the AI community. I'm using the 70b versatile version of Llama 3.1 on Groq because it offers higher token limits, which are essential for processing and extracting insights from lengthy articles.
\nIn this article, I'll walk you through how I combine Groq's fast AI inference capabilities with Llama 3.1 to efficiently extract meaningful insights from articles, showcasing a practical AI use case that's anything but bogus.
\nFind the URL of the article you want to extract insights from. (i.e. https://groff.dev/blog/ingesting-pdfs-with-gpt-vision)
\nGet the article's HTML content.
\n![\"Stargate](\"https://pbs.twimg.com/media/Gh2e3_OXkAA5DrA?format=jpg&name=large\")
html2text libraryturndownDetermine the type of article using JSON mode with Groq and Llama 3.1 (politics, sports, tech, etc.) since the insights you may want to extract could be different based on the type of article.
\nBased on the type of article, extract insights using a different System Message to Groq and Llama 3.1, explaining what fields you want to extract.
\nWhy would you want to extract insights from articles? What can I do with them?
\nBuilding a Knowledge Graph to store, visualize, and query relationships between entities. Examples of what you can do with a Knowledge Graph:
\nBut why build a Knowledge Graph? Because you can then query it to answer questions like:
\nIf you don't want to just take my word for it: There was a great talk by Dhagash Mehta from BlackRock's Applied AI Research at NVIDIA GTC about how they scrape financial documents to build their Knowledge Graph. You'll need to make a free NVIDIA developer account to sign up, but it's worth the hassle to get access to these talks from GTC.
\nI'm going to include some code snippets here to show you how you can use Groq and Llama 3.1 to extract insights from articles based on the logic I outlined above. This is an example in Python, but there's no reason why you couldn't apply this to any programming language.
\n!pip install html2text==2024.2.26 groq==0.9.0\nimport json\nimport html2text\nimport requests\nfrom groq import Groq\nI am using the official groq Python sdk to call Groq, but you can use the OpenAI package thanks to Groq's OpenAI compatibility.
url = "https://groff.dev/blog/ingesting-pdfs-with-gpt-vision"\nYou would replace this with the URL of the article you want to extract insights from.
\nresponse = requests.get(url)\nresponse.raise_for_status()\nhtml_content = response.text\nhtml_converter = html2text.HTML2Text()\nhtml_converter.ignore_images = True\nhtml_converter.ignore_links = True\nmarkdown_content = html_converter.handle(html_content)\nAt this stage we will have a Markdown version of the article but also everything else on the webpage including navigation elements, ads, etc.
\nclient = Groq()\ncompletion = client.chat.completions.create(\n model="llama-3.1-70b-versatile",\n messages=[\n {\n "role": "system",\n "content": "You are an article extraction expert. You take in the markdown content of a website and return just the content the user asks for. Do not preface it with 'Here is the article' or append 'Here you go' or 'How can I help you?' No intros or follow-ups just what is asked for."\n },\n {\n "role": "user",\n "content": "Give the user just the article content in markdown format, including only the date, the author, and the article. Do not include links that are outside of the article such as navigation links. Do not include images. Do not include JavaScript tags.\\n\\n---\\n\\n" + markdown_content\n }\n ],\n temperature=0,\n max_tokens=8000,\n top_p=1,\n stream=False,\n stop=None,\n)\nrefined_article = completion.choices[0].message.content\nWe now have a "cleaned up" version of the article that only includes the author, date, and content of the article. This step may not be absolutely necessary, but it can help to remove unnecessary elements from the article which could confuse Llama 3.1 and add extract tokens we don't care about. I say better to be safe than sorry, especially when working with models that don't have the best in class reasonsing capabilities.
\narticle_type_recognition_system_message = """\n# Your Purpose\n1. You will determine the type of the article content provided by the user.\n2. You will not try to generate or summarize the article content.\n3. You will only determine the article type based on the content provided.\n\n# Article Type Definitions\n- "SPORTS": Indicates that the article is about sports, athletes, or sporting events.\n- "POLITICS": Indicates that the article is about politics, politicians, or political events.\n- "DEVELOPMENT": Indicates that the article is about software development or programming.\n- "NOT_APPLICABLE": Indicates that the article type is not one of the above.\n\n# Valid Article Types\n- "SPORTS"\n- "POLITICS"\n- "DEVELOPMENT"\n- "NOT_APPLICABLE"\n\n# Output Format - JSON\nYou will respond in a valid JSON format with the article type and the reasoning behind choosing that article type over the others.\n\n{\n "reasoning": "<The reasoning behind selecting this article type>",\n "article_type_recognized": "<article_type_recognized>"\n}\n"""\n\ncompletion = client.chat.completions.create(\n model="llama-3.1-70b-versatile",\n messages=[\n {\n "role": "system",\n "content": article_type_recognition_system_message\n },\n {\n "role": "user",\n "content": refined_article\n }\n ],\n response_format={"type": "json_object"},\n temperature=0,\n max_tokens=8000,\n top_p=1,\n stream=False,\n stop=None,\n)\narticle_type_result = json.loads(completion.choices[0].message.content)\narticle_type = article_type_result['article_type_recognized']\nI use this System Message pattern when prompting models to determine the type of content. It's a simple way to get a model to classify content based on the content provided. You can further improve the prompt by providing examples of text that would classify an article as a certain type. I find that including a NOT_APPLICABLE case helps the model not hallucinate it's own article types that are not valid in addition to repeating the only valid options it can choose. Even more important I find is making the model output it's reasonsing before the option is chosen. This is similar to the popular "think step by step" prompt engineering technique and in my experience either improves results or at the very least you can see if the reason the model output is non-sensical.
\nmatch article_type:\n case "SPORTS":\n extraction_system_message = """\n # Your Purpose\n 1. You will extract insights related to sports from the article content provided by the user.\n 2. You will not generate or summarize the article content.\n 3. You will only extract insights related to sports.\n\n # Output Format - JSON\n You will respond in a valid JSON format with the extracted insights related to sports.\n\n {\n "sport_being_played": "<sport_being_played>",\n "teams_playing": ["<team_1>", "<team_2>"],\n "scores": {\n "<team_1>": <score_team_1>,\n "<team_2>": <score_team_2>\n },\n "mvp_player": "<mvp_player>"\n }\n """\n case "POLITICS":\n extraction_system_message = """\n # Your Purpose\n 1. You will extract insights related to politics from the article content provided by the user.\n 2. You will not generate or summarize the article content.\n 3. You will only extract insights related to politics.\n\n # Output Format - JSON\n You will respond in a valid JSON format with the extracted insights related to politics.\n\n {\n "politician_names": ["<politician_1>", "<politician_2>"],\n "parties": ["<party_1>", "<party_2>"],\n "bills_sponsored": ["<bill_1>", "<bill_2>"]\n }\n """\n case "DEVELOPMENT":\n extraction_system_message = """\n # Your Purpose\n 1. You will extract insights related to development from the article content provided by the user.\n 2. You will not generate or summarize the article content.\n 3. You will only extract insights related to development.\n\n # Output Format - JSON\n You will respond in a valid JSON format with the extracted insights related to development.\n\n {\n "code_snippets": ["<code_snippet_1>", "<code_snippet_2>"],\n "libraries_mentioned": ["<library_1>", "<library_2>"],\n "tools_mentioned": ["<tool_1>", "<tool_2>"]\n }\n """\n\n completion = client.chat.completions.create(\n model="llama-3.1-70b-versatile",\n messages=[\n {\n "role": "system",\n "content": sports_insights_system_message\n },\n {\n "role": "user",\n "content": refined_article\n }\n ],\n response_format={"type": "json_object"},\n temperature=0,\n max_tokens=8000,\n top_p=1,\n stream=False,\n stop=None,\n )\n extracted_info = completion.choices[0].message.content\nThese are just simple examples of what you can do once you know the type of article you want to extract insights from. You can customize the System Message to extract the insights that you find useful. If we go back to the Knowledge Graph example from before, instead of asking for JSON in this step we could ask for Cypher queries to insert the data into a Neo4j Graph database where you can then query the data to answer questions like the ones I mentioned before. Check out that video from NVIDIA GTC I mentioned earlier to see how they use an LLM to create Cypher queries to insert data into a Neo4j Graph database. There is a course from neo4j on "Knowledge Graphs for RAG" that I found was a great introduction to Knowledge Graphs and working with them alongside LLMs.
\nMy hope for this article is that it inspires you to think about more useful AI use cases so we can prove Jim Cramer wrong about 75% of AI use cases being bogus.
\nAs always feel free to connect with me on LinkedIn. I'd love to hear your feedback and thoughts.
", "url": "https://www.groff.dev/blog/extracting-insights-from-articles-using-groq-and-llama-31", "title": "Extracting Insights from Articles using Groq and Llama 3.1", "summary": "I will explain how and why I use Groq and Llama 3.1 to extract insights from articles.", "date_modified": "2024-08-10T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/semi-autonomous-linux-task-executor", "content_html": "I've been working on a little side project I've entitled SALTE the Semi-Autonomous Linux Task Executor.
\nAfter coming back from NVIDIA GTC 2024 I was inspired to create my own semi-autonomous system that could help me automate tasks on the linux command line. Two talks that really inspired me were "Generally Capable Agents in Open-Ended Worlds" by Jim Fan and "Fireside Chat With Kanjun Qiu and Bryan Catanzaro: Building Practical AI Agents that Reason and Code at Scale" by Bryan Catanzaro and Kanjun Qiu.
\nThe insights I found around Agents were:
\nFor the first version of SALTE I was using gpt-4 for every prompt, which so far is "ambiguity_check", "rewrite_user_provided_task", and "plan_task". I was getting good results but GPT-4 is still the most expensive model to use. I decided to try out ollama with mixtral to see if I could get similar results for a cheaper price. Ollama, for those unfamiliar, is a simple tool to "get up and running with large language models, locally" according to their website and I find that to be true.
Swapping over to Ollama meant giving up on OpenAI Function Calling which I've come to rely on as a neat trick to not only ensure JSON output, but with the right key value pairs and types.
\nOllama plans to support this in the future but as of March 29th, 2024 it's not available. However, I can use the same OpenAI package as GPT-4 and the response_mode JSON option thanks to a recent update to Ollama. I still had to rewrite my code to show apples to apples but it wasn't too painful. A little prompt engineering to get my desired result was the biggest change.
\nBonus: I added Claude 3 Opus to our tests as a third option.
\nPlease enter your task for SALTE: Delete the temporary files\n\nTask: Delete the temporary files\n\nChecking task for ambiguity...\n\nResponse time from mixtral for 'check_for_ambiguity': 3.4046363830566406\n\nAmbiguity detected: What is the file extension of the temporary files and what folder did you mean?\n\nYour response: .tmp files in /var/tmp\n\nResponse time from mixtral for 'rewrite_user_provided_task': 2.9240119457244873\n\nRevised task: Delete all .tmp files in the /var/tmp directory.\n\nTask: Delete all .tmp files in the /var/tmp directory.\n\nChecking task for ambiguity...\n\nResponse time from mixtral for 'check_for_ambiguity': 7.001227617263794\n\nAmbiguity detected: Is it necessary to backup or preserve any of the .tmp files in the /var/tmp directory before deleting them?\n\nYour response: no\n\nResponse time from mixtral for 'rewrite_user_provided_task': 3.564596652984619\n\nRevised task: Delete all .tmp files in the /var/tmp directory without backing up or preserving any of them.\n\nTask: Delete all .tmp files in the /var/tmp directory without backing up or preserving any of them.\n\nChecking task for ambiguity...\n\nResponse time from mixtral for 'check_for_ambiguity': 9.077196836471558\n\nAmbiguity detected: You asked me to delete all .tmp files in the /var/tmp directory. Are there any specific .tmp files you want me to delete, or should I proceed with deleting all .tmp files in that directory?\nNot exactly the desired income. Ollama and Mixtral don't have the same reasoning capabilities of GPT-4 that make it such a good ambiguity checker. It kept detecting ambiguties that were not there. Quality, not so good. Speed, not so fast either even on my rig with an RTX 4090 with 24GB of VRAM.
\nPlease enter your task for SALTE: Delete the temporary files\n\nTask: Delete the temporary files\n\nChecking task for ambiguity...\n\nResponse time from gpt-4 for 'check_for_ambiguity': 2.17018461227417\n\nAmbiguity detected: Could you please specify the location and/or the name pattern or extension of the temporary files you want to delete?\n\nYour response: .tmp files in /var/tmp\n\nResponse time from gpt-4 for 'rewrite_user_provided_task': 1.2635109424591064\n\nRevised task: Delete .tmp files located in /var/tmp directory.\n\nTask: Delete .tmp files located in /var/tmp directory.\n\nChecking task for ambiguity...\n\nResponse time from gpt-4 for 'check_for_ambiguity': 0.6644225120544434\n\nTask is clear and ready for execution. Proceeding with the task.\n\nFinal task: Delete .tmp files located in /var/tmp directory.\nWorking as intended, using Function Calling, fast, but not cheap.
\nPlease enter your task for SALTE: Delete the temporary files\n\nTask: Delete the temporary files\n\nChecking task for ambiguity...\n\nResponse time from mixtral for 'check_for_ambiguity': 8.036858320236206\n\nAmbiguity detected: What is the file extension of the temporary files and in which directory are they located?\n\nYour response: .tmp files in /var/tmp\n\nResponse time from Claude 3 Opus for 'rewrite_user_provided_task': 1.6924209594726562\n\nRevised task: Delete all .tmp files in the /var/tmp directory.\n\nTask: Delete all .tmp files in the /var/tmp directory.\n\nChecking task for ambiguity...\n\nResponse time from mixtral for 'check_for_ambiguity': 7.644544363021851\n\nAmbiguity detected: To clarify, you would like me to delete all files with the .tmp extension specifically in the /var/tmp directory, correct? Just want to make sure I have the right directory and file extension before proceeding.\n\nYour response: Yes\n\nResponse time from Claude 3 Opus for 'rewrite_user_provided_task': 4.24371075630188\n\nRevised task: Delete all files with the .tmp extension in the /var/tmp directory.\n\nTask: Delete all files with the .tmp extension in the /var/tmp directory.\n\nChecking task for ambiguity...\n\nResponse time from mixtral for 'check_for_ambiguity': 14.833811521530151\n\nError decoding JSON from LLM: $Here are a few clarifying questions to resolve ambiguities in the task:\n\n{\n "clarifying_question": "Are you sure you want to delete all files with the .tmp extension in the /var/tmp directory? This action could potentially delete important temporary files used by the system or other applications. Can you confirm this is the intended directory and that deleting these files will not cause any issues?"\n}\n\n{\n "clarifying_question": "Do you have the necessary permissions to delete files in the /var/tmp directory? Deleting files in system directories usually requires root or sudo privileges."\n}\n\n{\n "clarifying_question": "Is there a specific reason or criteria for deleting only the .tmp files in /var/tmp? Are there any .tmp files that should be excluded or retained?"\n}\n\nLet me know the answers to these questions before proceeding with deleting the specified files. It's important to be cautious when deleting files, especially in system directories, to avoid unintended consequences. Expecting value: line 1 column 1 (char 0)\n\nTask is clear and ready for execution. Proceeding with the task.\n\nFinal task: Delete all files with the .tmp extension in the /var/tmp directory.\nClaude 3 Opus is reportedly a better model than GPT-4 in some tasks. In my experience and what I've read online, Claude 3 Opus continues to be "overly safe" and refuse to execute tasks. While we didn't get a refusal here, Opus was hesitant to actually delete even temporary files. It was prompted not to worry about access or privileges same as the other models. I wouldn't choose it for an Ambiguity agent where it could potentially get too "afraid" of executing tasks.
\nYou can find the GPT-4 Ambiguity code on the master branch of my SALTE GitHub repository.
You can find the Ollama w/ Mixtral Ambiguity code on the ollama branch of my SALTE GitHub repository.
You can find the Claude 3 Opus branch of my SALTE GitHub repository to see the code differences.
\nOverall I'm excited for a future where better reasoning LLMs are available locally, but I don't think that day is here today. I'll be sticking with GPT-4 for now, but Claude's Opus 3 might give it a run for it's money.
\nI'd love to hear your thoughts on LinkedIn.
", "url": "https://www.groff.dev/blog/semi-autonomous-linux-task-executor", "title": "SALTE: Semi-Autonomous Linux Task Executor", "summary": "Comparing Ollama w/ Mixtral vs GPT-4 vs Claude 3 Opus for my command line application SALTE: Semi-Autonomous Linux Task Executor", "date_modified": "2024-03-29T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/ingesting-pdfs-with-gpt-vision", "content_html": "When it comes to document processing and analysis, PDFs represent a significant challenge. They're everywhere, from academic papers to business reports, but their unstructured nature makes it difficult to extract and understand the information they contain. This is where AI can play a crucial role, helping us transform PDFs into a more structured and machine-readable format.
\nI've found that simply extracting text from PDFs using python based tools is not exact. Some PDFs work better than others. Some might pull pieces from images and some might not. What we really want is to extract meaning from the document. Missing out on infographics means missing out on meaning.
\nAround 9 months ago I was tasked with building a RAG ChatBot at work for a retail chain that sells food. I was given two documents: an employee training manual and a food safety guide.
\nAt the time, the strategy I heard from other teams was to extract the text from the PDFs and split them up by every 1,000 characters into many "chunks." I had an engineer on another team swear up and down that the chunking strategy did not effect correctness of the ChatBot.
\nBeing the stubborn engineer I am I decided to manually go through the employee training manual and split it up into logical sections of text myself into Markdown files. If there was an infographic or a table I would manually type it out in Markdown. I did this for both documents.
\nThat 1,000 character chunking strategy from the other team? It didn't include these infographics or tables, which took up a lot of the content, especially around food safety and procedures.
\nWhile the other team at the time had early access to the GPT-4 API, we were stuck on GPT-3, but still our ChatBot was able to answer questions about food safety and procedures with a much higher accuracy than the other team's ChatBot.
\nOpenAI uses Markdown as one of the formats for prompting ChatGPT. It's a testament to Markdown's clear and structured nature, making it an ideal format for LLMs to parse and understand. If you try to pass in chunks of retrieved text into the context of a prompt to an LLM and it's nested inside JSON you won't have reliable results. If Markdown is good enough for OpenAI, it's good enough for me.
\nThe process begins with the conversion of PDF documents into images, one for each page, using the pdf2image library. This step is essential for capturing the entire content of the PDF, including charts and images that might be lost in simple text extractions.
Once we have these images, we leverage OpenAI's GPT-4 Vision to interpret and convert them into Markdown text. GPT-4 Vision's advanced capabilities allow it to understand complex layouts and visuals, ensuring that the converted Markdown retains the richness and structure of the original PDF. It is prompted to retain the original layout to the best of it's ability. We then have another pass of each Markdown file using GPT-4 Turbo to remove any irrelevant pieces like pretend images in the markdown, logos, pagent numbers, etc.
\nThis project, encapsulated in a set of Python scripts, provides a seamless workflow for converting PDFs into Markdown format. It includes:
\nimage_splitter.py: Converts a PDF into individual page images, stored in a page_jpegs folder.image_to_markdown.py: Processes each page image through GPT-4 Vision, converting it to Markdown and storing the results in a page_markdowns folder.cleanup_markdown.py: Refines the Markdown output, removing irrelevant content and ensuring a consistent structure.stitch_markdown_pages.py (Optional) Combines all cleaned markdown pages into one coherent Markdown document.While this approach isn't perfect, it's a significant step forward in the quest to make PDFs more accessible and machine-readable. Manually ingesting PDFs by hand into markdown is not scalable. Just extracting text and breaking into 1,000 character chunks wasn't the answer either.
\nHere's the GitHub repo with the scripts that I mentioned above.
\nI hope you find this approach useful and that it inspires you to explore the potential of AI in transforming unstructured data into a more structured and meaningful format. If you have any questions or ideas for improvement, feel free to reach out to me on LinkedIn.
", "url": "https://www.groff.dev/blog/ingesting-pdfs-with-gpt-vision", "title": "Ingesting PDFs using OpenAI GPT-4 Vision", "summary": "Being smart about how we ingest PDFs is crucial. Here's how I used OpenAI's GPT-4 Vision to convert them to Markdown.", "date_modified": "2024-02-18T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/function-agents", "content_html": "Here's how I crafted a Typescript NPM package with Bun called Function Agents:
\nI keep finding myself solving the same problems across different projects and found myself copying a lot of code inbetween them to save time. I figured that I could save myself a lot of time by creating a package that I could import into my projects and use to solve these problems, while also inviting others to contribute their OpenAI Function Agents (as I like to call them).
\nFirst, let's define what a Function Agent is.
\nFunction - In this case I am referring to an OpenAI Function Call. This is an API call using OpenAI's Chat Completion API and passing in well-defined function definitons in the functions argument. The gpt model will, based upon the user's message, system message, and the function definition descriptions, figure out which functions, if any, it should call. This gives you perfect results nearly everytime, by taking the arguments JSON object. It's a neat trick, and I've found it to be very useful in my projects. Each of these Function Agent's are abusing this trick in some way or another to get very consistent results without the use of LangChain or much prompt engineering at all.
Agent - This is a term I use loosely to the collection of Classes in the package. Each has a particular job to do and a persona to go along with it. We're letting OpenAI determine when to call a function, and with what parameters. The Agent guides the way with specific promptings and functions available in its toolset.
\nWith that out of the way let's get into the meat of this article. I'll be going over how I built this package using Bun.
\nmkdir new-project\ncd new-project\n\nbun init\n\n# bun init helps you get started with a minimal project and tries to guess sensible defaults. Press ^C anytime to quit\n\n# package name (test-project): \n# entry point (index.ts): \n\n# Done! A package.json file was saved in the current directory.\n# + index.ts\n# + .gitignore\n# + tsconfig.json (for editor auto-complete)\n# + README.md\n\n# To get started, run:\n# bun run index.ts\nI moved the index.ts file to src/index.ts and updated the package.json file to reflect this change.
The package.json that Bun makes looks like this:
\n{\n "name": "test-project",\n "module": "index.ts",\n "type": "module",\n "devDependencies": {\n "bun-types": "latest"\n },\n "peerDependencies": {\n "typescript": "^5.0.0"\n }\n}\nI'm going to share the package.json from function-agents and I'll highlight key pieces that are necessary:
{\n "name": "function-agents",\n "module": "index.ts",\n "main": "dist/index.js",\n "types": "dist/index.d.ts",\n "type": "module",\n "version": "0.0.17",\n "description": "A collection of Function Agents with specific purposes, utilizing the OpenAI API.",\n "scripts": {\n "build": "bun build --target=node ./src/index.ts --outfile=dist/index.js && bun run build:declaration",\n "build:declaration": "tsc --emitDeclarationOnly --project tsconfig.types.json",\n "postbuild": "rimraf tsconfig.types.tsbuildinfo"\n },\n "devDependencies": {\n "bun-types": "latest",\n "prettier": "^3.0.3",\n "rimraf": "^5.0.5",\n "typescript": "^5.2.2"\n },\n "peerDependencies": {\n "typescript": "^5.0.0"\n },\n "dependencies": {\n "openai": "^4.11.0"\n },\n "repository": {\n "type": "git",\n "url": "https://github.com/mattlgroff/function-agents.git"\n },\n "keywords": [\n "OpenAI",\n "agents",\n "functions",\n "function calling"\n ],\n "author": "Matthew Groff",\n "license": "MIT",\n "files": [\n "dist/*.js",\n "dist/*.d.ts"\n ]\n}\n "scripts": {\n "build": "bun build --target=node ./src/index.ts --outfile=dist/index.js && bun run build:declaration",\n "build:declaration": "tsc --emitDeclarationOnly --project tsconfig.types.json",\n "postbuild": "rimraf tsconfig.types.tsbuildinfo"\n }\nbuild - This is the script that builds the package. It uses Bun to build the package and then runs the build:declaration script. It does not include Type declaration, and Bun doesn't so this right now as of 1.0.3. So we added the following command to use tsc to build typeDefs into our /dist folder.build:declaration - This script uses tsc to build the typeDefs into our /dist folder.postbuild - This script removes the tsconfig.types.tsbuildinfo file that is created when we run build:declaration. This is a file that is created by tsc and is not needed in our package. If it's not deleted, then running bun run build will not generate type definitions. "main": "dist/index.js",\n "types": "dist/index.d.ts",\nmain - This is the entry point for our package. It's the file that will be run when someone imports our package. It's pointed towards the /dist folder, which is where our built code will be.types - This is the entry point for our type definitions. It's pointed towards the /dist folder, which is where our built type definitions will be. "files": [\n "dist/*.js",\n "dist/*.d.ts"\n ]\nFor whatever reason, npm publish wasn't picking up all of the files in the /dist folder (it was missing type defs), so I had to add this to the package.json file. This tells npm publish to include all of the .js and .d.ts files in the /dist folder.
Since Bun doesn't support generating type declaration files, I needed to make a separate tsconfig.types.json. Here's my two files below and I hope that they help you avoid the troubleshooting I had to get the types building.
\ntsconfig.json{\n "compilerOptions": {\n "outDir": "./dist",\n "rootDir": "./src",\n "baseUrl": "./",\n "paths": { \n "@/*": ["src/*"]\n },\n "lib": ["ESNext"],\n "module": "esnext",\n "target": "esnext",\n "moduleResolution": "bundler",\n "moduleDetection": "force",\n "allowImportingTsExtensions": true,\n "noEmit": true,\n "composite": true,\n "strict": true,\n "downlevelIteration": true,\n "skipLibCheck": true,\n "jsx": "react-jsx",\n "allowSyntheticDefaultImports": true,\n "forceConsistentCasingInFileNames": true,\n "allowJs": true,\n "types": [\n "bun-types"\n ]\n },\n "include": [\n "src/**/*.ts"\n ]\n}\ntsconfig.types.json{\n "extends": "./tsconfig.json",\n "compilerOptions": {\n "noEmit": false,\n "emitDeclarationOnly": true,\n "declaration": true,\n "outDir": "./dist",\n "rootDir": "./src",\n },\n "include": [\n "src/**/*.ts"\n ]\n}\n// Export Types\nexport {\n ChatCompletionMessageWithFunction,\n FunctionAgentCodeResponse,\n FunctionAgentJsonResponse,\n FunctionAgentMessageResponse,\n FunctionAgentMessageResponseWithCitation,\n} from './types';\nexport { MathInput, MathResult } from './agents/math';\nexport { Intent } from './agents/intent-classification';\n\n// Export Agents\nexport { default as CitationAgent } from './agents/citation';\nexport { default as DataTransformationAgent } from './agents/data-transformation';\nexport { default as IntentClassificationAgent } from './agents/intent-classification';\nexport { default as JavaScriptCodeInterpreterAgent } from './agents/javascript-code-interpreter';\nexport { default as JavaScriptFunctionCallTransformationAgent } from './agents/javascript-function-call-transformation';\nexport { default as JavascriptDeveloperAgent } from './agents/javascript-developer';\nexport { default as MathAgent } from './agents/math';\nexport { default as SentimentClassificationAgent } from './agents/sentiment-classification';\nThis is the central file, the entry point for our package. It's where we export all of our types and agents. Think of it as a table of content. If I make a new agent or type that I want consumes of function-agents to be able to import, I have to add it here to index.ts and run the build command (bun run build) again.
npm login\n\nnpm publish\nCombining my full-stack Typescript experience with Generative AI has been a lot of fun. I'm excited to see what others do with this package and what Function Agents they create. Feel free to connect with me on LinkedIn.
", "url": "https://www.groff.dev/blog/function-agents", "title": "Crafting a Typescript NPM package with Bun - Function Agents", "summary": "Here's how I crafted a Typescript NPM package with Bun called Function Agents.", "date_modified": "2023-10-01T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/ai-customer-service-agent-with-azure-function-node", "content_html": "Here's how I combined Azure Functions, Node.js, and OpenAI's GPT-4 to craft an efficient email-based AI Customer Service Agent.
\n\n![\"Example](\"/images/photos/AiCustomerServiceAgent.png\")
This Node-based Azure Functions solution focuses on receiving a customer's email, processing it, and then employing OpenAI's GPT-4 to generate an apt response. The response is then dispatched back to the customer using Sendgrid.
\nwhy is the project utilizes the ChatCompletion API when it's primarily not a chat system? The answer lies in OpenAI's Function Calling feature. GPT models, especially GPT-4, excel at orchestrating well-documented processes. In this project, the "sendEmail" function, with a clear definition of subject and body arguments, empowers GPT-4 to generate responses that are almost certain to be valid emails. While there's always room for improvement, a well-defined function minimizes errors.
\nEvery project has its hurdles. One of the challenges faced was handling newline /n characters. The initial approach resulted in generating invalid JSON for the "text" argument. However, by explicitly defining the use of newline characters, the issue was rectified.
\nThe potential to transform this prototype into a full-blown product is immense. Here's the roadmap:
\n*Integration with an Office 365 email inbox for direct customer emails.\n*Incorporation of Azure Logic App to activate the Azure Function upon email receipt.\n*Retrieving the last ten emails from a customer via Office 365's get_email_history to refine response quality.\n*Establishing a Vector Database to maintain and search Frequently Asked Questions, further enhancing response precision.
\ncustomer-service):customer-service to manage and organize the various components of my project.ASP-customerservice-b6bb):us-eastASP-customerservice-b6bb (they came up with the name not me), which essentially defines the hardware context where the Azure Function runs.NewEmailReceived):NewEmailReceived to handle the email processing and response generation. The configuration settings handle all of the environment variables or as Azure calls them "Application Settings".customerserviceaistorage):customerserviceaistorage to cater to this need. This storage account also contains specific containers like azure-webjobs-hosts and scm-releases to support the Azure Function's operations and deployment.In essence, these components formed the backbone of my project on Azure. By seamlessly integrating them, I was able to construct a robust infrastructure that efficiently supports the AI Customer Service Agent's functionalities. If you're diving into Azure for a similar project, I hope this breakdown offers clarity and direction. Happy coding!
", "url": "https://www.groff.dev/blog/ai-customer-service-agent-with-azure-function-node", "title": "Crafting an Email-based AI Customer Service System Using Azure Functions & GPT-4", "summary": "Explore how the synergy of Azure Functions, Node.js, and OpenAI's GPT-4 can revolutionize email customer support. Dive into the step-by-step process of building a responsive AI Customer Service Agent.", "date_modified": "2023-08-12T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/openai-chat-completion-function-calls", "content_html": "The Chat Completion Function Calling is a new and exciting feature in the OpenAI API that enables developers to have more control over the flow of chat-based applications. This functionality offers a way for developers to describe and define functions that the language model can decide to "call" in response to a given user query. This allows you to extract structured data from the model's output, which can then be used in various ways in your application.
\nAs an example, you can build chatbots that answer queries by calling external APIs. Functions like get_current_weather(location: string, unit: 'celsius' | 'fahrenheit') could be called upon to fetch the weather information for a specific location. This means that you can convert natural language into API calls, which can be hugely beneficial for creating intuitive and user-friendly interfaces.
\nImagine being able to simply ask "Who are my top customers?" and having the chatbot convert this to a function call like get_customers(min_revenue: int, created_before: string, limit: int) that interacts with your internal API.
\nOr perhaps you need to extract specific structured data from a block of text. This could be achieved by defining a function like extract_data(name: string, birthday: string), or sql_query(query: string). The possibilities are extensive and can greatly enhance the capabilities of your applications.
\nHowever, it's important to remember that while the model generates the JSON for the function call, it does not actually call the function itself. That step is up to your code. You'll need to parse the generated JSON into arguments for your function, then call the function, and handle the result in whatever way is appropriate for your application.
\nWhile this functionality is indeed powerful, it does also come with potential risks. It's advised to build user confirmation flows before executing any actions that could impact the world on behalf of users. This includes things like sending an email, making a purchase, or posting something online. This way, you can ensure that any function calls being made are in line with the user's intentions.
\nLet's jump right into our JavaScript code example that leverages OpenAI's Chat Completion Function Calling.
\nOur example demonstrates how to use the new feature with a server powered by Bun and Express. The server defines two example functions: get_current_weather and get_movie_info. The get_current_weather function returns a hardcoded response about the current weather in a specified location, while get_movie_info fetches information about a specified movie from the OMDB API.
\nIn order to enable the OpenAI model to use these functions, we explain them in the functionDefinitions object. This includes the function name, a description, and an object that outlines the parameters the function accepts.
\nHere's the code snippet that shows how we define the functions and explain them to OpenAI:
\n// Define the functions\nconst functions = {\n get_current_weather: (location, unit = "fahrenheit") => {\n return "The current weather in " + location + " is 72 degrees and sunny.";\n },\n get_movie_info: async (title, type = 'movie', year, plot = 'short') => {\n // Implementation details omitted for brevity\n },\n // Add your own functions here\n};\n\n// Explain the functions to OpenAI\nconst functionDefinitions = [\n {\n "name": "get_current_weather",\n "description": "Get the current weather and temperature in a given location",\n "parameters": {\n "type": "object",\n "properties": {\n "location": {\n "type": "string",\n "description": "The city and state, e.g. San Francisco, CA",\n },\n "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},\n },\n "required": ["location"],\n },\n },\n {\n "name": "get_movie_info",\n "description": "Get the information about a movie",\n "parameters": {\n "type": "object",\n "properties": {\n "title": {\n "type": "string",\n "description": "The title of the movie"\n },\n "type": {\n "type": "string",\n "enum": ["movie", "series", "episode"],\n },\n "year": {\n "type": "string",\n "description": "The year of release",\n },\n "plot": {\n "type": "string",\n "enum": ["short", "full"],\n },\n },\n "required": ["title"],\n },\n }\n];\nNow, when the OpenAI model's response includes a function call, our server executes the function with the specified parameters. The function call and result are then added to the messages and a new request is sent to the OpenAI API.
\nHere's the especially new pieces of the code, processChatCompletion and processFunctionCall:
// Function to process the OpenAI chat completion\nasync function processChatCompletion(query) {\n return await openai.createChatCompletion({\n model: "gpt-3.5-turbo-0613",\n messages: [{"role": "user", "content": query}],\n functions: functionDefinitions,\n function_call: "auto",\n });\n}\n\n// Function to process the function call\nasync function processFunctionCall(res, query, messages) {\n const lastMessage = messages[messages.length - 1];\n const functionName = lastMessage.function_call.name;\n const args = JSON.parse(lastMessage.function_call.arguments);\n\n console.log('\\nCalling function', functionName, args)\n\n if(functions[functionName]) {\n const argsArray = Object.keys(args).map(key => args[key]);\n const functionResult = await functions[functionName](...argsArray);\n\n // Add the function call and function result to the messages array\n messages.push({\n "role": "function",\n name: functionName,\n "content": JSON.stringify(functionResult),\n });\n\n const chatCompletion = await openai.createChatCompletion({\n model: "gpt-3.5-turbo-0613",\n messages: messages,\n functions: functionDefinitions,\n function_call: "auto",\n });\n\n const newMessage = chatCompletion.data.choices[0].message;\n\n if (newMessage.function_call) {\n // If there's another function call, process it recursively\n return await processFunctionCall(res, query, [...messages, newMessage]);\n } else {\n // If there's no more function call, return the message\n return newMessage;\n }\n } else {\n throw new Error(`Function ${functionName} is not defined.`);\n }\n}\nTo send a query to the server, you can use a tool like curl or Postman. Send a POST request to http://localhost:3000/query with a JSON body like this:
\n{\n "query": "Where city did the movie Forrest Gump start in? What is the weather and temperature there?"\n}\nThe response should look something like this:
\n{\n "message": "The movie Forrest Gump started in the state of Alabama. The current weather in Alabama is 72 degrees and sunny."\n}\nIn the console you can see the function calls happening and the arguments being used:
\nQuestion: Where city did the movie Forrest Gump start in? What is the weather and temperature there?\n\nCalling function get_movie_info {\n title: "Forrest Gump"\n}\n\nCalling function get_current_weather {\n location: "Greenbow, AL"\n}\n\nAnswer: The movie "Forrest Gump" starts in the city of Greenbow, Alabama. The current weather in Greenbow, AL is 72 degrees and sunny.\nTo run the example code, you'll need Bun installed. You'll also need to set two environment variables with a .env, OPENAI_API_KEY and OMDB_API_KEY, which are your API keys for OpenAI and OMDB, respectively. Once you have everything set up, simply use bun install and bun start to install dependencies and start the server.
You can find the full code example at my GitHub Repository.
\nJust like ChatGPT Plugins, this is giving powerful tools to OpenAI's GPT3/4 API to let their models make function calls on your behalf. It's a fun way to translate a human requests in one or many different function calls to get real data. I'm excited to see what people build with this new feature. Plugins are powerful, but end users aren't going to a Plugin Store and using ChatGPT outside of our applications to search for things or ask questions. They are doing it in the apps we build. This is a great way to have the best of both worlds.
\nAs always feel free to connect with me on LinkedIn. I'd love to hear your feedback and thoughts.
", "url": "https://www.groff.dev/blog/openai-chat-completion-function-calls", "title": "OpenAI Chat Completion's API Function Calling - JavaScript Example", "summary": "A code example of how to use the new Chat Completion Function Calling API with JavaScript (Bun and Express)", "date_modified": "2023-06-13T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/introducing-next-rails", "content_html": "Picture this: with just a few simple commands, you can set up a full-stack Next.js application, complete with PostgreSQL, migrations, API routes, and pages. This isn't a developer's fantasy, but the reality I've been crafting. Welcome to the world of next-rails, a CLI tool created to supercharge your Next.js development and make tedious setup a thing of the past.
\nYou can find next-rails on npm
\nYou can find the source code in my github repo
\n\nIn case the embed doesn't work here is a link to my youtube video introducing next-rails
\nI fell in love with the migration/seed/scaffolding features of the Ruby on Rails CLI.
\nI wanted to bring that power and syntax to Next.js applications.
\nSo I did.
\nYou can use it with npx or install it globally.
\nnpx next-rails new my-app\nThis will create a brand new Next.js application using create-next-app and then install a bunch of "extras" on top.
Some of the extras include:
\nshadcn compatibilityknex for migrations and seeds and query buildingdocker-compose.yml to instantly spin up a Next.js development environment with postgresql (including PGVector support!)After you've created your app is where the fun really begins. You can use the next-rails CLI to scaffold out a full-stack CRUD (Create Read Update Delete) application with:
Just like in Rails, your model name should be PascalCase and singular. The CLI will automatically pluralize it for you, the same way that Rails does.
\n# From the 'my-app' directory\nnpx next-rails generate scaffold User name:string age:integer\n\n# You can use g for short\nnpx next-rails g scaffold Todo title:string completed:boolean user:references\n\n# Migrate your changes to the database\nnpx next-rails db:migrate\nFor now I'm going to keep using this in my own projects and see how it goes. I'm sure there are bugs and I'm sure there are features that I haven't thought of yet. Please file an issue on GitHub with feature requests or bug reports. Also definitely contribute if you know how! I don't pay as much attention to making the front-end as pretty as I'd like, but the bones are there for someone to contribute to the page templates and make the apps scaffolded with next-rails to look gorgeous too.
That's all for now! Thanks for reading and I hope you enjoy using next-rails as much as I do!
If you found this post helpful, feel free to connect with me on LinkedIn. Please let me know if you have any questions or feedback!
", "url": "https://www.groff.dev/blog/introducing-next-rails", "title": "Introducing Next-Rails CLI", "summary": "I made an npm package called next-rails that gives the power and syntax of the Rails CLI to Next.js applications.", "date_modified": "2023-06-09T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/step-by-step-create-a-chatbot-with-openai-nextjs-postgres", "content_html": "I went over some of this in my previous post How to store and query your own data using OpenAI Embeddings and Supabase but I'll be doing a complete step by step guide here.
\nSign up on OpenAI's website if you don't already have an account.
\nAfter signing up you can go directly here to create your API key. Save it somewhere safe as you'll need it later. Do not leak this out or commit it in your code accidentally as it can be used to make API calls on your behalf and cause you financial or potentially legal harm.
\nThe important thing to note here is that your Postgres Database of choice must be able to support the extension pgvector.
\nTwo cloud PostgreSQL providers that I recommend that have both generous free tiers to get started as well as support for pgvector are Supabase and Neon.
\nToday I'm going to use Neon, but if you want to see the equivalent steps for Supabase refer past to my previous post.
\nCreate your project, and you'll receive a PostgreSQL connection string that looks something like this:
\npostgres://postgres:postgres@localhost:5432/postgres\nBut instead of localhost, it'll be a domain name that Neon provides you with a different user, passwword, etc. Save this somewhere safe as you'll need it later.
\nI'm going to use next-rails to create my Next.js project. You could use the plain create-next-app if you want, but this guide will be taking advantage of some of the dependencies and setup that next-rails handles for you like Knex.js.
npx next-rails new my-chatbot\n\ncd my-chatbot\nIn next-rails we're using Knex.js to manage our database migrations and seeds. The dependencies will already be installed for you and configured already if you use next-rails.
If you want to develop locally with Docker you can use the docker files that next-rails provides to spin up both Next.js and PostgreSQL (w/ pgvector) containers.
docker compose up\nThe default .env.local is setup with the correct environment variables to connect to the local PostgreSQL container, though you'll need to add your OpenAI API key in the next step.
Open up your .env.local at the root of your Next.js project and add the following environment variables:
OPENAI_API_KEY=your-openai-api-key-from-step-1\nDATABASE_URL=your-postgres-connection-string-from-step-2\nNOTE: Very important for neon you need to add ?sslmode=require to the end of your DATABASE_URL. So it should look like this:
OPENAI_API_KEY=your-openai-api-key-from-step-1\nDATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres?sslmode=require\nOn Neon, and in the default PostgreSQL setups if you want to enable pgvector you have to run a SQL command:
CREATE EXTENSION pgvector;\nYou "could" just go to the Neon console and run this manually (or Supabase has a checkbox to enable extensions on their site) but I prefer to have everything in code so I can easily recreate my database if I need to. That's why we're using Knex migrations.
\nWith next-rails we can use the magic of scaffolding to generate a migration that enables pgvector for us as well as create a table to store our data.
npx next-rails g scaffold Post title:string body:text embedding:vector\nGenerating scaffold Post...\n📁 Creating directory...\n✅ File /home/matt/working/pgtest/src/db/models/post.ts written successfully\n✅ File /home/matt/working/pgtest/src/pages/api/posts.ts written successfully\n✅ File /home/matt/working/pgtest/src/db/migrations/20230604224116_create_posts.js written successfully\n✅ File /home/matt/working/pgtest/src/pages/posts/index.tsx written successfully\nLet's break down the command:
\n npx next-rails g scaffold # Generate a scaffold, which is a model, api route, migration, and page\n\n Post # Our model name, this our db table name will be the plural, lowercase version of this "posts"\n\n title:string # Our first column, the name of the column is "title" and the type is "string"\n\n body:text # Our second column, the name of the column is "body" and the type is "text"\n\n embedding:vector # Our third column, the name of the column is "embedding" and the type is "vector"\nNow we can run the migration it made for us with:
\nnpx next-rails db:migrate\nNow that we have a table to store our data, let's seed it with some data. You can create a new seed file with:
\nnpx next-rails seed:generate seed-posts\nThis will create a new file in db/seeds that looks like this:
/**\n * @param { import("knex").Knex } knex\n * @returns { Promise<void> }\n */\nexports.seed = async function (knex) {\n // Deletes ALL existing entries\n await knex('table_name').del();\n await knex('table_name').insert([\n { id: 1, colName: 'rowValue1' },\n { id: 2, colName: 'rowValue2' },\n { id: 3, colName: 'rowValue3' },\n ]);\n};\nOur seed is going to be a little complicated as we're going to be using the OpenAI API to generate our embeddings before we insert them into our database.
\nFirst, let's install the openai and dotenv packages:
npm install --save openai dotenv\nNow we can update our seed file to look like this:
\n/* eslint-disable @typescript-eslint/no-var-requires */\nrequire('dotenv').config();\nconst { Configuration, OpenAIApi } = require('openai');\n\nconst configuration = new Configuration({\n apiKey: process.env.OPENAI_API_KEY,\n});\n\nconst openai = new OpenAIApi(configuration);\n\n/**\n * @param { import("knex").Knex } knex\n * @returns { Promise<void> }\n */\nexports.seed = async function (knex) {\n // Deletes ALL existing posts\n await knex('posts').del();\n\n // Define the array of posts\n const posts = [\n { title: 'Post 1', body: 'Hamburgers' },\n { title: 'Post 2', body: 'Bananas' },\n // Add more posts here...\n ];\n\n for (const post of posts) {\n // Call OpenAI API to create the embedding\n const embeddingResponse = await openai.createEmbedding({\n model: 'text-embedding-ada-002',\n input: post.body,\n });\n const [{ embedding }] = embeddingResponse.data.data;\n\n // Prepare the embedding for insertion\n const embeddingString = `'${JSON.stringify(embedding)}'`;\n\n // Insert the post into the posts table\n await knex.raw(\n `\n INSERT INTO posts (title, body, embedding) \n VALUES (?, ?, ${embeddingString})\n `,\n [post.title, post.body]\n );\n }\n};\nThis is a little different from a typical seed file because it's not just some static data that we're inserting into our database. We're actually calling the OpenAI API to generate the embeddings for us.
\nIf your data is very lengthy and you don't want to in-line it in your seed file you can store it in a separate file and read it in with fs like so:
/* eslint-disable @typescript-eslint/no-var-requires */\nrequire('dotenv').config();\nconst { Configuration, OpenAIApi } = require('openai');\nconst fs = require('fs');\n\nconst configuration = new Configuration({\n apiKey: process.env.OPENAI_API_KEY,\n});\n\nconst openai = new OpenAIApi(configuration);\n\n/**\n * @param { import("knex").Knex } knex\n * @returns { Promise<void> }\n */\nexports.seed = async function (knex) {\n // Deletes ALL existing posts\n await knex('posts').del();\n\n // Read the array of posts from the JSON file\n const posts = JSON.parse(fs.readFileSync('./posts.json', 'utf8'));\n\n for (const post of posts) {\n // Call OpenAI API to create the embedding\n const embeddingResponse = await openai.createEmbedding({\n model: 'text-embedding-ada-002',\n input: post.body,\n });\n const [{ embedding }] = embeddingResponse.data.data;\n\n // Prepare the embedding for insertion\n const embeddingString = `'${JSON.stringify(embedding)}'`;\n\n // Insert the post into the posts table\n await knex.raw(\n `\n INSERT INTO posts (title, body, embedding) \n VALUES (?, ?, ${embeddingString})\n `,\n [post.title, post.body]\n );\n }\n};\nEither way, now we can run the seed with:
\nnpx next-rails db:seed\nAssuming you have no errors, you should now have some data in your database, and each post should have an embedding column with an OpenAI embedding.
To validate that your table is setup correctly and you have data in it, you can use the startup your Next.js app and visit the posts page to see the data.
\n![\"Posts](\"/images/photos/PostsPage.png\")
To recap: we have a table in our database with some data in it, and each row has an OpenAI embedding. Now we need to create an API route that will take a query string and return the most similar post to that query.
\nBefore we make an API Route, we need to create a postgres function to find the closest embedding to a given embedding. That's the magic behind RAG (Retrieval Augmented Generation) and the OpenAI API.
Make a new migration:
\nnpx next-rails migration:generate match-posts-function\nIn our migration add the following code:
\n/**\n * @param { import("knex").Knex } knex\n * @returns { Promise<void> }\n */\nexports.up = function (knex) {\n return knex.raw(`\n create or replace function match_posts (\n query_embedding vector(1536),\n match_threshold float,\n match_count int\n )\n returns table (\n id uuid,\n body text,\n title text,\n similarity float\n )\n language sql stable\n as $$\n select\n posts.id,\n posts.body,\n posts.title,\n 1 - (posts.embedding <=> query_embedding) as similarity\n from posts\n where 1 - (posts.embedding <=> query_embedding) > match_threshold\n order by similarity desc\n limit match_count;\n $$;\n\n create index on posts using ivfflat (embedding vector_cosine_ops)\n with\n (lists = 100);\n `);\n};\n\n/**\n * @param { import("knex").Knex } knex\n * @returns { Promise<void> }\n */\nexports.down = function (knex) {\n return knex.raw(`\n drop function match_posts;\n `);\n};\nThis finds the n (match_count) number of matches, sorted by similarity, for a given similarity threshold (match_threshold). It also creates an index on the embedding column to speed up the query.
Yet again let's run our migration:
\nnpx next-rails db:migrate\nNow we can create a Next.js API route to query our data. Create a new file at pages/api/query.ts with the following code:
import type { NextApiRequest, NextApiResponse } from 'next';\nimport { Configuration, OpenAIApi } from 'openai';\nimport { getKnex } from '@deps/db';\n\nconst knex = getKnex();\n\nconst configuration = new Configuration({\n apiKey: process.env.OPENAI_API_KEY,\n});\n\nconst openai = new OpenAIApi(configuration);\n\nexport default async function handler(\n req: NextApiRequest,\n res: NextApiResponse\n) {\n if (req.method !== 'POST') {\n res.setHeader('Allow', ['POST']);\n res.status(405).end(`Method ${req.method} Not Allowed`);\n return;\n }\n\n try {\n const { query, previousMessages } = req.body;\n if (!query) {\n res.status(400).json({ error: 'Missing query' });\n return;\n }\n\n const input = query.replace(/\\n/g, ' ');\n const embeddingResponse = await openai.createEmbedding({\n model: 'text-embedding-ada-002',\n input,\n });\n const [{ embedding }] = embeddingResponse.data.data;\n\n const matchThreshold = 0.78;\n const matchCount = 10;\n\n const sql = `SELECT * FROM match_posts(ARRAY[${\n '??,'.repeat(embedding.length - 1) + '??'\n }]::vector(1536), ?, ?)`;\n const bindings = [...embedding, matchThreshold, matchCount];\n\n const { rows: posts } = await knex.raw(sql, bindings);\n\n let postsText = '';\n for (const post of posts) {\n postsText += `\\nTitle: ${post.title}\\nBody: ${post.body}\\n`;\n }\n\n let previousQuestionsText = '';\n for (const message of previousMessages) {\n previousQuestionsText += `\\nPrevious Question: ${message}`;\n }\n\n const systemMessage = `\n You are a friendly and knowledgeable chatbot, trained to provide accurate answers using the information found in your Context. Answer the question using only that information. Please answer in a conversational and factual manner, including the relevant information from the Context provided. If the answer is not explicitly written in the Context, say "Sorry, I can't help you with that."\n \n Context:\n ${postsText}\n \n Previously asked questions:\n ${previousQuestionsText}\n `;\n\n const userMessage = `Question: "${query}"`;\n\n console.log('systemMessage', systemMessage);\n console.log('userMessage', userMessage);\n\n // Now we can pass our messages to the gpt-3.5-turbo model to get our completion (result)\n const chatResponse = await openai.createChatCompletion({\n model: 'gpt-3.5-turbo',\n messages: [\n {\n role: 'system',\n content: systemMessage,\n },\n {\n role: 'user',\n content: userMessage,\n },\n ],\n max_tokens: 2048,\n temperature: 0.2,\n });\n\n if (\n !chatResponse.data.choices ||\n !chatResponse.data.choices[0]?.message?.content\n ) {\n throw new Error(\n 'There was a problem when asking your question, please try again.'\n );\n }\n\n const {\n id,\n choices: [\n {\n message: { content: text },\n },\n ],\n } = chatResponse.data;\n\n res.status(200).json({ id, text });\n } catch (error) {\n console.error(error);\n if (error instanceof Error) {\n res.status(500).json({ error: error.message });\n } else {\n res.status(500).json({ error: 'An unknown error occurred.' });\n }\n }\n}\nHere is an example you can toss into src/pages/index.tsx to replace the default Next.js homepage:
import { useState, useRef, useEffect, FormEvent } from 'react';\nimport Head from 'next/head';\n\ntype Message = {\n text: string;\n from: 'user' | 'bot';\n id: string;\n};\n\nconst Chat = () => {\n const [messages, setMessages] = useState<Message[]>([]);\n const [loading, setLoading] = useState(false);\n\n // Reference to the chat box\n const chatBoxRef = useRef<HTMLDivElement | null>(null);\n\n // Scroll to bottom of chatbox upon new messages\n useEffect(() => {\n if (chatBoxRef.current) {\n chatBoxRef.current.scrollTop = chatBoxRef.current.scrollHeight;\n }\n }, [messages]);\n\n // Fetch the response to the question\n const askQuestion = async (query: string) => {\n setLoading(true);\n\n try {\n // Get the last 5 messages sent by the user\n const previousMessages = messages\n .filter((message) => message.from === 'user')\n .map((message) => message.text)\n .slice(-5);\n\n const res = await fetch('/api/query', {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({ query, previousMessages }),\n });\n const data = await res.json();\n\n if (data.error) {\n setMessages((prev) => [\n ...prev,\n {\n text: 'Sorry, there was an issue with your request. Please try again.',\n from: 'bot',\n id: `bot_${Date.now()}`,\n },\n ]);\n } else if (data.id && data.text) {\n // Add the response to the messages\n setMessages((prev) => [\n ...prev,\n { text: data.text, from: 'bot', id: data.id },\n ]);\n }\n } catch (error) {\n setMessages((prev) => [\n ...prev,\n {\n text: 'Sorry, there was an issue with your request. Please try again.',\n from: 'bot',\n id: `bot_${Date.now()}`,\n },\n ]);\n }\n\n setLoading(false);\n };\n\n // Handle when the user submits a question\n const handleSubmit = (e: FormEvent<HTMLFormElement>) => {\n e.preventDefault();\n\n const input = e.currentTarget.querySelector('input') as HTMLInputElement;\n\n // Add the user's question to the messages immediately\n const query = input.value;\n setMessages((prev) => [\n ...prev,\n { text: query, from: 'user', id: `user_${Date.now()}` },\n ]);\n\n // Fetch the response and clear the input\n input.value = '';\n askQuestion(query);\n };\n\n return (\n <>\n <Head>\n <title>ChatBot</title>\n </Head>\n <main className="p-4">\n <h1 className="mb-4 text-center text-2xl font-semibold">ChatBot</h1>\n\n <div\n ref={chatBoxRef}\n className="mx-auto flex h-[500px] flex-col items-stretch space-y-3 overflow-auto rounded border bg-gray-50 p-4"\n >\n {messages.map((message) => (\n <div\n key={message.id}\n className={`text-${\n message.from === 'user' ? 'right' : 'left'\n } rounded-lg p-2 bg-${\n message.from === 'user'\n ? 'blue-500 text-white'\n : 'green-200 text-black'\n }`}\n >\n {message.text}\n </div>\n ))}\n </div>\n\n <form className="mt-4" onSubmit={handleSubmit}>\n <input\n required\n type="text"\n className="w-full rounded border p-2"\n placeholder="Ask a question..."\n />\n <button\n type="submit"\n className="mt-2 w-full rounded bg-blue-500 p-2 text-white"\n disabled={loading}\n >\n {loading ? 'Loading...' : 'Ask'}\n </button>\n </form>\n </main>\n </>\n );\n};\n\nexport default Chat;\nLet's visit the page and try asking a question about one of our posts:
\n![\"Post2GoesBananas\"/](\"/images/photos/Bananas.png\")
The easiest way to deploy your app is to use Vercel. You can sign up with your GitHub account and deploy your app in seconds. I recommend following their deployment guide here.
\nMake sure you add your environment variables to your Vercel project. You can do this by going to your project's settings and adding them under the "Environment Variables" section. You'll need your OPENAI_API_KEY and DATABASE_URL.
\nYou're done! You now have a chatbot that can answer questions about your data. I highly recommend playing around with your match_threshold, match_count, and the temperature of the GPT model to get the best results for your use case. You can definitely swap out gpt-3.5-turbo for a different model if you want to experiment with that as well. At the time of this writing gpt-4 is still waitlist only and it's much more expensive.
Happy coding!
\nIf you found this post helpful, feel free to connect with me on LinkedIn. Please let me know if you have any questions or feedback!
", "url": "https://www.groff.dev/blog/step-by-step-create-a-chatbot-with-openai-nextjs-postgres", "title": "Step by Step: Create a Chatbot with OpenAI & Next.js & Postgres For Free", "summary": "Learn how to create a chatbot with OpenAI, Next.js, and Postgres for free.", "date_modified": "2023-06-04T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/openai-embeddings-supabase", "content_html": "Inspired by this article by Supabase on the same topic
\nOpenAI Embeddings are a set of vectors that represent a word or phrase. They are trained on a large corpus of text and are able to represent the meaning of a word or phrase in a vector space. This means that similar words or phrases will have similar vectors. For example, the vectors for "cat" and "dog" will be closer together than the vectors for "cat" and "car".
\nVectors are a way of representing data in a multi-dimensional space. For example, a vector with two dimensions can be represented as a point on a graph. A vector with three dimensions can be represented as a point in 3D space. A vector with four dimensions can be represented as a point in 4D space, and so on.
\nYou may have heard of vector artwork, such as an .svg file. You can increase and decrease the size without distortion because it's made of math, rather than specific pixels. This is the same idea, but with words and phrases.
In the database, a vector will be stored as an array of numbers like so:
\n[0.015045881,-0.017935315, ...]\nThis would repeat way more times depending on how much text you have stored in your database as an embedding.
\nSo we want a way to store and query text data to OpenAI using these Embeddings in our database. We can do this using the pgvector extension for Postgres, or by using a specific Vector Database like pinecone which markets itself as "Long-term Memory for AI" which is mostly true.
\nWe can turn a blob of text (think like a blog post, article, chapter of a book, section of a book, etc) into an Embeddings (to query store in a Vector DB) using the OpenAI API.
\nYou can check out the specific docs around how this API works here but we're using the openai npm package.
const openai = require('openai-api');\n\nconst configuration = new Configuration({\n apiKey: 'youropenapikeygoeshere',\n});\n\nconst openai = new OpenAIApi(configuration);\n\n// Your text goes here.\nconst body = `This could be a blog post or anything that you want that is a unit of the things you are trying to index and search for.`;\n\n// This sends the text to the OpenAI API and returns the Embedding.\nconst embeddingResponse = await openai.createEmbedding({\n model: 'text-embedding-ada-002',\n input: body,\n});\n\n// This is our embedding. We'll store this in our Vector Database.\nconst [{ embedding }] = embeddingResponse.data.data\nOpenAI will transform our text value into an Embedding. We can then store this in our Vector Database.
\nIn my example today we'll be using Supabase (Postgres) with the pgvector extension that allows us to store vector files.
\nWe'll create a table called posts with a title and body column. We'll also add a embedding column that will store our Embedding.
create table posts (\n id serial primary key,\n title text not null,\n body text not null,\n embedding vector(1536)\n);\nNote, the only thing relevant to the embedding is the "body", we aren't sending the title along to the OpenAI API. You can still reference it in your query later if you so choose to.
\nNow we can store our Embedding in the database.
\n// code from before\nconst [{ embedding }] = embeddingResponse.data.data\n\n// Store the vector in Postgres\nconst { data, error } = await supabase.from('posts').insert({\n title,\n body,\n embedding,\n});\nYou'll want to run this piece of code for every new post that you'd want to store in your database. You can also run this on existing posts if you want to add Embeddings to them, say you had a DB of existing blogs, you can add the embedding vector column and update if you looped over them.
\nLet's assume you added the rest of the posts than you wanted to add to your database. Now we can query them.
\nIn order to "hook up" OpenAI to our embeddings we need to create a function in Postgres find the closest matching values when given a vector.
\n create or replace function match_posts (\n query_embedding vector(1536),\n match_threshold float,\n match_count int\n )\n returns table (\n id bigint,\n body text,\n title text,\n similarity float\n )\n language sql stable\n as $$\n select\n posts.id,\n posts.body,\n posts.title,\n 1 - (posts.embedding <=> query_embedding) as similarity\n from posts\n where 1 - (posts.embedding <=> query_embedding) > match_threshold\n order by similarity desc\n limit match_count;\n $$;\nAdditionally we can create an index on our posts table to speed up the query.
\ncreate index on posts using ivfflat (embedding vector_cosine_ops)\nwith\n (lists = 100);\nNow we can query our Embeddings in Supabase using the following code:
\nconst query = `What's cooler than being cool?`;\n\n// OpenAI recommends replacing newlines with spaces for best results\nconst input = query.replace(/\\n/g, ' ');\n\n// Generate a one-time embedding for the query itself\nconst embeddingResponse = await openai.createEmbedding({\n model: 'text-embedding-ada-002',\n input,\n});\n\nconst [{ embedding }] = embeddingResponse.data.data;\n\n// This is asking supabase to run our function and find the closests 10 matches. You can change the threshold and match as you see fit.\nconst response = await supabase.rpc('match_posts', {\n // match_posts should be the name of the postgres function you created previously)\n query_embedding: embedding,\n match_threshold: 0.78, // Choose an appropriate threshold for your data\n match_count: 10, // Choose the number of matches\n});\n\n// This is the data that we get back from our query.\nconst { data: posts } = response;\n\n// 10 Matched Documents found by the request to Supabase that closest match the query.\nconsole.log(JSON.stringify(posts, null, 2));\ntitle body id and similarity (how close the match is to our query).Now all that's left to do is ask OpenAI one more time, this time for the actual answer to our query "What's cooler than being cool?" using the 10 matches we got back from Supabase for context.
\nconst { data: posts } = response;\n\n// 10 Matched Documents\nconsole.log(JSON.stringify(posts, null, 2));\n\n// Prompting\nconst prompt = `\nYou are a very enthusiastic ChatBot who loves to help people! Given the following Posts from this collection, answer the question using only that information, outputted in markdown format. If you are unsure and the answer is not explicitly written in the collection provided, say "Sorry, I don't have that knowledge."\n\nContext Posts:\n${posts}\n\nQuestion: """\n${query}\n"""\n\nAnswer as markdown:\n`;\n\n// This is the answer to our query.\nconst completionResponse = await openai.createCompletion({\n model: 'text-davinci-003',\n prompt,\n max_tokens: 512, // Choose the max allowed tokens in completion\n temperature: 0, // Set to 0 for deterministic results\n});\n\nconst {\n id,\n choices: [{ text }],\n} = completionResponse.data;\n\nconsole.log('---');\nconsole.log('Query Response from OpenAI GPT Completions:');\nconsole.log(JSON.stringify({ id, text }, null, 2));\nAnd there we have it. To recap everything our query went through:
\ntitle body id and similarity (how close the match is to our query).This is taking the idea of ChatGPT Chat History to the next level. You can probably see if folks are storing their personal chat history in vector databases to effectively have long-term memory for their AI, they can also store their blog posts, articles, etc. and query them using the same method.
\nI hope this was helpful and you can see the power of OpenAI Embeddings and Supabase together. You can just as easily apply this to any other Vector Database like Pinecone, etc.
\nWhat do you think of Embeddings and Vector Databases? Let me know on LinkedIn.
", "url": "https://www.groff.dev/blog/openai-embeddings-supabase", "title": "How to store and query your own data using OpenAI Embeddings and Supabase", "summary": "Learn how to store and query your own data using OpenAI Embeddings and Supabase", "date_modified": "2023-05-31T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/next-rails-cli", "content_html": "I only really got started with Ruby on Rails earlier this year, but after Jordan Smith showed me rails generate scaffold and how easy it was to quickly generate new CRUD boilerplate I fell in love.
I've been working with Next.js for a while now, but I've been missing out on those backend cli features that make Rails so fast to use to get started.
\nSo, I conceptualized a package that would let me do the same thing with Next.js. I've never "really" made an npm package before, so I figured this would be a good opportunity to learn.
\nHere's what I've come up with: next-rails
A CLI for generating a Next.js app with Rails CLI-like features.
\n✅ = Implemented\n❌ = Not Implemented (Yet)
\nYou can either install it globally with npm install -g next-rails or use it with npx like npx next-rails.
The goal of this package is to make a scaffold command that uses the familiar Ruby on Rails CLI syntax to generate:
src/pages directory for each of the CRUD actions (index, show, new, edit)src/pages/api directory to support these pagessrc/db/models directory that includes a Typescript Interface for the model as well as some metadata that can be used to generate the content on the pagessrc/db/migrations directory that can be used to create the database table for the modelI've got a lot of work to do to get this to where I want it to be, but I'm excited to see where it goes. I currently have a very basic index page being created for the "views" but the api routes, model, and migrations are already working. If you run into any problems file an issue on the GitHub Repository.
\nNode seemed like a good fit as that's what Next.js already requires. I considered Bun, but it doesn't have child_process which is really critical in this app and it's found throughout it to automate the creation of files and directories.
\nI originally wanted to use sequelize for this project because it's migration/seed syntax matches that of the Rails CLI out of the box (i.e db:migrate db:seed db:migration:generate). However it turned out to not play nicely. Knex on the other hand only needed 2 files to get up and running on Next.js and supported the same type of migration commands as sequelize did, so it became a good match. I did not go with Prisma or DrizzleORM because they are schema-first and generate migrations automatically for you. That might be fine and perfectly safe but it didn't match the way I was used to doing things with Rails or Sequelize, so they were not a good fit for this package.
Embarking on this project was not without its challenges. So, what was the hardest part?
\nWriting code for code generation proved to be a challenging endeavor. As this was my initial venture into this domain, I'm aware that there might be more efficient methods than the ones I've utilized. Notably, AI technology such as ChatGPT has suggested a shift from solely using string interpolation towards the implementation of a templating engine like Handlebars. While I'm still pondering this transition, I'm confident that this is an area ripe with opportunities for improvement and innovation.\nHere's a code snippet to get an idea of what I'm talking about, this is generating the first version of the "index" page:
\nfunction generateIndexPage(singularModelName, pluralModelName, options) {\n const modelName =\n singularModelName.charAt(0).toUpperCase() + singularModelName.slice(1)\n const PascalPluralModelName =\n pluralModelName.charAt(0).toUpperCase() + pluralModelName.slice(1)\n\n // Helper function to convert snake_case to Title Case\n function toTitleCase(str) {\n return str\n .replace('_', ' ')\n .replace(\n /\\w\\S*/g,\n (txt) => txt.charAt(0).toUpperCase() + txt.substr(1).toLowerCase()\n )\n }\n\n // Construct the model metadata\n const modelMetadata = {\n id: { label: 'ID', display: (value) => value },\n created_at: {\n label: 'Created At',\n display: (value) => value?.toLocaleString() || '',\n },\n updated_at: {\n label: 'Updated At',\n display: (value) => value?.toLocaleString() || '',\n },\n }\n\n // Add each option as a field in the metadata\n options.forEach((option) => {\n const [name, type] = option.split(':')\n const label = toTitleCase(name)\n let displayFunction\n\n switch (type) {\n case 'boolean':\n displayFunction = (value) => (value ? 'Yes' : 'No')\n break\n case 'date':\n displayFunction = (value) => (value ? value.toISOString() : '')\n break\n default:\n displayFunction = (value) => value\n break\n }\n\n modelMetadata[name] = { label: label, display: displayFunction }\n })\n\n const fields = Object.keys(modelMetadata)\n\n // Constructing the JSX for each row of data\n const tdJsx = fields\n .map(\n (field) => `\n<td className="whitespace-nowrap px-3 py-4 text-sm font-medium text-gray-900">\n {${singularModelName}.${field} !== null && typeof ${singularModelName}.${field} !== 'undefined' ? modelMetadata['${field}'].display(${singularModelName}.${field}) : ""}\n</td>\n`\n )\n .join('')\n\n const mapJsx = `\nconst fieldData = ${pluralModelName}.map(${singularModelName} => (\n <tr key={${singularModelName}.id}>\n ${tdJsx}\n <td className="whitespace-nowrap px-3 py-4 text-sm text-gray-500 text-right">\n <Link href="/${pluralModelName}/{${singularModelName}.id}">\n Show\n </Link>\n </td>\n </tr>\n));\n`\n\n const viewCode = `\n import { GetServerSideProps } from "next";\n import { ${modelName}, ${singularModelName}Metadata as modelMetadata } from "@deps/db/models/${singularModelName}";\n import Link from "next/link";\n import { getKnex } from "@deps/db";\n \n interface ${PascalPluralModelName}PageProps {\n ${pluralModelName}: ${modelName}[];\n }\n \n const ${PascalPluralModelName}Page = ({ ${pluralModelName} }: ${PascalPluralModelName}PageProps) => {\n ${mapJsx}\n return (\n <div className="mt-3 flow-root">\n <div className="inline-block min-w-full py-2 align-middle sm:px-6 lg:px-8">\n <div className="overflow-hidden shadow ring-1 ring-black ring-opacity-5 sm:rounded-lg">\n <table className="min-w-full divide-y divide-gray-300">\n <thead className="bg-gray-50">\n <tr>\n ${fields\n .map(\n (field) => `\n <th\n scope="col"\n className="px-3 py-3.5 text-left text-sm font-semibold text-gray-900"\n >\n ${modelMetadata[field].label}\n </th>\n `\n )\n .join('')}\n <th\n scope="col"\n className="px-3 py-3.5 text-left text-sm font-semibold text-gray-900"\n >\n Actions\n </th>\n </tr>\n </thead>\n <tbody className="divide-y divide-gray-200 bg-white">\n {fieldData}\n </tbody>\n </table>\n </div>\n </div>\n </div>\n );\n };\n \n export const getServerSideProps: GetServerSideProps = async () => {\n const knex = getKnex();\n const ${singularModelName}sFromKnex = await knex("${singularModelName}s");\n const ${singularModelName}s = ${singularModelName}sFromKnex.map((${singularModelName}: ${modelName}) => ({\n ...${singularModelName},\n created_at: ${singularModelName}.created_at?.toISOString(),\n updated_at: ${singularModelName}.updated_at?.toISOString(),\n }));\n \n return {\n props: {\n ${singularModelName}s,\n },\n };\n };\n \n export default ${PascalPluralModelName}Page;\n `\n\n return viewCode\n}\n\nmodule.exports = generateIndexPage\nIt's not pretty code. I wouldn't even argue that its elegant at all, but it does work! I'm sure there are better ways to do this, but I'm not sure what they are yet. I'm sure I'll learn more as I continue to work on this project.
\nNow that you're familiar with next-rails, you might be wondering how to use it. Let's dive in! After installing globally or using npx next-rails we can use it to generate a brand new Next.js application with Knex, ES-Lint, Prettier, TailwindCSS, and PostgreSQL already set up for us.
next-rails new my-app\nReplace "my-app" with whatever you want the app to be called. Behind the scenes this is running the create-next-app CLI and then additionally installing and copying over the additional dependencies and files needed to get up and running.
You'll seen an output like this in your terminal when it's done:
\nCreating Next.js app with the following command: npx --no-install create-next-app my-app --ts --eslint --no-app --use-npm --src-dir --import-alias "@deps/*" --tailwind\n✔ Would you like to use experimental `app/` directory with this project? … No / Yes\nCreating a new Next.js app in /home/matt/working/test/my-app.\n\nUsing npm.\n\nInitializing project with template: default-tw\n\nInstalling dependencies:\n- react\n- react-dom\n- next\n- typescript\n- @types/react\n- @types/node\n- @types/react-dom\n- tailwindcss\n- postcss\n- autoprefixer\n- eslint\n- eslint-config-next\n\nadded 352 packages, and audited 353 packages in 6s\n\nInitialized a git repository.\nSuccess! Created my-app at /home/matt/working/test/my-app\nCopying over the new README...\nInstalling additional dependencies...\nCopying over the ESLint configuration...\nCopying over the Prettier configuration...\nCopying over the Knex configuration...\nCreating src/db for knex...\nCreating src/db/migrations for knex...\nCreating src/db/seeds for knex...\nCreating src/db/models for knex...\nCopying over the Knex index.js to src/db/index.js...\nNow that the app is created and has all the dependencies that we need installed, we can use the scaffold command to generate a new model, migration, api route, and pages for us.
next-rails generate scaffold Article name:string email:string password:string\nYou'll something like this in your terminal:
\nGenerating scaffold Article...\n✅ File /home/matt/working/test/my-app/src/db/models/article.ts written successfully\n✅ File /home/matt/working/test/my-app/src/pages/api/articles.ts written successfully\n✅ File /home/matt/working/test/my-app/src/db/migrations/20230530003527_create_articles.js written successfully\n✅ File /home/matt/working/test/my-app/src/pages/articles/index.tsx written successfully\nTo give you a better understanding of how next-rails works, let's take a look at the code that was generated.
\nHere's the migration src/db/migrations/20230530003527_create_articles.js:
exports.up = function (knex, Promise) {\n return knex.schema\n .raw('CREATE EXTENSION IF NOT EXISTS "uuid-ossp"')\n .createTable('articles', function (table) {\n table.uuid('id').primary().defaultTo(knex.raw('uuid_generate_v4()'))\n table.string('title').notNullable()\n table.string('author').notNullable()\n table.timestamp('created_at').defaultTo(knex.fn.now())\n table.timestamp('updated_at').defaultTo(knex.fn.now())\n })\n}\n\nexports.down = function (knex, Promise) {\n return knex.schema.dropTable('articles')\n}\nGeneration defaults to having a UUID for our primary key and we have created_at and updated_at timestamps. The nice thing about that is if you agree with this, great! If not, it's easy to go in and change it. These files are meant to be a started point for rapid development, not necessarily the best practice for every situation.
Here's the model src/db/models/article.ts:
export interface Article {\n id: string;\n title: string;\n author: string;\n created_at: Date;\n updated_at: Date;\n}\n\nexport interface ArticleMetadata {\n id: { label: string; display: (value: string) => string };\n title: { label: string; display: (value: string) => string };\n author: { label: string; display: (value: string) => string };\n created_at: { label: string; display: (value: Date) => string };\n updated_at: { label: string; display: (value: Date) => string };\n}\n\nexport const articleMetadata: ArticleMetadata = {\n id: { label: 'ID', display: (value: string) => value },\n title: { label: 'Title', display: (value: string) => value },\n author: { label: 'Author', display: (value: string) => value },\n created_at: { label: 'Created At', display: (value: Date) => value?.toLocaleString() || '' },\n updated_at: { label: 'Updated At', display: (value: Date) => value?.toLocaleString() || '' },\n};\nYou can import the Typescript interface anywhere in the app that you want to refer to it. The metadata is used to generate the table headers and the data for the index page (and other pages, eventually).
\nHere's the api route src/pages/api/articles.ts:
import type { NextApiRequest, NextApiResponse } from 'next';\nimport { getKnex } from '@deps/db';\n\nconst knex = getKnex();\n\nexport default async function handler(req: NextApiRequest, res: NextApiResponse) {\n const {\n method,\n query: { id },\n } = req;\n\n switch (method) {\n case 'GET':\n if (id) {\n const instructor = await knex('instructors').where('id', id).first();\n res.status(200).json(instructor);\n } else {\n const instructors = await knex('instructors');\n res.status(200).json(instructors);\n }\n break;\n\n case 'POST':\n const newInstructor = req.body;\n const [insertedInstructor] = await knex('instructors').insert(newInstructor).returning('*');\n res.status(201).json(insertedInstructor);\n break;\n\n case 'PUT':\n case 'PATCH':\n if (id) {\n const updatedInstructor = req.body;\n const [updatedEntry] = await knex('instructors').where('id', id).update(updatedInstructor).returning('*');\n res.status(200).json(updatedEntry);\n } else {\n res.status(400).json({ message: 'Missing ID' });\n }\n break;\n\n case 'DELETE':\n if (id) {\n await knex('instructors').where('id', id).del();\n res.status(200).json({ message: `Instructor with id ${id} deleted` });\n } else {\n res.status(400).json({ message: 'Missing ID' });\n }\n break;\n\n default:\n res.setHeader('Allow', ['GET', 'POST', 'PUT', 'PATCH', 'DELETE']);\n res.status(405).end(`Method ${method} Not Allowed`);\n }\n}\nThis I tried to stick as close to the Rails Controller that I could. I have not heavily tested these and some of them are AI generated, so again please if you have any problems file an issue on GitHub.
\nSo there's a few of the files I'm generating automatically when you run the scaffold command. Try it out yourself sometime!
const typeMapping = {\n string: 'string',\n integer: 'number',\n boolean: 'boolean',\n date: 'Date',\n text: 'string',\n // TODO: Add other type mappings as needed\n}\nThese are the supported types for generating a model or scaffold as of today (May 29th, 2023) but I plan on adding more as I go along.
\nThe biggest missing feature right now that need to be developed:
\nrails generate model Assemblies_parts assembly:references part:referencesAs always feel free to connect with me on LinkedIn. I'd love to hear your feedback on this project and if you have any ideas for how to make it better.
\nThanks for reading!
", "url": "https://www.groff.dev/blog/next-rails-cli", "title": "Bridging the Gap Between Next.js and Rails: Introducing the next-rails npm Package", "summary": "In this post, I introduce next-rails, a new npm package I developed to bring the efficiency of Rails-like features to Next.js, aimed at improving your full-stack development experience.", "date_modified": "2023-05-29T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/how-to-run-an-ai-scraping-hub-for-under-10-dollars-a-month", "content_html": "Running a web scraping service usually requires a substantial investment in infrastructure and computing power. However, with the right tech stack and a little ingenuity, you can create a robust AI scraping hub for under $10 a month. This post will guide you through the setup and use of an affordable scraping hub, based on my project on Github.
\nIf you're curious about why I chose to use Cloudflare Workers for scraping and want to delve into the code specifics and the restrictions, I recommend checking out my previous article on Scraping with Cloudflare Workers and OpenAI. In that article, I thoroughly discuss the rationale and advantages/disadvantages of this particular setup.
\nThis scraping hub project leverages a few key technologies: a Node.js Scheduler Service, two Cloudflare Workers, and a combination of Neon PostgreSQL, Redis via Upstash, and Render.com for deployment. Here's a quick rundown of how each component fits in:
\nNode.js Scheduler Service: This application polls the Neon PostgreSQL database every 15 minutes, retrieving the latest cron schedule from the scraping_job table. The service is deployed on Render.com.
Cloudflare Worker 1 - Scraper: As the workhorse of the hub, this worker is responsible for executing the web scraping tasks.
\nCloudflare Worker 2 - Apollo GraphQL Server: This worker provides a GraphQL API that allows clients to create, read, and update scraping cron jobs.
\nNeon PostgreSQL: A serverless PostgreSQL database where scraping job schedules and historical job results are stored.
\nRedis with Upstash.com: Used for caching the scraped data.
\nHere is a simplified diagram of how these services interact:
\nCloudflare Worker 1 (Scraper) <-------- Node.js Scheduler Service -------> Neon PostgreSQL\n ^ ^ |\n | | v\n | | Redis (Cache)\n | |\nCloudflare Worker 2 (GraphQL Server) <----|\nInitially, I considered using Rails with Sidekiq to create the UI dashboard and scheduling service. However, deploying Rails and Sidekiq turned out to be more complicated than anticipated, making the solution unattractive.
\nI also tried using Bun as a task scheduler, but it wasn't compatible with Neon Postgres or Node-Cron. Therefore, I settled on a simple Node Express server with a very basic dashboard for visualizing the jobs, their schedules, and the results of the last successful scrape.
\nHere's a preview of what the scheduler dashboard looks like:
\nBun Scheduler Dashboard\nUptime: 277 seconds\nJob ID\tURL\tSelector\tDescription\tLatest Content\tCron Schedule\tCron Schedule Described\nfcb4fb7e-f20c-4486-b0c7-568d948257e4\thttps://www.somecreditunion.org/loans-cards/mortgage/mortgage-rates/conventional-fixed-rate-mortgages.html\ttable, thead, tbody, tr, th, td\tThere is a Rates tables which has the headers: 'Term' 'Interest Rates As Low As' 'Discount Points' 'APR As Low As' please fetch them in the following format: 'term' 'rate' 'discount_points' and 'apr'. The key for the json array should be 'rates'\t❌\t0 7 * * *\tAt 07:00 AM US Eastern\nThe scheduler-service application can be spun up using Docker Compose. In the root directory of the project, you will find a docker-compose.yml file. Run the command docker-compose up in your terminal to start the service, and ensure Docker is installed on your machine.
More specific instructions on how to use the individual Cloudflare workers are available in their respective README.md files in the worker directories. You can use the wrangler CLI tool to deploy the workers to your Cloudflare account.
You might be surprised to learn that this entire setup, with its multiple services and advanced capabilities, costs less than $10 per month. Here's the cost breakdown:
\n| Service | Cost |
|---|---|
| Render.com | $7/mo |
| Cloudflare Workers | Free tier |
| Redis (Upstash.com) | Free tier |
| Neon Serverless PostgreSQL | Free tier |
| Total | $7/mo |
Yes, you read that right. For the price of a fancy coffee, you can have a powerful web scraping hub running round the clock. Coupled with OpenAI's reasonable costs, this solution remains well below the $10 a month mark. Get that data.
\nIf you're in need of a simple and cost-effective web scraping solution, this project might be the right fit for you. Check out the Github repository to get started. Embrace the world of affordable, efficient, and powerful web scraping, all for under $10 a month. Happy scraping!
\nI'd love to hear your thoughts on LinkedIn.
", "url": "https://www.groff.dev/blog/how-to-run-an-ai-scraping-hub-for-under-10-dollars-a-month", "title": "How to Run an AI Scraping Hub for Under $10 a Month", "summary": "By taking full advantage of generous free tiers of cloud services, you can run a scraping hub for under $10 a month.", "date_modified": "2023-05-27T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/scraping-in-cloudflare-workers-with-openai", "content_html": "Cloudflare Workers is a powerful service that runs on Cloudflare's edge runtime. This is very similar to something like AWS Lambda, however it is much more lightweight and has a much faster cold start time. It's also considerably cheaper.
\n| Plan Type | Requests | Duration |
|---|---|---|
| Free Plan | 100,000 / day | 10ms CPU time / invocation |
| Paid Plan - Bundled | 10 million / month, +$0.50/million | 50 ms CPU time / invocation |
| Paid Plan - Unbound | 1 million / month, + $0.15/million | 400,000 GB-s, + $12.50/million GB-s2,3 |
| Cloudflare Workers Pricing |
You only get charged for number of requests and duration of the request. You don't get charged for memory usage or bandwidth. CPU time is important here. You do NOT pay for time waiting on fetch API requests. You only pay for time spent processing the request. In my example today, while we're waiting for the scrape to resolve and while we're waiting for OpenAI to resolve we are not being charged a penny. That's awesome!
\nWatch this video from Theo - t3.gg on YouTube "That's It, I'm Done With Serverless." for a great explanation of Edge vs Serverless. tldr; Edge is faster and cheaper, but much more limited.
\nOne of the limitations of the Cloudflare edge runtime is that you can't bring in just any old dependency from NPM. Cloudflare has a list of Works on Workers if you want to bring in an external dependency.
\nUnfortunately you can't use typical Node scraping packages like cheerio or puppeteer on Cloudflare Workers. However, you can use plain old fetch, the one built into the browser and Cloudflare's edge runtime.
I followed an example project from Cloudflare's website called Web Scraper it was written by Adam Schwartz and you can find the GitHub repo here.
\nThis got me pretty far, it's output was a JSON object with the scraped text data. I wanted to take it a step further and use OpenAI's API to generate a JSON Array of the scraped response. The response should be trusted and ready to be inserted into a Redis cache or consumed by a frontend client.
\nHere's what I did:
\nwrangler init scraper\nenv variables to the wrangler.toml filename = "scraper"\nmain = "src/worker.ts"\ncompatibility_date = "2023-05-25"\n\n[vars]\nOPEN_AI_API_KEY = "your-open-ai-api-key"\nCORRECT_API_KEY = "your-correct-api-key"\nI added a "CORRECT_API_KEY", although in hindsight it's probably better named as a SECRET_KEY or something similar. This is just a simple way to add a layer of security to the endpoint.
\nexport default {\n async fetch(request: Request, env: Env): Promise<Response> {\n // We only want to accept a POST request.\n if (request.method !== 'POST') {\n return new Response('Method not allowed', { status: 405 });\n }\n\n // Verify the request has the CORRECT_API_KEY, so if someone tries to call your endpoint and they don't know your secret they will get a 401 Unauthorized response\n const authHeader = request.headers.get('Authorization');\n if (!authHeader || authHeader !== `Bearer ${env.CORRECT_API_KEY}`) {\n return new Response('Unauthorized', { status: 401 });\n }\n\n // TS interface for the POST body. We want an href, description and selector\n interface FetchData {\n href: string;\n description: string;\n selector: string;\n }\n\n // Validate all of the required fields are present in the POST body\n const body: FetchData = (await request.json()) as FetchData;\n if (!body.href || !body.description || !body.selector) {\n return new Response('Bad request. Please include an href, selector, and description in the POST body.', { status: 400 });\n }\n\n // Fetch the HTML content from the provided href that we wish to scrape\n const response = await fetch(body.href);\n if (!response.ok) {\n return new Response(`Failed to fetch ${body.href}`, { status: 404 });\n }\n\n // Initialize a new scraper and pass the fetched response to it\n const scraper = new Scraper();\n await scraper.fetch(body.href);\n\n // Use the scraper to get the text content of the specified elements\n const textContent = await scraper.querySelector(body.selector).getText({ spaced: false });\n\n // Serialize the text content to JSON\n const jsonData = JSON.stringify(textContent);\n\n try {\n // Send the JSON data to OpenAI's API\n const result = await sendToOpenAI(env, jsonData, body.href, body.description);\n\n // Return the result from OpenAI's API\n return new Response(JSON.stringify(result), { status: 200 });\n } catch (err: any) {\n console.error(err);\n\n // Do some error handling. If the error message includes "maximum context length is" then the JSON data provided was too large for OpenAI to process.\n if (err?.message?.includes('maximum context length is')) {\n return new Response('The HTML provided is too large for OpenAI to process.', { status: 413 });\n } else {\n return new Response(err?.message ? err.message : "OpenAI's API had a problem.", { status: 500 });\n }\n }\n },\n};\nFeel free to read the code comments, but basically I'm validating the request to my Cloudflare worker, scraping the text content from the provided href and selector, serializing the text content to JSON, sending the JSON data to OpenAI's API, and returning the result from OpenAI's API.
\nTo see how I'm crafting my prompt to send to OpenAI's API, check out the sendToOpenAI function below.
async function sendToOpenAI(env: Env, rawHtml: string, href: string, description: string) {\n const body = {\n model: 'gpt-3.5-turbo',\n messages: [\n {\n role: 'system',\n content: `You are a web scraper being asked to get content from ${href}. You have been provided with the scraped text values of certain DOM Element Tags. Your task is to parse these and extract the requested data as described. The format in your response should strictly be a JSON Array of the requested data with no additional text or formatting. If it's a singular item, it will be a JSON Array of 1 length. Here is a description of how to locate the content that we want from the site: ${description}. Treat this like you are an API endpoint, just return the JSON.`,\n },\n {\n role: 'user',\n content: `Parse the following scraped text values and locate the content as per the description provided: ${rawHtml}. Remember, no explanation. Just JSON.`,\n },\n ],\n temperature: 0.2,\n };\n\n const response = await fetch('https://api.openai.com/v1/chat/completions', {\n method: 'POST',\n headers: {\n Authorization: `Bearer ${env.OPEN_AI_API_KEY}`,\n 'Content-Type': 'application/json',\n },\n body: JSON.stringify(body),\n });\n\n if (!response.ok) {\n const errorMessage = await response.text();\n throw new Error(`OpenAI API error: ${errorMessage}`);\n }\n\n interface OpenAIChoice {\n message: {\n content: string;\n };\n }\n\n interface OpenAIResponse {\n choices: OpenAIChoice[];\n }\n\n const data: OpenAIResponse = await response.json();\n return data['choices'][0]['message']['content'];\n}\nThe key aspect is in the system and user prompts. I'm passing in the href for added context of what task it is assigned and what the mission objective is.
\nI am using the Chat Completions API, but you could also use the standard Text Completions API which would work in a similar manner. I like the idea of a System and User, and I've had success using Chat Completions, even though this is not a conversational task and there is no back and forth.
\nThe temperature is set to 0.2, which is a low value. This means the response will be more predictable and less random. I've found this to be a good value for the types of code generation/JSON work I'm doing. ChatGPT's default is 0.7 which is much more creative.
I'm using gpt-3.5-turbo which is faster and cheaper than gpt-4. I've found it to be sufficient for these rather "basic" text transformation tasks that are well defined.
curl --location 'https://scraper.yourcloudflareusername.workers.dev' \\\n--header 'Content-Type: application/json' \\\n--header 'Authorization: Bearer your-correct-api-key' \\\n--data '{\n "href": "https://website.com/page-to-scrape",\n "selector": "table, thead, tbody, tr, th, td",\n "description": "There is a Rates tables which has the headers: '\\''Term'\\'' '\\''Interest Rates As Low As'\\'' '\\''Discount Points'\\'' '\\''APR As Low As'\\'' please fetch them in the following format: '\\''term'\\'' '\\''rate'\\'' '\\''discount_points'\\'' and '\\''apr'\\''. The key for the json array should be '\\''rates'\\''"\n}'\nIn the href you send the url of the page you want to scrape.
In the selector you can use css selectors to choose which elements to scrape.
In the description, you can describe the data you want to scrape. This is used to help the AI understand what you want to scrape. The more detailed you are, the better the results will be. The default response is a JSON array, even if the result is a single value. Tell it what keys and values you want in your description.
As always you can find the source code for my projects on GitHub.
\nI think there are a lot of interesting use cases for this. In the future I may build upon this to create a Scraping Hub style platform where you can schedule these requests on as a cronjob to repeat however often you need and save historical results.
\nI'd love to hear your thoughts on LinkedIn.
", "url": "https://www.groff.dev/blog/scraping-in-cloudflare-workers-with-openai", "title": "Scraping with Cloudflare Workers and OpenAI", "summary": "How and why I made a scraper with Cloudflare Workers and OpenAI's API", "date_modified": "2023-05-25T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/scraping-in-nextjs", "content_html": "Next.js is a powerful React framework that allows you to build full-stack application. It has support for both React Client and Server components. You can get away with some interestings things including scraping web content, because basically anything you can do in a Node server you can do in a Next.js React Server Component, within reason. If you deploy your Next.js app to Vercel for instance, it uses a Lambda runtime which isn't quite 1-1 with Node.js.
\nI was working on a personal tool that needed to fetch the latest interest rate for home loans with a credit union. At first I was using puppetteer but it unfortunately won't work on Vercel and here's why:
\npuppetteer-core and chrome-aws-lambda but it's not going to work unfortunately because Vercel has a file size limit of 50MB and with this Chromium Lambda installed it just barely went over at 53MB. "The Serverless Function "index" is 53.95mb which exceeds the maximum size limit of 50mb."So wait, does that mean we can't actually scrape in Next.js?
\nWell, kind of!
\nIf you need puppetteer you can use it locally because Next.js runs on the Node V8 runtime when you are working locally off of npm run dev or npm run start. You might be able to get away with using Next.js with another provider who doesn't have the 50MB restrictions but I'm not aware of one.
Example code for scraping with puppetteer-core and chrome-aws-lambda, that would have worked if Chromium was 4MB smaller:
import chromium from 'chrome-aws-lambda';\n\ninterface CreditRange {\n value: string;\n min: string;\n max: string;\n rate?: string;\n}\n\n// This function is not actually called but I spent a lot of time making it work so I'm not removing it.\nexport async function scrapeNerdWallet(): Promise<CreditRange[]> {\n try {\n const browser = await chromium.puppeteer.launch({\n args: chromium.args,\n defaultViewport: chromium.defaultViewport,\n executablePath: await chromium.executablePath,\n headless: chromium.headless,\n ignoreHTTPSErrors: true,\n });\n const page = await browser.newPage();\n await page.setViewport({ width: 1000, height: 800 }); // Set viewport\n await page.goto('https://www.nerdwallet.com/mortgages/how-much-house-can-i-afford/calculate-affordability');\n\n await new Promise(resolve => setTimeout(resolve, 3000)); // Wait 3 seconds for any modals\n\n // Attempt to close modal if it exists\n if ((await page.$('[aria-label="Close dialog"]')) !== null) {\n await page.click('[aria-label="Close dialog"]');\n }\n\n // Press Escape key to close any possible pop-ups\n await page.keyboard.press('Escape');\n await new Promise(resolve => setTimeout(resolve, 3000)); // Wait another 3 seconds for any changes\n\n const options: string[] = await page.$$eval('.interest-rate-control-credit-score-select-option', options =>\n options.map(option => option.querySelector('.interest-rate-control-credit-score-select-option-value')?.textContent || '')\n );\n\n const categories: CreditRange[] = [];\n for (let i = 0; i < options.length; i++) {\n await page.click(`.interest-rate-control-credit-score-select-option:nth-child(${i + 1})`);\n await new Promise(r => setTimeout(r, 2000)); // Wait for the page to update\n\n // Check the specific elements of the page after each click\n const min = await page.$eval('.interest-rate-control-credit-score-select-option-min', el => el.textContent || '');\n const max = await page.$eval('.interest-rate-control-credit-score-select-option-max', el => el.textContent || '');\n const rate = await page.$eval('.interest-rate-control-method h3 span', span => span.textContent);\n categories.push({\n value: options[i],\n min: min,\n max: max,\n rate: parseFloat(rate?.replace('%', '') || '0').toFixed(3),\n });\n }\n\n await browser.close();\n return categories;\n } catch (err) {\n console.error(err);\n return [];\n }\n}\nIn this example I needed puppetteer because the content was hiding behind a pop-up modal that I needed to close as well as having to click through to get all of the interested rates for the credit score ranges.
\nBut if you don't need puppetteer, you can still use a React Server Component to scrape web content.
\nWe're going to use cheerio which is a simple HTML scraper with a jQuery-like syntax.
In my pages/index.tsx file I have this getServerSideProps function. I use getServerSideProps and not getStaticProps because I want the content to be dynamically rendered at runtime rather than build time.
export async function getServerSideProps() {\n const redis = process.env.REDIS_URL ? new Redis(process.env.REDIS_URL) : null;\n\n const interestRatesFromNavyFederal = await fetchInterestRates(redis);\n\n return { props: { interestRatesFromNavyFederal } };\n}\nThis calls a function called fetchInterestRates which I put in a separate file to prevent the page component from getting any longer than it already was.
import Redis from 'ioredis';\nimport { fetchNavyFederal } from './navy-federal';\n\nexport async function fetchInterestRates(redis: Redis | null) {\n try {\n if (redis) {\n let cachedData: any = await redis.get('navyFederalInterestRates');\n if (cachedData) {\n const data = JSON.parse(cachedData);\n\n if (data.length > 0) {\n return data;\n }\n }\n\n const freshData = await fetchNavyFederal();\n await redis.set('navyFederalInterestRates', JSON.stringify(freshData), 'EX', 1 * 60 * 60);\n\n return freshData;\n }\n } catch (err) {\n console.error(err);\n return {\n source: 'error',\n data: [],\n };\n }\n}\nI'm using Redis to cache the data for an hour so that I don't have to scrape the data every time someone visits the page. I'm using the ioredis package to connect to Redis. I'm using the free tier of Redis from Upstash.
Again this file imports another file where I actually do the scraping.
\nimport { InterestRate } from '@deps/types';\nimport { load } from 'cheerio';\n\nexport async function fetchNavyFederal(): Promise<InterestRate[]> {\n try {\n const response = await fetch(\n 'https://www.navyfederal.org/loans-cards/mortgage/mortgage-rates/conventional-fixed-rate-mortgages.html'\n );\n const body = await response.text();\n\n const $ = load(body);\n const loanTerms: InterestRate[] = [];\n\n $('.table-resp tbody tr').each((_, row) => {\n const columns = $(row).find('td');\n const term = $(row).find('th').text().trim();\n const rate = parseFloat(columns.eq(0).text().replace('%', ''));\n const discountPoints = parseFloat(columns.eq(1).text());\n\n loanTerms.push({\n term: term,\n rate: rate,\n discountPoints: discountPoints,\n rateWithoutDiscount: rate + discountPoints * 0.25, // Assuming each discount point reduces the rate by 0.25%\n });\n });\n\n return loanTerms;\n } catch (err) {\n console.error(err);\n return [];\n }\n}\nThis is a pretty great example of an easy-to-scrape site. The data is in a table and the table has a very specific class name. I can use cheerio to load the HTML and then use jQuery-like syntax to find the table and then loop through each row and column to get the data I need.
\nThe JSON content I end up deliverying to my pages/index.tsx and my Redis cache looks like:
[ \n { "term": "15 Year", "rate": 5.5, "discountPoints": 0.25, "rateWithoutDiscount": 5.5625 }, \n { "term": "15 Year Jumbo", "rate": 5.75, "discountPoints": 0.25, "rateWithoutDiscount": 5.8125 }, \n { "term": "30 Year", "rate": 6, "discountPoints": 0.5, "rateWithoutDiscount": 6.125 }, \n { "term": "30 Year Jumbo", "rate": 6, "discountPoints": 0.5, "rateWithoutDiscount": 6.125 } \n]\nThis credit union updates their rates daily so I'm fine with caching the data for an hour. If you're scraping a site that updates less frequently, you can cache the data for longer.
\nThe idea of scraping web content with a React Server Component is pretty exciting. It's a great way to get data from a site that doesn't have an API. It's also a great way to get data from a site that does have an API but the API is either too complicated or too slow. With Next.js you "almost" don't need a backend. I was very excited to get puppetteer working locally only to be dissapointed that it wouldn't run on Vercel, that would have been too cool. But I'm still very happy with the results.
", "url": "https://www.groff.dev/blog/scraping-in-nextjs", "title": "Scraping with Next.js and React Server Components", "summary": "How I made a web scraper using a React Server Component in a Next.js app.", "date_modified": "2023-05-23T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/creating-a-live-estimation-app-with-chatgpt-part-2", "content_html": "If you haven't yet please read Creating a Live Estimation App with ChatGPT - Part 1
\nSince the last post this I've created a Next.js app with Typescript, Tailwind, Pages directory, and started making some of the files suggested.
\nThe sql query it gave my for the create table room had a non-null requirement on lead_name, but the room would be generated before the lead_name was known. I changed the query to be nullable here:
\nALTER TABLE rooms ALTER COLUMN lead_name DROP NOT NULL;\nHere is my /lib/supabase.ts file, since I'm using TypeScript:
import { createClient } from '@supabase/supabase-js';\n\nconst SUPABASE_URL = process.env.SUPABASE_URL;\nconst SUPABASE_ANON_KEY = process.env.SUPABASE_ANON_KEY;\n\nif (!SUPABASE_URL || !SUPABASE_ANON_KEY) {\n throw new Error('Missing Supabase URL or anonymous key');\n}\n\nexport const supabase = createClient(SUPABASE_URL, SUPABASE_ANON_KEY);\nHere's my /api/join-room.ts file:
import type { NextApiRequest, NextApiResponse } from 'next';\nimport { supabase } from '@deps/lib/supabase';\n\ninterface JoinRoomResponse {\n success?: boolean;\n error?: string;\n is_lead?: boolean;\n}\n\nexport default async function handler(req: NextApiRequest, res: NextApiResponse<JoinRoomResponse>) {\n if (req.method !== 'POST') {\n return res.status(405).json({ error: 'Method not allowed.' });\n }\n\n const { name, room_id } = req.body;\n\n try {\n // Check if user already exists\n const { data: existingUser, error: existingUserError } = await supabase\n .from('users')\n .select('name')\n .eq('room_id', room_id)\n .eq('name', name);\n\n if (existingUserError) {\n throw existingUserError;\n }\n\n // If user does not exist, create a new one\n if (!existingUser?.length) {\n const { error: insertError } = await supabase.from('users').insert([{ name, room_id }]);\n\n if (insertError) {\n throw insertError;\n }\n }\n\n const { data: room, error: roomError } = await supabase.from('rooms').select('lead_name').eq('id', room_id);\n\n if (roomError) {\n throw roomError;\n }\n\n const is_lead = room?.[0]?.lead_name === name;\n\n return res.status(200).json({ success: true, is_lead });\n } catch (error) {\n if (error instanceof Error) {\n // Handle error\n return res.status(500).json({ error: error.message });\n }\n return res.status(500).json({ error: 'An error occurred when joining the room.' });\n }\n}\n\nI added logic to return if you are the lead or not. I also updated the getServerSideProps:
export async function getServerSideProps(context: GetServerSidePropsContext) {\n const { join_code } = context.params!; // Use "!" to assert that context.params is not undefined\n\n if (!join_code) {\n return {\n props: {\n error: 'Sorry this join code is invalid',\n },\n };\n }\n\n // Get current time and time 24 hours ago\n const now = new Date();\n const oneDayAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000);\n\n const { data: rooms } = await supabase.from('room').select('*').eq('join_code', join_code).gte('created_at', oneDayAgo);\n\n if (!rooms || rooms.length === 0) {\n return {\n props: {\n error: 'Sorry this join code is invalid',\n },\n };\n }\n\n const room = rooms[0];\n\n const { data: users } = await supabase.from('users').select('*').eq('room_id', room.id);\n\n return {\n props: { room, initialUsers: users },\n };\n}\nNext, I wanted to create a new API endpoint to create a room. Notice how I prompt ChatGPT with an example of the previous code.
\n...
\nUser:\nLet's make a create-room api call.
\nimport type { NextApiRequest, NextApiResponse } from 'next';\nimport { supabase } from '@deps/lib/supabase';\n\ninterface JoinRoomResponse {\n success?: boolean;\n error?: string;\n is_lead?: boolean;\n}\n\nexport default async function handler(req: NextApiRequest, res: NextApiResponse<JoinRoomResponse>) {\n if (req.method !== 'POST') {\n return res.status(405).json({ error: 'Method not allowed.' });\n }\n\n const { name, room_id } = req.body;\n\n try {\n // Check if user already exists\n const { data: existingUser, error: existingUserError } = await supabase\n .from('users')\n .select('name')\n .eq('room_id', room_id)\n .eq('name', name);\n\n if (existingUserError) {\n throw existingUserError;\n }\n\n // If user does not exist, create a new one\n if (!existingUser?.length) {\n const { error: insertError } = await supabase.from('users').insert([{ name, room_id }]);\n\n if (insertError) {\n throw insertError;\n }\n }\n\n const { data: room, error: roomError } = await supabase.from('rooms').select('lead_name').eq('id', room_id);\n\n if (roomError) {\n throw roomError;\n }\n\n const is_lead = room?.[0]?.lead_name === name;\n\n return res.status(200).json({ success: true, is_lead });\n } catch (error) {\n if (error instanceof Error) {\n // Handle error\n return res.status(500).json({ error: error.message });\n }\n return res.status(500).json({ error: 'An error occurred when joining the room.' });\n }\n}\n...
\nSystem:\nThe create-room API call is responsible for creating a new room, and possibly the lead user for that room. We need to modify your existing join-room API route to match the requirements for the create-room route. We will also generate a unique join code for the room.
\nHere is the updated code:
\nimport type { NextApiRequest, NextApiResponse } from 'next';\nimport { supabase } from '@deps/lib/supabase';\n\ninterface CreateRoomResponse {\n success?: boolean;\n room?: {\n id: number;\n join_code: string;\n };\n error?: string;\n}\n\n// Function to generate a random 6-digit alphanumeric string\nconst generateJoinCode = () => Math.random().toString(36).substring(2, 8).toUpperCase();\n\nexport default async function handler(req: NextApiRequest, res: NextApiResponse<CreateRoomResponse>) {\n if (req.method !== 'POST') {\n return res.status(405).json({ error: 'Method not allowed.' });\n }\n\n const { lead_name } = req.body;\n\n try {\n const join_code = generateJoinCode();\n\n // Create a new room\n const { data: room, error: roomError } = await supabase\n .from('rooms')\n .insert([{ join_code, lead_name }])\n .single();\n\n if (roomError) {\n throw roomError;\n }\n\n return res.status(200).json({ success: true, room });\n } catch (error) {\n if (error instanceof Error) {\n // Handle error\n return res.status(500).json({ error: error.message });\n }\n return res.status(500).json({ error: 'An error occurred when creating the room.' });\n }\n}\nNote from Matt: ChatGPT did an ok job but did not handle the edge case of creating a unique join code for a 24 hour period. I ask it to fix this.
\n...
\nUser:\nWe need to check before creating that room, that the join code has been used in the last 24 hours.
\nAlso we can return the "Room" interface instead of the id and join_code
\nexport interface Room {\n id: number;\n join_code: string;\n lead_name: string | null;\n created_at: string;\n}\n...
\nSystem:\nSure, here's the updated code that checks if the join code has been used in the last 24 hours:
\nimport type { NextApiRequest, NextApiResponse } from 'next';\nimport { supabase } from '@deps/lib/supabase';\nimport { Room } from '@deps/types';\n\ninterface CreateRoomResponse {\n success?: boolean;\n room?: Room;\n error?: string;\n}\n\n// Function to generate a random 6-digit alphanumeric string\nconst generateJoinCode = () => Math.random().toString(36).substring(2, 8).toUpperCase();\n\nexport default async function handler(req: NextApiRequest, res: NextApiResponse<CreateRoomResponse>) {\n if (req.method !== 'POST') {\n return res.status(405).json({ error: 'Method not allowed.' });\n }\n\n const { lead_name } = req.body;\n\n try {\n const join_code = generateJoinCode();\n\n // Get current time and time 24 hours ago\n const now = new Date();\n const oneDayAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000);\n\n // Check if the join code has been used in the last 24 hours\n const { data: existingRooms } = await supabase\n .from<Room>('rooms')\n .select('*')\n .eq('join_code', join_code)\n .gte('created_at', oneDayAgo);\n\n if (existingRooms && existingRooms.length > 0) {\n return res.status(400).json({ error: 'Join code has been used recently. Please try again.' });\n }\n\n // Create a new room\n const { data: room, error: roomError } = await supabase\n .from<Room>('rooms')\n .insert([{ join_code, lead_name }])\n .single();\n\n if (roomError) {\n throw roomError;\n }\n\n return res.status(200).json({ success: true, room });\n } catch (error) {\n if (error instanceof Error) {\n // Handle error\n return res.status(500).json({ error: error.message });\n }\n return res.status(500).json({ error: 'An error occurred when creating the room.' });\n }\n}\nThis code tries to generate a unique join code and checks if it has been used in the last 24 hours. If it has, it returns an error. If it hasn't, it creates the room and returns the room details.
\nimport type { NextApiRequest, NextApiResponse } from 'next';\nimport { supabase } from '@deps/lib/supabase';\nimport { Room } from '@deps/types';\n\ninterface CreateRoomResponse {\n success?: boolean;\n room?: Room;\n error?: string;\n}\n\n// Function to generate a random 6-digit alphanumeric string\nconst generateJoinCode = () => Math.random().toString(36).substring(2, 8).toUpperCase();\n\nexport default async function handler(req: NextApiRequest, res: NextApiResponse<CreateRoomResponse>) {\n if (req.method !== 'POST') {\n return res.status(405).json({ error: 'Method not allowed.' });\n }\n\n const { lead_name } = req.body;\n\n try {\n const join_code = generateJoinCode();\n\n // Get current time and time 24 hours ago\n const now = new Date();\n const oneDayAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000);\n\n // Check if the join code has been used in the last 24 hours\n const { data: existingRooms } = await supabase\n .from<Room>('rooms')\n .select('*')\n .eq('join_code', join_code)\n .gte('created_at', oneDayAgo);\n\n if (existingRooms && existingRooms.length > 0) {\n return res.status(400).json({ error: 'Join code has been used recently. Please try again.' });\n }\n\n // Create a new room\n const { data: room, error: roomError } = await supabase\n .from<Room>('rooms')\n .insert([{ join_code, lead_name }])\n .single();\n\n if (roomError) {\n throw roomError;\n }\n\n return res.status(200).json({ success: true, room });\n } catch (error) {\n if (error instanceof Error) {\n // Handle error\n return res.status(500).json({ error: error.message });\n }\n return res.status(500).json({ error: 'An error occurred when creating the room.' });\n }\n}\n\nThis code tries to generate a unique join code and checks if it has been used in the last 24 hours. If it has, it returns an error. If it hasn't, it creates the room and returns the room details.
\nNote from Matt: This was better, but it still didn't handle if the second code wasn't unique and so on.
\n...
\nUser:\nInstead of returning an error if the code has been used, have it create a new code, and check it again.
\nWe could use recursion, that would be fun!
\n...
\nSystem:\nRecursion is a great idea! We can create a new function getJoinCode that generates a join code and checks if it has been used in the last 24 hours. If it has, it will call itself again recursively to get a new code.
\nHere's how you can update your function:
\nimport type { NextApiRequest, NextApiResponse } from 'next';\nimport { supabase } from '@deps/lib/supabase';\nimport { Room } from '@deps/types';\n\ninterface CreateRoomResponse {\n success?: boolean;\n room?: Room;\n error?: string;\n}\n\n// Function to generate a random 6-digit alphanumeric string\nconst generateJoinCode = () => Math.random().toString(36).substring(2, 8).toUpperCase();\n\nconst getJoinCode = async (): Promise<string> => {\n const join_code = generateJoinCode();\n\n // Get current time and time 24 hours ago\n const now = new Date();\n const oneDayAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000);\n\n // Check if the join code has been used in the last 24 hours\n const { data: existingRooms } = await supabase\n .from<Room>('rooms')\n .select('*')\n .eq('join_code', join_code)\n .gte('created_at', oneDayAgo);\n\n if (existingRooms && existingRooms.length > 0) {\n // If join code has been used, generate a new one\n return getJoinCode();\n } else {\n // If join code has not been used, return it\n return join_code;\n }\n}\n\nexport default async function handler(req: NextApiRequest, res: NextApiResponse<CreateRoomResponse>) {\n if (req.method !== 'POST') {\n return res.status(405).json({ error: 'Method not allowed.' });\n }\n\n const { lead_name } = req.body;\n\n try {\n const join_code = await getJoinCode();\n\n // Create a new room\n const { data: room, error: roomError } = await supabase\n .from<Room>('rooms')\n .insert([{ join_code, lead_name }])\n .single();\n\n if (roomError) {\n throw roomError;\n }\n\n return res.status(200).json({ success: true, room });\n } catch (error) {\n if (error instanceof Error) {\n // Handle error\n return res.status(500).json({ error: error.message });\n }\n return res.status(500).json({ error: 'An error occurred when creating the room.' });\n }\n}\n...
\nThat's much better! It took some convincing. This is a great example of why you can't just create an application with ChatGPT with no experience as a developer. You need the expertise to understand when its wrong and why.
\nThe only thing I'm going to change is that we are not going to send a lead_name in the body.
Now let's add a homepage. I'm going to send it my existing Next.js default index page so it gets a better idea of what I want.
\n...
\nUser:\nLet's make this homepage branded like "Estimation Party" and have a button to "Create a new room" which calls the /api/create-room endpoint and then redirects to that room.
\nIt should have have a "Got a join code?" input that lets you here a 6 char alphanumeric join code (.toUppercase) and if valid submitting should take you to /room/<join_code> with router.push
import Image from 'next/image'\nimport { Inter } from 'next/font/google'\n\nconst inter = Inter({ subsets: ['latin'] })\n\nexport default function Home() {\n return (\n <main\n className={`flex min-h-screen flex-col items-center justify-between p-24 ${inter.className}`}\n >\n <div className="z-10 w-full max-w-5xl items-center justify-between font-mono text-sm lg:flex">\n <p className="fixed left-0 top-0 flex w-full justify-center border-b border-gray-300 bg-gradient-to-b from-zinc-200 pb-6 pt-8 backdrop-blur-2xl dark:border-neutral-800 dark:bg-zinc-800/30 dark:from-inherit lg:static lg:w-auto lg:rounded-xl lg:border lg:bg-gray-200 lg:p-4 lg:dark:bg-zinc-800/30">\n Get started by editing \n <code className="font-mono font-bold">src/pages/index.tsx</code>\n </p>\n <div className="fixed bottom-0 left-0 flex h-48 w-full items-end justify-center bg-gradient-to-t from-white via-white dark:from-black dark:via-black lg:static lg:h-auto lg:w-auto lg:bg-none">\n <a\n className="pointer-events-none flex place-items-center gap-2 p-8 lg:pointer-events-auto lg:p-0"\n href="https://vercel.com?utm_source=create-next-app&utm_medium=default-template-tw&utm_campaign=create-next-app"\n target="_blank"\n rel="noopener noreferrer"\n >\n By{' '}\n <Image\n src="/vercel.svg"\n alt="Vercel Logo"\n className="dark:invert"\n width={100}\n height={24}\n priority\n />\n </a>\n </div>\n </div>\n\n <div className="relative flex place-items-center before:absolute before:h-[300px] before:w-[480px] before:-translate-x-1/2 before:rounded-full before:bg-gradient-radial before:from-white before:to-transparent before:blur-2xl before:content-[''] after:absolute after:-z-20 after:h-[180px] after:w-[240px] after:translate-x-1/3 after:bg-gradient-conic after:from-sky-200 after:via-blue-200 after:blur-2xl after:content-[''] before:dark:bg-gradient-to-br before:dark:from-transparent before:dark:to-blue-700/10 after:dark:from-sky-900 after:dark:via-[#0141ff]/40 before:lg:h-[360px]">\n <Image\n className="relative dark:drop-shadow-[0_0_0.3rem_#ffffff70] dark:invert"\n src="/next.svg"\n alt="Next.js Logo"\n width={180}\n height={37}\n priority\n />\n </div>\n\n <div className="mb-32 grid text-center lg:mb-0 lg:grid-cols-4 lg:text-left">\n <a\n href="https://nextjs.org/docs?utm_source=create-next-app&utm_medium=default-template-tw&utm_campaign=create-next-app"\n className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800/30"\n target="_blank"\n rel="noopener noreferrer"\n >\n <h2 className={`mb-3 text-2xl font-semibold`}>\n Docs{' '}\n <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">\n ->\n </span>\n </h2>\n <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>\n Find in-depth information about Next.js features and API.\n </p>\n </a>\n\n <a\n href="https://nextjs.org/learn?utm_source=create-next-app&utm_medium=default-template-tw&utm_campaign=create-next-app"\n className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800/30"\n target="_blank"\n rel="noopener noreferrer"\n >\n <h2 className={`mb-3 text-2xl font-semibold`}>\n Learn{' '}\n <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">\n ->\n </span>\n </h2>\n <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>\n Learn about Next.js in an interactive course with quizzes!\n </p>\n </a>\n\n <a\n href="https://vercel.com/templates?framework=next.js&utm_source=create-next-app&utm_medium=default-template-tw&utm_campaign=create-next-app"\n className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800/30"\n target="_blank"\n rel="noopener noreferrer"\n >\n <h2 className={`mb-3 text-2xl font-semibold`}>\n Templates{' '}\n <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">\n ->\n </span>\n </h2>\n <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>\n Discover and deploy boilerplate example Next.js projects.\n </p>\n </a>\n\n <a\n href="https://vercel.com/new?utm_source=create-next-app&utm_medium=default-template-tw&utm_campaign=create-next-app"\n className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800/30"\n target="_blank"\n rel="noopener noreferrer"\n >\n <h2 className={`mb-3 text-2xl font-semibold`}>\n Deploy{' '}\n <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">\n ->\n </span>\n </h2>\n <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>\n Instantly deploy your Next.js site to a shareable URL with Vercel.\n </p>\n </a>\n </div>\n </main>\n )\n}\n...
\nSystem:\nYou'll want to use the Next.js useRouter hook to handle the redirect, and fetch to make the API call. For the join code input, you can create a state variable to hold its value and update it on input change. Then, you can use the useRouter hook again to handle the redirect when the form is submitted.
\nHere's how you can do it:
\nimport { useState } from 'react';\nimport { useRouter } from 'next/router';\nimport Image from 'next/image'\n\nexport default function Home() {\n const router = useRouter();\n const [joinCode, setJoinCode] = useState('');\n\n const handleCreateRoom = async () => {\n const res = await fetch('/api/create-room', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json'\n },\n body: JSON.stringify({ lead_name: 'Your Lead Name' }) // You may want to handle lead_name differently\n });\n\n const data = await res.json();\n\n if (data.success) {\n router.push(`/room/${data.room.join_code}`);\n } else {\n // Handle error\n console.error(data.error);\n }\n };\n\n const handleJoinRoom = (event) => {\n event.preventDefault();\n router.push(`/room/${joinCode.toUpperCase()}`);\n };\n\n return (\n <main>\n <h1>Estimation Party</h1>\n\n <button onClick={handleCreateRoom}>\n Create a new room\n </button>\n\n <form onSubmit={handleJoinRoom}>\n <input\n type="text"\n value={joinCode}\n onChange={(event) => setJoinCode(event.target.value)}\n maxLength={6}\n placeholder="Enter join code"\n />\n <button type="submit">\n Join a room\n </button>\n </form>\n\n {/* Your existing code... */}\n </main>\n )\n}\nThis sets up a basic input and button for entering a join code and navigating to the corresponding room, as well as a button for creating a new room. Remember to replace 'Your Lead Name' with appropriate logic for setting the lead's name when creating a room.
\nPlease note that this is a simplistic implementation, and you might want to add more checks or error handling. For example, you might want to check whether a room with the entered join code actually exists before trying to navigate to it, and show an error message if it doesn't. Also, you may want to validate the join code input to ensure it's in the correct format.
\n...
\nWe struck out on the example. It didn't even use tailwind and suggested I use the existing code. I should have been more clear.
\nUser:\nCan you please follow Tailwind css class styles?
\n...
\nSystem:\nSure, here's how you can use Tailwind CSS to style the components:
\nimport { useState } from 'react';\nimport { useRouter } from 'next/router';\n\nexport default function Home() {\n const router = useRouter();\n const [joinCode, setJoinCode] = useState('');\n\n const handleCreateRoom = async () => {\n const res = await fetch('/api/create-room', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json'\n },\n body: JSON.stringify({ lead_name: 'Your Lead Name' }) // You may want to handle lead_name differently\n });\n\n const data = await res.json();\n\n if (data.success) {\n router.push(`/room/${data.room.join_code}`);\n } else {\n // Handle error\n console.error(data.error);\n }\n };\n\n const handleJoinRoom = (event) => {\n event.preventDefault();\n router.push(`/room/${joinCode.toUpperCase()}`);\n };\n\n return (\n <main className="flex flex-col items-center justify-center min-h-screen py-2">\n <h1 className="text-4xl font-bold mb-8">Estimation Party</h1>\n\n <button\n onClick={handleCreateRoom}\n className="px-4 py-2 font-bold text-white bg-blue-500 rounded hover:bg-blue-700"\n >\n Create a new room\n </button>\n\n <form onSubmit={handleJoinRoom} className="mt-8">\n <input\n type="text"\n value={joinCode}\n onChange={(event) => setJoinCode(event.target.value)}\n maxLength={6}\n placeholder="Enter join code"\n className="shadow appearance-none border rounded w-full py-2 px-3 text-gray-700 leading-tight focus:outline-none focus:shadow-outline"\n />\n <button\n type="submit"\n className="mt-4 px-4 py-2 font-bold text-white bg-blue-500 rounded hover:bg-blue-700 w-full"\n >\n Join a room\n </button>\n </form>\n </main>\n )\n}\n...
\nSo it gave us Tailwind classnames. Let's remove the body of the POST since we aren't putting a lead_name in the home page but waiting until they join a room.
It doesn't look too bad at all.
\n![\"Estimation](\"/images/photos/EstimationPartyHome.png\")
I got the application code pushed to (GitHub)[https://github.com/mattlgroff/estimation-party] deployed to Vercel @ (estimationparty.com)[https://estimationparty.com].
\nThat's it for Part 2! In Part 3, we'll add the ability to join a room and continue working on the room page.
", "url": "https://www.groff.dev/blog/creating-a-live-estimation-app-with-chatgpt-part-2", "title": "Creating a Live Estimation App with ChatGPT - Part 2", "summary": "Prompting ChatGPT to help me create a Live Estimation App with Next.js and Supabase.", "date_modified": "2023-05-21T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/creating-a-live-estimation-app-with-chatgpt", "content_html": "I'm going to work with ChatGPT 4 to help me build a live estimation app. I'll be using Next.js and Supabase. Below is my back and forth with ChatGPT. Labels are User (Me, Matt Groff) and System (ChatGPT).
\nAt work we have used some "free" live estimation systems in conjuction with Zoom or Google Meet, but they are not great. I want to build something that is easy to use and can be used for free. I also want to use this as a way to learn more about Next.js and Supabase.
\nSome of my co-workers have expressed intrests in seeing my prompts and how I use ChatGPT, so the main reason I'm writing this is to share how I use ChatGPT.
\nUser:\nLet's make a Next.js full-stack application (using Tailwind CSS) that can be used for estimating user stories as a group.
\nI want to be able to (without signup):\nCreate a new Estimation Party.
\nJoin an existing Estimation Party from a link or invite code (let's say 6 random letters/numbers all caps).
\nEstimation Parties should have a lifespan of 24 hours. If they live longer than that the invite code is invalided and could be recycled.
\nWhoever joins the room first gets Lead, who can Reveal Estimations and Estimate New Work Item (with a ticket name and optionally a URL to the ticket).
\nAnyone joining needs to put a name and the name must be unique to the room. (Matt, Matt G, etc).
\nWhen it's time to a Estimate New Work Item, anyone is allowed to provide a Fibbonaci sequence estimation 0, 1, 2, 3, 5, 8, 13, 21, etc or they can choose ? which means they have a clarifying question.
\nWhen the Lead selects Reveal Estimations, everyone can see the number (or ?) of the estimate and the person who made it.
\nCan we accomplish this with just Next.js (deployed to Vercel) or do we need to host a websocket service separately?
\n...
\nSystem:\nThis can be accomplished mostly with Next.js, but it will require a real-time data exchange mechanism like websockets, long polling, or server-sent events. While Next.js does not have built-in support for WebSockets, this doesn't necessarily mean you need to host a separate WebSocket service. Instead, you can use a real-time database or a service like Pusher, Socket.IO, or Firebase Realtime Database that handles the real-time data exchange for you.
\nFor simplicity, we will use Firebase Realtime Database in this case, which uses WebSockets behind the scenes and works well with Next.js applications. Firebase also provides data persistence, which is important for managing the Estimation Parties' lifespans.
\nHere's an outline of how you might structure your application:
\nProject setup: Setup your Next.js application with Tailwind CSS. This can be done easily using create-next-app with the Tailwind CSS starter template.
\nFirebase integration: Setup Firebase in your project. You can start with Firebase Realtime Database for managing your data in real-time.
\nEstimation Party creation: For creating a new estimation party, generate a unique 6 character code (can be alphanumeric and all caps). Use this as a key in your Firebase Realtime Database with its value being the details of the party (such as the timestamp of creation). The details should include the unique names of the people who have joined the party. The first one to join gets the "Lead" status.
\nJoining an Estimation Party: When someone wants to join an existing estimation party, they enter the unique code. The application checks with Firebase if a party with that code exists and if it's not older than 24 hours. If yes, they're asked to enter a unique name and are added to the party.
\nEstimation: When the Lead creates a new work item to estimate, they input the details and it's added to the Firebase database under that party's details. Everyone in the party can see this and can add their estimation in Fibonacci sequence or a "?" as a question. When everyone has estimated, the Lead can reveal the estimations.
\nRevealing Estimations: When the Lead clicks to reveal the estimations, the details of each person's estimation is shown. This can be achieved by fetching the data from Firebase.
\nParty Lifecycle: Firebase allows you to create Cloud Functions which are pieces of code that run in response to events in your Firebase app. You could create a cloud function that runs every hour, checks all the parties, and removes any that are more than 24 hours old, invalidating their code.
\nThis is just a rough outline and there are many details you'll need to work out based on your specific requirements. But this should give you a good idea of how to structure your application and how Firebase can help you manage real-time data in a Next.js application.
\n...
\nUser:\nCan we use Supabase Realtime instead of Firebase?
\n...
\nSystem:\nAbsolutely! Supabase's Realtime and database services can certainly be used as an alternative to Firebase. Supabase is an open-source Firebase alternative that provides key features such as a Postgres database, authentication and authorization, storage, and serverless functions.
\nHere's how the steps would look like with Supabase:
\nProject setup: Setup your Next.js application with Tailwind CSS using create-next-app.
\nSupabase integration: Set up Supabase in your project. You'll use Supabase's Realtime and Postgres database service to manage real-time data.
\nEstimation Party creation: For creating a new estimation party, generate a unique 6 character code (alphanumeric and all caps). This can be used as a unique key in your Supabase Database with its value being the details of the party (such as the timestamp of creation). The details should include the unique names of the people who have joined the party. The first one to join gets the "Lead" status.
\nJoining an Estimation Party: When someone wants to join an existing estimation party, they enter the unique code. The application checks with Supabase if a party with that code exists and if it's not older than 24 hours. If yes, they're asked to enter a unique name and are added to the party.
\nEstimation: When the Lead creates a new work item to estimate, they input the details and it's added to the Supabase database under that party's details. Everyone in the party can see this and can add their estimation in Fibonacci sequence or a "?" as a question. When everyone has estimated, the Lead can reveal the estimations.
\nRevealing Estimations: When the Lead clicks to reveal the estimations, the details of each person's estimation are shown. This can be achieved by fetching the data from Supabase.
\nParty Lifecycle: Supabase, at the time of my knowledge cutoff in September 2021, does not support serverless functions like Firebase Cloud Functions, so you'll need to handle the 24-hour lifespan of the estimation parties differently. You might consider using a CRON job running on a separate server or a service like EasyCron to regularly delete parties that are older than 24 hours.
\nAgain, this is just a rough outline and there are many details you'll need to work out based on your specific requirements, but this should give you a good idea of how to structure your application with Supabase. Supabase has solid real-time capabilities, which are crucial for this application, and they provide good compatibility with Next.js.
\n...
\nUser:\nCool let's get started. I made a project in Supabase and I have an API Key and Postgres connection config. How can I start adding schema and setting up the realtime estimation rooms with 24 hour expiration, and a unique "per 24 hours" 6 character uppercase alphanumeric code?
\n...
\nSystem:\nLet's start by setting up the database schema in Supabase.
\nCreate a new table named rooms: This will represent an estimation party or room. Here are the columns you should have:
\nid: This should be a text field that will store your 6 character code. Make sure to set this as a primary key to ensure uniqueness.
\nlead_name: This should be a text field to store the name of the person who joined the room first and will act as the lead.
\ncreated_at: This is a timestamp field that will store the creation time of the room. Supabase automatically generates this field when you create a new row.
\nCreate a new table named users: This will represent a user in a room. Here are the columns you should have:
\nid: This is an auto-incrementing integer and will be your primary key.
\nname: This is a text field to store the unique name of the person in the room.
\nroom_id: This is a foreign key to the id in the rooms table. This will link a user to a room.
\nCreate a new table named estimations: This will represent an estimation for a work item. Here are the columns you should have:
\nid: This is an auto-incrementing integer and will be your primary key.
\nwork_item_name: This is a text field to store the name of the work item.
\nwork_item_url: This is a text field to store the optional URL of the work item.
\nroom_id: This is a foreign key to the id in the rooms table. This will link an estimation to a room.
\nuser_id: This is a foreign key to the id in the users table. This will link an estimation to a user.
\nestimate: This is a text field to store the Fibonacci sequence estimation or a question mark.
\nThe above schema should be sufficient for your needs. You can create the tables and columns in the Supabase dashboard's table editor.
\nAs for the unique 6 character code and 24-hour expiration, here's a basic approach:
\nUnique Code: Whenever a room is created, generate a random 6 character uppercase alphanumeric string. Before saving the room in the database, check if a room with the same id already exists. If it does, generate a new code. You can use the Math.random().toString(36).substring(2, 8).toUpperCase() snippet in JavaScript to generate such a string.
\n24-hour Expiration: As for the 24-hour expiration, Supabase doesn't have built-in support for automatic deletion of rows based on conditions. Therefore, you will need to handle this on the client-side application. Whenever a user attempts to join a room, compare the current timestamp with the created_at timestamp of the room. If the difference is more than 24 hours, the room is considered expired and the user is not allowed to join.
\nThis should give you a good starting point for setting up your database schema and handling the unique code generation and room expiration. From here, you can start building your Next.js application to interact with this database and handle the various user actions.
\n...
\nUser:\nHere's the default create table query. Can you give me code samples for these?
\ncreate table table_name (\n id bigint generated by default as identity primary key,\n inserted_at timestamp with time zone default timezone('utc'::text, now()) not null,\n updated_at timestamp with time zone default timezone('utc'::text, now()) not null,\n data jsonb,\n name text\n);\n...
\nSystem:\nSure, here are the SQL statements to create the rooms, users, and estimations tables in your Supabase database:
\nCreating the rooms table:
\nCREATE TABLE rooms (\n id CHAR(6) PRIMARY KEY,\n lead_name TEXT NOT NULL,\n created_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL\n);\nCreating the users table:
\nCREATE TABLE users (\n id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,\n name TEXT NOT NULL,\n room_id CHAR(6) REFERENCES rooms(id),\n inserted_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL,\n updated_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL\n);\nCreating the estimations table:
\nCREATE TABLE estimations (\n id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,\n work_item_name TEXT NOT NULL,\n work_item_url TEXT,\n room_id CHAR(6) REFERENCES rooms(id),\n user_id BIGINT REFERENCES users(id),\n estimate TEXT,\n inserted_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL,\n updated_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL\n);\nPlease note that we are using CHAR(6) for id in rooms table and room_id in users and estimations tables, as we're creating a fixed length 6 character code.
\nAlso, these queries make sure that every room has at least one user (the lead) and every estimation is linked to a user and a room.
\nEnsure you replace these queries in the Supabase SQL editor and run them.
\nRemember, as per your requirements, the users and estimations tables are linked to rooms table with a foreign key room_id. If a room is deleted, you'll need to decide whether to delete all related users and estimations or not (CASCADE or SET NULL or NO ACTION). Please consider your use case carefully before making this decision.
\n...
\nUser:\nLet's not use the 6 char id as the primary key for room since it's only "unique" for 24 hours and possible to collide in the future.
\nLet's use a bigint generated for the id.
\nWe can use a "join_code" char(6) with no special characters.
\n...
\nSystem:\nUnderstood, your point makes sense. The id field as a bigint will serve as a unique identifier across all times, and the join_code will serve as a session key valid for only 24 hours. Here are the revised SQL statements:
\nCreating the rooms table:
\nCREATE TABLE rooms (\n id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,\n join_code CHAR(6) NOT NULL,\n lead_name TEXT NOT NULL,\n created_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL\n);\nNote from Matt: the rest of them are the same
\nPlease run these revised queries in the Supabase SQL editor to set up your tables.
\n...
\nUser:\nEstimate shouldn't be null, but text seems good.
\n...
\nSystem:\nAlright, in the estimations table, you can add a NOT NULL constraint on the estimate field, which means an estimate has to be given when a record is created. Here's the updated SQL command for the estimations table:
\nCREATE TABLE estimations (\n id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,\n work_item_name TEXT NOT NULL,\n work_item_url TEXT,\n room_id BIGINT REFERENCES rooms(id),\n user_id BIGINT REFERENCES users(id),\n estimate TEXT NOT NULL,\n inserted_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL,\n updated_at TIMESTAMP WITH TIME ZONE DEFAULT timezone('utc'::text, now()) NOT NULL\n);\nPlease run this query in the Supabase SQL editor to set up your estimations table accordingly.
\n...
\nUser:\nGreat, we have our 3 tables created in Supabase. Now let's create our Next.js app. We need a page/room/[join_code].tsx that uses getServerSideProps to validate the room is valid and available. Then on the client side we can join the room with websockets.
\nIf they are the first one to join and enter their name, we need to make them the Lead for the Estimation Party and save that in the "room" table as the lead name.
\n...
\nSystem:\nAwesome! To begin, you'll first need to set up your Next.js project. If you haven't already done this, you can use the following command:
\nnpx create-next-app@latest --use-npm --example with-tailwindcss estimation-party-app\nThis command will create a new Next.js application with Tailwind CSS already set up.
\nNext, install the Supabase client library:
\nnpm install @supabase/supabase-js\nNow, create a new file at pages/room/[join_code].tsx. Here's a basic setup of how this page might look:
\nimport { useEffect, useState } from 'react';\nimport { createClient } from '@supabase/supabase-js';\n\n// Initialize Supabase client\nconst supabase = createClient("YOUR_SUPABASE_URL", "YOUR_SUPABASE_ANON_KEY");\n\nexport async function getServerSideProps(context) {\n const { join_code } = context.params;\n\n const { data: room, error } = await supabase\n .from('rooms')\n .select()\n .eq('join_code', join_code)\n .single();\n\n if (error || !room || new Date().getTime() - new Date(room.created_at).getTime() >= 24 * 60 * 60 * 1000) {\n // If an error occurred, or if no room was found, or if the room is older than 24 hours, return a 404 status.\n return {\n notFound: true,\n };\n }\n\n return {\n props: {\n room: room,\n },\n };\n}\n\nexport default function Room({ room }) {\n const [leadName, setLeadName] = useState(null);\n\n useEffect(() => {\n if (!room.lead_name) {\n // Prompt the user for their name and set them as the lead.\n const name = window.prompt("Please enter your name:");\n if (name) {\n setLeadName(name);\n supabase\n .from('rooms')\n .update({ lead_name: name })\n .eq('id', room.id);\n }\n }\n }, [room]);\n\n // The rest of your component logic here...\n return (\n <div>\n {/* Your room UI here... */}\n </div>\n );\n}\nThe getServerSideProps function is run by Next.js on each request to /room/[join_code], and it fetches the corresponding room from the Supabase database. If the room is not found, or if it is older than 24 hours, a 404 status is returned, which shows a default error page.
\nIn the Room component, the useEffect hook checks if the room has a lead. If it doesn't, it prompts the user to enter their name, and it sets them as the lead. This happens only on the first render, as room is listed as a dependency.
\nThis code gives you a basic structure for your estimation party rooms. It does not include the real-time updating of the room state, and it does not handle the case where the user cancels the name prompt. You'll need to add this functionality as you build out your application.
\nRemember to replace "YOUR_SUPABASE_URL" and "YOUR_SUPABASE_ANON_KEY" with your actual Supabase URL and anon key. You can get these from the Supabase dashboard. Also remember to secure your anon key, as it has access to your Supabase database. Do not expose it in the client-side code.
\nOk, so ChatGPT did a few things wrong here but was mostly on the right track.
\nThe command for Next.js was almost right. I'm going to use the commands from the Tailwind docs instead.
\nnpx create-next-app@latest my-project --typescript --eslint\ncd my-project\n\nnpm install -D tailwindcss postcss autoprefixer\nnpx tailwindcss init -p\nThen configure my tailwind.config.js include my /src folder.
\n/** @type {import('tailwindcss').Config} */\nmodule.exports = {\n content: [\n "./app/**/*.{js,ts,jsx,tsx,mdx}",\n "./pages/**/*.{js,ts,jsx,tsx,mdx}",\n "./components/**/*.{js,ts,jsx,tsx,mdx}",\n \n // Or if using `src` directory:\n "./src/**/*.{js,ts,jsx,tsx,mdx}",\n ],\n theme: {\n extend: {},\n },\n plugins: [],\n}\nChatGPT got this one right. I always double check the dependency since it's easy to be suggested the wrong or outdated package. This is indeed the right package.
\nnpm install @supabase/supabase-js\nLet's look at the logic it wrote for the useEffect.
\n useEffect(() => {\n if (!room.lead_name) {\n // Prompt the user for their name and set them as the lead.\n const name = window.prompt("Please enter your name:");\n if (name) {\n setLeadName(name);\n supabase\n .from('rooms')\n .update({ lead_name: name })\n .eq('id', room.id);\n }\n }\n }, [room]);\nBased on what we prompted, we told it that it should ask for the Name of the user regardless of them being the first to join the room. The first to join should become the lead. Based on what they provided if you are not the lead you will NOT be prompted to enter a name. Let's prompt ChatGPT about this and clarify why we need to useState to manage the lead_name. This should be a realtime room update to listen for changes to the lead_name. We should also be able to see the other users in the room.
\nUser:\nBased on what we prompted, we asked that it should prompt for the Name of the user regardless of them being the first to join the room. The first to join should become the lead. Based on what they provided if you are not the lead you will NOT be prompted to enter a name. Let's prompt ChatGPT about this and clarify why we need to useState to manage the lead_name. This should be a realtime room update to listen for changes to the lead_name. We should also be able to see the other users in the room.
\n...
\nSystem:\nApologies for the confusion. You're right; it makes more sense to prompt for a name from every participant and assign the role of lead to the first participant.
\nFirst, let's refactor the logic to accommodate this change.
\nimport { useEffect, useState } from 'react';\nimport { createClient } from '@supabase/supabase-js';\n\n// Initialize Supabase client\nconst supabase = createClient("YOUR_SUPABASE_URL", "YOUR_SUPABASE_ANON_KEY");\n\nexport async function getServerSideProps(context) {\n const { join_code } = context.params;\n\n const { data: room, error } = await supabase\n .from('rooms')\n .select()\n .eq('join_code', join_code)\n .single();\n\n if (error || !room || new Date().getTime() - new Date(room.created_at).getTime() >= 24 * 60 * 60 * 1000) {\n // If an error occurred, or if no room was found, or if the room is older than 24 hours, return a 404 status.\n return {\n notFound: true,\n };\n }\n\n return {\n props: {\n room: room,\n },\n };\n}\n\nexport default function Room({ room }) {\n const [name, setName] = useState(null);\n const [users, setUsers] = useState([]);\n const [isLead, setIsLead] = useState(false);\n\n useEffect(() => {\n const fetchUsers = async () => {\n const { data: users, error } = await supabase\n .from('users')\n .select()\n .eq('room_id', room.id);\n\n if (!error && users) {\n setUsers(users);\n }\n };\n\n fetchUsers();\n\n const name = window.prompt("Please enter your name:");\n if (name) {\n setName(name);\n if (!room.lead_name) {\n supabase\n .from('rooms')\n .update({ lead_name: name })\n .eq('id', room.id);\n setIsLead(true);\n }\n supabase\n .from('users')\n .insert([\n { name, room_id: room.id }\n ]);\n }\n }, [room]);\n\n useEffect(() => {\n const subscription = supabase\n .from(`rooms:id=eq.${room.id}`)\n .on('*', payload => {\n setIsLead(payload.new.lead_name === name);\n })\n .subscribe();\n\n return () => {\n supabase.removeSubscription(subscription);\n };\n }, [room, name]);\n\n useEffect(() => {\n const subscription = supabase\n .from(`users:room_id=eq.${room.id}`)\n .on('*', () => fetchUsers())\n .subscribe();\n\n return () => {\n supabase.removeSubscription(subscription);\n };\n }, [room]);\n\n // The rest of your component logic here...\n return (\n <div>\n {/* Your room UI here... */}\n </div>\n );\n}\nThis version of the Room component does the following:
\nWhen the component mounts, it fetches the existing users in the room. Then, it prompts the user for their name. If they enter a name, it checks if the room already has a lead. If it doesn't, it sets the user as the lead. Then, it adds the user to the users table in the database.
\nIt subscribes to changes to the rooms table and updates the isLead state whenever the lead_name of the room changes.
\nIt also subscribes to changes to the users table and refetches the users whenever a change is made.
\nPlease remember to replace "YOUR_SUPABASE_URL" and "YOUR_SUPABASE_ANON_KEY" with your actual Supabase URL and anon key. Also, you need to secure your anon key, as it has access to your Supabase database. Do not expose it in the client-side code.
\nThe Room component now keeps track of whether the user is the lead and the list of users in the room. It updates these states in real-time as changes are made in the database. Please adjust the JSX to present these details as you'd like.
\n...
\nUser:\nCould we initialize the supabase client in the getServersideProps and pass it to the client as a prop? I don't want to leak the anon key, and initializing outside of the getServerSideProps: "Is that safe to do?" Will it be revealed in the client if we do that?
\n...
\nSystem:\nNo, you should not pass the Supabase client as a prop from getServerSideProps. Anything returned from getServerSideProps will be serialized and sent to the client, so this would expose your Supabase client, including your anon key, to the client.
\nWhen you use the Supabase client on the server side (as in getServerSideProps), you should use a service role key that has more permissions than the anon key. But you must be careful to keep this key secret, as it can give full access to your Supabase database.
\nInitializing the Supabase client outside of getServerSideProps on the client side, like in the example I provided earlier, is safe and does not expose the anon key. The key will not be included in the client-side bundle, because Next.js knows not to include it.
\nFor example, you can create a new file called lib/supabase.js with the following code:
\nimport { createClient } from '@supabase/supabase-js'\n\nexport const supabase = createClient(\n process.env.NEXT_PUBLIC_SUPABASE_URL,\n process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY\n)\nIn next.config.js, add:
\nmodule.exports = {\n env: {\n NEXT_PUBLIC_SUPABASE_URL: 'YOUR_SUPABASE_URL',\n NEXT_PUBLIC_SUPABASE_ANON_KEY: 'YOUR_SUPABASE_ANON_KEY',\n },\n}\nThen in your pages/room/[join_code].tsx, you can import this client:
\nimport { supabase } from '../../lib/supabase'\nThis way, the client is created once and reused throughout your application, and you won't have to worry about leaking the key.
\nRemember to replace 'YOUR_SUPABASE_URL' and 'YOUR_SUPABASE_ANON_KEY' with your actual Supabase URL and anon key, and keep these variables secret.
\n...
\nUser:\nWhile that seems correct, I worry about using NEXT_PUBLIC since wouldn't that include my Anon Key in the client side bundle? That should be backend only like with a .env file, and not prepended with NEXT_PUBLIC, right?
\nNote from Matt: I argued a bit with ChatGPT and provided the docs from Next.js for environment variables.
\n...
\nSystem:\nThank you for pointing out this important detail from the Next.js documentation.
\nIn a Next.js application, server-side environment variables should be stored in a .env.local file. They can then be accessed in getStaticProps(), getServerSideProps(), or API routes using process.env.
Therefore, for your use case, you should store your Supabase credentials in the .env.local file as follows:
SUPABASE_URL=your-supabase-url\nSUPABASE_SERVICE_KEY=your-supabase-service-key\nThese variables can then be accessed securely in server-side code as follows:
\nimport { createClient } from '@supabase/supabase-js'\n\nexport const supabase = createClient(\n process.env.SUPABASE_URL,\n process.env.SUPABASE_SERVICE_KEY\n);\nYou're correct that environment variables should not be prefixed with NEXT_PUBLIC_ unless you explicitly want them to be available on the client side. For your Supabase credentials, you should only use them server-side, so you should not use the NEXT_PUBLIC_ prefix.
\nKeep in mind that while these environment variables are available in getStaticProps(), getServerSideProps(), and API routes, they are not exposed to the client side and will not be included in the client-side JavaScript bundle, ensuring their security.
...
\nUser:\nDo we need room in the state if that's being provided from getServerSideProps?
\nOur supabase select query should filter only for rooms created in the last 24 hours. If there is no room found we should display "Sorry this join code is invalid" with a link to go to the homepage '/'.
\nThere is no reason to call it "initialRoom" because there is either 0 or 1 room(s). It's just room. initialUsers is valid, because we need to update in the client if there are new users that join or leave after initial load.
\nCan you also show UI code in Tailwind UI to prompt for the name as you join a valid room?
\nWe'll show an <h2> conditionally that says "You are the lead!" and give you the button options to "Reveal Estimations" and "Start New Estimation" (which opens a modal to enter a ticket name and optionally a ticket url), with a "Start" or "Cancel" option.
Start New Estimation is only shown if there is not a current estimation.\nReveal Estimations is only shown while there is a current estimation.
\nWe need a list of users visually. All users need a visual indicator of when each user has made an estimation.
\nWe also need to display a "Card" for each User, which is revealed with their answer if they made one.
\n...
\nSystem:\nAbsolutely, your point about the room state is valid. As we are directly providing the room from getServerSideProps, there's no need to manage it in the local state. We can use it directly from the props.
\nAlso, as you correctly mentioned, the select query for the rooms should indeed filter for rooms created in the last 24 hours. I'll update that in the code.
\nNote from Matt: ChatGPT gave me the getServerSideProps code and the jsx separately, so I'm combining them here.\nHere's the updated getServerSideProps method. Next, for the UI code to prompt for a user's name and other components, we will use Tailwind UI. Let's create a simple form to get the user's name and update the Room component:
import { useState, useEffect } from 'react';\nimport { supabase } from '../../lib/supabase';\n\nconst Room = ({ room, initialUsers, error }) => {\n const [users, setUsers] = useState(initialUsers);\n const [lead, setLead] = useState(room.lead_name);\n const [name, setName] = useState('');\n const [estimationInProgress, setEstimationInProgress] = useState(false); // Mock state for now\n\n // ...\n\n if (error) {\n return (\n <div className="text-center">\n <p className="text-2xl">{error}</p>\n <a className="text-blue-600" href="/">Go to Homepage</a>\n </div>\n );\n }\n\n return (\n <div className="flex flex-col items-center justify-center min-h-screen py-2">\n <div className="max-w-md w-full space-y-8">\n <div>\n <h2 className="mt-6 text-center text-3xl font-extrabold text-gray-900">\n Welcome to the estimation room\n </h2>\n <p className="mt-2 text-center text-sm text-gray-600">\n Please enter your name\n </p>\n </div>\n <form className="mt-8 space-y-6">\n <input type="hidden" name="remember" defaultValue="true" />\n <div className="rounded-md shadow-sm -space-y-px">\n <div>\n <label htmlFor="name" className="sr-only">Name</label>\n <input id="name" name="name" type="text" required className="appearance-none rounded-none relative block w-full px-3 py-2 border border-gray-300 placeholder-gray-500 text-gray-900 rounded-t-md focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 focus:z-10 sm:text-sm" placeholder="Name" onChange={e => setName(e.target.value)} />\n </div>\n </div>\n <div>\n <button type="submit" className="group relative w-full flex justify-center py-2 px-4 border border-transparent text-sm font-medium rounded-md text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500">\n <span className="absolute left-0 inset-y-0 flex items-center pl-3">\n </span>\n Join Room\n </button>\n </div>\n </form>\n </div>\n\n {lead === name && \n <h2>You are the lead!</h2>\n }\n\n {/* Mocked estimation controls for lead */}\n {lead === name && !estimationInProgress && \n <button onClick={() => setEstimationInProgress(true)}>Start New Estimation</button>\n }\n {lead === name && estimationInProgress && \n <button onClick={() => setEstimationInProgress(false)}>Reveal Estimations</button>\n }\n\n {/* Mocked list of users */}\n <div>\n {users.map(user => (\n <div key={user.id}>\n {user.name} {user.hasEstimated && <span>Has estimated</span>}\n </div>\n ))}\n </div>\n </div>\n );\n};\n\nexport async function getServerSideProps(context) {\n const { join_code } = context.params;\n \n // Get current time and time 24 hours ago\n const now = new Date();\n const oneDayAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000);\n\n const { data: rooms } = await supabase\n .from('room')\n .select('*')\n .eq('join_code', join_code)\n .gte('created_at', oneDayAgo);\n\n const room = rooms[0];\n\n if (!room) {\n return {\n props: {\n error: 'Sorry this join code is invalid'\n },\n };\n }\n\n const { data: users } = await supabase\n .from('users')\n .select('*')\n .eq('room_id', room.id);\n\n return {\n props: { room, initialUsers: users },\n };\n}\nexport default Room;\nThis is a basic UI setup that you can start with. You would need to further customize this to match your exact requirements, including setting up actual estimation controls, user list, estimation cards, etc. The above code also does not yet handle the submission of the name input, you would need to add code to store this in the backend and handle the joining of the user to the room.
\nAlso, remember to install the Tailwind CSS library in your project and set up your tailwind.config.js accordingly if you haven't done so already.
\n...
\nUser:\nCan we have validation that the name is valid?
\nAlso we should use a Next.js API route when submitting the name, so as to conceal our API Key.
\nCan you show me the useEffects again based off our last prompt?
\n...
\nSystem:\nAbsolutely, we can add some simple front-end validation to ensure the name is entered before submitting the form. For the sake of simplicity, we'll just ensure the name field isn't empty, but you can add more validation rules as per your requirements.
\nNote from Matt: I'm combining all the code together for brevity and adding extra comments for my own use later.\nThe updated component would look like this:
\nimport { useState, useEffect } from 'react';\nimport { supabase } from '../../lib/supabase';\n\nconst Room = ({ room, initialUsers, error }) => {\n const [users, setUsers] = useState(initialUsers);\n const [lead, setLead] = useState(room.lead_name);\n const [name, setName] = useState('');\n const [estimationInProgress, setEstimationInProgress] = useState(false); // Mock state for now\n const [formValid, setFormValid] = useState(false); // State for form validation\n\n useEffect(() => {\n // Validation logic here. Update the formValid state accordingly.\n // For simplicity, we're just checking the name isn't empty.\n if (name.trim()) {\n setFormValid(true);\n } else {\n setFormValid(false);\n }\n }, [name]);\n\n \n useEffect(() => {\n const subscription = supabase\n .from(`rooms:id=eq.${room.id}`)\n .on('*', payload => {\n setIsLead(payload.new.lead_name === name);\n })\n .subscribe();\n\n return () => {\n supabase.removeSubscription(subscription);\n };\n }, [room, name]);\n\n useEffect(() => {\n const subscription = supabase\n .from(`users:room_id=eq.${room.id}`)\n .on('*', () => fetchUsers())\n .subscribe();\n\n return () => {\n supabase.removeSubscription(subscription);\n };\n }, [room]);\n\n const handleSubmit = async (e) => {\n e.preventDefault();\n\n if (!formValid) {\n return;\n }\n\n // Create API endpoint in Next.js, such as /api/join-room\n // Send a POST request to this endpoint with the name and room id as the payload.\n // The backend will handle joining the room and return the appropriate response.\n const res = await fetch('/api/join-room', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json'\n },\n body: JSON.stringify({\n name,\n roomId: room.id,\n })\n });\n\n const { success, error } = await res.json();\n\n if (success) {\n // handle success...\n } else {\n // handle error...\n }\n };\n\n if (error) {\n return (\n <div className="text-center">\n <p className="text-2xl">{error}</p>\n <a className="text-blue-600" href="/">Go to Homepage</a>\n </div>\n );\n }\n\n return (\n <div className="flex flex-col items-center justify-center min-h-screen py-2">\n <div className="max-w-md w-full space-y-8">\n <div>\n <h2 className="mt-6 text-center text-3xl font-extrabold text-gray-900">\n Welcome to the Estimation Party Room! Join Code: {room.join_code}\n </h2>\n <p className="mt-2 text-center text-sm text-gray-600">\n Please enter your name\n </p>\n </div>\n <form className="mt-8 space-y-6">\n <input type="hidden" name="remember" defaultValue="true" />\n <div className="rounded-md shadow-sm -space-y-px">\n <div>\n <label htmlFor="name" className="sr-only">Name</label>\n <input id="name" name="name" type="text" required className="appearance-none rounded-none relative block w-full px-3 py-2 border border-gray-300 placeholder-gray-500 text-gray-900 rounded-t-md focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 focus:z-10 sm:text-sm" placeholder="Name" onChange={e => setName(e.target.value)} />\n </div>\n </div>\n <div>\n <button type="submit" className="group relative w-full flex justify-center py-2 px-4 border border-transparent text-sm font-medium rounded-md text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500">\n <span className="absolute left-0 inset-y-0 flex items-center pl-3">\n </span>\n Join Room\n </button>\n </div>\n </form>\n </div>\n\n {lead === name && \n <h2>You are the lead!</h2>\n }\n\n {/* Mocked estimation controls for lead */}\n {lead === name && !estimationInProgress && \n <button onClick={() => setEstimationInProgress(true)}>Start New Estimation</button>\n }\n {lead === name && estimationInProgress && \n <button onClick={() => setEstimationInProgress(false)}>Reveal Estimations</button>\n }\n\n {/* Mocked list of users */}\n <div>\n {users.map(user => (\n <div key={user.id}>\n {user.name} {user.hasEstimated && <span>Has estimated</span>}\n </div>\n ))}\n </div>\n </div>\n );\n};\n\nexport async function getServerSideProps(context) {\n const { join_code } = context.params;\n \n // Get current time and time 24 hours ago\n const now = new Date();\n const oneDayAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000);\n\n const { data: rooms } = await supabase\n .from('room')\n .select('*')\n .eq('join_code', join_code)\n .gte('created_at', oneDayAgo);\n\n const room = rooms[0];\n\n if (!room) {\n return {\n props: {\n error: 'Sorry this join code is invalid'\n },\n };\n }\n\n const { data: users } = await supabase\n .from('users')\n .select('*')\n .eq('room_id', room.id);\n\n return {\n props: { room, initialUsers: users },\n };\n}\nexport default Room;\nNext, you'll want to create an API route /api/join-room.js to handle the joining of the room. Here's a basic example of what the API route could look like:
\nimport { supabase } from '../../lib/supabase';\n\nexport default async function handler(req, res) {\n if (req.method !== 'POST') {\n return res.status(405).json({ error: 'Method not allowed.' });\n }\n\n const { name, roomId } = req.body;\n\n const { error } = await supabase.from('users').insert([\n { name, room_id: roomId },\n ]);\n\n if (error) {\n return res.status(500).json({ error: error.message });\n }\n\n return res.status(200).json({ success: true });\n}\nNow, when a user submits the form, their name and the room id is sent to this API route which inserts a new record into the users table in your Supabase database. It then returns a success or error message depending on the outcome, which you can then handle in your Room component.
\nI'm running out of messages for ChatGPT Plus so I'm gonna end this as a Part 1 in a series of using ChatGPT to create a live Estimation app with Next.js and Supabase. In the next part, we'll actually create the Next.js application and see how far we get with the suggestions from ChatGPT.
\nPlease reach out on LinkedIn or via Email if you have any questions or feedback. I'd love to hear from you!
", "url": "https://www.groff.dev/blog/creating-a-live-estimation-app-with-chatgpt", "title": "Creating a Live Estimation App with ChatGPT - Part 1", "summary": "Prompting ChatGPT to help me create a Live Estimation App with Next.js and Supabase.", "date_modified": "2023-05-20T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/ask-openai-from-bash-terminal", "content_html": "Inspired by this article by Norah Sakal on asking OpenAI questions from the terminal using python.
\nI often find myself searching for the correct syntax for a command line tool or trying to remember the name of a specific flag. Instead of going to ChatGPT or asking GitHub Copilot, I wanted to see if I could get a quick answer from OpenAI directly in my terminal.
\nMeet ask, a bash command line program that uses OpenAI's GPT-3.5 Turbo model to answer your questions right in the terminal. This small program not only sends your question to OpenAI, but also the output of pwd and ls -al to provide additional context for your question.
You can setup ask by adding the following lines in your ~/.bashrc:
export OPEN_AI_API_KEY="myopenaiapikey"\nalias ask='ruby ~/ask/ask.rb'\nAfter that, run source ~/.bashrc to reload your shell. Remember to replace "myopenaiapikey" with your actual OpenAI API key. The ask.rb script should be located in the ask folder in your home directory.
\nTo use ask, simply type ask followed by your question in quotes:
\nask "How do I extract my discord file?"\nLet's say you have a discord.tar.gz file in your current directory and you don't know how to extract it. Here's how ask can help:
\nmatt@ubuntu-desktop:~$ ls\nask Desktop discord.tar.gz Documents Downloads Music Pictures Public snap Templates Videos working\nmatt@ubuntu-desktop:~$ ask "How do I extract my discord file?"\nYou can extract your discord file using the following command:\n\ntar -xvf discord.tar.gz\n\nThis will extract the contents of the `discord.tar.gz` file in the current directory.\nAs you can see, ask not only gave the correct command to extract a tar.gz file, but it also used the correct filename from the context provided.
\nCurrently, ask is configured to use the gpt-3.5-turbo model, but you can change it to gpt-4 in the ask.rb script if you prefer. The gpt-3.5-turbo model is generally sufficient for command line questions and provides cost savings.
You can find my GitHub Repo with ask here.
I'd love to hear your thoughts on LinkedIn.
", "url": "https://www.groff.dev/blog/ask-openai-from-bash-terminal", "title": "Ask Anything from your Bash Terminal with OpenAI.", "summary": "Interact with OpenAI's GPT-4 model directly from your bash terminal for quick answers to your questions.", "date_modified": "2023-05-12T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/monorepo-git-hooks", "content_html": "Despite its amusing name, Mookme is a "simple, easy-to-use, yet powerful, and language-agnostic git hook designed specifically for monorepos." The brains behind Mookme, escape.tech, are also responsible for the GraphQL and REST API Security Tools. Their innovative contributions are worth a follow, so consider checking them out on LinkedIn.
\nHusky has been a valuable tool in preventing wasted time through its pre-commit hooks on various projects. The last thing anyone wants is to commit something that fails to pass eslint or hasn't been formatted with prettier. Mookme offers similar functionality but excels in managing pre-commit hooks in monorepos without needing alignment to a package.json.
\nSimply drop a .hooks folder into any subdirectory, and the hooks will only execute if there are commits making changes within those subdirectories. If there's a need for a hook to run across the entire repo, a global .hooks can be set at the root.
In an ongoing internal project, we have a Rails API in /backend and a Next.js app in /frontend. I've added eslint and prettier with lint-staged exclusively to the Next.js app by positioning the hook here:\n/frontend/.hooks/pre-commit.json
The json syntax is straightforward:
\n{\n "steps": [\n {\n "name": "Lint",\n "command": "npx lint-staged"\n }\n ]\n}\nOur /frontend/package.json houses our lint-staged configuration:
{\n "lint-staged": {\n "src/**/*.(ts|tsx|js|jsx)": ["eslint --fix", "prettier --write"],\n "src/**/*.(css|scss)": ["prettier --write"]\n }\n}\nFor the future, I plan to explore integrating a hook into /backend/.hooks/ to run a Ruby gem that would generate new TypeScript interfaces for use in the /frontend Next.js app, ensuring we always have the latest types if the Rails API schema changes.
What are some of your favorite uses for git hooks? I'd love to hear your thoughts on LinkedIn.
", "url": "https://www.groff.dev/blog/monorepo-git-hooks", "title": "Leveraging Git Hooks in Monorepos with Mookme.", "summary": "An advanced Git hook solution for monorepos, similar to Husky.", "date_modified": "2023-05-11T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/eliminate-unnecessary-rerenders-in-react", "content_html": "Inspired by How I eliminate ALL unnecessary Rerenders in React by Vitaliysteffensen
\nThe useEffect hook is one of the most powerful hooks in React. It allows you to perform side effects in your components. It's also one of the most misused hooks in React. If you're ever using useEffect to update state, you may have gotten yourself into an infinite loop. If any of the values in the dependency array have changed since the last render, the callback function will be called again. Setting the state in the callback function will cause a rerender, which will cause the callback function to be called again, and so on.
Don't do this.
\nuseEffect({\n setSomeState()\n}, [someState])\nThe above code will cause an infinite loop.
\nAnother common cause for infinite loops in React is calling a function instead of referencing it.
\n<button onClick={updateStateFunction()}>Click me</button>\nThe code above includes a function that is being called within the onClick attribute. When we attempt to render the page, this function is immediately executed, which can trigger another render. This cycle of triggering the function and causing a render can continue indefinitely until it ultimately causes a runtime error in React.
\nAvoid this pattern and instead use one of the following:
\n<button onClick={updateStateFunction}>Click me</button>\n<button onClick={() => updateStateFunction()}>Click me</button>\nThe useMemo hook is used to memoize a value. Instead of React rerendering your component every time any prop or state in its parent are changed, you can wrap your component with useMemo and React will only rerender your component when one of the items in your dependency array changes. This is useful when you have a component that is expensive to render, but you know that it will only need to be rerendered when a specific prop or state changes.
const expensiveComponent = useMemo(() => <ExpensiveComponent someProp={someProp} />, [someProp])\nLet's say this component is returned on the same page as a state that is constantly changing. If that state is irrelevant to our expensive component, we can wrap our expensive component in useMemo and React will only rerender it when someProp changes. It will not rerender when the state changes, if that state is not included in the dependency array.
You can also use useMemo to memoize a function or variable. This is useful when you have a function that is expensive to run, but you know that it will only need to be run when a specific prop or state changes.
I'll use my Mortgage Calculator React App as an example. There's so many variables that can be changed, but many of our values should only re-calculate based upon specific parts of the form. For example, the monthly payment should only be recalculated when the loan amount, interest rate, or loan term changes. We can use useMemo to memoize our monthly payment function and only recalculate it when one of those values changes.
const MonthlyPayment = ({\n housePrice,\n percentDown,\n propertyTaxPercentage,\n homeownersInsuranceYearlyCost,\n hoaCostMonthly,\n yearsOfMortgage,\n interestRate,\n monthlyPaymentShouldBeAtOrBelow,\n hasPmiUnder20Down,\n}) => {\n const downPayment = useMemo(() => {\n return Number(Number(housePrice) * (Number(percentDown) / 100)).toFixed();\n }, [housePrice, percentDown]);\n\n const loanAmount = useMemo(() => {\n return Number(Number(housePrice) - Number(downPayment)).toFixed();\n }, [housePrice, downPayment]);\n\n const principalInterestPayment = useMemo(() => {\n const monthlyInterestRate = Number(interestRate) / 100 / 12;\n const numberOfPayments = Number(yearsOfMortgage) * 12;\n const monthlyPayment =\n Number(loanAmount) *\n (monthlyInterestRate /\n (1 - Math.pow(1 / (1 + monthlyInterestRate), numberOfPayments)));\n return Number(monthlyPayment).toFixed();\n }, [loanAmount, interestRate, yearsOfMortgage]);\n // And so on...\nYou can find the rest of my component on (GitHub)[https://github.com/mattlgroff/mortgage-calc/blob/master/src/components/MonthlyPayment.jsx]
\nWhen using the useMemo hook, it's important to consider the trade-off between memory and performance. As a general rule, you should only apply useMemo when:
*It re-renders given the same props or state\n*It re-renders often
\nHowever, if your component introduces side effects, it should not be pure or memoized. Some examples of side effects include:
\n*Modifying any external variable or object property\n*Logging data to the console\n*Writing data to a file\n*Writing data to the network\n*Triggering any external process\n*Calling any other functions with side-effects\n*Making asynchronous data calls
\nBut what if your component deals with non-primitive data types?
\nuseMemo does not work with non-primitive types. The reason is that non-primitive types can't be directly compared by their value. An example of this would be:
a = {key: 10};\n b = {key: 10};\n c = a;\n a === b //returns false\n a === c //returns true\nHow do we prevent rerendering, when using non-primitive data types?
\nIn a memoized component we do have the option to control the logic of the memo, by using a custom areEqual function.
function areEqual(prevProps, nextProps) {\n/*\nreturn true if passing nextProps to render would return\nthe same result as passing prevProps to render,\notherwise return false\n*/\n}\nYou could grab isEqual from lodash or write your own for your specific use case.
function areEqual(prevProps, nextProps) {\n return _.isEqual(prevProps, nextProps);\n}\nThough it's better to just pass in primitive types to your memoized component or function (if possible). You can also pass a key to a primitive in the object instead.
\nconst obj = {key: 10};\n\nuseMemo(() => {\n // if obj.key changes, do something\n}, [obj.key])\nDon't do this:
\nexport default function App() {\n const [useRole, setUserRole] = useState("defaultUser");\n\n if (userRole === "admin") {\n return (\n <div className="admin-page">\n <AdminHeaderComponent />\n <HeaderComponent />\n <ContentComponent />\n </div>\n )\n }\n\n return (\n <div className="user-page">\n <HeaderComponent />\n <ContentComponent />\n </div>\n )\n}\nThe above code demonstrates poor conditional rendering, as both the <HeaderComponent/> and <ContentComponent/> are remounted whenever the state changes, even though it's unnecessary.
If you find that you're rendering the exact same component on both sides of a ternary operator, it's usually a sign of poor conditional rendering.
\nA better solution would be to extract the <HeaderComponent/> and <ContentComponent/> from the conditional render, as demonstrated below.
export default function App() {\n const [useRole, setUserRole] = useState("defaultUser");\n\n return (\n <div className={userRole === "admin" ? "admin-page" : "user-page"}>\n {userRole === "admin" && <AdminHeaderComponent />}\n <HeaderComponent />\n <ContentComponent />\n </div>\n )\n}\nOne common issue with useEffect() is specifying the dependencies incorrectly. Since useEffect() often involves state manipulation, it can cause unnecessary re-rendering if not used properly. To avoid this, it's important to be specific with the dependencies you pass to useEffect().
\nBy being specific with the dependencies, you can ensure that the useEffect() hook only runs when it's necessary. This can help improve the performance of your application and prevent unnecessary re-rendering.
\nIn addition to being specific, you can also use memoization to prevent running the useEffect() hook when it's not necessary.
\nuseEffect(() => {\n // do something\n}, [someState])\nRemember, non-primitives can't be directly compared by their value. So if you're using a non-primitive type as a dependency, you'll need to use a custom areEqual function or better yet pass in a primitive type or key to the object.
useEffect(() => {\n // do something\n}, [someState.key])\nAnother technique to prevent unnecessary re-renders is to use memoization with the useMemo() hook. By using useMemo(), you can create a memoized value that only updates when its dependencies change, reducing the number of updates to that value. Here's an example:
\nconst memoizedValue = useMemo(() => computeExpensiveValue(a, b), [a, b]);\n\nuseEffect(() => {\n // do something with memoizedValue\n}, [memoizedValue])\nOur useEffect will trigger only if the memoizedValue changes. This is a great way to prevent unnecessary re-renders.
useReducer is a hook that lets you add a reducer to your component. If you're familiar with Redux, you'll find that useReducer is very similar to Redux's useReducer hook.
Read more about refactoring useState to useReducer here.
It won't fix re-renders on it's own, but it will make your code more readable and easier to maintain which might help you find the source of your re-renders.
\nAlways remember to consider the trade-off between memory and performance when using useMemo. Memoizing values that are not expensive to calculate can actually hurt performance. Be sure to test and profile your application to ensure that it's running as efficiently as possible.
\nHonorable mention but not something I've worked with a ton yet is the useCallback hook which is specifically designed to memoize functions. It can be used for:
\nIt wasn't covered in today's article but maybe I'll focus on it in the future. I've yet to use it in a project but it seems like a great tool to have in your toolbelt.
\nThe React Developer Tools extension can help track down the source of re-renders.
\nIn conclusion, there are several techniques you can use to eliminate unnecessary rerenders in your React application. You can use useMemo to memoize expensive calculations and prevent unnecessary renders of components. It's also important to optimize the dependencies in useEffect to ensure that the hook only runs when necessary. You can use the useReducer hook to add a reducer to your component, which can make your code more readable and easier to maintain.
\nRemember, every time your component rerenders, it can negatively impact the performance of your application. By using these techniques, you can prevent unnecessary rerenders and make your application faster and more efficient.
", "url": "https://www.groff.dev/blog/eliminate-unnecessary-rerenders-in-react", "title": "Eliminating Unnecessary Rerenders in React.", "summary": "useMemo, useState, useEffect, and useReducer are all you need to eliminate all unnecessary rerenders in React.", "date_modified": "2023-05-02T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/react-hook-click-outside", "content_html": "On one of my recent projects at work we had one component that required a way to handle the action of clicking outside of the component, it was a custom Search Select dropdown. We wanted it to close when clicking outside. In this case, the engineer assigned to that component created a useOutsideClick hook, but it was kind of specific to that particular use case and not a universal way to handle this action.
Then comes along another engineer who needed to create a similar way to handle a click outside of their component and the hook didn't work for their use case because it was a nested child element that was being clicked. They ended up importing an npm package called react-outside-click-handler. It's still rather popular with 694,000 weekly downloads as of the time of this writing. The only problem is that it was last updated over 4 years ago.
I decided to dig into the repository and found out that it's a single React Class component written by AirBNB called OutsideClickHandler.jsx. It was written long before the days of React Typescript and Hooks. I decided to take a stab at converting it to a React Hook and it was a lot easier than I thought it would be. I was able to get it working in a matter of minutes using ChatGPT's assistance.
\nThe original component's imports:
\nimport React from 'react'\nimport PropTypes from 'prop-types'\n\nimport { forbidExtraProps } from 'airbnb-prop-types'\nimport { addEventListener } from 'consolidated-events'\nimport objectValues from 'object.values'\n\nimport contains from 'document.contains'\n\nconst DISPLAY = {\n BLOCK: 'block',\n FLEX: 'flex',\n INLINE: 'inline',\n 'INLINE-BLOCK': 'inline-block',\n CONTENTS: 'contents',\n}\n\nconst propTypes = forbidExtraProps({\n children: PropTypes.node.isRequired,\n onOutsideClick: PropTypes.func.isRequired,\n disabled: PropTypes.bool,\n useCapture: PropTypes.bool,\n display: PropTypes.oneOf(objectValues(DISPLAY)),\n})\n\nconst defaultProps = {\n disabled: false,\n\n // `useCapture` is set to true by default so that a `stopPropagation` in the\n // children will not prevent all outside click handlers from firing - maja\n useCapture: true,\n display: DISPLAY.BLOCK,\n}\nWe really didn't need PropTypes in this case, so I removed them. I also removed the forbidExtraProps and objectValues imports because they were only used for the PropTypes. I also removed the DISPLAY object because it was only used for the display prop. Lastly I removed the addEventListener and contains imports because they were also outdated packages that were simply for polyfills for older browsers, which we don't need to worry about anymore.
At this point I prompted ChatGPT my copy pasting in the code from the original component and it helped me convert it to a functional React component instead of class based.
\nimport React, { useEffect, useRef } from 'react'\n\ntype Display = 'block' | 'flex' | 'inline' | 'inline-block' | 'contents'\n\ninterface OutsideClickHandlerProps {\n children: React.ReactNode\n onOutsideClick: (event: MouseEvent) => void\n disabled?: boolean\n useCapture?: boolean\n display?: Display\n}\n\nconst OutsideClickHandler: React.FC<OutsideClickHandlerProps> = ({\n children,\n onOutsideClick,\n disabled = false,\n useCapture = true,\n display = 'block',\n}) => {\n const childNode = useRef<HTMLDivElement>(null)\n\n const onMouseDown = (e: MouseEvent) => {\n const isDescendantOfRoot =\n childNode.current && childNode.current.contains(e.target as Node)\n if (!isDescendantOfRoot) {\n document.addEventListener('mouseup', onMouseUp, { capture: useCapture })\n }\n }\n\n const onMouseUp = (e: MouseEvent) => {\n const isDescendantOfRoot =\n childNode.current && childNode.current.contains(e.target as Node)\n document.removeEventListener('mouseup', onMouseUp, { capture: useCapture })\n\n if (!isDescendantOfRoot) {\n onOutsideClick(e)\n }\n }\n\n useEffect(() => {\n if (!disabled) {\n document.addEventListener('mousedown', onMouseDown, {\n capture: useCapture,\n })\n }\n\n return () => {\n document.removeEventListener('mousedown', onMouseDown, {\n capture: useCapture,\n })\n document.removeEventListener('mouseup', onMouseUp, {\n capture: useCapture,\n })\n }\n }, [disabled, useCapture])\n\n return (\n <div ref={childNode} style={display !== 'block' ? { display } : undefined}>\n {children}\n </div>\n )\n}\n\nexport default OutsideClickHandler\nThat was easy! Nice. Now I asked it to turn it into a React Hook that took in the same props as the previous useOutsideClick hook that the first engineer made and it gave me the final result:
import { Dispatch, RefObject, SetStateAction, useEffect } from 'react'\n\nexport const useOutsideClick = (\n ref: RefObject<HTMLElement>,\n isOpen: boolean,\n setIsOpen: Dispatch<SetStateAction<boolean>>,\n useCapture = true\n) => {\n useEffect(() => {\n const onMouseDown = (e: MouseEvent) => {\n const isDescendantOfRoot =\n ref.current && ref.current.contains(e.target as Node)\n if (!isDescendantOfRoot) {\n document.addEventListener('mouseup', onMouseUp, { capture: useCapture })\n }\n }\n\n const onMouseUp = (e: MouseEvent) => {\n const isDescendantOfRoot =\n ref.current && ref.current.contains(e.target as Node)\n document.removeEventListener('mouseup', onMouseUp, {\n capture: useCapture,\n })\n\n if (!isDescendantOfRoot) {\n setIsOpen(false)\n }\n }\n\n if (isOpen) {\n document.addEventListener('mousedown', onMouseDown, {\n capture: useCapture,\n })\n }\n\n return () => {\n document.removeEventListener('mousedown', onMouseDown, {\n capture: useCapture,\n })\n document.removeEventListener('mouseup', onMouseUp, {\n capture: useCapture,\n })\n }\n }, [isOpen, useCapture])\n\n return isOpen\n}\nIn order to use this you would import it like so:
\nimport React, { useRef, useState } from 'react'\nimport { useOutsideClick } from './src/hooks/useOutsideClick'\n\nconst MyComponent: React.FC = () => {\n const [isOpen, setIsOpen] = useState(true)\n const ref = useRef<HTMLDivElement>(null)\n\n useOutsideClick(ref, isOpen, setIsOpen)\n\n return <div ref={ref}>{/* Your component content */}</div>\n}\n\nexport default MyComponent\nHooray! Now we have all the upsides of a single reusable React Hook that works for both use cases and we didn't need to import a four year old package to do it. I hope this helps someone else out there who is trying to figure out how to use make a useOutsideClick hook. If you have any questions or comments please let me know!
I created this guide after reading a few different resources, mainly Apollo's post from November 2021 which is linked in the resources at the bottom of this post. I ran through these step by step and made some changes along the way. I hope this helps you get started with GraphQL and Ruby on Rails.
\nrails new rails-graphql -d postgresql --skip-action-mailbox --skip-action-text --skip-spring --webpack=react -T\nThis command will create a new rails application with the following options:
\nrails new rails-graphql This command creates a new Rails application with the name rails-graphql.
-d postgresql Use postgresql as database
--skip-action-mailbox Skip action-mailbox gem
--skip-action-text Skip action-text gem
--skip-spring Skip spring gem
--webpack=react: This flag tells Rails to use Webpack for managing the application's JavaScript assets and to configure it for React. Webpack is a powerful bundler for JavaScript applications, and React is a popular JavaScript library for building user interfaces. By specifying --webpack=react, the Rails application will be pre-configured to work with React.
-T: This flag tells Rails not to include the default test suite (Minitest). This is useful if you plan to use a different testing framework, such as RSpec, for your application.
action-mailbox - Action Mailbox routes incoming emails to controller-like mailboxes for processing in Rails. It ships with ingresses for Mailgun, Mandrill, Postmark, and SendGrid. You can also handle inbound mails directly via the built-in Exim, Postfix, and Qmail ingresses.
action-text - Action Text brings rich text content and editing to Rails. It includes the Trix editor that handles everything from formatting to links to quotes to lists to embedded images and galleries.
spring - Spring speeds up development by keeping your application running in the background so you don't need to boot it every time you run a test, rake task or migration.
Not required, but I prefer using a UUID for my primary keys. Edit your config/application.rb file and add the following line to the Application class:
# Default to UUIDs for primary keys\n config.generators do |g|\n g.orm :active_record, primary_key_type: :uuid\n end\nNext, since we're using postgres, we need to add a migration for postgres to generate uuids. Create a new migration file with the following command:
\nrails generate migration enable_pgcrypto_extension\nEdit the migration file and add the following line to the change method:
enable_extension 'pgcrypto' unless extension_enabled?('pgcrypto')\nThis will only enable the extension if it's not already enabled. The migration should look something like this:
\nclass EnablePgcryptoExtension < ActiveRecord::Migration[7.0]\n def change\n enable_extension 'pgcrypto' unless extension_enabled?('pgcrypto')\n end\nend\nHere are the typeDefs we're going to use for our models:
\n"A person"\n type Person {\n id: ID!\n first_name: String!\n last_name: String!\n email: String!\n job_title: String!\n avatar: String\n comments: [Comment!]!\n }\n\n "A comment"\n type Comment {\n id: ID!\n comment: String!\n }\n`;\nNow we can create our first model. We'll create a Person model with the following command:
rails g model Person first_name:string last_name:string email:string job_title:string avatar:string\nThis will create a Person model with the following attributes:
first_name: stringlast_name: stringemail: stringjob_title: stringavatar: stringActive Record will generate a model and a migration file for us. The migration file will look something like this:
\nclass CreatePeople < ActiveRecord::Migration[7.0]\n def change\n create_table :people, id: :uuid do |t|\n t.string :first_name\n t.string :last_name\n t.string :email\n t.string :job_title\n t.string :avatar\n\n t.timestamps\n end\n end\nend\nNext we'll make the Comment model.
rails g model Comment person:references comment:string\nThis will create a Comment model with the following attributes:
comment: stringperson: references // This will create a foreign key to the Person modelThe migration file will look something like this:
\nclass CreateComments < ActiveRecord::Migration[7.0]\n def change\n create_table :comments, id: :uuid do |t|\n t.references :person, null: false, foreign_key: true, type: :uuid\n t.string :comment\n\n t.timestamps\n end\n end\nend\nMake sure you have Postgres running before running the migration.
\nRunning Postgres with Docker
\ndocker run -d --name my-postgres -p 5432:5432 -e POSTGRES_PASSWORD=mysecretpassword -e POSTGRES_DB=rails_graphql_development postgres\n\nYou can configure your development database in the config/database.yml file.
development:\n <<: *default\n database: rails_graphql_development\n username: postgres\n password: mysecretpassword\n host: localhost\nNow you can run rails db:migrate to create the tables in your database.
rails db:migrate\nIf successful, you should see something like this:
\n== 20230423020452 EnablePgcryptoExtension: migrating ==========================\n-- extension_enabled?("pgcrypto")\n -> 0.0111s\n-- enable_extension("pgcrypto")\n -> 0.0082s\n== 20230423020452 EnablePgcryptoExtension: migrated (0.0194s) =================\n\n== 20230423020459 CreatePeople: migrating =====================================\n-- create_table(:people, {:id=>:uuid})\n -> 0.0125s\n== 20230423020459 CreatePeople: migrated (0.0126s) ============================\n\n== 20230423020504 CreateComments: migrating ===================================\n-- create_table(:comments, {:id=>:uuid})\n -> 0.0116s\n== 20230423020504 CreateComments: migrated (0.0116s) ==========================\nTo add the relationship between the Person and their Comments, navigate to the app/models/person.rb file and add has_many :comments, dependent: :destroy
We will need some pre-generated data to work with and render to our page. In the db/seeds/rb file add the following contents and save the file.
\n# This file should contain all the record creation needed to seed the database with its default values.\n# The data can then be loaded with the bin/rails db:seed command (or created alongside the database with db:setup).\n#\n# Examples:\n#\n# movies = Movie.create([{ name: "Star Wars" }, { name: "Lord of the Rings" }])\n# Character.create(name: "Luke", movie: movies.first)\n\nmatt = Person.create!(\n first_name: "Matt",\n last_name: "Groff",\n email: "matt@example.com",\n job_title: "Director of Engineering",\n avatar: "https://www.gravatar.com/avatar/b21bbd4c0b7f75a0fbb469c238639eb7"\n)\n\nComment.create!(\n [\n {\n person: matt,\n comment: "This is a comment from Matt Groff",\n },\n {\n person: matt,\n comment: "This is another comment from Matt Groff",\n }\n ]\n)\nNow you can run rails db:seed to seed the database with the data we just created.
rails db:seed\nTo create our Rails-GraphQL API, let’s use a ruby gem called graphql-ruby. It will add many files to our project. It will add a lot of files that will help run our project. To add the gem, run the following line in your console followed by the generator.
\nbundle add graphql\n\nrails generate graphql:install\n\nbundle install\nNote: If you want GraphiQL in Production, go into your Gemfile and change the line for the graphiql-rails gem from this:
\ngem "graphiql-rails", group: :development\nto this:
\ngem "graphiql-rails"\nA Rails generator is used for automating the process of creating files with boilerplate code. It creates and updates files based on templates, etc.
\nLet’s poke around in the files and see what we got! Check out the schema file, rails_graphql_schema.rb. This is where it declares where all the queries should go and set up mutations.
class RailsGraphqlSchema < GraphQL::Schema\n mutation(Types::MutationType)\n query(Types::QueryType)\nend\nLet’s get this app running. Look at the config/routes.rb file. The generator is very helpful here! It is mounting graphiql::Rails::Engine for us. This allows us to test queries and mutation using the handy web interface, GraphiQL. Think of it as building out documentation and a fun place to test out your queries on the web.
\nRails.application.routes.draw do\n if Rails.env.development?\n mount GraphiQL::Rails::Engine, at: "/graphiql", graphql_path: "/graphql"\n end\n post "/graphql", to: "graphql#execute"\n # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html\n\n # Defines the root path route ("/")\n # root "articles#index"\nend\nThis is giving us a GraphiQL interface on get requests but only in development. We can change this to be available in production if we want. Which in this case, I do want this.
\nRails.application.routes.draw do\n mount GraphiQL::Rails::Engine, at: "/graphiql", graphql_path: "/graphql"\n post "/graphql", to: "graphql#execute"\n # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html\n\n # Defines the root path route ("/")\n # root "articles#index"\nend\nAlternatively, you can use the Apollo Studio Explorer. It’s Apollo’s web IDE for creating, running, and managing your GraphQL operations.
\nWe are going to add more information to our Rails GraphQL API so we can write our first GraphQL query in our Rails project.
\nWe’re going to remove some of the example content and add a field called :comments in the query_type.rb file, so we can get all the comments returned. Notice the new comments method added here. Each field type contains a name (comments), a result type/options ([Types::CommentType], and :null is required and set to true or false. The description is optional but good to have since it helps with documentation.
module Types\n class QueryType < Types::BaseObject\n # Add `node(id: ID!) and `nodes(ids: [ID!]!)`\n include GraphQL::Types::Relay::HasNodeField\n include GraphQL::Types::Relay::HasNodesField\n\n # Add root-level fields here.\n # They will be entry points for queries on your schema.\n\n field :comments,\n [Types::CommentType],\n null: false,\n description: "Return a list of comments"\n\n def comments\n Comment.all\n end\n end\nend\nWe now want to generate the CommentType using the GraphQL Ruby gem. In your console, enter the following command:
\nrails g graphql:object comment\nThe response should look like this if successful:
\ncreate app/graphql/types/comment_type.rb\nNow we need to update the types/comment_type.rb file to include the fields that have a type and nullable option.
module Types\n class CommentType < Types::BaseObject\n field :id, ID, null: false\n field :comment, String, null: false\n field :person, Types::PersonType, null: false\n field :created_at, GraphQL::Types::ISO8601DateTime, null: false\n field :updated_at, GraphQL::Types::ISO8601DateTime, null: false\n end\nend\nYou might be thinking, how does this all work? It looks for the method with the same name defined in the class time (thanks rails magic!) Now let’s do the same thing, but for the PersonType.
\nrails g graphql:object person\nAgain, if successful, you should see the following response:
\ncreate app/graphql/types/person_type.rb\nNow we need to update the types/person_type.rb file to include a full_name method and full_name field:
module Types\n class PersonType < Types::BaseObject\n field :id, ID, null: false\n field :first_name, String\n field :last_name, String\n field :email, String\n field :job_title, String\n field :avatar, String\n field :created_at, GraphQL::Types::ISO8601DateTime, null: false\n field :updated_at, GraphQL::Types::ISO8601DateTime, null: false\n\n # Add a field to the PersonType that returns a full name\n field :full_name, String, null: false\n def full_name\n [object.first_name, object.last_name].compact.join(" ")\n end\n end\nend\nWe now have enough code to start your rails server. Run rails s in your console and open up GraphiQL: http://localhost:3000/graphiql in your web browser. In GraphiQL run the following query. We can type in a query to run with the data we added to our db/seeds file and get a response back.
Start the server:
\nrails s\nThe output should look like:
\n=> Booting Puma\n=> Rails 7.0.4.3 application starting in development\n=> Run `bin/rails server --help` for more startup options\nPuma starting in single mode...\n* Puma version: 5.6.5 (ruby 3.2.2-p53) ("Birdie's Version")\n* Min threads: 5\n* Max threads: 5\n* Environment: development\n* PID: 38587\n* Listening on http://127.0.0.1:3000\n* Listening on http://[::1]:3000\nUse Ctrl-C to stop\nNow go to http://localhost:3000/graphiql and run the following query:
\nquery {\n comments {\n id\n comment\n person {\n firstName\n lastName\n email\n createdAt\n }\n }\n}\nThe response should look similar to this:
\n{\n "data": {\n "comments": [\n {\n "id": "cfd6f460-edae-4e88-88d1-a882b38ee5a1",\n "comment": "This is a comment from Matt Groff",\n "person": {\n "firstName": "Matt",\n "lastName": "Groff",\n "email": "matt@example.com",\n "createdAt": "2023-04-23T02:44:22Z"\n }\n },\n {\n "id": "e5821082-6f31-44cb-8b97-2f590a55ba29",\n "comment": "This is another comment from Matt Groff",\n "person": {\n "firstName": "Matt",\n "lastName": "Groff",\n "email": "matt@example.com",\n "createdAt": "2023-04-23T02:44:22Z"\n }\n }\n ]\n }\n}\nHow's this all work? How is Rails doing all this?
\n Person Load (0.8ms) SELECT "people".* FROM "people"\n Comment Load (0.5ms) SELECT "comments".* FROM "comments" WHERE "comments"."person_id" = $1 [["person_id", "49f53c0b-3c55-47d2-8842-9b7bffeb3d1b"]]\nCompleted 200 OK in 24ms (Views: 0.2ms | ActiveRecord: 6.6ms | Allocations: 19936)\nThe GraphQL gem created the GraphqlController for us. It is where requests are sent to. Within this file, you can see that the execute method/action does a lot of work for us.
\ndef execute\n variables = prepare_variables(params[:variables])\n query = params[:query]\n operation_name = params[:operationName]\n context = {\n }\n result = TaypiSchema.execute(query, variables: variables, context: context, operation_name: operation_name)\n render json: result\n rescue StandardError => e\n raise e unless Rails.env.development?\n handle_error_in_development(e)\nend\nWe did a GraphQL query with Ruby on Rails! We are now fetching person/people along with comments.
\nI wanted to add a "person" query which takes in an ID of the Person and returns that Person object. We can do this by adding a new field to the QueryType and a new resolver.
# Ask for a person by ID\n field :person,\n Types::PersonType,\n null: true,\n description: "Find a person by ID" do\n argument :id, ID, required: true\n end\n\n def person(id:)\n Person.find(id)\n end\nAnd inside of PersonType I added a comments field to return all the comments for a person.
# Add a field to the PersonType that returns a list of comments\n field :comments, [Types::CommentType], null: false\n def comments\n object.comments\n end\nNow I can query a person like so:
\nquery {\n person(id: "3c4ceab0-1b13-4fe1-a4bb-ea4ec8b042f8") {\n id\n firstName\n lastName\n email\n comments {\n id\n comment\n }\n }\n}\nIf I also want to ask for multiple people, let's add to the QueryType once again:
# Ask for a list of people\n field :people,\n [Types::PersonType],\n null: false,\n description: "Return a list of people"\n\n def people\n Person.all\n end\nNow I can query a list of people like so:
\nquery {\n people {\n id\n firstName\n lastName\n email\n comments {\n id\n comment\n }\n }\n}\nLet's add a Mutation to add a new Comment to a Person. Go to mutation_type.rb and add the following to the class MutationType like so:
module Types\nclass MutationType < Types::BaseObject\n # Add a field to the MutationType that adds a new comment\n field :add_comment, Types::CommentType, null: false do\n argument :comment, String, required: true\n argument :person_id, ID, required: true\n end\n\n def add_comment(comment:, person_id:)\n Comment.create!(\n comment: comment,\n person_id: person_id\n )\n end\n\nend\nend\nThis creates a new mutation called addComment that takes in a comment and person_id and returns a Comment.
addComment(\ncomment: String!\npersonId: ID!\n): Comment!\nNow we can add a new comment to a person like so:
\nmutation {\n addComment(\n comment: "This is a new comment"\n personId: "3c4ceab0-1b13-4fe1-a4bb-ea4ec8b042f8"\n ) {\n id\n comment\n person {\n id\n firstName\n lastName\n email\n comments {\n id\n comment\n createdAt\n }\n }\n }\n}\nLet's create a new person. Go to mutation_type.rb and add the following to the class MutationType like so:
module Types\nclass MutationType < Types::BaseObject\n # Add a field to the MutationType that adds a new comment\n field :add_comment, Types::CommentType, null: false do\n argument :comment, String, required: true\n argument :person_id, ID, required: true\n end\n\n def add_comment(comment:, person_id:)\n Comment.create!(\n comment: comment,\n person_id: person_id\n )\n end\n\n # Add a field to the MutationType that adds a new person\n field :add_person, Types::PersonType, null: false do\n argument :first_name, String, required: true\n argument :last_name, String, required: true\n argument :email, String, required: true\n argument :job_title, String, required: true\n argument :avatar, String, required: true\n end\n\n def add_person(first_name:, last_name:, email:, job_title:, avatar:)\n Person.create!(\n first_name: first_name,\n last_name: last_name,\n email: email,\n job_title: job_title,\n avatar: avatar\n )\n end\nend\nend\nThis creates a new mutation called addPerson that takes in a first_name, last_name, email, job_title, and avatar and returns a Person.
addPerson(\nfirstName: String!\nlastName: String!\nemail: String!\njobTitle: String!\navatar: String!\n): Person!\nNow we can add a new person like so:
\nmutation {\n addPerson(\n firstName: "Jane"\n lastName: "Doe"\n email: "jane@example.com"\n jobTitle: "Senior Design Crafter"\n avatar: "https://www.gravatar.com/avatar/df56f48c7a057d2b45915c96011aaf42"\n ) {\n id\n firstName\n lastName\n email\n jobTitle\n avatar\n }\n}\nTo disable CSRF protection for your GraphQL endpoint, you can modify your app/controllers/application_controller.rb file as follows:
class ApplicationController < ActionController::Base\n protect_from_forgery with: :exception, unless: :graphql_controller?\n\n private\n\n def graphql_controller?\n controller_name == 'graphql'\n end\nend\nIn config/environments/production.rb, update the following line to the config.public_file_server.enabled section:
# Disable serving static files from the `/public` folder by default since\n # Apache or NGINX already handles this.\n config.public_file_server.enabled = true\nI wanted to dockerize and deploy this app. I followed this tutorial to do so.
\nCreate a Dockerfile in the root directory of the project.
FROM ruby:3.2.2-alpine\nWORKDIR /app\nCOPY . .\nRUN apk add --no-cache build-base tzdata nodejs postgresql-dev\nRUN gem install bundler\nRUN bundle install\nENV RAILS_ENV=production\nRUN bundle exec rails assets:precompile\nEXPOSE 3000\nCMD ["rails", "server", "-b", "0.0.0.0"]\nIn your terminal create a secret with rails secret. Copy the secret.
rails secret\nWe need to add a secret_key_base for the production environment in order to deploy. Edit the config/environments/production.rb file and add the following line:
config.secret_key_base = <your copied secret here>\nNOTE: If we were using any encrypted data this would be a bad security practice for obvious reasons. We can and should use an ENV variable for this in production deployments.
\nBuild the docker image to make sure it works.
\ndocker build -t rails-graphql .\nIf successful, you should see something like this:
\n[+] Building 37.2s (13/13) FINISHED\n => [internal] load build definition from Dockerfile 0.0s\n => => transferring dockerfile: 32B 0.0s\n => [internal] load .dockerignore 0.0s\n => => transferring context: 2B 0.0s\n => [internal] load metadata for docker.io/library/ruby:3.2.2-alpine 1.1s\n => [auth] library/ruby:pull token for registry-1.docker.io 0.0s\n => [internal] load build context 0.1s\n => => transferring context: 209.06kB 0.1s\n => [1/7] FROM docker.io/library/ruby:3.2.2-alpine@sha256:697038d90aa973dfa8bb3613f3d57d58b38bdf7957b83a 0.0s\n => CACHED [2/7] WORKDIR /app 0.0s\n => [3/7] COPY . . 0.3s\n => [4/7] RUN apk add --no-cache build-base tzdata nodejs postgresql-dev 11.5s\n => [5/7] RUN gem install bundler 1.7s\n => [6/7] RUN bundle install 18.1s\n => [7/7] RUN bundle exec rails assets:precompile 2.5s\n => exporting to image 1.9s\n => => exporting layers 1.9s\n => => writing image sha256:7d437c9e05b4cc85f17d7f23acd11f03af3a3f98bd38306f15b4f62ad5d3a02f 0.0s\n => => naming to docker.io/library/rails-graphql\nRun the docker image to make sure it works.
\ndocker run -p 3000:3000 \\\n -e DATABASE_URL="postgres://postgres:mysecretpassword@localhost:5432/rails_graphql_development" \\\n rails-graphql\nusing-graphql-with-ruby-on-rails
\ndockerize-deploy-and-run-a-ruby-on-rails-app
", "url": "https://www.groff.dev/blog/how-to-make-a-graphql-api-with-ruby-rails", "title": "How to make a GraphQL API with Ruby on Rails.", "summary": "Explaining step by step how to make a GraphQL API with Ruby on Rails.", "date_modified": "2023-04-23T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/gpt-test-generator", "content_html": "Today, I had an exciting experience working with OpenAI's ChatGPT to create a Bun app called GPT Test Generator. This application generates test cases for your code with the help of ChatGPT. In this post, I'll walk you through the process of creating this app and how I collaborated with the AI.
\nAs a developer, I always struggle with writing tests. While it's an essential part of the development process, it can be time-consuming and tedious, taking away from the creative side of coding. That's where GPT Test Generator comes in. It automates the process of generating test cases, giving me more time to focus on building the core functionality of my applications.
\nGo check out the GitHub Repo for this project here. There is a README.md that explains how to run this yourself with our own API key from OpenAI.
Throughout the development process, I worked closely with ChatGPT, asking it questions and receiving assistance in various aspects such as improving the UI, handling errors, and enhancing the user experience. The AI helped me to create a better application, demonstrating the potential of AI-assisted development.
\nIn the GPT Test Generator app, the test_types are the different types of tests that can be generated for a given code input. These test types were prompted by passing example tests and test APIs into the prompt of the API call, making them more accurate and up-to-date.
\nCurrently, the app uses the GPT-3.5-turbo model to generate the test cases, but I'm eagerly waiting for my turn on the GPT-4 API waitlist. The addition of these test types ensures that the app generates a wide range of test cases, improving the accuracy and effectiveness of the generated tests.
\nI had a great time pair-programming this with ChatGPT.
\nThat's all for now!
", "url": "https://www.groff.dev/blog/gpt-test-generator", "title": "Creating a GPT Test Generator Bun App with AI Assistance.", "summary": "Learn how I built a GPT Test Generator Bun app with ChatGPT.", "date_modified": "2023-04-22T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/cypress-component-tests-with-chatgpt", "content_html": "Writing unit tests for React components can be time-consuming, especially if you have a large codebase with many components to test. Fortunately, with the help of Chat GPT and Cypress, you can save time by automating the process of writing unit tests.
\nLet's say you have a React component called MyComponent that you want to test. Start by creating an empty file called my-component.spec.js in the cypress/integration directory.
\nNext, go to chat.openai.com and start a new chat with the Chat GPT app. In the chat, copy and paste your MyComponent code and ask Chat GPT to "write me a cypress component spec test for the following React component."
\nHere's an example of what you might paste into the Chat GPT app:
\nCopy code\nimport React from 'react';\nimport { mount } from '@cypress/react';\n\nfunction MyComponent() {\n return (\n <div>\n <h1>Hello, world!</h1>\n <p>This is my component.</p>\n </div>\n );\n}\nAfter you paste the code, wait for Chat GPT to generate the tests. Depending on the complexity of your component, this may take a few seconds or a few minutes.
\nOnce Chat GPT has generated the tests, copy and paste the output into the my-component.spec.js file that you created earlier.
\nHere's an example of what the generated tests might look like:
\ndescribe('MyComponent', () => {\n it('renders correctly', () => {\n mount(<MyComponent />);\n cy.get('h1').should('contain', 'Hello, world!');\n cy.get('p').should('contain', 'This is my component.');\n });\n});\nTo run your tests, open the Cypress test runner by running the following command:
\nnpx cypress open\nThis will open the Cypress test runner, where you can select the my-component.spec.js file and run the tests.
\nIf all goes well, you should see the tests pass and be confident that your component is working as expected.
\nWith the help of Chat GPT and Cypress, you can save time by automating the process of writing Cypress component tests for your React components. Simply copy and paste your component code, let Chat GPT do the rest, and run the tests. With this approach, you can ensure that your components are reliable and working as expected, while saving time and effort in the testing process.
\nI prompted ChatGPT to write this article for me. It needed a lot of guidance.
\nFor example:
\nYou don't do this: Let's say that you have a React component called MyComponent that you want to test. To get started, copy the component code and paste it into a new file in the cypress/integration directory. Let's call this file my-component.spec.js.\n\nDo this:\nLet's say you have a React component called MyComponent that you want to test. Create an empty file called my-component.spec.js. Now go to ChatGPT and copy and paste your MyComponent code into it, asking ChatGPT to "write me a cypress component spec test for the following React component" and paste it in.\nIn a previous post, I wrote about how to make a GraphQL Server with Hono but today I'm super excited to show you how to make an Apollo Server with Bun, which was previously not possible prior to Bun v0.5.4.
\nTo get started:
\ncurl -fsSL https://bun.sh/install | bash\nbun-apollo:mkdir bun-apollo\ncd bun-apollo\npackage.json{\n "name": "bun-apollo",\n "version": "1.0.0",\n "description": "Bun Apollo Server",\n "main": "src/index.js",\n "scripts": {\n "start": "bun index.js"\n }\n}\nbun add graphql @apollo/server\nIf successful you'll see something like this:
\nbun add v0.5.4 (0dfde6f8)\n\n installed @apollo/server@4.3.2\n installed graphql@16.6.0\n\n[1321.00ms] done\nimport { ApolloServer } from '@apollo/server';\nimport { startStandaloneServer } from '@apollo/server/standalone';\n\nconst typeDefs = `#graphql\n type Book {\n title: String\n author: String\n }\n\n type Query {\n books: [Book]\n }\n`;\n\nconst books = [\n {\n title: 'The Awakening',\n author: 'Kate Chopin',\n },\n {\n title: 'City of Glass',\n author: 'Paul Auster',\n },\n];\n\nconst resolvers = {\n Query: {\n books: () => books,\n },\n};\n\nconst server = new ApolloServer({\n typeDefs,\n resolvers,\n});\n\nconst port = process.env.PORT || 4000;\n\nawait startStandaloneServer(server, {\n listen: { port },\n});\n\nconsole.log(`🚀 Server ready at: http://localhost:${port}/graphql`);\nThis is based off the example from the Apollo's getting started documentation
\nbun run index.js\nYou should see something like this:
\n🚀 Server ready at: http://localhost:4000/graphql\nDockerfile and add the following code:FROM oven/bun:0.5\nWORKDIR /app\nCOPY package.json package.json\nCOPY bun.lockb bun.lockb\nRUN bun install\nCOPY index.js index.js\nEXPOSE 4000\nENTRYPOINT ["bun", "run", "index.js"]\nYou can test the Dockerfile locally if you have Docker installed by running the following command:
\ndocker build -t bun-apollo .\n\ndocker run -p 4000:4000 bun-apollo\nCommit and push up the code to a new repository. I suggest adding node_modules/ to the .gitignore file.
I deployed using Render, but there are tons of options to deploy an application in a Docker Container.
\nI selected "New +" and then selected "Web Service". Once I connected my repository to the service, it detected my Dockerfile and was building and deploying in minutes.
\nTo test that the deployment works, run the books query against the Apollo Server on the newly deployed endpoint.
Congrats, if everything worked you have an Apollo GraphQL Server running on Bun in the cloud!
\nYou can find the source code for my example Apollo Server w/ Bun on GitHub here.
", "url": "https://www.groff.dev/blog/bun-apollo-express", "title": "How To Make An Apollo Server With Bun.", "summary": "Apollo Server used to be exclusive to Node.js, not anymore. Here's how to make an Apollo Server with Bun.", "date_modified": "2023-02-02T00:00:00.000Z", "author": { "name": "Matthew Groff" } }, { "id": "https://www.groff.dev/blog/bun-express", "content_html": "I've been using Node for a long while and express almost goes hand and hand. It's the go-to http server with countless middleware packages. Let's use it in Bun.
You can read all about Bun here bun.sh.
\nNow to get started:
\ncurl -fsSL https://bun.sh/install | bash\nnpm init to create a package.json file.mkdir bun-express\ncd bun-express\nnpm init\nbun add express\nimport express from "express";\n\n// Initialize the express app\nconst app = express();\n\n// Hello World GET endpoint\napp.get("/", (req, res) => {\n res.send("Hello World!");\n});\n\n// Set the default port to 3000, or use the PORT environment variable\nconst port = process.env.PORT || 3000;\n\napp.listen(port, () => console.log(`Express 🥟 Server Listening on port ${port}`));\nbun index.js\nYou should see something like this in the console:
\nExpress 🥟 Server Listening on port 3000\nDockerfile and add the following code:FROM jarredsumner/bun:0.5.1\nWORKDIR /app\nCOPY package.json package.json\nCOPY bun.lockb bun.lockb\nRUN bun install\nCOPY index.js index.js\nEXPOSE 3000\nENTRYPOINT ["bun", "index.js"]\nYou can test the Dockerfile locally if you have Docker installed by running the following command:
\ndocker build -t bun-express .\n\ndocker run -p 3000:3000 bun-express\nCommit and push up the code to a new repository. I suggest adding node_modules/ to the .gitignore file.
I deployed using Render, but there are tons of options to deploy an application in a Docker Container.
\nI selected "New +" and then selected "Web Service". Once I connected my repository to the service, it detected my Dockerfile and was building and deploying in minutes.
\nTo test that deployment works, visit the Express server on the newly deployed endpoint. You should see "Hello World!"
\nCongrats, if everything worked you have an Express server running on Bun in the cloud!
\nYou can find the source code for my example Express Server w/ Bun on GitHub here. The code is going to look a bit different than the example here in this blog because I opted to put my index.js into a /src folder.
UPDATE 2/4/2023 - I have a new blogpost for a much better method of running a GraphQL server with Bun here with Apollo Server
\nThis is my first post on my new blog, how exciting!
\nToday, I was trying to find an example of how to make a GraphQL server with Bun, but I couldn't find one. I decided to create one instead.
\nYou can read all about Bun here bun.sh.
\nBun has a lot of the Node API built in, but it provides its own, faster, HTTP server. This causes incompatibilities with the traditional methods I use to create a GraphQL server (Apollo Server or ExpressJS). Thankfully, Hono is compatible with Bun. It claims to be "a small, simple, and ultrafast web framework for Cloudflare Workers, Deno, Bun, and others."
\nAdditionally, luck would have it, there is a new package for GraphQL Server with Hono which we can utilize to get a GraphQL server up and running. It was build as a fork of graphql-express, and it depends on the GraphQL.js package to work.
\nThe GraphQL.js package does not fully support Bun at this time, but we only need one part of it in order to get this server to work, the buildSchema function.
Now to get started:
\ncurl -fsSL https://bun.sh/install | bash\nnpm init to create a package.json file.mkdir bun-graphql\ncd bun-graphql\nnpm init\nbun add graphql\nbun add hono\nbun add @honojs/graphql-server\nimport { Hono } from 'hono'\nimport { graphqlServer } from '@honojs/graphql-server'\nimport { buildSchema } from 'graphql/utilities/buildASTSchema.js'\n\n// Initialize Hono\nexport const app = new Hono()\n\n// Builds schema from a string\nconst schema = buildSchema(`\n type Query {\n hello: String\n }\n`)\n\n// Resolvers\nconst rootResolver = (ctx) => {\n return {\n hello: () => 'Hello Hono!',\n }\n}\n\n// Hono.js GraphQL Server\napp.use(\n '/graphql',\n graphqlServer({\n schema,\n rootResolver,\n })\n)\n\n// Set the default port to 3000, or use the PORT environment variable\nconst port = process.env.PORT || 3000\n\nconsole.log(`Hono 🥟 GraphQL Server Listening on port ${port}`)\n\nexport default {\n port,\n fetch: app.fetch,\n}\nThis is based off the example file from Hono's GraphQL Server package, but tweaked to match Hono's Bun example.
\nIt's important to import buildSchema directly from ''graphql/utilities/buildASTSchema.js', otherwise your app will fail because the GraphQL.js library isn't fully Bun compatible, but this essential function is.
bun index.js\nYou won't see any logs in the console, but you should see a GraphQL server running if you query the endpoint. http://localhost:3000/graphql
Dockerfile and add the following code:FROM jarredsumner/bun:0.5.1\nWORKDIR /app\nCOPY package.json package.json\nCOPY bun.lockb bun.lockb\nRUN bun install\nCOPY index.js index.js\nEXPOSE 3000\nENTRYPOINT ["bun", "index.js"]\nYou can test the Dockerfile locally if you have Docker installed by running the following command:
\ndocker build -t bun-graphql .\n\ndocker run -p 3000:3000 bun-graphql\nCommit and push up the code to a new repository. I suggest adding node_modules/ to the .gitignore file.
I deployed using Render, but there are tons of options to deploy an application in a Docker Container.
\nI selected "New +" and then selected "Web Service". Once I connected my repository to the service, it detected my Dockerfile and was building and deploying in minutes.
\nTo test that deployment works, run the same hello query against the GraphQL server on the newly deployed endpoint.
Congrats, if everything worked you have a GraphQL server running on Bun in the cloud!
\nYou can find the source code for my example GraphQL Server w/ Bun on GitHub here. The code is going to look a bit different than the example here in this blog because I added GraphQL Playground and I'm reading my schema from a graphql.schema file.