The wrapper is the right first move (usually)
If you can validate your idea by piping a prompt to an LLM and rendering the response, do that. A "ChatGPT wrapper" — a thin app that formats a prompt, calls an API, and shows the answer — is the fastest way to learn whether anyone wants the thing you're building. We tell most early clients to start here. Shipping in a weekend beats architecting for six weeks.
The mistake isn't starting with a wrapper. The mistake is not knowing the five signals that mean you've outgrown it — and pouring more money into prompt tweaks when the real problem is architectural.
Signal 1: You're stuffing more into the prompt than the model can act on
A wrapper's only lever is the prompt. When requirements grow, the prompt grows: more rules, more examples, more "IMPORTANT: do not…". Past a point, the model starts ignoring the middle of your instructions (the classic "lost in the middle" problem) and behavior gets non-deterministic in ways no amount of wording fixes.
A custom agent moves that logic out of the prompt and into code the model calls — tools, validators, and control flow. The model decides what to do; your code decides how, and enforces the rules deterministically.
Signal 2: It needs to do things, not just say things
The moment your product needs to book the appointment, update the CRM, hit an internal API, or move a file — not just describe how — you need tool/function calling with real error handling. A wrapper that "returns instructions for the user to follow" is a demo. An agent that completes the task is a product.
This is the single most common line we see crossed. "It writes a great draft email" becomes "it should just send the email," and suddenly you need auth, idempotency, retries, and an audit log. None of that lives in a prompt.
Signal 3: You can't tell why it did what it did
Wrappers are black boxes: prompt in, text out. When a customer says "it gave me the wrong answer," you have nothing to inspect. A custom agent logs each step — which tools ran, with what arguments, what they returned — so you can replay a bad run and fix the actual cause instead of guessing at prompt wording.
If you're doing anything regulated, financial, or high-trust, this stops being a nice-to-have. You need the trace.
Signal 4: Costs scale with the wrong thing
In a wrapper, every request re-sends the entire context. As your instructions and history grow, per-call token cost grows with them — you pay more for every user, forever, whether or not the extra context helped. A custom agent controls context deliberately: retrieve only what this step needs, cache what's stable, and route cheap requests to cheap models. Cost scales with work done, not with prompt bloat.
Signal 5: Quality is a vibe, not a number
With a wrapper you ship prompt changes and hope. With an agent you build an eval set — real inputs with known-good outputs — and every change gets scored before it ships. The first time a "harmless" prompt tweak silently breaks 20% of cases and your evals catch it, you'll never go back.
The honest cost of switching
Going from wrapper to agent isn't a rewrite of your product — it's adding a spine. Typically:
- A tool layer: your real actions, as typed functions with validation.
- A control loop: decide → act → observe → repeat, with a step budget.
- Observability: structured logs of every step.
- Evals: a scored test set that gates changes.
You keep your UI. You keep your value prop. You replace "one big prompt" with "a small model orchestrating reliable code."
The rule of thumb
Stay on the wrapper while your product's job is to produce text a human reviews. Move to an agent the moment its job is to take actions a human trusts. The wrapper proves people want it; the agent is what they'll actually pay for.
If you're staring at signals 2 and 3 right now, that's the tell. That's the conversation we have with clients every week — and usually the switch is smaller than they feared.
