Sweet PalA model-agnostic AI product comes down to three choices: every provider speaks through the same interface, each role's model is picked from a database row rather than a deploy, and the decisions that matter are taken away from the model and handed to the architecture. That way, when a model changes, disappears or starts behaving differently, the product stays the same. It's what we learned building Sweet Pal.
Language models change faster than any other component of an app. Every month there's a new one that is faster, cheaper or better at something. If the product is written around one specific model, every release turns into a small house move; if the model is a replaceable part, a release is something to try in an afternoon. Above all, the model isn't under your control: the provider decides when to update it, when to retire it and how its parameters change.
We found out in the most concrete way possible. At the end of September, the role that does Sweet Pal's background jobs, extracting memories from conversations, compacting long chats, writing analyses, started failing. The configuration was right: the provider had stopped serving that model, an older version of the kind that sooner or later gets retired. The catalogue still listed it, but calls came back "not available". That day five memory extractions, three compactions and one analysis failed. The lesson isn't "pick a better provider". It's that a model in the catalogue isn't a model that answers, and that the product has to be able to switch models in minutes, without shipping anything.
That's why Sweet Pal talks to three providers today without the rest of the code knowing. Two use the same API format and the third has its own, but they all sit behind a single function that takes the conversation, the available tools and the output cap, and returns text, any tool calls and the token counts. Every difference lives inside that function: how the request is built, how the stream is read, how reasoning blocks are sent back on the next round when a provider requires it. Adding a provider means writing an adapter, not rewriting the chat.
The model is picked in the database, not in a deploy
In Sweet Pal, models aren't in the code, or even in environment variables. They live in a table: for each role (the chat, background jobs, image reading) there's a row with the model, the reasoning level and the temperature. Changing model means editing that row in the internal panel, and the next message already runs on the new model. A few weeks ago the table gained one more dimension, the subscription: each role can have one model for Expert and one for Pro, so the chat turn of people who pay more can use a stronger model without touching a line of code.
Three rules make this flexibility safe. First, the code checks the shape, not the list. An allow-list of models written in code would force a deploy to add one, exactly what we want to avoid; instead the code checks that the provider is one of ours and that the model name has a valid shape, and that's enough to stop a badly written row from sending traffic somewhere else. Second, one read per turn. The configuration is read once at the start and the resolved model travels as an object to the end; read it again halfway and the model that answered and the model the cost is billed to might not match. Third, no fallback model. Falling back to a model hard-coded in the app when the database doesn't answer sounds kind, but a fallback model has no price list, a call without a price list costs zero, and a call that costs zero doesn't move the spending counter: a database hiccup would turn the usage cap into nothing. Better a visible error. After the September episode we also added a "test" button next to every model choice, which makes a minimal call to the provider before applying the change: if the model isn't being served, you find out there, not in the next day's logs.
A common interface doesn't make models equal, though, and the place where the differences show most is reasoning, the time a model spends thinking before it writes. Every provider controls it its own way, and sometimes changes it from one generation to the next. An open model, unless explicitly told not to reason, reasons anyway: with the role set to "no reasoning", a round went from two or three seconds to seven. In one model family, "budget" reasoning worked on the older versions and returns an error on the newer ones. To turn it off, one model accepts "disabled", another rejects that word and wants a different one, a third never turns off and can only be set to its lowest level; yet another, without a budget, came back with an empty answer. None of these is a product decision: they're provider details, and they all live next to the adapter. The rest of the system asks for "flat voice" or "reason at medium level", and the adapter translates it into the form that model accepts. When a provider changes the rules, you fix one function, not ten.
Taking the decisions away from the model
The choice that truly made us independent from the model, though, is a different one. In J2LM, the architecture we described in Decide first, speak later, the language model no longer makes the decisions. Specialised layers do, answering closed questions in parallel in about three hundred milliseconds; a script written in code turns their choices into instructions; the model only writes, in a single round and without tools. If the model only has to voice decisions already made, its talent for reasoning matters much less. We saw it in the measurements: two very different models, one open and one proprietary, both completed the same reported meal, reply and cards included, in about a second and a half. With the classic agent, which decided everything on its own, the same two models delivered the cards at 3 and 4.5 seconds.
The same principle applies to how much the model should think. If the model decides, the chat's speed depends on how much it likes to reason, and every new model has a different temperament. That's why Sweet Pal's voice has two speeds: normally it's flat, with reasoning truly switched off in whatever form each model accepts, and it goes deep, at the level the subscription provides, only when the decision layers ask for it, because the person wants to dig in, the message wasn't understood with confidence, or there's a computation on their data to explain. With the same model at medium level, the flat voice starts writing in 0.9–1.4 seconds, the deep one in 7.5–8. The model no longer decides how much to think: the product does, the same way whatever model sits underneath.
There's still work to do. The reasoning rules change too often, and every change has to be checked by hand on a test bench that reproduces the real chat. The next step is making that check automatic: a set of reference sentences every model has to pass before it can be picked in the panel.
In short:
- Providers sit behind a single interface: their differences live in the adapters, not in the product.
- Each role's model is a database row, checked for shape, read once per turn, with no fallback models that escape cost tracking.
- A model in the catalogue isn't a model that answers: test before you switch.
- Taking decisions away from the model makes the product less sensitive to which model you use.
- How much the model reasons is decided by the architecture, not by the model.