Sweet PalAn AI assistant for people living with type 1 diabetes can do a lot: understand a meal described in plain words, put the data in order, show how glucose moved the last few times. There is one thing it must never do: decide in place of the person and their diabetes team. In Sweet Pal that isn't a promise buried at the bottom of a page. It's a set of rules built into the product, some inside the model and some, the most important ones, outside it.
Let's start with an uncomfortable fact. A language model is excellent at writing and at reading the tone of a sentence, but it is also unpredictable in a way this field can't afford: the same message, at two different moments or with two different models, can get two different answers. For a joke, that's fine. For a low glucose reading, it isn't. So we gave ourselves a simple principle: a safety rule is validated on the worst case, not on the best engine. If a safeguard only works when the model behaves, it isn't a safeguard. That's why we write the most delicate rules twice: once in the model's instructions, and once in code, where they behave the same way every time.
Risky values override everything
In the instructions of Sweet Pal's chat there is a rule that comes before every other one. We call it rule zero: acute safety beats everything. With a value in the hypoglycemia zone, with ketones, or with feeling seriously unwell, the reply changes nature: two or three sentences, written calmly, inviting the person to act on the protocol agreed with their own team. No offers, no suggestions, no charts. And the rule applies even when the person says they feel fine, or that "it happens all the time". Feeling fine at a low value doesn't disprove the risk: it's called hypoglycemia unawareness, and it is exactly the case where a reassuring reply would do harm.
We learned this in practice. When we separated decisions from the writing of replies, in the architecture we described in Decide first, speak later, a message like "I'm at 62, I feel fine, it happens all the time" was read as a story, calm, almost chatty. The fix was a dedicated question, a single one, asking whether the message contains a risky value; if the answer is yes, it overrides every other decision. Erring on the side of caution here costs little; erring the other way doesn't.
The model isn't the only one watching, though. On the phone, the app reads what the person wrote, not the model's reply, and decides on its own whether the moment is urgent. It looks for words that need no numbers, about hypoglycemia, ketones or feeling faint, and for a declared glucose value, which it compares with the lower limit of that person's range, not a threshold that's the same for everyone. It is cautious by construction: a number alone is never enough, because "I ate 60 g" is not a glucose reading. It takes a number with its unit, or a phrasing like "I'm at 52" (in Italian, for now: that's the language the app speaks). Better not to raise the alarm than to raise it by mistake in a quiet conversation: an alarm that sounds too often stops being heard.
When the rule triggers, the reply bubble changes: a solid mark in a muted red and one line that says, in effect, "this is what matters now". We chose seriousness, not fear, because whoever is reading at that moment already feels bad and a bright red background would only add weight. The same goes for glucose entered by hand from the quick-entry button: there the reply isn't written by the model but composed by the app from the data it already has, and if the value is low the voice drops. No analysis, no forecast, no amounts: only a pointer to the person's own care protocol, and some company.
Estimates to confirm, not decisions
The rest of the day, which thankfully isn't an emergency, follows another family of rules. When you describe or photograph a meal, Sweet Pal estimates the carbohydrates and presents them for what they are: a value to confirm or correct, not a number to take on trust. When you describe a bolus you took, the app picks up dose and time and offers to log them in your diary. But a question like "how much should I bolus?" is not treated as a bolus to log, and it doesn't get a dose as an answer. It's the rule that matters most, written without nuance in the model's instructions: never recommend a treatment or a dose. Our terms of use say so explicitly too: the app does not diagnose, does not prescribe, does not change the care plan, and is not connected to pumps, connected pens or automated systems. You decide and take the dose, with your team.
Avoiding advice isn't enough: you also have to avoid sounding like advice. The Simulator shows how the curve might move with a meal, an activity or a bolus, and the first version of the text next to the chart read like a plan, in the second person and the future tense. No sentence contained a dose, and it was still wrong. Today those texts talk about the curve, in the third person: "glucose starts from", "after about half an hour the meal pushes the curve up", "the bolus insulin slows the rise". Times are given as time elapsed since the start of the illustration, not as "in half an hour", which would sound like an appointment.
The last rule is about statistical honesty. An assistant that sees your data every day is tempted to find patterns everywhere, and "this always happens to you" is an easy sentence to write and almost always false. In Sweet Pal's instructions the scale is precise: a single episode is an anecdote, two or three times are "the last few times", and real trends only come from the statistical engine, which tells an early hint apart from an established link. When the app shows a link, it also shows how many episodes it rests on.
That helps the person using the app, who can give what they see the right weight, and it helps their diabetologist, who can tell at a glance whether an observation is an impression or something worth discussing. Put together, the rules draw a clear line. On one side, what an AI assistant for type 1 diabetes does well: understanding everyday language, taking the effort out of the diary, remembering how past times went, making visible what would stay hidden in the numbers. On the other, what isn't its job: deciding how much insulin, changing a treatment, reassuring when the numbers say otherwise. It isn't a limit we put up with. It's the reason you can trust what the app does.
In short:
- Sweet Pal's safety rules are validated on the worst case: the most delicate ones live both in the model's instructions and in the app's code.
- A risky value, ketones or feeling seriously unwell override everything, even when the person says they feel fine.
- Carbohydrates are estimates to confirm; described boluses get logged, but the app never suggests a dose.
- Simulation texts describe the curve in the third person, never like instructions.
- Every link shown says how many episodes it rests on: one episode is an anecdote, not a pattern.