Same prompts, new reader

On 14 January 2027, GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini stop working on the OpenAI API. A lot of production systems were built on those models, because for two years they were the sensible default: capable, fast and cheap enough to put in front of customers.

We've moved a production assistant across, one that answers product and support questions and summarises conversations. The model names live in configuration, so on paper this was a change to two settings. In practice the settings were the least of it, and the interesting part was not the API at all.

First, the good news: it costs less

At October 2026 list prices, the like-for-like tier (GPT-6 Sol at $2 in and $10 out per million tokens) is cheaper on input than GPT-4o ($2.50 and $10), and the small tier (GPT-6 Luna at $0.10 and $0.50) is cheaper than GPT-4o mini ($0.15 and $0.60). Running the same conversations through each model, our cost per conversation came out roughly a quarter lower, partly from the prices and partly because the newer model writes shorter answers.

The more useful lesson was about our own forecast. We had modelled cost per conversation from prompt sizes, and the measured figure came in at well under half of it. Neither number was measured until we started logging token usage on every call. More on that below.

It is not a configuration change

Newer models are reasoning models, and they reject parts of a request the older ones accepted. Two things to note:

• Temperature. Any value other than the default returns an error. If somewhere in your code a call sets temperature to 0.1 for "consistency", that call fails outright the moment the model changes.
• Token limits. max_tokens is refused; the parameter is now max_completion_tokens.

The tempting fix is to edit each call site. The better one is to shape requests in one place, in the client every call goes through, so that call sites never need to know which generation of model is configured. Then a model change really is a configuration change, and rolling back is too.

Turn the thinking off unless you want it

This is a change in approach and worth calling out on its own. GPT-4o answered straight away. The latest models reason by default: they think before they answer unless told not to, and that thinking is billed as output. On our short, retrieval-grounded calls it added around 30 to 50 output tokens and half a second to a second, on every call, for no visible improvement. Setting the reasoning effort to none gave us the old behaviour, and the old latency. Decide this per kind of call rather than discovering it on the invoice: some tasks do benefit from the model thinking first, but a lookup against a knowledge base usually does not.

Instructions you forgot about now run

This was the surprise. Product questions on the new model suddenly got worse in a specific way: asked to compare two products from the same range, with both specification sheets sitting in the retrieved context, it asked the customer what mattered most to them instead of answering. Asked for current offers, with the offers page retrieved, it asked whether they wanted a monthly or a one-off price.

The code had not changed. The cause was a line of prompt stored in configuration rather than in the codebase, attached to the shopping intent long ago: if constraints are missing, ask one concise follow-up question before recommending. The old model had been treating it as background advice and answering anyway. The new one did exactly what it said.

We only found it by holding everything else still: same build, same retrieved context, swap the model, compare. The old model answered with a comparison table; the new one asked a question. Once we had rewritten the instruction to answer from the context first and ask only when the context gives nothing to choose by, the new model answered with the table and then asked one short question to narrow the choice, which is better than either model managed before.

And some rules turn out to be preferences

The reverse also happened, which is why "follows instructions more strictly" is not quite the right summary. Our main prompt said to answer based only on the provided context. The old model read that as a refusal rule: given a request the knowledge base had nothing to say about, it declined. The new model read it as a preference: it answered the same request from general knowledge, fluently, every time.

Our test request was harmless. The same looseness about a returns policy or a warranty term is an invented answer given in the company's name. The pattern we took from it: the new model follows procedures literally ("if X, do Y") and treats descriptions loosely ("answer only from the context"). So we rewrote the rule as behaviour. Every fact about the company, its products, prices and policies must come from the context; general knowledge may only be used to connect the question to it (working out which listed store is nearest to a town the data does not mention, for instance); and when the context has nothing, say so briefly and invite another question.

That last part needed its own instruction too. Asked something out of scope, the model helpfully explained what it could help with, summarising whatever had been retrieved for the question. On a knowledge base covering dozens of topics, it described itself by the handful that happened to come back. A refusal should decline, not guess at a description of the service.

A new model exposes old coupling

The links shown under an answer were drawn from the best-scoring retrieved items rather than from the items the answer actually used. With the old model those two lists mostly overlapped, so nobody noticed. The new model cites more widely, including brand pages and editorial content, and the overlap fell apart: a recommendation for one thing, with links to three others underneath.

The model was not wrong. The design had depended on a habit of the old model without anyone writing that down. The fix was to link only what the answer cites. Expect a model change to find a few of these, and treat them as findings rather than regressions.

Measure every call, before and after

A single customer turn is rarely one model call, and most assistants make several: rewriting the question in the light of the conversation, routing it, matching products, then the answer. The answer is usually most of the cost and the rest is small but not nothing. Conversation summaries cost almost nothing by comparison.

We only know that because we added usage logging at the client layer, so every chat call reports the model, the input, cached, output and reasoning tokens, whatever code made it. Do this first. It turns the migration from a pricing argument into a measurement, and it will probably correct your forecast as it corrected ours.

A short checklist

• Find every call that sets temperature or max_tokens, and shape requests centrally rather than per call.
• Set reasoning effort explicitly for each kind of call. Where GPT-4o mini moves to Luna it can usually be set to none, which is how the older model behaved.
• Log token usage on every call, then price real conversations on both models.
• Collect every prompt, including the ones stored as data, and read them as if for the first time.
• Rewrite descriptive rules as procedures: what to do, and what to do when the context has nothing.
• A/B with retrieval held fixed, so you are testing the model and not your search.
• Deploy the code on the old model first, then switch the configuration, so rollback is one setting.

None of this is a reason to delay. The deadline is fixed and the newer models are better value. But the move is a re-test of your whole prompt surface by a stricter, and differently strict, reader, and it is far better to run that test in January's planning than in January's support queue.