Reasoning Models in Practice: Test-Time Compute and When It Pays
UIT University 365 Institute of Technology
Series AI Foundations
Duration 25 minutes
IT Engineering, AI and Applied AI, Data Science, Software Development, Digital Transformation
UNOP Sound (University 365 Neuroscience Oriented Pedagogy)
Take five minutes to prepare your brain. Play the isochronous tone track (40Hz gamma frequency) with your eyes closed. Gamma-frequency tones before a learning session raise attention and make the material easier to absorb.
[Audio player: UNOP Pre-Lecture Isochrone (40Hz, 5 minutes)]
In this Lecture
Reasoning Models in Practice: Test-Time Compute and When It Pays
The Dial You Did Not Ask For
Your assistant used to answer in four seconds. This week it takes forty, the answers look the same, and the invoice for the month tripled. Nobody touched the prompt. Somebody set an effort level, or a vendor turned thinking on by default, or a library you inherited switched a parameter it calls "reasoning" from medium to high because a release note said the new default was better.
In November 2026 every frontier vendor sells thinking as a setting rather than as a separate model: OpenAI exposes reasoning.effort, Anthropic exposes named effort levels, and Google replaced its thinking budget with a thinking level. The setting is one line in a request. The consequence is a bill, a delay and a quality difference that all move together, and none of the three is visible in the interface where most people choose the setting.
The pitch for more effort is easy to repeat: the model thinks longer, so it gets harder problems right. The pitch is true inside a range and false outside it. On one independently measured model, the lowest setting to the highest multiplies output tokens by fifteen and time per task by thirteen, for a quality index that rises from 26 to 55. Whether that trade is good depends on four things you can test, not on a preference.
This lecture gives you those four tests, the arithmetic behind the trade, and a one-page decision sheet you can defend in a meeting. It is written for someone who has to choose a setting per task and justify it to a manager who asks why the same task costs $66 one month and $532 the next.
Reasoning Models in Practice: Test-Time Compute and When It Pays
What the Model Does Before It Answers
A reasoning model writes before it answers. You send a request, and the model produces a span of internal tokens, sometimes visible as a summary, usually not, and then produces the answer you read. On current Anthropic models the thinking step is adaptive: unless you ask for it, the model decides per turn whether the problem needs it. On current OpenAI models the effort parameter tells the model how much thinking to do. On current Google models a thinking level does the same job and replaces the older token budget.
Three properties of that step decide your cost, and each one is easy to get wrong.
Thinking tokens are billed at the output rate. The output line is the expensive line on every rate card in this lecture, and thinking lands there. Input tokens describe your problem; output tokens include the model working on it.
Effort is a behavioural signal, not a token budget. Anthropic's effort documentation states it plainly: the same level can consume different token counts on different tasks. That sentence is the reason you cannot read a setting and predict a price. A high setting on a short, easy question can cost less than a low setting on a long, tangled one, because the model reacts to the problem in front of it.
The thinking budget competes with the answer. On Anthropic's Haiku 5.5, thinking is on by default and thinking tokens count toward max_tokens, which reaches 128,000. A long deliberation can therefore consume the room reserved for the answer. The failure looks like truncation, not like a cost problem, which is why teams misdiagnose it.
Put together, these three facts explain the surprise in the first paragraph. Nothing broke. A dial with a soft stop moved the work from the input side of the ledger to the output side, where a token costs five to ten times more.
Reasoning Models in Practice: Test-Time Compute and When It Pays
The Effort Ladder as It Stands in November 2026
Each vendor invented the same idea with different names, ranges and defaults. Read on 14 November 2026:
OpenAI's reasoning.effort accepts none, minimal, low, medium, high, xhigh and max, with the supported set depending on the model. GPT-5.5 accepts none, low, medium (the default), high and xhigh, and has no max. Its successor GPT-5.6 Sol adds max above xhigh.
Anthropic's effort accepts low, medium, high, xhigh and max, and the default differs by model: high on the Claude API for Sonnet 4.6, medium on Haiku 5.5. On the newest models the older control is gone: a request that sends budget_tokens returns a 400 error, and so does a request that tries to switch thinking off at xhigh or max.
Google's thinking_level accepts minimal, medium and high, and replaces thinking_budget. Gemini 3.6 Flash defaults to medium; Gemini 3.5 Flash-Lite defaults to minimal, the right shape for extraction, routing and classification at volume.
Two conclusions follow, and both change how you write a specification.
The default is not the cheap setting. A model whose default is medium or high has already made a cost decision on your behalf. Gemini 3.5 Flash-Lite defaults to minimal because Google built it for high-volume work; Anthropic's Sonnet 4.6 defaults to high on the API because Anthropic built it for difficult work. Inheriting a default is a choice you did not audit.
The names do not translate. A high on one vendor is not a high on another, and a medium after a model upgrade is not the same medium as before. The levels are labels over a behaviour, not shared units, so a policy that says "use high effort" without naming the model and the version will drift on the next release.
Reasoning Models in Practice: Test-Time Compute and When It Pays
The Measured Ladder: What Each Turn of the Dial Buys
Labels are not evidence. Independent measurement is. Artificial Analysis publishes, for each model, the same evaluation run at every effort level, which is the only shape that lets you see the marginal return of one more turn of the dial. Read on 14 November 2026 for Claude Sonnet 5.5, the numbers are these.
Effort | Intelligence index | Time per task | Output tokens across the run |
low | 26 | 2.6 minutes | 21.3 million |
medium | 31 | 4.0 minutes | 29.4 million |
high | 42 | 6.7 minutes | 54.7 million |
xhigh | 50 | 14.3 minutes | 111 million |
max | 55 | 33.8 minutes | 335 million |
The table is a curve, and the shape of the curve is the whole lecture. From low to high the model gains 16 index points for 2.6 times the output tokens, which works out at about 2.1 million extra tokens per point. From high to max it gains 13 points for six times the tokens, about 21.6 million per point. The last step is roughly ten times less efficient than the first three, and the published cost per task at max is $10.02 against 33.8 minutes of waiting.
Read it as a purchasing decision rather than a benchmark. The top of the ladder is not a rip-off, since some tasks need it and the same page ranks the model 13th of 80 at max. It is a specialist instrument with a steep price per point, and most work sits where the first three rungs are cheap.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Why the Best Amount of Thinking Depends on the Prompt
The reason the curve bends is not vendor pricing. It is the mathematics of search. A model that thinks longer is allocating more attempts, more checks and more revisions to one problem. On a problem where the first attempt is likely wrong, extra attempts find the right answer. On a problem where the first attempt is already right, extra attempts mostly re-confirm it, and occasionally talk themselves out of it.
The clearest published statement of this is Snell, Lee, Xu and Kumar, "Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters", submitted on 6 August 2024. The paper compares two ways of spending inference compute, searching against a process reward model and adaptively revising the answer, and finds that the effectiveness of each approach "critically varies depending on the difficulty of the prompt". Their conclusion is about allocation: a fixed budget spent the same way on every problem is the wrong shape, because difficulty decides which strategy pays.
That finding arrived before the product dials existed and predicted what they would expose. A single global effort setting is a compromise: it overspends on the easy third of your traffic, underspends on the hard tenth, and the average looks acceptable while both tails waste money.
Two practical consequences follow for anyone setting a policy in 2026.
Route by difficulty before you route by price. If you can tell, from the request itself, that a task is a lookup or a reformatting, you already know the answer is a low rung. The expensive mistake is sending everything to one setting because one setting is easier to govern.
Measure on your own distribution. A vendor's ladder is measured on a benchmark suite with a particular mix of difficulties. Your traffic has a different mix. The rung that pays for you is an empirical question about your cases, and no published table can answer it.
Reasoning Models in Practice: Test-Time Compute and When It Pays
The Overthinking Problem
If extra thinking only ever helped, the ladder would be a simple spending decision. It also hurts. Chen and twelve co-authors published the first broad study of the failure in "Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs", submitted on 30 December 2024 and revised on 1 February 2025. Their finding is that these models allocate "excessive computational resources for simple problems with minimal benefit", and they introduce efficiency metrics that separate the quality of an answer from the rationality of the process that produced it.
The paper's own remedy is a self-training method that shortens reasoning without losing accuracy. You do not need the method, only the finding, to justify a floor on your effort policy: for a task the model can already do, a level above the floor buys latency and tokens and returns nothing measurable.
Overthinking has a second cost that the paper's metrics make visible and that your users feel directly. A model that deliberates for four minutes on a question a junior colleague answers in ten seconds reads as broken, even when the answer is correct. The credibility loss is real and it is not in the token invoice.
The practical test is blunt. Take twenty cases you already know the right answer to. Run them at the lowest rung and at the rung you are considering. If the answers agree, the extra rung is buying insurance you have not needed yet, and the only open question is whether the cases you cannot check look more like the easy twenty or less.
Reasoning Models in Practice: Test-Time Compute and When It Pays
What a Thought Costs
Thinking is priced at the output rate, context is priced at the input rate, and a repeated context is priced at a cache rate that is a fraction of both. The November 2026 rate cards make the relationship concrete.
Model | Input per million | Output per million | Cache read per million |
Claude Sonnet 5.5 | $2.00 | $10.00 | $0.10 |
Claude Opus 5.5 | $4.00 | $20.00 | $0.20 |
Claude Fable 5.2 | $8.00 | $40.00 | Not quoted here |
GPT-5.5 | $5.00 | $30.00 | $0.50 |
GPT-6.1 Sol | $2.00 | $10.00 | $0.10 |
Gemini 3.6 Flash | $1.50 | $7.50 | Not quoted here |
The cache column is the one most teams under-use. On Sonnet 5.5 a cached read costs five percent of the input rate, and on Opus 5.5 and GPT-5.5 it costs ten percent. In an agent that resends a long instruction block on every turn, the cache discount is often a larger number than the effort decision.
A single comparison makes the point. Thirty thousand tokens of instructions resent across twelve turns is 360,000 input tokens per task. On Sonnet 5.5 that is $0.72 uncached and $0.036 with the cache hit. On GPT-5.5 it is $1.80 uncached and $0.18 cached. No effort setting in this lecture moves a bill by a factor of twenty; the cache does, and it is usually already available.
Reasoning Models in Practice: Test-Time Compute and When It Pays
A Worked Invoice: Three Settings, One Task
Numbers become decisions when they land on one task. Take a brief of 1,200 words drafted from a 4,000-token source pack, on Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens. Assume, for the sake of the arithmetic, that the three settings produce the token counts below. The prices are the vendor's; the token counts are an illustration you should replace with your own measurement.
At the lowest rung the model writes 900 thinking tokens and 1,600 answer tokens. That is 2,500 output tokens, $0.025 for output plus $0.008 for input, about $0.033 per task. At a middle rung the thinking grows to 4,000 tokens and the answer to 1,700, which is $0.065 per task. At the top rung the thinking grows to 24,000 tokens and the answer to 1,800, which is $0.266 per task.
Multiply by 2,000 tasks a month and the three settings cost $66, $130 and $532. The difference between the cheapest and the most expensive is $466 a month, which is a real number and not a catastrophe. Multiply by 200,000 tasks a month, which is the volume of a support classification queue or a document pipeline in a mid-sized organisation, and the same difference becomes $46,600 a month.
Two lessons come out of that invoice, and they hold whatever your real token counts are.
The dial is cheap on small volume and expensive on large volume. A team of five drafting documents will not notice high effort. A pipeline that runs a million tasks will notice it before it notices anything else in the budget.
The answer length barely moves. Across the three settings the answer grew by 12 percent while thinking grew by a factor of 26. If your users judge quality by length, the dial changes almost nothing they can see.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Latency Decides Before Price Does
A cost you can see is easier to argue about than one you cannot, and latency is the second kind. The run behind the ladder table records 2.18 seconds to first answer token at low and 272.39 seconds at max, with time per task rising from 2.6 to 33.8 minutes.
Four and a half minutes before the first visible token is not a slow answer. It is a different product: interactive tools lose users, agents accumulate turns, and a queue that holds a customer for five minutes per ticket stops being a queue.
Three latency facts deserve a place in your policy.
Time per task is the number to quote. Time to first token matters for chat. Time per task matters for anything with a completion target, and the two diverge: at max the measured run spent 272 seconds before the first answer token and 33.8 minutes on the task.
Agent turns multiply the wait. A twelve-turn agent that waits 30 seconds per turn costs six minutes before any retry, so a high setting inside a loop is a high setting multiplied by the loop count.
Waiting has a wage. A specialist who waits 30 minutes for a draft is not a $0.27 token event: if the wait blocks one hour of a 60 euro per hour professional, the delay costs more than a month of tokens at any setting here. Put the human cost in the same table, or the comparison is not a comparison.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Four Tests Before You Raise the Dial
The ladder, the curve and the invoice now give you a decision procedure. Before you move a task from a low rung to a higher one, answer four questions. All four have to come back yes.
One: is the end state checkable? Can you write down, before the run, what a correct answer contains, and can a person or a script confirm it? Code with tests, arithmetic with a key, a reconciliation with a control total and a data extraction with a schema all qualify. A strategy memo does not.
Two: do errors compound across steps? A task with eight dependent steps and a 90 percent success rate per step succeeds about 43 percent of the time. That arithmetic is why reasoning pays on multi-step work: a better per-step decision improves the product more than the sum of the improvements suggests. On a single-step task there is nothing to compound.
Three: is the failure cost above the token cost? A $0.27 request that prevents a $400 rework is the easiest purchase in the building. A $0.27 request that improves a sentence nobody will litigate is a rounding error with a latency cost attached.
Four: is the latency budget compatible? If the answer has to arrive in two seconds, the top of the ladder is excluded no matter how good it is. If the answer is due tomorrow morning in a batch, latency is free and the only remaining question is quality per dollar.
When all four come back yes, raise the rung and measure the result. When one comes back no, the fix is usually not in the dial: it is in the prompt, the context you supply, or the decomposition of the task into steps a cheaper setting can handle.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Task Families and the Effort Each Deserves
The tests are general; the settings are specific. Group your work into families with the same shape.
Classification, routing and extraction. The answer space is closed and the evidence is in the text you supplied. Use the lowest rung, or a non-reasoning model. This is where the overthinking literature applies most directly, and where Google's minimal default on a Flash-Lite model is the right shape.
Drafting against a brief. The model has the facts and needs prose in a voice. Use a middle rung and spend your effort on the brief and the examples. The answer barely grows, so the reader sees little change.
Multi-step analysis with numbers. A financial reconciliation, a unit-economics model, a comparison across sources. Use a high rung, and check the arithmetic independently. This is the family where the measured quality gain is largest relative to the token cost.
Code with tests. Use a high or very high rung and let the test suite be the judge. On GPT-5.5 the independent Terminal-Bench 2.1 score runs from 61.05 percent in the non-reasoning setting to 84.27 percent at xhigh, which is the clearest published case for the dial in this lecture.
Open-ended judgment. Strategy, diagnosis, ethical framing, a decision memo. A high rung helps the model hold more considerations at once, and it cannot supply the judgment. Budget human review in the same plan, or the effort was decoration.
Long-horizon agents. Many turns, tool calls and side effects. The top rung is defensible here only with a verifier, a step budget and a stop condition. Without those three, extra thinking extends a trajectory that was already going the wrong way.
Reasoning Models in Practice: Test-Time Compute and When It Pays
When Reasoning Pays: The Checkable End State
Strip the families down and one property predicts most of the value: whether the task has a checkable end state. Reasoning is a search over candidate answers, and a search is only as good as its ability to tell a good candidate from a bad one.
The GPT-5.5 evidence is worth stating precisely because it comes from independent measurement rather than a launch page. In the same evaluation suite, the model's GDPval Elo rises from 978 in the non-reasoning setting to 1,357 at xhigh, and Terminal-Bench 2.1 rises from 61.05 percent to 84.27 percent. Those are agentic and occupational tasks with verifiable end states: a task completed, a test passed.
The rule that follows is short enough to memorise. If you can write the test, effort buys accuracy. If you cannot, effort buys prose. That is not an argument against prose. It is an argument against paying the search premium for it.
There is a second condition that teams forget. The verifier has to be independent of the model that did the work. A reasoning model checking its own arithmetic will agree with itself, and the agreement is not evidence. Where the check is the same model reading its own answer, you have bought confidence, not correctness.
Reasoning Models in Practice: Test-Time Compute and When It Pays
When Reasoning Does Not Pay
The inverse list is as practical as the positive one, and it is where most of the money sits.
Reasoning does not pay for tone, house style and rewriting, where the constraint is your voice and the model already knows the language. It does not pay for translation between two well-resourced languages. It does not pay for summarising a document you supplied, because the information is already bounded by the source. It does not pay for retrieval questions with a good source and a clear answer. It does not pay for schema extraction with a fixed output shape, and it does not pay for classification into a closed set. In all seven cases the answer is determined by the material, and deliberation mostly re-reads it.
There is also a harder case, where more effort makes the output worse. Anthropic reports that Sonnet 5.5 scored lower at max than at xhigh on FrontierCode, because at the top setting it more often invoked a code-review skill that split the work across subagents, and in two cases that led to a timeout or to edits outside the scope of the task. The vendor's own explanation is an orchestration effect rather than a reasoning failure, and the practical lesson survives either way: the top rung changes how the model works, not just how long it works, and the change can cost quality.
That is why the ladder needs a ceiling in your policy, not just a floor. The ceiling is the rung beyond which you have measured no gain, and it protects you from a setting that a future release may reinterpret.
Reasoning Models in Practice: Test-Time Compute and When It Pays
The Sweep That Justifies the Setting
A defended setting is a measured setting, the measurement fits on one page, and it has five parts.
Cases. Forty examples of the real task, drawn from production traffic and labelled with the outcome you would accept. Include the easy ones: a sweep on hard cases only will over-recommend the top rung, which is exactly the bias the overthinking literature warns about.
Levels. Four effort levels, from the lowest your provider offers to the highest you would ever allow. Four is enough to see the bend in the curve and few enough to stay cheap.
Repeats. Three runs per case per level, because reasoning models vary between runs and a single pass will show you noise as signal.
Measures. Pass rate on the labelled outcome, cost per completed task including retries, and the 95th percentile latency. Cost per completed task is the only cost number that survives contact with a manager, because it charges failures to the setting that produced them.
The rule. Choose the lowest rung whose pass rate is within your noise band of the best rung. Write the band down before the run.
The band is arithmetic, not instinct. With 40 cases and a true pass rate near 90 percent, the standard error of the measured rate is about five points, and repeats tighten it only modestly because the cases are the same 40. A four-point gap is inside the noise; a 15-point gap is not. State the band and the sample size beside the result, or the next reviewer will read noise as a finding.
Then record the model, its version, the rung, the date, the pass rate and the cost per completed task where the next person will find it. A setting without that record expires silently on the next release.
Reasoning Models in Practice: Test-Time Compute and When It Pays
The Price Cliffs That Outrank the Dial
Three pricing structures move a bill more than the effort dial does, and all three are easy to miss because they sit outside the per-token headline.
The long-context cliff. GPT-5.5 prices prompts above 272,000 input tokens at $10 per million input and $45 per million output, and the higher rate applies to the whole request, not to the tokens above the line. A 300,000-token prompt costs $3.00 in input against $1.50 at the standard rate, so a change that feels small doubles the input line.
The retry loop. A failed run is billed, and reasoning failures are expensive because they burn the thinking budget before failing. Two extra points of failure rate on a 100,000-task queue costs more than the price gap between two rungs, which is the strongest argument for cost per completed task.
None of these three is an argument against reasoning models. Each is an argument for reading the rate card before the benchmark page.
Reasoning Models in Practice: Test-Time Compute and When It Pays
What Breaks When You Raise the Dial
The dial changes more than a number: on the current generation it changes what the API accepts and how the response ends, and that breaks integrations written for the previous generation.
The output ceiling becomes shared. When thinking tokens count toward max_tokens, a request with a tight output limit can finish with the ceiling consumed and no visible answer. The response is not an error; it is an empty or truncated answer that reads like a model failure. Raise the output limit when you raise the rung, and assert on the text you expect rather than on the HTTP status.
The knobs you relied on may be gone. On Anthropic's newest models, budget_tokens returns a 400 error, and so do temperature, top_p and top_k from API version 2026-10-21, along with assistant prefill and forced tool choice. A client that pinned temperature to zero now fails at the first call.
Refusals can arrive without a fallback. Haiku 5.5 returns a stop_reason of refusal that earlier models did not return, and there is no server-side fallback behind it. A pipeline that treats every non-error response as an answer will store the refusal as content.
None of this is a reason to stay on an old model. It is a reason to treat an effort change as a release: run the suite, check the ceiling, re-read the parameters the vendor accepts, and compare cost per completed task before and after.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Six Mistakes That Survive Every Review
Four errors appear in almost every effort policy that reaches a review, and each has a one-line correction.
Comparing a new rung against a different model. The gain came from the model and is credited to the setting. Hold the model and move one variable.
Measuring on hard cases only. Everything then looks like it needs the top rung. Include the easy third of your traffic.
Leaving the default unnamed. A vendor default changes on a release you did not attend, and the invoice changes with it.
None of the four needs new tooling. All six need the setting to be treated as a decision, which is the argument of this lecture.
Reasoning Models in Practice: Test-Time Compute and When It Pays
A Decision Sheet You Can Defend
Everything in this lecture collapses into six lines per task. Write them once, review them quarterly, and you will spend less time defending the setting than you spend choosing it.
Task. One sentence naming the work and its volume per month.
Model and version. The exact model string, because a level name does not survive an upgrade.
Effort level. The rung, with the date it was measured.
Evidence. Pass rate at the chosen rung against the rung above, the number of cases, and cost per completed task at each.
Ceiling and trigger. The rung you will not exceed, and the signal that would justify moving up: a pass rate below a threshold, or a failure the cheaper rung produces reproducibly.
Owner and review date. The person who answers for the setting, and the date the evidence expires, which is the next model release or the next quarter, whichever comes first.
A filled line reads like this: "Contract clause extraction, 40,000 documents a month. Claude Sonnet 5.5, effort low. Measured 14 November 2026 on 40 labelled documents: 94 percent pass at low, 95 percent at high, cost per completed document $0.004 against $0.021. Ceiling high, triggered by any month below 90 percent. Owner: legal operations. Review: next model release."
That line is what the dial was missing. The setting is now a decision with a price, a date and a name attached to it, and the next person who asks why the invoice moved will get an answer instead of a hypothesis. The next lecture in this series turns the sheet into a routing policy, where different items in the same queue take different rungs without anyone choosing per request.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Feynman Summary: Explain It Like You Are 12
Imagine two students taking the same test. One answers at once from memory. The other writes a plan, checks the plan, tries an answer, checks the answer, then writes it out. On a hard question the second student usually wins. On a question whose answer is printed on the page, the second student wastes ten minutes and sometimes talks himself into a wrong answer.
A reasoning model is the second student, and the effort setting is how much scratch paper you allow. You pay for the paper by the sheet, at the price of the finished answer, and you wait while it is used. Raise the setting when the question is hard and the answer can be checked. Leave it low when the question is easy, when the answer is in the material you handed over, or when someone is waiting at the counter.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Mindmap: The Complete Picture
The map puts the four tests at the centre and hangs the cost lines, the task families and the failure modes off them. Use it as a checklist before you change a setting.
UNOP Sound (University 365 Neuroscience Oriented Pedagogy)
Take five minutes to consolidate your memory. Play the isochronous tone track (10Hz alpha frequency) with your eyes closed. Alpha-frequency tones after a learning session support consolidation, helping move what you just learned from short-term to long-term memory.
[Audio player: UNOP Post-Lecture Isochrone (10Hz, 5 minutes)]
Reasoning Models in Practice: Test-Time Compute and When It Pays
Practical Exercise: Price One Task in Ninety Minutes
The exercise takes one real task and produces one defensible line of the decision sheet.
Pick the task and count it. Choose a task that runs at least 200 times a month and write its monthly volume. A setting that cannot be justified at volume is a preference.
Label twenty cases. Take twenty real inputs and write the outcome you would accept for each: twelve typical, five hard, three that have failed before.
Run four levels. Run each case at four effort levels, three times each, and record pass or fail, tokens in, tokens out and wall-clock time. Keep the log, because someone will challenge the result.
Compute cost per completed task. Multiply tokens by the rate card, add retries, divide by the cases that passed. Put the four numbers in one row with the four pass rates above them.
Choose and write the line. Take the lowest rung inside your noise band, fill the six lines of the decision sheet, and file them with the runbook.
Send the finished sheet through the U365 Fellow portal if you want a second reading. The interesting part of the review is usually not the rung you chose but the cases you decided to exclude.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Glossary
Term | Definition |
Reasoning model | A model that generates internal thinking tokens before it writes the answer a reader sees. |
Test-time compute | The computation a model spends while answering one request, as opposed to the computation spent training it. |
Reasoning effort | A named setting such as low, medium, high, xhigh or max that tells a model how much thinking to do. It is a behavioural signal, not a token budget. |
Thinking level | Google's version of the same control, with the values minimal, medium and high, which replaced the older thinking budget. |
Adaptive thinking | A mode in which the model decides per turn whether a request needs deliberation, instead of being handed a fixed allowance. |
Thinking tokens | The tokens a model produces before its answer. They are billed at the output rate and count toward the output ceiling. |
Cost per completed task | Total spend, retries included, divided by the number of tasks that passed. The only cost figure that survives a budget review. |
Long-context cliff | A rate threshold above which a whole request is billed at a higher price, for example prompts above 272,000 input tokens on GPT-5.5. |
Cache read | A repeated input token charged at a fraction of the input rate, between five and ten percent on the models in this lecture. |
Overthinking | Spending heavy computation on a problem that needs little, with minimal benefit to the answer and a real cost in time. |
Noise band | The pass-rate difference that a small sample cannot distinguish from chance, about five points on 40 cases at a 90 percent rate. |
Verifier | An independent check that separates a correct answer from an incorrect one. A search is only as good as its verifier. |
Reasoning Models in Practice: Test-Time Compute and When It Pays
Quiz: TEST YOUR UNDERSTANDING
1. What does the reasoning effort setting actually control?
A) The model version that serves the request
B) How much thinking the model does before it answers
C) The price charged per thousand tokens
D) The language of the answer
2. Which line of the invoice absorbs the thinking tokens?
A) Input tokens
B) Cached input tokens
C) Output tokens
D) They are free of charge
3. On the measured Sonnet 5.5 ladder, what happens between high and max?
A) Quality rises faster than tokens
B) The gain per extra token falls by roughly a factor of ten
C) Output tokens fall
D) Latency falls
4. Which task is the strongest case for a high effort level?
A) Rewriting a paragraph in house style
B) Classifying a support ticket into one of six categories
C) Fixing a failing unit test until the suite passes
D) Summarising a policy document you supplied
5. A sweep on 40 cases shows 94 percent at low and 98 percent at high. What is the defensible reading?
A) Adopt the high rung immediately
B) Treat the four-point gap as inside the noise band and keep the low rung
C) Double the token budget
D) Switch to another vendor
Answers: 1-B, 2-C, 3-B, 4-C, 5-B
Related Resources
U365 INSIDE Publications
External Resources
Related U365 Lectures (Coming Soon)
Routing by Difficulty: One Queue, Several Effort Levels (AI Engineering)
The Cost of a Thought: Unit Economics for AI Workflows (Finance Series)
Evaluating Agents: Traces, Tools and Side Effects (AI Engineering)
Reasoning Models in Practice: Test-Time Compute and When It Pays
U.Copilot for This Lecture
You are my cost and quality analyst for one AI task. USER-PERSONA: I am a manager responsible for an AI workflow and its budget. FULL-CONTEXT: Task: [describe the unit of work in one sentence]. Monthly volume: [number]. Current model and effort setting: [model string and level]. Current cost per completed task: [amount or unknown]. Current p95 latency: [seconds or minutes]. Failure cost when the output is wrong: [amount or description]. AI-PROFILE-ROLE-EXPERTISE: Act as an AI operations analyst who knows the effort ladders of the frontier vendors as of November 2026 and refuses to invent numbers. USER-FINAL-GOAL: Recommend one effort level for this task and defend the cost. AI-EXACT-TASK: Do five things. 1. State which of the four tests the task passes: checkable end state, compounding error, failure cost above token cost, compatible latency budget. 2. Design a 20-case sweep with four effort levels and three repeats, and list the cases by category. 3. Tell me which numbers to record per run and how to compute cost per completed task including retries. 4. Give the decision rule for choosing a level, including the noise band for 20 cases. 5. Draft the six lines of the decision sheet: task, model and version, effort level, evidence, ceiling and trigger, owner and review date. AUDIENCE: My operations lead and my finance partner, neither of whom will read a benchmark table. EXPECTED-FORMAT: Six short sections with the headings above, then the decision sheet as a table. No preamble. EXAMPLES: One filled example line for a contract clause extraction task at 40,000 documents a month; one filled example line for a support classification task at 200,000 tickets a month; one counter-example showing a task where the low rung wins. Ask me one clarifying question before you start if the task description is ambiguous.
Reasoning Models in Practice: Test-Time Compute and When It Pays
Next Steps
Pick one task that runs at least 200 times a month and write its monthly volume today.
Label twenty cases, twelve typical and eight hard, before you change any setting.
Run the four-level sweep with three repeats and compute cost per completed task, retries included.
Write the six-line decision sheet, including the ceiling and the review date, and file it with the runbook.
Send the sheet through the U365 Fellow portal if you want a second reading before the policy goes live.
Reasoning Models in Practice: Test-Time Compute and When It Pays
IMPORTANT NOTICE
Copyright University 365, Inc. All rights reserved. This lecture is provided for educational purposes and does not constitute legal, financial or professional advice. For enrollment, visit university-365.com/tuition. For permissions or inquiries, contact uda@university-365.com.
Published by the Department of Academics, University 365.
Lecture delivered by the University 365 Institute of Technology (UIT).
Sam Utteker, Dean of Technology, UIT
Signed for the academic year 2026.
Comments