The model does not write the number

You ask a language model a question about your data. It answers with a number. The number is plausible, and it is wrong. Nobody signs off on a board decision on the strength of a plausible number, and that is what blocks more enterprise deployments than budget and regulation combined.

The answer is not to ask more politely. It is to take away the model's ability to be wrong about the number at all: it does not write the answer, it writes the query. The database answers what it actually contains. The shift looks small; it changes the nature of the error. A wrong query is visible, loggable, replayable and fixable. A hallucinated number is invisible, because it looks exactly like a correct one.

This article follows four questions really put to an enterprise deployment, in the order they come up in a meeting: last month's number, a row-by-row prediction, a three-month forecast, then a chart. Three storeys on one foundation, each named like an address: txt2sql for the number, txt2sql2ml for the prediction, txt2sql2viz for the figure. The first storey is well-trodden industry ground; the other two are far rarer, and they are the ones I built.

"How many invoices are overdue?"

The answer lands in two seconds: thirteen. And, unfolded underneath it, the query that produced it.

SELECT COUNT(*) FROM invoices WHERE status = 'overdue' LIMIT 500

Showing that query is not a flourish of transparency. It is the evidence: a reader who knows the database can read it, dispute it, replay it. The number is not there to be believed, it is there to be checked.

Writing SQL in general is something a recent language model does very well. The difficulty lies elsewhere, and it is the whole difficulty: writing the right query for this database. The right table names, the right joins, and above all the right filter values. Is an unpaid invoice's status spelled overdue, unpaid or 3? A model that does not know the schema invents names that are plausible and wrong. That is where all the value of a text-to-SQL tool sits: anchoring generation in the reality of one given database.

Two families of solution exist. Fine-tuning retrains a model on the customer's data: expensive, slow, and to be redone at every schema change. Retrieval-augmented generation leaves the model untouched and hands it, for each question, the context that matters². Vanna, the open source reference on this ground, belongs to the second family¹.

One point is disconcerting from the outset, and it is the most important: in Vanna, the word "train" means filling a knowledge base that the system consults at answer time. No neural network weight is modified. You file texts away, you retrieve them by similarity of meaning, you paste them into the prompt³.

Three materials, in the reverse order of intuition

Three kinds of content go into that knowledge base, and intuition gets their relative weight wrong.

The table structure comes first in time: each table's CREATE TABLE, its columns, types and keys. It teaches the model what exists and stops it inventing. One table per entry, so each is retrieved independently.

The business glossary encodes what the structure does not say: the definition of a measure, in-house vocabulary, the meaning of coded values. One self-contained statement per entry, phrased the way a user would say it. "Revenue excluding tax on orders is the sum of orders.total_ht_eur." This is where the largest gains sit, precisely where a model without context goes wrong most often.

The question-to-query pairs come last in the order of work and first in the order of payoff. A question in plain language, the correct query beside it. Vanna's documentation says it without hedging: their quality is the single largest determinant of the result¹. The mechanism explains why. Only what is retrieved influences a given answer; the knowledge base is never poured into the prompt whole. One correct example close to the question asked is therefore worth a thousand distant ones. You curate for coverage, not for volume: twenty to a hundred verified pairs are enough to start, provided they cover the families of query users will actually produce, from counting to joining, from filtering on a coded value to ranking the top N⁹.

Two curation rules dominate, and the second is a safety rule. Always verified SQL, meaning actually executed and confirmed correct: a wrong pair teaches the system to produce wrong SQL. And never SELECT *, which teaches nobody a single column name.

Two locks between the model and the database

The model produces a query, and nothing more. Execution happens elsewhere, under a read-only database role, behind guardrails that parse the query before running it. Parsing here means reading it the way a compiler would, as a syntax tree⁴, and refusing anything that is not a read. The separation is a security decision, not an aesthetic one: the component that talks to the model never has the right to write to the database.

The second lock was born of a precise failure, dated 4 September 2026. Until then the model received column names and types, never their domain, the set of values they actually take. So it wrote WHERE status = 'confirmed' against the invoices table. Syntactically valid. Accepted by the guardrails. Executed without error. Zero rows. The real values were paid, overdue and sent; confirmed belonged to the orders table, a different table. The user saw an empty result with no explanation whatsoever.

That flaw is fixed from both ends. As prevention, every low-cardinality column now carries its real domain, read from the database and written into the context given to the model: first the enumerated types, which the database knows exactly and for free, then the text columns the database statistics declare low-cardinality, whose domain is re-read with a bounded query. A column with no statistics is never probed: no domain beats a wrong domain. As a cure, a query that runs without error but returns no rows, and that filters on a literal absent from the known domain, no longer hands back a silent blank; a second attempt is made with the real values given to the model, and failing that the answer names the column, the invented literal and the real domain.

Proving that all this work is worth something

This is the step people skip, and the only one that proves the curation paid. You hold back a few pairs you do not train on, a control set. Measured on questions already filed away, accuracy says nothing beyond the system's ability to regurgitate a memorised example.

Three metrics are available, from weakest to most reliable. Valid-SQL rate is a mere guardrail: a query can be valid and wrong. Exact textual match, which compares the two queries character by character, badly undercounts accuracy, since two correct queries are almost always written differently⁵. The right metric is execution match: you run both the generated query and the reference query read-only, and compare the results, order aside. It is the metric of the field's benchmarks, Spider⁶ and BIRD⁷, because it judges the answer delivered, which is the only thing that matters.

One global score, on its own, hides the family of queries that fails⁸. So you break it down by pattern, and that reading drives what comes next: weak joins call for join examples, not for a bigger model. That is exactly what happened here, and it is the jump from 87.5 % to 100 % on the chart.

"Which customers are at risk of leaving, and how confident are you?"

An exact number answers a question about the past. How much did we invoice last month, which customers went over budget. It does not answer the one the executive asks next, which is the one that decides something. Until now that second question meant a data-science project of several months, per customer and per question.

The idea that changes this fits in one sentence. The query no longer brings back a number: it brings back a table whose last column is the question, known for some rows, empty for the ones to predict.

One question, one query, and the gaps fill in. SQL cannot predict; it can fetch the right shape of table.

That table holds the example and the question at once. Which is exactly what a tabular foundation model knows how to read. Such a model was pre-trained once and for all, at its laboratory, on synthetic tables; it does not know your data, it knows how tables behave. Show it this one and it fills the gaps in a single pass, with no gradient descent, no training on your data, no model to store.

This is called in-context learning, and the analogy that lands is one everybody has already lived: giving a chat assistant an example inside the conversation itself, and watching it catch on without retraining¹⁰. The analogy has a limit, and the limit is instructive: text reads left to right, a table has no order, neither across rows nor across columns. That is precisely the problem these models had to solve.

The direct consequence: zero training projects, zero artefacts to maintain. The day you plug in a new customer, there is no model to retrain. One and the same code covers classification, regression, gap filling and top-N ranking.

Two European schools, and why I picked the French one

This ground is held by two teams, and both are European. The German school, in Freiburg, around Frank Hutter, produced TabPFN, published in Nature in January 2025¹¹. Its industrial arm, Prior Labs, was acquired by SAP: announced in May 2026, closed on 17 July, with more than a billion euros committed over four years¹². A German vendor putting a billion into an eighteen-month-old German laboratory validates the subject better than any argument I could make.

The French school is at Inria Saclay, in the SODA team, around Gaël Varoquaux, co-creator of scikit-learn. Its model is called TabICL. Across two hundred classification datasets it matches TabPFN v2 while being consistently faster, by up to ten times, and it overtakes it on tables of more than ten thousand rows¹³.

I picked TabICL, and not out of patriotism: the code and the weights are published¹⁴. It is the only one of the two a customer can audit line by line and run on their own machines, with nothing leaving the building. When you answer "this customer is about to leave" to an executive, being able to show where the answer came from is not a nicety.

Knowing who is leaving without knowing why is useless. Every prediction therefore comes with the weight of each variable and its direction, by both reference methods: Shapley-value attribution, borrowed from game theory, which splits the prediction across variables the way a payoff is split across players¹⁵, and local surrogate explanation, which approximates the model around the row under examination with a simple, readable one¹⁶. Both are shown side by side: when they disagree, that is information in itself.

"What is the revenue forecast for the next three months?"

Same foundation, same model, different question. This time the query brings back a series through time, and the answer is about months that do not exist yet.

A single number about the future is fortune-telling; a number with its range is a decision. And this particular chart says something uncomfortable that has to be read properly: the range is wider than the gap between the three months. August looks better than July, but the bars overlap almost entirely, so the gap between them is not interpretable.

That is not a flaw in the tool, it is a verdict on the data. The monthly series in this demonstration database is noise: over twenty-one months, 293 euros in one month against 72,380 a year later, and three months missing. No method can forecast that, and the only honest answer is a wide range. A tool that had returned three neat, tidy numbers would have dressed the situation up. It is the same principle as on the first storey: the guarantee is not about the quality of the answer, it is that the answer cannot lie about its own uncertainty.

One implementation detail says a lot about the world you work in. The forecast horizon is read out of the user's question, which is to say out of free text. "In a hundred thousand months" translated into a hundred thousand periods to forecast, which means out-of-range dates and a computation that ties up the machine for a result nobody seriously asked for. The horizon is now capped: past that point a forecast has no statistical value anyway, so bounding it removes nothing real.

"Show me where revenue is concentrated"

Last question, last link different: the result is no longer a number or a table, it is a figure. It is chosen automatically from the shape of the data and the intent of the question, and rendered live, with no pre-built dashboard to maintain. That distinction has a price for whoever decides: a dashboard answers the questions of six months ago.

Except that this choice is exactly where a language model can do anything at all. Show it a hundred and twenty-seven chart names and it can invent one, misspell one, or pick one incompatible with the columns present, a map for instance when no column carries geography. The guarantee therefore cannot be a promise of good behaviour. It has to live in the structure.

The model picks a dish from a menu it did not write. That is a guarantee of a different order from a temperature setting or an instruction added at the top of the conversation: it does not depend on the model's compliance, it depends on the shape of the program. And, as on the first storey, it makes the remaining error visible and tractable: a misranked figure is fixed with one click, it does not blend in with a correct one.

The rendering runs on sprezzature.ai, an open source figure engine¹⁹. Its catalogue holds a hundred and twenty-seven chart kinds, from lines to sankeys, treemaps, waterfalls, beeswarms and maps. This is not greed: the wider the catalogue, the richer the list of candidates compatible with a given set of columns, and the more often the automatic pick lands right.

Every figure also meets a standard of accessibility and chromatic rigour: contrast checked, readable under colour blindness, never a colour alone carrying information, polarity stated on any axis that has a good direction. None of that is cosmetic. A poorly contrasted chart projected in a room is read by nobody; and a chart that requires telling two greens apart excludes one man in twelve. The rules inherited from graphical semiology¹⁷ and from perception experiments¹⁸ are code here, not recommendations.

What production learns and preparation cannot guess

No amount of upfront curation knows how users will phrase their requests, nor the turns of phrase, shortcuts and special cases proper to each organisation. Production reveals that real distribution of questions. Every question handled well, and above all every correction, is information the initial curation did not have.

The loop is simple to state: a person asks a question, the system produces the query, it runs read-only, a human validates the result or fixes the query, and only once validated does the pair join the index. The next similar question then retrieves that new example.

The safety rule is absolute and it is the easiest to get wrong: never blind learning. Automatically capturing every generated query and feeding it back as an example poisons retrieval for every similar question to come. Bad SQL teaches the system to produce bad SQL. A pair is therefore only accepted through a dedicated feedback path, which replays the guardrails and re-executes the query before accepting it: no blind trust, not even in what the caller presents as already verified.

Four properties separate a mechanism that works on one laptop from a production loop. Persistence, because the index lives in the database already in place²⁰ rather than in a cache lost on the first restart. Idempotence, a pair's identifier being derived from its content, so resubmitting it creates no duplicate to clog retrieval. Traceability, every field-sourced pair being marked as such, distinct from the curated corpus, which makes it possible to audit what production taught the system and to remove one bad example. And measurement: after a batch of additions, you replay the control set. Progress is verified, not assumed.

What this changes, and what it does not

Before, a business question leaves as a ticket for the data team and comes back in three days, often after the decision has been made without it. A predictive question leaves as a project. After, the question is asked out loud, the number lands in seconds, and so does the follow-up: the meeting keeps its rhythm. What matters is not that the machine does analysis, it is that the delay between the question and a reliable number drops from three days to three seconds without any loss of reliability, since the number comes from the database.

What it does not change deserves to be said just as plainly. The model can still write a bad query: the guarantee is about the visibility of the error, not its absence. The real limit is not data volume, since the pipeline goes at the database rather than an export, and it is SQL that does the filtering; the limit is the quality of the schema and the vocabulary. A database whose columns are called col_17 will not let itself be queried in plain language, and no model will change that.

The three lines

txt2sql question → SQL → database → the real number txt2sql2ml question → SQL → table with gaps → the prediction and its range txt2sql2viz question → SQL → rows → the right chart, live

One foundation, three storeys. At each one, the same discipline: the model proposes the shape, the machine checks it, and the data settles it. That is what separates a number you can sign from a number that merely looks right.

Bibliography

The works below are the ones this pipeline actually rests on, in the order the article calls on them. Each note says what the source establishes and where it is used here.

1. Vanna AI. Training advice, project documentation. vanna.ai/docs/training-advice States without hedging the hierarchy of the three materials and the safety rule taken up here: the quality of question-to-query pairs is the single largest determinant of the result, and bad SQL learned teaches the system to produce bad SQL.
2. Lewis, P. et al. (2020). "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", NeurIPS. arXiv:2005.11401 The paper that named retrieval-augmented generation. It establishes the principle used on the first storey: leave the model untouched and supply, for each question, the context that matters, rather than retraining its weights.
3. Vanna AI (2023). How Vanna works, how to train it, data security. medium.com/vanna-ai Walks step by step through the retrieval mechanism everything else depends on: the question is embedded, the nearest knowledge is pulled per content type, and only that enters the prompt. This is where the rule "coverage beats volume" comes from.
4. Mao, T. sqlglot, SQL parser and transpiler in Python, MIT licence. github.com/tobymao/sqlglot The parser that turns the generated query into a syntax tree before any execution. It is what makes the guardrails possible: you inspect the structure of the query, not a string, and refuse anything that is not a read.
5. Pourreza, M. and Rafiei, D. (2023). "Evaluating Cross-Domain Text-to-SQL Models and Benchmarks", EMNLP. arXiv:2310.18538 Shows that benchmark scores underestimate models, because a natural-language question admits several correct queries. Justifies the choice of execution match over textual comparison, made here.
6. Yu, T. et al. (2018). "Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task", EMNLP. arXiv:1809.08887 The benchmark that set the field's habits, starting with evaluation on databases never seen in training. The held-out control set described here is that idea transposed to the scale of a single customer.
7. Li, J. et al. (2023). "Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs", NeurIPS. arXiv:2305.03111 The BIRD benchmark, built on real databases, dirty and large. It documents the gap between a syntactically correct query and a correct one, which is exactly the gap the column-domain bug exposed here.
8. Zavaleta, G. (2026). "Why 90 % accuracy in text-to-SQL is 100 % useless", Towards Data Science. towardsdatascience.com Argues for per-category reading against the global score: a high average accuracy hides the family of queries that fails, and that family decides real-world usefulness. It is the reading that singled out joins and one filter here, and enabled the targeted top-up.
9. NVIDIA. NeMo Agent Toolkit, text-to-SQL, technical documentation. docs.nvidia.com Independently gives the same order of magnitude for the example corpus and the same requirement of validation on a held-out set. Two sources of very different origin converging on "twenty to a hundred well-chosen pairs".
10. Brown, T. et al. (2020). "Language Models are Few-Shot Learners", NeurIPS. arXiv:2005.14165 The paper that established in-context learning: one example given inside the conversation is enough to steer the model, without touching its weights. Tabular foundation models transpose exactly that idea from text to tables.
11. Hollmann, N., Müller, S., Purucker, L. et al. (2025). "Accurate predictions on small data with a tabular foundation model", Nature, 637, 319-326. 10.1038/s41586-024-08328-6 TabPFN, the German school. Establishes that a model pre-trained once on synthetic tables beats, in a single pass and with no tuning, methods fitted on the data itself. That is the result that makes this second storey possible.
12. SAP (2026). SAP Completes Prior Labs Acquisition, press release of 17 July 2026. news.sap.com The acquisition of the Freiburg laboratory, announced in May and closed in July 2026, with more than a billion euros committed over four years. Cited here as a measure of industrial interest in tabular models, not as a technical argument.
13. Qu, J., Holzmüller, D., Varoquaux, G. and Le Morvan, M. (2025). "TabICL: A Tabular Foundation Model for In-Context Learning on Large Data", ICML. arXiv:2502.05564 The French school. Column-then-row attention builds a fixed-size representation of each row, which is what lets it hold much larger tables. On a par with TabPFN v2 across two hundred datasets, up to ten times faster, and ahead beyond ten thousand rows.
14. SODA team, Inria. tabicl, model code and weights. github.com/soda-inria/tabicl The concrete reason for the choice made here: pretraining code, inference code and weights are published, hence auditable and runnable on the customer's own machines, with no data leaving.
15. Lundberg, S. and Lee, S.-I. (2017). "A Unified Approach to Interpreting Model Predictions", NeurIPS. arXiv:1705.07874 Imports the Shapley value into machine learning, the rule that splits a collective payoff across players by average marginal contribution. Applied to a prediction, it says how much each variable weighed, and in which direction.
16. Ribeiro, M. T., Singh, S. and Guestrin, C. (2016). "Why Should I Trust You? Explaining the Predictions of Any Classifier", KDD. arXiv:1602.04938 Local surrogate explanation: the model is approximated, in the neighbourhood of the single row under examination, by a simple readable one. Shown here beside the previous method, because two methods disagreeing on one row is a signal in itself.
17. Bertin, J. (1967). Sémiologie graphique. Les diagrammes, les réseaux, les cartes, Gauthier-Villars. English edition: Semiology of Graphics, University of Wisconsin Press, 1983. The treatise establishing that a graphical form is not decoration but an encoding, and that each visual variable has properties of its own. It is the foundation of the deterministic filter: a figure incompatible with the columns present must never be offered.
18. Cleveland, W. S. and McGill, R. (1984). "Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods", Journal of the American Statistical Association, 79(387), 531-554. 10.1080/01621459.1984.10478080 Ranks visual encodings experimentally by reading precision, position on a common scale ahead of length, itself ahead of angle and area. It is the measurement that justifies the rules applied automatically to this article's figures.
19. Harchaoui, W. sprezzature-figures, open source figure engine. github.com/warith-harchaoui/sprezzature-figures The catalogue of a hundred and twenty-seven chart kinds used on the last storey, and the checker that ships with each: contrast, stated polarity, labels that do not collide. The six figures in this article come out of it.
20. pgvector, vector search extension for PostgreSQL, PostgreSQL licence. github.com/pgvector/pgvector Lets the knowledge base live in the PostgreSQL already in place, so nothing new has to be deployed. That is what gives the improvement loop its persistence: validated examples survive restarts and are shared across every instance of the service.

For further reading, two books broader than this article's subject: my favourite AI books for the annotated overview, and Edward Tufte's The Visual Display of Quantitative Information for everything the last storey automates without being able to replace it: the judgement about which question the figure should ask.