Skip to main content

The Model Does Not Write the Number

Four plain-language questions put to a company database call for a number, a prediction, a forecast and a chart. All four times, the language model writes the query. The answer comes from elsewhere.

Three names in coloured monospace type: text2sql in green and purple, text2sql2machinelearning in orange, text2sql2visualisation in blue.

Ask a chat assistant a question about your data. The language model behind it answers with a number, as confidently as a student who never opened the book. The number is plausible. It is wrong. Hardly anyone signs off on a board decision on the strength of a plausible number. That alone is enough to stop an enterprise deployment.

First, three words. A database is a set of large tables, cousins of spreadsheets: a table of customers, a table of invoices, thousands of rows each. A query is a question put to those tables. You write it in SQL, a rigid language with no double meanings that says which rows to take and what to do with them. The language model understands plain language and it also writes SQL.

The cure takes the number away from the model. Asking the question better would change nothing. The model writes the query and the database answers, with what it holds. The difference looks like nothing and it changes the nature of the error. A wrong query can be seen, kept, replayed, fixed. A made-up number cannot be seen, because it looks just like a correct one.

This article follows four questions really put to an enterprise deployment. They come up in a meeting in this order: last month's number, a row-by-row prediction, a three-month forecast, then a chart. They climb three stories on one foundation. Their names read like addresses: txt2sql for the number, txt2sql2ml for the prediction and the forecast, txt2sql2viz for the figure. Industry knows the first story well. The other two are far rarer. Those are the ones I built.

The chain shared by all three stories: a plain-language question, a SQL query written by the model, guardrails that parse the query, the read-only database, then three possible outputs: a number, a table to complete, rows to draw.

The foundation is the same at all three stories: the question becomes a query, the query is parsed before it runs, the database answers. Only the last link changes.


"How many invoices are overdue?"

Thirteen, says the screen two seconds later. And unfolded beneath that thirteen, the query that produced it.

SELECT COUNT(*) FROM invoices WHERE status = 'overdue' LIMIT 500

You can decipher it without ever having read SQL. It counts (COUNT) the rows of the invoices table (invoices) whose status equals overdue and never brings back more than five hundred of them (LIMIT 500).

That displayed line is the proof behind the number. Anyone who knows the database can read it, argue with it, run it again. You check the number instead of believing it.

A recent language model writes SQL in general very well. The trouble starts with this database. You need the right table names, the right joins and above all the right filter values. A join matches two tables through a shared column, invoices with their customers for instance. Is an unpaid invoice's status written overdue, unpaid or 3? The names and joins are in the schema, the list of tables and their columns. The values are not: they have to be read from the tables themselves. A model that does not know the schema invents names that look right and are wrong. So a tool that turns a question into a query is judged on how well it sticks to the database in front of you.

Two roads lead there. The first sends the model back to school: fine-tuning resumes its training on the customer's data until it knows them by heart. It costs a lot, takes a long time and starts over whenever the database changes. The second leaves the model as it is and slips it, for each question, a revision sheet with the few pieces of information that matter for that one question2. That method is called retrieval-augmented generation, or RAG. Vanna, the open source reference in the field, takes the second road1.

In Vanna, "train" means something unusual. This is the point that trips people up most: it means filling a knowledge base that the system will consult when answering. The model itself stays untouched: none of its internal dials moves. Picture a library where texts get filed away. When a question comes in, you look for the texts that talk about the same thing, even in other words. Then you attach them to the question you put to the model3.


What goes into the library and what counts most

Three kinds of content go into that library and intuition gets their weight wrong.

First shelf, the table structure: each table's CREATE TABLE. That is its identity card: its name, its columns, the type of each (number, text or date) and its keys, which link it to other tables. It teaches the model what exists and keeps it from inventing. Each table gets its own card, so that it comes back alone when a question touches it.

Second shelf, the business glossary says what the structure leaves out: how a measure is computed, what the in-house vocabulary means, what the coded values stand for. Each card holds one sentence, phrased the way a user would say it. "Revenue before tax on orders is the sum of orders.total_ht_eur." This is where you gain the most, because this is where a model with no context goes wrong.

Third shelf, the question-query pairs, last to be written and first in the results. Each one sets a question in plain words beside its correct query. Vanna's documentation says it without hedging: their quality decides the result1. The mechanism shows why. For each question, the system shows the model only the few closest cards and the rest of the library does not exist for it. One good example, close to the question asked, is worth a thousand distant ones. You file things away to cover cases, never to pile up volume.

Twenty to a hundred verified pairs are enough to start9. They have to cover the families of question people will ask: counting, crossing two tables, filtering on a coded value, ranking the biggest. That order of magnitude holds for a database this size. Elsewhere, it depends on the number of tables, on how ambiguous the vocabulary is, on how many business areas are covered and above all on the questions people really ask.

Two rules decide what gets into the library.

  • Always verified SQL, actually run and confirmed correct: a wrong pair teaches the system to produce wrong SQL.
  • Never SELECT *: that query brings back every column without naming any, so it teaches no column name.


Two locks between the model and the database

The model's job stops at the query. The query runs elsewhere, under an account allowed to read the database and never to change it. Before it runs, guardrails parse it, cutting it up the way you diagram a sentence in grammar class: subject, verb, complements. That breakdown is called a syntax tree4 and reading it is how the chain refuses anything that is not a plain read. The separation protects the database: the part that talks to the model never has the right to write to it.

A dated failure forged the second lock. On September 4, 2026, the model knew the names and types of the columns and nobody had shown it the values they hold. On the invoices table, it wrote WHERE status = 'confirmed'. The query was correct, the guardrails accepted it, it ran without error and it returned zero rows. Invoices only knew paid, overdue and sent. The word confirmed did exist, but in the orders table. On the screen, an empty result and not a word of explanation.

The flaw gets fixed from both ends. Prevention first: the pipeline goes after the columns with few distinct values, a status, a category, never an amount. The database supplies the real list of their values and the model receives it with the rest. First served are the columns whose exact list of allowed values the database already knows. Next come the text columns its statistics flag as low-variety, re-read with a bounded query. A column with no statistics never gets probed, because a wrong list would do worse than no list.

Then the cure. A query that runs without error and returns no rows, because it filters on a value absent from the column, no longer hands back a silent blank. The system takes one more shot, real values in hand. On a second failure, the answer names the column, the invented value and the values actually present.


The exam that proves the filing paid off

People skip this step too often, yet it is the only one that proves all that filing paid off. You set aside a few question-query pairs and leave them out of the library: they will serve as the exam. No serious teacher grades a student on exercises they have the answers to. A score on questions already filed only measures the system's ability to bring back an example it has learned.

You can score this in three ways, from the least to the most reliable.

  • The valid-SQL rate is a mere guardrail, since a query can be valid and wrong.
  • Character-by-character comparison of the two queries badly underrates accuracy, because two correct queries are almost always written differently5.
  • Comparing the results is the right measure and also the simplest: you run the generated query and the reference query, then compare what they return.

The field's benchmarks, Spider6 and BIRD7, use this measure. Every team scores itself against these public question sets. It wins because it looks at what the user will actually receive, the answer.

Waterfall chart: accuracy starts at 62.5 % with table structure alone, gains 25 points from the glossary and worked examples, then 12.5 more from a targeted top-up, reaching 100 %. The last bar stacks the three: the 62.5 % start, then the 37.5 points the corpus adds.

On the control set, the business glossary and the worked examples take the share of correct answers from 62.5% to 87.5%. Three examples added on the two lagging families, joins and one filter, take it to 100%. Replayed on three database sizes, the same measurement gives the same result: the library of examples moves the score and the data volume does not. Careful not to over-read these figures. The control set is small and scores move across it in steps of 12.5 points, which says enough about how fine-grained it is. That 100% means nothing is missing on those questions. It does not say the pipeline is correct in general, because a case study is not a benchmark.

A single overall score has a flaw: it hides the family of questions that fails8. Broken down by family (counting, crossing tables, filtering, ranking), it starts telling you what to do next. Weak crossings call for examples of crossings, not a bigger model. The jump from 87.5% to 100% on the figure came from that.


"Which customers are at risk of leaving and how confident are you?"

An exact number answers a question about the past: how much did we invoice last month, which customers went over budget? The executive asks a different question right afterward, about the future. Until now, it meant a project of several months, handed to data specialists and redone for every customer and every question.

Change what the query brings back. It now returns a table whose last column is the one to predict. That column is filled for some rows and empty for the ones you want to predict.

Customer Segment Recency Frequency Tickets Left?
Studio Moreau SMB 12 d 8 1 no
Fabrique Horizon Enterprise 190 d 1 4 yes
Maison Central SMB 45 d 5 0 ?

One question, one query: the gaps fill in. SQL cannot predict, it can fetch the right shape of table.

That table holds the examples and the question together. A tabular foundation model can read it as is. Its schooling is over: it was trained one single time, at its laboratory, on millions of tables made up for the exercise. It knows nothing of your data and a great deal about how tables behave. Think of a grader who, having marked thousands of papers, spots a mistake at first glance on a paper they have never seen. You show it the table and it fills the gaps in one go. Nothing is trained on your data, no internal dial moves, no model is left to store.

This is called in-context learning and everyone has already lived it. You give a chat assistant an example inside the conversation itself: it catches on without being retrained10. The comparison has a limit, though. Text reads left to right, whereas a table has no order, across its rows or across its columns. These models had to solve that problem.

The consequence is direct: no training project, nothing to maintain, no model to retrain the day you plug in a new customer. And the same code does four jobs.

  • It predicts a category, such as will leave or will stay.
  • It predicts a number, such as next month's amount.
  • It fills gaps.
  • It ranks the top N.


Two European schools, then NVIDIA: why I picked TabICL

Until the fall of 2026, two European teams led this field. In Freiburg, around Frank Hutter, the German school produced TabPFN, published in Nature in January 202511. SAP bought Prior Labs, its industrial arm, with more than a billion euros committed over four years12. That is a market signal, with no value as technical proof.

The French school works at Inria Saclay, in the SODA team, around Gaël Varoquaux, co-creator of scikit-learn. Its model, TabICL, was presented at the ICML conference in 2025. It matches TabPFN v2's accuracy across two hundred classification datasets. It runs up to ten times faster and pulls ahead in accuracy beyond ten thousand rows13. The second version, TabICLv2, came out in February 202614. The comparisons in this article are about the first.

I picked TabICL, and not out of patriotism. Both models run on the customer's machines with nothing leaving. Both publish their weights, the dials learned during training. TabICL goes further and publishes the code that trained it14, data generator included. With the weights alone, you rerun the predictions. With the training code, you rebuild the model from scratch. When you tell an executive "this customer is about to leave," you have to be able to show where the answer came from.

On September 29, 2026, NVIDIA released Kumo Tabular, which also learned on tables made up from scratch. With no tuning at all, it comes first on TabArena23. This public benchmark was presented in 2025 at the NeurIPS conference25. Every model sits the same test there, on fifty-one real datasets, picked out of one thousand fifty-three. That rank comes from NVIDIA's own runs and I found no independent reproduction. On video, Jure Leskovec, a Stanford professor, goes further on NVIDIA's behalf24: the model would beat the "grandmasters," the best human competitors, on many Kaggle competitions. That claim is spoken and no published measurement backs it.

TabArena ranks models the way chess ranks players, with the Elo rating. On each dataset, whichever model predicts better wins the game. A random forest on default settings, a classic and sturdy model, sits at 1,000 points as the landmark. A 400-point lead predicts ten wins for every loss. Kumo Tabular posts 1,950 points, 950 above the forest. Do the sum: two stretches of 400 points make ten times ten, a hundred wins for every loss, with 150 points still to spare. The math therefore predicts that it beats the forest on more than 99 datasets out of 100. The rating counts wins and ignores their size, as its authors write themselves: winning by a hair counts as much as winning by a mile.

That first place leaves my choice untouched. NVIDIA releases the weights under OpenMDW 1.1, a free license that allows commercial use26. So Kumo Tabular can be installed at a customer's site just like TabICL, which is BSD licensed like scikit-learn. But the training recipe and the table generator are still promised for "soon." With Kumo Tabular, you rerun the predictions. With TabICL, you can also rebuild the model from scratch. If NVIDIA publishes its recipe, the question will reopen.

The teams read one another and say so. TabICL's forecasting code comes from TabPFN-TS, written by the German team: it therefore stays under Prior Labs' license27. Kumo Tabular's announcement says it reads the table "as introduced in TabICL and TabPFN." Its first author, Jingang Qu, is also the first author of TabICL, written at Inria. He now describes himself as an engineer at NVIDIA, in Paris. SAP bought TabPFN's company, NVIDIA hired TabICL's first author. Twenty months passed between the Nature paper and NVIDIA's announcement. They were enough for these models to leave the laboratory and beat automated pipelines tuned for four hours on the data itself, such as AutoGluon, TabArena's reference.

Three freedoms get confused easily.

  • Reading a model, understanding how it works, is open to everyone through papers and announcements.
  • Installing it at a customer's site hangs on the license alone.
  • Rebuilding it from scratch demands the training code.

Today Kumo Tabular grants the first two and TabICL all three. Between two models I am allowed to install, I pick the one I know how to rebuild.

Horizontal bars of the estimated probability of leaving for five customers, with the 50 % threshold: Maison Central 69 %, Agence Concorde 66 % and Atelier Lumière 59 %, called leavers; Comptoir Saint-Jean 22 % and Boulangerie Concorde 20 %. No prediction goes above 69 %.

The row-by-row answer, on the customers whose departure status is not consolidated. Three departures called, none above 69 %: each prediction comes with its confidence and that confidence stays modest.

That 69% is almost always read the wrong way. People see in it a 69% certainty that this customer will leave. Take instead the customers who resemble this one on the columns we can see: about sixty-nine out of a hundred have left. The number describes an observed frequency. It predicts nothing about one person.

Two very different things make that figure uncertain and confusing them is expensive. The first is our own ignorance: the history holds few comparable customers, so the estimate would move if we added data. The second comes from the randomness of the world: a perfect history would not stop two customers identical on paper from deciding differently. More data reduces the first. The second never shrinks, which is why an honest prediction stays cautious.

One rule holds across all three stories: a figure without its uncertainty is an opinion dressed as a fact. On this data, no prediction goes above 69% and that is a good sign. A tool that had announced 97% would have been lying.

Knowing who will leave is useless without knowing why. You keep a customer with a reason precise enough to act on, never with a probability.

Commerce named this problem long before statistics did. Its most famous phrasing runs like this: "Half the money I spend on advertising is wasted; the trouble is I don't know which half." The line is credited to the American department-store owner John Wanamaker, though his archives hold no record of it15.

The problem itself is real enough. It is even ill-posed: no measurement can settle it. The sale happened once. We will never know what would have happened without the poster in the subway, because that world never existed. So any split of the credit between channels is a convention: you pick a rule, then you stand by it.

Let us weigh the choice of rule on a small experiment. Take two advertising channels, billboards and online, bought together. They rise and fall in step: their correlation is 0.84 across two hundred simulated weeks, on a scale that stops at 1. By construction, only the billboards act on sales and online is a passenger. Now ask each channel what it explains.

Four bars: the online channel explains 55.3 % of sales when asked on its own, 0.0 % when added to billboards; billboards explain 80.1 % alone and 24.8 % added to online. A last bar gives the Shapley split: 52.5 % to billboards and 27.7 % to online, 80.1 % together.

The same channel, the same data, two incompatible answers: the only difference is the order in which the question is asked.

It all happens in the first two bars. Asked on its own, the online channel explains 55.3% of the variation in sales, enough for a handsome dashboard. Added to billboards, it explains nothing at all: 0.0%. Wanamaker's trouble fits in those two numbers.

The Shapley split, which I come back to below, averages the two orders and settles it: 52.5% to billboards, 27.7% to online. Check it: billboards get the average of 80.1 and 24.8, online the average of 55.3 and 0.0. The two shares come to 80.1%, give or take rounding, which is what the two channels explain together. That is no happy accident, because Shapley required the shares to add up. That is what lets you bring the result to a board. The protocol is spelled out, to be rerun rather than taken on trust: two hundred simulated weeks, random seed 20260925, linear regression. A group of channels is worth the share of the variation in sales it explains.

Look, though, at the 27.7% handed to online: we know, since we built the example, that this channel causes nothing. The method has not failed, though. It answers faithfully the question it was asked: what did the model use? That differs from another question: what caused the sale? That one has no shortcut. You need an experiment: cut one channel's budget in one region and compare with the others. The split explains the model and only the experiment reaches the cause.

Back to our 69%. The model produced it in one go, looking at a dozen columns at once. What share comes from the age of the contract, what share from the number of complaints, what share from the last invoiced amount? Cutting a result into shares, one per variable, is called attribution. Nothing guarantees in advance that it is possible, because the model did not compute its number as a sum. It mixed everything. So attribution rebuilds, once the computation is over, a split that never existed in it.

Try the naive route and the difficulty appears. To measure what a variable brings, remove it and watch how far the prediction drops. But the answer depends on the order in which you remove the others. With nothing removed yet, taking out the age of the contract drops the prediction a long way. Once the number of complaints has gone, the same removal changes almost nothing, because the two say partly the same thing. So there are as many answers as there are orders.

The solution dates from 1953 and had an entirely different purpose. Lloyd Shapley was after a fair way to share a team's payoff among its members, when each brings something different and the contributions overlap16. His idea is simple: since the order changes the result, look at every possible order and average. Think of a restaurant bill. Depending on who arrives first and what gets shared, each person's share changes. Averaging over every order of arrival gives a split nobody can dispute. The Nobel Memorial Prize in Economic Sciences rewarded that work in 2012. Applied to a prediction instead of a dinner, the same computation says how much each variable weighed and in which direction17.

A second method starts from another idea, just as simple. A complicated model stays complicated, but around one single customer it behaves almost like a straight line, the way a curve looks straight when you zoom in far enough. So you build, in that small neighborhood only, a simple model that imitates the big one. All that is left is to read its coefficients18.

The tool shows both methods side by side. If they agree, you have a solid reading. When they disagree, the disagreement is already information: the prediction depends on a fragile balance between variables and it is better to know that before calling the customer.


"What is the revenue forecast for the next three months?"

Same foundation, same model, different question. This time the query brings back a series through time and the answer is about months that do not exist yet.

Three monthly forecasts with their range from the 10th to the 90th percentile: July 14,557 euros, from 2,743 to 32,581; August 19,096, from 4,374 to 37,675; September 14,334, from 2,529 to 31,772. A band shows the three forecasts lie within 4,762 euros, while each range covers close to 30,000. Below the chart, two bars on the same scale compare the narrowest range, 29,243 euros, with the gap between the forecasts, 4,762 euros.

Each forecast month shows its range, from the tenth to the ninetieth percentile. A percentile cuts the possible values into a hundred parts. The tenth is the one below which there is only a one-in-ten chance of landing, the ninetieth the one below which there are nine chances in ten. Between the two, eight chances in ten remain and that is what the bar says.

A single number about the future is fortune telling. A number with its range is a decision. This figure delivers uncomfortable news that you have to know how to read: the range outstrips the gap between the three months. August may look better than July, but the bars overlap almost entirely and the gap between those two months means nothing.

The data is bad and the tool has nothing to do with it. In this demonstration database, the monthly series is pure noise, jumps with no regularity at all. Its twenty-one months show 293 euros in one month, 72,380 a year later and three months missing. No method forecasts that. Only a wide range answers honestly. Three neat, tidy numbers would have dressed the situation up. The principle is the same as on the first story: the promise applies to the uncertainty shown beside the answer, never to the quality of the answer itself.

One implementation detail shows the world you work in. The number of months to predict comes from the user's question, so from free text. "In a hundred thousand months" turned into a hundred thousand months to forecast. A hundred thousand months, more than eight thousand years! The machine spat out absurd dates and choked on a computation nobody had seriously asked for. That number is now bounded: past that point a forecast has no value anyway, so nothing is lost.


"Show me where revenue is concentrated"

Last question, last link that differs: the result becomes a figure, chosen automatically from the shape of the data and what the question asks, then drawn on the spot. Nobody has a dashboard to maintain. A dashboard is a page of frozen charts prepared in advance. It answers the questions foreseen while building it, six months ago.

This choice is where a language model can go off the rails. Show it a hundred and sixty-eight chart names and it may invent one, misspell another or pick one that does not go with the columns present. It might offer a map, for instance, when no column holds a place. Counting on its good will would protect nothing. The protection comes from the shape of the program, where the model cannot get past it.

Five steps on two lanes, no model below, the model above: column profile with no model, the question's intent by the model, list of compatible figures with no model, ranking and explanation by the model, rendering with no model. A bracket under steps 3 and 4 notes that the model only sees compatible figures: it can reorder them, never add one.

Three deterministic steps alternate with the two model calls. Steps 1, 3 and 5 involve no model, so no surprises. The third step eliminates every structurally incompatible candidate before the model sees anything at all. The fourth can only reorder and annotate that already-safe list, never extend it.

The model picks a dish from a menu it did not write. This guarantee differs in kind from a temperature setting, which makes the model more or less adventurous, or from an instruction added at the top of the conversation. It depends on the shape of the program and never on the model's compliance. As on the first story, it makes the remaining error visible and fixable: a badly chosen figure is corrected with one click and never blends in with a correct one.

Pareto chart laid on its side: nineteen bars of revenue per customer, from 52,257 euros down to 1,642, and beside them the cumulative-share line, which crosses 80 % at the twelfth customer. The first twelve customers make 317,370 euros out of 386,015, or 82 %.

The answer to the question asked: twelve of the nineteen customers the query returned make 82% of revenue. The cumulative share is computed over those nineteen customers and the caveat matters. A Pareto chart ranks the contributions from largest to smallest, then draws beside them the curve of their running total. You read at a glance how few customers make up most of the total. Drawn on a ranking already truncated upstream, it announces a share "of the total" that is nothing of the kind, for all its neat curve.

The drawing falls to sprezzature.ai, an open source figure engine21. One hundred and sixty-eight chart types fill its catalog, from curves to flow diagrams, treemaps, waterfalls and maps. The wider the catalog, the more charts fit the columns you have and the more often the automatic choice lands right.

Every figure also obeys rules on accessibility and color.

  • Contrast is checked.
  • A color-blind person reads the figure without trouble.
  • No information depends on a color alone.
  • Any axis where up is better than down states its reading direction.

Cosmetics have nothing to do with it. A poorly contrasted chart projected in a room goes unread. A chart that asks you to tell two greens apart excludes one man in twelve. These rules descend from graphical semiology, which studies what visual forms can really convey19, and from experiments on perception20. Here they run as lines of code, not as recommendations.


What production teaches and preparation cannot guess

No amount of filing done in advance guesses how people will phrase their requests, their turns of phrase, their shortcuts, the special cases of each company. The real questions only appear in production. Every question handled well, above all every correction, teaches something the initial filing did not know.

The loop is simple. Someone asks a question. The system writes the query, which runs read-only. A human validates the result or fixes the query. Once validated, the pair joins the library of examples and the next question that resembles it will find that new example.

One safety rule allows no exception and it is the easiest to get wrong: never blind learning. Automatically capturing every query produced and feeding it back as an example poisons the library for every similar question to come. Bad SQL teaches the system to produce bad SQL. A pair therefore enters through a dedicated path, which replays the guardrails and re-runs the query before accepting it. Even what we are told has already been verified goes through it.

A mechanism that works on one laptop differs by four properties from a loop that holds in production.

  • Persistence: the library stays in the database already in place22 and escapes the memory that the first restart would wipe.
  • Idempotence: the identifier of a pair comes from its content, so the same pair submitted twice makes no duplicate.
  • Traceability: every pair from the field wears a mark that keeps it apart from the hand-filed corpus. You can then check what production taught the system and remove one bad example.
  • Measurement: after each batch of additions, you sit the exam again. Progress gets observed instead of assumed.


Three days become three seconds, and the limit that stays

Before, a business question left as a ticket for the data team and came back three days later, often after the decision had been made without it. A question about the future went off as a project. Now the question is asked out loud, the number lands in seconds and so does the follow-up. The meeting keeps its rhythm. The gain is measured in delay: between the question and a reliable number, you go from three days to three seconds, because the number comes from the database. Three days against three seconds is a ratio of 86,400, the number of seconds in a day.

The model can still write a bad query. The architecture guarantees that we only read the database, within known limits. Whether the query matches what the person meant to ask, it cannot guarantee. On that point, it reduces the ways of going wrong and makes them visible without removing them. The real limit comes from the quality of the schema and the vocabulary, never from the volume of data. The chain queries the database directly and SQL does the filtering. A database whose columns are called col_17 will not let itself be queried in plain language and no model will change that.


The three lines

txt2sql question → SQL → database → the real number txt2sql2ml question → SQL → table with gaps → the prediction and its range txt2sql2viz question → SQL → rows → the right chart, live

One foundation, three stories and everywhere the same discipline: the model writes the query, the rest of the chain checks and answers. The plausible number from the first paragraph only looked right. This one you can read, rerun and sign.


Acknowledgments

Warm thanks to Marie Sacksick, Edmond Jacoupeau, Bachir Zerroug and Vincent Sammiez for our fruitful discussions on these topics.

Let's talk AI.

The figures in this article were made with sprezzature.ai by Deraison.ai


Bibliography

Each of these titles justifies a design decision taken above. The notes say which.

  1. Vanna AI. Training advice, project documentation. vanna.ai/docs/training-advice States without hedging the hierarchy of the three kinds of content and the safety rule taken up here: the quality of question-to-query pairs is the single largest determinant of the result and bad SQL learned teaches the system to produce bad SQL.
  2. Lewis, P. et al. (2020). "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", NeurIPS. arXiv:2005.11401 The paper that named retrieval-augmented generation. It establishes the principle used on the first story: leave the model untouched and supply, for each question, the context that matters, rather than retraining its weights.
  3. Vanna AI (2023). How Vanna works, how to train it, data security. medium.com/vanna-ai Walks step by step through the retrieval mechanism everything else depends on: the question is embedded, the nearest knowledge is pulled per content type and only that enters the prompt. This is where the rule "coverage beats volume" comes from.
  4. Mao, T. sqlglot, SQL parser and transpiler in Python, MIT license. github.com/tobymao/sqlglot The parser that turns the generated query into a syntax tree before any execution. It is what makes the guardrails possible: you inspect the structure of the query, not a string and refuse anything that is not a read.
  5. Pourreza, M. and Rafiei, D. (2023). "Evaluating Cross-Domain Text-to-SQL Models and Benchmarks", EMNLP. arXiv:2310.18538 Shows that benchmark scores underestimate models, because a natural-language question admits several correct queries. Justifies the choice of execution match over textual comparison, made here.
  6. Yu, T. et al. (2018). "Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task", EMNLP. arXiv:1809.08887 The benchmark that set the field's habits, starting with evaluation on databases never seen in training. The held-out control set described here is that idea transposed to the scale of a single customer.
  7. Li, J. et al. (2023). "Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs", NeurIPS. arXiv:2305.03111 The BIRD benchmark, built on real databases, dirty and large. It documents the gap between a syntactically correct query and a correct one, which is exactly the gap the column-domain bug exposed here.
  8. Zavaleta, G. (2026). "Why 90 % accuracy in text-to-SQL is 100 % useless", Towards Data Science. towardsdatascience.com Argues for per-category reading against the global score: a high average accuracy hides the family of queries that fails, which is the family deciding real-world usefulness. It is the reading that singled out joins and one filter here and made the targeted top-up possible.
  9. NVIDIA. NeMo Agent Toolkit, text-to-SQL, technical documentation. docs.nvidia.com Independently gives the same order of magnitude for the example corpus and the same requirement of validation on a held-out set. Two sources of very different origin converging on "twenty to a hundred well-chosen pairs".
  10. Brown, T. et al. (2020). "Language Models are Few-Shot Learners", NeurIPS. arXiv:2005.14165 The paper that established in-context learning: one example given inside the conversation is enough to steer the model, without touching its weights. Tabular foundation models transpose exactly that idea from text to tables.
  11. Hollmann, N., Müller, S., Purucker, L. et al. (2025). "Accurate predictions on small data with a tabular foundation model", Nature, 637, 319-326. 10.1038/s41586-024-08328-6 TabPFN, the German school. Establishes that a model pre-trained once on synthetic tables beats, in a single pass and with no tuning, methods fitted on the data itself. That is the result that makes this second story possible.
  12. SAP (2026). SAP Completes Prior Labs Acquisition, press release of 17 July 2026. news.sap.com The acquisition of the Freiburg laboratory, announced in May and closed in July 2026, with more than a billion euros committed over four years. Cited here as a measure of industrial interest in tabular models, not as a technical argument.
  13. Qu, J., Holzmüller, D., Varoquaux, G. and Le Morvan, M. (2025). "TabICL: A Tabular Foundation Model for In-Context Learning on Large Data", ICML. arXiv:2502.05564 The French school. Column-then-row attention builds a fixed-size representation of each row, which is what lets it hold much larger tables. On a par with TabPFN v2 across two hundred datasets, up to ten times faster and ahead beyond ten thousand rows.
  14. SODA team, Inria. tabicl, model code and weights. github.com/soda-inria/tabicl The concrete reason for the choice made here: pretraining code, inference code and weights are published, hence auditable and runnable on the customer's own machines, with no data leaving.
  15. "One-Half the Money I Spend for Advertising Is Wasted, But I Have Never Been Able To Decide Which Half", Quote Investigator, 11 April 2022. quoteinvestigator.com Traces the saying credited to John Wanamaker. The earliest known match is a 1919 speech already attributing it to him; neither the Wanamaker archives nor Unilever's, for the Leverhulme variant, hold any record of it. It is the source for the paragraph on attribution, and the reason this article presents the quotation as apocryphal rather than as a fact.
  16. Shapley, L. S. (1953). "A Value for n-Person Games", in H. Kuhn and A. Tucker (eds.), Contributions to the Theory of Games II, Princeton University Press, 307-317. The original paper, written to share a team's payoff among its members. It states the axiom that does the work: since the order of arrival changes each share, average over every order. Shapley received the Nobel Memorial Prize in Economic Sciences in 2012, with Alvin Roth. Sixty years separate this text from its use in machine learning.
  17. Lundberg, S. and Lee, S.-I. (2017). "A Unified Approach to Interpreting Model Predictions", NeurIPS. arXiv:1705.07874 Imports the Shapley value into machine learning, the rule that splits a collective payoff across players by average marginal contribution. Applied to a prediction, it says how much each variable weighed and in which direction.
  18. Ribeiro, M. T., Singh, S. and Guestrin, C. (2016). "Why Should I Trust You? Explaining the Predictions of Any Classifier", KDD. arXiv:1602.04938 Local surrogate explanation: the model is approximated, in the neighborhood of the single row under examination, by a simple readable one. Shown here beside the previous method, because two methods disagreeing on one row is a signal in itself.
  19. Bertin, J. (1967). Sémiologie graphique. Les diagrammes, les réseaux, les cartes, Gauthier-Villars. English edition: Semiology of Graphics, University of Wisconsin Press, 1983. The treatise establishing that a graphical form is not decoration but an encoding and that each visual variable has properties of its own. It is the foundation of the deterministic filter: a figure incompatible with the columns present must never be offered.
  20. Cleveland, W. S. and McGill, R. (1984). "Graphical Perception: Theory, Experimentation and Application to the Development of Graphical Methods", Journal of the American Statistical Association, 79(387), 531-554. 10.1080/01621459.1984.10478080 Ranks visual encodings experimentally by reading precision, position on a common scale ahead of length, itself ahead of angle and area. It is the measurement that justifies the rules applied automatically to this article's figures.
  21. Harchaoui, W. sprezzature-figures, open source figure engine. github.com/warith-harchaoui/sprezzature-figures The catalog of a hundred and sixty-eight chart kinds used on the last story and the checker that ships with each: contrast, stated polarity, labels that do not collide. The seven figures in this article come out of it.
  22. pgvector, vector search extension for PostgreSQL, PostgreSQL license. github.com/pgvector/pgvector Lets the knowledge base live in the PostgreSQL already in place, so nothing new has to be deployed. That is what gives the improvement loop its persistence: validated examples survive restarts and are shared across every instance of the service.
  23. Qu, J. et al. (2026). Kumo Tabular, release announcement, NVIDIA, September 29, 2026. huggingface.co/blog/nvidia/kumo-tabular The third model of the kind, in three sizes from twenty-eight to two hundred fifteen million internal dials, trained on synthetic tables drawn from causal models. First on TabArena with an Elo rating of 1,950 on default settings, also announced first on three other benchmarks. Measurements run by the authors themselves, to be read as such. The training recipe and data generator are announced for later. The first author describes himself as "Senior AI Engineer at NVIDIA" on github.com/jingangQu. Weights: huggingface.co/nvidia/Kumo-Tabular.
  24. Leskovec, J. (2026). Video presentation of Kumo Tabular on NVIDIA's behalf, LinkedIn, late September 2026. linkedin.com/posts/leskovec A two-minute presentation. The claims quoted here, one hundred million rows in context, more than ninety-one models compared on TabArena, Kaggle grandmasters beaten, are spoken and come with no published measurements. They are reported as such.
  25. Erickson, N., Purucker, L., Tschalzev, A., Holzmüller, D., Mutalik Desai, P., Salinas, D. and Hutter, F. (2025). "TabArena: A Living Benchmark for Machine Learning on Tabular Data", NeurIPS 2025, Datasets and Benchmarks. arXiv:2506.16791 The living benchmark for tabular models: fifty-one datasets picked out of one thousand fifty-three against public criteria, an Elo rating anchored on a default random forest at 1,000 points, AutoGluon tuned for four hours as the reference. The live leaderboard is at tabarena.ai.
  26. OpenMDW License 1.1, license of Kumo Tabular's weights. openmdw.ai/license/1-1 The text that settles the question for professional use. It grants, free of charge, the right to use, modify and redistribute the model "without restriction," and imposes nothing on what the model produces. Only two conditions: keep the license and notices of origin when redistributing, and give up the model if you sue over a patent or copyright claim against it.
  27. SODA team, Inria. tabicl, license file. github.com/soda-inria/tabicl Three-clause BSD for the most part, with one declared exception: the forecasting directory is a derived work from TabPFN-TS and stays under an Apache license in Prior Labs' name. Concrete proof that the two schools read each other, in a document nobody writes for show.

Two books to widen the view. My favorite AI books gives the guided overview. Edward Tufte's The Visual Display of Quantitative Information takes up what the last story automates without being able to replace: the judgment about which question the figure should ask.