Same Code, Different Seed

In short

Reinforcement learning (RL) trains a machine by trial and error, guided by a reward. In 2018, Joelle Pineau showed the ICLR conference that its published results did not survive a simple examination: the best algorithm changes from one task to the next; three implementations of the same algorithm give three results; two batches of five runs of the same program look like two different methods; papers publish three runs where ten are needed.

Eight years later, RL has earned its place: it fine-tunes ChatGPT and it taught DeepSeek-R1 to reason. The traps of 2018 came back with it, at a larger scale.

In 2026 we ticked again the grid she applied to the papers of 2018, on fifty recent papers drawn at random: held-out evaluation has progressed, baseline tuning has fallen back, statistical testing is still absent. My answer: three conditions, then a reading grid.

• When RL is worth its cost. An environment that runs for almost nothing, a reward you check without error, the budget for ten seeds and for a baseline tuned with the same care.
• How to read an RL announcement. How many seeds, which interval, which baseline, how many model families, which code.

Why I never went

I have been asked the question for ten years, in two forms. Students ask: should I go to RL? Executives ask: does RL really work? I have never done RL. I do applied mathematics with a computer, for companies, almost always supervised learning: you show the machine examples whose right answer you know, it learns to find that answer, then you judge it on examples it has never seen.

RL changes the contract. Nobody gives the right answer. An agent acts in an environment, a game, a simulated robot, a market; from time to time it receives a reward, a number; it adjusts its way of acting to receive more. The idea is superb. It beat the best backgammon players as early as the 1990s¹, then the best Go players in 2016². My suspicions came from everything else. A single number, sometimes at the very end of a long sequence of actions, to learn everything from. Learning curves that zigzag. Spectacular results that nobody around me managed to redo.

I was not alone. In February 2018, Alex Irpan, an engineer at Google, published a post that stayed famous, "Deep Reinforcement Learning Doesn't Work Yet". He wrote: "Whenever someone asks me if reinforcement learning can solve their problem, I tell them it can't. I think this is right at least 70% of the time."³ A shared suspicion is still a suspicion. I needed someone from the trade, who knew RL from the inside and who had measured.

Vancouver, May 3, 2018

On Thursday, May 3, 2018, at 9 a.m., Joelle Pineau opened the day at ICLR, one of the three major machine learning conferences, with an invited talk: "Reproducibility, Reusability, and Robustness in Deep Reinforcement Learning"⁴. She was a professor at McGill University and head of the Facebook AI Research lab in Montreal. She had been doing RL since 1999. At her first workshop on the subject, in 2000, there were thirty-five people in the room. In 2016, she counted about thirteen thousand RL papers published in the year. Nobody could accuse the speaker of holding a grudge against the field.

She started with a definition, from the US National Science Foundation: a result is reproducible when another researcher finds it again with the same materials; this is the minimum condition for a result to be believable and informative⁵. Then a number. In 2016, the journal Nature surveyed 1,576 researchers: 52% saw a "significant" reproducibility crisis, 38% a "slight" one; more than 70% had already failed to reproduce a colleague's experiment⁶. She had asked the same question to twenty machine learning experts at a workshop in December 2017: 80% saw a crisis.

Finally, a sentence from 1995, by two Stanford statisticians, Jonathan Buckheit and David Donoho, who took it from the geophysicist Jon Claerbout: "An article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures."⁷ The rest of the talk put that sentence to the test, on the ground she knew best.

The best algorithm changes with the task

The ground was policy gradients. A policy is the agent's rule of conduct: in this situation, take that action. You give it to a neural network and correct it little by little in the direction that increases the average reward. Four algorithms of this family served as baselines in almost every paper: TRPO, PPO, DDPG and ACKTR. She checked that the seven papers of this kind accepted at ICLR 2018 all compared themselves to them.

The test bench was MuJoCo, a physics simulator where jointed puppets learn to move forward: HalfCheetah, a two-legged half cheetah; Hopper, a hopping leg; Walker2d, a walker; Swimmer, a swimmer. The four tasks look alike: a few joints, continuous controls, unsurprising physics. You would expect the same winner everywhere. Her team ran the four algorithms on the four tasks, with five seeds each⁸.

DDPG dominates HalfCheetah; it comes last on the three other tasks. TRPO, last on HalfCheetah, wins everywhere else. On Swimmer, the HalfCheetah winner learns almost nothing. The whole ranking depends on the choice of task. An author who tests a method on HalfCheetah against TRPO and an author who tests it on Hopper against DDPG publish two easy victories, without anyone cheating.

Three implementations, three results

She went further. Perhaps her team had coded these baselines badly? A search for "TRPO" on GitHub returns about ten public repositories, which she first found delightful: this is exactly the sharing she argues for. Her team's paper compares three of them, all serious: the original code of TRPO's author, the one in the OpenAI Baselines library and the one in the rllab library. Same algorithm, same task, same settings: three very different curves. The experiment repeated on three implementations of DDPG gives the same gap⁸.

Then come the details that a paper almost never mentions. The size of the neural network changes the result. So does the activation function, the small computation each neuron applies to its input. The scale of the reward, between 0 and 1 or between 0 and 10, upsets learning, whereas with the methods that came before neural networks you could change it without consequence. These settings interact: the effect of the scale reverses depending on whether the network's layers are normalised.

Her next slide asked the awkward question: "When testing a baseline, how motivated are we to find the best hyper-parameters?" Every setting left untuned in a baseline becomes an apparent gain for the new method. Nobody needs to be dishonest. It is enough to tune your own algorithm for a month and the others for an afternoon.

Same code, different seed

Then came the slide I have never forgotten. Two learning curves on HalfCheetah. The orange one climbs towards 4,000; the blue one levels off around 2,700. Each is the average of five runs, with its uncertainty band. The two bands separate clearly. Any reviewer would see two algorithms, one better than the other. She let the suspense last, then gave the answer: both curves come from the same TRPO code, with the same settings, the best known. Only the random seed changes.

The seed is the number that initialises a program's random generator. It decides the network's starting weights, the order of episodes, the small perturbations of the simulator. In supervised learning, changing the seed usually moves a score by little. In RL, early chance decides what the agent explores, what it explores decides what it learns; what it learns decides what it will explore next. A small difference at the start becomes a different behaviour at the end.

The published paper says exactly what was done. Ten runs of the same program, split into two groups of five, then a classic statistical test comparing two means: t = −9.09, p = 0.0016⁸. With the usual 5% threshold, this test declares the gap significant. Yet it compares the same algorithm with itself. To understand how, I redid the experiment in simulation.

The two batches of five do not come from two different distributions; they come from the two ends of a single, very spread-out one. With five draws per group, these two ends often fall far enough apart to fool a test that assumes well-behaved, bell-shaped results. Add that a lab tries several seeds, several tasks and several settings before publishing: it always finds a combination that "works". That was exactly my suspicion, better measured.

Three runs, never ten

How many runs do papers publish? She read the number in seven of the most cited RL papers of the time: the best five out of an unstated number, between three and nine, five, three, five, five, the best two or three⁸. None reached ten. She said so with sympathy, since a run lasts hours, even days, after weeks of tuning. Then she showed what this economy costs.

Keeping the best three runs out of ten does two things at once. The mean goes up: that is a positive bias. The spread melts, since the bad cases were thrown away: the method looks more reliable than it is. A student who arrives with ten curves hears "your algorithm is learning"; with the best three, "we've got a trend, let's write the NIPS paper". She told the scene to make the room laugh. The room laughed, because it recognised itself.

She ended on the stakes. For twenty years, RL had only been used to play, backgammon then Go; its imprecision cost nothing. It was now arriving at medical treatment design, energy management, finance. Answering questions from the room, she listed her personal habits: ask as many questions of the good results as of the bad ones, document everything, rerun your experiments in the hours before submission. She also wished the major conferences would stop accepting papers without their code.

What she obtained

She did not just talk. In the autumn of 2017, she had given her roughly 180 McGill students a final course project: pick a paper submitted to ICLR 2018 and try to reproduce its results. Other courses in several countries followed. The official tally, published later, counts 123 papers claimed by teams from 31 institutions⁹. Among participants who answered her survey, 54.5% reproduced part of the work and 32.7% most of it; 22% found the exercise very difficult. On the authors' side, 79% wanted their 2019 paper to go through the same examination.

In December 2018, in Montreal, she gave the talk again at NeurIPS with a new number. Her team had read fifty RL papers published in 2018 at NeurIPS, ICML and ICLR, then ticked a grid of eight boxes¹⁰.

Nine papers in ten give the settings of their method; six in ten those of their baselines. One in five says how its settings were chosen. One in ten evaluates the agent somewhere other than where it was trained. One in twenty applies a statistical test. The same talk asked a question I often quote: "Myth or fact? Reinforcement Learning is the only case of ML where it is acceptable to test on your training set." In supervised learning this would be a beginner's mistake. In the RL of 2018, it was the custom: you train the agent on HalfCheetah and publish its score on HalfCheetah.

She proposed a reproducibility checklist at the end of 2018. NeurIPS made it mandatory in 2019, at submission and again at camera-ready, together with a code policy and a reproducibility challenge. The report on this programme, published in 2021, measures the effect: fewer than 50% of accepted NeurIPS 2018 papers provided code in their final version; 74.4% did in 2019⁹. The checklist has since become the norm at the field's major conferences.

Measured again in 2026

A grid ticked in 2018 can be ticked again. We took her eight questions to fifty RL papers drawn at random, with a published seed, from the 1,252 papers whose title announces RL at the three most recent conferences: ICLR 2026, ICML 2026 and NeurIPS 2025. The readers are artificial intelligence agents (Claude, by Anthropic) that justify each answer with an exact quotation from the paper and its page; two independent readers, on ten papers read twice, agree on 127 decisions out of 130, or 98%.

The limits are four. Filtering by title leaves out RL papers that do not say so in their title. Only papers whose PDF is freely accessible enter the sample. Fifty papers give wide intervals. Joelle Pineau does not describe how she chose hers. To find out how our readers compare with her team, we first re-read twelve RL papers from 2018, drawn from ICML and NeurIPS, with the same grid: a difference counts only if it stands both against her numbers and against our re-reading of 2018. The protocol was written before the first paper was read; its deviations are recorded¹¹.

The 2026 sample contains 27 papers of RL applied to large language models, 14 of classic RL (control, games, robotics) and 9 of theory, 5 of them without any experiment. Percentages are computed over the 45 experimental papers.

What improved: held-out evaluation. Seven papers in ten now evaluate the agent somewhere other than on what it was trained on, against one in ten for Pineau and four in ten in our own re-reading of 2018, which is more lenient on this point. The progress comes mostly from RL on large language models: you train on one set of problems and judge on separate benchmarks, 26 papers out of 27. In classic RL, only 5 papers out of 13 do so. The "myth" of 2018, testing on the training set, survives where it was born.

What regressed: baseline settings. In 2018, six papers in ten gave all the settings of the methods they compare against. In 2026, fewer than two in ten, 18%, while our re-reading of 2018 still found six in eleven. Only one experimental paper in twenty, 5%, says it tuned its baselines with the same care as its own method. Her slide's question, "how motivated are we to tune a baseline?", now has a numerical answer.

What did not move. Linked code rises, from 55% to 67%, without our interval excluding the 2018 figure. One paper in four says how it chose its settings, against one in five. The significance test remains the exception: 4%, two papers out of forty-five.

Ambiguous, and why. Method settings go from 90% for Pineau to 67% for us; but our re-reading of 2018 gave 60%, because our grid requires every setting where hers probably accepted the essential ones. The gap measures our readers, not the field. The share of papers without experiments rises with theory; a sample of fifty does not allow us to say more.

The questions we added, taken from her Vancouver talk, say more. Of the 41 experimental RL papers, 20 do not say how many independent training runs support their main results; 4 rely on a single one; the others report five as the median. Uncertainty is shown by all 14 classic RL papers out of 14, but by 4 of the 27 papers on large models: the discipline Pineau asked for took hold in her own field, then got lost on the way to large models. Among those 27 papers, 15 try a single family of base models, exactly the trap that the random-reward study describes below. One in two, 47%, reports the compute used. Finally, best-run selection is still visible: an ICLR 2026 paper reports as its main result a score of 23.00, the best of three identical training runs scored 19.87, 21.58 and 23.00, without saying so. You have to read its hyperparameter sweep table to see it.

The data, the protocol and its deviations, and the paper-by-paper codings with their quotations are online: protocol, results and the full archive. The protocol and results are written in French.

What has changed since

Her warnings were confirmed by others, with means she did not have. In 2020, a team at MIT took PPO and TRPO apart: most of PPO's lead came from "code-level optimizations" that the original paper treated as secondary details¹². In 2021, a team at Google trained more than 250,000 agents to measure the effect of more than fifty implementation choices of this kind¹³. The same year, Rishabh Agarwal and his co-authors showed that, with three to five runs, published rankings on Atari games often flipped when re-analysed with confidence intervals; they released a library, rliable, to do the computation properly. NeurIPS gave them an outstanding paper award¹⁴.

Meanwhile, RL found the job it had been missing. In January 2022, OpenAI described InstructGPT: a language model fine-tuned by RL from human preferences, whose 1.3-billion-parameter version is preferred by raters to GPT-3 and its 175 billion¹⁵. On November 30, 2022, ChatGPT came out, built on the same method¹⁶. In March 2025, Andrew Barto and Richard Sutton received the Turing Award for laying the conceptual and algorithmic foundations of RL¹⁷. In April, Sutton and David Silver announced "the era of experience", in which agents would learn mostly from their own attempts¹⁸.

The decisive turn came in January 2025. DeepSeek released R1, whose R1-Zero version learns to reason on mathematics problems by RL alone, without reasoning examples written by humans. The reward is verifiable: the final answer is right or wrong; a program checks it. The paper, published in Nature in September 2025 after peer review, reports a first-try success rate that rises from 15.6% to 77.9% on the AIME 2024 competition. It also gives the cost of the RL stage: $294,000 of compute, not counting the training of the starting model¹⁹.

The same traps, only bigger

The RL of 2025 no longer starts from scratch. It fine-tunes a language model that already knows a great deal. Two 2025 studies then ask the 2018 question in a new form: what does it actually learn?

The first, by Yang Yue and co-authors at Tsinghua University, presented as an oral at NeurIPS 2025, compares a model before and after RL with verifiable rewards. It measures pass@k, the share of problems solved within at most k attempts. For a single attempt, the trained model wins clearly. When the base model is given hundreds of attempts, it solves as many problems, or more. The authors conclude that the reasoning found after RL was already in the base model: RL teaches the model to produce it on the first try, without adding new reasoning. Distillation, which makes a model imitate a stronger one, does add some²⁰.

The second, by Rulin Shao and co-authors at the University of Washington, is the closest to the Vancouver slide²¹. On the Qwen2.5-Math-7B model, RL with the true reward gains 29.1 points on the MATH-500 benchmark. With a reward that praises wrong answers, 24.1 points. With a reward drawn at random, with no relation to whether the answer is right, 21.4 points.

The explanation lies in a quirk of the training algorithm, which amplifies a behaviour already present in this model: writing its reasoning as code, without running it. This habit goes from 65% to more than 90% of answers; it happens to pay on this benchmark. The same treatment gives almost nothing, or lowers the score, on the Llama and OLMo families. The authors recommend validating any RL method on several model families rather than on the one model everybody uses. In 2018 the trap was called the seed; in 2025 it is also called the starting model.

The noise has not gone away either. A team led from the University of Tübingen re-ran in 2025 the evaluations of nine reasoning models with twenty seeds: on the AIME 2024 competition, the standard deviation of the first-try score ranges from 5 to 15 points from one seed to another; a single question more or less moves the score by 3.3 points out of thirty. Several gains announced for RL fell within that margin²². The cost has changed scale: a 2025 study on how to scale RL for large models used more than 400,000 GPU hours²³. At that price, ten seeds become a budget line.

Andrej Karpathy, a co-founder of OpenAI, summed up the state of things in October 2025: "Reinforcement learning is terrible. It just so happens that everything that we had before it is much worse." He describes the agent that works a whole minute to receive a single grade at the end, then applied to every one of its actions, good and bad alike: "you're sucking supervision through a straw"²⁴.

Joelle Pineau went on to lead Meta's fundamental AI research. She left in the spring of 2025; in August, the Canadian company Cohere named her chief AI officer, based in Montreal²⁵. She remains a professor at McGill.

Should I go to RL?

My answer has changed in form since 2018, not in substance. RL is an expensive, powerful, temperamental tool that pays under precise conditions. I check them in this order.

An environment that costs almost nothing. RL learns by trying millions of times. It needs a simulator, a game, an automatic checker, something you can restart without cost or danger. A real robot, a patient, a customer cannot be restarted a million times. Without a faithful simulator, RL remains an idea.

A reward you check without error. The success of 2025 comes from rewards that a program checks: the answer to the maths problem is right, the code passes its tests. As soon as the reward becomes an approximation, a grade given by another model, a business indicator you hope is linked to the goal, the agent learns to push the indicator up, which then drifts away from the goal. The random-reward experiment is a reminder that a measured gain does not prove the reward did the work.

The budget for ten seeds and a well-tuned baseline. If you cannot run each configuration ten times and compare against a baseline tuned with the same care and the same budget, you will not know whether you won. In that case, I start with the simplest method that works: supervised learning, distillation from a stronger model, better data. That is almost always where I end up.

When the three conditions are met, I go without second thoughts: that is what the teams now training models to reason or to code have done, with automatic checkers and compute budgets to match. When one is missing, I do not go. I then explain it to the client with the Vancouver slide.

The grid for reading an RL announcement

When someone shows me an RL paper, press release or demo, I ask seven questions. None requires knowing how to program.

• How many seeds? Fewer than five, I learn nothing from it; "the best runs", I close the document.
• Which interval? A mean without spread says nothing; I want a confidence interval, or at least a band on the curve.
• Which baseline, tuned how? If the new method got a month of tuning and the baseline its default values, the gap measures the month.
• On how many tasks, or model families? One task, one starting model: the result holds for that pair.
• Tested where? On the training task, the agent may have learned by heart; I want an evaluation on situations it has not seen.
• Which code? Without published code, I do not know what the paper actually did; the decisive details are almost never in the text.
• What cost? The compute hours, in plain figures. A result you cannot afford cannot be reproduced.

The seed

I still have not gone to RL. What I did take up, that May morning in 2018, was asking every result the question she asked her students: did you question your good results as much as your bad ones? It applies to RL, to supervised learning, to my own curves. RL simply has the honesty to make the problem visible: its seeds produce gaps large enough that nobody can ignore them any more.

The two Vancouver curves had the same code. They did not have the same seed. The experimental science of this field plays out in what you do with that gap: publish it as a discovery, or measure it as noise.

References

Each title below was used to write a specific section; the note says which. The video of the Vancouver talk is on YouTube; the slides of the NeurIPS 2018 version are published by the conference.

1. Tesauro, G. (1995). Temporal Difference Learning and TD-Gammon. Communications of the ACM, 38(3), pp. 58-68. bkgm.com TD-Gammon, the backgammon player trained by playing against itself: for twenty years, the result RL quoted for lack of a better one.
2. Silver, D., Huang, A., Maddison, C. J. et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529, pp. 484-489. nature.com AlphaGo, the result that replaced TD-Gammon in talks; Pineau regretted that it came without a blueprint for redoing it.
3. Irpan, A. (2018). Deep Reinforcement Learning Doesn't Work Yet. Blog post, February 14, 2018. alexirpan.com A practitioner's suspicion, cited by Pineau in her NeurIPS version: "at least 70% of the time".
4. Pineau, J. (2018). Reproducibility, Reusability, and Robustness in Deep Reinforcement Learning. Invited talk, ICLR 2018, Vancouver, May 3, 2018. iclr.cc The talk itself, whose video and slides are the main source of this article.
5. Bollen, K., Cacioppo, J. T., Kaplan, R. M. et al. (2015). Social, Behavioral, and Economic Sciences Perspectives on Robust and Reliable Science. Report to the National Science Foundation. Copy hosted by Harvard The definition of reproducibility that opens the talk.
6. Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533, pp. 452-454. nature.com The survey of 1,576 researchers: 52% significant crisis, 38% slight crisis. Pineau says "50%" in the talk; her slide says 52%.
7. Buckheit, J. B. and Donoho, D. L. (1995). WaveLab and Reproducible Research. In Wavelets and Statistics, Springer, pp. 55-81. doi.org The sentence about the article that is merely the advertising of the scholarship, taken from Jon Claerbout.
8. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D. and Meger, D. (2018). Deep Reinforcement Learning that Matters. AAAI 2018. arxiv.org The written source of the slides: the ranking by task (table 3), the three implementations of TRPO and DDPG, the two groups of five seeds (figure 5, t = −9.09, p = 0.0016) and the number of runs in published papers.
9. Pineau, J., Vincent-Lamarre, P., Sinha, K. et al. (2021). Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program). Journal of Machine Learning Research, 22(164), pp. 1-20. jmlr.org The tally of the reproducibility challenges (123 papers claimed in 2018) and of the checklist: code provided rises from fewer than 50% to 74.4% of accepted papers.
10. Pineau, J. (2018). Reproducible, Reusable, and Robust Reinforcement Learning. Invited talk, NeurIPS 2018, Montreal, December 5, 2018. Slides The Montreal version: the two-seed slide with its written answer, the grid of fifty 2018 papers and the first checklist.
11. Harchaoui, W. (2026). Remesurer Pineau: Pineau's 2018 grid applied again to 50 RL papers from 2025-2026. Protocol, data and codings, November 8, 2026. deraison.ai Our re-measurement: sampling, grid, calibration on twelve 2018 papers, double reading, deviations from the protocol.
12. Engstrom, L., Ilyas, A., Santurkar, S. et al. (2020). Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO. ICLR 2020. arxiv.org The "code-level optimizations" that explain most of PPO's lead over TRPO.
13. Andrychowicz, M., Raichuk, A., Stańczyk, P. et al. (2021). What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study. ICLR 2021. arxiv.org More than 250,000 agents to measure the effect of more than fifty implementation choices.
14. Agarwal, R., Schwarzer, M., Castro, P. S., Courville, A. and Bellemare, M. G. (2021). Deep Reinforcement Learning at the Edge of the Statistical Precipice. NeurIPS 2021, outstanding paper award. arxiv.org With three to five runs, rankings that flip; confidence intervals and the rliable library.
15. Ouyang, L., Wu, J., Jiang, X. et al. (2022). Training language models to follow instructions with human feedback. arxiv.org InstructGPT: RL from human preferences; the 1.3-billion-parameter model preferred to GPT-3.
16. OpenAI (2022). Introducing ChatGPT. November 30, 2022. openai.com The release of ChatGPT, trained with the same method.
17. Association for Computing Machinery (2025). ACM A.M. Turing Award Honors Two Researchers Who Led the Development of Cornerstone AI Technology. March 5, 2025. acm.org The 2024 Turing Award to Barto and Sutton.
18. Silver, D. and Sutton, R. S. (2025). Welcome to the Era of Experience. Chapter to appear in Designing an Intelligence, MIT Press. storage.googleapis.com The era of experience: agents that learn mostly from their own attempts.
19. DeepSeek-AI (2025). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645, pp. 633-638. nature.com R1-Zero: from 15.6% to 77.9% on the first try on AIME 2024 (71.0% in the January 2025 version); $294,000 for the RL stage.
20. Yue, Y., Chen, Z., Lu, R. et al. (2025). Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? NeurIPS 2025. arxiv.org pass@k: the base model catches up with the trained one when given enough attempts; distillation, on the other hand, adds.
21. Shao, R., Li, S. S., Xin, R. et al. (2025). Spurious Rewards: Rethinking Training Signals in RLVR. ICML 2026. arxiv.org The random reward that gains 21.4 points for Qwen2.5-Math-7B but almost nothing for Llama or OLMo.
22. Hochlehnert, A., Bhatnagar, H., Udandarao, V. et al. (2025). A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility. arxiv.org Twenty seeds, nine models: 5 to 15 points of standard deviation on AIME 2024.
23. Khatri, D., Madaan, L., Tiwari, R. et al. (2025). The Art of Scaling Reinforcement Learning Compute for LLMs. arxiv.org More than 400,000 GPU hours to study how RL scales.
24. Patel, D. and Karpathy, A. (2025). Andrej Karpathy: AGI is still a decade away. Dwarkesh Podcast, October 17, 2025. dwarkesh.com "Reinforcement learning is terrible" and supervision sucked through a straw.
25. Zeff, M. (2025). Cohere hires long-time Meta research head Joelle Pineau as its chief AI officer. TechCrunch, August 14, 2025. techcrunch.com Joelle Pineau's arrival at Cohere after leaving Meta.