The future of AI training data: why proprietary data is the next frontier.

Is AI running out of training data?

Public human-written text is close to used up. Epoch AI puts the effective stock of public human text at about 300 trillion tokens and estimates, with 80% confidence, that it will be fully used at some point between 2026 and 2032 if training runs keep growing at the current rate. That is a forecast, and the range is wide, but the direction is not in dispute.

Lab leaders say the same. At NeurIPS 2024, OpenAI co-founder Ilya Sutskever said that "data is not growing because we have but one internet" and that "we've achieved peak data." Compute keeps getting cheaper. The supply of new, high-quality human data does not grow at the same speed. That gap is what people mean by the data wall.

The open web is also getting harder to collect. On 1 July 2025 the network provider Cloudflare changed its default to block AI crawlers unless they pay for content, and began testing a pay-per-crawl model. Data that was once free to scrape now often comes with a price or a refusal.

Can synthetic data replace real data?

Not by itself. In a 2024 Nature paper, Shumailov and colleagues showed that when models are trained on data generated by earlier models, generation after generation, the rare parts of the original data disappear. They call this model collapse, and they found it in language models and in simpler statistical models.

The same paper is careful about the limits of that finding. Training on generated data is possible, but the data has to be filtered, and the authors note that genuine human data becomes more valuable as more of the web is written by models. In practice synthetic data is useful for:

• Expanding data where real examples already exist and can check the output
• Practice problems with answers that can be verified, such as code and math
• Rephrasing or translating real data a lab already holds

It cannot create information about the world that no real source contains. A model cannot invent how a specific claims process runs or how a hand grips a part on an assembly line. That has to be recorded.

Where is the data that AI models have not seen?

Most of it sits in two places: inside companies and in the physical world.

• Operational logs and telemetry that show how systems and people behave over years
• Transactions, with the sequence of decisions behind them
• Call and meeting audio with transcripts, which record how experts reason out loud
• Internal documents: manuals, case files, reports and correspondence
• First-person video of hands doing real tasks, for robotics and physical AI

Labs have asked for this kind of data directly. OpenAI's Data Partnerships program, launched in November 2023, asked organizations for large datasets that are hard to find online and offered a private track in which a partner's data trains OpenAI's own models.

Robotics shows the physical side. Robots need examples of tasks seen from the worker's point of view, and the web has little of it. In 2026 NVIDIA researchers trained a robot manipulation model on 20,854 hours of egocentric human video in their EgoScale paper. Data at that scale mostly has to be recorded on purpose. More on egocentric video data.

How big is the AI data licensing market?

The largest public deals so far involve content platforms. Figures are as reported; see sources.

Around those deals, a market for data supply is forming. The market size below is a forecast from one research firm. The other rows are reported facts.

Mercor's figure is gross revenue, most of which is paid out to the experts who produce the data. It shows how much labs now spend on expert-made data for post-training, where people write, rank and check answers in their own field.

What changes for AI labs?

When the useful data is spread across thousands of companies, finding it becomes a job of its own. A lab has to know who holds what, agree terms with each holder, and keep track of rights for every record.

• Sourcing becomes a function: scoping a need, finding holders, pricing and contracting
• Provenance matters: a lab needs to show where each record came from and what license covers it
• Cleaning comes first: duplicates and personal data have to be removed before training
• Where no data exists, it has to be recorded, for example first-person video of a task

What changes for companies that hold data?

Data that was a by-product of running the business becomes an asset that can be licensed. A license gives a lab the right to train on a defined copy. The company keeps the data and can license it again unless it agrees to exclusivity.

Value depends on rarity, depth of history, structure and clear rights. Before selling, a company has to check its contracts and privacy notices and remove personal data. How to sell your company's data to AI labs.

What will AI training data look like in the next few years?

These are our predictions, not facts. They follow from the trends above.

• Prediction: more training budgets go to licensed and commissioned data than to web collection
• Prediction: per-record provenance and license terms become a standard part of dataset delivery
• Prediction: egocentric and sensor data for robotics becomes one of the fastest-growing data types
• Prediction: more companies with operational records treat licensing as a line of revenue
• Prediction: synthetic data grows, but stays tied to real data that anchors and checks it

What should labs and data owners do now?

Labs should write down the data a model is missing, then source it with the rights and provenance attached. Data owners should take stock of what they hold, check their rights and get a valuation against real buyer demand.

Indri Labs sources proprietary training data for AI labs. Scoping is free, and the brief is priced: every source named and priced, a sample of each, and our fee on its own line. On approval there is one contract. We license data from the companies that hold it, usually ones that have never sold it, such as logs, telemetry, transactions, call and meeting audio with transcripts, and documents. Where none exists, we record new first-person video on smart glasses and head cameras. We remove duplicates and personal data and deliver one dataset with the source and license for every record. We also help companies sell their data to AI labs. How Indri Labs works.