Artificial intelligence is in the news every day, usually described in terms of models: bigger networks, new chatbots, smarter assistants. The less visible half of the story is data. Every one of those models learned what it knows from examples, and the quality, quantity and variety of those examples decide what it can do.
This post is a plain-language starting point. It explains what AI, machine learning and deep learning are and how they relate, why data has become the fuel of all three, and where the popular phrase “data is the new oil” stops being helpful. It ends with the eight main challenges that come with data, two of which, data locked away in separate organisations and privacy law, lead into my series on federated learning. Two short pieces of Python show the key ideas running.
AI, machine learning and deep learning
The three terms are often used as if they meant the same thing. They are better pictured as circles inside each other.
- Artificial intelligence (AI) is the broadest term: any computer system that does tasks we associate with intelligence, such as planning a route, recognising speech or playing chess. Early AI was mostly hand-written rules: experts wrote down what they knew as “if this, then that”, and the computer followed it.
- Machine learning (ML) is the part of AI where the system learns the rules from examples instead. You do not tell a spam filter which words are suspicious; you show it thousands of emails marked “spam” or “not spam”, and it works out the patterns itself.
- Deep learning is machine learning with neural networks of many layers. Older methods needed people to describe the data first, for example by measuring the edges in an image. Deep networks learn those features on their own, straight from raw pixels, sound or text. That is why they took over image recognition, speech and translation.
- Generative AI and large language models (LLMs) are deep learning models that produce new content, such as text, images or code, rather than only labelling what they are shown. ChatGPT is the best-known example.
A short history, told through data
AI as a field is about 70 years old, and for most of that time progress was limited by computers and by ideas. Since about 2010, it has been limited mostly by data.
The turning point was a dataset, not an algorithm. ImageNet, first presented in 2009, set out to label millions of images by hand, each with what it shows, and grew to over 14 million. In 2012, a deep network called AlexNet, trained on 1.2 million of those images, beat every other method in the ImageNet competition by a wide margin. The ideas behind it were decades old; what was new was enough labelled data, and graphics cards fast enough to learn from it.
The same pattern has repeated since, at a scale that is hard to picture. GPT-3, in 2020, was trained on about 300 billion tokens, pieces of words (Brown et al., 2020). Meta’s Llama 3, four years later, used over 15 trillion: fifty times more. Researchers at Epoch AI estimate the stock of good-quality public text on the web at around 300 trillion tokens. If current trends continue, the largest models will be trained on datasets of about that size between 2026 and 2032 (Villalobos et al., 2024). The easy data is running out.
How a model learns from data
Learning from examples has a simple consequence: more examples, and more varied examples, make a better model, up to a point. You can see this with a classic dataset of 1,797 small images of handwritten digits that ships with the Python library scikit-learn. The code trains the same simple model on more and more images, and tests it each time on 500 images it has never seen.
import numpy as np
from sklearn.datasets import load_digits
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
# 1,797 small images of handwritten digits, 0 to 9, that ship with scikit-learn
X, y = load_digits(return_X_y=True)
X_pool, X_test, y_pool, y_test = train_test_split(X, y, test_size=500, random_state=0, stratify=y)
rng = np.random.default_rng(0)
print("training images accuracy on 500 new images")
for n in [10, 30, 100, 300, 1000, len(X_pool)]:
scores = []
for _ in range(5): # average over 5 random draws
pick = rng.choice(len(X_pool), n, replace=False)
model = LogisticRegression(max_iter=2000).fit(X_pool[pick], y_pool[pick])
scores.append(model.score(X_test, y_test))
print(f"{n:>15,} {np.mean(scores):.1%}")
training images accuracy on 500 new images
10 44.6%
30 67.6%
100 89.2%
300 93.8%
1,000 96.2%
1,297 96.4%
With ten random examples, about one per digit, the model is right less than half the time. With a hundred, it is right nine times out of ten, and by a thousand it has nearly reached its limit. Two lessons follow. Data is what turns a model from useless to useful. And once the common cases are covered, the remaining gains come from the rare, hard cases, which are exactly the examples that are scarce in any single dataset.
Data is the new oil, up to a point
“Data is the new oil” is a phrase often attributed to the British mathematician Clive Humby, from 2006, and it became famous when The Economist declared data “the world’s most valuable resource” in 2017. As a metaphor, it gets some things right. Data is a raw material that has to be extracted, refined and transported before it is useful, a few large companies control much of it, and whole industries now run on it.
But data is unlike oil in ways that matter more:
- It is not used up. The same dataset can train a thousand models. That makes it hugely valuable, and it also means a leaked copy can be misused forever.
- It is often about people. Oil does not have rights. A patient record or a message does, and laws such as the GDPR say who may use it and for what.
- Refining is most of the work. Raw data has to be cleaned, checked and often labelled by hand before a model can learn from it. In a well-known study of AI teams, 92% of practitioners had run into at least one problem caused by poor data that compounded through the project (Sambasivan et al., 2021).
- More is not always better. A barrel of oil is a barrel of oil. A million examples from the wrong population can make a model worse for the people it is meant to serve.
- Spills are invisible. When data leaks, nobody sees a slick on the beach, but the damage, from identity theft to discrimination, can last for years.
So data is better thought of not as a fuel to burn, but as something closer to a valuable, sensitive and reusable asset. That changes which problems you have to solve.
The eight challenges
1. Too little data
Common cases are easy to collect; rare ones are not. A hospital may see thousands of ordinary chest X-rays a year but only a handful showing a rare disease, and those few are exactly the cases where a second opinion from a model would help most. The learning curve above shows the problem: a model with too few examples of something is mostly guessing.
2. Poor quality
Real data has typing errors, missing values, duplicates, inconsistent units and wrong labels. Even ImageNet, one of the most carefully built datasets in AI, has thousands of mislabelled images in the validation set used to test models (Northcutt et al., 2021). A model has no way of knowing which examples are wrong; it learns from all of them.
3. Bias and missing groups
A model learns the world as its data shows it. If some groups are underrepresented, it works worse for them, often without anyone noticing, because the overall accuracy still looks fine. In a well-known audit, commercial face-analysis systems misclassified the gender of darker-skinned women up to 34.7% of the time, against at most 0.8% for lighter-skinned men (Buolamwini and Gebru, 2018).
The effect is easy to reproduce. Here, a condition depends on the same two measurements in two groups of patients, say from two regions, but in a different way. The code trains a model on different mixes of the two groups and tests it on each.
# Two groups of patients, say from two regions. The same two measurements
# predict the same condition, but the relationship differs between the groups.
def patients(n, group):
x = rng.normal(0, 1, (n, 2))
if group == "A":
sick = x[:, 0] + 0.3 * x[:, 1] > 0.5
else: # in group B, the second measurement matters more
sick = 0.3 * x[:, 0] + x[:, 1] > 0.5
return x, sick, np.full(n, group == "B")
def features(x, is_B, knows_group):
if not knows_group:
return x
return np.column_stack([x, is_B, x * is_B[:, None]]) # lets the model treat B differently
test_A, test_B = patients(5000, "A"), patients(5000, "B")
def train_and_test(n_A, n_B, knows_group=False):
xa, ya, ga = patients(n_A, "A")
xb, yb, gb = patients(n_B, "B")
x, y, g = np.vstack([xa, xb]), np.concatenate([ya, yb]), np.concatenate([ga, gb])
model = LogisticRegression(max_iter=1000).fit(features(x, g, knows_group), y)
return [model.score(features(t[0], t[2], knows_group), t[1]) for t in (test_A, test_B)]
print("training data group A group B")
for n_A, n_B, knows in [(1900, 100, False), (1000, 1000, False), (1900, 100, True), (1000, 1000, True)]:
a, b = train_and_test(n_A, n_B, knows)
label = f"{n_A:>5} A + {n_B:>4} B" + (", group-aware" if knows else "")
print(f"{label:<37} {a:>7.1%} {b:>7.1%}")
training data group A group B
1900 A + 100 B 98.9% 73.6%
1000 A + 1000 B 86.4% 85.7%
1900 A + 100 B, group-aware 99.7% 92.9%
1000 A + 1000 B, group-aware 98.8% 99.6%
Trained mostly on group A, the model is excellent for A and poor for B: 98.9% against 73.6%, while its overall accuracy still looks good. Simply balancing the data shares the pain instead of removing it. What works is enough data from both groups, and a model that can learn how they differ: then both reach about 99%.
In real life, group B’s data is usually held by someone else: another hospital, another region, another country. Getting it is the hard part.
4. Drift
A model is a snapshot of the world at the time its data was collected. When the world changes, the model does not. In spring 2020, many models for forecasting demand, detecting fraud and planning logistics struggled within weeks, because people’s behaviour changed overnight. Models need to be monitored and retrained on fresh data, which means the data problem is never solved once and for all.
5. Poisoning
If a model learns from whatever data it is given, an attacker can teach it the wrong things by planting bad examples. This is not only a theoretical risk: researchers showed that, for about 60 US dollars, they could have poisoned 0.01% of two widely used image datasets collected from the web, by buying expired domain names that the datasets still pointed to (Carlini et al., 2023).
6. Ownership and consent
Who owns the data a model learns from? Writers, artists and news organisations have taken AI companies to court over training on their work, The New York Times against OpenAI and Microsoft among them. Inside organisations, the question is just as real: data collected to treat patients or run machines was not necessarily collected to train AI, and using it for a new purpose may need new permission.
7. Data locked in silos
The most valuable data is often spread across many owners, none of whom has enough on their own. Every hospital has some rare cases, every factory has some failures, every bank sees some fraud. Together, they would have the variety the bias example above needed. But each keeps its data to itself, for reasons of law, business and trust.
8. Privacy law
Much of that data is about people, and the law protects it. The GDPR in Europe, HIPAA for health care in the United States, and similar laws elsewhere limit how personal data may be used, shared and moved across borders, with large fines for getting it wrong. Simply removing names is rarely enough to make data anonymous.
Where this leads: learning without sharing
The last two challenges are connected, and together they block much of the most valuable AI work: models that need data from many organisations, which cannot legally or practically pool it. There are several ways to work around that, from anonymisation and synthetic data to encryption, and each has limits. For training a model across several organisations, the most practical answer today is usually federated learning: instead of bringing the data to the model, it sends the model to the data.
That is the subject of my series on federated learning:
- Data you cannot move: GDPR, HIPAA and the alternatives to sharing: what GDPR, HIPAA and other laws require, the alternatives and their limits, and why federated learning is usually the best fit.
- Federated learning explained: training on data that never moves: how it works, where it is used and what makes it hard.
- Beyond FedAvg: how to choose a federated learning aggregator.
- Federated learning frameworks compared: Flower, NVIDIA FLARE, FEDn and more.
Further reading
- Brown et al. (2020). Language Models are Few-Shot Learners. The GPT-3 paper.
- Vaswani et al. (2017). Attention Is All You Need. The Transformer, the architecture behind today’s language models.
- Villalobos et al. (2024). Will we run out of data? Limits of LLM scaling based on human-generated data.
- Sambasivan et al. (2021). “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI.
- Buolamwini and Gebru (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.
Want to go further? Browse my publications, or look at the open thesis topics if you would like to work on these problems with us.