More Varied Training Helps Small AI Models Learn Abstract Rules
Greater variety in training examples helped small neural networks develop internal mechanisms for applying abstract rules, a Princeton University study reports.
Published November 16, 2026Briefs / Artificial Intelligence / Generative Models

Greater variety in training examples helped small neural networks develop internal mechanisms for applying abstract rules, a Princeton University study reports.
Melody Zixuan Li and Taylor Webb trained models from scratch on controlled sequence problems. They varied the vocabulary, number of examples and rules, then examined how the networks reached their answers.
The underlying question concerns transfer: can a model recognize a relationship when the words or symbols change? That ability is valuable whenever a system encounters a situation beyond its familiar examples.
The November 9 paper also connects with earlier work tracing symbolic operations inside much larger language models.
A rule that survives different words
In an A–B–A puzzle, the third item repeats the first. A model that learns the relationship can complete a new sequence even when its items look different from those in the example.
The Princeton team investigated representations of those roles inside networks. It used interventions on internal activity to test which signals affected the output. More diverse data produced stronger symbolic signatures in the tested tasks, including in simpler multilayer perceptrons.
A multilayer perceptron passes information through connected layers of processing units. It lacks a transformer’s attention machinery, which helps a transformer relate different parts of its input. Finding similar signatures in both designs broadens the question of what training can teach different architectures.
Building on earlier mechanistic research
A 2025 study led by Yukang Yang, with Webb among its authors, described a three-stage symbolic mechanism in pretrained language models. Early processing represented relations as abstract variables, intermediate processing continued a pattern over those variables, and later processing recovered the corresponding token for the answer.
That work investigated a mechanism already present in trained models. The new controlled experiments ask what conditions encourage related behavior to emerge during learning.
Changing the wording of training examples can leave their underlying rules much the same. The study suggests that varying those rules and situations deserves attention alongside expanding the number of examples.
The experiments establish that relationship in limited tasks. Extending it to a larger training program would require testing how new examples affect generalization across the work the model is meant to perform.