AI Models May Retain a Trace of the Machines That Initialized Them

A trained artificial intelligence (AI) model may retain a numerical trace of the setup used to create its starting values. That could give researchers a new way to examine the history of systems whose development is only partly documented.

In a November 12 report, Will DePue of Unemployment Labs describes initialization patterns in six models from DeepSeek, Moonshot AI, MiniMax and Ant Group that match layouts associated with NVIDIA’s Blackwell chips. The findings concern the creation of the starting weights; identifying the equipment used throughout training requires further evidence. Unemployment Labs report

How a starting pattern can survive

Weights are the adjustable numbers a neural network uses to process information. Before learning begins, software fills them with initial values. A pseudorandom-number generator produces an apparently irregular sequence from a starting number called a seed. Within a fixed implementation, repeating the seed and generator settings repeats the sequence. NVIDIA’s cuRAND documentation describes how seeds, subsequences and offsets control that process. NVIDIA cuRAND documentation

Parallel processing adds an arrangement to those values. The relevant PyTorch code sets the work grid using the data size and two properties of the graphics processing unit (GPU): its processing-unit count and thread capacity. Workers generate sequences and place their values at assigned positions. Changing the grid can change where values land, even when the seed stays fixed. PyTorch distribution code

DePue reports recovering traces in weights that training preserved or only partly changed. Comparing candidate layouts against separate, held-out weights helps check whether a match extends beyond the values used to find it. His report also describes reproductions on physical GPUs.

Reproducing a pattern and identifying a chip

A public repository, checkpoint-forensics-repro, reports reproducing the proposed layouts for the six models and the older DeepSeek-V3 from published embedding weights. It emulates the calculation on a central processing unit (CPU) and takes the mappings from layouts to chip families from the original report. Its tests exclude physical hardware checks, the AMD comparison and the weaker traces in trained layers. Public reproduction and limits

DePue also reports that specially configured AMD MI300X hardware can reproduce these layouts. His additional numerical comparison favors NVIDIA for DeepSeek-V4-Flash, while the AMD alternative remains open for the other five models. The weights also leave location, ownership and the terms of access undetermined. Report limitations

PyTorch warns that reproducibility depends on software versions and platforms. A useful forensic result therefore needs its software assumptions and comparison data recorded alongside the match. PyTorch reproducibility guidance

Sources

• Will DePue’s November 12 report and its limitations

• Public CPU reproduction, results and untested components

• PyTorch code governing the random-number work grid

• NVIDIA cuRAND device API documentation

• PyTorch reproducibility guidance