Reddit User Says Hugging Face Mythos-Class Weights Look Synthetic

Reddit user johnxreturn said a limited sample of a Hugging Face repository claiming to hold an Anthropic Mythos-class checkpoint showed most of the sampled weights appeared synthetic.

The repository, burner22/numbat, was created Oct. 23 and circulated on Reddit Sunday, Nov. 1. Its description calls the files a mid-training checkpoint from Anthropic's next Mythos-class run, with about 14 trillion parameters and 1,224 experts. It lists 7.72 terabytes of data split across 772 shards and does not identify a numbered Mythos version.

johnxreturn said the analysis sampled only a small portion of the files and compared them with Alibaba's Qwen2.5-7B. The analysis described the checkpoint format, tokenizer and word-embedding matrix as plausible, but said most other sampled weights appeared synthetic. johnxreturn estimated a 0.5% chance that the files came from an authentic Anthropic checkpoint.

The analysis pointed to similarities between all 131,072 rows in the output layer and corresponding word embeddings, with scores around 0.89. johnxreturn compared that with an average of 0.0144 for Qwen2.5-7B. The analysis also cited calibration values of 0.185 and 0.087 for two modules said to receive the same input; johnxreturn said those values should match.

johnxreturn also said statistics across three types of weight matrices differed by 0.4%, while values across input channels in a real model would be expected to vary by two to five times. The listed parameter count was near 6.85 trillion, but johnxreturn said that figure omitted one data format and estimated the adjusted count at about 13.68 trillion, close to the repository's claim.

The repository description says the files use Nvidia's NVFP4 format for testing on Amazon Trainium2 chips. Reddit user Strider96 challenged that description, saying NVFP4 did not match Trainium2's listed format support. Repository files list two different question counts for an MMLU test: 14,042 and 281.

Strider96 warned against running custom Python code included with the repository. HideousSerene raised a concern that the Reddit post could be bait to encourage downloads of a malicious model.

Anthropic limited Mythos 5.1 to trusted-access programs; Japan had access to Mythos Preview while participating in Project Glasswing.