0G Labs (Home of Infinite AI) (@0G_labs): 107B parameters. No supercluster. Here's how.

107B parameters. No supercluster. Here's how. Jensen Huang went on the All-In Podcast last week and said something that moved markets: decentralized AI training works. $TAO jumped 17%. Grayscale filed a TAO ETF. Every crypto outlet ran the story. Bittensor's Covenant-72B, trained by 70+ community contributors, became the proof point. But the biggest model ever trained on decentralized infrastructure didn't come from Bittensor. The paper no one read 0G published a framework called DiLoCoX in June 2025. Nine months before the Jensen clip went viral. DiLoCoX trained a 107 billion parameter model (modified Qwen1.5) across 160 GPUs spread over 20 nodes, all connected by standard 1 Gbps bandwidth. Not high-speed InfiniBand. Regular internet. The numbers: 3,728 tokens per second versus AllReduce's 10.4. That is a 357x speedup in communication efficiency. How it works Standard distributed training breaks on slow networks. Nodes spend more time sending gradients than training. DiLoCoX solves this with four techniques working together. Pipeline parallelism splits the model across nodes so computation and communication happen at the same time. A dual optimizer keeps local node updates aligned with the global model. One-step-delay overlap lets training continue using the previous step's gradient while the current one transmits. Adaptive gradient compression squeezes gradients 1,000x before sending. The ablation data shows each piece earning its place. Remove compression: loss improves slightly but throughput drops 3x. Remove overlap: similar trade-off. The full framework is the only configuration that makes 107B feasible at standard bandwidth. Covenant trained a model. DiLoCoX builds the framework Covenant-72B is a real achievement. 72B parameters, 70+ contributors, a working model on HuggingFace. Credit where it's due. The difference is scope. Covenant proved one model could be trained by a distributed community. DiLoCoX proved a framework can scale to 107B parameters while keeping communication costs 357x lower than standard methods. One produces a model. The other builds infrastructure for training any model. The verification gap Both projects face the same question: how do you verify that every node trained honestly? Bittensor relies on economic incentives. Bad actors lose stake. 0G uses Trusted Execution Environments: the hardware itself proves the computation was correct. As training runs get more expensive and models handle higher-stakes decisions, the gap between "incentives should work" and "hardware proved it worked" matters more. Michael Heinrich ( @michaelh_0g ) will expand on this at EthCC Cannes on April 1. What's next 0G is retraining DiLoCoX with extended data and open community participation. Weights and checkpoints will be released publicly when complete. The full 0G stack makes this practical: 0G Compute provides TEE-verified GPU resources, 0G Storage handles training data at 30+ MB/s throughput, 0G DA ensures data availability at a fraction of Ethereum costs, and 0G Chain settles everything onchain. No other project ships all four layers under one roof. Jensen validated the space. Bittensor trained 72B. 0G trained 107B. The real question is what gets built on these frameworks next. What would you train first on open decentralized infrastructure? Full deep dive from @0G_labs: https://0g.ai/blog/dilocox-107b-decentralized-ai-training