Picture this: someone drops a GitHub link in your Discord with the bold claim "Someone just built AGI!" The repository name? hyperspaceai/agi. Red flags everywhere, but curiosity wins. What unfolds is a fascinating case study in impressive engineering wrapped around a fundamental misunderstanding of intelligence.
The Promise vs. The Reality
Hyperspace AI marketed itself as the first distributed Artificial General Intelligence (AGI) system. The numbers were impressive: 660 AI agents running across a peer-to-peer network, conducting 27,000 experiments autonomously. The infrastructure featured cutting-edge distributed computing techniques, gossip protocols, and collaborative training runs across 32 anonymous nodes.
But here's the kicker: their biggest "discovery"—the breakthrough that propagated to 23 agents within hours and served as proof their system worked—was Kaiming initialization.
If you're familiar with deep learning, you probably just winced. Kaiming initialization has been in PyTorch's standard library since 2015. It's literally covered in week two of every introductory deep learning course. The original paper by Kaiming He was published eleven years ago.
The Tech That Actually Impresses
Before we dismiss this entirely, let's acknowledge what Hyperspace did get right: the underlying infrastructure is genuinely impressive. They implemented several cutting-edge techniques:
DiLoCo (Decoupled Local Communication)
Standard distributed training requires constant synchronization—every GPU waits for every other GPU after each step. This works in data centers with high-speed connections but falls apart over the internet due to latency and bandwidth limitations.
DiLoCo solves this by allowing each node to train independently for many steps (called "inner steps"), then sync only occasionally. Instead of syncing gradients constantly, they share the net "delta" (weights_after - weights_before).
Compression Magic
Even sharing these deltas would be prohibitive—a 500M parameter model creates roughly 2GB of delta data per sync. Hyperspace stacked two compression techniques:
- SparseLoCo: Only send the largest-magnitude weight updates (top 2%), since most parameter updates are near-zero noise
- Parcae: Group adjacent transformer layers and average their gradients before sparsification
The result? A staggering 195× compression ratio, reducing sync payload from ~1GB to just 5.5MB per round.
Where Intelligence Goes to Die
Here's where the impressive engineering meets the harsh reality of what intelligence actually requires. Every single one of those 660 agents follows the exact same loop:
- Check the leaderboard for best scores
- Read last 5 experiment results
- Prompt an LLM: "Given these results, generate a hypothesis"
- Run the experiment
- Record results
- Share with peers
- Repeat
The fatal flaw? The agents have no memory beyond their current context window. When a session resets, everything resets. There's no persistent understanding, no causal reasoning about why things work, no structured knowledge building.
Hyperspace stores flat logs like: "run_047: threshold 0.30, score 0.67"
But it doesn't store:
- Why threshold 0.30 worked
- What conditions make it effective
- What failed before and why
- How different approaches relate to each other
The Kaiming "Discovery" Explained
So how did the system "discover" Kaiming initialization? Simple:
- The LLM generating hypotheses was trained on data including He et al. 2015
- A prompt mentioned "improving initialization"
- The model retrieved Kaiming init from its training weights
- An agent tested it
- It worked (as it should)
- 23 agents adopted it via gossip protocol
This isn't emergence or swarm intelligence—it's a very expensive way to retrieve information already embedded in pretrained models.
The Real Lesson
Hyperspace represents a classic case of solving the wrong problem brilliantly. They built impressive distributed computing infrastructure that could genuinely advance the field of collaborative AI training. The gossip protocols, compression techniques, and peer-to-peer coordination are all valuable contributions.
But they confused computational distribution with intelligence distribution. True AI breakthroughs require more than parallel processing—they need persistent memory, causal reasoning, and the ability to build structured knowledge over time.
The next time you see claims about revolutionary AI swarms or distributed AGI systems, ask the critical questions: Do these agents actually learn and remember? Can they build on previous insights? Or are they just very sophisticated ways of running parallel searches?
As for that GitHub repository that started this journey—well, at least the engineering was solid, even if the AGI claims were a bit premature.
Source: Original analysis by Vektor Memory, published on Towards AI