Five Small Models, One Team That Beats a Giant

Last week, the Memdoor project published a result that deserves serious attention in the artificial intelligence ecosystem: five instances of the Qwen3.8-27B model, organized into a multi-agent workflow, achieved a mean score of 9.8 out of 10 on LiveCodeBench Hard—outperforming Claude Fable 5, a much more expensive model with many more parameters, which scored 8.0 on the same benchmark.

This is not an insignificantly small margin. 9.8 versus 8.0 is the difference between solving 49 of 50 proposed problems and managing only 40. And the cost? Each median run cost just $0.39. Fable 5, run via Anthropic's API, costs significantly more per call.

The architecture: division of labor without conversation

What makes Memdoor interesting is not the model itself—Qwen3.8-27B is already competent for coding tasks—but the way the agents organize themselves. Five distinct roles, none sharing chat context:

- Planner: reads the problem and writes a `plan.md` - Ideator-A and Ideator-B (two parallel idea generators): each produces an algorithm proposal in its own file - Solver: reads both proposals and writes the `solution.py` - Manager: reads test output and decides whether repair is needed - Repair worker: fixes the `solution.py` based on the manager's instructions

Each agent enters "cold"—without any prior conversation history. Each reads only the specific files needed for its task. Communication happens exclusively through files written to disk. There is no shared context, no memory between steps.

The architecture faithfully reproduces the GVS5H harness, published by researcher Lee in 2026. Memdoor did not invent this approach—it implemented it, tested it, and published the results transparently, with every API call and every result logged directly in the repository.

The numbers

The results are documented in the project's README, with raw data from every execution:

| Pass | Memdoor (this workflow) | GVS5H (original harness) | Fable 5 (alone) | |------|------------------------|-------------------------|-----------------| | 1 | 10/10 | 9 | 7 | | 2 | 10/10 | 10 | 8 | | 3 | 10/10 | 10 | 10 | | 4 | 10/10 | 9 | 8 | | 5 | 9/10 | 10 | 7 | | Mean | 9.8 | 9.6 | 8.0 |

Fifty problems were tested (ten problems × five passes). The Memdoor team solved 49 of 50. The total cost for the 50 successful runs was $35.79, consuming 15.3 million output tokens. Including the 16 incomplete attempts, the total cost rose to $38.77.

A median run took 40 minutes and cost $0.39. Each problem therefore costs less than 40 cents.

The contamination question

The authors themselves are transparent about a critical limitation: the LiveCodeBench Hard problems date from March 29 to April 6, 2025. Qwen3.8-27B may have seen these problems during training. Contamination cannot be ruled out for any column—neither Memdoor's nor Fable 5's.

This does not invalidate the result, but places it correctly: it is a signal, not definitive proof of superiority. What Memdoor demonstrates robustly is that multi-agent architecture works—and that a well-designed harness can extract significantly more performance from smaller models than a single call can.

What this means for the future of development

There are two practical implications worth considering.

The first is economic. $0.39 per competitive programming problem is a cost that any API-based coding agent tool has difficulty matching. If development teams can replicate part of this performance for real-world tasks—not just synthetic benchmarks—the cost of AI-assisted development drops dramatically.

The second is about autonomous agent architecture. Memdoor shows that you do not need shared context or persistent memory to coordinate agents. File-based communication on disk is robust enough for complex problem-solving tasks. This has direct implications for security: agents that communicate via files are easier to audit than agents that share conversations, and it is easier to isolate a compromised agent when each operates in its own directory with minimal file access.

The Hacker News community already pointed out an interesting detail: with Bonsai 2 quantizations, Qwen 27B can run at 50 tokens per second on reasonable gaming hardware (16 GB VRAM, 32 GB RAM). This means the same performance that beat Fable 5 could, in theory, run locally on an affordable machine.

The future of AI-assisted software development does not necessarily depend on ever-larger models. It depends on finding the right architecture—and Memdoor is a step in that direction.

Sources: Emergent, AI Weekly, GitHub — Memdoor team-solve

✓ Independent sources cross-checked and verified before publishing