Experiments
Track A — gpt-oss 20B context ladder
Verdict: partial pass. On Owlforge’s RTX 3090 Ti, gpt-oss:20b passed strict five-needle retrieval at 17,445 and 68,916 prompt tokens. At 107,832 prompt tokens it recovered all five values internally and emitted all five exact values, but violated the preregistered strict-JSON contract. That cell is therefore scored 0/5, not rescued after the fact.
| Nominal archive | Actual prompt tokens | Strict score | Wall time | Peak VRAM | Peak power | Verdict |
|---|---|---|---|---|---|---|
| 16,000 words | 17,445 | 5/5 | 2.384 s | 15,918 MiB | 380.63 W | PASS |
| 64,000 words | 68,916 | 5/5 | 16.658 s | 15,918 MiB | 474.16 W | PASS |
| 100,000 words | 107,832 | 0/5 | 30.081 s | 15,918 MiB | 475.17 W | FAIL — output contract |
The 107,832-token response was not a retrieval miss: the reasoning trace identified every needle correctly, and the final channel contained each value. It returned newline-separated text instead of the required JSON object. The preregistered evaluator therefore failed it closed.
Two earlier runs are recorded as infrastructure failures rather than model-quality results:
- v1: Ollama 0.32.5’s JSON-Schema response mode exposed an empty final channel for this model.
- v2: the 256-token generation budget truncated an otherwise correct final object.
Public receipt: experiments/track-a/public/gpt-oss-20b-v3-receipt.json
Receipt SHA-256: e4ef0db25d7aee05f2eaecbbd5b5a6e43ac7cde01d67d658ea49f209292ab148
This control measures exact synthetic retrieval only. It does not establish long-context summarization, reasoning, tool use, or million-token usefulness.