# v0.0.7 frozen blind result: Qwen2.5-Coder-7B

## Outcome

The v0.0.7 publication gate failed on the untouched HumanEval 40-59 slice.

| Metric | LIR | Python |
|---|---:|---:|
| Tasks | 20 | 20 |
| Pass@1 | 0.0% | 100.0% |
| Final pass rate after one repair | 0.0% | 100.0% |
| Generated o200k tokens | 2,266 | 927 |
| Repair-prompt o200k tokens | 3,568 | 0 |
| Session-amortized total tokens | 8,034 | 2,964 |
| Tokens per correct solution | n/a | 148.2 |

Because LIR solved no tasks, it has no finite tokens-per-correct value and no compression claim can be made from this run.

## Frozen-evaluation configuration

- Model: `Qwen/Qwen2.5-Coder-7B-Instruct`
- Tasks: official HumanEval 40-59, prepared without `canonical_solution`
- Decoding: temperature 0, maximum 1,200 output tokens
- Repair budget: one repair for both languages
- Runtime: OpenAI-compatible vLLM `v0.11.0`, FP16, eager mode, 4,096-token context
- Frozen language version: `0.0.7`
- Frozen language SHA-256: `e2a9592b895f9d4c16f3ba639d09c51d12ce0df1bf0558657be5fef79d875f31`
- Prepared blind-task SHA-256: `d0d5b952b14761a897c551d3e55d2f9ef2db97b65ce2a18fca13181268b31c5b`

The transport-only `User-Agent` correction was made after RunPod's proxy returned HTTP 403 to Python's default `urllib` user agent. It did not alter prompts, tasks, decoding, language semantics, or generated outputs.

## Failure taxonomy

All 40 LIR attempts failed at compilation before functional tests ran.

- 16 of 20 first attempts used a space before the function-body brace, for example `f(a) {`, while the v0.0.7 line parser accepts only `f(a){`.
- 3 of 20 used spaces around an expression-function equals sign, for example `f(a) = expr`, while the parser accepts only `f(a)=expr`.
- 1 of 20 initially emitted Python.
- 13 of 20 repair attempts repeated the first output exactly.

Primary classification: syntax/parse failure caused by whitespace-sensitive parser rules plus model misunderstanding of the compact grammar. The algorithms were generally not reached, so this run does not establish whether the underlying generated logic would have passed after normalization.

## Decision

Do not publish a leadership, no-quality-loss, or token-saving claim for v0.0.7. Preserve these raw results. A v0.0.8 candidate should be developed separately with whitespace-tolerant lexical normalization and clearer canonical examples, then frozen before testing on a new untouched slice.

Raw files:

- `benchmarks/external/blind/results/qwen25coder7b_lir.json`
- `benchmarks/external/blind/results/qwen25coder7b_python.json`
- `benchmarks/external/blind/results/qwen25coder7b_compare.md`