PACC-Hybrid v0.2 — first real-model run, on a local 9B open-weight model

Hypothesis

hypothesis

A real language model under the PACC runtime shows the coherence / valid-novelty gains and recoverable breadth loss seen in the synthetic witness (v0.2 predeclared interpretation).

Setup

model_ids

• MOD-2026-0006 Qwythos-9B-v2 (Q4_K_M, local, Ollama)

benchmark_ids

• BEN-2026-0003 PACC-LLM Hybrid A/B/C benchmark

software_environment

Python 3.14, openai SDK 3.0.0 against Ollama 0.33.3 /v1/responses; PACC-Hybrid-Lab v0.2 package unchanged (25 tests green before the run).

Procedure

procedure

scripts/run_real_local_ollama.py --candidate-count 4 --repetitions 2 (after a 6×2×3 smoke); summary by scripts/summarize_real_local.py; both scripts and both frozen caches are in the bundle.

Runs

run_count

1

random_seeds

• model nondeterminism, single run; frozen response cache in the bundle for exact replay

controls

• identical candidate ledger for A/B/C
• equal accounted calls
• condition-blind judge with deduplicated judge calls
• deterministic literal checks

metrics

verdict

REAL_LOCAL_9B_MIXED: supersession and repair up, coherence flat, valid novelty slightly down, breadth unmeasurable at two repetitions

execution_status

EXECUTED_REAL_MODEL

protocol

model

qwythos-9b-v2-q4km-ctx8k:latest

judge_model

qwythos-9b-v2-q4km-ctx8k:latest

task_count

16

repetitions

2

candidate_count

4

equal_accounted_calls

true

architecture_call_counts

A_llm_only

224

B_hard_verifier

224

C_pacc_runtime

224

rows_per_architecture

32

overall

A_llm_only

hard_adherence

0.9875

derived_coherence

0.9812

intent_persistence

0.9875

supersession_alignment

0.8094

repair_success

0.775

usefulness

0.9528

semantic_novelty

0.8084

valid_novelty

0.8475

literal_check_mean

0.9792

within_task_pattern_entropy_mean

1.0

B_hard_verifier

hard_adherence

0.9859

derived_coherence

0.9797

intent_persistence

0.9875

supersession_alignment

0.875

repair_success

0.7984

usefulness

0.9503

semantic_novelty

0.7725

valid_novelty

0.8606

literal_check_mean

0.9792

within_task_pattern_entropy_mean

1.0

C_pacc_runtime

hard_adherence

1.0

derived_coherence

0.9906

intent_persistence

0.9853

supersession_alignment

0.9062

repair_success

0.8688

usefulness

0.9516

semantic_novelty

0.7897

valid_novelty

0.8153

literal_check_mean

0.9792

within_task_pattern_entropy_mean

1.0

deltas

C-B

hard_adherence

0.0141

derived_coherence

0.0109

intent_persistence

-0.0022

supersession_alignment

0.0312

repair_success

0.0703

usefulness

0.0012

semantic_novelty

0.0172

valid_novelty

-0.0453

literal_check_mean

0.0

within_task_pattern_entropy_mean

0.0

C-A

hard_adherence

0.0125

derived_coherence

0.0094

intent_persistence

-0.0022

supersession_alignment

0.0969

repair_success

0.0938

usefulness

-0.0012

semantic_novelty

-0.0188

valid_novelty

-0.0322

literal_check_mean

0.0

within_task_pattern_entropy_mean

0.0

B-A

hard_adherence

-0.0016

derived_coherence

-0.0016

intent_persistence

0.0

supersession_alignment

0.0656

repair_success

0.0234

usefulness

-0.0025

semantic_novelty

-0.0359

valid_novelty

0.0131

literal_check_mean

0.0

within_task_pattern_entropy_mean

0.0

selection_agreement

A=B

0.5625

A=C

0.40625

B=C

0.46875

all_same

0.3125

usage

input_tokens

350535

latency_ms_sum

3633534.6304999674

output_tokens

108991

wall_seconds

3370.0

smoke_run

6 tasks × 2 × 3 candidates executed first, 128 calls, all outputs parsed; kept in the bundle

Interpretation

interpretation

Against the v0.2 predeclared interpretation: the predicted coherence and intent-persistence gains over B are not observed; raw novelty did not decrease (semantic novelty +0.017 vs B); the predicted breadth collapse cannot be tested at this repetition count. What did appear — governance gains on supersession and repair with a valid-novelty cost concentrated in multi_constraint and repair tasks — is mechanism-consistent but small, untested statistically, and runs opposite to the synthetic v0.1 valid-novelty picture (+0.196 there). One model, one run, one same-model judge: a first real data point, not a verdict on the architecture.

Limitations

limitations

• One open-weight 9B model at 4-bit, one run, 32 rows per architecture; no significance or equivalence test — deltas are descriptive.
• Judge = the same 9B model; no human rating, no second judge; pattern labels are noisy, so entropy is fragile.
• Thinking disabled for every call (see docs/REAL_LOCAL_RUN_EVIDENCE_BOUNDARY.md); a thinking-enabled run is a different experiment.
• Says nothing about frontier models.
• Not replicated: the second run with four repetitions (EXP-2026-0024, 64 rows per condition) shows supersession +0.0005 and repair −0.0125 vs B — the run-1 gains were run-to-run variation.
• The thinking-enabled run exists now: EXP-2026-0025 (2026-09-11).

Reproduction

reproduction_instructions

Extract the bundle; python -m pytest -q; replay exactly with CachedReplayProvider('.pacc_real_cache_local', reasoning_effort='none'); or rerun scripts/run_real_local_ollama.py against any OpenAI-compatible endpoint serving the same model tag.

Results

Recorded fields

completed_at

2026-09-11

Relations

History and provenance

Canonical URL

https://evemisslab.com/ai/experiments/EXP-2026-0023/

Machine-readable

/ai/experiments/EXP-2026-0023/index.json

Snapshot

AI-SNAPSHOT-v0.1-fe85b9694a45

Provenance

source

EveMissLab research collection: Adaptive Epistemic Systems (真本體論13)

extracted_by

Splice (Claude Code), reading the canonical UTF-8 sources and each lab's own result reports

extracted_at

2026-09-11

generator

tools/extract_aes/extract.py

claim_boundary

status, evidence level and result type follow the source artifact's own stated claim boundary; nothing is upgraded beyond what the report supports