I'm a sysops/devops engineer at a hosting company and wanted a local model for daily work (Terraform, Cloudflare, AWS, log digging, small scripts, mail and ticket triage). Generic benchmarks didn't tell me much about my work, so I wrote my own: 28 one-shot tasks with deterministic grading. Over 6 days I ran it against my local setup, a handful of cloud models and a dozen config variants. Sharing in case it helps people with a similar box. A lot of the usual advice turned out not to apply to a 2-bit quant.

TL;DR

• IQ2_XS on Strata scores on par with full Qwen3.8 Flash from the API: 47–50/56 locally vs 48–49/56 on OpenRouter/Alibaba, with about half the wall-clock time (median latency 12 s vs 22 s).
• Greedy (Strata's default) is clearly the best for this quant. Qwen's recommended sampling (temp 1.0, top_p 0.95, top_k 20) dropped me from 50 to 37/56. temp 0.6 gave 43, and temp 1.0 + min_p 0.05 gave 40.
• reasoning_effort: high is worse than medium (42 vs 50/56) and twice as slow. On IQ2 the model loops in its thinking. A 16k reasoning budget didn't help either.
• ./setup.sh --calibrate on Strata 0.1.43 gave +25% decode (median 118 → 143–147 tok/s). The gain came from --pool-workers 8 and the adaptive expert tier, not from --pcie-frac.
• Run-to-run noise is about ±3 points out of 56, even with greedy. Check which tasks flipped before concluding anything.

Hardware / software

• RTX 5070 Ti 16 GB (PCIe 5.0 x16), driver 580 → 615.71
• Ryzen 9 9950X3D (16C/32T, AVX-512)
• 64 GB DDR5-4800, EXPO off on purpose (see below)
• ASRock X870 Pro RS WiFi, BIOS 4.10
• Fedora 44, kernel 7.2, Strata in Docker (built from the repo's Dockerfile)
• Model: Qwen3.8-Flash-Next GSQ-RCO IQ2_XS (ISTA-DASLab), Strata v0.1.38 (Oct 9–10) → v0.1.43 (Oct 13+)
• Engine config: 131k context, KV int8, --kv-resident 32768, MTP --spec 4, reasoning_budget_tokens: 8192, conversation cache 4 GiB, vision encoder on CPU
• Also tested locally: Qwen3.8 27B GSQ-RCO IQ3_S on llama.cpp (fits fully in VRAM)

Why the RAM runs at 4800 MT/s. This is my third 9950X3D: the first two died on this ASRock board and went back under RMA. If you missed the saga, the ASRock AM5 + Ryzen 9000X3D failures had several suspected causes:

• ASRock blamed its own PBO/power settings in older BIOS versions.
• AMD pointed at memory compatibility on older BIOS.
• Failures kept being reported after the fix BIOS releases.

Nobody has established a single cause, so I'm not taking chances: EXPO off, stock memory speed, current BIOS. That costs bandwidth. With an MoE model where the CPU computes the experts that don't fit in VRAM, memory bandwidth matters, so the numbers below are probably a few to ~20% lower than the same GPU/CPU with DDR5-6000 EXPO would give. I haven't measured it and won't on this board (propably.. 😉 ). If you run a 5070 Ti + 9950X3D with EXPO on, I'd like to see your numbers.

The benchmark

28 tasks, each run twice (56 answers per run). One-shot: no tools, no retries, no agent loop. Every answer is graded by code, with no LLM judge. Code from the answer runs in network-less containers. All data is synthetic (documentation IP ranges, fake account IDs). The mail tasks are in Polish which is worth knowing if you want to compare.

category	tasks	how it's graded
hcl	S3 module, count → for_each with moved blocks, fix a buggy module	terraform init/validate/test with mock_provider (aws 6.46), parse of moved pairs
cloudflare	DNS records in provider v5, WAF expression	terraform test (cloudflare 5.27), a Rules-language evaluator over 12 requests
aws	least-privilege IAM policy, jq over describe-instances, CloudTrail event analysis	IAM policy simulator (17 requests), exact jq -r output, JSON fields
logs	ALB 5xx breakdown, nginx incident (brute force + scanner), top-5xx one-liner, journal with OOM kills	exact values, command output in a sandbox
python	fix a duration parser, CIDR allowlist (IPv4/IPv6), p95 per endpoint	hidden unittest suites
sysops	backup rotation in bash, systemd unit + timer, nginx rate limit behind Cloudflare	shellcheck + run on fixtures, systemd-analyze verify, nginx -t + real 429 test
mail	thread with a moved deadline, triage of 8 / 23 / 443 mails (~40k tokens), invoice extraction, anonymization (Polish name inflection, secrets in URLs), brief for an external model, prompt injection, complaint reply	JSON fields, regex leak checks, placeholder consistency, word and deadline counts

A task passes only when every check passes; the score gives partial credit. The grader has a self-test that grades reference answers (28/28).

Model comparison (Oct 9, same tasks, 2 repeats)

model	where	pass /56	median latency	cost of one full run
GPT-6.1-sol high	Codex subscription	55	17 s	subscription
DeepSeek V4.1 Flash	OpenRouter → DeepInfra	51	30 s	$0.07
Qwen3.8 Flash-Next IQ2_XS	local, Strata	49	12 s	electricity
Qwen3.8 Flash (full)	OpenRouter → Alibaba, budget 8k	49	22 s	$0.09
GLM 5.3	OpenRouter → Novita	46	8 s	$0.13
Qwen3.8 27B bf16	OpenRouter → DeepInfra	44	66 s	$0.28
Qwen3.8 27B IQ3_S	local, llama.cpp	42	46 s	electricity
Kimi K3	OpenRouter → Moonshot	42	12 s	$0.62 (some 429s)
GLM 5.3 Flash	OpenRouter → Z.AI	41	6 s	$0.04

Notes:

• The 2-bit local quant matched the full-precision API model, so I stopped thinking about IQ3_XXS. On 64 GB it would cost RAM headroom and speed for no visible gain on my tasks.
• The 27B is roughly 3–4x slower here and lost overall. It only won at exact counting in logs.
• Flash's weak spots: nginx config details (limit_req inside if), subtle Python (CIDR/IPv6 edge cases), and exact log arithmetic.
• OpenRouter → Alibaba ignored reasoning_effort for Flash. Only reasoning.max_tokens had an effect.

Tuning the local setup, Oct 9–15

All runs below use the local IQ2_XS, reasoning_effort: medium unless stated, and the same 28 tasks × 2.

date	change	pass /56	median tok/s	total time
Oct 9	Strata 0.1.38, driver 580, greedy	49	108	18.5 min
Oct 10	driver 615	47	116	18.0 min
Oct 13	Strata 0.1.43	50	118	20.6 min
Oct 13	reasoning_effort: high (8k budget)	42	112	41.3 min
Oct 14	high + 16k reasoning budget	43	118	61.6 min
Oct 14	medium + sampling temp 1.0 / top_p 0.95 / top_k 20	37	119	20.4 min
Oct 14	high + the same sampling	42	95	58.9 min
Oct 14	medium + temp 0.6 (top_p/top_k as above)	43	122	18.1 min
Oct 14	medium + temp 1.0 + min_p 0.05	40	115	22.2 min
Oct 15	greedy + --calibrate results	44	147	16.0 min
Oct 15	same, control run	44	143	15.4 min

1. Sampling vs greedy on a 2-bit quant

Several threads here say Strata's greedy default causes doom loops and that you should switch to Qwen's recommended sampling. On my tasks that made things much worse. Medium: 50 → 37/56 (16 tasks lost, 3 gained, sign test p≈0.004).

Most new failures were single-token slips, not worse reasoning:

• A JSON string opened with a Polish „ and closed with ASCII ", which made the JSON invalid.
• Invalid \escape sequences in JSON.
• days instead of noncurrent_days in an S3 lifecycle rule.
• A misplaced paren in jq.

My guess: a 2-bit quant has a flatter, noisier token distribution, so every sampled token has a higher chance of being wrong. With medium reasoning the model has little room to notice and fix it. Lower temperature and min_p reduced the damage but never got back to greedy.

Sampling did fix the loops on high (11 looping answers → 0), but the score stayed at 42 and decode got slower (MTP drafts are accepted less often).

2. "High" reasoning on IQ2 = loops

On high, the model uses whatever budget it gets:

medium	high, 8k budget	high, 16k budget
median completion tokens	1.6k	8.3k
hit the budget wrap-up	3	29
answers with >30% duplicated sentences in reasoning	0	8

One answer repeated 847 of 903 reasoning paragraphs. Looping text is very predictable, so MTP hits 170–185 tok/s inside the loops, which makes a 16k run finish faster than you'd expect. Even on medium, a 16k budget gave one task back and lost another: the model hand-wrote 128-bit two's complement as 1111…0000… until Strata's repeat detector cut it off with an empty answer. With 8k, the budget wrap-up would have stopped it first. For this quant, the 8k budget works as a safety fuse.

3. Calibration: +25% speed, but read the score carefully

./setup.sh --calibrate (run inside the container, about 25 minutes) measured:

• --pcie-frac: 0 → 124.5, 0.2 → 130.8, 0.35 → 132.9, 0.55 → 131.5, 1.0 → 102 tok/s. It kept my old 0.35. 0.2, suggested in another thread, wasn't better here.
• --spec-min-p: 0.3 / 0.5 / 0.7 → 133.9 / 131.6 / 136.6. It kept 0.7.
• --pool-workers: 15 → 133.5, 10 → 138.0, 8 → 141.6, 4 → 118.2. That's +6%. On a 16-core X3D, fewer workers are faster.
• Adaptive expert tier: default 148.4 → --adapt-every 1 --adapt-swaps 160 --adapt-decay 0.97 159.1 (+7%).

--trim-stage-weights and --pipeline-windows, also suggested here recently, are multi-GPU-only according to the docs, so I skipped them.

On the bench, decode went from a median of 118 to 143–147 tok/s and the full run from 20.6 to 15.4–16.0 min. But the score dropped to 44/56, twice.

Before blaming the flags, I split the failures into two groups:

• hard: the code doesn't work. nginx -t fails, terraform validate fails, hidden tests fail.
• soft: a defensible interpretation or a strict exact-match check.

run	pass	soft	hard
Oct 9	49	1	6
Oct 10	47	2	7
Oct 13	50	2	4
Oct 15, calibrated	44	5	7
Oct 15, control	44	5	7

The hard failures are the same as before (the Oct 13 run with 4 was the lucky one). All of the drop is soft cases:

• An IAM condition of exactly incoming/ instead of incoming/*. The task said "list only these prefixes", so this is a stricter reading of "only".
• Two inbox triage tasks where the model flagged a disk alert as needing action even though a later mail in the same mailbox had closed it.

The calibration changes which experts run on GPU vs CPU and the CPU summation order. The numerical differences are tiny, but greedy decoding flips at near-ties. Those flips then replay identically across runs: the same IAM mistake and the same inbox list in both calibrated runs. So two 44s are closer to one dice roll repeated than to two independent samples. I kept the calibration.

4. Other things worth knowing

• Noise: even with greedy, the two repeats inside one run agree on only ~24 of 28 tasks (Strata's adaptive expert cache isn't bit-reproducible). Treat ±3/56 as noise and look at which tasks flipped, not just the total.
• Vision on CPU ("vision" section in the config, BF16 mmproj, 16 threads) doesn't touch VRAM, so text speed stays the same. A quick check after enabling it gave ~112 tok/s including prefill, in line with the bench. A 1089×787 infrastructure diagram took 17 s end to end.
• Real agent workload (Hermes agent, 54k-token context): about 100–104 tok/s decode, 78–83% expert-cache hit rate, 5–7% of routed experts over PCIe. That was before calibration. (waits for some free time to retest)
• "tool_call_recovery": true in the Strata config is worth turning on for agent use. Per the docs, 98.3% vs 92.9% of tool calls are parsed as intended.
• A Strata config without a sampling block decodes greedy. If your client sends no temperature, that's what you get. Check what your agent actually sends.

Caveats

• 2 repeats per config. One-shot tasks, not agentic. Synthetic data. Some prompts are in Polish.
• Built around my job, so heavy on Terraform/AWS/Cloudflare/mail, light on general coding.
• The cloud-model comparison is a single day. Providers and quants behind OpenRouter can change.
• The findings on sampling and high reasoning are about IQ2_XS. Q4 or IQ3 users may well see the opposite. I'd be curious if someone runs the same comparison on IQ3_S.

Feedback welcome

I'm open to suggestions on what else to test or what to fix in the methodology: other Strata flags, other quants (if they fit in 64 GB), task ideas, grading that's too strict or too lenient. The tasks are synthetic, so I'm happy to share them, along with the grading code, if anyone wants to reproduce this on their hardware or compare against a different setup. Just ask in the comments.

Disclosure: I prepared the benchmark, ran the analysis and wrote this post together with Claude (Anthropic). The runs, hardware and decisions are mine; Claude helped with the bench code, statistics and write-up.

EDITS: Formatting