Original ten tasks · raw model
5/10
4/6 DAX and 1/4 report contracts, straight from the model.
Inspect the baselineWebGPU lab · Experiment 01
A 4B language model, running in your browser.
Qwen3.5 4B on WebGPU, entirely client-side. Inspect every recorded answer, the audited repairs and a fresh transfer test, or load your own copy of the model files and run it live.
- Model
- Qwen3.5 4B, text-compatible copy, Q4_K_M
- Files
- 7 local GGUF shards · 2.72 GB · SHA-256 verified
- Runtime
- wllama 3.8.1 on WebGPU
- Decoding
- 4096-token context · ≤768 output tokens · greedy
- Uploads
- None. Model files never leave your browser.
Fixed trial setup
Results at a glance
What passed, and what that means.
Same ten tasks · after repairs
10/10
Not general accuracy. The repair rules were informed by these same ten tasks.
Inspect the repairsFresh, harder fourteen · frozen pipeline
3/14
Strict score on a transfer test frozen before its first run. Manual review found one more valid DAX expression.
See the protocolThese are response-contract checks. They test response structure and the requested DAX/report contracts. DAX was not executed; numerical correctness, native Power BI rendering and export were not validated. Passing them is not proof of native Power BI correctness.
The hosted rerun
Raw answers. Audited repairs.
Recorded hosted inference · no live inference is running here
Loading the verified hosted rerun…
Pipeline improvement, with the raw model score retained. The same ten initial prompts and strict validators were used. One ambiguous DAX declaration received one request-grounded model retry; conservative rules corrected the other four failed outputs. These repairs were informed by this benchmark. This is not held-out model accuracy, executed DAX correctness, or native Power BI rendering evidence.
Inspect all ten rerun answers, original outputs and repair audits
The original baseline
Ten tasks. Every answer.
Recorded on Intel Iris Xe · not live inference
14 October 2026 · Chrome · wllama 3.8.1 · Text-compatible COPY of Qwen3.5 4B Q4_K_M · 34/34 GPU layers
What these checks mean. They test response structure and requested DAX/report contracts. DAX was not executed; numerical correctness, native report rendering, and Power BI export were not validated. This isolated ten-task, single-device trial excludes the app’s production grounding and repair pipeline.
Loading the recorded responses…
Optional · Bring your own files
Run it on your GPU.
The website serves the interface and runtime. Your browser reads the model files you explicitly choose; it does not upload them or use a cloud inference service.
Raw model and repaired pipeline are scored separately. Each task starts with the unchanged benchmark prompt. Conservative rules can correct a measure declaration, remove SUM around an existing measure, relocate empty report metadata, or omit unused empty fields. An ambiguous bracketed DAX declaration gets at most one model retry with request-grounded formatting feedback. Every original answer, retry and transformation is retained for inspection; these repairs were informed by the ten benchmark cases.
Compatible model files required
Choose all seven prepared GGUF shards (2.72 GB total). This site does not host model weights. The original Ollama model file needs compatibility transformations before this browser runtime can load it.
Model preparation & source instructions
Chrome with hardware WebGPU, enough GPU memory, and HTTPS required. File names, sizes and every SHA-256 are verified before loading. Performance depends on your hardware.
- 1Check WebGPU
A hardware compute shader must pass.
- 2Choose the model files
Select all seven shards together.
No local files selected.
- 3Verify & load
Hashes are checked before anything loads onto the GPU.
Your GPU has not been checked. Viewing recorded results needs no GPU.
Live inference has not started.
Fixed trial settings: 4096 context tokens · up to 768 output tokens · Q8 KV cache · greedy sampling · 99 requested GPU layers. Static hosting without cross-origin isolation uses one CPU helper thread; the recorded trial used four. Actual layers and threads appear after loading.
Local runtime diagnostics
No runtime loaded.
What we learned
Feasible. Measurable.
Still an experiment.
The model completed all ten tasks entirely on a browser GPU. The contract failures show the next work clearly: tighter report schemas, validated DAX execution, and stronger hardware coverage.