Modelcraft LAB Open workspace

Fresh tasks · Frozen repairs

A fresh test. Same frozen pipeline.

Fourteen harder synthetic tasks: eight DAX measures and six report proposals. New schemas, combined filter requirements, multiple report tables, and distractor identifiers.

A transfer test, not another tuning round. Tasks and scoring rules are fixed before the first model run. The previously deployed repair code, model, greedy sampling, 4096-token context and 768-token output cap remain unchanged. Results count raw and repaired answers separately; failures will not trigger edits to the repairs or these tests.

First hosted run · strict scoring

3/14

Contract passes with the frozen pipeline. Manual review identified one additional valid DAX expression, which is not counted in the strict score. Contract checks are not proof of native Power BI correctness.

Protocol

Inspect the test before running.

Loading tasks

Checking frozen source fingerprints…

Inspect all task prompts and the scoring scope

These are constrained response-contract checks, not native DAX execution, numerical engine validation, report rendering, or export validation. Equivalent syntax outside a requested subset may need manual review. A truncated or timed-out answer fails completion and is retained.

Local inference

Run the full model in Chrome.

Local inference

Select the same seven compatible local GGUF shards. The model stays on your computer; the website serves the interface and runtime.

Same model and hardware checks.

Qwen3.5 4B text-compatible COPY · 2.72 GB · each shard verified by SHA-256. Positive native GPU offload and hardware compute are required.

Model preparation instructions

At most one existing request-grounded model retry per task. Each task has the same 180-second deadline, including any retry. A task timeout is recorded and the next task is attempted; Stop ends the suite.

  1. 1
    Check WebGPU

    A hardware compute shader must pass.

  2. 2
    Choose the model files

    Select all seven shards together.

    No files selected.

  3. 3
    Verify & load

    Hashes are checked before anything loads onto the GPU.

Hardware has not been checked.

No inference has started.

Local runtime diagnostics
No runtime loaded.