# QA Audit — Session Handoff (resume after /clear)

## ⚡ RESUME HERE FIRST (2026-06-12, session restart mid-fan-out)
**Mode: FULLY AUTONOMOUS fan-out** (Pam: "go autonomous… fan them out, complete all non-image
questions"). Mega-batch pipeline: fetch 300 → split 10×30 (audit_s1..10.json) → 10 parallel agents
(each WRITES _work_barmodels/spec_out_N.json directly, returns only a summary) → build → deploy.

**STATE: mega-batch 2 (ids 7975–8985) is AUDITED but NOT DEPLOYED.**
All 10 spec files sit ready in `_work_barmodels/spec_out_1..10.json` (297 ok / 3 flagged:
8085 ambiguous distractor "7400+23"≡key; 8216 & 8223 inconsistent stems — totals don't divide,
stored solutions silently used different totals 240/1000). NO key flips needed in this batch.

**Next actions, in order (all scripts exist):**
1. `php artisan tinker _work_barmodels/mega_build.php`  (globs spec_out_*.json → chunk_sols/
   chunk_hints/chunk_flags.json + renders any q<id>_d.png diagrams)
2. scp chunk_sols.json chunk_hints.json chunk_flags.json → prod /tmp/; tar+scp any NEW q*_d.png
   to prod storage/app/public/questions/solutions/
3. ssh: `php artisan tinker --execute="require '/tmp/deploy_chunk2.php';"` then chown storage +
   cache:clear. (deploy_chunk2.php already on prod /tmp — reads the 3 JSONs.)
4. Fetch mega-batch 3: sed the id cursor in `_work_barmodels/fetch_mega2.php` (7974→8985) →
   scp+run on prod → pull /tmp/chunk.json → split 10×30 → `rm -f spec_out_*.json` → fan out 10
   agents with the standard prompt (see cont 8/9 below; agents write spec_out_N.json directly).
5. Repeat until the WITH-solutions path is exhausted (~220 left after mega2), then start the
   **coverage build**: 5,663 public no-image questions with NO solution (same pipeline; author
   from scratch; INSERT solutions as status_id=3, source='ai', human_checked_at=now, user_id=2 —
   mirror deploy_htmlsols.php insert pattern but INSERT not UPDATE since no row exists).
**Deploy rules:** never background the scp/ssh compound commands (they die silently — run
foreground); no concurrent SSH (fail2ban); verify "solutions updated:" output every time.
**Defect policy:** verified key/slot fixes applied directly + needs_revision + [audit] note;
unfixable → flag only. All conventions: docs/solution-authoring/SOLUTION-STYLE-SPEC.md.

**Goal:** comprehensive QA audit + remediation of AllGifted Math questions on PROD
(mathapi.allgifted.com). You can `/clear` and resume — memory files + this doc hold everything.

## Prod access (read [[feedback_ssh_via_ip]])
`ssh root@152.42.223.228` (IP + HostKeyAlias, never hostname). App `/var/www/html/mathapi`,
DB `api`. Query/edit via `php artisan tinker /tmp/<script>.php`.
**NEVER run concurrent SSH** (me + an agent, or two agents) — it trips fail2ban and blocks
this IP on port 22. Agents do PURE REASONING (no SSH); the parent applies all DB writes
serially. `chown -R www-data:www-data storage` after any artisan-as-root that writes files.

## Flag mechanism
Wrong/uncertain question → `qa_status='needs_revision'` + append reason to `qa_notes`.
Fix only where the intended answer is clear; otherwise flag.

## What's DONE this session
- **Bar-model solutions P3–P6 complete** (public): models + (P6) algebra. FIB input-type
  cleanup done (6,978 → numeric). 28 FIB→MCQ. Currency/word/unit fixes.
- **Answer audits → 286 flagged `needs_revision`:** 133 deterministic (place-value/rounding/
  in-words/arithmetic across all 26,344), 150 reasoning (all 739 *unreviewed* public no-image),
  3 from a 150-q *approved* sample (approved set ~2% error rate — clean).
- **Systemic finding:** a broken AI-generated batch (~ids 10246–10510, percentage/average/ratio)
  shipped public with **placeholder answer "1"** — all flagged; investigate at source.
- **Comprehensive 4-dimension audit pipeline built + proven** (see below). Public no-image
  questions WITH solutions audited+remediated up to **id 3221** (batch 476–2282 + 2283–3221).

## RESUME HERE — comprehensive no-image audit (the active task)
Per the user: bar model by default for solutions; **formula/guess-&-check where a model is
harder**; **algebra only for convoluted P6** ([[feedback_solution_model_first]]). Hints
**ASCII-only** ([[feedback_hints_ascii_only]]).

**Dimensions per question:** (1) question_text sense/typos/secretly-needs-image,
(2) answer (solve & verify), (3) hints (correct/ASCII/non-spoiling),
(4) solution — **READ the `[[model:…png]]` image with vision** to verify the bar model is
relevant + correct; build a model where one is warranted and missing.

**Pipeline (per chunk) — refresh files every time, verify each step finished before the next:**
1. Fetch: `questions` where status_id=3, (question_image null/''), `question NOT LIKE '%base64%'
   AND NOT LIKE '%<img%'`, has a status-3 solution, **id > LAST_DONE**, limit 60. Dump
   `{id,question,answers[4],correct_index,solution,hints[]}` to /tmp/chunk.json; `tar` the
   `[[model:path]]` PNGs to /tmp/chunk_pngs.tgz.
2. scp chunk.json + tgz back; extract PNGs into `storage/app/public/`.
3. Split chunk.json into batch files of 30 (`cb1.json`, `cb2.json`). **Confirm they hold the
   NEW ids before spawning** (the staleness bug on 2026-06-10 re-audited an old chunk).
4. Spawn ONE Sonnet agent per batch — prompt = the 4-dimension comprehensive checker; it
   Reads `_work_barmodels/cbN.json` and each model PNG at
   `C:\allgifted\mathapi11v2\storage\app\public\questions\solutions\q<id>.png`. Returns JSON
   per question: `{id,qtext:{ok,issue},answer:{ok,stored,correct,issue},solution:{status,model_image,issue},hints:{status,issue}}`.
5. Collect → remediate (fix answers/hints/qtext; rebuild/author models per method policy;
   flag the rest needs_revision) → deploy in ONE script. Report touched PUBLIC qids (tables).
6. Next chunk: LAST_DONE = last id of this chunk.

**After the WITH-solutions path:** the ~5,400 public no-image **WITHOUT** solutions are the
coverage-build (author solutions+models / formula where simpler) — the big remaining effort.
**Defer image questions** (question_image set, or base64-inline) — figures are being drawn by
another agent; audit those only after that finishes.

## Open items to carry over
- q3195: hint 4 states the $650 difference, not Mark's final **$975** — fix.
- q2285/2346/2425/3183: bar-model brace font renders `0` rounded (reads like `o`) — values are
  CORRECT; optional: give the generator a clearer-zero font + regen those PNGs.
- q478: MCQ has only 2 options (flagged) — needs distractors.
- 328 draft answer-key solutions ("Answer key …") — deferred, all status≠3 (not served).
- Figure-dependent P6 left for the image agent: 5654, 10594, 10597, 10601, 10603, 6650–6654.
- Question-text errors awaiting human: (earlier) 8216, 8223, 8376, 8383, 9594, 7561, 4038.

## Tooling / scratch in `_work_barmodels/`
- `lib.php` — bar-model generator (seg/brace/nameLbl + bm_rasterize; per-PID Chrome profile).
- `audit_answers.php` — deterministic verifier (place-value/rounding/in-words/arithmetic).
- Data: `audit_candidates.json` (6,509 public no-image non-deterministic w/ answer, pri 0=unreviewed/1=approved),
  `all_questions.json` (full dump, may be stale — re-dump if needed), `audit_found.json`/`audit_flags.json` (applied flags).
- Backups (rollback): `fib_backup.json`, `qa_latex_backup.json`, `hint_ascii_backup.json`,
  `mcq74_backup.json`, `p6_backup*.json`. Safe to keep until review complete.
- Dev DB = localhost `api` (XAMPP, ~6,589 q, OLDER snapshot). Holds correct versions of some
  *older* questions but NOT the newer (8000+) ones; useful only for old-question dev→prod copy.

## PROGRESS UPDATE 2026-06-11
Comprehensive no-image audit (WITH-solutions path) done through **id 5455** (476-2282, 2283-3221, 3222-4356, 4357-5455). **RESUME at id > 5455.** Recurring issue types: non-ASCII in hints, qtext typos, boilerplate solutions (flag), noun mismatches, occasional ungradeable FIB (answer in answers[1]).

## PROGRESS 2026-06-11 (cont 8) — AUTONOMOUS MODE; chunks 7110-7193 + 7194-7378
**Pam: "go autonomous."** Agents now WRITE chunk_vo.json/chunk_ro.json directly (cheaper loop).
Chunk 7110-7193 (fraction products, mixed×whole, decimal rounding, %-of, reverse price):
60/60 verified, 60 solutions + 180 hints deployed, 0 flags.
Chunk 7194-7378 (ratios: equivalent/simplest/missing-term/word, unit conversions, volume m³/cm³,
tank fill, patterns 2-blank FIB): 57 verified + flips 7215 (->5:1:2 idx1; keyed option wasn't
fully simplified) and 7247 (->6 idx3) applied; 7195 flagged (computed $1071; options hold "1271"
typo). Cuboid diagrams for 7288-7304. **RESUME at id > 7378.**

## PROGRESS 2026-06-11 (cont 7) — CONTINUOUS MODE + chunk 7031-7109 done
**Pam: "No need to pause to check with me. Just keep going."** — continuous mode is standing
policy (memory: feedback_audit_continuous_mode): verified key/data fixes applied directly (always
+ needs_revision flag + [audit] note), broken = flag only, report tables as chunks land.
Applied: 6958 key 0->1 (1/5). Chunk 7031-7109 (60 q: 8 round-estimate, 23 fraction conversions/
division-sentences, 21 fraction +/- incl mixed-number regrouping, 10 fraction ×): 58 solutions +
174 hints deployed (rule-boxes, no diagrams); 7087 key 0->1 (3 1/3) applied + flagged; 7099
flagged (stem corrupted: 15/64 × 15/64 vs options from original 5/8 × 3/4). Mixed-number answers
stored as LaTeX — written inline \(\frac{}{}\) in Final Answer lines (display $$ would split).
**RESUME at id > 7109.**

## PROGRESS 2026-06-11 (cont 6) — APP DEPLOYED + chunk 6919-7030 done
**Flutter app rebuilt & deployed** to quiz.allgifted.com (/var/www/html/quiz/build/web; backup
web.bak-20260611). Built from HEAD (4367f72) + ONLY the HTML-dialect renderer changes in
math_text_utils.dart — the held WIP (diagnostic exit-flush etc., 5 files) was stash-protected and
restored to the working tree after the build (still uncommitted). Students now see HTML solutions.
**Chunk 6919-7030 audited+authored**: 55 solutions deployed (24 advanced volume: wall-thickness,
%-dimension-change, GCF packing, painted-cube, melted-reshape, leak-rate w/ cuboid+fill diagrams;
31 rounding/estimation w/ rule-box, no triangle), 165 hints, 12 cuboid PNGs. 5 flagged broken:
6919 (cube+pi hole, no option correct, dup options), 6922/6923 (capacity inconsistent, no option),
6958 (key wrong: correct 1/5 = index 1, geometric fit ignored), 6959 (0.36 matches no option).
3 self-contradictory old solutions replaced (6996, 7030, 6942 wrong final lines). chunk_build.php
now handles cuboid|balance|compare diagrams generically. **RESUME at id > 7030.**

## PROGRESS 2026-06-11 (cont 5) — AVERAGE FAMILY PROPAGATED bank-wide
6813 pattern (TAN triangle in box + balance/compare bar models + totals working) propagated to ALL
53 remaining public average questions w/ solutions: 4668-4699 (P5 two-unknown comparisons) +
10202-10251 (changed-average/balance, comparison, weighted-combine, find-last). 23 new bar-model
PNGs (balance_svg/compare_svg in avg_build.php — reusable spec-driven generators). 13 data repairs
(missing FIB answers 10205/10206/10236; wrong slot 10227/10231/10232/10233; wrong value
10210/10211/10212/10230; stray distractors 10209; 10234 MCQ->FIB) — all repaired AND flagged
needs_revision for review. Triangles now sit INSIDE the formula box, side-by-side with the formula
(ags_dst_triangle_2.png / ags_tan_triangle_2.png, near-equilateral). NOTE pitfalls: PHP array_merge
REINDEXES numeric ids (use + operator); tinker closure scope breaks `global` (define locals).

## PROGRESS 2026-06-11 (cont 4) — CANONICAL FORMAT LOCKED: AGS HTML dialect (supersedes cont 3)
**Pam-approved format** = HTML body + PNG diagrams (template q6643 v5) — full spec in
**docs/solution-authoring/SOLUTION-STYLE-SPEC.md §2** (formula box crimson #960000 + "AGS Formula
Box" footer, <img> diagram q<id>_d.png, ags-working 3-col table, inline \(..\) fractions only,
1.5 spacing, KaTeX font-inherit CSS in both admin blades). FE renderer extended in flutter_demo
(math_text_utils.dart, UNCOMMITTED) — **app rebuild+deploy pending**; until then students see raw
tags on converted solutions. FANNED OUT: 72 questions converted (6624-6660 volume, 6745-6753 +
6757-6808 speed/avg-speed via shared ags_dst_triangle.png, 6810-6861 avg/money/reverse-%,
6871-6903 ratio w/ bar-model PNGs, 6911/6914/6916 m^3 volume) + 162 formula-first ASCII hints +
30 diagram PNGs. Tooling: build_html.php (specs->HTML), render_diagrams.php, spec_*.json.
Chunk 6757-6918 audit DONE (key flips 6911/6914/6916 applied; 6907,6909,6910,6915,6917,6918
flagged needs_revision). **RESUME comprehensive audit at id > 6918.**

## PROGRESS 2026-06-11 (cont 3) — combined Raleway solution figures
NEW solution format for formula/measurement Qs — see **docs/solution-authoring/SOLUTION-STYLE-SPEC.md**
(canonical). One combined transparent **Raleway PNG** per Q via `_work_barmodels/solution_fig.php`
(+`cuboid.php`): **bold formula in a highlighted box** (cream/lime) → labelled cuboid (steel, unknown
"?" crimson, length+width on base plane) → Raleway working w/ real fractions → crimson Final Answer.
NO KaTeX (font inconsistent on app). Body = just `[[model:.../q<id>_3.png]]`. Applied to 14 volume +
5 speed in 6608-6753. Generators: cuboid.php (transparent raster), solution_fig.php (composer).

## PROGRESS 2026-06-11 (cont 2) — through id 6753 + NEW POLICY
**Policy change (user):** don't just flag — UPDATE solution+hints first (easy/ASCII, bar-model
preferred, NO algebra below P6), provide the answer I believe right, THEN needs_revision. Do NOT
auto-flip answer keys — surface key changes to Pam in a TABLE for approval.
- Backfilled 6518-6607: 8 wrong solutions rewritten + 6542 distractor typo fixed (already flagged).
- Chunk 6608-6753 (60 q): 50 solutions + 150 hints redeployed, 37 needs_revision. **RESUME id > 6753.**
- AWAITING PAM REVIEW (tabled): key-flips 6643(->25),6651(->500),6658(->15),6660(->10),6702(sqrt1600->40);
  broken/no-matching-option 6639,6640,6642,6650,6652,6653,6654,6659,6661,6662;
  out-of-syllabus sqrt/cbrt 6691-6720; near-dups 6704(=6691),6751(=6748).

## PROGRESS 2026-06-11 (cont) — through id 6517
Added chunks 5456-6517. **RESUME at id > 6517.** Found a heavily-broken machine-generated region ~6365-6517: number-series + "count the letters" questions with boilerplate or WRONG-WORD solutions (e.g. 6505 solves KANGAROO not BEETLE) and missing hints — answers mostly correct, solutions flagged needs_revision for rewrite. Also q5466 ungradeable FIB (answer in answers[1]) fixed.