Evaluation - Bring AI Home

Defined in qvac-service/evaluation.ts and qvac-service/evaluation-cli.ts.

Cases

All four are the same fictional chat with six labelled regions: name, email, address, phone, account, reference.

Case ID	Image	Condition
`chat-clear`	`public/test-files/redact/evaluation/chat-clear.png`	Clean
`chat-rotated`	`…/chat-rotated.png`	Rotated 90° clockwise
`chat-low-contrast`	`…/chat-low-contrast.png`	Low contrast
`chat-partially-obscured`	`…/chat-partially-obscured.png`	Partly covered

Run one case with REDACT_EVALUATION_CASE=<case ID> npm run qvac:evaluate.

Boxes are compared in percent of page width and height.

Pair each expected region with every proposal of the same label.
Keep pairs with intersection-over-union (IoU) ≥ 0.45.
Take pairs from highest IoU down; each expected region and each proposal is used once.

A case passes when all three hold:

The run passes when every case passes. The command exits non-zero otherwise.

JSON on stdout:

Field	Content
`mode`	Run mode
`identifiers`	SDK version (`@qvac/sdk@0.22.0`) and model IDs
`passed`	`true` when every case passed
`modelLoadMs`	Load time per model
`timings[]`	Per case: OCR and reasoning inference time
`memory`	Peak RSS and heap
`results[]`	Per case: `caseId`, `proposals`, `result` (matches, counts, precision, recall, F1, thresholds), `elapsedMs`

@qvac/sdk 0.22.0, Laya, Apple Silicon, two runs with identical output, at redact-11-copy-check.

Case	F1	Precision	Recall
`chat-clear`	0.92	0.86	1.00
`chat-rotated`	0.92	0.86	1.00
`chat-low-contrast`	0.92	0.86	1.00
`chat-partially-obscured`	0.83	0.83	0.83