course/redact-qvac).
For the current questions and threshold, see Decision questions.
Add a question for a new kind of field
-
Open
qvac-service/reasoning.ts. -
Add an entry to
privateQuestions(asked at both levels) orconfidentialQuestions(Confidential only). Write it as a statement that is true or false about one line of text: - If the ID isn’t one of Redact’s labels, add the label too: follow How to add a sensitive-field label. Otherwise the suggestion is dropped by the evidence rules.
-
Run
npm test, then measure the change.
Reword a question that misfires
If Laya flags ordinary lines for a field, or misses obvious ones:- Make the statement concrete: “This text contains a street address” works better than “This is location information”.
- Ask one thing per question. Two fields in one statement blur the probability.
- Don’t turn the questions into one multiple-choice question with a “none” option. In testing, Laya then answered “none” for real names, emails and addresses.
Change how sure Laya must be
DECISION_THRESHOLD in reasoning.ts is 0.5.
- To catch more fields at the cost of more suggestions to reject, lower it (for example
0.4). - To show fewer, more certain suggestions, raise it (for example
0.7).
result.answers inside classify() and run one page through the app.
Measure the change
Stop the service, then run the labelled evaluation:f1, precision and recall with your previous run.
The gate requires all three at 0.70 or above.
See Evaluation.
Related
- About decision models
- Decisions (official)