Pilot plan — version 0.1

Separate states: technical verification; diagnostic assessment; professional review; fitness for the use the mandate names. Each state is earned separately in audit and in investigation.

Preparation

Before any access to real data, establish authorisations, a suitable environment, data minimisation, roles and retention. The reviewer prepares the expected reference separately: inventory, verifiable facts, errors and decoys, expected anomalies, alternative explanations and adjudication criteria. The reviewer keeps the reference apart from the analyst's package. Freeze frameworks, prompts, sources and the first output. Record the extraction/OCR tools and versions used in the run.

Minimum cases to cover — a coverage list

1. Benign documentary duplicates, each paid once.
2. Superseded versions and documents that look marginal on their own and become relevant once linked.
3. Gaps and illegible material: the correct result declares limited coverage and leaves the gap as a gap.
4. A controlled irregularity, with corroboration and at least one innocent explanation to verify.
5. A personal relationship or a coincidence of names, which the correct result records as a fact, leaving conflict and responsibility to further evidence.
6. An investigative case with a plausible adverse reconstruction refuted by evidence favourable to the defence.
7. An open investigative case: the correct result keeps the alternatives open and closes with them listed.

Measurement

For each expected finding: found, partial, missed, justified abstention. For additional findings: correct, false positive, unresolved. Precision and recall are computed on adjudicated cases, with explicit denominators and exclusions. Errors are separated by type: acquisition/OCR, citation, linkage, diagnosis, confidence, legal interpretation. Premature exclusions, dependence between sources and contrary elements left out of the analysis are assessed as well.

These metrics are populated by adjudicated pilot cases. The first diagnostic assessments calibrate the protocol. After corrections, use cases held out from the protocol's design. A version promotion requires a reasoned decision on adjudicated cases and calibrated scores.

Professional criminal-law review

The competent reviewer checks: the relevant legal system and time frame; roles and mandates; the rules applicable to documentation, statements and acquisitions; secrecy and confidentiality; personal and judicial data; conflicts; AI vendors and tools; archiving; procedural guarantees and admissibility. The list is open. Resolved and open questions are recorded under the reviewer's name and authority.