L. van Westen — register

Ref 004 · writing ·

Evidence, not screenshots

Compliance evidence that is generated, timestamped, and diffable — because a screenshot of a settings page proves almost nothing.

Every audit season, somewhere, a competent engineer opens a settings page, presses print-screen, pastes the result into a document named Evidence_AC-3_final_v2.docx, and uploads it to a shared drive. Multiply by forty controls and three frameworks and you have a quarter of someone’s job.

The screenshot is the worst evidence format we routinely accept. It proves that a page looked a certain way, on someone’s monitor, at a moment nobody recorded, in an environment nobody can name. It can’t be diffed against last quarter. It goes stale the moment it’s taken, and everyone involved knows it — which is why the same screenshots get retaken every year, by hand, forever.

Evidence is a query with a timestamp

The alternative is boring and available today: treat evidence as an artifact that a scheduled job produces by asking the real system a real question.

  • A control says MFA is enforced. The evidence is the output of the query that lists users without MFA, dated, with the count on the first line. The healthy version of that file is empty.
  • A control says access is reviewed quarterly. The evidence is the diff between this quarter’s group membership export and the last one, plus the sign-off recorded next to it.
  • A control says backups are tested. The evidence is the restore job’s log, not a paragraph asserting that restores are tested.

Each of these is a few dozen lines of PowerShell or TypeScript against an API you already have credentials for. The output goes into a dated folder — or better, a repo, where the history of the control is the git log:

$report = Get-MgUser -All -Property UserPrincipalName, StrongAuthenticationMethods |
  Where-Object { -not $_.StrongAuthenticationMethods } |
  Select-Object UserPrincipalName

$stamp = Get-Date -Format 'yyyy-MM-dd'
$report | Export-Csv "evidence/ac-3/mfa-gaps-$stamp.csv" -NoTypeInformation

Nothing clever is happening here. That’s the point. The value isn’t in the script; it’s in the properties the artifact acquires by being generated: it has a date, it has a provenance, it can be regenerated, and two versions of it can be compared mechanically.

What changes with the auditor

Less than you’d fear, more than you’d expect. Auditors don’t actually want screenshots; they want to close findings without arguing. A CSV with a timestamp and the query that produced it closes faster than a screenshot, because the auditor’s own next question — “can you show me this is current?” — answers itself: run it again.

The real resistance is internal. Screenshot evidence feels like work you can see; a scheduled job that quietly writes a file feels like nothing happened. Keep a register of which controls are automated and when each last produced its artifact, and that feeling goes away. The register also tells you, honestly, which controls still run on print-screen. Mine did, for longer than I’d like to write down.

Where to start

Not with a platform. Pick the three controls whose evidence you retake most often, automate only those, and put the outputs somewhere with history. If after one quarter the diffs haven’t saved you an afternoon and one awkward audit conversation, stop. They will have.