🧭 THIS WEEK AT BuildProven
Howdy. Last issue landed on a Saturday. I've moved BuildProven to my evenings, so this one should be on time.
As usual, share it, vote in the poll at the end, or best of all, hit reply and tell me what you need help with on AI. I answer every one.
🧰 Worth Your Click
Two from this week's reading.
Introducing AlphaProof Code. DeepMind had agents write machine-checked proofs for Signal's cryptography code, with human experts reviewing the specs. Same idea as A3 below, done at a scale I won't be trying at home.
Slop Is Moving Up the Stack. A benchmark study: agents now clear more of the checks, and the mess has moved into design and error handling. Exactly the stuff a reviewer skims past.
🗺️ FEATURED INSIGHT
Treat the AI as a supplier
In October an AI agent wrote a change to my own content system, PR 667. Every one of its own tests passed. A review then caught it telling readers an issue "went out" before I'd sent it. A second review, by a different model, found two more defects after it had merged. Under the rule below, I reject it.
25 years in automotive programs taught me one thing about suppliers. You don't accept a part because the supplier says it's good. You check it against your own spec, and somebody with a name signs for it.
So I now treat AI-written code as a supplier delivery. The team that merges it is the customer, and owns acceptance.
The standard
It uses ASPICE-style words so a quality lead can map it to what they already run, but it doesn't claim conformance to anything. A pass means the evidence exists. Whether the change is right is still on you.
AI CHANGE ACCEPTANCE
A1 Requirement before work
Written and dated before the first commit:
When (trigger), the (system) shall (response)
A2 Test named for each requirement
It must fail on the code before the change
A3 Independent check before merge
A person, or a different model family
Same model, new session = Partly met
A check after merge doesn't count
A4 Traceable both ways
Requirement, commits, tests, review, merge
A5 A named person releases it
An auto-merge rule is not a release
Score each: Met / Partly met / Not met
Accept: all five Met
Deviation: one Partly met, risk written
down, release owner signs
Reject: any Not met, or a finding openHow PR 667 scored
A1, Met. The ticket had five "shall" lines, written before the first commit.
A2, Met. The new tests failed on the old code, then passed.
A3, Partly met. Before merge, only a reviewer from the same model family saw it. It caught the false "went out" line, which every test passed because the tests encoded the same wrong rule. The different-family review came about 7 hours after merge, so it doesn't count. It found two more defects, one against a written requirement, still open when I checked.
A4, Met. Ticket ID in the PR and commits, a test-run record per commit, one merged commit.
A5, Not met. No human reviewed it. The agent merged it under my own standing auto-merge rule the moment the gate and CI passed.
That one stung, because I wrote the auto-merge rule. A green pipeline is evidence for someone to decide on. Nobody did.
Decision: Reject. A5 Not met, A3 Partly met, and a finding still open.
Try it this week
Pick one AI-written change your team merged recently.
Paste the standard above into your team's wiki or the change template.
Score A1 to A5 from what's already recorded.
Write down the verdict and the one criterion that failed.
Next change, record the evidence before merge, not after.
I've only run it on code. I reckon A1, A3 and A5 carry straight over to an AI-drafted test plan or FMEA row. If you try that, tell me how it went.
Proven or not?
The claim: if the AI's tests pass and CI is green, the change is good to ship.
The evidence: PR 667 passed all its own tests and still made a false claim to readers. After merge, a different-family review found two more defects, one against a written requirement.
Verdict: NOT PROVEN. Green tells you the supplier's own checks ran. It says nothing about whether the checks were right, or whether anyone signed.
The standard itself is one change in, on my own repo, so I'm not calling it proven either. It did catch what the pipeline let through.
Take this
A requirements to test traceability matrix, in one file. Open trace-matrix.html
A4 as a page. Paste your requirements and your test list, and it shows which requirements nothing tests and which tests trace to nothing. It runs in your browser, uploads nothing, and exports a CSV for Excel.
Check it: save the file, open it, click Load sample, and confirm it shows two uncovered requirements, one orphan test and one unknown requirement ID.
Want a second pair of eyes on your first scoring? Reply with the word "supplier" and we'll find 15 minutes for a free call. Or just reply with the program job you'd most like AI to take off your plate.
Know someone who'd use this? Share your link: {{rp_refer_url}}. One sign-up gets you my content engine setup prompt; three get you a free Proven or Not? Review call.
Brett
