An assessor who cannot say “we were wrong about this one” has no standing to say “you are wrong about that one”. We have been wrong. Here is what that looked like, and what we did about it.
Seven determinations, overturned by ourselves
On a first Report on Compliance for a payments company, before issuing, we ran the report through our own pre-issue quality review. The review turned seven of our earlier determinations into “not in place”. For three of them the note to the client said plainly that our earlier determination had been made too readily, and that correcting it was our responsibility, not theirs.
That was not a comfortable email to send to a client who had worked hard for months and was already unhappy with the pace. It was the right one. The quality review exists so that the signed report is right. It does not exist to make the earlier emails right. A firm that skipped the review, or softened its result to avoid the conversation, would be signing a report it knew to be wrong, and the whole point of the signature is that it is not.
The quality review exists so that the signed report is right. It does not exist to make the earlier emails right.
“Two corrections from us”
A scoping exchange with a cloud start-up opened with two corrections to our own earlier notes. We had misread one of their subnet classifications and overstated one of their flows. The exchange then went on to the point that mattered: a cache instance whose security group accepted traffic from the whole production network had to be classed as part of the cardholder environment unless that group was narrowed, and the client’s attempt to lower some subnets to “connected” just to clear warnings was not going to work.
Opening with our own corrections cost us nothing and made the rest of the letter easier to receive. A client who has just watched you correct yourself has less reason to suspect you are padding the findings.
The vendor was right about the constraint
Under the PIN security standard, we recommended that a payment-infrastructure vendor make its hardware security module’s host-command authorisation temporary. The vendor rejected it: its terminal management system has to generate and export keys on demand, so the authorisation cannot be switched off between uses. Our own preparation notes for the next meeting say it in so many words: they are right about that, it should be conceded without reservation, and that is what will give weight to the rest. The notes also record two further points on which we had been wrong, and one finding of ours that was badly written.
We conceded all of it at the start of the meeting. Then we came to the point that still stood: permanent authorisation combined with exportable keys is the dangerous combination, and a control you cannot remove can still be bounded, with a minimal command list, locked key attributes and logged exports. The vendor accepted that, and we believe it accepted it because the concessions came first.
What people get wrong
They think an assessor’s authority comes from never backing down. It comes from the opposite: from a client being able to see that every determination was tested, including against the assessor’s own earlier view.
They do not challenge. Clients often accept a finding they believe is wrong because arguing with the assessor feels risky. A good assessor wants the challenge. A finding that survives it is stronger; a finding that does not should not be in the report.
They challenge the wrong way. “The last auditor accepted it” is not an argument. “Here is the constraint, here is why your recommendation cannot work in our architecture, and here is what we can do instead” is one, and it usually wins.