PCI DSS asks for penetration testing in specific terms, and most of the testing reports we are handed do not meet them. Not because the testers were bad, but because nobody told them what the report would be used for.
PCI DSS asks for penetration testing in specific terms, and most of the testing reports we are handed do not meet them.
What requirement 11.4 asks
A defined methodology, based on industry-accepted approaches, covering the whole perimeter of the cardholder environment and the critical systems inside it, at both the network layer and the application layer, with the application tests at least covering the vulnerability classes the standard lists.
External penetration testing at least once every twelve months and after any significant change. Internal penetration testing at least once every twelve months and after any significant change. Exploitable vulnerabilities and security weaknesses found are corrected, and the test is repeated to verify the corrections. For service providers, the methodology has to include segmentation testing every six months.
The tester has to be qualified and organisationally independent. Internal teams can do it if they are not the ones who manage the systems.
How it differs from the scan
The quarterly external scan by an approved scanning vendor, under requirement 11.3.2, is a tool run against your Internet-facing addresses, with a passing report. The internal quarterly scan, under 11.3.1, is a tool run inside. Both find known vulnerabilities by signature.
A penetration test is a person trying to get in, chaining what the scanner found with what the scanner cannot see: a trust relationship, a default credential in a supplier’s manual, a peered network with a default allow rule, a service listening that the inventory says should be off. On a recent assessment, the two findings that mattered most, peered untrusted networks reaching the cardholder environment on every port and an unapproved service on a worker node, were exactly what a network-layer internal test is designed to find, and no scan had flagged either.
How we scope and deliver it
Our QSAs scope the test against the requirements your evidence depends on: the perimeter as the scope document defines it, the critical systems inside, the segmentation controls, the applications that touch card data. Where we do not perform the testing ourselves, it is executed by specialist testing partners we have worked with, under our scope and our review. The assessor who will rely on the report is the one who reads it first, which means the report is written for the person who has to defend it.
How an assessor reads the report
In this order. Is the scope stated, and does it match the scope document? Is the methodology named? Are the dates inside the assessment period? Is the tester independent? Are the source networks named? Are the exploitable findings listed with severity, and is there a re-test showing each one closed? Is the application layer covered, with the vulnerability classes the standard names?
A report that answers all of those is evidence. A report that answers half of them is a conversation about whether the test needs to be done again.
What people get wrong
They treat the scan as the test. They test from the outside and skip the inside, where the segmentation lives. They test before the environment is finished and never after the changes. They fix the findings and forget the re-test, which is the part the requirement actually asks you to evidence.