I ran into an excellent presentation done by Ernest Lopez & Matt Linton of NASA on the VA/PT debate. As I delve into this rather innocuous sounding issue, let me tell you that it is not. As a responsible security team we have absolute nightmares when this term is used interchangeably and we have to assume on behalf of the customer as to what it could be!
My educated hunch on why this happens:
1. Lack of awareness of what constitutes a security assessment, just that one is required!
2. PT or penetration testing as a term is less technical than a VA – Vulnerability assessment!
3. The industry accepted notation of them is “VA/PT” which suggests that they are interchangeable terms!
4. Security teams for the fear of losing business do not dare to differentiate between the two!
5. Also most security teams have similar methodologies for either of them!
Having had the context now let’s look at what Mr. Lopez and Linton have to say on this.
“The difference between theory in practice is small in theory, but great in practice.”
Wikipedia defines: A penetration test is a method of evaluating the security of a computer system or network by simulating an attack from a malicious source, known as a Black Hat Hacker, or Cracker.
Wikipedia defines: A vulnerability assessment is the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system
One epoch-making statement they make is (if I concoct some of their observations) pen testing gets you what vulnerability assessment misses out…..though it is not as dramatic as that, it is in a way essential that a PT be carried out not in substitution of a VA but nevertheless.
And then the absolute zapper of what a picture says that would have required a 1000 words…
There are some absolute crackers where they have defined typical Pen Testing offering to rules of engagement through very well-known quotes:
Network Vulnerability Testing - “The only rules that really matter are these: what a man can do and what a man can't do.” – Jack Sparrow
Web vulnerability Testing - “Just cause you got the monkey off your back doesn't mean the circus has left town.” – George Carlin
Social engineering/Phishing Tests - "Foolproof systems don't take into account the ingenuity of fools." — Gene Brown.”
Authored by Srimant Acharya