The VA/PT Conundrum!

I ran into an excellent presentation done by Ernest Lopez & Matt Linton of NASA on the VA/PT debate. As I delve into this rather innocuous sounding issue, let me tell you that it is not. As a responsible security team we have absolute nightmares when this term is used interchangeably and we have to assume on behalf of the customer as to what it could be!

My educated hunch on why this happens:

 1. Lack of awareness of what constitutes a security assessment, just that one is required!

 2. PT or penetration testing as a term is less technical than a VA – Vulnerability assessment!

 3. The industry accepted notation of them is “VA/PT” which suggests that they are interchangeable terms!

 4. Security teams for the fear of losing business do not dare to differentiate between the two!

 5. Also most security teams have similar methodologies for either of them!

Having had the context now let’s look at what Mr. Lopez and Linton have to say on this.

“The difference between theory in practice is small in theory, but great in practice.”

Wikipedia defines: A penetration test is a method of evaluating the security of a computer system or network by simulating an attack from a malicious source, known as a Black Hat Hacker, or Cracker.

Wikipedia defines: A vulnerability assessment is the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system

One epoch-making statement they make is (if I concoct some of their observations) pen testing gets you what vulnerability assessment misses out…..though it is not as dramatic as that, it is in a way essential that a PT be carried out not in substitution of a VA but nevertheless.

And then the absolute zapper of what a picture says that would have required a 1000 words…

There are some absolute crackers where they have defined typical Pen Testing offering to rules of engagement through very well-known quotes:

Network Vulnerability Testing - “The only rules that really matter are these: what a man can do and what a man can't do.”  – Jack Sparrow

Web vulnerability Testing - “Just cause you got the monkey off your back doesn't mean the circus has left town.”  – George Carlin

Social engineering/Phishing Tests - "Foolproof systems don't take into account the ingenuity of fools."    — Gene Brown.”

Authored by Srimant Acharya

Rate this article: 
Average: 1 (1 vote)
Article category: 

There are 4 Comments

Very True Srimant. !!  Day in Day out interaction with clients/external teams sometimes makes us re-look at the definition of VA & PT and try to relate it to their actual business requirement ….This could be because they have less time to get this executed or less funding. or they dont understand whats actually needed.... but nevertheless the first discussion with most of the clients on this topic are always confusing & definitely interesting

Penetration Testing is the technique which allows the Tester / hired attacker / ethical hacker to provide the more comprehensive and detail view of security loopholes associated with the likelihood of vulnerabilities getting exploited as well.PT is the phase that starts after the VA has been completed and PT solely depends on the output of VA.
PT has got the scope of exploiting a vulnerability (if agreed by the customer), demonstarte the extent of explotation of vulnerability  .
Where as VA is exlsusive to vulnerability findings , risk rating , validation of those findings only.
Irrespective of Black BOX or Gray BOX approach on VA, a security tester do not have knowledge of various security controls implemented on the application as well as the infrastructure.
PT phase do possess the capability of exploiting the found weakness/vulnerability not only limited to validation of the vulnerbilities getting exploited but simultaneosuly validate the security controls as well that whether these controls are capable enough to provide security to the infra /application with the presence of vulnerabilities  on them.

As per my view PT gives more efficient approach to assess security of the scope defined as PT is always the super set of VA. 
 

From Customer point of view , out of my past experience i have conducted exclusive VA and Full Fledged VA/PT as well.
Mostly when customer scope is in production Env. , they do not allow Penetration Testing for their is always a risk of services getting disrupted on procution environment. Rather they have opted for safe approach with VA only.
To be very specific PT cannot exist without a VA.
But from business point of view , we have categorised PT as well to give a clear picture on what methodology we are going to adopt , such as :

  • PT with passive mode or safe checks: Where we proceed to the next phase of VA , do not exactly discrupt a service , or gain a remote shell or change the configuration but demonstrate or provide artifact on the same could be possible.
  • PT with Active approach : These are mostly conducted on testing environment , where we do exploit the vulnerabilities in real showing the customer ; "irrespective of the security controls applied, an attacker can damage the application / infrastructure to its maximum possible extent".

I also agree with Sameer. Though having a difference in approach, these terms seems confusing. A matured approach for complete Threat and Vulnerability Management should consists of both, VA and PT.