Penetration test vs vulnerability scan
Both find weaknesses, which is why they get confused. The difference is whether anyone proves the weakness is real, and SOC 2 asks for each in a different place.
A vulnerability scan and a penetration test both look for weaknesses, and vendors blur the line further by calling some scanners pentests. Here’s how the standard definitions separate them, where DAST fits, and what SOC 2 needs from each.
The definitions
NIST’s testing guide, SP 800-115, defines a pentest by what the tester does1:
Penetration testing is security testing in which assessors mimic real-world attacks to identify methods for circumventing the security features of an application, system, or network.
It adds that most pentests look for “combinations of vulnerabilities on one or more systems that can be used to gain more access than could be achieved through a single vulnerability.”1
A vulnerability scan is the automated part: a tool compares what it sees against known issues and reports matches. NIST’s caution is that “vulnerability scanners can have a high false positive error rate (i.e., reporting vulnerabilities when none exist).”1
DAST is a kind of scan aimed at a running web app. OWASP describes web application scanners as tools that “scan web applications, normally from the outside, to look for security vulnerabilities,” and says this category “is frequently referred to as Dynamic Application Security Testing (DAST) Tools.”2
Side by side
| Vulnerability scan | DAST | Penetration test | |
|---|---|---|---|
| Looks at | Hosts, ports, software versions | A running web app | Whatever is in scope: apps, APIs, networks |
| Done by | A tool, on a schedule | A tool, often in the release pipeline | A tester, using tools |
| Proves impact | No | Sometimes | Yes |
| False positives | Common1 | Common1 | Fewer, because a person checks each finding8 |
| Typical price | Monthly subscription | Monthly subscription | From about $5,000 a test5 |
| SOC 2 fit | CC7.1 scans | CC7.1, and testing changes under CC8.1 | CC4.1 evaluations |
What SOC 2 asks from each
The criteria mention both, in different places.
- Scans belong to CC7.1. Its point of focus describes infrastructure and software scans “on a periodic basis and after significant changes are made to the environment,” with problems fixed “in a timely manner.”3 That means a schedule, not a one-off.
- Pentests belong to CC4.1, whose point of focus lists “vulnerability scans, security assessment, penetration testing, and third-party assessments” among the evaluations management can use.3
- DAST can also count as testing changes under CC8.1, whose point of focus names “static and Dynamic application source code” testing among its examples.3
Vanta sums up how the two fit: “The vulnerability scan identifies your systems’ vulnerabilities and the pen test tells you how exploitable they are.”5 The pentest firm Software Secured frames the bar the same way: “auditors and enterprise buyers expect evidence of vulnerability management.”4 And if your auditor doesn’t require a pentest, Secureframe says they’ll still typically want you to run scans, rank the risks and fix the highest ones on a regular basis.6
Where manual testing still wins
Alex Lauerman, who runs the pentest firm TrustFoundry, is blunt: “Completely automated analysis can only find 10-20% of vulnerabilities in most cases.” And: “If someone is just running a scan, this does not constitute a penetration test.”7 He sells manual testing, but the point holds for anything that needs to understand your app: one customer reading another customer’s data, a checkout that accepts a negative price, a permission check that exists only in the UI. Vanta says a pentest “should also test your application’s business logic.”5 Tools don’t know your business rules, so they rarely catch these.
Where an automated external pentest sits
Between the two sits a newer category: tools that automate the outside part of a tester’s job. They map the attack surface, then prove each weakness instead of guessing from a version number. NIST calls this step vulnerability validation, where “The objective is to prove that a vulnerability exists.”1
That’s what Pentrail does for one domain. It doesn’t report that a Redis version might be vulnerable; it checks whether the port answers without a password. It doesn’t warn that .git could be exposed; it requests it and puts the response in the report. Then it stops, with no exploitation beyond proving the weakness, no password guessing and no denial of service.
What it can’t do is what Lauerman is talking about: testing behind a login, business logic, internal networks, social engineering and source code review. If your auditor or a customer needs those, a manual tester still has to do them.
See your own grade
Scan your domain and see what an attacker sees.
A setup that covers both
- Run scans on a schedule and after major changes, and keep the results (CC7.1).
- Before your audit window, run an external test that proves what an outsider can reach, fix what it finds and retest.
- Add a manual application pentest when an auditor, a contract or a customer asks for one, or when you ship features with complex permissions.
- Keep each report, ticket and retest together for the auditor.
A scan tells you what might be wrong. A pentest proves what an attacker can actually do. SOC 2 describes regular scans under CC7.1 and lists penetration testing under CC4.1, so plan for both.
Questions
Is DAST the same as penetration testing?
Does SOC 2 require vulnerability scanning?
How often should I run vulnerability scans for SOC 2?
Can an automated pentest replace a manual one?
Sources
- NIST, SP 800-115: Technical Guide to Information Security Testing and Assessment:
Although the process itself is highly automated, vulnerability scanners can have a high false positive error rate (i.e., reporting vulnerabilities when none exist). … application vulnerability scanners typically have high false positive rates … The objective is to prove that a vulnerability exists, and to demonstrate the security exposures that occur when it is exploited. … Penetration testing is security testing in which assessors mimic real-world attacks to identify methods for circumventing the security features of an application, system, or network. … Most penetration tests involve looking for combinations of vulnerabilities on one or more systems that can be used to gain more access than could be achieved through a single vulnerability.
- OWASP Foundation, Vulnerability Scanning Tools:
Web Application Vulnerability Scanners are automated tools that scan web applications, normally from the outside, to look for security vulnerabilities such as Cross-site scripting, SQL Injection, Command Injection, Path Traversal and insecure server configuration. This category of tools is frequently referred to as Dynamic Application Security Testing (DAST) Tools.
- AICPA 2017 Trust Services Criteria with revised points of focus (2022), as transcribed in the open-source CISO Assistant library, SOC 2 2017 TSC (rev. 2022), points of focus CC4.1, CC7.1 and CC8.1:
Depending on the entity’s objectives, such risk and control evaluations may include first- and second-line monitoring and control testing, internal audit assessments, compliance assessments, resilience assessments, vulnerability scans, security assessment, penetration testing, and third-party assessments. … Conducts Vulnerability Scans — The entity conducts infrastructure and software vulnerability scans designed to identify potential vulnerabilities or misconfigurations on a periodic basis and after significant changes are made to the environment. Action is taken to remediate identified deficiencies in a timely manner to support the achievement of the entity’s objectives. … Tests System Changes — A process is in place to test internally developed and acquired system changes prior to implementation into the production environment. Examples of testing may include unit, integration, regression, static and Dynamic application source code, quality assurance, or automated testing (whether point in time or continuous).
- Software Secured (penetration testing firm), SOC 2 Penetration Testing Services:
SOC 2 Penetration testing isn’t explicitly mandated, but auditors and enterprise buyers expect evidence of vulnerability management.
- Vanta, Penetration testing 101:
A penetration test should also test your application’s business logic and ensure there are no bypasses that could lead to serious issues. … Typically these tests start at $5,000 and can go up to $15,000 depending on the scope and the testing company you choose. … The vulnerability scan identifies your systems’ vulnerabilities and the pen test tells you how exploitable they are.
- Secureframe, Penetration Testing 101: A Guide to Testing Types, Processes, and Costs:
In cases where auditors don't require you to have a third-party pen test completed, they will still typically require you to run vulnerability scans, rank risks resulting from these scans, and take steps to mitigate the highest risks on a regular basis.
- Alex Lauerman (TrustFoundry), interviewed by Secureframe, Frequently Asked Questions about Pen Testing:
Completely automated analysis can only find 10-20% of vulnerabilities in most cases. If someone is just running a scan, this does not constitute a penetration test and will be not effective in identifying all potential vulnerabilities.
- Synack (penetration testing firm), How Much Does a Pentest Cost? 2026 Trends and Insights:
Manual testing by senior researchers costs more per day than automated scanning, but it also produces fewer false positives and more reliable findings.