Fuzz, load, penetration and abuse-case testing: what the IEC 62443-4-1 SVV practice asks of a supplier

Of the eight practices in IEC 62443-4-1, the one that generates the most preparation effort before an ISASecure SDLA audit is the fifth: Security Verification and Validation Testing, SVV for short. It is the largest practice by row count, 20 rows to security management's 19, and also the one where the scheme is most specific about what it wants to see and least willing to accept a documented intention in place of a result.

A companion article in the CSA series covers who runs which test when a component is evaluated. This one looks at the other side: when your development process is audited, what must the SVV practice define, and what must it prove it has done?

Five requirements, 20 rows

IEC 62443-4-1 states five SVV requirements. ISASecure expands them into 20 assessable rows in the SDLA-312 workbook, and the expansion is uneven.

• SVV-1, security requirements testing (5 rows) covers testing what you specified: a plan, execution against it and recorded results, and two further rows: one for performance and scalability, one for boundary, stress and malformed input.
• SVV-2, threat-mitigation testing (2 rows) is abuse-case testing. The tests are meant to try, one by one, the threats your threat model claims to have mitigated, and to record what happened.
• SVV-3, vulnerability testing (11 rows) is where fuzz testing, network-load testing, attack-surface analysis, known-vulnerability testing, binary composition analysis and runtime-resource testing all live.
• SVV-4, penetration testing (1 row) requires the process to include penetration testing, keep its results and route the findings into defect handling.
• SVV-5, tester independence (1 row) requires tester independence, graded by the kind of test being run.

The test types, row by row

The table maps each test type the catalogue names to the rows that carry it. The two right-hand columns come from ISASecure SDLA-300: rows where execution artifacts are mandatory for a full evaluation, and minimum requirements, where evidence of inconsistent application is a major nonconformity rather than a minor one.

Three notes on the table.

The 3A rows are presented as a group. SVV-3A1 to SVV-3A5 together cover fuzz testing and network-load testing: a plan for each, the quality of the artifacts each produces and the results. We do not pin a test type to each row; the current workbook is the reference for row-level labels. For planning it does not matter: you need a plan, good artifacts and results for both kinds of test, and three of the five rows require proof of execution.

Appendix B exists, and it is short. The workbook's Appendix B attaches eight artifact-quality criteria to SVV-3A2 and SVV-3A4. They are the yardstick an assessor uses on your fuzz and load artifacts; their content belongs to the workbook, which ISCI publishes. On Perseus's reading of the activity columns, those two rows are examined in a product certification rather than in the process audit; the plan and results rows around them are examined in both. The practical point is that "we ran a fuzzer" is not the standard being applied.

Two rows are conditional on tooling. Binary composition analysis and dynamic runtime-resource testing are expected where tools exist for the platform in question. That judgement needs to be written down, because "not applicable" without a recorded reason reads as a gap.

Why SVV decides more than any other practice

SDLA-300 defines two overlays on the 102 rows, and SVV dominates both.

Twenty-one rows require execution artifacts for a full evaluation. Ten of them are SVV rows; the next-largest practice, security management, has four. This list matters beyond the audit itself: the two evaluation methods, full and readiness, differ only on these 21 rows, and a certificate with even one requirement passed by readiness runs for 12 months rather than 36. Nearly half of the rows that can shorten your certificate are testing rows.

Thirty-one rows are minimum requirements. Eight are SVV rows, second only to the security-guidelines practice. On a minimum requirement, evidence that the process is followed sometimes but not consistently is a major nonconformity.

Six rows appear on both lists: SVV-1A1, SVV-3A1, SVV-3A3, SVV-3C1, SVV-3C2 and SVV-3D. Across the whole scheme only ten rows have that double status, so testing accounts for more than half of SDLA's hardest core.

Two rows only the process audit ever sees

Each row in the workbook can carry two validation activities: one for the process audit and one for the per-product artifact review that CSA, ICSA and SSA perform. For two SVV rows there is no product-side activity at all: SVV-4, penetration testing, and SVV-5, tester independence.

On Perseus's reading of the SDLA-312 activity columns, a product certification checks that your fuzz, abuse-case and known-vulnerability testing actually ran for the product in front of the assessor. It never checks that you penetration-tested that product, and it never checks who was independent of whom. Those two questions are answered once, when the process is audited, and then relied on by every product certification that follows. On the same reading, the other 18 SVV rows also appear in the per-product artifact review of a CSA or ICSA evaluation.

If you are sequencing certifications, get the pen-test and independence provisions of your process right before the SDLA audit rather than hoping to fix them in a product certification. There is no product-side check for them.

Who runs the tests

Nobody at the certifier. SDLA is a review of documents and an audit of a process. For every SVV row that carries a process-side activity, the auditor reads what the process says will happen; on the ten SVV rows where execution artifacts are mandatory, a full evaluation then examines artifacts sampled from products developed under that process, or, under a readiness evaluation, the evidence that the organization is equipped to execute: trained people, tools in place, templates ready.

The one hands-on activity the catalogue names for a certifier is witnessing a vulnerability test that the supplier runs, carried on the SVV-3 carrier row. It is optional, and on Perseus's reading of the activity columns it belongs to the product-certification side rather than to the process audit. As an ISASecure certification body, Perseus assesses the SVV practice by reading the process and sampling its artifacts, not by testing anything.

The product side, including what the lab does test itself, is covered in the CSA article on fuzzing, load and penetration testing. The same fuzz and load artifacts are reviewed there for coverage per interface, and the process behind them is not re-audited because your SDLA certificate already answers that question.

Decide up front: the SM-5 connection

One row outside the SVV practice belongs in this article. SM-5, in security management, is about scoping the process for a given product: recording which lifecycle activities apply to it and which do not. It is the one management row that names fuzz testing and penetration testing explicitly, and it is itself one of the 21 mandatory-artifact rows.

So the decision on where fuzzing and penetration testing apply is not made inside the SVV practice at test time. It is made at the start, per product, and documented. An auditor can then trace a chain: the scoping record says these tests apply, the SVV plans cover them, the artifacts show they ran. A break anywhere in that chain is a finding against the process.

What to have ready

• A security test plan per product or product family, with execution records and results that can be tied to it.
• Abuse-case tests that reference the threat model, so an assessor can see that each mitigated threat was attempted.
• Fuzz and network-load plans and results, structured so that the artifacts can be judged against the Appendix B criteria rather than against a generic "tests passed" statement.
• An attack-surface analysis and a known-vulnerability test plan with pre-release results.
• A recorded decision on composition and runtime-resource tooling for each target platform, including the reason where no tool exists.
• Penetration-test records with issue routing, and a documented independence assignment for each test type. Both are checked here and nowhere else.
• The SM-5 scoping record that says, per product, which of these tests apply.

Prepared this way, the SVV practice stops being the place where a full evaluation quietly turns into a readiness evaluation. The rest of the SDLA Fundamentals series covers the two evaluation methods, the minimum requirements and the certificate lifecycle.