Independent testing in ISASecure CSA: what the lab must test itself

"Independent testing" is a phrase in the ISASecure CSA scheme that suppliers read two ways, and both are wrong. One camp hears "the lab will test every requirement on my product" and budgets for a long stay on the bench. The other hears "the lab will read my test reports" and ships a unit that nobody can log into. The scheme does something narrower than either, and it is written down requirement by requirement.

This article explains what the independent-test flag is, how many requirements carry it, what the laboratory does with the rest, why your interface count matters more than most suppliers expect, and what to have ready.

Myth 1: the lab tests every functional requirement

It does not. The CSA evaluation workbook, CSA-311, lists the 166 functional requirements drawn from IEC 62443-4-2 and marks, for each one, whether the requirement has to be validated by an independent test. On the CSA requirement set, 34 rows carry that mark.

For those 34, the laboratory exercises the requirement on the product itself. It configures the unit, drives the behaviour in question, observes the outcome and records it. Your own test report is useful context, but the result rests on the lab's execution, not on your report.

For every other requirement, the evaluation is evidence review. The assessor examines what you provide, typically test results, design documentation and, where it helps, a live demonstration, and records a result for the row. It is still a real evaluation with a real outcome, but the hands are yours and the judgement is the assessor's.

The flag is applied row by row across that whole set; it is not confined to one family. Which of the 34 apply to your product follows the usual workbook rules: component type and target security level decide which rows are in scope, and the flag comes with the row.

Myth 2: independent testing means the lab re-runs your lifecycle testing

This is the more expensive misunderstanding, because it points suppliers at the wrong preparation.

"Independent" in the flag means one thing: for that requirement, the test is executed by the laboratory, on the product, rather than accepted from the supplier. It does not mean the lab fuzzes your protocols, floods your interfaces or attempts to break in. Those belong to your secure development lifecycle under IEC 62443-4-1, and the lab's role there is to review what you did and how far it reached. The testing article walks through where each of those lives.

So the laboratory's hands-on work in a CSA evaluation comes down to two things: the 34 flagged functional requirements, and the vulnerability identification scan of every accessible network interface. Both are laboratory activities performed under ISO/IEC 17025 accreditation, with the method control and records that implies. Evidence review of the remaining requirements is assessment work, and the certification decision is governed by ISO/IEC 17065. As an ISASecure certification body, Perseus keeps those three roles distinct on every CSA evaluation: laboratory testing, assessor review, and a separate decision.

Myth 3: the evidence-review rows are the easy part

Suppliers sometimes treat the non-flagged requirements as paperwork. They are not. The decision rule for a CSA certificate is that no applicable functional requirement may be left not met, and a single requirement in that state, whether tested by the lab or reviewed from your evidence, means no certificate. Evidence review is the majority of the functional stream by count and where most preparation effort goes.

What the assessor needs for a non-flagged requirement is evidence that it is met on the version under evaluation: test results that name the version and the interface they were run against, design documentation that shows how the behaviour is implemented, and the ability to demonstrate on request if the documents leave a question open. Evidence that is generic, undated or tied to a different version tends to produce a follow-up request rather than a result.

Interface count drives effort

A subset of the functional requirements is evaluated once per accessible network interface rather than once for the product, and the results are rolled up worst-case: the requirement is met for the component only if it is met on every interface where it applies. Several of the common component security constraints are assessed this way. The certificate still states a single capability security level for the whole component; there is no per-interface level.

Two practical consequences follow. A flagged requirement that is evaluated per interface is tested by the lab on each interface, so bench time scales with your interface list. And the vulnerability scan is also run per accessible interface, so the same list drives that effort too. The evaluated interfaces are recorded in the assessment report, not on the certificate, which makes the accessible-interface list you supply one of the most consequential documents in the evaluation.

A concrete example: the SL 3 step

The SL 2 to SL 3 step makes the pattern visible. It adds 23 requirements, and nine of them carry the independent-test flag:

Three things to read from that list. The flagged set is not confined to one foundational requirement. It mixes common requirements with type-specific ones, so the hands-on increment at SL 3 is larger for a network device, which sees seven of the nine rows, than for a software application, which sees three; an embedded or host device sees four. And CR 2.7, the only base requirement added at SL 3, is one of the flagged rows, so the SL 3 step brings hands-on testing for every component type, not just network devices.

The full list of 34 lives in the workbook, row by row. Before you finalise the test unit and the interface list, ask for the flagged rows that apply to your component type at your target level.

Who decides, and why it is not the tester

One more independence rule sits at the end of the process. The certification decision is made by someone who was not one of the stream leads: not the lead for the development-artifact review, not the lead for the functional assessment, and not the engineer who ran the vulnerability scan. That separation is a requirement on certification bodies under ISO/IEC 17065 and a condition of issuing the certificate, not a preference. The assessor who tested your product and the person who signs the certificate are never the same individual.

What to prepare

• A test-ready unit at the exact version under evaluation, in a configuration the lab can exercise, with the installation, administration and hardening guidance needed to operate it.
• Access to every accessible interface: physical connections, network reachability and any adapters or cabling the lab will need. If an interface is only reachable in a particular operating mode, say so.
• Credentials and accounts at every role the flagged requirements will need, administrative and low-privilege alike, plus any certificates or keys the product expects.
• The accessible-interface list, complete and consistent with the unit you ship. It sets the per-interface scope for the flagged tests and the scan.
• The evidence package for the non-flagged rows: test results and design documentation tied to the version under evaluation and organised by requirement identifier, plus a named contact who can demonstrate on request.

Prepared that way, the lab's hands-on testing is a known quantity: a defined set of requirements on a defined set of interfaces, with the outcome resting on what the product does rather than what the paperwork says. The series index has the rest of the CSA articles, and the overview article covers how the three streams fit together.