ACSSA evaluation results: ten result types, and the one that fails

The first result table an asset owner sees from an ACSSA evaluation is a page of abbreviations, most of them starting with "NR", with no obvious way to tell good news from bad. ACSSA records exactly ten result types, defined in ACSSA-300 and tabulated in ACSSA-303's Table 5, and only one of them, Not met, is a failure. The other nine are ways of passing, some unconditional and some conditional on documentation the asset owner must have approved. This article walks through the ten, shows how many of the 422 evaluation methods permit each, and points to where they appear in the report.

Four parts, four vocabularies

ACSSA evaluates four IEC 62443 parts against one installed IACS: the asset owner's security program (62443-2-1, plus two ACSSA-defined policy-and-procedure-support items), its risk assessment (62443-3-2), its service providers (62443-2-4) and the technical capabilities configured and used in each zone (62443-3-3).

The security-program part shows seven result types across its two maturity levels but six at either one, since NR-TI at level 2 gives way to NR-TZ at level 3; the capability part six, service providers five, the risk assessment only two.

The ten in plain words

M, Met. The criteria were satisfied at the maturity level in question: level 2 means the policies and procedures are documented, level 3 that they are practised for this IACS.

NM, Not met. The criteria were not satisfied. This is the only failing result.

Four results say a requirement does not apply, each for a different reason.

NR-A, not required by the asset owner. Service-provider requirements only: the asset owner has not delegated the task in question to this provider.

NR-TI, not required because the technology is not used in the part of the IACS examined. A level-2 result for security-program requirements.

NR-TZ, not required because the technology is not used in the zone or conduit. The level-3 counterpart, available for capability requirements too.

NR-S, not required for the zone's target security level. The capability is defined at a level above the target the asset owner's risk assessment set for that zone, and nothing in that assessment calls for it; or, for a security-program requirement carried at several security levels because its associated capabilities span them, the method for a level above the target does not apply. Since a level-1 method can never sit above any target, NR-S is offered only at security level 2 and above.

Three results are what ACSSA-300 calls special circumstances. Each passes, but only with documentation approved by the asset owner and every affected stakeholder.

NR-R, not required with a documented risk justification. The asset owner has decided, in writing and with a risk argument, not to implement the requirement.

CSM, compensating security measure. Capability requirements only: the zone lacks the capability as specified, but a documented alternative covers the risk.

NF, not feasible or not allowed by law. The requirement cannot be implemented, or the law forbids it.

One result is a shortcut.

VIC, verified by independent certification. Service-provider requirements at level 2 only: the provider's maturity-level-3 IEC 62443-2-4 certificate, from a body accredited by an IAF MRA signatory, stands in for the document review.

Only Not met fails

A Not met at either maturity level is a nonconformity. ACSSA-303 records each one in a short list (Table 10) and a detailed record (Table 11) carrying maturity and security level, description, evidence, the zones, conduits and service providers affected, the normative and associated security-program requirements, date, evaluator and the asset owner's contact. Nonconformities are numbered NC-YYYY-NNN.

Results feed composite verdicts on the 89 asset-owner requirements (87 from IEC 62443-2-1, as ACSSA-303 states, plus the two support items), and under ACSSA-303 a Not met on the requirement's own process aspect or on any associated requirement means the composite does not pass. Certification requires every requirement to pass at level 3, composite or not, so one open nonconformity blocks the certificate. For an initial evaluation, ACSSA-300 gives the asset owner 30 days to have a correction plan accepted and 90 days to close the nonconformity.

Three results that need approved paperwork

NR-R, CSM and NF are neither soft failures nor accepted on a verbal explanation. They pass on one condition: documentation that the asset owner and all affected stakeholders have approved.

Under ACSSA-300 R17 that documentation must make a risk case, not state a preference: what the requirement would have protected against, how serious the exposure is, and why the asset owner can live with it. A compensating measure must also describe the substitute and the risk it leaves; a not-feasible result must cite the law or explain why implementation is impossible.

"All affected stakeholders" means those inside the asset owner's organisation, and is the phrase to plan around: a compensating measure in a zone that operations, engineering and security all touch is rarely one department's call. Section 5.5 of the report lists every accepted special circumstance with its zone and recorded result, so a reader can see which passes rest on documentation rather than on the control itself.

VIC: a certificate substitutes at level 2 only

The provider's certificate counts only when issued at maturity level 3 by a body accredited by an IAF MRA signatory, and it settles only the level-2 question: are the provider's processes documented? Whether they are executed for this IACS, the level-3 question, still needs evidence from this IACS. A level-2 provider certificate is not accepted at all. The workbook offers VIC on 122 of the 123 service-provider methods at level 2, and on none at level 3.

NR-S, the target level, and the zones you will not see

NR-S is the result most often misread as a grade. ACSSA awards no security level; the target for each zone and device-bearing conduit is the asset owner's input from its IEC 62443-3-2 risk assessment, and NR-S records that a capability or method above that target was not required there. The product schemes use security levels differently, as the SSA article on capability levels per zone explains.

Equally important is what NR-S is not used for. Level-3 sampling can leave some zones and conduits unexamined for a given requirement. Such entities receive no result: the report marks them as not examined and omits them from that requirement's result tables.

How many methods can produce each result

The workbook states, for every method at each maturity level it evaluates, which of the ten results may be recorded. Three workbook rows are grouping headers with no result of their own, so the totals below (308 at level 2, 419 at level 3) are derived from the workbook rather than stated by it.

Source: ISASecure ACSSA-311 v1.3; totals derived from the workbook.

Three things stand out. NF is rare by design: one security-program requirement (ORG 1.2) and two service-provider requirements (SP.01.04 and its enhancement), all three on personnel background screening. NR-R is broad but not universal: 14 security-program methods never offer it, among them eight of the nine requirements evaluated once for the whole IACS (the ninth, ORG 1.2, keeps NR-R). And CSM appears on every capability method and nowhere else.

Where the results appear in the report

Results appear in the ACSSA-303 report at three depths.

• Table 5 defines the ten result types; the annex (Table 35) repeats them.
• Table 12 is the composite view: one row per asset-owner requirement with its folded verdict, the table most readers need.
• Table 19 is the capability view: every IEC 62443-3-3 requirement against every examined zone and conduit, each paired with the security-program requirement it supports.
• Tables 10 and 11 list and detail the nonconformities. A passing certification report drops them and carries the three statements of conformity instead.

Two reading rules help. First, results at the two maturity levels are not independent: a requirement met at level 3 must also be met at level 2, and NR-R or NF must appear at both levels or at neither. ACSSA-300 lists ten such consistency conditions for checking the complete result set, but forbids using them to fill in a result that was never evaluated, except between the two maturity levels. Second, the risk-assessment part uses only Met and Not met, and three of its 33 statements are recommendations, so ACSSA records their results but does not score them.

As an ISASecure certification body, we would rather an asset owner read the column before the abbreviation and treat anything other than NM as a pass with a reason attached, not a gap. The rest of the ACSSA series is on the Insights index.