NIS2 Security Awareness ROI: What a 19,500-Employee Trial Says Your Training Budget Buys

The largest randomised controlled trial ever run on security awareness training followed more than 19,500 employees for eight months across ten simulated phishing campaigns. Users who received embedded anti-phishing training failed subsequent simulations at a rate 1.7 percentage points lower than users who received nothing at all. For employees who had recently completed the mandatory annual e-learning module, the study found no measurable reduction in clicking at all [6].

That is the honest starting point for any NIS2 training business case — and it is not an argument against training. Under NIS2 you have to train regardless of what the return looks like, because two separate provisions make it compulsory. What the evidence changes is the question. You are not deciding whether to fund awareness. You are deciding how much of the compliance floor to exceed, and where the money above that floor does the most good.

Two training obligations, and only one of them is a dial

Most guidance treats “NIS2 training” as one requirement. It is two, they sit in different articles, and they behave differently under audit.

Article 20(2) requires Member States to ensure that members of management bodies “are required to follow training” and — in the same sentence — to “encourage essential and important entities to offer similar training to their employees on a regular basis” [2]. Read the verbs. Management training is mandated; staff training, in Article 20(2), is only encouraged.

The binding staff obligation lives in Article 21(2)(g): “basic cyber hygiene practices and cybersecurity training”, one of ten minimum risk-management measures [3][4]. Article 21(1) attaches a proportionality test to all ten — measures must be “appropriate and proportionate”, taking into account “the degree of the entity’s exposure to risks, the entity’s size and the likelihood of occurrence of incidents and their severity, including their societal and economic impact” [4]. Germany’s transposition makes the split explicit: management training sits in § 38(3) BSIG, staff training in § 30(2) sentence 2 no. 7 BSIG, with separate BSI guidance for each [11][12].

Obligation	Legal basis	Who	Proportionality applies?	Budget behaviour
Management body training	Art. 20(2), first limb	Every member of the management body	No — the duty is unconditional	Fixed floor. There is no “how much” argument to have.
Staff cyber hygiene and training	Art. 21(2)(g), read with Art. 21(1)	All employees, and management as users	Yes — scaled to exposure, size and incident severity	A dial. You choose the level and defend it.
Role-specific security training	Art. 21(2)(g); CIR Annex 8.2	Roles needing “security relevant skill sets”	Yes	A dial, usually the most underfunded one.
Assessing whether any of it works	Art. 21(2)(f)	The entity	Yes	Its own obligation — see below.

That last row is the one almost every awareness vendor skips, and it is where the ROI conversation stops being optional. We come back to it after the evidence.

What the strongest available evidence says your programme buys

The study is Ho, Mirian, Luo and colleagues, “Understanding the Efficacy of Phishing Training in Practice”, presented at the 46th IEEE Symposium on Security and Privacy in 2025 [6]. Eight months at UC San Diego Health: ten phishing lures, more than 19,500 employees, randomised assignment and a genuine no-training control group. Both the simulation platform and the annual training content were commercial products — this is the deployment most in-scope entities are buying.

The results, in the authors’ own numbers:

• Annual mandated training showed no measurable protective effect. Time since a user last completed it had no significant association with failing a simulation (odds ratio 0.998 per 30-day increase, 95% CI 0.996–1.000, P = 0.06) [6].
• Embedded training worked, barely. Trained users were 9.5% less likely to fail in relative terms (OR 0.905, 95% CI 0.863–0.950, P < 0.001) — which the authors translate to a 1.7 percentage point absolute difference. For several campaigns, at least 10% of users in every group failed [6].
• Almost nobody engages with it. The majority spent under 30 seconds on the training page; fewer than 24% formally completed it; depending on the variant, between 37% and 51% of training sessions recorded zero seconds of engagement [6]. The university’s own summary puts it plainly: one third closed the page immediately [7].
• More of it is not better. Completing more sessions showed no benefit (OR 1.092, 95% CI 1.034–1.153), and neither did accumulating more time on training (OR 1.008, 95% CI 0.992–1.025). Users who completed multiple static training sessions were more likely to fail a later simulation (OR 1.185, 95% CI 1.110–1.266) [6].
• One thing did work. Among users who actually completed interactive training, the likelihood of clicking a later lure fell by around 19% in relative terms (OR 0.809, 95% CI 0.730–0.896) [6].

Two honest caveats, because this is one study. It ran at a single healthcare organisation, and the authors say so directly — the findings may not generalise to every sector [6]. And it measures one outcome: clicking a simulated link, not policy knowledge or reporting behaviour. What it does establish is that the claim underwriting most awareness ROI models — that training substantially reduces the probability an employee clicks — is far weaker than the market assumes. An earlier randomised study found embedded-training users failing at a higher rate than controls [6].

The KPI most entities report is the one the evidence cannot connect to outcomes

Article 21(2)(f) requires “policies and procedures to assess the effectiveness of cybersecurity risk-management measures” [4] — a separate minimum measure sitting alongside training rather than inside it. Whatever you spend on awareness, you owe a documented answer to whether it worked. For entities the Implementing Regulation binds, CIR (EU) 2024/2690 Annex point 8.1.3 puts it beyond doubt: the awareness raising programme “shall, where appropriate, be tested in terms of effectiveness”, and ENISA’s guidance under that point directs entities to the common KPIs at Annex point 7.2 [5][8].

Look at what those KPIs are. ENISA’s indicative list includes “the cost of implementation and maintenance, for example capital expenditure (CAPEX) / operational expenditure (OPEX)” and “the number of employees who have attended cybersecurity trainings” [8].

Attendance. The suggested training KPI is an attendance count — and attendance is precisely the variable the trial found unrelated to phishing outcomes [6]. An entity can report a green 98% completion rate, satisfy an indicative KPI, and have measured nothing about effectiveness. That is not a criticism of ENISA, whose list is explicitly indicative and whose guidance in the same breath tells you to weigh “the cost of their implementation” when choosing what to measure [8]. It is a warning about what happens when an indicative list becomes the whole programme.

One scope note. The CIR binds eleven categories of digital provider only — DNS providers, TLD registries, cloud and data centre services, CDNs, managed service and managed security service providers, online marketplaces, search engines, social networking platforms, and trust service providers [5]. If you are a hospital, a utility or a manufacturer, your obligations come from national transposition of Article 21, though supervisors increasingly read the CIR as interpretive guidance. Our complete Article 21 breakdown maps the ten measures in full.

Four things worth putting in front of a board instead of a click rate

None of these is validated by the trial — it measured clicking, not detection. They are practitioner measures, and they are defensible because each one tests something the click rate structurally cannot.

Measure	What it actually tests	Why it beats click rate
Report rate on simulated and real lures	Whether the organisation detects an attack in progress	ENISA’s guidance under CIR Annex 8.1.2 names “event reporting” among the cyber hygiene practices an awareness programme should cover [8]. Click rate has a floor; reporting has headroom.
Median time from delivery to first report	How fast your response clock starts	Feeds directly into incident-handling evidence under Art. 21(2)(b) rather than sitting in an HR spreadsheet.
Completion of interactive training, tracked separately from assignment	The one engagement variable with a measured effect	Completed interactive training reduced later failures by about 19% relative; static repetition was associated with worse outcomes [6].
Coverage of non-email vectors	Whether the programme matches how people are actually attacked	Verizon’s 2026 DBIR puts median successful click rates in mobile-centric vectors such as voice and text 40% higher than email; the human element appears in 62% of breaches [9].

The fourth line is where most programmes are quietly obsolete. Simulation platforms are email-shaped because email is easy to instrument. Pretexting — the concocted-scenario approach, frequently by voice — reached 6% of all breaches in the 2026 DBIR, while phishing held at 16% [9]. A programme that only ever tests the inbox is optimising a control surface attackers are steadily moving off.

Where the next euro should go

Fix the floor first, then allocate. The floor is neither negotiable nor expensive: management body training under Article 20(2), a scheduled awareness programme that repeats and covers new starters (CIR Annex 8.1.2), and records proving both happened [2][8]. The BSI’s practical formulation for staff training is an induction session plus annual updates on what has changed [11]; our breakdown of NIS2 training costs covers what each delivery tier runs to. Above the floor, the allocation looks different once you stop assuming awareness spend converts linearly into risk reduction.

Where the money goes	Effort	What the evidence supports	Article point it also evidences
Replace static modules with interactive training, and measure completion not assignment	Low	The only training variable with a measured protective effect [6]	21(2)(g)
Phishing-resistant multi-factor authentication	Medium	The trial’s authors explicitly recommend refocusing on technical countermeasures, naming two-factor authentication [7]	21(2)(j)
Password managers restricted to correct domains	Low	Also named by the researchers; removes credential entry on look-alike domains from human judgement entirely [7]	21(2)(i), 21(2)(j)
Role-specific training for privileged and security-relevant roles	Medium	Required in its own right under CIR Annex 8.2.1 and routinely thinner than all-staff awareness [8]	21(2)(g), 21(2)(i)
Extend simulation and reporting drills to voice and SMS	Medium	Mobile-centric vectors show materially higher success rates [9]	21(2)(g), 21(2)(b)
A tenth annual repeat of the same static module	Low	No benefit found from more sessions or more time; static repetition was associated with worse outcomes [6]	Attendance only

The compounding argument is the one for a budget meeting: rows two and three produce audit evidence under a different point of Article 21 while also reducing the risk the awareness programme was bought to reduce. The same euro does two compliance jobs and one security job. A tenth annual module does one compliance job and, on the available evidence, no security job.

ENISA’s NIS Investments 2025 survey of 1,080 organisations found cybersecurity holding at 9% of IT budgets, median €1.5 million, and characterised the period as a shift from people to technology — though 33% still plan to expand training and awareness [10]. The same survey put awareness-raising among the least difficult NIS2 requirements, cited by roughly one in five entities against 50% for vulnerability and patch management [10]. Ease is exactly why awareness absorbs budget the harder measures need, and ENISA’s read on the outcome is blunt: “Despite awareness and training programmes, phishing continues to challenge the operational resilience of organisations” [10].

The proportionality file that justifies whatever number you pick

Because Article 21(1) makes staff training proportionate rather than absolute, the defensible position is not a spend level — it is a documented reasoning chain from your risk assessment to your training design. That file doubles as your Article 21(2)(f) evidence, so it is not extra work. CIR Annex point 7.2 asks you to determine six things: what is monitored and measured, the methods, when measurement happens, who is responsible, when results are analysed, and who analyses them [8]. Answer those six for training and you have the file.

Role	Owns	Evidence produced
Management body	Approving the measures and completing its own training	Board resolution approving the programme; individual training records (Art. 20(1) and 20(2)) [2]
CISO / IT security	Content, delivery, simulation design, non-email coverage	Programme outline with objectives, content, frequency, syllabus and schedule [8]
HR / L&D	Induction coverage, new starters, contractors and external users	Attendance logs, sign-in sheets, completion certificates [8][11]
Compliance / risk	The effectiveness assessment and its cadence	Test and quiz results, review-and-update records, annual effectiveness review [8]

ENISA’s examples of evidence are worth matching exactly: the programme outline, copies of materials distributed, attendance logs and completion certificates, quiz or assessment results, employee feedback, and records showing the programme is reviewed and updated at least annually [8]. The BSI frames the same duty as continuous review and development of the measures, evidenced by documentation that they were carried out [11]. Our training audit checklist works through the controls an assessor tends to test first, and the Article 20(2) board training guide covers the management-body half.

One caution on the reasoning chain. A file arguing “our training is proportionate because 98% of staff completed it” restates an attendance number as an effectiveness finding. A file saying “we assessed effectiveness by report rate and time-to-report, found X, and redirected budget to interactive content and phishing-resistant MFA in response” describes an entity that actually operates Article 21(2)(f). Only the second survives a supervisor asking what the assessment concluded.

Frequently asked questions

Does NIS2 require security awareness training for all staff? Yes, but through Article 21(2)(g) rather than Article 20(2). Article 20(2) mandates training for management bodies and only obliges Member States to encourage entities to offer similar training to employees [2]. The binding all-staff duty is the cyber hygiene and training measure in Article 21(2)(g), which is subject to the proportionality test in Article 21(1) [3][4].

Can we justify a smaller awareness budget using this evidence? Partly. You cannot use it to skip training — the obligation is not conditional on demonstrated ROI, and Article 21(2)(f) separately requires you to assess effectiveness [4]. What the evidence supports is reallocating from repeated static modules, which showed no benefit, toward interactive content and technical controls [6][7], and documenting that reasoning in your proportionality file.

How often must training be delivered? The Directive says regularly without setting an interval. For entities in CIR scope, the awareness programme must be “scheduled over time, so that the activities are repeated and cover new employees”, and reviewed at least annually per ENISA guidance [5][8]. The BSI’s practical guidance is induction training plus annual updates on current developments [11].

Is phishing simulation itself required? No. Neither Article 21(2)(g) nor the CIR Annex names phishing simulation. CIR Annex 8.1.3 requires the awareness programme to be tested for effectiveness “where appropriate”, and the examples in ENISA’s guidance are quizzes or real scenarios [5][8]. Simulation is one way to satisfy that, not the mandated way.

The bottom line

A business case built on “awareness training prevents breaches” rests on a claim the best randomised evidence puts at 1.7 percentage points [6]. Build it on the two things that are true instead: the obligation is not optional, so the floor is a cost of operating; and Article 21(2)(f) turns the effectiveness question into an artefact you must produce anyway [4]. Spend the floor without arguing about it, measure something that can move, and let the measurement decide where the euro above the floor goes — into interactive content, into the roles carrying real privilege, into the vectors your simulations do not touch, and into the technical controls that work whether or not anyone was paying attention.

This article provides general information only and does not constitute legal or regulatory advice. Requirements may vary by jurisdiction and organisation type. Consult a qualified legal professional or compliance specialist for advice specific to your situation.

Sources

1. Directive (EU) 2022/2555 (NIS 2 Directive) — EUR-Lex, OJ L 333, 27.12.2022.
2. NIS 2 Directive, Article 20 — Governance.
3. NIS 2 Directive, Article 21 — Cybersecurity risk-management measures.
4. Directive (EU) 2022/2555, Article 21, points (a)–(j) — NIS2 Resources.
5. Commission Implementing Regulation (EU) 2024/2690 of 17 October 2024 — EUR-Lex.
6. Ho, G., Mirian, A., Luo, E., et al., “Understanding the Efficacy of Phishing Training in Practice” — 46th IEEE Symposium on Security and Privacy, 2025.
7. “Cybersecurity Training Programs Don’t Prevent Employees from Falling for Phishing Scams” — UC San Diego Today.
8. Technical Implementation Guidance on cybersecurity risk-management measures, version 1.0 — ENISA, June 2025.
9. 2026 Data Breach Investigations Report — Verizon.
10. NIS Investments 2025 — Main Report — ENISA.
11. Grundlegende Schulungen und Sensibilisierungsmaßnahmen (§ 30 BSIG) — BSI.
12. NIS-2-Geschäftsleitungsschulung (§ 38(3) BSIG) — BSI.