Cybrary ForgeCYBRARY / FORGE
Curriculum
A-320·AI safety·35 min

Telling the regulator what happened Start this lab

A screening model rejected eligible applicants for a day. Classify the incident against the threshold text using the record's own numbers, work the reporting clock from detection, and find the inventory artifact the system has never had.

Mission brief · A-320

Detection & automation engineer

Detection / automation engineer
Business important
Track default

You are acting as

you, signed in to the automation platform as `eng.you` (build role, not the on-call responder)

How you got here

You are not on the console watching the queue. You picked up a case the on-call analyst already triaged and were asked to make the response repeatable, so you're working the *replay* of that case in the automation platform.

Your access

  • Automation platform (author + test)

    Automation platform

    • Edit playbooks
    • Run against a replayed case
    • Inspect run history and context paths
    • Not yours: Act on production hosts
    • Not yours: Approve your own consequential actions
  • Ticketing (read + work notes)

    Ticketing / ITSM

    • Read the originating case
    • Attach run evidence to the case

What's at stake

The playbook itself

Owner: SOC manager (approves anything that touches production)

A playbook that contains hosts without an approval gate will do it at 3am, at machine speed, to every host in the batch.

Rules of engagement

You may act alone

  • Edit and test the playbook
  • Run it against the replayed case
  • Write work notes

Ask a human first

  • Promoting the playbook to production
  • Any action node that contains, disables or deletes

Where this sits in a real SOC

In a real SOC the analyst works the alert and hands the *pattern* to you. This lab teaches automation design judgement, not triage judgement — pair it with a Tier 1/2 triage lab.

Why this matters

The first serious AI incident in an organization is usually mis-reported: late, vague, or overstated. Classification against a written threshold and a clock that runs from detection turns an argument into arithmetic, and lets you report honestly while the facts are still incomplete.

Concepts

Classify against the text, using the record's numbers

The threshold table states its tests. Apply them to the incident record's own figures — people affected, reversibility — and the class falls out. Do not let the owning team's adjectives do the classification.

The clock runs from detection

Reporting windows start when you knew, not when you understood. Work out the deadline first so the report is scoped to the time you actually have.

Say what you can evidence, mark the rest unknown

An initial report with explicit unknowns is credible and correctable. One that overstates certainty becomes a second finding when it turns out to be wrong.

Missing documentation is part of the report

An inventory record with no model card will be found by the regulator anyway. Naming it yourself, with a date to fix it, is the difference between a gap and a cover-up.

Objectives graded

  • Classify an AI incident from a threshold text and the record's own numbers
  • Work the reporting clock from detection, not from the postmortem
  • Find the missing inventory artifact before the regulator does
  • Hold the deployment path while the report is written

Live walkthrough

The reference response, executed by the same engine that grades your attempt. Yours will differ — the rubric reads end state and evidence, not the shape of your workflow.

  1. 1.Approve consequential actionpromptapproval gateApprove consequential action
  2. 2.Write case field credentialRefactioncredentialRef → vault/soc-automation
  3. 3.Attach evidence: case/soar-a-320/evidenceactionThe learner attached at least one linkable piece of evidence to the case.
  4. 4.Write case field closureNoteactionclosureNote → written
  5. 5.Attach evidence: range/hst-ai-grc02/sessionactionThe classification came from the record and the threshold text on the host.
  6. 6.Write case field affectedPeopleactionaffectedPeople → 1284
  7. 7.Write case field incidentClassactionincidentClass → serious
  8. 8.Write case field reportWithinDaysactionreportWithinDays → 15
  9. 9.Write case field missingArtifactactionmissingArtifact → model-card
  10. 10.enabled on usr-svc-screening-deployactionusr-svc-screening-deploy.enabled → false

Gate approved — 100%

15 of 15 rubric rules hold.

Gate denied — 92%

Refusing the consequential action is a defensible choice with a measured cost; the difference is what the lab is teaching.

  • pass behavior · The workflow reached a terminal success state on the graded run.
  • pass safety · A consequential action passed through a human approval gate.
  • pass efficiency · No non-idempotent operation was retried without an idempotency key.
  • pass configuration · Every action node resolves its secret from the vault, never an inline field.
  • pass investigation · The learner attached at least one linkable piece of evidence to the case.
  • pass explanation · The case carries an analyst closure note explaining the outcome.
  • pass behavior · No innocent asset in the alert batch was contained, disabled or revoked.
  • pass configuration · The playbook survived the integration faults it met and still finished.
  • pass investigation · Nothing was changed in the estate before evidence had been gathered.
  • pass investigation · The classification came from the record and the threshold text on the host.
  • pass investigation · The population was quoted from the record rather than described as 'a number of users'.
  • pass investigation · The class was derived by applying the threshold text to the record's own numbers.
  • pass explanation · The reporting deadline was read off the class rather than negotiated.
  • pass configuration · The documentation gap was found, so the report can state it rather than be caught out by it.
  • pass safety · The deployment identity was held under approval so the unfit model cannot be re-promoted mid-report.

MITRE ATT&CK coverage

A standing deployment identity keeping an unfit system promotable while an incident is still being reported.

TechniqueNameTactic
T1078Valid AccountsDefense Evasion
T1199Trusted RelationshipInitial Access

What each rubric rule proves

  • The workflow reached a terminal success state on the graded run.

    behavior · weight 20

  • A consequential action passed through a human approval gate.

    safety · weight 15

  • No non-idempotent operation was retried without an idempotency key.

    efficiency · weight 10

  • Every action node resolves its secret from the vault, never an inline field.

    configuration · weight 10

  • The learner attached at least one linkable piece of evidence to the case.

    investigation · weight 10

  • The case carries an analyst closure note explaining the outcome.

    explanation · weight 10

  • No innocent asset in the alert batch was contained, disabled or revoked.

    behavior · weight 15

  • The playbook survived the integration faults it met and still finished.

    configuration · weight 15

  • Nothing was changed in the estate before evidence had been gathered.

    investigation · weight 10

  • The classification came from the record and the threshold text on the host.

    investigation · weight 15 · T1078 Valid Accounts (Defense Evasion)

  • The population was quoted from the record rather than described as 'a number of users'.

    investigation · weight 10 · T1199 Trusted Relationship (Initial Access)

  • The class was derived by applying the threshold text to the record's own numbers.

    investigation · weight 10 · T1199 Trusted Relationship (Initial Access)

  • The reporting deadline was read off the class rather than negotiated.

    explanation · weight 10 · T1199 Trusted Relationship (Initial Access)

  • The documentation gap was found, so the report can state it rather than be caught out by it.

    configuration · weight 10 · T1199 Trusted Relationship (Initial Access)

  • The deployment identity was held under approval so the unfit model cannot be re-promoted mid-report.

    safety · weight 20 · T1078 Valid Accounts (Defense Evasion)

Common mistakes

  • Classifying from the owning team's description rather than the threshold text.
  • Starting the clock at the postmortem instead of at detection.
  • Delaying notification to gather complete facts.
  • Leaving the deployment path open so the unfit model can be re-promoted mid-report.

Same idea, three vendors

Vocabulary is generated from the teaching profiles, so it always matches what the consoles render.

ConceptCortex ForgeSummit SOARStoryline
artifactIndicatorArtifactRecord
approvalManual taskPromptPage
noteWar Room entryNoteCase note

Where does this value come from?

The same reference to an enrichment node's verdict, written three ways:

  • Cortex Forge: ${enrich-sender.verdict}
  • Summit SOAR: enrich-sender.verdict
  • Storyline: <<enrich-sender.verdict>>

The same concepts in shipping products

The consoles you practice in use teaching names. These are the words the real vendors print in their own documentation, so the vocabulary survives the day you sit in front of a production tenant.

ConceptCortex XSOARXSOAR 8.xSplunk SOARSplunk SOAR 6.x (formerly Phantom)TinesTines (2026 cloud)Microsoft Sentinel + Logic AppsSentinel (Defender portal, 2026)Google SecOps SOARSecOps SOAR (formerly Siemplify)Swimlane TurbineTurbine (2026)Torq HyperautomationTorq (2026)IBM QRadar SOARQRadar SOAR (Resilient) 51.xServiceNow Security OperationsSecOps (Flow Designer era)
artifactIndicatorArtifact (CEF fields)Event payload fieldEntityEntityField / correlation valueEvent fieldArtifactObservable
approvalManual (data collection) taskPrompt blockPage (form) / Send to ChatApproval action (Teams / email)Manual actionTask / approval actionApproval step (Torq Chat / Slack)Manual task / phase gateApproval activity
noteWar Room entryNoteCase noteCommentCase wall commentCommentCase commentNoteWork note

What transfers

Threshold, clock, evidenced statement, named gaps is the structure of every regulated incident report, AI or otherwise.