Skip to main content

Human-in-the-Loop

Not every change should be merged without a human looking at it first. Human-in-the-loop (HITL) adds approval gates to the agent workflow — high-risk work items pause after the merge request is created and wait for a human to review and approve before proceeding.

Risk levels

Every work item and worker definition can carry a risk level that determines whether human approval is required:

Risk levelBehaviour
LOWFully autonomous — the agent completes the work without pausing
MEDIUMFully autonomous — the agent completes the work without pausing
HIGHPauses after MR creation — requires human approval to continue
CRITICALPauses after MR creation — requires human approval to continue

If no risk level is specified, the default is LOW (fully autonomous).

The risk level can be set in two places:

SourceScope
Worker definitionApplies to all work items dispatched to workers of this definition
Work item metadataApplies to a specific work item, overriding the worker definition

The work item's risk level takes precedence over the worker definition's. This lets you set a sensible default on the definition and override it for specific tasks — for example, a backend worker definition might default to MEDIUM, but a particularly sensitive refactoring task might be elevated to HIGH.

tip

Some roles carry higher inherent risk. DB_MIGRATION and DEVOPS roles default to HIGH or CRITICAL because infrastructure and schema changes have a wider blast radius than typical code changes.

The approval flow

When an agent completes work on a HIGH or CRITICAL work item, the normal flow is interrupted:

The flow works as follows:

  1. Agent completes work — the agent runs normally, creates a branch, and opens a merge request. This part is identical to the autonomous flow.
  2. Runner checks risk level — after MR creation, the runner checks the work item's risk level (from the work item metadata first, then the worker definition metadata).
  3. If HIGH or CRITICAL — the work item status is set to NEEDS_APPROVAL. The runner creates an activity record noting the MR URL and exits. The worktree is cleaned up since the branch is already on the remote.
  4. Human reviews — a team member reviews the merge request in GitLab as they would any other MR.
  5. Human decides:
    • Approve — the work item is moved back to IN_PROGRESS, and the system finalises the work (merging or completing as appropriate).
    • Reject — the work item is set to CANCELLED, and the human closes the MR manually.
important

The agent's work — including the merge request — is fully visible before the approval decision. The human reviewer sees the complete diff, commit messages, and any eval or guardrail results before choosing to approve or reject.

The NEEDS_APPROVAL status

NEEDS_APPROVAL is a work item status that signals the work is paused for human review. It sits in the STARTED status category, meaning the work item is considered in-flight but not progressing.

While in NEEDS_APPROVAL:

  • The agent is not running — the runner has exited and the worktree has been cleaned up
  • The branch and merge request exist on the remote
  • No further automated action is taken until a human responds
  • The work item is excluded from the dispatch queue

Approval timeouts and escalation

Work items should not sit in NEEDS_APPROVAL indefinitely. The system uses configurable timeouts to ensure that pending approvals are surfaced:

TimeoutWhat happens
1x timeoutA reminder activity is created on the work item
2x timeoutAn escalation activity is created (which can trigger notifications via FND)

The timeout period is configurable. For example, with a 48-hour timeout:

  • After 48 hours without a decision, a reminder activity is logged
  • After 96 hours without a decision, an escalation activity is logged

These activities serve as nudges — the system does not auto-approve or auto-reject. A human must always make the final decision.

Configuring risk levels

On worker definitions

Set a default risk level for all work dispatched to a worker definition by including risk_level in the definition's metadata:

metadata:
risk_level: "HIGH"

This is appropriate when the nature of the work a definition handles is consistently high-risk — for example, a worker definition for database migrations should default to HIGH because schema changes carry inherent risk.

On work items

Override the default by setting risk_level in the work item's metadata:

metadata:
risk_level: "CRITICAL"

This is appropriate for one-off escalations — a normally low-risk task that happens to touch sensitive code, or a normally high-risk definition handling a straightforward change that can be trusted to run autonomously.

How HITL connects to other modules

  • PPM — the NEEDS_APPROVAL status and approval/rejection actions use existing PPM status transitions. The approval flow creates PPM activities that provide an audit trail.
  • Agent runs — the agent run completes normally regardless of risk level. HITL does not affect the agent's execution — it gates what happens after the agent finishes.
  • Guardrails — guardrail results are available before the approval decision. A human reviewer can check whether guardrails flagged any warnings before approving. See Guardrails for details.
  • Orchestration — in multi-agent workflows, a work item in NEEDS_APPROVAL blocks any dependent work items from starting (since it has not reached a completed status). See Orchestration for details.