What Human-in-the-Loop AI Actually Means in Production

The phrase gets used loosely. In some contexts it means a human can override an AI system after the fact. In others it means a human reviews a sample of outputs periodically. Neither of those is what the term means in a production workflow context.

In production, human-in-the-loop means the workflow cannot proceed past a defined checkpoint without a human decision. The system pauses. It waits. The human sees the relevant data, makes a judgment, and the workflow continues on the path that judgment dictates. If the human never acts, the workflow stays paused. This is a hard dependency, not a soft suggestion.

That distinction matters for three reasons.

• Accountability. When a human decision is a hard dependency, there is a clear record of who decided what and when. The decision is not inferred from downstream behavior: it is an explicit, timestamped event tied to a specific person and a specific workflow instance.
• Correctability. Because the workflow pauses before continuing, a human can catch errors, hallucinations, policy violations, or quality problems before they propagate downstream. A human review that happens after the fact, when content has already been published or an action already taken, is not a checkpoint: it is a post-mortem.
• Trust. Teams adopt AI workflows faster when they know that defined categories of decision still require a human. The question is not whether to use AI: it is which steps benefit from automation and which steps require judgment that only a person can provide.

The practical design question is therefore not "should we have human-in-the-loop?" but "at which specific nodes in this workflow does human judgment add more value than it costs in latency?" The answer is different for every workflow and every organisation.

In DAVE, human-in-the-loop is implemented through a task system. When a running workflow instance reaches an interaction or review node, the workflow engine pauses execution and generates a task: a discrete work item assigned to a specific user. The workflow does not continue until the assignee submits a decision.

Tasks appear in the Task Inbox, accessible by navigating to Tasks in the sidebar. The Task Inbox lists every task assigned across the tenant, with a status filter in the top-right corner that lets assignees narrow the list to pending, in progress, completed, failed, or cancelled tasks. Each row in the list shows the task type, a color-coded status badge, the parent instance ID, the assignee, and the due date if one is set.

Clicking a task's type link opens the task detail page, which shows everything the assignee needs to make a decision:

• The task type and ID at the top of the page.
• Three metadata cards: the parent instance ID, the assignee, and the due date.
• A Payload section containing the JSON data passed from the workflow to the task. This is the content, output, or context the assignee is being asked to evaluate.
• A decision panel at the bottom (visible only for pending or in-progress tasks) with an optional Comment field and three decision buttons: Approve, Reject, and Request Changes.

When the assignee submits a decision, the workflow resumes from the node that generated the task. The decision determines which outgoing edge the workflow follows: approve routes forward, reject routes to a rejection branch, and request changes routes to a revision loop. The decision is recorded as a Submitted Result on the task detail page, visible after completion.

This model makes human-in-the-loop concrete and auditable. Every pause is a task. Every task has an assignee, a payload, a decision, and a timestamp. Nothing continues without a record.

DAVE generates four distinct task types, each corresponding to a different kind of human checkpoint in a workflow. Understanding which task type a node generates helps you design workflows where the right person sees the right information at the right moment.

user_interaction

A user_interaction task is generated when a workflow reaches a User Interaction node. It asks the assignee to provide input that the workflow needs to continue: a content brief, a topic selection, a set of parameters, or any other information that cannot be determined automatically. The assignee reads the payload (which describes what input is needed), enters a comment if useful, and clicks Approve to submit their response and advance the workflow.

User interaction tasks are the entry point for human-driven workflows. They are how a workflow collects the original brief, the source material, or the runtime parameters that shape everything that follows.

human_review

A human_review task is generated when a workflow reaches a Human Review node. It asks the assignee to evaluate content or output and render a judgment: approve, reject, or request changes. The payload contains the content to review. The assignee's decision directly determines which branch the workflow follows next.

Human review tasks are the quality gate of a production workflow. They are where an editor approves a draft, a compliance officer signs off on a communication, or a senior reviewer accepts or returns a deliverable. Adding a comment when rejecting or requesting changes is especially important here: the comment explains the reasoning to the workflow owner and to any downstream step that needs to act on the feedback.

agent_review

An agent_review task is generated when a workflow reaches an Agent Review node and the result requires human verification. The payload typically contains output from an AI agent. The assignee's role is to verify or override the agent's assessment before the workflow continues. This task type is the bridge between automated AI review and human judgment: the agent does the mechanical work, and the human confirms or corrects it.

safety_check

A safety_check task is generated when a workflow reaches a Safety node and the content is flagged rather than automatically blocked. The assignee reviews the flagged content and decides whether it should proceed, be rejected, or be escalated. Safety check tasks are how DAVE keeps a human in the loop for content that triggers a policy concern without automatically blocking every flagged item.

Not every user in a DAVE tenant can access the Task Inbox. Task access is controlled by role, and the design reflects the purpose of the task system: tasks are for people who actively participate in workflow execution, not for administrators or analysts who need visibility into the system but do not make workflow decisions.

DAVE ships with six default tenant roles: Admin, Create, Curate, Use, Reporting, and Financial. Roles are editable, so your administrator can adjust permissions to match your team's structure. For tasks specifically, the access model is straightforward:

• Admin and Use roles can view the Task Inbox, open task detail pages, submit decisions, and add comments.
• Create, Curate, Reporting, and Financial roles cannot access the Task Inbox or task detail pages.

If a user does not see the Tasks item in the sidebar, their role does not include task access. An administrator can verify and adjust role assignments through the admin user management section.

Task assignment is controlled by the workflow configuration, not by the role system alone. The workflow engine assigns tasks to users based on the configuration of the interaction or review node in the workflow graph. A task with no assignee (shown as a dash in the Assignee column) has not been claimed or the workflow did not specify a target user. Only the assigned user or a user with an appropriate role can view and complete a given task.

Every task operation is logged for compliance and accountability. Task completions, including the decision taken and the acting user, are recorded. Submission data is captured with sensitive fields redacted. These events support SOC 2 CC6.1 (logical access) and GDPR Article 30 (processing records) when those compliance modes are active for the tenant.

Human-in-the-loop is a design choice, not a default setting. Adding human review nodes to every step of a workflow defeats the purpose of automation. Adding none removes the accountability and correctability that make AI output trustworthy in production. The goal is to place human checkpoints where they add the most value relative to the latency they introduce.

Place review nodes where the cost of an error is highest

Not all errors are equal. A hallucinated fact in a draft that goes to a human editor before publication is caught and corrected. The same hallucination in a customer-facing communication that bypasses review causes reputational damage. Map the cost of an error at each step of your workflow and place Human Review nodes where the cost is highest: before publication, before a regulated action, before content reaches an external audience.

Use the payload to give reviewers what they need

A human reviewer can only make a good decision if the task payload contains the right information. Configure the workflow node's payload to include the content being reviewed, the context that produced it (the original brief, the agent's instructions, the source data), and any relevant metadata. A reviewer who sees only the output without the context cannot evaluate whether the output is correct. A reviewer who sees too much irrelevant data will slow down. The payload is a design surface: treat it as one.

Use comments to build an audit trail

The optional Comment field on every task decision is more valuable than it appears. A brief note explaining a rejection or a request for changes creates a record that the workflow owner, the next reviewer, and any future auditor can read. Encourage reviewers to use comments consistently, especially for reject and request changes decisions. Over time, the comment history of a workflow becomes a record of the judgment calls that shaped its output.

Watch due dates and pending tasks

A pending task blocks workflow progress. Tasks with a due date may time out and fail if not completed in time, causing the workflow instance to fail rather than pause indefinitely. Use the status filter in the Task Inbox to filter to pending tasks and identify any that are approaching their deadline. Workflows with high-volume review steps benefit from clear assignment rules and realistic due date configurations that match the reviewer's actual availability.

Automate the mechanical, review the consequential

The most effective human-in-the-loop workflows use AI to handle the mechanical steps (drafting, formatting, classification, extraction) and reserve human review for the consequential decisions (is this accurate, is this appropriate, does this meet our standard). An Agent Interaction node drafts the content. An Agent Review node checks it against defined criteria automatically. A Human Review node presents the result to a person only when the automated check finds something worth a human's attention, or when the content is consequential enough to require sign-off regardless. This pattern keeps humans in control without making them do mechanical work that a model can do reliably.