> ## Documentation Index
> Fetch the complete documentation index at: https://intunedhq.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Human in the loop

> Pause an automation to ask a person for input, or hand them the live browser

> [!NOTE]
>
> This feature is in **preview**. The API may change before general availability. To try it out, contact [support@intunedhq.com](mailto:support@intunedhq.com) and we'll enable it for your workspace.

## Overview

Some automations can't be fully automated. A portal asks a security question only your customer can answer, a document needs a judgment call, or a final submission needs a person to review it. Human in the loop lets an automation stop, involve a person, and carry on.

Intuned gives you two runtime helpers:

| Helper | What the person does | Automation gets back |
| - | - | - |
| `humanInTheLoop` | Answers a question in **your** application | Structured data matching a schema you define |
| `handOverControlToHuman` | Drives the **live browser** and finishes the task | Nothing—it returns when they're done |

Both work the same way: Intuned posts a request to an endpoint on your system, then waits. You own the operator experience—who gets asked, what the UI looks like, how requests are queued and escalated. Intuned supplies the trigger, the callback, and the browser stream.

> [!NOTE]
>
> Waiting on a person does **not** count toward the attempt's `requestTimeout`. The `timeout` you pass to the helper is the only limit on the wait, so a Run can sit waiting for hours without its automation timeout expiring.

## When to use which

**Use `humanInTheLoop`** when you need a *value* and your automation can take it from there:

* A security question, a one-time code read off a device, or a CAPTCHA your solver can't handle
* A classification or approval decision—"is this the right invoice?"
* Disambiguation—three accounts matched, which one is it?

**Use `handOverControlToHuman`** when the remaining work is easier to *do* than to describe:

* A multi-step identity verification flow that changes every time
* A final submission someone must review on screen before confirming
* An exception path that's too rare to be worth automating

> [!TIP]
>
> If the task needs both—a decision, then more automation, then a person to finish—call `humanInTheLoop` mid-automation and `handOverControlToHuman` as the last step.

## Prerequisites

* A backend service that accepts HTTP POST requests from Intuned, on a publicly accessible HTTPS URL.
* For `handOverControlToHuman`: the project must run in headful mode (`"headful": true`). There's no browser to stream otherwise.

## Configure your endpoints

Register your endpoints in [`Intuned.json`](https://intunedhq.com/docs/main/05-references/intuned-json#param-human-in-the-loop). The two helpers use separate endpoints—the payloads have little in common, and the handoff flow has no callback at all.

```jsonc Intuned.json theme={null}
{
  "humanInTheLoop": {
    "enabled": true,
    "requestEndpoint": {
      "url": "https://your-app.com/intuned/human-in-the-loop",
      "headers": {
        "Authorization": "Bearer <your-secret>"
      }
    },
    "handoffEndpoint": {
      "url": "https://your-app.com/intuned/control-handoff",
      "headers": {
        "Authorization": "Bearer <your-secret>"
      }
    }
  }
}
```

Configure only the endpoint you use. Calling a helper with no matching endpoint throws a [non-retryable error](https://intunedhq.com/docs/main/05-references/runtime-sdk-typescript/non-retryable-error.md).

> [!TIP]
>
> Put the shared secret in an [environment variable](https://intunedhq.com/docs/main/02-features/environment-variables-secrets.md) and reference it, rather than committing it to `Intuned.json`.

## How it works

### Asking a person for data

1. Your automation calls `humanInTheLoop` with a prompt, a data schema, and optional supporting material.
2. Intuned POSTs a request to your `requestEndpoint`, including a **callback URL**.
3. Your endpoint returns `2xx` to acknowledge it. The automation stays paused—acknowledging is not answering.
4. A person answers in your application. Your backend POSTs the answer to the callback URL.
5. Intuned validates the answer against the schema and resumes the automation, which receives the data as the helper's return value.

If no valid answer arrives before `timeout`, the call throws `HumanInTheLoopTimeoutError`.

### Handing over control

1. Your automation calls `handOverControlToHuman` as its final step.
2. Intuned POSTs a request to your `handoffEndpoint`, including a **stream URL**.
3. Your application shows the stream URL to an operator—in an iframe, or a new tab. They see the live browser and have full keyboard and mouse control.
4. The operator finishes the task and selects **Complete** in the top-right corner of the stream. The stream UI also shows how much time is left.
5. The helper returns and the Run completes.

If nobody completes the session before `timeout`, the stream closes and the call throws `HumanInTheLoopTimeoutError`.

## Request payloads

### `human_in_the_loop`

```json theme={null}
{
  "type": "human_in_the_loop",
  "requestId": "hitl_2v9QxLm4Rt",
  "callbackUrl": "https://api.intuned.io/human-in-the-loop/hitl_2v9QxLm4Rt/callback?token=...",
  "prompt": "Which category does this document belong to?",
  "dataSchema": {
    "type": "object",
    "properties": {
      "category": { "type": "string", "enum": ["invoice", "statement", "other"] },
      "confirmed": { "type": "boolean" },
      "note": { "type": "string" }
    },
    "required": ["category", "confirmed"]
  },
  "supportingMaterial": [
    { "type": "image", "url": "https://assets.intuned.io/...signed..." },
    { "type": "text", "text": "Document ID: D-4471" }
  ],
  "metadata": { "documentId": "D-4471", "queue": "billing-review" },
  "timeout": 7200,
  "expiresAt": "2026-10-25T20:14:03Z",
  "workspaceId": "e95cb8d1-f212-4c04-ace1-c0f77e8708c7",
  "project": { "id": "482bf507-5fcc-43ed-9443-d8fff86015c4", "name": "claims-portal" },
  "apiInfo": { "name": "classify", "runId": "656CxOdANRlR5lWUAt_eC", "attempt": 1 }
}
```

### `control_handoff`

Same envelope, with `streamUrl` in place of `callbackUrl` and `dataSchema`:

```json theme={null}
{
  "type": "control_handoff",
  "requestId": "handoff_7KpR2nWq8Y",
  "streamUrl": "https://stream.intuned.io/handoff_7KpR2nWq8Y?token=...",
  "prompt": "Complete the identity verification step, then submit the application and select Complete.",
  "supportingMaterial": [],
  "metadata": { "applicantId": "A-8821" },
  "timeout": 1800,
  "expiresAt": "2026-10-25T19:02:11Z",
  "workspaceId": "e95cb8d1-f212-4c04-ace1-c0f77e8708c7",
  "project": { "id": "482bf507-5fcc-43ed-9443-d8fff86015c4", "name": "claims-portal" },
  "apiInfo": { "name": "submit-application", "runId": "9fKd1pQsVz0Lb_aXm", "attempt": 1 }
}
```

> [!NOTE]
>
> Image supporting material passed as a buffer—for example a `page.screenshot()` result—is uploaded by the runtime and delivered as a short-lived signed URL. Fetch it promptly or copy it into your own storage.

## The callback contract

For `humanInTheLoop` only. POST the answer to the `callbackUrl` from the request, with the data under a `data` key:

```bash theme={null}
curl -X POST "https://api.intuned.io/human-in-the-loop/hitl_2v9QxLm4Rt/callback?token=..." \
  -H "Content-Type: application/json" \
  -d '{
    "data": {
      "category": "invoice",
      "confirmed": true,
      "note": "Matched to PO-5512"
    }
  }'
```

| Status | Meaning |
| - | - |
| `200` | Accepted. The automation resumes with this data. |
| `400` | `data` didn't match `dataSchema`. The request stays open—fix the payload and post again. |
| `404` | Unknown `requestId`, or the callback token is wrong. |
| `409` | Already answered. Callbacks are single-use. |
| `410` | The request timed out or the Run was canceled. |

A `400` doesn't consume the request, so you can validate interactively—post, show the person the error, let them correct it.

## Implementing the endpoints

Your endpoint's only job is to accept the request quickly and queue it. Return `2xx` within 5 seconds; do the work of finding a person afterward.

**Express (Node.js)**

```typescript theme={null}
interface HumanInTheLoopRequest {
  type: "human_in_the_loop";
  requestId: string;
  callbackUrl: string;
  prompt: string;
  dataSchema: Record<string, unknown>;
  supportingMaterial: Array<
    { type: "text"; text: string } | { type: "image"; url: string }
  >;
  metadata: Record<string, unknown>;
  timeout: number;
  expiresAt: string;
  workspaceId: string;
  project: { id: string; name: string };
  apiInfo: { name: string; runId: string; attempt: number };
}

app.post("/intuned/human-in-the-loop", async (req, res) => {
  if (req.headers.authorization !== `Bearer ${process.env.INTUNED_HITL_SECRET}`) {
    return res.status(401).json({ error: "unauthorized" });
  }

  const request = req.body as HumanInTheLoopRequest;

  // Store the request, including the callback URL, and acknowledge immediately.
  await reviewQueue.enqueue({
    id: request.requestId,
    callbackUrl: request.callbackUrl,
    prompt: request.prompt,
    schema: request.dataSchema,
    material: request.supportingMaterial,
    expiresAt: new Date(request.expiresAt),
    queue: request.metadata.queue,
  });

  res.status(202).json({ received: true });
});

// Later, when a person submits their answer in your UI:
async function submitAnswer(requestId: string, data: unknown) {
  const { callbackUrl } = await reviewQueue.get(requestId);

  const response = await fetch(callbackUrl, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ data }),
  });

  if (response.status === 400) {
    const { error } = await response.json();
    throw new Error(`Answer rejected by schema validation: ${error}`);
  }
  if (!response.ok) {
    throw new Error(`Callback failed with ${response.status}`);
  }
}
```

**Flask (Python)**

```python theme={null}
import os
from datetime import datetime

import requests
from flask import jsonify, request


@app.route("/intuned/human-in-the-loop", methods=["POST"])
def intuned_human_in_the_loop():
    expected = f"Bearer {os.environ['INTUNED_HITL_SECRET']}"
    if request.headers.get("Authorization") != expected:
        return jsonify({"error": "unauthorized"}), 401

    payload = request.get_json()

    # Store the request, including the callback URL, and acknowledge immediately.
    review_queue.enqueue(
        id=payload["requestId"],
        callback_url=payload["callbackUrl"],
        prompt=payload["prompt"],
        schema=payload["dataSchema"],
        material=payload["supportingMaterial"],
        expires_at=datetime.fromisoformat(payload["expiresAt"].replace("Z", "+00:00")),
        queue=payload["metadata"].get("queue"),
    )

    return jsonify({"received": True}), 202


# Later, when a person submits their answer in your UI:
def submit_answer(request_id: str, data: dict):
    entry = review_queue.get(request_id)

    response = requests.post(entry.callback_url, json={"data": data}, timeout=10)

    if response.status_code == 400:
        raise ValueError(f"Answer rejected by schema validation: {response.json()}")
    response.raise_for_status()
```

For the handoff endpoint, the same pattern applies—store `streamUrl` and surface it to an operator. There's no callback to make.

> [!NOTE]
>
> If your endpoint doesn't return `2xx`, Intuned retries with exponential backoff while the request is still within its `timeout`. A request that's never acknowledged times out like one that's never answered.

## Timeouts and retries

* **`timeout` is per call**, in seconds, defaulting to one hour. Set it from how long a person realistically needs—a queued review might be hours, a live handoff minutes.
* **The attempt timeout doesn't apply to the wait.** `requestTimeout` keeps governing your automation's own execution time.
* **A timeout fails the attempt.** If the Run has retries left, the next attempt runs from the top and issues a **new** request—the person gets asked again. Wrap the call and throw [`NonRetryableError`](https://intunedhq.com/docs/main/05-references/runtime-sdk-typescript/non-retryable-error) if asking twice would be wrong.
* **Requests are tied to an attempt.** The `attempt` field in `apiInfo` lets you detect and collapse duplicates in your queue.

## Security

* **Authenticate inbound requests.** Intuned sends the `headers` you configure on every request. Verify them before acting, and reject anything else.
* **Treat `callbackUrl` and `streamUrl` as secrets.** Both carry a bearer token. Anyone with the callback URL can answer on your behalf; anyone with the stream URL can drive the browser. Don't log them, don't put them in URLs you render to untrusted users, and scope them to the operator who's handling the request.
* **Both URLs expire** with the request and are single-use.
* **A handoff exposes whatever is on screen.** The operator sees the live session, including any data already loaded. Only hand control to people authorized to see it.
* **Pause tracing around a handoff** if the person might type a password or a one-time code. The trace keeps recording while they work. See [Pause tracing](https://intunedhq.com/docs/main/05-references/runtime-sdk-typescript/pause-tracing.md).

## Best practices

* **Write prompts for a person who lacks your context.** "Pick the account ending 4471" beats "disambiguate account".
* **Attach a screenshot.** A `page.screenshot()` as supporting material is usually the difference between a fast answer and a wrong one.
* **Keep schemas small.** Ask for the decision, not a transcription of the page. Fewer fields means fewer rejected callbacks.
* **Use `metadata` for routing**, so your endpoint doesn't have to parse prompts to decide who should handle a request.
* **Make your endpoint idempotent on `requestId`.** Retries and attempt-level retries both produce repeat deliveries.
* **Alert on requests nearing `expiresAt`**, so a forgotten review doesn't silently fail a Run.

## Limitations

* **You build the operator UI.** Intuned doesn't provide a review queue or an inbox. The endpoint contract is the integration point.
* **Your endpoint must be publicly reachable** over HTTPS. There's no polling alternative during preview.
* **Handoff requires headful mode**, which uses more resources than headless—see the machine `size` guidance in [Intuned.json](https://intunedhq.com/docs/main/05-references/intuned-json.md).
* **A handoff has no return value.** The person's work shows up as page state, not as data. Call it last.
* **Waiting occupies a machine.** A Run paused on a person still holds its slot against your project's `maxCount`, so a queue of waiting Runs can block new ones. Size `replication` with that in mind.
* **One request at a time per automation.** Both helpers block until they resolve; concurrent requests from a single Run aren't supported.

## FAQs

> **Is `handOverControlToHuman` a separate feature from `humanInTheLoop`?**
>
> They're two helpers for the same problem, documented together because they share project configuration and the same request envelope. They use separate endpoints, and you can adopt either on its own.

> **What does the person see during a handoff?**
>
> The live browser session, streamed over VNC and embeddable in an iframe, with full keyboard and mouse control. A **Complete** button sits in the top-right corner, next to the time remaining on the handoff `timeout`.

> **Can I cancel a request that's waiting?**
>
> Cancel the Run with the [Run cancel endpoint](https://intunedhq.com/docs/client-apis/api-reference/projectsruns/run-api--cancel.md). The pending request is invalidated and a later callback gets `410`.

> **Does the automation keep the same page and session state?**
>
> Yes. The browser stays open and the attempt stays alive for the whole wait. When the helper returns, your code continues against the same `page`—though after a handoff, the person may have changed what's on it.

> **Can I use this inside a Job?**
>
> Yes, but think about the shape of it. A JobRun that stalls on dozens of human requests will hold machines and run long. Prefer human in the loop on [Runs](https://intunedhq.com/docs/main/02-features/runs-single-executions.md) triggered by your application, where a person is already involved.

> **What if my endpoint is down when a request fires?**
>
> Intuned retries with exponential backoff for as long as the request's `timeout` allows. If it's never acknowledged, the helper throws `HumanInTheLoopTimeoutError` and the attempt fails.

## Related resources

> **[Human in the loop (TypeScript)](https://intunedhq.com/docs/main/05-references/runtime-sdk-typescript/human-in-the-loop.md)**
>
> `humanInTheLoop` and `handOverControlToHuman`

> **[Human in the loop (Python)](https://intunedhq.com/docs/main/05-references/runtime-sdk-python/human-in-the-loop.md)**
>
> `human_in_the_loop` and `hand_over_control_to_human`

> **[Intuned.json reference](https://intunedhq.com/docs/main/05-references/intuned-json.md#param-human-in-the-loop)**
>
> Endpoint configuration properties

> **[Webhook integration](https://intunedhq.com/docs/main/04-integrations/webhook.md)**
>
> The same endpoint-building patterns, for Run results


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.