Human in the loop

This feature is in preview. The API may change before general availability. To try it out, contact support@intunedhq.com and we’ll enable it for your workspace.

Pause an automation and wait for a person. Use humanInTheLoop to ask a question and get a structured answer back, or handOverControlToHuman to give an operator the live browser and let them finish the task themselves. Both helpers call an endpoint on your system, which you register in Intuned.json. Intuned doesn’t provide the operator UI—you decide who gets asked and how.

Time spent waiting on a person does not count toward the attempt’s requestTimeout. The timeout you pass to these helpers is the only limit on the wait. See Human in the loop for the full request and callback contract.

Available helpers

humanInTheLoop — Ask a person for data and wait for the answer
handOverControlToHuman — Hand the live browser to a person

Function reference

humanInTheLoop

import { humanInTheLoop } from "@intuned/runtime";

function humanInTheLoop<T>(options: {
  prompt: string;
  dataSchema: JsonSchema | z.ZodSchema;
  timeout?: number;
  supportingMaterial?: SupportingMaterial[];
  metadata?: Record<string, unknown>;
}): Promise<T>;

Sends a request to your configured human-in-the-loop endpoint and waits. Resolves with the data your system posts back to the callback URL, validated against dataSchema. Parameters

string

required

What you need from the person, in plain language. Shown to whoever handles the request.

JsonSchema | z.ZodSchema

required

The shape of the answer you expect. Sent to your endpoint as JSON Schema, and used to validate the callback payload before the automation resumes. Accepts a JSON Schema object or a Zod schema, like extractStructuredData.

number

default:"3600"

How long to wait for an answer, in seconds. After this, the call rejects with HumanInTheLoopTimeoutError.

SupportingMaterial[]

Context to help the person answer—a screenshot of the page they’re deciding about, extracted text, or a link. See SupportingMaterial.

Record<string, unknown>

Arbitrary JSON passed through to your endpoint untouched. Use it to route the request to the right queue, team, or end user.

Returns Returns the validated answer. When you pass a Zod schema, the return type is inferred from it; with a JSON Schema object, supply the type parameter yourself. Throws

HumanInTheLoopTimeoutError — No valid answer arrived within timeout.
NonRetryableError — No human-in-the-loop endpoint is configured for the project.

Example

import { humanInTheLoop } from "@intuned/runtime";
import { z } from "zod";

const ClassificationSchema = z.object({
  category: z.enum(["invoice", "statement", "other"]),
  confirmed: z.boolean(),
  note: z.string().optional(),
});

export default async function classify(params: { documentId: string }, page: Page) {
  const screenshot = await page.screenshot();

  const answer = await humanInTheLoop({
    prompt: "Which category does this document belong to?",
    dataSchema: ClassificationSchema,
    timeout: 7200,
    supportingMaterial: [
      { type: "image", buffer: screenshot },
      { type: "text", text: `Document ID: ${params.documentId}` },
    ],
    metadata: { documentId: params.documentId, queue: "billing-review" },
  });

  // answer is typed as { category: ...; confirmed: boolean; note?: string }
  return { category: answer.category, reviewed: answer.confirmed };
}

handOverControlToHuman

import { handOverControlToHuman } from "@intuned/runtime";

function handOverControlToHuman(options: {
  prompt: string;
  timeout?: number;
  supportingMaterial?: SupportingMaterial[];
  metadata?: Record<string, unknown>;
}): Promise<void>;

Sends a request to your configured handoff endpoint with a stream URL, and waits for the person to finish. The stream URL renders the live browser session in an iframe or a browser tab, with full keyboard and mouse control and a Complete button in the top-right corner. The call resolves when the person selects Complete.

Call this as the last step of your automation. Once control is handed over, the person can navigate, submit forms, and change page state—anything your code does afterward runs against a page you no longer control.

Requirements

The project must run in headful mode ("headful": true in Intuned.json). There’s no browser to stream otherwise.
A handoffEndpoint must be configured. See Configure your endpoints.

Parameters

string

required

What the person needs to do once they take over. Shown alongside the stream.

number

default:"3600"

How long the session stays open, in seconds. The remaining time is shown to the person in the stream UI. When it expires, the stream closes and the call rejects with HumanInTheLoopTimeoutError.

SupportingMaterial[]

Context to show next to the stream. See SupportingMaterial.

Record<string, unknown>

Arbitrary JSON passed through to your endpoint untouched.

Returns Returns void when the person selects Complete. Throws

HumanInTheLoopTimeoutError — The person didn’t complete the session within timeout.
NonRetryableError — No handoff endpoint is configured, or the project isn’t running in headful mode.

Example

import { handOverControlToHuman, withTracingPaused } from "@intuned/runtime";

export default async function submitApplication(params: { applicantId: string }, page: Page) {
  await page.goto("https://portal.example.com/applications/new");
  await page.getByLabel("Applicant ID").fill(params.applicantId);

  // Everything is filled in except the identity check, which only a person can clear.
  await withTracingPaused(async () => {
    await handOverControlToHuman({
      prompt:
        "Complete the identity verification step, then submit the application and select Complete.",
      timeout: 1800,
      metadata: { applicantId: params.applicantId },
    });
  });
}

A handoff streams a live browser to a person who may type a password or a one-time code. The trace keeps recording while they work, so wrap the handoff in withTracingPaused when the session could involve credentials.

`SupportingMaterial`

type SupportingMaterial =
  | { type: "text"; text: string }
  | { type: "image"; url: string }
  | { type: "image"; buffer: Buffer };

Optional context delivered to your endpoint with the request.

{ type: "text", text } — Plain text. Use it for extracted values, a page excerpt, or an explanation of why the automation stopped.
{ type: "image", url } — An image at a URL your endpoint can fetch.
{ type: "image", buffer } — An image as raw bytes. Buffers are uploaded by the runtime and delivered to your endpoint as short-lived signed URLs, so a page.screenshot() result can be passed straight through.

Human in the loop — Endpoint configuration, request payloads, and the callback contract.
Intuned.json reference — humanInTheLoop project configuration.
Pause tracing — Keep credentials out of traces during a handoff.
Non-retryable errors — Stop a Run instead of asking a person twice.

Getting started

Intuned Agent

Web Tasks

Platform features

Learn

How-To

Integrations

References

Resources

Human in the loop

Available helpers

Function reference

humanInTheLoop

handOverControlToHuman

`SupportingMaterial`

​Available helpers

​Function reference

​humanInTheLoop

​handOverControlToHuman

​SupportingMaterial

​Related

Available helpers

Function reference

humanInTheLoop

handOverControlToHuman

`SupportingMaterial`

Related