Intercept API responses directly instead of parsing the DOM. This is often 10x faster and more reliable. Use browser dev tools to find which API endpoints return the data you need.The following examples show the same scraper—first using DOM manipulation, then using network interception.
DOM Manipulation (Slower)
TypeScript
import { BrowserContext, Page } from "playwright";import { goToUrl } from "@intuned/browser";export default async function automation( params: any, page: Page, context: BrowserContext) { await goToUrl({ page, url: "https://www.ycombinator.com/companies" }); // Wait for companies to load await page.getByText("Loading companies...").waitFor({ state: "hidden" }); await page.locator('a[href^="/companies/"]').first().waitFor(); // Get all company card links const companyCards = page.locator('a[href^="/companies/"]:not([href="/companies"])'); const count = await companyCards.count(); const companies: { name: string; industry: string; tags: string[] }[] = []; for (let i = 0; i < count; i++) { const card = companyCards.nth(i); const href = await card.getAttribute("href"); if (!href || href === "/companies") continue; const tagLinks = card.locator('a[href^="/companies?"]'); const tagCount = await tagLinks.count(); const tags: string[] = []; let industry = ""; for (let j = 0; j < tagCount; j++) { const tagText = (await tagLinks.nth(j).innerText()).trim(); const tagHref = await tagLinks.nth(j).getAttribute("href"); if (tagHref?.includes("industry=") && !industry) { industry = tagText; } tags.push(tagText); } const slug = href.replace("/companies/", ""); companies.push({ name: slug.replace(/-/g, " "), industry, tags }); } return companies;}
Avoid waitForTimeout() with arbitrary delays—they waste time when pages load fast and fail when pages load slowly. Instead, wait for something specific.
// Wait for the table to appearawait page.locator('#data-table').waitForElementState("visible");// Wait for at least one row to existawait page.locator('.table-row').first().waitForElementState("visible");
When scraping lists, avoid iterating with locators—each locator call auto-waits, adding milliseconds per row that compounds over hundreds of elements.Instead:
Wait for the list container to be visible
Extract all data in a single evaluate() call using querySelectorAll
Example: Scraping a list
TypeScript
import { BrowserContext, Page } from "playwright";export default async function automation( params: any, page: Page, context: BrowserContext) { await page.goto('https://example.com/products'); // Wait for the list container to be visible await page.locator('.product-list').waitFor({ state: 'visible' }); // Extract all data in a single evaluate call const products = await page.evaluate(() => { return Array.from(document.querySelectorAll('.product-item')).map(el => ({ name: el.querySelector('.name')?.textContent?.trim(), price: el.querySelector('.price')?.textContent?.trim(), })); }); return { products };}
Move expensive operations outside the browser context. Extract raw data first, then process it after the automation completes. This includes image processing, LLM calls, data transformation, and file conversions.
Example
TypeScript
import { BrowserContext, Page } from "playwright";export default async function automation( params: any, page: Page, context: BrowserContext) { await page.goto('https://example.com/products'); // Extract raw data only - process later const rawData = await page.evaluate(() => { return Array.from(document.querySelectorAll('.product')).map(el => ({ imageUrl: el.querySelector('img')?.src, title: el.querySelector('.title')?.textContent, price: el.querySelector('.price')?.textContent })); }); return { rawData };}
Instead of logging in every run, use AuthSessions to reuse authenticated browser state. This can save 5-30 seconds per run depending on the login complexity.
AI agents require multiple LLM calls per action. If your automation does predictable, repeatable steps, replace AI code with direct selectors. Use AI only for unpredictable page structures or as a fallback when deterministic code fails.
Stagehand Agent (Slower)
TypeScript
import z from "zod";import { Stagehand } from "@browserbasehq/stagehand";import type { BrowserContext, Page } from "playwright";import { attemptStore, getAiGatewayConfig } from "@intuned/runtime";interface Params { query: string;}async function getWebSocketUrl(cdpUrl: string): Promise<string> { if (cdpUrl.includes("ws://") || cdpUrl.includes("wss://")) { return cdpUrl; } const versionUrl = cdpUrl.endsWith("/") ? `${cdpUrl}json/version` : `${cdpUrl}/json/version`; const response = await fetch(versionUrl); const data = await response.json(); return data.webSocketDebuggerUrl;}export default async function automation( { query }: Params, page: Page, _context: BrowserContext) { const { baseUrl, apiKey } = await getAiGatewayConfig(); const cdpUrl = attemptStore.get("cdpUrl") as string; const webSocketUrl = await getWebSocketUrl(cdpUrl); const stagehand = new Stagehand({ env: "LOCAL", localBrowserLaunchOptions: { cdpUrl: webSocketUrl, viewport: { width: 1280, height: 800 }, }, model: { modelName: "openai/gpt-5-mini", apiKey, baseURL: baseUrl, }, }); await stagehand.init(); try { await page.goto("https://example.com/products"); // Each Stagehand action still requires LLM calls. await stagehand.act(`Search for "${query}" and select the first result.`); const productSchema = z.object({ product: z .object({ name: z.string(), price: z.string(), }) .nullable(), }); return await stagehand.extract( "Extract the product name and price.", productSchema ); } finally { await stagehand.close(); }}
If optimizations don’t help and simple actions are still slow, the site may be JavaScript-heavy. Consider these configuration changes:
Use a larger machine — Upgrade your machine size in replication settings for resource-intensive sites.
Turn off unnecessary features — Headful mode, stealth mode, and proxies add overhead. Disable them in intuned.json if your automation works without them.
Build URLs directly — Instead of clicking through filters, build the final URL with query parameters (e.g., ?category=electronics&price=under-100).
Go to iframe URLs directly — Instead of using frameLocator, navigate directly to the iframe’s source URL to avoid loading the parent page.
Use fill() instead of pressSequentially() — pressSequentially() types character-by-character. Use fill() for instant input unless you need keystroke events.
Avoid returning large data from evaluate() — Extract only the fields you need, not entire DOM elements or large HTML strings.