Navigate

The second phase of the Pluck pipeline. Bytes in from connect, prepared content out to extract. Passthrough by default; Readability, Playwright, and LLM-driven agents by choice.

The mental model

Connect gives you raw bytes. Extract pulls structured text. But sometimes the bytes aren't yet in the form extract can work with – an SPA hasn't rendered, a page needs a cookie consent dismissed, a multi-step form needs clicking, or an HTML page is buried under 50KB of navigation chrome that a naive HTML strip would flatten into noise.

That's what navigate is for. It's the preparation phase between connect and extract: take a ConnectResult, do whatever transformation the caller asks for, hand a clean NavigateResult to extract.

The pipeline looks like this:

Connect → Navigate → Extract → Shape → Act (or Sense) → Output

Navigate is a phase, not a standalone verb. You control it via the mode option on pluck():

import { pluck } from "@sizls/pluck";

// Default: direct passthrough. Fastest, no dependencies.
const raw = await pluck("https://example.com");

// Readability: strip navigation chrome, extract the article body.
const article = await pluck("https://news.example.com/post", {
  mode: "read",
});

All seven modes ship today. The four browser-backed modes (navigate, interact, agent, screenshot) light up once playwright is installed as an optional peer – pnpm add playwright && npx playwright install chromium. Without it, calling those modes throws a clear MISSING_PEER_DEP error that tells you exactly which install command to run.

The navigation modes

The full set is captured in the NavigationMode type. The IMPLEMENTED_NAVIGATION_MODES constant is your source of truth for which modes fire and which fall through to direct with a warning.

import { IMPLEMENTED_NAVIGATION_MODES } from "@sizls/pluck";

console.log(IMPLEMENTED_NAVIGATION_MODES);
// ["direct", "read", "navigate", "interact", "agent", "screenshot", "watch"]

direct vs read – what the modes actually do

Before reaching for a browser, know what the two shipping modes buy you. Given a cluttered news article:

import { pluck } from "@sizls/pluck";

const URL = "https://news.example.com/posts/big-tech-announcement";

// direct – raw HTML, nav + footer + ads + comments all in the text.
const raw = await pluck(URL, { mode: "direct" });
raw.text.length; // → 52_000 chars of DOM noise

// read – Readability-style extraction. The article body, title,
// byline, and little else.
const article = await pluck(URL, { mode: "read" });
article.text.length;         // → 4_200 chars of actual content
article.metadata.title;      // → "Big Tech announces thing"
article.metadata.byline;     // → "By Jane Reporter, 2026-04-21"
article.metadata.excerpt;    // → 160-char lede

// Any subsequent extract strategy sees the cleaner text.
const { data } = await pluck(URL, {
  mode: "read",
  extract: {
    strategy: "regex",
    patterns: [/\$([\d.]+B?)/g], // Prices and dollar amounts only
  },
});

Rule of thumb: start with direct; jump to read when the extract step keeps pulling in boilerplate. Jump to navigate / interact when the content only exists after JavaScript renders or a user action fires.

Navigate vs. Act – opposite intents

Navigate's browser-backed modes (navigate, interact, agent, screenshot) and the act phase's browser actors (browser, browser-agent) look similar on paper – both load URLs, click things, return data. They're not redundant; they do opposite things:

If you're reading a JavaScript-rendered page, dismissing a modal, or scrolling to trigger lazy-loads → navigate. If you're posting a form, buying a ticket, or sending a DM → act.

The line is: does this need to be undoable / auditable / policy-checked? If yes, it's act.

The Navigator contract

Every navigator follows the same tiny interface:

interface Navigator {
  name: string;
  navigate: (
    source: ConnectResult,
    options: NavigateOptions,
  ) => Promise<NavigateResult>;
}

interface NavigateOptions {
  mode?: NavigationMode;
  browser?: BrowserConfig;
  waitFor?: string;         // CSS selector to wait for (browser modes)
  scrollToBottom?: boolean; // Useful for infinite-scroll pages
  clickSelectors?: string[];
  timeout?: number;
  signal?: AbortSignal;
  steps?: InteractionStep[];    // "interact" mode
  screenshot?: ScreenshotOptions; // "screenshot" mode
  instruction?: string;         // "agent" mode
  maxSteps?: number;            // "agent" mode
  llm?: LlmConfig;              // "agent" mode
}

interface NavigateResult {
  content: string | Buffer;
  contentType: string;
  url: string;
  screenshot?: Buffer;
  metadata: Partial<PluckMetadata>;
}

NavigateResult is the handoff to extract. content and contentType are load-bearing – extract's canExtract(source) reads them to pick which extractor claims the result.

Typical flow

Direct passthrough – read a JSON API:

const data = await pluck("https://api.example.com/items");
// mode defaults to "direct"; connect hands bytes straight to extract.

Readability – clean article body:

const post = await pluck("https://news.example.com/article/42", {
  mode: "read",
});
console.log(post.text); // article body only, nav + footer stripped

Browser with wait-for – SPA rendering:

const rendered = await pluck("https://dashboard.example.com", {
  mode: "navigate",
  waitFor: ".data-grid-loaded",
  timeout: 10_000,
});

Agent – natural-language navigation:

const result = await pluck("https://shop.example.com", {
  mode: "agent",
  instruction:
    "Click 'Sign in', enter the credentials from env, then navigate to Orders and screenshot the page.",
  maxSteps: 8,
  llm: { apiKey: process.env.ANTHROPIC_KEY, model: "claude-sonnet-4-6" },
});

The agent navigator emits structured trace events you can follow in DevTools – see the studio command for replay.

Custom navigators

Same pattern as every other phase. Register at instance creation:

import { createPluck, type Navigator } from "@sizls/pluck";

const stealthBrowser: Navigator = {
  name: "stealth",
  async navigate(source, options) {
    // Drive puppeteer-extra-plugin-stealth, return rendered HTML.
    const html = await fetchWithStealth(source.metadata.url, options);
    return {
      content: html,
      contentType: "text/html",
      url: String(source.metadata.url ?? ""),
      metadata: source.metadata,
    };
  },
};

const pluck = createPluck({ navigators: [stealthBrowser] });
await pluck("https://bot-hostile.example.com", { mode: "stealth" });

The registry matches by navigator.name === mode, with one special case: mode: "read" routes to a navigator named reader. Custom navigators are prepended, so they win over built-ins that share a name.

Why navigate is a separate phase

Two reasons.

1. The extract phase only sees well-formed inputs. Without a navigate phase, every extractor would need to handle "raw bytes from an API", "rendered DOM from a browser", "Readability-cleaned article body", "screenshot PNG waiting to be OCR'd", and "agent-scripted interaction log" – all in one function. That design never stays clean. Separating navigate from extract means each extractor does one thing and the navigator picks the right preparation strategy up front.

2. Browser-backed work is a peer-dep decision. Playwright is heavy (Chromium alone is ~120 MB). Most Pluck users never need it. The browser-backed modes (navigate, interact, agent, screenshot) dynamic-import playwright on first use – your bundle stays lean unless you actually need SPA rendering, and installs that never touch those modes never pay the peer-dep cost.

What's next

• Extract – the next phase. Bytes → structured data.
• Concepts: Connect – the phase before navigate.
• Reference: Connectors – every URI scheme the connect phase speaks.