Expand AI logo
DocsDocs
Glow Active
API Reference
Login

Documentation

Get Started

OverviewWhy ExpandQuickstartWays to Use Expand

Agent Quickstarts

OverviewExpand SkillClaude CodeCursorCodexOpenCodeSkill-Based AgentsOther MCP Clients

Fetch

OverviewHow Fetch WorksOutput ModelInclude OptionsBrowser BehaviorHighlightsPlayground & ReplayBatched Fetch

Reference

API ReferenceCLI CommandsMCP Tools & ResourcesTypeScript SDK

Account & Billing

Pricing & UsageTiersFAQ

Machine-Readable Docs

start.mdllms.txtllms-full.txtDocs as Markdown
Browse docs

Get Started

OverviewWhy ExpandQuickstartWays to Use Expand

Agent Quickstarts

OverviewExpand SkillClaude CodeCursorCodexOpenCodeSkill-Based AgentsOther MCP Clients

Fetch

OverviewHow Fetch WorksOutput ModelInclude OptionsBrowser BehaviorHighlightsPlayground & ReplayBatched Fetch

Reference

API ReferenceCLI CommandsMCP Tools & ResourcesTypeScript SDK

Account & Billing

Pricing & UsageTiersFAQ

Machine-Readable Docs

start.mdllms.txtllms-full.txtDocs as Markdown

TypeScript SDK

Use @expandai/sdk to call Fetch from TypeScript and read Main Markdown plus State JSON in code.

Use @expandai/sdk to call Fetch from TypeScript: Main Markdown, State JSON, Highlights, Batched Fetch, and citation helpers.

npm install @expandai/sdk
pnpm add @expandai/sdk
bun add @expandai/sdk
import { ExpandClient } from '@expandai/sdk'

const expand = new ExpandClient({ apiKey: process.env.EXPAND_API_KEY })
const page = await expand.fetchJson({
url: 'https://news.ycombinator.com',
})
const mainMarkdown = page.markdown
const stateJson = page.json
const snapshotId = page.meta.snapshotId
const playground = page.meta.playground

Use fetchJson() for application code that needs Main Markdown plus State JSON. Use fetch() when you only want the Markdown string.

Jump to: JSON Mode · Markdown-only Fetch · Highlights · Batched Fetch · Citation Helpers · Errors · Effect

Install and auth

Install the package, then set your API key. The SDK reads EXPAND_API_KEY from the environment by default.

export EXPAND_API_KEY="xpnd_..."
import { ExpandClient } from '@expandai/sdk'

const expand = new ExpandClient({
  apiKey: process.env.EXPAND_API_KEY,
})

The SDK sends your key on every request as the x-expand-api-key header.

OptionDefaultDescription
apiKeyprocess.env.EXPAND_API_KEYExpand API key.
baseUrlhttps://api.expand.aiAPI base URL.

Method index

MethodUse whenReturns
fetchJson(params, options?)App code needs Main Markdown, State JSON, snapshot metadata, or inline Highlights.Object-mode Fetch result.
fetch(params, options?)You only need the Markdown string.string

JSON Mode with fetchJson

fetchJson() is the recommended SDK method for applications. It returns the Main Markdown, State JSON, snapshot metadata, and optional search results in one object.

const page = await expand.fetchJson({
  url: 'https://example.com',
})

console.log(page.markdown)
console.log(page.json)
console.log(page.meta.snapshotId)
console.log(page.meta.playground)

The result shape, abbreviated:

type FetchJsonResult = {
  meta: {
    snapshotId: string
    playground: string
    url: string
    capturedAt: string
    // ...
  }
  markdown: string
  json: Array<unknown>
  data?: {
    search?: {
      query: string
      snippets: Array<{
        source: 'markdown' | 'appendix' | 'statejson'
        text: string
        json?: unknown
        score: number
  • markdown is the Main Markdown.
  • json is State JSON and extracted evidence.
  • meta.snapshotId is the handle for later Highlights with fetchSearch.
  • meta.playground is the human inspection link.
  • data.search appears when inline search is requested.

This shape is not exhaustive. See Output Model and the API Reference for exact fields.

Markdown-only Fetch with fetch

fetch() is the convenience method for scripts and agents that only need the Markdown string.

const markdown = await expand.fetch({
  url: 'https://example.com',
})

console.log(markdown)
  • The return type is string.
  • This calls /v1/fetch.
  • Use fetchJson() if the application needs State JSON, snapshot metadata, or structured search results.

Include options

Both fetch and fetchJson accept query include options through the second argument.

const markdownWithAppendix = await expand.fetch(
  { url: 'https://example.com' },
  { include: 'appendix,statejson' },
)
const objectMode = await expand.fetchJson(
  { url: 'https://example.com' },
  { include: 'appendix' },
)
  • include is a query option passed as the second argument.
  • fetchJson body include options are available through the request body when you need exact structured control.
  • Passing { include: null } omits the include query param.

Exact include semantics belong to Include Options and the API Reference.

Highlights

Highlights ("search") run two ways: inline with a fresh Fetch, or against a stored snapshot.

Inline Highlights with fetchJson

const page = await expand.fetchJson({
  url: 'https://docs.example.com',
  search: {
    query: 'authentication limits',
    maxResults: 5,
    minScore: 0.6,
  },
})

const snippets = page.data?.search?.snippets ?? []
  • This starts a new Fetch.
  • Snippets live under data.search.snippets.
  • State JSON snippets may include json.
  • Raw SDK snippets expose location, not the MCP-only citationUrl field.

Snapshot Highlights with fetchSearch

const page = await expand.fetchJson({
  url: 'https://docs.example.com',
})

const result = await expand.fetchSearch({
  snapshotId: page.meta.snapshotId,
  search: {
    query: 'authentication limits',
    maxResults: 5,
    minScore: 0.6,
  },
  include: {
    markdown: true,
    json: true,
  • fetchSearch calls /v1/fetch/search.
  • It searches stored artifacts without recapturing the URL.
  • Use it to refine results after a previous Fetch.
  • The exact request schema belongs to the API Reference. See Highlights for behavior.

Citation Helpers

Raw SDK results expose snippet location. Use Playground helpers when you want to turn a snippet into a user-visible citation link.

import { citationUrl, resolvePlaygroundHost } from '@expandai/sdk/Playground'
const host = resolvePlaygroundHost(page.meta.playground, 'https://expand.land')

for (const snippet of page.data?.search?.snippets ?? []) {
  const url = citationUrl(page.meta.snapshotId, snippet.location, host)
  console.log(snippet.text, url)
}
HelperPurpose
playgroundBase(snapshotId, host?)Build the whole-snapshot Playground URL.
playgroundOrigin(metaPlayground)Extract the origin from meta.playground.
resolvePlaygroundHost(metaPlayground, fallback)Prefer the server-provided Playground host, fall back if missing.

SDK API responses do not already contain citationUrl. MCP adds that field; SDK users build it with these helpers. See Playground & Replay for URL semantics.

Batched Fetch

const run = await expand.batched({
  urls: [
    'https://example.com',
    'https://example.com/about',
  ],
})

let status = await expand.getBatched(run.id, { limit: '10', offset: '0' })

while (status.batchedStatus === 'QUEUED' || status.batchedStatus === 'RUNNING') {
  await new Promise((resolve) => setTimeout(resolve, 1000))
  • batched() returns a run ID.
  • getBatched() polls the run.
  • Each result includes its persisted status alongside data, so handle failed or cancelled URLs independently.
  • When omitted, the SDK generates one idempotencyKey and reuses it for the configured retry policy.
  • Supply a stable idempotencyKey to deduplicate separate calls or calls made after a process restart.
  • Reusing a key with a different payload returns 409 BatchedIdempotencyConflict with reason: 'payload_mismatch'.
  • The current generated TypeScript getBatched query params use strings for limit, offset, and include.

The full lifecycle belongs to Batched Fetch.

Effect Service

Use ExpandService when your application is already Effect-native. It exposes the same operations as ExpandClient, but failures are typed in the Effect error channel.

import { NodeRuntime } from '@effect/platform-node'
import { Effect } from 'effect'
import { ExpandService } from '@expandai/sdk'

const program = Effect.gen(function* () {
  const expand = yield* ExpandService
  const page = yield* expand.fetchJson({ url: 'https://example.com' })
  yield* Effect.log(page.markdown)
})

NodeRuntime.runMain(

You do not need this section to use the SDK from plain Promise code.

Errors and retries

Promise-side errors, thrown by ExpandClient:

ErrorMeaning
ExpandClientErrorBase Promise-side SDK error and invalid client options.
ExpandClientApiErrorNon-2xx API response. Inspect status and body.
ExpandClientConnectionErrorNetwork failure.

Effect-side errors, surfaced in the ExpandService error channel:

ErrorMeaning
ExpandSdkErrorInvalid service options or SDK setup error.
ExpandApiErrorNon-2xx API response.
ExpandConnectionErrorNetwork failure.
ExpandTimeoutError

Retry and timeout behavior:

  • Default timeoutMs is 60000.
  • Default maxRetries is 2.
  • Retryable failures include timeouts, connection errors, 408, 409, 429, and 5xx.
  • FetchNavigationFailed (502) and FetchCaptureTimeout (504) are exempt from automatic retries. A 504 is the conservative fallback when no result arrived and queue-only expiry was not proven; it does not prove the page caused the timeout. Avoiding automatic retries prevents amplification during control-plane incidents, though a deliberate manual retry may still be appropriate.
  • FetchCapacityTimeout (529) means Expand proved the request stayed queue-only until its deadline. It is retryable, and the SDK honors it through the normal retry policy; the response advertises Retry-After: 5.
import { ExpandClientApiError } from '@expandai/sdk'

try {
  await expand.fetchJson({ url: 'https://example.com' })
} catch (error) {
  if (error instanceof ExpandClientApiError) {
    console.error(error.status, error.body)
  }
  throw error
}

Choosing timeoutMs

timeoutMs is a total wall-clock budget across all SDK attempts, and it is enforced client-side. Synchronous Fetch has one 150-second capture deadline created before Hatchet dispatch; queueing and every worker retry or reassignment share that immutable epoch. Timeout classification can then use at most five seconds, so use 180 seconds to leave 25 seconds for response serialization and network transit:

const page = await expand.fetchJson(
  { url: 'https://example.com' },
  { timeoutMs: 180_000, maxRetries: 0 },
)
  • 180000 covers the capture-work budget, bounded classification, and delivery headroom. A result that arrives while timeout classification runs can still be returned successfully; otherwise, the API produces a typed FetchCaptureTimeout/FetchCapacityTimeout by 155 seconds even when the classification call hangs. The remaining 25 seconds are for the HTTP response and network.
  • The 60000 default is sized for typical pages; it aborts long captures with ExpandClientTimeoutError before Expand answers.
  • The budget covers retries and backoff too, so raise timeoutMs per additional attempt you allow, or set maxRetries: 0 when you only want one verdict.
  • FetchCaptureTimeout.timeoutMs is 150000 by default even when a replacement worker receives only a small remainder. The tag means timeout classification won and queue-only expiry was not proven; a proven queue-only expiry is reported separately as FetchCapacityTimeout.

Related pages

TopicWhere to go
First SDK or API setupQuickstart
Choosing SDK vs CLI/MCP/APIWays to Use Expand
Product behaviorFetch Overview
Main Markdown and State JSON
PreviousMCP Tools & Resources
NextPricing & Usage

On This Page

Install and authMethod indexJSON Mode with fetchJsonMarkdown-only Fetch with fetchInclude optionsHighlightsInline Highlights with fetchJsonSnapshot Highlights with fetchSearchCitation HelpersBatched FetchEffect ServiceErrors and retriesChoosing timeoutMsRelated pages
timeoutMs
60000
Total wall-clock budget across attempts.
maxRetries2Retry attempts for retryable failures.
fetchSearch(params, options?)
You already have snapshotId and want Highlights from stored artifacts.
Snapshot Highlights result.
batched(params, options?)You want to start many async Fetch jobs. The SDK generates options.idempotencyKey when omitted.{ id }
getBatched(id, options?)You want to poll a Batched Fetch run.Batched status/results.
location?: unknown
}>
}
}
}
appendix: false,
},
})
console.log(result.search.snippets)
citationUrl(snapshotId, location, host?)
Build a citation link from location.evidenceId.
status = await expand.getBatched(run.id, { limit: '10', offset: '0' })
}
console.log(status.results)
program.pipe(
Effect.provide(
ExpandService.layer({ apiKey: process.env.EXPAND_API_KEY }),
),
),
)
ExpandClientTimeoutError
Request exceeded timeoutMs.
Request exceeded timeoutMs.
Output Model
Include semanticsInclude Options
Highlights behaviorHighlights
Citation URLs and replayPlayground & Replay
Batched lifecycleBatched Fetch
Exact endpoint schemasAPI Reference