distill

Clean Markdown an agent can actually use.

Local-first CLI and MCP server. Strip the chrome, keep the tables, resolve the links. Same input → same Markdown, every time.

Try it in the browser Install

npx -y -p distill-md distill-mcp

Measured, not marketed

50-URL corpus across articles, docs, product pages, forums, tables, and SPAs — process time is network-free median on the current build.

Reproduce in bench/

coverage
usable output returned

~0 ms

per page
median process time

code kept
fenced blocks preserved

tables kept
structure preserved

Structural preservation vs local and cloud alternatives
Tool	Tables	Code	Coverage	Runs
distill	0.62	0.94	92%	local
jina	0.27	0.17	100%	cloud
trafilatura	0.18	0.69	92%	local
markitdown	0.57	0.68	66%	local
readability	0.00	0.00	80%	local

Structural scores are micro-average Σkept / Σsource. Higher isn’t always better on links/headings (boilerplate inflation). Tables and code blocks are the high-signal features for agent use. Deterministic: byte-identical Markdown on re-run.

Try it

real Rust core → WASM, in your browser

Paste HTML and distill converts it to Markdown client-side. No server, nothing leaves the page.

Note: the CLI normally takes a URL ( distill <url>. Browsers can’t fetch arbitrary sites (CORS), so this demo runs the HTML → Markdown step only.

HTML in

Markdown out

Output appears here…

Built for agents, not reader mode

Most extractors optimize for blog posts. Agents need API docs, pricing tables, forum threads, and reference pages — structural fidelity first.

Local & private

No cloud API, no per-page cost. Nothing leaves your machine.

Deterministic

Byte-identical Markdown for the same input, every run.

Structure-preserving

Real tables, fenced code with language tags, nested lists, blockquotes.

Agent-ready JSON

Opt-in --agent-ready: sectioned Markdown, RAG chunks, and a schema layer — see how.

Metadata & links

Title, author, date in YAML frontmatter; relative links resolved absolute.

opt-in · --agent-ready

Want clean data for your agents?

Markdown is the default — and the right default. When your pipeline needs more than a flat blob, flip on agent-ready JSON: the same local extract, reshaped into three layers agents can retrieve, cite, and query without re-parsing the page.

01 · sectioned_markdown

Normalized headings plus a Contents outline — readable prose that still knows its shape.

02 · chunks

RAG units with id, heading, level, text, and source — ready to embed or hand to a retriever.

03 · schema

Structured extract: meta, outline, links, tables, and code blocks — no LLM rewrite.

page.agent-ready.json · shape

deterministic · local

{
  "sectioned_markdown": "# Widget Docs\n\n## Contents\n- Install\n- API\n…",
  "chunks": [
    { "id": "chunk-1", "heading": "Install", "text": "…", "source": "https://…" }
  ],
  "schema": {
    "title": "Widget Docs",
    "outline": […],
    "links": […],
    "tables": […],
    "code_blocks": […]
  }
}

CLI

distill https://example.com/docs --agent-ready
distill https://example.com/docs --ars -o page.json
distill https://example.com/docs -A

MCP

{
  "name": "distill_url",
  "arguments": {
    "url": "https://example.com/docs",
    "agent_ready": true
  }
}

Default output stays plain Markdown. Agent-ready only changes the final shape — same extract pipeline, no cloud call, same bytes on re-run.

Install

Published as distill-md; the installed command is still distill.

Agents · MCP server (no install)

npx -y -p distill-md distill-mcp

macOS / Linux

curl -LsSf https://github.com/gokulnair2001/distill/releases/latest/download/distill-md-installer.sh | sh

Rust

cargo install distill-md

Prebuilt binaries for macOS, Linux, and Windows on every GitHub Release.

Usage

# From a URL
distill https://en.wikipedia.org/wiki/Markdown

# From a local file, or stdin
distill page.html
curl -s https://example.com | distill -

# Multiple inputs → one file each
distill https://a.com https://b.com -o out/

Use with an agent

Run distill as an MCP server and any local agent (Claude Code, etc.) can call distill_url, distill_urls, distill_html. Pass agent_ready: true when you want the three-layer JSON. SSRF-guarded by default.

claude mcp add distill -- npx -y -p distill-md distill-mcp