How do I extract website content to Markdown?
For one public page, paste the URL into a free converter such as Firecrawl's or Microlink's and download the result. For a page your session must already be able to open, or for a bounded list you want converted together, give the URLs to your coding agent in ego(lite). It waits for each rendered page, keeps the main content while leaving out navigation and ads, and writes Markdown files with the source URL, title, checked-at time, and access status. The output is a conversion of what your authorized browser can see—not a paywall bypass, hidden-content extractor, or site-wide crawler.
How do I convert a URL to Markdown?
For a single public page, paste it into a free web converter such as Firecrawl's or Microlink's — that is the fastest route and this workflow is not trying to replace it. For pages those tools cannot reach, install ego(lite) and give your coding agent the URLs, the output shape you want, and a rule to stop at anything needing a human. The agent opens each page in a visible browser Space using your own session, converts what rendered, and writes .md files that keep the source URL, title, and check time.
Why do URL-to-Markdown tools fail on pages behind a login?
Because they fetch the page from their own servers, with no access to your cookies or session. Firecrawl's free tool states it directly: it works on publicly accessible pages, and content behind a login or paywall is not accessible without credentials. What you usually get back is not an error but clean Markdown of a login form or a paywall teaser. ego(lite) converts inside the browser context you authorized, so it sees the page you see, and it records the access status alongside the content.
Is this a free webpage-to-Markdown converter?
ego(lite) is free to download for Mac, and the work is done by the coding agent you already use, so the only cost is your agent's tokens. There are no conversion credits or per-page charges in this workflow. Compare that to the usual meaning of free in this category: Microlink allows 25 conversions a day, Firecrawl's free plan runs on a monthly credit allowance and limits its free tool to one page at a time, and Jina AI's Reader offers a rate-limited free tier before paid plans. The trade-off here is setup: you need a Mac and a coding agent.
Can I convert a webpage to Markdown without installing anything?
Yes, and for one public page you should. A hosted converter or the r.jina.ai URL prefix will do it in seconds with nothing installed. Choose this workflow when the thing you actually need is different: a page that requires your session, a list of pages in one run, or files that record which page state was converted. If none of those apply, a web converter is the better tool and this page will not pretend otherwise.
What is the best way of converting HTML to Markdown in code?
If you already have the HTML, use a library: Turndown in JavaScript or TypeScript, markdownify or html2text in Python, or Pandoc on the command line. They are mature, free, and the right answer for HTML in hand. Their limit is that they convert what you give them — they do not go and render a page, and they have no access to your logged-in session. ego(lite) covers that earlier step: getting the correct rendered page out of a real browser. The two compose; use a library for HTML you have, this for pages you need to open first.
How do I convert HTML or a webpage to PDF?
If you already have an HTML file, use your browser's print-to-PDF command or a server-side HTML-to-PDF library. For a webpage that must render in an authorized browser session, ask your coding agent to open the URL in ego(lite), wait for the required state, and print the visible page with a defined paper size, orientation, margins, and background setting. The output is a PDF of that rendered state with source URL, checked-at time, and access status recorded. It does not bypass login walls or guarantee that interactive widgets, animations, fonts, or complex layouts reproduce exactly.
Can I save a logged-in webpage as a PDF?
Yes, when the page is already openable in a browser session you are authorized to use. The agent can print the visible page from its ego(lite) Space after you handle any required login, consent, CAPTCHA, or verification step. It never enters credentials or circumvents a control, and the PDF should retain the source URL, checked-at time, and access status so you can review what was archived. Check the source's terms and any sensitive data before sharing the file.
Does it handle JavaScript-rendered pages and SPAs?
Yes. The agent works in a real Chromium browser and converts the page after it has rendered, so content a plain HTTP request would miss is present in the Markdown. Some hosted converters also render pages now, so this is no longer a unique advantage on public URLs. It matters most in combination with your session: a client-rendered page that also requires a login is where anonymous fetchers fail on both counts.
How do I save a web page as a Markdown file?
Ask for it in the prompt. Turning a URL to a Markdown file is the default output: the agent writes one .md per page into the location you specify, or a single combined document if you prefer, with source_url, title, and checked_at in the front matter. Because links and images are resolved to absolute URLs, the file still works after you move it into a notes vault or a repository. Pages that did not fully open appear in the run summary with their status rather than as files.
Can it convert a whole site or many pages at once?
It converts a bounded list of URLs, each in its own Space, in parallel — a docs section or a reading list is one run. It is not a site crawler: it will not discover and follow links across a domain, and this page will not claim otherwise. If you need site-wide crawling with scheduling and retries, a crawler API such as Firecrawl's is the built-for-purpose category. If you need a supervised run over pages you chose, including ones that require your session, this fits better.
What happens to images, links, and tables?
Tables become GitHub-flavored Markdown tables, code blocks become fenced blocks carrying the language when the page shows it, and emphasis, headings, and lists map to their Markdown equivalents. Links and image sources are rewritten to absolute URLs so they resolve outside the original site. You can ask for images as Markdown image links, as a separate list, or omitted. Text inside an image is not extracted; there is no OCR in this workflow.
Why convert web pages to Markdown for an LLM or RAG pipeline?
Markdown carries the document's structure — headings, lists, tables, code — in far fewer tokens than the equivalent HTML, so the same content costs less context and its hierarchy survives chunking. That is why Markdown has become the common target format for feeding web content to models. What this workflow adds is provenance and reach: each file records the URL and time it came from, so you can re-check a retrieved chunk against its source, and internal documentation that a hosted fetcher cannot open can go into the pipeline too.
Is the Markdown identical to the original page?
No, and no converter's is. Markdown has no equivalent for many things a page can do: interactive widgets, forms, embedded players, complex nested layouts, and heavy styling do not survive, and unusual table structures may need a manual pass. The agent converts the main content and leaves out navigation, ads, and consent banners, which is normally what you want but is still a judgment about what counts as content. The front matter keeps the source URL so you can always go back and compare.
Is it legal to convert a web page to Markdown?
Converting a page you are authorized to view into another format for your own reading or research is ordinary use of your browser, and this workflow does nothing your browser cannot already do. Two things still apply: the source's copyright does not change because the format did, so republishing converted content is a separate question, and a site's terms may restrict automated access regardless of format. This is not legal advice. Keep runs bounded to pages you have legitimate access to, and never use this to get around a paywall.
How do I clean web content for RAG ingestion without losing hierarchy?
Convert the main article or documentation element, keep its heading levels, lists, tables, and code fences, and remove navigation, ads, cookie banners, and repeated footer text. Preserve source_url, title, and checked_at in front matter, then chunk by headings with a small overlap and review a sample against the page. Markdown reduces structural noise, but it does not make a lossy extraction factual; provenance and spot checks still matter.
Can it extract clean Markdown from Cloudflare-protected or JavaScript-heavy sites?
It can convert a page after it renders in the authorized browser session, including client-rendered documentation that an anonymous HTTP fetch misses. Cloudflare, CAPTCHA, login, paywall, and consent screens remain access controls: when one appears, the agent stops and records the status instead of bypassing it. A visible browser improves reach for pages you are allowed to open; it does not promise access to a protected site.
How can I reduce tokens and hallucinations when an AI extracts web content?
Send the agent only the target URLs and the main-content boundary, ask for conversion rather than a summary, and exclude navigation, ads, and repeated boilerplate. Return the Markdown file and a short status list instead of pasting the entire page back into the conversation. Keep source links and checked-at times so a reviewer can compare a chunk with the live page; no cleaner can guarantee that an ambiguous page was interpreted correctly without that check.
Can I extract structured JSON from a product page without XPath?
Yes, for fields the authorized page visibly exposes. Ask the agent for a JSON schema such as name, SKU, displayed price, currency, availability, and source_url, require null or ‘not displayed’ for missing fields, and keep the rendered page as the evidence source. This avoids hand-written XPath for a one-off task, but it is not a guarantee that hidden metadata or an internal API is available or permitted.
Can a browser extension bulk-export webpages or AI chats to Markdown?
A browser extension can be convenient for one visible page, and an authorized chat or document URL can be included in a bounded ego(lite) conversion run. ego(lite) does not claim a background export API for every ChatGPT, Claude, or Gemini conversation, and it will not crawl your history automatically. List the pages you want, keep each source and access status, and manually review sensitive conversation content before sharing the files.
How do I import a full documentation website into Gemini or Claude?
Choose the specific documentation URLs or a bounded section, convert them to Markdown with headings and absolute links preserved, then upload the resulting files or add them to the assistant's supported knowledge source. A full-site crawl is a different, resource-intensive job; this workflow does not discover links beyond the URLs you provide. Keep the source URL and date in front matter so the assistant's answers can be traced back to the current docs.
Can I export Feishu or Lark cloud documents to Markdown for an AI agent?
If you can open the document in an authorized browser session, add its URL to the same bounded conversion prompt and ask for headings, tables, links, and checked-at metadata to be preserved. The agent stops at a permission dialog, login, or export restriction; it does not use private APIs or copy documents a session cannot access. For an organization-wide export, use Feishu/Lark's official export or API with the workspace administrator's approval.