Twitter (X) scraping for public post data

Free Twitter scraper for public X posts with your AI agent

Use ego(lite) as a free Twitter scraper tool for public X post data. Send post, thread, profile, or your own X bookmarks page to Codex, Claude Code, or another coding agent: it reads only what your authorized session can already see in a visible browser Space on your Mac and returns a source-linked CSV, Markdown report, or newsletter draft.

Please use /ego-browser to collect the public X post data from and return a source-linked CSV and Markdown report.

Try

Trusted by developers from

How to scrape public X posts or turn bookmarks into a newsletter

Give your agent a bounded list of public X post, thread, or profile URLs—or the X bookmarks page visible in your authorized session—and a clear output contract. ego(lite) supplies the visible browser, a separate Space per task, and the takeover point for anything X wants a human to decide.

1
Install ego(lite) and choose an authorized browser context
Download ego(lite) for Mac, then import only the Chrome context you are authorized to use. A direct link to a public X post may load without signing in, but X's login wall gates scrolling, search, full reply threads, and most profile browsing, so sustained collection uses the session you already control. The agent never enters credentials or works around a control.
2
Send the X scraping prompt to your AI agent
Give your agent a short scraping prompt: the public X post, thread, profile, or bookmarks page, the fields you need, and CSV, Markdown, or newsletter draft as the output format. Keep the rule that the agent stops when X asks for login, verification, or another human decision.
3
Watch each post being read on screen
Your agent opens each source in its own ego(lite) Space and records the post text, timestamps, and engagement counts the page actually displays. For a bookmarks run, it reads only the items visible in your own authorized bookmarks page. Nothing happens off screen: open the Space at any point and you can see exactly which source an item came from.
4
Run several X research tasks in parallel Spaces
Give each profile, thread, or comparison its own Space. Independent collections run side by side without mixing browser state, and your own tabs stay untouched while the agent works. Open any Space to check progress or pause one task.
5
Review the source-linked report and flag anything odd
The agent returns a CSV or Markdown report—or a newsletter draft—where every row keeps its post URL, checked-at time, and access status. Fields X hid or did not display stay marked unavailable, and every draft claim keeps its source link for your review.

Try this X scraping workflow for free

Why use ego(lite) as your Twitter scraper?

Most tweet scrapers run in a hosted cloud you never see, on accounts and proxies you do not control. ego(lite) keeps the browser visible, the tasks separate, and the report tied to its sources. It also stops at X's controls instead of pretending they do not exist.

Finish browser tasks 52% faster with ego(lite)

Use the same coding agent for the collection and the analysis that follows. It reads each public post in a real browser and returns one source-linked report you can hand to a teammate. Across our 31-task benchmark on live sites, ego(lite) averaged 398 seconds per task against 605 for agent-browser, 52% faster. Actual timing varies by website, workflow, and network conditions.

Run X research tasks in parallel

Give each profile, thread, or comparison its own ego(lite) Space. Collections run side by side without mixing browser state, and you can return to the exact post where a count was visible or where X asked for your attention.

Watch any Space and take over at the login wall

Open a Space to see which post the agent is reading. When X shows its login wall, a verification step, or a consent prompt, the agent stops, records the status, and hands the browser back to you instead of trying to slip past the control.

Use only a browser context you authorize

Import the Chrome context you choose instead of lending your account to a scraping service. What the agent can see matches what that browser session can see, and X's login, privacy, and rate-limit rules stay in effect throughout.

ego (lite) Chrome context import for an authorized X browser session

What this X scraper can and cannot collect

X's terms prohibit scraping without written permission at any scale, and since late 2024 they set liquidated damages for bulk access. This workflow does not change what X permits. It stays deliberately small by using a bounded list, your own authorized session, and only what the page visibly displays. It organizes public post data; it does not unlock anything.

What your agent can record

Public post data the current authorized session visibly displays.

Post text, author handle and display name, and the post URL for public posts your session can open
The visible items in your own authorized X bookmarks page, with each original post URL retained
Displayed engagement counts, including replies, reposts, quotes, likes, bookmarks, and views, recorded exactly as X formats them
Public reply threads, profile timelines, and the public image or video URLs loaded by each post
A source-linked CSV or Markdown report with checked-at time and access status on every row

What stays out of scope

No bypass, no protected content, no bulk crawling, no engagement actions.

Protected accounts, deleted posts, or fields the current session cannot visibly open
Another person's bookmarks or hidden bookmark items, and the identities of people who liked or bookmarked a post
Bypassing the login wall, CAPTCHA, verification, rate limits, or blocks, or signing in on its own
Bulk or unbounded crawling, since the run stays limited to the source list you provide and moves at a supervised pace
Liking, reposting, replying, following, posting, or any other engagement action
Automatically sending, subscribing, or publishing a newsletter; the output is a draft for your review

Read X's terms of service

Scrape public X post data you can verify later

Give your agent the public post, profile, or authorized bookmarks source and the fields you need. ego(lite) turns the run into a source-linked report or newsletter draft with visible collection, separate Spaces, and a stopping point at every control X puts up. Review the draft yourself before sending it.

Try the free Twitter/X scraper

Twitter/X scraper FAQ

What is a Twitter scraper?

A Twitter scraper, sometimes called a tweet scraper, is a tool or workflow that collects selected data from X (formerly Twitter) pages into a structured result such as a CSV. Most Twitter scrapers run in a hosted cloud on accounts you never see. With ego(lite), your own coding agent does the reading in a visible browser on your Mac, records only what the public page displays in your authorized session, and keeps the source URL and check time beside every row.

How do I scrape tweets from a public profile or thread?

Install ego(lite), then give your AI agent a short prompt with the public profile or post URLs, the fields you need, and the rule to stop at any login or verification step. The agent opens each URL in its own ego(lite) Space, records the visible post text, timestamps, and engagement counts, and returns a CSV or Markdown report. You can watch the Space while it works and take over whenever X asks for a human decision.

Is there a free Twitter scraper that still works?

The free command-line scrapers many people remember stopped working when X removed guest access in 2023, and most tools now advertising "free" resolve to short trials or small credit packs. ego(lite) is free to download for Mac, and the work is done by the coding agent you already use, so the only cost is your agent's tokens. The trade-off is scale: it is built for bounded, supervised collection, not industrial crawling.

Is ego(lite) a Twitter scraper API?

No. ego(lite) is a local Mac app, not a hosted Twitter scraper API. There are no endpoints, request quotas, or per-call billing. If you need to embed X data into production software at volume, X's own paid API is the built-for-purpose category, and X's scraping terms apply to hosted scraper vendors just as they do here. ego(lite) fits the other job: personal-scale research where you want to see the browser doing the work and verify every row.

Do I need X API keys, proxies, or a scraping subscription?

No. The agent works inside a browser session you already control, so there is nothing to apply for and no proxy pool to rent. That matters because X's API no longer has a free tier for reading posts. Programmatic read access is paid at every level. This workflow reads only what your own session can already display, which is also why it cannot go beyond that session's access.

Can I use it as a Twitter media scraper for images and video?

It can record references to the images and video visible on the public posts in your list. These are the public URLs that the browser loads from each post, so your report keeps a link to the original media. Media remains the copyright of whoever posted it. Keeping a reference for research is different from republishing, and the workflow does not bulk-harvest media libraries or open anything your session cannot see.

Is it legal to scrape Twitter (X)?

X's Terms of Service prohibit scraping without written permission, and since late 2024 they include a liquidated-damages clause for anyone viewing or accessing more than a million posts in a day. U.S. courts have separately declined to treat viewing public web data as computer intrusion in cases like hiQ v. LinkedIn and X Corp. v. Bright Data, but those rulings do not erase contract terms for account holders. This is not legal advice: keep collection small, public, and supervised, and you are responsible for how you use what you collect.

Can it scrape Twitter (X) without logging in?

Only barely, and not reliably. A direct link to a single public post usually loads while logged out, but scrolling, search, full reply threads, and most profile browsing hit X's login wall. This workflow therefore runs in the signed-in browser context you authorize and reads only what that session can already see. The agent does not create accounts or bypass the wall.

Will using a Twitter scraper get my X account banned?

This workflow is designed to lower that risk rather than pretend it away: it works at a visible, supervised pace on a bounded list, never bypasses the login wall, CAPTCHA, verification, or rate limits, and stops for every checkpoint so you decide what happens. We cannot predict how X's rules or enforcement will change, which is one more reason to keep runs small and reviewable.

How much data can one run collect?

A run is bounded by the source list you provide. Think hundreds of source-linked rows in a supervised session, not millions of posts. ego(lite) is not an unbounded crawler, and X's terms set explicit damages for bulk access. If your project genuinely needs millions of records, this is honestly the wrong tool; this page's workflow is for research you can read, check, and stand behind.

What does the export look like?

A CSV or Markdown file where each row is one post: its visible text, author, timestamp, displayed engagement counts, and media references, plus source_url, checked_at, and access_status. Anything X hid or did not render stays marked unavailable with the reason. Because collection happened in a visible browser, any row can be spot-checked by opening its source URL again.

Can I turn my X bookmarks into a newsletter?

Yes—if your authorized browser session can open your X bookmarks page, the agent can organize the visible bookmarked posts into a Markdown or CSV-backed newsletter draft. Each item keeps its original URL, displayed author and date, checked-at time, and a claim list so you can verify the context. This is a reviewable draft workflow, not an automatic newsletter sender: it does not read someone else's bookmarks, bypass login or verification, or send or publish anything. X may still require you to log in or complete a human checkpoint.

Can I use a Twitter scraper for lead generation?

Yes, for a bounded list of public posts, profiles, or threads that your authorized session can open. Ask the agent to capture a public author handle, post URL, date, visible text, and the signal you want to qualify, then export a source-linked CSV for human review. Do not infer private attributes, scrape beyond the list, or auto-message people; treat the output as research leads rather than permission to contact anyone.

What can I use after Nitter shut down?

There is no dependable universal Nitter replacement. For a single public post, the canonical X URL may work while logged out; profiles, search, replies, and scrolling commonly require an X session. A read-only RSS mirror, official API, licensed dataset, or your own authorized browser session can be appropriate depending on the use case. ego(lite) is the last option: it gives your agent a visible local browser and records only what that session displays, stopping at login or verification.

How do I avoid blocks and rate limits when collecting X data?

You cannot guarantee that a scraper will avoid X blocks, and this workflow does not try to evade them. Keep the source list bounded, use a conservative pace, cache results, back off on 429 responses, and stop at login, CAPTCHA, verification, or consent. If your volume or freshness requirement exceeds what X permits, use its supported API, export, or a licensed provider instead of adding proxies, rotating identities, or replaying cookies.

Can it collect a full timeline, replies, media, or historical trends?

It can collect the posts, visible replies, and media references that the authorized browser session actually loads for the bounded sources you provide. Infinite timelines and historical trend archives are not guaranteed: X may paginate, hide older items, require login, or change the interface. The report marks unavailable fields, keeps each source URL and checked-at time, and does not pretend a partial view is a complete historical dataset.

Can I automate replies, DMs, or social engagement with this scraper?

No. This page's workflow is read-only: it can find and summarize relevant public posts, but it does not like, follow, reply, post, send DMs, or publish on your behalf. You may use the source-linked report to draft a thoughtful response for human approval, then perform the action yourself through X's normal interface and policies. Automation that tries to avoid bans or disguise a bot is outside scope.

Is there a free Twitter scraper without a paid API?

ego(lite) is free to download for Mac and needs no X API key or hosted scraper subscription for a bounded, supervised run. The trade-off is deliberate: your agent reads a visible browser session, so collection is limited by what that session can display and by X's rules. It is not a free replacement for a high-volume API or an unbounded crawler, and your agent's model usage may still have a cost.