What does the website data extractor do?
Extract Data from Website Pages reads a site from the URL you give it and returns JSON shaped by a schema you define. You list the fields, such as plan name, price and features, and every item the pages show comes back as one record with those keys, so ten competitor sites give you ten sets of identical columns.
You do not write CSS selectors or parsing rules. The extractor reads the page text and fills your fields, so there are no selectors to fix after a site redesign. With fact checking on, a value the page does not state comes back empty instead of guessed.
You ask
Extract every plan from https://github.com/pricing with its price, billing unit, tagline, button label and top five features.
It runs
From your AI assistant over MCP or from your code over REST
You get
- Your fields
- From our pricing test
- Crawl report
- Freshness
- Track competitor pricingPull plan names, prices and billing units from pricing pages into the same columns every month.
- Build product recordsTurn product and spec pages into rows with name, price, sizes and materials for a catalog or a comparison.
- Compare features side by sideCollect the feature list of each plan or product and see which ones only one vendor offers.
- Research companiesRead company sites for headquarters city, product lines and pricing model and file them in one sheet.
- Give agents typed dataHand Claude or ChatGPT JSON with fixed keys, so it compares values instead of re-reading prose.
What data can you extract from a website?
You decide the fields. Each run returns your records plus a short report on what the crawl did. These are the parts of the response and the kinds of fields a schema can hold.
Your fields
- Any field you name in the schema
- Text, numbers and true or false values
- Lists, such as features or specs
- Nested records, such as plans with their features
- Required fields you mark in the schema
From our pricing test
- Plan name
- Price as written on the page
- Billing unit
- Tagline
- Button label
- Feature list
Crawl report
- URLs the extractor analyzed
- Maximum link depth
- Pages succeeded, failed, skipped and blocked
Freshness
- Fresh read or cached result
- Age of a cached result
A record holds only what the pages state. If the start page lists prices but the features sit on another page, allow a link depth of 1 or more so the extractor can reach it. When you want the whole text of a page rather than chosen fields, use URL to Markdown.
Sample output from a real pricing page
The table at the top of this page is a real run on 16 November 2026: the GitHub pricing page, read as a single page with fact checking on and no cached copy. We asked for six fields per plan and the extractor returned all three plans the page shows. Below is the Team plan as one full record.
One result in full
One record in full: Team
- plan_name
- Team
- price
- $4USD
- price_unit
- per user / month
- tagline
- Advanced collaboration for individuals and organizations
- call_to_action
- Continue with Team
- key_features
- Access to GitHub Codespaces; Repository rules; Multiple reviewers in pull requests; Draft pull requests; Code owners
What this run tells you
All 3 plans came back with all 6 fields filled and 5 features each, so no row needs a second pass.
Prices came back as text with the currency attached, “$0USD”, “$4USD” and “$21USD”. Convert them to numbers before you sort or chart them.
The billing unit landed in its own field: “per user / month” for 2 plans and “/ month” for the $0 plan. Ask for the unit separately so you never compare a per-user price with a flat one.
The crawl report showed 1 page analyzed, 1 succeeded and none failed, skipped or blocked, and the result was a fresh read rather than a cached one.
How to extract data from a website with your AI assistant
Connect AI/ML API
Click Try it now and add the MCP server to Claude, ChatGPT, Cursor or another client. You sign in with your AI/ML API account once.
Name the page and the fields
Give the URL and list the values you want per item, in plain words. Your assistant turns the list into a schema.
Check the records
You get one row per plan, product or listing. Open the page once and compare two rows before you trust the rest.
Reuse the field list
Run the same fields on competitor pages to get identical columns, then ask for a CSV or a comparison.
Prompts to start with
Use the AI/ML API MCP Extract Data from Website Pages tool. Extract every plan from https://github.com/pricing with its price, billing unit, tagline, button label and top five features.
Use the AI/ML API MCP Extract Data from Website Pages tool. From https://www.python.org/downloads/, extract each Python version listed as maintained with its status and end of support date.
Use the AI/ML API MCP Extract Data from Website Pages tool. Read https://slack.com/pricing and https://www.notion.com/pricing and return plan name, monthly price and billing unit for each in one table.
Use the AI/ML API MCP Extract Data from Website Pages tool. Extract the title, price, star rating and stock status of every book on https://books.toscrape.com/, start page only.
Extraction settings
Two settings are required: the start URL and the schema. The rest control how far the extractor reads and how strictly it fills fields.
| Setting | What it controls | Tip |
|---|---|---|
| Website URL | The page where reading starts | Start on the page that lists the items, such as /pricing |
| Schema | A JSON Schema with the fields and types you want back | In chat, list the fields in words and your assistant writes it |
| Instructions | Up to 2,000 characters on what to prioritize and how to read unclear values | “Use prices exactly as written” keeps currency and format |
| Pages and depth | How many pages to read, 5 by default and 50 at most, and how many links away from the start page | Depth 0 reads the start page only, which suits a single pricing page |
| Fact check | When on, every value must be stated on the page and missing fields come back empty | Turn it on for prices and numbers you will publish |
| Cache age | Reuse a result younger than this, one day by default and 30 days at most | Set it to 0 when the page changes often |
| Wait and time limits | Extra wait after load, up to 30 seconds, and a crawl budget of 10 to 110 seconds | Add a wait for pages that draw prices with JavaScript |
Split values that mean different things. A single “price” field that sometimes holds “$4 per user” and sometimes “contact sales” is harder to compare than a price field plus a unit field.
Ways teams use extracted website data
Competitor price tracking
Run one field list on every competitor's pricing page each month and keep the records to see who changed what.
PlanPriceBilling unit
Product catalog records
Turn shop or spec pages into rows with name, price and attributes for a comparison site or an internal catalog.
NamePriceAttributes
Feature matrices
Collect feature lists per plan from several vendors and let your assistant build a yes or no matrix.
PlanFeatures
Company research
Read company websites for the facts your brief needs, such as product lines, locations and how they sell.
Any field the site states
Hiring and roles
List open roles with team and location from careers pages to see where a company is growing.
TitleTeamLocation
Structured input for agents
Give an agent typed JSON with fixed keys so its next step compares values instead of re-reading a page.
Your schema
Website data extraction API for your code
In your code
REST · your AI/ML API key, JSON records back
The same extractor runs over REST with your AI/ML API key. Send the start URL and a JSON Schema, which you can generate from a Pydantic or Zod model, and the response holds your data in that shape plus the crawl report. Because the keys are yours, the result goes straight into your database or spreadsheet without a mapping step.
Use MCP when someone explores a page in chat and REST when a job runs the same schema across a list of sites. Both share one AI/ML API account and balance. Start with the Toolbox REST quickstart.
Other ways to pull data from a website
Every method below can get the plan prices from a pricing page. They differ in setup, in what you get back and in what happens when the site changes its layout.
| Copy values into a spreadsheet | Your own scraper with CSS selectors | Paste the page into a chatbot | ||
|---|---|---|---|---|
| Setup | Connect once, describe the fields | None | Code, selectors per site, fixes when markup changes | None |
| What you get | JSON with your keys, from chat or code | Whatever you typed | Whatever your code emits | A table whose columns can change between runs |
| Best for | Repeat extractions across many sites | A few values from one page | High-volume jobs on sites you know well | One-off questions about a page |
Pricing
Pay as you goBilled per run from your AI/ML API balance
- No subscription
- No seat fee
- One balance for every tool
Each extraction is billed per run from your AI/ML API balance, with no subscription. The number of pages a run reads does not change its price. Our one-page pricing test cost less than three cents. Check the current price in your dashboard before you schedule large batches.
FAQ
Is it legal to extract data from websites?
Reading public pages such as pricing or product listings is common for market research. The site's terms, copyright and database rights still apply, and privacy laws apply if you collect personal data. Ask a lawyer about your specific use.
How many pages does one run read?
Five by default and up to 50, following links as deep as you allow. Depth 0 keeps the run on the start page, which is what we used for the pricing sample.
What happens when a field is not on the page?
With fact checking on, the field comes back empty. With it off, the extractor may fill a reasonable inference while keeping checkable specifics such as prices faithful to the page. Turn it on whenever a wrong value would cost you.
Do I have to write JSON Schema myself?
Not in chat. List the fields in plain words and your assistant writes the schema. In code you send JSON Schema, which most languages can generate from a typed model.
How is this different from a site's official API?
An official API returns the data model the company chose to publish, with its own keys and limits. Most pricing and product pages have no API at all. This tool reads what a visitor sees on the page and returns it in the fields you defined.
Can I use it from Claude, ChatGPT or Cursor?
Yes. Add the AI/ML API MCP server once, then ask for fields from a URL. Any MCP client works the same way.
Can I export the records to CSV or Google Sheets?
Ask your assistant for the records as CSV, or write the JSON from the REST response to a file or a sheet. Because every record has the same keys, the columns line up across runs.
How fresh is the data?
By default a result up to one day old can be served from cache. Set the cache age to 0 to read the live page, as we did in the test, and the response tells you whether it was a fresh read.
Does it work on pages built with JavaScript?
Add an extra wait after load, up to 30 seconds, and the extractor reads the page once scripts have drawn the content. You can also have it include iframe contents and let animations settle first.
Extract your first records
Connect once, then name a page and the fields you need.