Get contents

curl -X POST 'https://api.exa.ai/contents' \
  -H 'x-api-key: YOUR-EXA-API-KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "urls": ["https://arxiv.org/abs/2307.06435"],
    "text": true
  }'

{
  "requestId": "e492118ccdedcba5088bfc4357a8a125",
  "results": [
    {
      "title": "A Comprehensive Overview of Large Language Models",
      "url": "https://arxiv.org/pdf/2307.06435.pdf",
      "publishedDate": "2023-11-16T01:36:32.547Z",
      "author": "Humza  Naveed, University of Engineering and Technology (UET), Lahore, Pakistan",
      "id": "https://arxiv.org/abs/2307.06435",
      "image": "https://arxiv.org/pdf/2307.06435.pdf/page_1.png",
      "favicon": "https://arxiv.org/favicon.ico",
      "text": "Abstract Large Language Models (LLMs) have recently demonstrated remarkable capabilities...",
      "highlights": [
        "Such requirements have limited their adoption..."
      ],
      "highlightScores": [
        0.4600165784358978
      ],
      "summary": "This overview paper on Large Language Models (LLMs) highlights key developments...",
      "subpages": [
        {
          "id": "https://arxiv.org/abs/2303.17580",
          "url": "https://arxiv.org/pdf/2303.17580.pdf",
          "title": "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face",
          "author": "Yongliang  Shen, Microsoft Research Asia, Kaitao  Song, Microsoft Research Asia, Xu  Tan, Microsoft Research Asia, Dongsheng  Li, Microsoft Research Asia, Weiming  Lu, Microsoft Research Asia, Yueting  Zhuang, Microsoft Research Asia, [email protected], Zhejiang  University, Microsoft Research Asia, Microsoft  Research, Microsoft Research Asia",
          "publishedDate": "2023-11-16T01:36:20.486Z",
          "text": "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face Date Published: 2023-05-25 Authors: Yongliang Shen, Microsoft Research Asia Kaitao Song, Microsoft Research Asia Xu Tan, Microsoft Research Asia Dongsheng Li, Microsoft Research Asia Weiming Lu, Microsoft Research Asia Yueting Zhuang, Microsoft Research Asia, [email protected] Zhejiang University, Microsoft Research Asia Microsoft Research, Microsoft Research Asia Abstract Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are abundant AI models available for different domains and modalities, they cannot handle complicated AI tasks. Considering large language models (LLMs) have exhibited exceptional ability in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks and language could be a generic interface to empower t",
          "summary": "HuggingGPT is a framework using ChatGPT as a central controller to orchestrate various AI models from Hugging Face to solve complex tasks. ChatGPT plans the task, selects appropriate models based on their descriptions, executes subtasks, and summarizes the results. This approach addresses limitations of LLMs by allowing them to handle multimodal data (vision, speech) and coordinate multiple models for complex tasks, paving the way for more advanced AI systems.",
          "highlights": [
            "2) Recently, some researchers started to investigate the integration of using tools or models in LLMs  ."
          ],
          "highlightScores": [
            0.32679107785224915
          ]
        }
      ],
      "extras": {
        "links": []
      }
    }
  ],
  "context": "<string>",
  "statuses": [
    {
      "id": "https://example.com",
      "status": "success",
      "error": {
        "tag": "CRAWL_NOT_FOUND",
        "httpStatusCode": 404
      }
    }
  ],
  "costDollars": {
    "total": 0.005,
    "breakDown": [
      {
        "search": 0.005,
        "contents": 0,
        "breakdown": {
          "neuralSearch": 0.005,
          "deepSearch": 0.015,
          "contentText": 0,
          "contentHighlight": 0,
          "contentSummary": 0
        }
      }
    ],
    "perRequestPrices": {
      "neuralSearch_1_25_results": 0.005,
      "neuralSearch_26_100_results": 0.025,
      "neuralSearch_100_plus_results": 1,
      "deepSearch_1_25_results": 0.015,
      "deepSearch_26_100_results": 0.075
    },
    "perPagePrices": {
      "contentText": 0.001,
      "contentHighlight": 0.001,
      "contentSummary": 0.001
    }
  }
}

POST

contents

curl -X POST 'https://api.exa.ai/contents' \
  -H 'x-api-key: YOUR-EXA-API-KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "urls": ["https://arxiv.org/abs/2307.06435"],
    "text": true
  }'

{
  "requestId": "e492118ccdedcba5088bfc4357a8a125",
  "results": [
    {
      "title": "A Comprehensive Overview of Large Language Models",
      "url": "https://arxiv.org/pdf/2307.06435.pdf",
      "publishedDate": "2023-11-16T01:36:32.547Z",
      "author": "Humza  Naveed, University of Engineering and Technology (UET), Lahore, Pakistan",
      "id": "https://arxiv.org/abs/2307.06435",
      "image": "https://arxiv.org/pdf/2307.06435.pdf/page_1.png",
      "favicon": "https://arxiv.org/favicon.ico",
      "text": "Abstract Large Language Models (LLMs) have recently demonstrated remarkable capabilities...",
      "highlights": [
        "Such requirements have limited their adoption..."
      ],
      "highlightScores": [
        0.4600165784358978
      ],
      "summary": "This overview paper on Large Language Models (LLMs) highlights key developments...",
      "subpages": [
        {
          "id": "https://arxiv.org/abs/2303.17580",
          "url": "https://arxiv.org/pdf/2303.17580.pdf",
          "title": "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face",
          "author": "Yongliang  Shen, Microsoft Research Asia, Kaitao  Song, Microsoft Research Asia, Xu  Tan, Microsoft Research Asia, Dongsheng  Li, Microsoft Research Asia, Weiming  Lu, Microsoft Research Asia, Yueting  Zhuang, Microsoft Research Asia, [email protected], Zhejiang  University, Microsoft Research Asia, Microsoft  Research, Microsoft Research Asia",
          "publishedDate": "2023-11-16T01:36:20.486Z",
          "text": "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face Date Published: 2023-05-25 Authors: Yongliang Shen, Microsoft Research Asia Kaitao Song, Microsoft Research Asia Xu Tan, Microsoft Research Asia Dongsheng Li, Microsoft Research Asia Weiming Lu, Microsoft Research Asia Yueting Zhuang, Microsoft Research Asia, [email protected] Zhejiang University, Microsoft Research Asia Microsoft Research, Microsoft Research Asia Abstract Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are abundant AI models available for different domains and modalities, they cannot handle complicated AI tasks. Considering large language models (LLMs) have exhibited exceptional ability in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks and language could be a generic interface to empower t",
          "summary": "HuggingGPT is a framework using ChatGPT as a central controller to orchestrate various AI models from Hugging Face to solve complex tasks. ChatGPT plans the task, selects appropriate models based on their descriptions, executes subtasks, and summarizes the results. This approach addresses limitations of LLMs by allowing them to handle multimodal data (vision, speech) and coordinate multiple models for complex tasks, paving the way for more advanced AI systems.",
          "highlights": [
            "2) Recently, some researchers started to investigate the integration of using tools or models in LLMs  ."
          ],
          "highlightScores": [
            0.32679107785224915
          ]
        }
      ],
      "extras": {
        "links": []
      }
    }
  ],
  "context": "<string>",
  "statuses": [
    {
      "id": "https://example.com",
      "status": "success",
      "error": {
        "tag": "CRAWL_NOT_FOUND",
        "httpStatusCode": 404
      }
    }
  ],
  "costDollars": {
    "total": 0.005,
    "breakDown": [
      {
        "search": 0.005,
        "contents": 0,
        "breakdown": {
          "neuralSearch": 0.005,
          "deepSearch": 0.015,
          "contentText": 0,
          "contentHighlight": 0,
          "contentSummary": 0
        }
      }
    ],
    "perRequestPrices": {
      "neuralSearch_1_25_results": 0.005,
      "neuralSearch_26_100_results": 0.025,
      "neuralSearch_100_plus_results": 1,
      "deepSearch_1_25_results": 0.015,
      "deepSearch_26_100_results": 0.075
    },
    "perPagePrices": {
      "contentText": 0.001,
      "contentHighlight": 0.001,
      "contentSummary": 0.001
    }
  }
}

Get your Exa API key

Authorizations

x-api-key

string

header

required

API key can be provided either via x-api-key header or Authorization header with Bearer scheme

Body

application/json

urls

string[]

required

Array of URLs to crawl (backwards compatible with 'ids' parameter).

Example:

["https://arxiv.org/pdf/2307.06435"]

ids

string[]

deprecated

Deprecated - use 'urls' instead. Array of document IDs obtained from searches.

Example:

["https://arxiv.org/pdf/2307.06435"]

text

If true, returns full page text with default settings. If false, disables text return.

highlights

object

Text snippets the LLM identifies as most relevant from each page.

Show child attributes

highlights.numSentences

integer

The number of sentences to return for each snippet.

Required range: x >= 1

Example:

1

highlights.highlightsPerUrl

integer

The number of snippets to return for each result.

Required range: x >= 1

Example:

1

highlights.query

string

Custom query to direct the LLM's selection of highlights.

Example:

"Key advancements"

summary

object

Summary of the webpage

Show child attributes

summary.query

string

Custom query for the LLM-generated summary.

Example:

"Main developments"

summary.schema

object

JSON schema for structured output from summary. See https://json-schema.org/overview/what-is-jsonschema for JSON Schema documentation.

Example:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Title",
  "type": "object",
  "properties": {
    "Property 1": {
      "type": "string",
      "description": "Description"
    },
    "Property 2": {
      "type": "string",
      "enum": ["option 1", "option 2", "option 3"],
      "description": "Description"
    }
  },
  "required": ["Property 1"]
}

livecrawl

enum<string>

Options for livecrawling pages. 'never': Disable livecrawling (default for neural search). 'fallback': Livecrawl when cache is empty. 'always': Always livecrawl. 'preferred': Always try to livecrawl, but fall back to cache if crawling fails.

Available options:

never,

fallback,

always,

preferred

Example:

"always"

livecrawlTimeout

integer

default:10000

The timeout for livecrawling in milliseconds.

Example:

1000

subpages

integer

default:0

The number of subpages to crawl. The actual number crawled may be limited by system constraints.

Example:

1

subpageTarget

Term to find specific subpages of search results. Can be a single string or an array of strings, comma delimited.

Example:

"sources"

extras

object

Extra parameters to pass.

Show child attributes

extras.links

integer

default:0

Number of URLs to return from each webpage.

Example:

1

extras.imageLinks

integer

default:0

Number of images to return for each result.

Example:

1

context

Return page contents as a context string for LLM. When true, combines all result contents into one string. We recommend using 10000+ characters for best results, though no limit works best. Context strings often perform better than highlights for RAG applications.

Example:

true

Response

200 - application/json

requestId

string

Unique identifier for the request

Example:

"e492118ccdedcba5088bfc4357a8a125"

results

object[]

Show child attributes

results.title

string

The title of the search result.

Example:

"A Comprehensive Overview of Large Language Models"

results.url

string<uri>

The URL of the search result.

Example:

"https://arxiv.org/pdf/2307.06435.pdf"

results.publishedDate

string | null

An estimate of the creation date, from parsing HTML content. Format is YYYY-MM-DD.

Example:

"2023-11-16T01:36:32.547Z"

results.author

string | null

If available, the author of the content.

Example:

"Humza Naveed, University of Engineering and Technology (UET), Lahore, Pakistan"

results.id

string

The temporary ID for the document. Useful for /contents endpoint.

Example:

"https://arxiv.org/abs/2307.06435"

results.image

string<uri>

The URL of an image associated with the search result, if available.

Example:

"https://arxiv.org/pdf/2307.06435.pdf/page_1.png"

results.favicon

string<uri>

The URL of the favicon for the search result's domain.

Example:

"https://arxiv.org/favicon.ico"

results.text

string

The full content text of the search result.

Example:

"Abstract Large Language Models (LLMs) have recently demonstrated remarkable capabilities..."

results.highlights

string[]

Array of highlights extracted from the search result content.

Example:

[
  "Such requirements have limited their adoption..."
]

results.highlightScores

number<float>[]

Array of cosine similarity scores for each highlighted

Example:

[0.4600165784358978]

results.summary

string

Summary of the webpage

Example:

"This overview paper on Large Language Models (LLMs) highlights key developments..."

results.subpages

array

Array of subpages for the search result.

Example:

[
  {
    "id": "https://arxiv.org/abs/2303.17580",
    "url": "https://arxiv.org/pdf/2303.17580.pdf",
    "title": "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face",
    "author": "Yongliang  Shen, Microsoft Research Asia, Kaitao  Song, Microsoft Research Asia, Xu  Tan, Microsoft Research Asia, Dongsheng  Li, Microsoft Research Asia, Weiming  Lu, Microsoft Research Asia, Yueting  Zhuang, Microsoft Research Asia, [email protected], Zhejiang  University, Microsoft Research Asia, Microsoft  Research, Microsoft Research Asia",
    "publishedDate": "2023-11-16T01:36:20.486Z",
    "text": "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face Date Published: 2023-05-25 Authors: Yongliang Shen, Microsoft Research Asia Kaitao Song, Microsoft Research Asia Xu Tan, Microsoft Research Asia Dongsheng Li, Microsoft Research Asia Weiming Lu, Microsoft Research Asia Yueting Zhuang, Microsoft Research Asia, [email protected] Zhejiang University, Microsoft Research Asia Microsoft Research, Microsoft Research Asia Abstract Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are abundant AI models available for different domains and modalities, they cannot handle complicated AI tasks. Considering large language models (LLMs) have exhibited exceptional ability in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks and language could be a generic interface to empower t",
    "summary": "HuggingGPT is a framework using ChatGPT as a central controller to orchestrate various AI models from Hugging Face to solve complex tasks. ChatGPT plans the task, selects appropriate models based on their descriptions, executes subtasks, and summarizes the results. This approach addresses limitations of LLMs by allowing them to handle multimodal data (vision, speech) and coordinate multiple models for complex tasks, paving the way for more advanced AI systems.",
    "highlights": [
      "2) Recently, some researchers started to investigate the integration of using tools or models in LLMs  ."
    ],
    "highlightScores": [0.32679107785224915]
  }
]

results.extras

object

Results from extras.

Show child attributes

results.extras.links

string[]

Array of links from the search result.

Example:

[]

context

string

Return page contents as a context string for LLM. When true, combines all result contents into one string. Context strings often perform better than highlights for LLMs.

statuses

object[]

Status information for each requested URL

Show child attributes

statuses.id

string

The URL that was requested

Example:

"https://example.com"

statuses.status

enum<string>

Status of the content fetch operation

Available options:

success,

error

Example:

"success"

statuses.error

object

Error details, only present when status is "error"

Show child attributes

statuses.error.tag

enum<string>

Specific error type

Available options:

CRAWL_NOT_FOUND,

CRAWL_TIMEOUT,

CRAWL_LIVECRAWL_TIMEOUT,

SOURCE_NOT_AVAILABLE,

CRAWL_UNKNOWN_ERROR

Example:

"CRAWL_NOT_FOUND"

statuses.error.httpStatusCode

integer | null

The corresponding HTTP status code

Example:

404

costDollars

object

Show child attributes

costDollars.total

number<float>

Total dollar cost for your request

Example:

0.005

costDollars.breakDown

object[]

Breakdown of costs by operation type

Show child attributes

costDollars.breakDown.search

number<float>

Cost of your search operations

Example:

0.005

costDollars.breakDown.contents

number<float>

Cost of your content operations

Example:

0

costDollars.breakDown.breakdown

object

Show child attributes

costDollars.breakDown.breakdown.neuralSearch

number<float>

Cost of your neural search operations

Example:

0.005

costDollars.breakDown.breakdown.deepSearch

number<float>

Cost of your deep search operations

Example:

0.015

costDollars.breakDown.breakdown.contentText

number<float>

Cost of your text content retrieval

Example:

0

costDollars.breakDown.breakdown.contentHighlight

number<float>

Cost of your highlight generation

Example:

0

costDollars.breakDown.breakdown.contentSummary

number<float>

Cost of your summary generation

Example:

0

costDollars.perRequestPrices

object

Standard price per request for different operations

Show child attributes

costDollars.perRequestPrices.neuralSearch_1_25_results

number<float>

Standard price for neural search with 1-25 results

Example:

0.005

costDollars.perRequestPrices.neuralSearch_26_100_results

number<float>

Standard price for neural search with 26-100 results

Example:

0.025

costDollars.perRequestPrices.neuralSearch_100_plus_results

number<float>

Standard price for neural search with 100+ results

Example:

1

costDollars.perRequestPrices.deepSearch_1_25_results

number<float>

Standard price for deep search with 1-25 results

Example:

0.015

costDollars.perRequestPrices.deepSearch_26_100_results

number<float>

Standard price for deep search with 26-100 results

Example:

0.075

costDollars.perPagePrices

object

Standard price per page for different content operations

Show child attributes

costDollars.perPagePrices.contentText

number<float>

Standard price per page for text content

Example:

0.001

costDollars.perPagePrices.contentHighlight

number<float>

Standard price per page for highlights

Example:

0.001

costDollars.perPagePrices.contentSummary

number<float>

Standard price per page for summaries

Example:

0.001

Search Find similar links

⌘I

Getting Started

API Reference

Concepts

Integrations

Admin

Get contents

Get your Exa API key

Authorizations

Body

Response