Crawling Subpages with Exa

results = exa.get_contents([company_url], subpages=5, subpage_target=["about", "products"])

result = exa.get_contents(
  ["https://exa.ai"],
  subpages=9,
  subpage_target=["docs", "tutorial"]
)

{
  "results": [
    {
      "id": "https://exa.ai",
      "url": "https://exa.ai/",
      "title": "Exa API",
      "author": "exa",
      "text": "AIs need powerful access to knowledge. But search engines haven't improved since 1998.         API features built for AI apps         FLEXIBLE SEARCH Handle any type of query For queries that need semantic understanding, search with our SOTA web embeddings model\nover our custom index. For all other queries, we offer keyword-based search.       PAGE CONTENT Instantly retrieve the content from any page Stop learning how to web scrape or parse HTML. Get the clean, full text of any page in our\nindex, or intelligent embeddings-ranked\nhighlights related to a query.       CUSTOMIZABILITY Apply filters to search over your own subset of the web Select any date range, include or exclude any domain, select a custom data vertical, or get up to 10 million results.       SIMILARITY SEARCH Search the web using examples of what you’re looking for Your query can even be a URL or multiple paragraphs on a webpage.      Explore the documentation",
      "image": "https://exa.imgix.net/og-image.png",
      "subpages": [
        {
          "id": "https://docs.exa.ai/reference/getting-started",
          "url": "https://docs.exa.ai/reference/getting-started",
          "title": "Getting Started",
          "author": "",
          "text": "Exa provides search for AI.  \n       Exa is a knowledge API for LLMs..\n Search\nUsing the /search endpoint, your LLM can search using natural language and return a list of relevant webpages from our neural database. Exa's neural search allows your LLM to query in natural language. And if a query doesn't benefit from neural search, Exa also supports traditional Google-style keyword search.\n Contents\nRight alongside the search results, you can obtain clean, up-to-date HTML content, no web crawling or scraping required. We also support returning highlights from pages - highly intelligent extracts calculated using RAG models.\nSee \"Examples\" to see Exa in action. Or just ask one of our GPTs how to build what you want!\n  🔑 Getting a Exa API Key  Exa is free to use up to 1000 requests per month, for individual developers. Get an API key here. \nThere's no need to learn how to use our API all alone. Below are ChatGPT GPTs that you can ask about how to implement anything.\nFor Python SDK assistance, go here.\nFor Typescript SDK assistance, go here.\nFor any other language, go here.\n  # pip install exa_py\nfrom exa_py import Exa\nexa = Exa(\"EXA_API_KEY\")\nresults = exa.search('hottest AI agent startups', use_autoprompt=True)\n  \n Recent News Summarizer \n Exa Researcher \n Basic search \n Basic link similarity \n Basic content retrieval \n         Table of Contents   \n What is Exa? \n Getting Access \n GPT-assisted implementation \n A simple example \n More examples"
        },
        {
          "id": "https://docs.exa.ai/reference/recent-news-summarizer",
          "url": "https://docs.exa.ai/reference/recent-news-summarizer",
          "title": "Recent News Summarizer",
          "author": null,
          "publishedDate": "2024-03-02T11:36:31.000Z",
          "text": "https://docs.exa.ai/reference/recent-news-summarizer\nRecent News Summarizer\n2024-03-02T11:36:31Z\n   \nIn this example, we will build a LLM-based news summarizer app with the Exa API to keep us up-to-date with the latest news on a given topic.\nThis Jupyter notebook is available on Colab for easy experimentation. You can also check it out on Github, including a plain Python version if you want to skip to a complete product.\nTo play with this code, first we need a Exa API key and an OpenAI API key. Get 1000 Exa searches per month free just for signing up!\n  # install Exa and OpenAI SDKs\n!pip install exa_py\n!pip install openai\n  \n  from google.colab import userdata # comment this out if you're not using Colab\nEXA_API_KEY = userdata.get('EXA_API_KEY') # replace with your api key, or add to Colab Secrets\nOPENAI_API_KEY = userdata.get('OPENAI_API_KEY') # replace with your api key, or add to Colab Secrets\n  \nFirst, let's try building the app just by using the OpenAI API. We will use GPT 3.5-turbo as our LLM. Let's ask it for the recent news, like we might ask ChatGPT.\n  import openai\nopenai.api_key = OPENAI_API_KEY\nUSER_QUESTION = "What's the recent news in physics this week?"\ncompletion = openai.chat.completions.create(\nmodel="gpt-3.5-turbo",\nmessages=[\n{"role": "system", "content": "You are a helpful assistant."},\n{"role": "user", "content": USER_QUESTION},\n],\n)\nresponse = completion.choices[0].message.content\nprint(response)\n  \n One recent news in physics is that researchers at the University of Illinois have discovered a new topological state of matter. They created a material composed of interacting particles called quadrupoles, which can exhibit unique behavior in their electrical properties. This finding has the potential to pave the way for the development of new types of electronic devices and quantum computers.\nAnother interesting development is in the field of cosmology. The European Space Agency's Planck satellite has provided new insights into the early universe. By analyzing the cosmic microwave background radiation, scientists have obtained more accurate measurements of the rate at which the universe is expanding, which could challenge current theories of physics.\nAdditionally, scientists at CERN's Large Hadron Collider (LHC) have observed a rare phenomenon called charm mixing. They found that particles containing both charm and strange quarks can spontaneously transition between their matter and antimatter states. This discovery could contribute to our understanding of the puzzle of why the universe is primarily made of matter and why there is very little antimatter.\n \nOh no! Since the LLM is unable to use recent data, it doesn't know the latest news. It might tell us some information, but that info isn't recent, and we can't be sure it's trustworthy either since it has no source. Luckily, Exa API allows us to solve these problems by connecting our LLM app to the internet. Here's how:\nLet's use the Exa neural search engine to search the web for relevant links to the user's question.\nFirst, we ask the LLM to generate a search engine query based on the question.\n  import openai\nfrom exa_py import Exa\nopenai.api_key = OPENAI_API_KEY\nexa = Exa(EXA_API_KEY)\nSYSTEM_MESSAGE = "You are a helpful assistant that generates search queries based on user questions. Only generate one search query."\nUSER_QUESTION = "What's the recent news in physics this week?"\ncompletion = openai.chat.completions.create(\nmodel="gpt-3.5-turbo",\nmessages=[\n{"role": "system", "content": SYSTEM_MESSAGE},\n{"role": "user", "content": USER_QUESTION},\n],\n)\nsearch_query = completion.choices[0].message.content\nprint("Search query:")\nprint(search_query)\n  \n Search query:\nRecent news in physics this week\n \nLooks good! Now let's put the search query into Exa. Let's also use start_published_date to filter the results to pages published in the last week:\n  from datetime import datetime, timedelta\none_week_ago = (datetime.now() - timedelta(days=7))\ndate_cutoff = one_week_ago.strftime("%Y-%m-%d")\nsearch_response = exa.search_and_contents(\nsearch_query, use_autoprompt=True, start_published_date=date_cutoff\n)\nurls = [result.url for result in search_response.results]\nprint("URLs:")\nfor url in urls:\nprint(url)\n  \n URLs:\nhttps://phys.org/news/2024-01-carrots-reveals-mechanics-root-vegetable.html?utm_source=twitter.com&utm_medium=social&utm_campaign=v2\nhttps://phys.org/news/2024-01-astrophysicists-theoretical-proof-traversable-wormholes.html\nhttps://gizmodo.com/proton-physics-strong-force-quarks-measurement-1851192840\nhttps://www.nytimes.com/2024/01/24/science/space/black-holes-photography-m87.html\nhttps://phys.org/news/2024-01-liquid-lithium-walls-fusion-device.html?utm_source=twitter.com&utm_medium=social&utm_campaign=v2\nhttps://physics.aps.org/articles/v17/s13\nhttps://phys.org/news/2024-01-validating-hypothesis-complex.html?utm_source=twitter.com&utm_medium=social&utm_campaign=v2\nhttps://phys.org/news/2024-01-scientists-previously-unknown-colonies-emperor.html?utm_source=twitter.com&utm_medium=social&utm_campaign=v2\nhttps://phys.org/news/2024-01-reveals-quantum-topological-potential-material.html?utm_source=twitter.com&utm_medium=social&utm_campaign=v2\nhttps://phys.org/news/2024-01-shallow-soda-lakes-cradles-life.html?utm_source=twitter.com&utm_medium=social&utm_campaign=v2\n \nNow we're getting somewhere! Exa gave our app a list of relevant, useful URLs based on the original question.\nBy the way, we might be wondering what makes Exa special. Why can't we just search with Google? Well, let's take a look for ourselves at the Google search results. It gives us the front page of lots of news aggregators, but not the news articles themselves. And since we used Exa's search_and_contents, our search came with the webpage contents, so can use Exa to skip writing a web crawler and access the knowledge directly!\n  results = search_response.results\nresult_item = results[0]\nprint(f"{len(results)} items total, printing the first one:")\nprint(result_item.text)\n  \n 10 items total, printing the first one:\nCredit: CC0 Public Domain\nChopped carrot pieces are among the most universally enjoyed foods and a snacking staple—a mainstay of school lunchboxes, picnics and party platters year-round.\nNow researchers from the University of Bath have uncovered the secret science of prepping the popular root vegetable and quantified the processes that make them curl up if left uneaten for too long.\nMechanical Engineering student Nguyen Vo-Bui carried out the research as part of his final-year studies, in the limited circumstances of COVID-19 lockdowns of 2021. The research paper, "Modelling of longitudinally cut carrot curling induced by the vascular cylinder-cortex interference pressure", is published in Royal Society Open Science.\nWithout access to labs, Nguyen aimed to identify the geometrical and environmental factors that have the most influence on carrots' longevity. Working in his kitchen, he characterized, analytically modeled and verified the aging of over 100 Lancashire Nantes carrot halves, cut lengthways, using finite-element (FE) models normally used in structural engineering.\nThe research team concluded that residual stresses and dehydration were the two key factors behind the curling behavior. The starchy outer layer of the carrot (the cortex) is stiffer than the soft central vein (also known as the vascular cylinder). When cut lengthwise, the two carrot halves curl because the difference in stress becomes unbalanced. Dehydration leads to further loss of stiffness, further driving the curling effect.\nTheir recommendations to manufacturers include handling carrots in cold, moist, airtight and humidity-controlled environments to protect their natural properties and increase their edible life span.\nThey say the study provides a methodology to predict the deformation of cut root vegetables, adding that the procedure is likely to apply to other plant structures. The study gives food producers a new mathematical tool that could be applied to the design of packaging and food handling processes, potentially reducing food waste.\nOne of the world's top crops by market value, carrots are known for their high production efficiency—but despite this, wastage is high. Around 25–30% of this occurs prior to processing and packaging—due to deformities, mechanical damage or infected sections. Fresh cut and minimally processed carrots are a convenient ready-to-use ingredient that make possible the use of carrots that might otherwise be discarded, reducing food waste.\nDr. Elise Pegg, a senior lecturer in Bath's Department of Mechanical Engineering, is one of the research paper authors and oversaw the study. She said, "We have mathematically represented the curl of a cut carrot over time, and showed the factors that contribute to curling.\n"Our motivation was to look for ways to improve the sustainability of carrot processing and make them as long-lasting as possible. We have produced a methodology that a food producer could use to change their processes, reducing food waste and making packaging and transportation more efficient. Understanding the bending behavior in such systems can help us to design and manufacture products with higher durability.\n"A question like this would normally be investigated from a biological perspective, but we have done this work using purely mechanical principles. I'm so pleased for Nguyen—it's a measure of his resourcefulness and dedication to produce such interesting research in a challenging situation."\nOver the course of a week, the curl of the carrot halves increased—with the average radius of each carrot's curvature falling from 1.61m to 1.1m. A 1.32-times reduction in stiffness was also seen, correlating with the carrots drying out; on average, their weight fell by 22%.\nNguyen added, "This was interesting research—to apply mechanical principles to vegetables was surprising and fun.\n"One of the big challenges was to devise an experiment that could be done in a lockdown setting, without access to normal labs and equipment. To now be in a position to have this work published in an academic journal and potentially be used by the food industry is really rewarding.\n"This project has inspired me to continue my studies at the University of Bath and I now study residual stresses in porous ferroelectric ceramics for my Ph.D."\nAs well as having to use a suitcase to collect the 30kg of carrots the experiment demanded from a farmers' market, a further challenge was finding ways to use them afterward. Carrot cake, the Indian carrot dessert Gajar Ka Halwa, carrot pesto and many other dishes kept Nguyen and his flatmates fed for several days.\nMore information:\nModelling of longitudinally cut carrot curling induced by the vascular cylinder-cortex interference pressure, Royal Society Open Science (2024). DOI: 10.1098/rsos.230420. royalsocietypublishing.org/doi/10.1098/rsos.230420\nCitation:\nWhy do carrots curl? Research reveals the mechanics behind root vegetable aging (2024, January 23)\nretrieved 24 January 2024\nfrom https://phys.org/news/2024-01-carrots-reveals-mechanics-root-vegetable.html\nThis document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no\npart may be reproduced without the written permission. The content is provided for information purposes only.\n \nAwesome! That's really interesting, or it would be if we had bothered to read it all. But there's no way we're doing that, so let's ask the LLM to summarize it for us:\n  import textwrap\nSYSTEM_MESSAGE = "You are a helpful assistant that briefly summarizes the content of a webpage. Summarize the users input."\ncompletion = openai.chat.completions.create(\nmodel="gpt-3.5-turbo",\nmessages=[\n{"role": "system", "content": SYSTEM_MESSAGE},\n{"role": "user", "content": result_item.text},\n],\n)\nsummary = completion.choices[0].message.content\nprint(f"Summary for {urls[0]}:")\nprint(result_item.title)\nprint(textwrap.fill(summary, 80))\n  \n Summary for https://phys.org/news/2024-01-carrots-reveals-mechanics-root-vegetable.html?utm_source=twitter.com&utm_medium=social&utm_campaign=v2:\nWhy do carrots curl? Research reveals the mechanics behind root vegetable aging\nResearchers from the University of Bath have conducted a study on the curling\nbehavior of chopped carrot pieces. The study found that residual stresses and\ndehydration were the main factors behind the curling effect. The starchy outer\nlayer of the carrot is stiffer than the soft central vein, and when cut\nlengthwise, the difference in stress causes the carrot to curl. Dehydration\nfurther contributes to the curling effect. The research provides recommendations\nto manufacturers on how to handle carrots to increase their edible lifespan. The\nstudy also offers a methodology that can be used to predict the deformation of\ncut root vegetables and potentially reduce food waste. The findings have\nimplications for the design of packaging and food handling processes. Carrots\nare a highly produced crop, but wastage is still high, with a significant amount\noccurring before processing and packaging. The study was carried out by\nMechanical Engineering student Nguyen Vo-Bui during the COVID-19 lockdowns of\n2021.\n \nAnd we're done! We built an app that translates a question into a search query, uses Exa to search for useful links, uses Exa to grab clean content from those links, and summarizes the content to effortlessly answer your question about the latest news, or whatever we want.\nWe can be sure that the information is fresh, we have the source in front of us, and we did all this with a Exa queries and LLM calls, no web scraping or crawling needed!\n With Exa, we have empowered our LLM application with the Internet. The possibilities are endless."
        },
        {
          "id": "https://docs.exa.ai/reference/company-analyst",
          "url": "https://docs.exa.ai/reference/company-analyst",
          "title": "Company Analyst",
          "author": null,
          "publishedDate": "2024-03-02T11:36:42.000Z",
          "text": "https://docs.exa.ai/reference/company-analyst\nCompany Analyst\n2024-03-02T11:36:42Z\n    In this example, we'll build a company analyst tool that researches companies relevant to what you're interested in. If you just want to see the code, check out the Colab notebook.\nThe code requires an Exa API key and an OpenAI API key. Get 1000 Exa searches per month free just for signing up!\nSay we want to find companies similar to Thrifthouse, a platform for selling secondhand goods on college campuses. Unfortunately, googling “companies similar to Thrifthouse” doesn't do a very good job. Traditional search engines rely heavily on keyword matching. In this case we get results about physical thrift stores. Hm, that's not really what I want.\nLet’s try again, this time searching based on a description of the company, like by googling “community based resale apps.” But, this isn’t very helpful either and just returns premade listicles...\n      What we really need is neural search.\nExa is a fully neural search engine built using a foundational embeddings model trained for webpage retrieval. It’s capable of understanding entity types (company, blog post, Github repo), descriptors (funny, scholastic, authoritative), and any other semantic qualities inside of a query. Neural search can be far more useful for these types of complex queries.\nLet's try Exa, using the Python SDK! We can use thefind_similar_and_contents function to find similar links and get contents from each link. The input is simply a URL, https://thrift.house and we set num_results=10(this is customizable up to thousands of results in Exa).\nBy specifying highlights={"num_sentences":2} for each search result, Exa will also identify and return a 2 sentence excerpt from the content that's relevant to our query. This will allow us to quickly understand each website that we find.\n  !pip install exa_py\nfrom exa_py import Exa\nimport os\nEXA_API_KEY= os.environ.get("EXA_API_KEY")\nexa = Exa(api_key=EXA_API_KEY)\ninput_url = "https://thrift.house"\nsearch_response = exa.find_similar_and_contents(\ninput_url,\nhighlights={"num_sentences":2},\nnum_results=10)\ncompanies = search_response.results\nprint(companies[0])\n  \nThis is an example of the full first result:\n [Result(url='https://www.mystorestash.com/',\nid='lMTt0MBzc8ztb6Az3OGKPA',\ntitle='The Airbnb of Storage',\nscore=0.758899450302124,\npublished_date='2023-01-01',\nauthor=None,\ntext=None,\nhighlights=["I got my suitcase picked up right from my dorm and didn't have to worry for the whole summer.Angela Scaria /Still have questions?Where are my items stored?"],\nhighlight_scores=[0.23423566609247845])]\n \nAnd here are the 10 titles and urls I got:\n  # to just see the 10 titles and urls\nurls = {}\nfor c in companies:\nprint(c.title + ':' + c.url)\n  \n rumie - College Marketplace:https://www.rumieapp.com/\nThe Airbnb of Storage:https://www.mystorestash.com/\nBunction.net:https://bunction.net/\nHome - Community Gearbox:https://communitygearbox.com/\nNOVA SHOPPING:https://www.novashoppingapp.com/\nRe-Fridge: Buy, sell, or store your college fridge - Re-Fridge:https://www.refridge.com/\nJamble: Social Fashion Resale:https://www.jambleapp.com/\nBranded Resale | Treet:https://www.treet.co/\nSwapskis:https://www.swapskis.co/\nEarn Money for Used Clothing:https://www.thredup.com/cleanout?redirectPath=%2Fcleanout%2Fsell\n \nLooks pretty darn good! As a bonus specifically for companies data, specifying category="company" in the SDK will search across a curated, larger companies dataset - if you're interested in this, let us know at  [email protected] !\nNow that we have 10 companies we want to dig into further, let’s do some research on each of these companies.\nNow let's get more information by finding additional webpages about each company. To do this, we're going to do a keyword search of each company's URL. We're using keyword because we want to find webpages that exactly match the company we're inputting. We can do this with the search_and_contents function, and specify type="keyword" and num_results=5. This will give me 5 websites about each company.\n  # doing an example with the first companies\nc = companies[0]\nall_contents = ""\nsearch_response = exa.search_and_contents(\nc.url, # input the company's URL\ntype="keyword",\nnum_results=5\n)\nresearch_response = search_response.results\nfor r in research_response:\nall_contents += r.text\n  \nHere's an example of the first result for the first company, Rumie App. You can see the first result is the actual link contents itself.\n <div><div><div><div><p><a href="https://www.rumieapp.com/"></a></p></div><div><p>The <strong>key</strong> to <strong>your</strong> college experience. </p><p><br/>Access the largest college exclusive marketplace to buy, sell, and rent with other students.</p></div></div><div><h2>320,000+</h2><p>Users in Our Network</p></div><div><div><p><h2>Selling is just a away.</h2></p><p>Snap a pic, post a listing, and message buyers all from one intuitive app.</p><div><p></p><p>Quick setup and .edu verification</p></div><div><p></p><p>Sell locally or ship to other campuses</p></div><div><p></p><p>Trade with other students like you</p></div></div><div><p><h2>. From local businesses around your campus</h2></p><h4>Get access to student exclusive discounts</h4><p>rumie students get access to student exclusive discounts from local and national businesses around their campus.</p></div></div><div><p><h2>Rent dresses from </h2></p><p>Wear a new dress every weekend! Just rent it directly from a student on your campus.</p><div><p></p><p>Make money off of the dresses you've already worn</p></div><div><p></p><p>rumie rental guarantee ensures your dress won't be damaged</p></div><div><p></p><p>Find a new dress every weekend and save money</p></div></div><div><p><h2>. The only place to buy student tickets at student prices</h2></p><h4>Buy or Sell students Football and Basketball tickets with your campus</h4><p>rumie students get access to the first-ever student ticket marketplace. No more getting scammed trying to buy tickets from strangers on the internet.</p></div><div><div><div><p></p><h4>Secure</h4><p>.edu authentication and buyer protection on purchases.</p></div><div><p></p><h4>Lightning-fast</h4><p>Post your first listing in under a minute.</p></div><div><p></p><h4>Verified Students</h4><p>Trade with other students, not strangers.</p></div><div><p></p><h4>Intuitive</h4><p>List an item in a few simple steps. Message sellers with ease.</p></div></div><p><a href="https://apps.apple.com/us/app/rumie-college-marketplace/id1602465206">Download the app now</a></p></div><div><p><h2>Trusted by students.</h2></p><div><div><p></p><p>Saves me money</p><p>Facebook Marketplace and Amazon are great but often times you have to drive a long way to meet up or pay for shipping. rumie let’s me know what is available at my school… literally at walking distance. </p></div><div><p></p><p>5 stars!</p><p>Having this app as a freshman is great! It makes buying and selling things so safe and easy! Much more efficient than other buy/sell platforms!</p></div><div><p></p><p>Amazing!</p><p>5 stars for being simple, organized, safe, and a great way to buy and sell in your college community.. much more effective than posting on Facebook or Instagram!</p></div><div><p></p><p>The BEST marketplace for college students!!!</p><p>Once rumie got to my campus, I was excited to see what is has to offer! Not only is it safe for students like me, but the app just has a great feel and is really easy to use. The ONLY place I’ll be buying and selling while I’m a student.</p></div></div></div><div><p><h2>Easier to than GroupMe or Instagram.</h2></p><p>Forget clothing instas, selling groupme's, and stress when buying and selling. Do it all from the rumie app.</p></div></div></div>\n \nFinally, let's create a summarized report that lists our 10 companies and gives us an easily digestible summary of each company. We can input all of this web content into an LLM and have it generate a nice report!\n  import textwrap\nimport openai\nimport os\nSYSTEM_MESSAGE = "You are a helpful assistant writing a research report about a company. Summarize the users input into multiple paragraphs. Be extremely concise, professional, and factual as possible. The first paragraph should be an introduction and summary of the company. The second paragraph should include pros and cons of the company. Things like what are they doing well, things they are doing poorly or struggling with. And ideally, suggestions to make the company better."\nopenai.api_key = os.environ.get("OPENAI_API_KEY")\ncompletion = openai.chat.completions.create(\nmodel="gpt-4",\nmessages=[\n{"role": "system", "content": SYSTEM_MESSAGE},\n{"role": "user", "content": all_contents},\n],\n)\nsummary = completion.choices[0].message.content\nprint(f"Summary for {c.url}:")\nprint(textwrap.fill(summary, 80))\n  \n Summary for https://www.rumieapp.com/:\nRumie is a college-exclusive marketplace app that allows students to buy, sell,\nand rent items with other students. It has over 320,000 users in its network and\noffers features such as quick setup, .edu verification, local and campus-wide\nselling options, and exclusive discounts from local businesses. Students can\nalso rent dresses from other students, buy or sell student tickets at student\nprices, and enjoy secure and intuitive transactions. The app has received\npositive feedback from users for its convenience, safety, and effectiveness in\nbuying and selling within the college community.\nPros of Rumie include its focus on college students' needs, such as providing a\nsafe platform and exclusive deals for students. The app offers an intuitive and\nfast setup process, making it easy for students to start buying and selling.\nThe option to trade with other students is also appreciated. Users find it convenient\nthat they can sell locally or ship items to other campuses. The app's rental\nguarantee for dresses provides assurance to users that their dresses won't be\ndamaged. Overall, Rumie is highly regarded as a simple, organized, and safe\nplatform for college students to buy and sell within their community.\nSuggestions to improve Rumie include expanding its reach to more colleges and\nuniversities across the nation and eventually internationally. Enhancing\nmarketing efforts and fundraising can aid in raising awareness among college\nstudents. Additionally, incorporating features such as improved search filters\nand a rating/review system for buyers and sellers could enhance the user\nexperience. Continual updates and improvements to the app's interface and\nfunctionality can also ensure that it remains user-friendly and efficient.\n \nAnd we’re done! We’ve built an app that takes in a company webpage and uses Exa to \nDiscover similar startups\nFind information about each of those startups\nGather useful content and summarize it with OpenAI\nHopefully you found this tutorial helpful and are ready to start building your very own company analyst! Whether you want to generate sales leads or research competitors to your own company, Exa's got you covered."
        },
        {
          "id": "https://docs.exa.ai/reference/exa-researcher",
          "url": "https://docs.exa.ai/reference/exa-researcher",
          "title": "Exa Researcher",
          "author": null,
          "publishedDate": "2024-03-02T11:36:30.000Z",
          "text": "https://docs.exa.ai/reference/exa-researcher\nExa Researcher\n2024-03-02T11:36:30Z\n    In this example, we will build Exa Researcher, a Javascript app that given a research topic, automatically searches for different sources about the topic with Exa and synthesizes the searched contents into a research report.\nThis interactive notebook was made with the Deno Javascript kernel for Jupyter. Check out the plain JS version if you prefer a regular Javascript file you can run with NodeJS, or want to skip to the final result. If you'd like to run this notebook locally, Installing Deno and connecting Deno to Jupyter is fast and easy.\nTo play with this code, first we need a Exa API key and an OpenAI API key. Get 1000 Exa searches per month free just for signing up! \nLet's import the Exa and OpenAI SDKs and put in our API keys to create a client object for each.\nMake sure to pick the right imports for your runtime and paste or load your API keys.\n  // Deno imports\nimport Exa from 'npm:exa-js';\nimport OpenAI from 'npm:openai';\n// NodeJS imports\n//import Exa from 'exa-js';\n//import OpenAI from 'openai';\nconst EXA_API_KEY = "" // insert or load your API key here\nconst OPENAI_API_KEY = ""// insert or load your API key here\nconst exa = new Exa(EXA_API_KEY);\nconst openai = new OpenAI({ apiKey: OPENAI_API_KEY });\n  \nSince we'll be making several calls to the OpenAI API to get a completion from GPT 3.5-turbo, let's make a simple utility wrapper function so we can pass in the system and user messages directly, and get the LLM's response back as a string.\n  async function getLLMResponse({system = 'You are a helpful assistant.', user = '', temperature = 1, model = 'gpt-3.5-turbo'}){\nconst completion = await openai.chat.completions.create({\nmodel,\ntemperature,\nmessages: [\n{'role': 'system', 'content': system},\n{'role': 'user', 'content': user},\n]\n});\nreturn completion.choices[0].message.content;\n}\n  \nOkay, great! Now let's starting building Exa Researcher. The app should be able to automatically generate research reports for all kinds of different topics. Here's two to start:\n  const SAMA_TOPIC = 'Sam Altman';\nconst ART_TOPIC = 'renaissance art';\n  \nThe first thing our app has to do is decide what kind of search to do for the given topic. \nExa offers two kinds of search: neural and keyword search. Here's how we decide:\nNeural search is preferred when the query is broad and complex because it lets us retrieve high quality, semantically relevant data. Neural search is especially suitable when a topic is well-known and popularly discussed on the Internet, allowing the machine learning model to retrieve contents which are more likely recommended by real humans. \nKeyword search is useful when the topic is specific, local or obscure. If the query is a specific person's name, and identifier, or acronym, such that relevant results will contain the query itself, keyword search may do well. And if the machine learning model doesn't know about the topic, but relevant documents can be found by directly matching the search query, keyword search may be necessary.\nSo, Exa Researcher is going to get a query, and it needs to automatically decide whether to use keyword or neural search to research the query based on the criteria. Sounds like a job for the LLM! But we need to write a prompt that tells it about the difference between keyword and neural search-- oh wait, we have a perfectly good explanation right there.\n  // Let's generalize the prompt and call the search types (1) and (2) in case the LLM is sensitive to the names. We can replace them with different names programmatically to see what works best.\nconst SEARCH_TYPE_EXPLANATION = `- (1) search is usually preferred when the query is a broad topic or semantically complex because it lets us retrieve high quality, semantically relevant data. (1) search is especially suitable when a topic is well-known and popularly discussed on the Internet, allowing the machine learning model to retrieve contents which are more likely recommended by real humans.\n- (2) search is useful when the topic is specific, local or obscure. If the query is a specific person's name, and identifier, or acronym, such that relevant results will contain the query itself, (2) search may do well. And if the machine learning model doesn't know about the topic, but relevant documents can be found by directly matching the search query, (2) search may be necessary.\n`;\n  \nHere's a function that instructs the LLM to choose between the search types and give its answer in a single word. Based on its choice, we return keyword or neural.\n  async function decideSearchType(topic, choiceNames = ['neural', 'keyword']){\nlet userMessage = 'Decide whether to use (1) or (2) search for the provided research topic. Output your choice in a single word: either "(1)" or "(2)". Here is a guide that will help you choose:\\n';\nuserMessage += SEARCH_TYPE_EXPLANATION;\nuserMessage += `Topic: ${topic}\\n`;\nuserMessage += `Search type: `;\nuserMessage = userMessage.replaceAll('(1)', choiceNames[0]).replaceAll('(2)', choiceNames[1]);\nconst response = await getLLMResponse({\nsystem: 'You will be asked to make a choice between two options. Answer with your choice in a single word.',\nuser: userMessage,\ntemperature: 0\n});\nconst useKeyword = response.trim().toLowerCase().startsWith(choiceNames[1].toLowerCase());\nreturn useKeyword ? 'keyword' : 'neural';\n}\n  \nLet's test it out:\n  console.log(SAMA_TOPIC, 'expected: keyword, got:', await decideSearchType(SAMA_TOPIC));\nconsole.log(ART_TOPIC, 'expected: neural, got:', await decideSearchType(ART_TOPIC));\n  \n Sam Altman expected: keyword, got: keyword\nrenaissance art expected: neural, got: neural\n \nGreat! Now we have to craft some search queries for the topic and the search type. There are two cases here: keyword search and neural search. Let's do the easy one first. LLMs already know what Google-like keyword searches look like. So let's just ask the LLM for what we want:\n  function createKeywordQueryGenerationPrompt(topic, n){\nreturn `I'm writing a research report on ${topic} and need help coming up with Google keyword search queries.\nGoogle keyword searches should just be a few words long. It should not be a complete sentence.\nPlease generate a diverse list of ${n} Google keyword search queries that would be useful for writing a research report on ${topic}. Do not add any formatting or numbering to the queries.`\n}\nconsole.log(await getLLMResponse({\nsystem: 'The user will ask you to help generate some search queries. Respond with only the suggested queries in plain text with no extra formatting, each on it\\'s own line.',\nuser: createKeywordQueryGenerationPrompt(SAMA_TOPIC, 3),\n}));\n  \n Sam Altman biography\nY Combinator founder\nInvestments made by Sam Altman\n \nThose are some good ideas!\nNow we have to handle the neural Exa search. This is tougher: you can read all about crafting good Exa searches here. But this is actually a really good thing: making the perfect Exa search is hard because Exa is so powerful! Exa allows us to express so much more nuance in our searches and gives us unparalleled ability to steer our search queries towards our real objective.\nWe need to our app to understand our goal, what Exa is, and how to use it to achieve the goal. So let's just tell the LLM everything it needs to know.\n  function createNeuralQueryGenerationPrompt(topic, n){\nreturn `I'm writing a research report on ${topic} and need help coming up with Exa keyword search queries.\nExa is a fully neural search engine that uses an embeddings based approach to search. Exa was trained on how people refer to content on the internet. The model is trained given the description to predict the link. For example, if someone tweets "This is an amazing, scientific article about Roman architecture: <link>", then our model is trained given the description to predict the link, and it is able to beautifully and super strongly learn associations between descriptions and the nature of the content (style, tone, entity type, etc) after being trained on many many examples. Because Exa was trained on examples of how people talk about links on the Internet, the actual Exa queries must actually be formed as if they are content recommendations that someone would make on the Internet where a highly relevant link would naturally follow the recommendation, such as the example shown above.\nExa neural search queries should be phrased like a person on the Internet indicating a webpage to a friend by describing its contents. It should end in a colon :.\nPlease generate a diverse list of ${n} Exa neural search queries for informative and trustworthy sources useful for writing a research report on ${topic}. Do not add any quotations or numbering to the queries.`\n}\nconsole.log(await getLLMResponse({\nsystem: 'The user will ask you to help generate some search queries. Respond with only the suggested queries in plain text with no extra formatting, each on it\\'s own line.',\nuser: createNeuralQueryGenerationPrompt(ART_TOPIC, 3),\n//model: 'gpt-4'\n}));\n  \n Hey, check out this comprehensive guide to Renaissance art:\nCan you recommend any scholarly articles on Renaissance art?\nI found an excellent website that explores the influence of religion on Renaissance art:\n \nNow let's put them together into a function that generates queries for the right search mode.\n  async function generateSearchQueries(topic, n, searchType){\nif(searchType !== 'keyword' && searchType !== 'neural'){\nthrow 'invalid searchType';\n}\nconst userPrompt = searchType === 'neural' ? createNeuralQueryGenerationPrompt(topic, n) : createKeywordQueryGenerationPrompt(topic, n);\nconst completion = await getLLMResponse({\nsystem: 'The user will ask you to help generate some search queries. Respond with only the suggested queries in plain text with no extra formatting, each on it\\'s own line.',\nuser: userPrompt,\ntemperature: 1\n});\nconst queries = completion.split('\\n').filter(s => s.trim().length > 0).slice(0, n);\nreturn queries;\n}\n  \nLet's make sure it works, and check out some more queries:\n  const samaQueries = await generateSearchQueries(SAMA_TOPIC, 3, 'keyword');\nconst artQueries = await generateSearchQueries(ART_TOPIC, 3, 'neural');\n  \n  console.log(samaQueries);\nconsole.log(artQueries);\n  \n [\n"Sam Altman biography",\n"Y Combinator founder",\n"Sam Altman startup advice"\n]\n[\n"Check out this comprehensive guide to Renaissance art:",\n"Discover the key characteristics of Renaissance art and its influential artists.",\n"Explore the development of perspective and human anatomy in Renaissance paintings."\n]\n \nNow it's time to use Exa to do the search, either neural or keyword. Using searchAndContents, we can get clean text contents bundled with each link.\n  async function getSearchResults(queries, type, linksPerQuery=2){\nlet results = [];\nfor (const query of queries){\nconst searchResponse = await exa.searchAndContents(query, { type, numResults: linksPerQuery, useAutoprompt: false });\nresults.push(...searchResponse.results);\n}\nreturn results;\n}\n  \n  const artLinks = await getSearchResults(artQueries, 'neural');\nconsole.log(artLinks[0]); // first result of six\n  \n {\ntitle: "How to Look at and Understand Great Art",\nurl: "https://www.wondrium.com/how-to-look-at-and-understand-great-art?lec=29%3Futm_source%3DSocialMedia&p"... 5 more characters,\npublishedDate: "2013-11-19",\nauthor: "Doc",\nid: "dq0L1GOKroUBuryT3ypSsQ",\ntext: "\\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" Trailer\\n" +\n" \\n" +\n" \\n" +\n"\\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" \\n" +\n" 01: The Importance of First Impressions\\n" +\n" \\n" +\n" Examine the conte"... 14543 more characters,\nscore: 0.1785949170589447\n}\n \nIn just a couple lines of code, we've used Exa to go from some search queries to useful Internet content.\nThe final step is to instruct the LLM to synthesize the content into a research report, including citations of the original links. We can do that by pairing the content and the urls and writing them into the prompt.\n  async function synthesizeReport(topic, searchContents, contentSlice = 750){\nconst inputData = searchContents.map(item => `--START ITEM--\\nURL: ${item.url}\\nCONTENT: ${item.text.slice(0, contentSlice)}\\n--END ITEM--\\n`).join('');\nreturn await getLLMResponse({\nsystem: 'You are a helpful research assistant. Write a report according to the user\\'s instructions.',\nuser: 'Input Data:\\n' + inputData + `Write a two paragraph research report about ${topic} based on the provided information. Include as many sources as possible. Provide citations in the text using footnote notation ([#]). First provide the report, followed by a single "References" section that lists all the URLs used, in the format [#] <url>.`,\n//model: 'gpt-4' //want a better report? use gpt-4\n});\n}\n  \n  const artReport = await synthesizeReport(ART_TOPIC, artLinks);\n  \n Research Report: Renaissance Art\nRenaissance art is a significant period in the history of art characterized by technical innovation and a richly symbolic visual language. It is known for combining the advancements in technique with the exploration of deeper layers of meaning in artworks. In Renaissance paintings, the identification of the patron or donor often provides insights into the intended message of the artwork. For example, a painting discussed in an article from the Royal Academy[^1] was commissioned by Jacopo Pesaro, the Bishop of Paphos on the island of Cyprus, and was likely painted by Titian during his early twenties. Analyzing the patronage and symbolism in Renaissance paintings allows for a better understanding of the multifaceted meanings conveyed by the artists.\nAnother key aspect of studying Renaissance art is understanding the contexts and environments in which art is encountered and viewed. The influence of the viewer's point of view and focal point plays a critical role in shaping the experience of art. Lectures provided by Wondrium[^3] discuss the importance of first impressions and explore how the artist positions the viewer with respect to the image. Additionally, the genres of Western art and the artist's media, tools, and techniques are explored in these lectures, providing a comprehensive understanding of the various elements that contribute to the creation and perception of Renaissance art.\nOverall, studying Renaissance art encompasses an exploration of not only the technical skills and innovations of the artists but also the cultural and historical contexts in which the artworks were created. By analyzing the patronage, symbolism, and viewing experience, researchers gain a deeper appreciation and interpretation of this significant period in the history of art.\nReferences:\n[1] How to Read a Renaissance Painting. (2016, April 1). Royal Academy. Retrieved from https://www.royalacademy.org.uk/article/how-to-read-a-renaissance-painting\n[3] How to Look at and Understand Great Art (#29). Wondrium. Retrieved from https://www.wondrium.com/how-to-look-at-and-understand-great-art?lec=29\n \nLet's wrap up by putting it all together into one researcher() function that starts from a topic and returns us the finished report. We can also let Exa Researcher generate us a report about our keyword search topic as well.\n  async function researcher(topic){\nconst searchType = await decideSearchType(topic);\nconst searchQueries = await generateSearchQueries(topic, 3, searchType);\nconsole.log(searchQueries);\nconst searchResults = await getSearchResults(searchQueries, searchType);\nconsole.log(searchResults[0]);\nconst report = await synthesizeReport(topic, searchResults);\nreturn report;\n}\n  \n  console.log(await researcher(SAMA_TOPIC));\n  \n [\n"Sam Altman biography",\n"Y Combinator founder",\n"Sam Altman startup advice"\n]\n{\ntitle: "Sam Altman - Wikipedia",\nurl: "https://en.wikipedia.org/wiki/Sam_Altman",\nauthor: null,\nid: "8942e4e1-a37d-42fd-bec1-cf1715ef8d35",\ntext: "\\n" +\n"From Wikipedia, the free encyclopedia\\n" +\n"\\n" +\n"Sam AltmanAltman in 2019BornSamuel Harris AltmanApril 22, 19"... 7424 more characters\n}\nResearch Report: Sam Altman\nSam Altman is an American entrepreneur, investor, and former CEO of OpenAI[^1^]. He is widely known for his significant contributions as the president of Y Combinator from 2014 to 2019[^1^]. Altman was born on April 22, 1985, in Chicago, Illinois[^2^]. He grew up in St. Louis, Missouri, where he attended John Burroughs School[^3^]. Altman's interest in computers began at a young age, and he received his first computer, an Apple Macintosh, at the age of eight[^3^]. His childhood idol was Steve Jobs[^4^]. Sam Altman dropped out of Stanford University after one year to pursue his entrepreneurial endeavors[^1^].\nAltman's career has been marked by his involvement in various tech ventures. He notably served as the CEO of OpenAI from 2019 to 2023[^1^]. Additionally, Altman played a pivotal role as the president of Y Combinator, a startup accelerator[^1^]. His influence in the tech industry has drawn comparisons to renowned figures like Steve Jobs and Bill Gates[^2^]. Altman firmly believes in the potential of artificial general intelligence (AGI) and its ability to accomplish tasks comparable to those performed by humans[^2^].\nReferences:\n[^1^] Wikipedia. (n.d.). Sam Altman. Retrieved from https://en.wikipedia.org/wiki/Sam_Altman\n[^2^] Britannica. (n.d.). Sam Altman. Retrieved from https://www.britannica.com/biography/Sam-Altman\n \nFor a link to a complete, cleaned up version of this project that you can execute in your NodeJS environment, check out the alternative JS-only version."
        },
        {
          "id": "https://docs.exa.ai/reference/exa-rag",
          "url": "https://docs.exa.ai/reference/exa-rag",
          "title": "Exa RAG",
          "author": null,
          "publishedDate": "2024-03-02T11:36:43.000Z",
          "text": "https://docs.exa.ai/reference/exa-rag\nExa RAG\n2024-03-02T11:36:43Z\n   \nLLMs are powerful because they compress large amounts of data and patterns into a format that allows convenient access, but this compressions isn't lossless. Exa can bring the most relevant data into context. This lets us to combine the compressed data of the LLM with a select quantity of uncompressed data for the problem at hand for the best generations possible.\nExa's SDKs make incorporating quality data into your LLM pipelines quick and painless. Install the SDK by running this command in your terminal:\n pip install exa-py \n  # Now, import the Exa class and pass your API key to it.\nfrom exa_py import Exa\nmy_exa_api_key = "YOUR_API_KEY_HERE"\nexa = Exa(my_exa_api_key)\n  \nFor our first example, we'll set up Exa to answer questions with OpenAI's popular GPT 3.5 model. (You can use GPT 4 or another model if you prefer!) We'll use Exa's highlight feature, which directly returns relevant text of customizable length for a query. You'll need to run pip install openai to get access to OpenAI's SDK if you haven't used it before. More information about the OpenAI Python SDK can be found here.\n  # Set up OpenAI' SDK\nfrom openai import OpenAI\nopenai_api_key = "YOUR_API_KEY_HERE"\nopenai_client = OpenAI(api_key=openai_api_key)\n  \nNow, we just need some questions to answer!\n  questions = [\n"How did bats evolve their wings?",\n"How did Rome defend Italy from Hannibal?",\n]\n  \nWhile LLMs can answer some questions on their own, they have limitations:\nLLMs don't have knowledge past when their training was stopped, so they can't know about recent events\nIf an LLM doesn't know the answer, it will often 'hallucinate' a correct-sounding response, and it can be difficult and inconvenient to distinguish these from correct answers\nBecause of the opaque manner of generation and the problems mentioned above, it is difficult to trust an LLM's responses when accuracy is important \nRobust retrieval helps solve all of these issues by providing quality sources of ground truth for the LLM (and their human users) to leverage. Let's use Exa to get some information about our questions:\n  # Parameters for our Highlights search\nhighlights_options = {\n"num_sentences": 7, # how long our highlights should be\n"highlights_per_url": 1, # just get the best highlight for each URL\n}\n# Let the magic happen!\ninfo_for_llm = []\nfor question in questions:\nsearch_response = exa.search_and_contents(question, highlights=highlights_options, num_results=3, use_autoprompt=True)\ninfo = [sr.highlights[0] for sr in search_response.results]\ninfo_for_llm.append(info)\n  \n [['As the only mammals with powered flight, the evolutionary\\xa0history of their wings has been poorly understood. However, research published Monday in Nature and PLoS Genetics has provided the first comprehensive look at the genetic origins of their incredible wings.But to appreciate the genetics of their wing development, it’s important to know how crazy a bat in flight truly\\xa0looks.Try a little experiment: Stick your arms out to the side, palms facing forward, thumbs pointing up toward the ceiling. Now imagine that your fingers are\\xa0long, arching down toward the floor like impossibly unkempt fingernails — but still made of bone, sturdy and spread apart. Picture the sides of your body connecting to your hands, a rubbery membrane attaching your leg and torso to those long fingers, binding you with strong, stretchy skin. Then, finally, imagine using your muscles to flap those enormous hands.Bats, man.As marvelous as bat flight is to behold, the genetic origins of their storied wings has remained murky. However, new findings from an international team of researchers led by Nadav Ahituv, PhD, of the University of California at San Francisco, Nicola Illing, PhD, of the University of Cape Town\\xa0in\\xa0South Africa\\xa0and Katie Pollard, PhD of the UCSF-affiliated Gladstone Institutes has shed new light on how, 50 million years ago, bats took a tetrapod blueprint for arms and legs and went up into the sky.Using a sophisticated set of genetic tools, researchers approached the question of how bats evolved flight by looking not only at which genes were used in the embryonic development of wings, but at what point during development the genes were turned on and off, and — critically — what elements in the genome were regulating the expression of these genes. Genes do not just turn themselves on without input; genetic switches, called enhancers, act to regulate the timing and levels of gene expression in the body.',\n"Since flight evolved millions of years ago in all of the groups that are capable of flight today, we can't observe the changes in behavior and much of the morphology that the evolution of flight involves. We do have the fossil record, though, and it is fairly good for the three main groups that evolved true flight. We'll spare you an in-depth description of how each group evolved flight for now; see the later exhibits for a description of each group and how they developed flight.",\n"It's easy to forget that one in five species of mammal on this planet have wings capable of delivering spectacularly acrobatic flying abilities. Equally incredibly, two-thirds of these 1,200 species of flying mammal can fly in the dark, using exquisite echolocation to avoid obstacles and snatch airborne prey with stunning deftness. These amazing feats have helped make bats the focus not only of folkloric fascination, but also of biological enquiry and mimicry by human engineers from Leonardo da Vinci onwards. Recent research in PLOS journals continues to add surprising new findings to what we know about bats, and how they might inspire us to engineer manmade machines such as drones to emulate their skills. Bats, unlike most birds and flying insects, have relatively heavy wings – something that might appear disadvantageous. But a recent study in PLOS Biology by Kenny Breuer and colleagues shows that bats can exploit the inertia of the wings to make sharp turns that would be near-impossible using aerodynamic forces alone. The authors combined high-speed film of real bats landing upside-down on ceiling roosts with computational modelling to tease apart aerodynamic and inertial effects."],\n["things, gold and silver, could buy a victory. And this Other Italian cities, inspired by Rome's example, overpowered occupying troops, shut their gates again and invited a second siege. Hannibal could not punish them without dividing his he had no competent leadership to do so, what with one member of",\n'A group of Celts known as the Senone was led through Italy by their commander, Brennus. The Senone Gauls were threatening the nearby town of Clusium, when Roman Ambassadors from the Fabii family were sent to negotiate peace for Clusium. The Romans were notoriously aggressive, and so it is only a little surprising that when a scuffle broke out between the Gauls and Clusians, the Fabii joined in and actually killed a Senone chieftain. The Roman people voted to decide the fate of those who broke the sacred conduct of ambassadors, but the Fabii were so popular that they were instead voted to some of the highest positions in Rome. This absolutely infuriated Brennus and his people and they abandoned everything and headed straight for Rome. Rome was woefully unprepared for this sudden attack. The Gauls had marched with purpose, declaring to all the towns they passed that they would not harm them, they were heading straight for Rome.',\n"Hannibal had no intention to sit and recieve the romans in spain.Hannibal clearly considered the nature of roman power-and came to the conclusion that Rome could only be defeated in Italy.The cornerstone of Rome's power was a strategic manpower base that in theory could produce 7,00,000 infantry and 70,000 cavalry.More than half of this manpower base (4,00,000) was provided by rome's Italian allies,who paid no taxes but had to render military service to rome's armies.Not all were content.Carthage on the other hand rarely used its own citizens for war,bulk of its army being mercenaries.In any case its manpower could never even come close to Rome,the fact that had aided roman victory in the 1st Punic war.Hannibal thus understood that Rome could afford to raise and send army after army to spain and take losses. Meanwhile any carthiginian losses in spain would encourage the recently conquered iberian tribes to defect. The only way to defeat Rome,was to fight in italy itself.By winning battle after battle on italian soil and demonstrating to the italian allies rome's inability to protect them and weakness,he could encourage them to break free of Rome eroding Rome's manpower to sizeable proportions. But there was one problem,his fleet was tiny and Rome ruled the seas.By land,the coastal route would be blocked by Roman forces and her ally-the great walled city of massalia.Hannibal thus resolved to think and do the impossible - move thousands of miles by land through the pyranees mountains,uncharted territory inhabited by the fierce gauls ,then through the Alps mountains and invade italy. Even before the siege of Saguntum had concluded,Hannibal had set things in motion.Having sent a number of embassies to the Gallic tribes in the Po valley with the mission of establishing a safe place for Hannibal to debouch from the Alps into the Po valley. He did not desire to cross this rugged mountain chain and to descend into the Po valley with exhausted troops only to have to fight a battle.Additionally the fierce gauls would provide a source of manpower for Hannibal's army.The romans had recently conquered much territory from the gauls in this area,brutally subjagating them ,seizing their land and redistributing it to roman colonists.Thus securing an alliance proved to be easy. After the sack of Saguntum he dismissed his troops to their own localities."]]\n \nNow, let's give the context we got to our LLM so it can answer our questions with solid sources backing them up!\n  responses = []\nfor question, info in zip(questions, info_for_llm):\nsystem_prompt = "You are RAG researcher. Read the provided contexts and, if relevant, use them to answer the user's question."\nuser_prompt = f"""Sources: {info}\nQuestion: {question}"""\ncompletion = openai_client.chat.completions.create(\nmodel="gpt-3.5-turbo",\nmessages=[\n{"role": "system", "content": system_prompt},\n{"role": "user", "content": user_prompt},\n]\n)\nresponse = f"""\nQuestion: {question}\nAnswer: {completion.choices[0].message.content}\n"""\nresponses.append(response)\n  \n  from pprint import pprint # pretty print\npprint(responses)\n  \n ['\\n'\n' Question: How did bats evolve their wings?\\n'\n' Answer: Recent research has shed new light on how bats evolved their '\n'wings. An international team of researchers used genetic tools to study the '\n'embryonic development of bat wings and the genes involved in their '\n'formation. They also investigated the regulatory elements in the genome that '\n'control the expression of these genes. By analyzing these factors, the '\n'researchers discovered that bats took a tetrapod blueprint for arms and legs '\n'and adapted it to develop wings, allowing them to fly. This research '\n'provides a comprehensive understanding of the genetic origins of bat wings '\n'and how they evolved over 50 million years ago.\\n'\n' ',\n'\\n'\n' Question: How did Rome defend Italy from Hannibal?\\n'\n' Answer: Rome defended Italy from Hannibal by using various strategies. One '\n'of the main defenses relied on the Roman manpower base, which consisted of a '\n'large army made up of Roman citizens and Italian allies who were obligated '\n"to render military service. Rome's strategic manpower base was a cornerstone "\n'of their power, as it could produce a significant number of infantry and '\n'cavalry. This posed a challenge for Hannibal, as Carthage relied heavily on '\n"mercenaries and could not match Rome's manpower.\\n"\n'\\n'\n'Hannibal realized that in order to defeat Rome, he needed to fight them in '\n'Italy itself. His plan was to win battles on Italian soil and demonstrate '\n"Rome's inability to protect their Italian allies, with the intention of "\n"encouraging them to break free from Rome. This would erode Rome's manpower "\n'base to a sizeable proportion. However, Hannibal faced several obstacles. '\n'Rome ruled the seas, making it difficult for him to transport troops and '\n'supplies by sea. Additionally, the coastal route to Italy would be blocked '\n'by Roman forces and their ally, the walled city of Massalia.\\n'\n'\\n'\n'To overcome these challenges, Hannibal devised a daring plan. He decided to '\n'lead his troops on a treacherous journey through the Pyrenees mountains, '\n'inhabited by fierce Gauls, and then through the Alps mountains to invade '\n'Italy. He sent embassies to Gallic tribes in the Po valley, securing '\n'alliances and establishing a safe place for his army to enter the Po valley '\n'from the Alps.\\n'\n'\\n'\n'Overall, Rome defended Italy from Hannibal by leveraging their manpower '\n'base, their control of the seas, and their strategic alliances with Italian '\n'allies. They also had the advantage of better infrastructure and control '\n'over resources within Italy itself. These factors ultimately played a '\n"significant role in Rome's defense against Hannibal's invasion.\\n"\n' ']\n \nExa can be used for more than simple question answering. One superpower of embeddings-based search is that we can search for the meaning of sentences or even paragraphs:\n  paragraph = """\nGeorgism, also known as Geoism, is an economic philosophy and ideology named after the American\npolitical economist Henry George (1839–1897).This doctrine advocates for the societal collective,\nrather than individual property owners, to capture the economic value derived from land and other\nural resources. To this end, Georgism proposes a single tax on the unimproved value of land, known\nas a "land value tax," asserting that this would deter speculative land holding and promote efficient\nuse of valuable resources. Adherents argue that because the supply of land is fundamentally inelastic,\ntaxing it will not deter its availability or use, unlike other forms of taxation. Georgism differs\nfrom Marxism and capitalism, underscoring the distinction between common and private property while\nlargely contending that individuals should own the fruits of their labor."""\nquery = f"The best academic source about {paragraph} is (paper: "\ngeorgism_search_response = exa.search_and_contents(paragraph, highlights=highlights_options, num_results=5, use_autoprompt=False)\n  \n  for result in georgism_search_response.results:\nprint(result.title)\nprint(result.url)\npprint(result.highlights)\n  \n Henry George\nhttps://www.newworldencyclopedia.org/entry/Henry_George\n["George's theory of interest is nowadays dismissed even by some otherwise "\n'Georgist authors, who see it as mistaken and irrelevant to his ideas about '\n'land and free trade. The separation of the value of land into improved and '\n"unimproved is problematic in George's theory. Once construction has taken "\n'place, not only the land on which such improvements were made is affected, '\n'the value of neighboring, as yet unimproved, land is impacted. Thus, while '\n'the construction of a major attraction nearby may increase the value of '\n'land, the construction of factories or nuclear power plants decreases its '\n'value. Indeed, location is the single most important asset in real estate. '\n'George intended to propose a tax that would have the least negative impact '\n'on productive activity. However, even unimproved land turns out to be '\n'affected in value by productive activity in the neighborhood.']\nWikiwand\nhttps://www.wikiwand.com/en/Georgism\n['Georgism is concerned with the distribution of economic rent caused by land '\n'ownership, natural monopolies, pollution rights, and control of the commons, '\n'including title of ownership for natural resources and other contrived '\n'privileges (e.g. intellectual property). Any natural resource which is '\n'inherently limited in supply can generate economic rent, but the classical '\n'and most significant example of land monopoly involves the extraction of '\n'common ground rent from valuable urban locations. Georgists argue that '\n'taxing economic rent is efficient, fair and equitable. The main Georgist '\n'policy recommendation is a tax assessed on land value, arguing that revenues '\n'from a land value tax (LVT) can be used to reduce or eliminate existing '\n'taxes (such as on income, trade, or purchases) that are unfair and '\n'inefficient. Some Georgists also advocate for the return of surplus public '\n"revenue to the people by means of a basic income or citizen's dividend. The "\n'concept of gaining public revenues mainly from land and natural resource '\n'privileges was widely popularized by Henry George through his first book, '\n'Progress and Poverty (1879).']\nHenry George\nhttps://www.conservapedia.com/Henry_George\n['He argued that land, unlike other factors of production, is supplied by '\n'nature and that rent is unearned surplus. The landless deserve their share '\n'of this surplus as a birthright, according to George. Henry George was born '\n'in Philadelphia, Pennsylvania, on the 2nd of September 1839. He settled in '\n'California in 1858; then later removed to New York in 1880; was first a '\n'printer, then an editor, but finally devoted all his life to economic and '\n'social questions. In 1860, George met Annie Corsina Fox. Her family was very '\n'opposed to the relationship, and in 1861 they eloped. In 1871 he published '\n'Our Land Policy, which, as further developed in 1879 under the title of '\n'Progress and Poverty, speedily attracted the widest attention both in '\n'America and in Europe.']\nGeorgism - Wikipedia\nhttps://en.wikipedia.org/wiki/Georgism\n['A key issue to the popular adoption of Georgism is that homes are illiquid '\n'yet governments need cash every year. Some economists have proposed other '\n'ways of extracting value from land such as building government housing and '\n'selling homes to new buyers in areas of fast-rising land value. The '\n'government would theoretically collect revenue from home sales without much '\n'cost to current homeowners while slowing down land value appreciation in '\n'high-demand areas. Henry George, whose writings and advocacy form the basis '\n'for Georgism Georgist ideas heavily influenced the politics of the early '\n'20th century. Political parties that were formed based on Georgist ideas '\n'include the Commonwealth Land Party in the United States, the Henry George '\n'Justice Party in Victoria, the Single Tax League in South Australia, and the '\n"Justice Party in Denmark. In the United Kingdom, George's writings were "\n'praised by emerging socialist groups in 1890s such as the Independent Labour '\n'Party and the Fabian Society, which would each go on to help form the '\n'modern-day Labour Party.']\nGeorgism\nhttps://rationalwiki.org/wiki/Georgism\n['Even with mostly primitive methods, land values are already assessed around '\n'the world wherever property/council taxes exist, and some municipalities '\n'even collect all their revenue from land values. Though these are '\n'market-based measures, they can still prove difficult and require upfront '\n'investment. Georgists believe that the potential value of land is greater '\n'than the current sum of government spending, since the abolition of taxes on '\n'labor and investment would further increase the value of land. Conversely, '\n'the libertarian strain in Georgism is evident in the notion that their land '\n'tax utopia also entails reducing or eliminating the need for many of the '\n'things governments currently supply, such as welfare, infrastructure to '\n'support urban sprawl, and military & foreign aid spending to secure '\n"resources abroad. Therefore, many Georgists propose a citizen's dividend. "\n'This is a similar concept to basic income but its proponents project its '\n'potential to be much larger due to supposedly huge takings from the land '\n'tax, combined with lowered government spending. It has been recognized since '\n'Adam Smith and David Ricardo that a tax on land value itself cannot be '\n'passed on to tenants, but instead would be paid for by the owners of the '\n'land:']\n \nUsing Exa, we can easily find related papers, either for further research or to provide a source for our claims. This is just a brief intro into what Exa can do. For a look at how you can leverage getting full contents, check out this article."
        },
        {
          "id": "https://docs.exa.ai/",
          "url": "https://docs.exa.ai/",
          "title": "Introduction",
          "author": "",
          "publishedDate": "2023-03-03T23:47:48.000Z",
          "text": "Exa is a search engine made for AIs.  \n Exa has three core functionalities. \n Search Find webpages using Exa's embeddings-based or Google-style keyword search.\n Get contents Obtain clean, up-to-date, parsed HTML from Exa search results. Supports full text, LLM summaries, and highlights (snippets of relevant text).\n Find similar pages Based on a link, find and return pages that are similar in meaning\nLearn how to do an Exa search in your project\nPython Quickstart\nJavaScript Quickstart\ncURL Quickstart\nLearn how to do Exa-powered RAG\nGetting Started with RAG in Python\nGetting Started with RAG in TypeScript\nGetting Started with LangChain\nGetting Started with OpenAI\nGetting Started with CrewAI\nGetting Started with LlamaIndex\n              Table of Contents   \n Exa finds the exact content you're looking for on the web."
        },
        {
          "id": "https://exa.ai/blog/announcing-exa",
          "url": "https://exa.ai/blog/announcing-exa",
          "title": "Exa API",
          "author": "exa",
          "text": "Steps toward the mission Today, we're excited to announce that we're renaming Metaphor to \"Exa” to better reflect our\nmission. We're also launching a new type of search experience – highlights. With highlights, you can instantly extract any webpage content from each search result using a\ncustomizable embedding model. All together, we’re taking big steps toward our core mission – organize the world’s knowledge.  Why Exa? The internet contains the collective knowledge output of mankind – all the great works of art\nand literature, millions of essays, hundreds of millions of research papers, billions of\nimages and videos, trillions of ideas sprinkled across tweets, forums, and memes. Searching the internet should feel like navigating a grand library of knowledge, where you\ncould weave insights across cultures, industries, and millenia. Of course it doesn’t feel that way. Today, searching the internet feels more like navigating a\nlandfill.  Many. have. debated. what's. wrong. with. search.  But the core problem is actually simple – knowledge on the internet is buried under an\noverwhelming amount of information. The core solution is also simple – we need a better search algorithm to filter all that\ninformation and organize the knowledge buried inside. Exa is going to organize the world’s knowledge.  To illustrate the problem, try googling \"startups working on climate change\".     You get 43,800,000 results. That’s a lot of information! But how much actual knowledge? I see\nmany listicle results, but no actual startups working on climate change. That's because Google still uses a keyword based algorithm. Keywords as a filtering\nmechanism may have worked in 1998, but they don't work for an internet with a thousand times more\ncontent and an SEO industry devoted to hacking keywords. Exa, in contrast, is the first web-scale neural search engine. Our algorithm uses\ntransformers end-to-end, the same technology that built ChatGPT. This enables us to filter the\ninternet by meaning, not by keyword. Here's the same search on Exa:  startups working on climate change   Rewind Climate Change https://www.rewind.earth/ Stored on the bottom of the sea, the carbon will be sequestered for thousands of years.     Holocene — Harnessing organic chemistry to remove CO2 https://www.theholocene.co/  Harnessing organic chemistry to remove CO 2 from the atmosphere, forever.     A NOVEL PHOTOCHEMICAL PROCESS TO CAPTURE CARBON DIOXIDE https://www.banyucarbon.com/ Combined they have >130 peer-reviewed publications, have managed research projects worth millions of dollars, and have both presented before the National Academies of Science.     Living Carbon https://www.livingcarbon.com/  Restoring ecosystems from the ground up More biomass and faster growth means more carbon capture Our trees accumulate up to 53% more biomass than control seedlings We incorporated a photosynthesis enhancement trait to help trees grow faster We’ve developed a metal accumulation trait so trees can absorb more metals in their roots and stem These metals naturally slow wood decay, creating durable wood products and retaining carbon in wood for a longer period of time Our trees are unique in their ability to grow on degraded land with high concentrations of heavy metals As our trees grow, they can clean soil made toxic by industrial activity, store more carbon, and create investment opportunities on otherwise abandoned land      Phykos, PBC https://www.phykos.co/ We've built houses and farms and families, and we've travelled widely to see firsthand just how precious this world is.     Eion Carbon | Carbon removal that checks all the right boxes https://eioncarbon.com/ For every 10,000 tons of carbon removal, we create about five local jobs.     Climate Robotics https://www.climaterobotics.com/ In order to save ourselves from catastrophic climate change, we must automate sequestering carbon as well.     Carbon Dioxide Removal | Carba https://www.carba.com/ When biomass biodegrades, up to half of the carbon can be converted into methane, which carries 28 times the greenhouse warming potential.     Unemit https://unemit.com/ We firmly believe humanity can and will reach net zero greenhouse gas emissions, and ours will be a significant contribution to stopping and then reversing climate change by driving permanent Carbon Dioxide Removal (CDR) of more than 10% of historical emissions by the end of the century.\t     Twelve | The Carbon Transformation Company https://www.twelve.co/ To get there, we are working with brands and industrial partners to implement our technology at scale and eliminate emissions from global manufacturing supply chains.     Note that these results don't necessarily mention the words \"climate change\" or \"startup\". Exa\nfiltered out all the noisy webpages – the listicles talking about startups – and returned the\nactual knowledge – the startups themselves. Exa's goal is to understand any query – no matter how complex – and filter the internet to\nexactly the knowledge required for that query. This is the solution to the problem of overwhelming information. Instead of 40 million\nnoisy/redundant links, we’ll filter down to exactly the pieces of knowledge needed. In doing\nso, we’ll convert an internet landfill into a library of knowledge with each and every search. That's why we chose the name Exa. Exa means 10^18th power. Google means 10^100th power. While\nGoogle aims to surface all information, Exa aims to filter all information into organized\nknowledge. And when it comes to organized knowledge, 10^18 is greater than 10^100. Why .ai? It just so happens that a creature recently emerged on this planet with a particularly strong\nneed for organized, high quality knowledge. AIs, powered by LLMs like ChatGPT, are overhauling the way we do knowledge work. But to\nperform knowledge work optimally, AIs must incorporate the internet’s knowledge. That means\nthey need to be paired with a web retrieval system, like Exa. The quality of web retrieval particularly matters for LLMs. If retrieval returns junk\ninformation, the LLM will output junk content. If the retrieval is powerful enough to filter\nthe internet to the right knowledge, the LLM will output the highest quality content possible. Most of our customers have tried search APIs like Bing and Google SERP. They come to us\nbecause those APIs just don’t work for their use cases. Those APIs work for humans, but not\nfor LLMs, which have quite different needs. We’ve been on the frontlines witnessing this transformation of knowledge work and of\nretrieval. We believe AI applications will soon search the internet more than humans. That’s why Exa wasn’t just built with AI, it was also built for AI. Building with the Exa API Exa was designed with all the features necessary for AI applications:  Filter out noisy SEO results to only the type of content the AI requests Handle long queries – a sentence, a paragraph, or even a whole webpage can be a query Retrieve the page content of any url for downstream processing  Today we're adding highlights. Highlights means Exa can instantly extract any\npiece of content from any result's webpage. Behind the scenes, we’re chunking and embedding\nfull webpages with a paragraph prediction model. Because this happens live, you can customize\nhighlights length, # per page, and specify a secondary query specific to what content you want\nto find. Customers using highlights have seen significant increases in user conversion compared to Bing\nand other search providers. It's exciting to see customers using Exa for so many applications that just weren't possible a\nyear ago, from research paper writing assistants, to lead generation research bots, to\nlearning tools for students. A mission we take seriously As builders at the doorway to the world’s knowledge, we recognize the power we wield. The\nknowledge people consume underlies nearly everything, from our politics to our scientific\nprogress to our daily perceptions of the world. We believe that organizing the world's knowledge, and thereby giving users immense power to\nfilter the internet, is critical societal infrastructure that does not currently exist. Luckily, our incentives are aligned with our values. We have no ad-based need to monetize\nattention. Our API customers want the highest quality search possible. So do we. When the\ninternet feels like a library of all knowledge, we'll know we succeeded. If you want to help us build some of the coolest technology – designing internet-scale web\ncrawling infrastructure, training foundation models for search on our own GPU cluster, serving\ncustom vector databases at scale – you can check out our roles here. And if you're ready to build with the Exa API, you can sign up here. Thanks for reading and we'll keep writing more posts soon so you can join us on the ride!  See more",
          "image": "https://exa.imgix.net/og-image.png"
        },
        {
          "id": "https://dashboard.exa.ai/",
          "url": "https://dashboard.exa.ai/",
          "title": "Exa API Dashboard",
          "author": "Exa",
          "publishedDate": "2012-01-06T00:00:00.000Z",
          "text": "Get started with Exa No credit card required   If you are supposed to join a team, please contact your team's administrator for an invite link."
        }
      ]
    }
  ]
}

result = exa.get_contents(
  ["https://www.apple.com/"],
  livecrawl="always",
  subpage_target=["news", "product"],
  subpages=10
)

{
  "results": [
    {
      "id": "https://www.apple.com/",
      "url": "https://www.apple.com/",
      "title": "Apple",
      "author": "",
      "publishedDate": "2024-10-30T16:54:13.000Z",
      "text": "Apple Intelligence is here.\nExperience it now on the latest iPhone, iPad, and Mac models with a free software update.1 \nMacBook Pro\nA work of smart.\nAvailable starting 11.8\n Hello, Apple Intelligence. \nApple Intelligence is here.\nExperience it now on the latest iPhone, iPad, and Mac models with a free software update.1 \nMac mini\nSize down. Power up.\nAvailable starting 11.8\n Hello, Apple Intelligence. \nApple Intelligence is here.\nExperience it now on the latest iPhone, iPad, and Mac models with a free software update.1 \niMac\nBril l l l l liant.\nAvailable starting 11.8\n Hello, Apple Intelligence. \niPhone 16 Pro\nHello, Apple Intelligence.\niPhone 16\nHello, Apple Intelligence.\nAirPods Pro 2\nHearing Test, Hearing Aid, and Hearing Protection features in a free software update.2\n Apple Intelligence \nAI for the rest of us.\n Apple Trade In \nGet $180-$650 in credit when you trade in iPhone 12 or higher.3 \n Apple Card \nGet up to 3% Daily Cash back with every purchase.\nApple TV+\nFAM Gallery",
      "image": "https://www.apple.com/ac/structured-data/images/open_graph_logo.png?202110180743",
      "subpages": [
        {
          "id": "https://www.apple.com/apple-news/",
          "url": "https://www.apple.com/apple-news/",
          "title": "Apple News+",
          "author": "",
          "publishedDate": "2024-05-07T20:24:00.000Z",
          "text": "Get 3 months of Apple News+ free with a new iPhone, iPad, or Mac. 1  Learn more\nA world of journalism. One trusted subscription.\n Access top stories from over 400 leading publications.\n Get the latest news from local, national, and international titles.\n Solve daily puzzles created exclusively for Apple News+.\n Listen to the week’s best articles with Apple News+ audio stories.\nBuy an Apple device\n3 months free\nGet 3 months of Apple News+ free with the purchase of an eligible device. 1  \n  Check eligibility \n \nFree 1-month trial\nApple News+\nNew subscribers get 1 month of Apple News+ free, then pay\n$12.99 per month. \nFree 1-month trial\nApple One\nBundle Apple News+ with five other great services for one low monthly price.  Learn more    \nRewriting the reading experience.\nExplore an incredibly diverse and wide range of global publications in just one app. Expert editors surface the most compelling, must-read articles in Top Stories, Spotlight, and other collections. Vivid photography and animation, rich videos, and immersive layouts bring each story to life in striking detail. And you can even download issues to read offline.\nSolve puzzles \n \nApple News+ includes original mini and full-size crossword puzzles, ranging in difficulty from easy to challenging. New crosswords are available daily, and you can take on past games from the archive.\n The stories you need to hear. Everywhere you want to listen.\n \nSubscribers to Apple News+ can hear professionally narrated articles in the iPhone app or on the Apple News+ Narrated podcast. And everyone can listen weekday mornings as our host, Shumita Basu, talks you through the day’s headlines on Apple News Today — and catch weekly interviews with top journalists on In Conversation.\nA global news source.Personalized for you.\nStay close to what’s happening close to home.\nGet all the latest news from local publications in a growing number of cities — with coverage on politics, sports, dining, culture, and more.\nThe hard-hitting, fast-breaking sports news you need.\nGet highlights, scores, and schedules for professional and college teams and leagues. Apple News+ unlocks access to The Athletic, Sports Illustrated, local newspapers, and more.\nThe feed that feeds your interests.\nAs you read, Apple News gets a better understanding of your interests and suggests relevant stories that you can easily find throughout the app.\nOnly you see what you read.\nApple News only uses on-device intelligence to recommend stories and doesn’t access your information without your permission. We never share it with others or allow advertisers to track you.\n Read, listen, and play Apple News on your favorite devices.\n \niPhone\nMac\niPad\nCarPlay\nHomePod\nApple Watch\nExtra. Extra.Apple News+ delivers more.\n $12.99/mo. after free trial *  \nHundreds of magazines and leading newspapers\nincluded\n not included\n Apple News+ audio stories \nincluded\n not included\nLocal news from top regional sources\nincluded\n not included\nSports coverage from local and premium publications\nincluded\n not included\nExclusive daily puzzles\nincluded\n not included\nReading online and off across devices\nincluded\n not included\nCover-to-cover magazines\nincluded\n not included\nFamily Sharing for up to six\nincluded\n not included\nTop stories chosen by editors, personalized for you\nincluded\nincluded\nMy Sports with scores, standings, and highlights\nincluded\nincluded\n Apple News Today and In Conversation \nincluded\nincluded\nLocal news\nincluded\nincluded\nPrivate and secure reading\nincluded\nincluded\nCarPlay\nincluded\nincluded\nApple News+\nGet 3 months of Apple News+ free with a new iPhone, iPad, or Mac. 1  \n \nApple One\nBundle Apple News+ with 5 other great services. And enjoy more for less.\n \n Questions? Answers.\n \nApple News is the easiest way to stay up to date with the news and information that matter most, with a seamless reading experience across all your devices. Experienced Apple News editors curate the day’s top stories from trusted sources, and advanced algorithms help you discover stories you’ll find interesting. Our editors create an audio briefing called Apple News Today, covering the biggest stories each weekday morning. You can also subscribe to a daily email newsletter from the Apple News editors highlighting the news you need to know to start your day.\nApple News and Apple News+ both feature the world’s best journalism from trusted sources, curated by human editors and personalized to your interests. With Apple News+ you unlock access to premium content from hundreds of magazines and leading local, national, and international newspapers, cover-to-cover magazine issues you can read online or off, and audio stories — professionally narrated versions of some of the best stories available in Apple News+.\nApple News+ costs just\n$12.99 per month after a free trial. Apple News+ is also included in the Apple One Premier plan, which bundles five other Apple services into a single monthly subscription for\n$37.95 per month.\nWith an Apple News+ subscription, you get access to more than 400 of the world’s best magazines, newspapers, and digital publishers. The magazines cover a wide range of interests, from food to fashion to politics and much more. Newspapers include leading titles such as The Wall Street Journal, Los Angeles Times, Houston Chronicle, and San Francisco Chronicle. Subscribers also receive access to audio stories — professionally narrated versions of some of the best articles available in Apple News+ — and exclusive puzzles updated daily.\nYou can download full issues of your favorite magazines to your Apple devices and access them anywhere, anytime, without an internet connection. You can also listen to Apple News+ audio stories, Apple News Today, and In Conversation offline.\n \nUpdate to the latest version of iOS or macOS to start your Apple News+ free trial.",
          "image": "https://www.apple.com/v/apple-news/l/images/shared/apple-news__6xg2yiktruqy_og.png?202401091100"
        },
        {
          "id": "https://www.apple.com/us/shop/goto/store",
          "url": "https://www.apple.com/us/shop/goto/store",
          "title": "Apple Store Online",
          "author": "",
          "publishedDate": "2024-06-18T09:56:09.000Z",
          "text": "∆ Apple Intelligence is available in beta on all iPhone 16 models, iPhone 15 Pro, and iPhone 15 Pro Max, with Siri and device language set to U.S. English, as an iOS 18 update. English (Australia, Canada, New Zealand, South Africa, UK) language support available this December. Some features and support for additional languages, like Chinese, English (India, Singapore), French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, and others, will be coming over the course of the next year. \n^ Apple Intelligence is available in beta on all Mac models with M1 and later, with Siri and device language set to U.S. English, as a macOS Sequoia update. English (Australia, Canada, New Zealand, South Africa, UK) language support available this December. Some features and support for additional languages, like Chinese, English (India, Singapore), French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, and others, will be coming over the course of the next year. \n± Apple Intelligence is available in beta on iPad mini (A17 Pro) and iPad models with M1 and later, with Siri and device language set to U.S. English, as an iPadOS 18 update. English (Australia, Canada, New Zealand, South Africa, UK) language support available this December. Some features and support for additional languages, like Chinese, English (India, Singapore), French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, and others, will be coming over the course of the next year.\n∆∆ Apple Intelligence is available in beta on all iPhone 16 models, iPhone 15 Pro, iPhone 15 Pro Max, iPad mini (A17 Pro), and iPad and Mac models with M1 and later, with Siri and device language set to U.S. English, as part of an iOS 18, iPadOS 18, and macOS Sequoia update. English (Australia, Canada, New Zealand, South Africa, UK) language support available this December. Some features and support for additional languages, like Chinese, English (India, Singapore), French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, and others, will be coming over the course of the next year.\n* Pricing for iPhone 16 and iPhone 16 Plus includes a $30 connectivity discount that requires activation with AT&T, Boost Mobile, T-Mobile, or Verizon. Available to qualified customers and requires 24-month installment loan when you select Citizens One or Apple Card Monthly Installments (ACMI) as payment type at checkout at Apple. You’ll need to select AT&T, Boost Mobile, T-Mobile, or Verizon as your carrier when you check out. An iPhone purchased with ACMI is always unlocked, so you can switch carriers at any time. Subject to credit approval and credit limit. Taxes and shipping are not included in ACMI and are subject to your card’s variable APR. Additional Apple Card Monthly Installments terms are in the Apple Card Customer Agreement (Opens in a new window) . Additional iPhone Payments terms are here (Opens in a new window) . ACMI is not available for purchases made online at special storefronts. The last month’s payment for each product will be the product's purchase price, less all other payments at the monthly payment amount. ACMI financing is subject to change at any time for any reason, including but not limited to, installment term lengths and eligible products. See https://support.apple.com/kb/HT211204 (Opens in a new window)  for information about upcoming changes to ACMI financing.\n◊ Apple Card Monthly Installments (ACMI) is a 0% APR payment option that is only available if you select it at checkout in the U.S. for eligible products purchased at Apple Store locations, apple.com (Opens in a new window) , the Apple Store app, or by calling 1-800-MY-APPLE, and is subject to credit approval and credit limit. See support.apple.com/kb/HT211204 (Opens in a new window)  for more information about eligible products. APR ranges may vary based on when you accepted an Apple Card. Cardholders who accept an Apple Card on and/or after August 1, 2024: Variable APRs for Apple Card, other than ACMI, range from 19.24% to 29.49% based on creditworthiness. Rates as of August 1, 2024. Existing cardholders: See your Customer Agreement for applicable rates and fee. If you buy an ACMI-eligible product by choosing to pay in full with Apple Card (instead of using ACMI), that purchase is subject to the Apple Card variable APR, not 0% APR. Taxes and shipping on ACMI purchases are subject to the variable APR, not 0% APR. When you buy an iPhone with ACMI, you’ll need to select AT&T, Boost Mobile, T-Mobile, or Verizon as your carrier when you check out. An iPhone purchased with ACMI is always unlocked, so you can switch carriers at any time. ACMI is not available for purchases made online at the following special stores: Apple Employee Purchase Plan; participating corporate Employee Purchase Programs; Apple at Work for small businesses; Government and Veterans and Military Purchase Programs; or on refurbished devices. The last month’s payment for each product will be the product’s purchase price, less all other payments at the monthly payment amount. ACMI financing is subject to change at any time for any reason, including but not limited to installment term lengths and eligible products. See support.apple.com/kb/HT211204 (Opens in a new window)  for information about upcoming changes to ACMI financing. See the Apple Card Customer Agreement (Opens in a new window)  for more information about ACMI financing.\nTo access and use all Apple Card features and products available only to Apple Card users, you must add Apple Card to Wallet on an iPhone or iPad that supports and has the latest version of iOS or iPadOS. Apple Card is subject to credit approval, available only for qualifying applicants in the United States, and issued by Goldman Sachs Bank USA, Salt Lake City Branch.\nIf you reside in the U.S. territories, please call Goldman Sachs at 877-255-5923 with questions about Apple Card.\n † Monthly pricing is available when you select Apple Card Monthly Installments (ACMI) as payment type at checkout at Apple, and is subject to credit approval and credit limit. Financing terms vary by product. Taxes and shipping are not included in ACMI and are subject to your card’s variable APR. See the Apple Card Customer Agreement (Opens in a new window)  for more information. ACMI is not available for purchases made online at special storefronts. The last month’s payment for each product will be the product’s purchase price, less all other payments at the monthly payment amount. ACMI financing is subject to change at any time for any reason, including but not limited to, installment term lengths and eligible products. See support.apple.com/kb/HT211204 (Opens in a new window)  for information about upcoming changes to ACMI financing.\n1. Special pricing available to qualified customers. To learn more about how to start qualifying toward special pricing, talk to an Apple Specialist in a store or give us a call at 1‑800‑MY‑APPLE.\n2. $9 two-hour delivery on eligible Apple products in most metros. Offer is not available on customized Mac, engraved products, and for certain order types including orders paid for with financing or by bank transfer. Delivery times vary according to your selected delivery address, availability of your items, and the time of day you place your order. Find a store to view local store hours or see checkout for estimated delivery. A signature is required for delivery. Drivers may ask for verbal confirmation of receipt from a safe distance to satisfy the signature requirement. See  https://www.apple.com/shop/shipping-pickup/ for more information.\n3. Trade-in values will vary based on the condition, year, and configuration of your eligible trade-in device. Not all devices are eligible for credit. You must be at least the age of majority to be eligible to trade in for credit or for an Apple Gift Card. Trade-in value may be applied toward qualifying new device purchase, or added to an Apple Gift Card. Actual value awarded is based on receipt of a qualifying device matching the description provided when estimate was made. Sales tax may be assessed on full value of a new device purchase. In-store trade-in requires presentation of a valid photo ID (local law may require saving this information). Offer may not be available in all stores, and may vary between in-store and online trade-in. Some stores may have additional requirements. Apple or its trade-in partners reserve the right to refuse, cancel, or limit quantity of any trade-in transaction for any reason. More details are available from Apple’s trade-in partner for trade-in and recycling of eligible devices. Restrictions and limitations may apply.\n4. AT&T iPhone 16 Special Deal: Monthly price (if shown) reflects net monthly payment, after application of AT&T trade-in credit applied over 36 months with purchase of an iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus and trade-in of eligible smartphone. Receive credit with purchase of an iPhone 16 Pro or iPhone 16 Pro Max of either $1000, $700, or $350 (based upon the model and condition of your trade-in smartphone). Receive credit with purchase of an iPhone 16 or iPhone 16 Plus of either $700 or $350 (based upon the model and condition of your trade-in smartphone). Max bill credits will not exceed the cost of the device. Requires upgrade of an existing line or activation of a new line and purchase of a new iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus on qualifying 36 month 0% APR installment plan, subject to carrier credit qualification. Customers purchasing this offer through Apple cannot add the Next Up Anytime option. $0 down for well qualified customers only, or down payment may be required and depends on a variety of factors. Tax on full retail price due at sale. Requires activation on eligible AT&T unlimited plan. AT&T may temporarily slow data speeds if the network is busy. If you cancel eligible wireless service, credits will stop and you will owe the remaining device balance. Activation/Upgrade Fee: $35. Trade in device may not be on existing installment plan. Bill credits are applied as a monthly credit over the 36 month installment plan. Credits start within 3 bills. Will receive catchup credits once credits start. Wireless line must be on an installment agreement, active, and in good standing for 30 days to qualify. Installment agreement starts when device is shipped. To get all credits, device must remain on agreement for entire term and you must keep eligible service on device for entire installment term. Limited time offer; subject to change. Limits: one trade-in per qualifying purchase and one credit per line. May not be combinable with other offers, discounts, or credits. Purchase, financing, other limits, and restrictions apply. Price for iPhone 16 and iPhone 16 Plus includes $30 AT&T connectivity discount. Activation required.\n  AT&T iPhone 15 Special Deal: Buy an iPhone 15 128 GB and get $334.36 in bill credits applied over 36 months. Buy an iPhone 15 256 GB and get $254.36 in bill credits applied over 36 months. Buy an iPhone 15 512 GB and get $274.36 in bill credits applied over 36 months. Requires upgrade of an existing line (or activation of a new line) and purchase on qualifying 36-month 0% APR installment plan, subject to carrier credit qualification. $0 down for well-qualified customers only, or down payment may be required and depends on a variety of factors. Tax on full retail price due at sale. Requires activation on eligible AT&T unlimited plan. AT&T may temporarily slow data speeds if the network is busy. If you cancel eligible wireless service, credits will stop and you will owe the remaining device balance. Activation/Upgrade Fee: $35. Bill credits are applied as a monthly credit over the 36-month installment plan. Credits start within 3 bills. Will receive catch-up credits once credits start. Wireless line must be on an installment agreement, active, and in good standing for 30 days to qualify. Installment agreement starts when device is shipped. To get all credits, device must remain on agreement for entire term and you must keep eligible service on device for entire installment term. Limited-time offer; subject to change. Limits: one credit per line. May not be combinable with other offers, discounts, or credits. Purchase, financing, other limits, and restrictions apply. Activation required.\n AT&T iPhone SE Special Deal: Buy an iPhone SE 64 GB and get $214.36 in bill credits applied over 36 months. Buy an iPhone SE 128 GB and get $84.36 in bill credits applied over 36 months. Buy an iPhone SE 256 GB and get $4.36 in bill credits applied over 36 months. Requires upgrade of an existing line (or activation of a new line) and purchase on qualifying 36-month 0% APR installment plan, subject to carrier credit qualification. $0 down for well-qualified customers only, or down payment may be required and depends on a variety of factors. Tax on full retail price due at sale. Requires activation on eligible AT&T unlimited plan. AT&T may temporarily slow data speeds if the network is busy. If you cancel eligible wireless service, credits will stop and you will owe the remaining device balance. Activation/Upgrade Fee: $35. Bill credits are applied as a monthly credit over the 36-month installment plan. Credits start within 3 bills. Will receive catch-up credits once credits start. Wireless line must be on an installment agreement, active, and in good standing for 30 days to qualify. Installment agreement starts when device is shipped. To get all credits, device must remain on agreement for entire term and you must keep eligible service on device for entire installment term. Limited-time offer; subject to change. Limits: one credit per line. May not be combinable with other offers, discounts, or credits. Purchase, financing, other limits, and restrictions apply. Activation required.\n Boost Mobile iPhone 16 Special Deal: Buy an iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus and get $1000 in bill credits (not to exceed the cost of the iPhone) applied over 36 months. No trade-in required. If you are trading in a device with this deal, trade-in value will be applied as additional bill credits over 36 months. Monthly price (if shown) reflects net monthly payment, after application of $1000 in bill credit (not to exceed the cost of the iPhone purchased) and trade-in credit (if applicable) applied over 36 months respectively. Requires activation of a new line, Boost Mobile Infinite Access plan and purchase on qualifying 36-month 0% APR installment plan, subject to carrier credit qualification. After making 12 installment payments, you may upgrade to a new iPhone and get up to $1000 in bill credits (not to exceed the cost of the iPhone) applied over 36 months for the new iPhone on the Infinite Access plan and purchase on new qualifying 36-month 0% APR installment plan, subject to carrier credit qualification. Tax on full retail price due at sale. If you cancel eligible wireless service, credits will stop and you will owe the remaining device balance. Bill credits are applied as a monthly credit over the 36-month installment plan. Trade-in credits start within 3 bills. Installment agreement starts when device is shipped. To get all credits, device must remain on agreement for entire term and you must keep eligible service on device for entire installment term. Limited-time offer; subject to change. Limits: one credit per line. May not be combined with other offers, discounts, or credits. Purchase, financing, other limits, and restrictions apply. Price for iPhone 16 and iPhone 16 Plus includes $30 Boost Mobile connectivity discount. Activation required.\n T-Mobile iPhone 16 Special Deal: Monthly price (if shown) reflects net monthly payment, after application of T-Mobile trade-in credit applied over 24 months with purchase of an iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus and trade-in of eligible smartphone. Receive credit with purchase of an iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus of $1000 or $500 for customers on a Go5G Next plan (based upon the model and condition of your trade-in smartphone); or $800 or $400 for customers on a Go5G Plus plan (based upon the model and condition of your trade-in smartphone). Offer excludes customers on Go5G Next First Responder, Go5G Plus First Responder, Go5G Next Military, Go5G Plus Military, Go5G Next 55, and Go5G Plus 55 plans. Max bill credits will not exceed the cost of the device. Credit comprised of (i) Apple instant trade-in credit at checkout and (ii) T-Mobile monthly bill credits applied over 24 months. Allow 2 bill cycles from valid submission and validation of trade-in. Tax on pre-credit price due at sale. Limited-time offer; subject to change. Qualifying credit, data plan, and trade-in in good condition required. Max 4 promotional offers on any iPhone per account. May not be combinable with some offers or discounts. Price for iPhone 16 and iPhone 16 Plus includes $30 T-Mobile connectivity discount. Activation required. Contact T-Mobile before cancelling service to continue remaining bill credits on current device, or credits stop & balance on required finance agreement is due. \n T-Mobile iPhone 15 Special Deal: Monthly price (if shown) reflects net monthly payment, after application of T-Mobile trade-in credit applied over 24 months with purchase of an iPhone 15 or iPhone 15 Plus and trade-in of eligible smartphone. Receive credit with purchase of an iPhone 15 or 15 Plus of $1000 or $500 for customers on a Go5G Next plan (based upon the model and condition of your trade-in smartphone); or $800 or $400 for customers on a Go5G Plus plan (based upon the model and condition of your trade-in smartphone). Offer excludes customers on Go5G Next First Responder, Go5G Plus First Responder, Go5G Next Military, Go5G Plus Military, Go5G Next 55, and Go5G Plus 55 plans. Max bill credits will not exceed the cost of the device. Credit comprised of (i) Apple instant trade-in credit at checkout and (ii) T-Mobile monthly bill credits applied over 24 months. Allow 2 bill cycles from valid submission and validation of trade-in. Tax on pre-credit price due at sale. Limited-time offer; subject to change. Qualifying credit, data plan, and trade-in in good condition required. Max 4 promotional offers on any iPhone per account. May not be combinable with some offers or discounts. Price for iPhone 15 and iPhone 15 Plus includes $30 T-Mobile connectivity discount. Activation required. Contact T-Mobile before cancelling service to continue remaining bill credits on current device, or credits stop & balance on required finance agreement is due. \n T-Mobile iPhone 14 Special Deal: Monthly price (if shown) reflects net monthly payment, after application of T-Mobile trade-in credit applied over 24 months with purchase of an iPhone 14 or iPhone 14 Plus and trade-in of eligible smartphone. Receive credit with purchase of an iPhone 14 or 14 Plus of $999 or $500 for customers on a Go5G Next plan (based upon the model and condition of your trade-in smartphone); or $800 or $400 for customers on a Go5G Plus plan (based upon the model and condition of your trade-in smartphone). Offer excludes customers on Go5G Next First Responder, Go5G Plus First Responder, Go5G Next Military, Go5G Plus Military, Go5G Next 55, and Go5G Plus 55 plans. Max bill credits will not exceed the cost of the device. Credit comprised of (i) Apple instant trade-in credit at checkout and (ii) T-Mobile monthly bill credits applied over 24 months. Allow 2 bill cycles from valid submission and validation of trade-in. Tax on pre-credit price due at sale. Limited-time offer; subject to change. Qualifying credit, data plan, and trade-in in good condition required. Max 4 promotional offers on any iPhone per account. May not be combinable with some offers or discounts. Price for iPhone 14 and iPhone 14 Plus includes $30 T-Mobile connectivity discount. Activation required. Contact T-Mobile before cancelling service to continue remaining bill credits on current device, or credits stop & balance on required finance agreement is due. \n T-Mobile iPhone SE 3 Special Deal: Monthly price (if shown) reflects net monthly payment, after application of T-Mobile trade-in credit applied over 24 months with purchase of an iPhone SE 3 and trade-in of eligible smartphone. Receive credit with purchase of an iPhone SE 3 of $579 or $500 for customers on a Go5G Next plan (based upon the model and condition of your trade-in smartphone); or $579 or $400 for customers on a Go5G Plus plan (based upon the model and condition of your trade-in smartphone). Offer excludes customers on Go5G Next First Responder, Go5G Plus First Responder, Go5G Next Military, Go5G Plus Military, Go5G Next 55, and Go5G Plus 55 plans. Max bill credits will not exceed the cost of the device. Credit comprised of (i) Apple instant trade-in credit at checkout and (ii) T-Mobile monthly bill credits applied over 24 months. Allow 2 bill cycles from valid submission and validation of trade-in. Tax on pre-credit price due at sale. Limited-time offer; subject to change. Qualifying credit, data plan, and trade-in in good condition required. Max 4 promotional offers on any iPhone per account. May not be combinable with some offers or discounts. Activation required. Contact T-Mobile before cancelling service to continue remaining bill credits on current device, or credits stop & balance on required finance agreement is due. \n Verizon iPhone 16 Special Deal: Monthly price (if shown) reflects net monthly payment, after application of Verizon trade-in credit applied over 36 months with purchase of an iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus. Customers on an Unlimited Ultimate plan receive: $1000 credit (based upon the model and condition of your trade-in smartphone) with purchase of an iPhone 16 Pro or iPhone 16 Pro Max; $930 credit (based upon the model and condition of your trade-in smartphone) with purchase of an iPhone 16 Plus; or $830 credit (based upon the model and condition of your trade-in smartphone) with purchase of an iPhone 16. Customers on an Unlimited Plus plan receive $730 credit (based upon the model and condition of your trade-in smartphone) with purchase of an iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus. Credit comprised of (i) Apple instant trade-in credit at checkout and (ii) Verizon monthly bill credits applied over 36 months. Customer must remain in the Verizon Device Payment Program for 36 months to receive the full benefit of the Verizon bill credits. Bill credits may take 1-2 bill cycles to appear. If it takes two cycles for bill credits to appear, you'll see the credit for the first cycle on your second bill in addition to that month's credit. Requires purchase and activation of a new iPhone 16 Pro, iPhone 16 Pro Max, iPhone 16, or iPhone 16 Plus with the Verizon Device Payment Program at 0% APR for 36 months, subject to carrier credit qualification, and iPhone availability and limits. Taxes and shipping not included in monthly price. Sales tax may be assessed on full value of new iPhone. Requires eligible unlimited service plan. Requires trade-in of eligible device in eligible condition. Must be at least 18 to trade-in. Apple or its trade-in partners reserve the right to refuse or limit any trade-in transaction for any reason. In-store trade-in requires presentation of a valid, government-issued photo ID (local law may require saving this information). In-store promotion availability subject to local law; speak to a Specialist to learn more. Limited-time offer; subject to change. Additional terms from Apple, Verizon, and Apple's trade-in partners may apply. Price for iPhone 16 and iPhone 16 Plus includes $30 Verizon connectivity discount. Activation required.\n⁺ New subscribers only. $10.99/month after trial. Offer is available for new Apple Music subscribers with a new eligible device for a limited time only. Offer redemption for eligible audio devices requires connecting or pairing to an Apple device running the latest iOS or iPadOS. Offer redemption for Apple Watch requires connecting or pairing to an iPhone running the latest iOS. Offer good for three months after eligible device activation. Only one offer per Apple ID, regardless of the number of eligible devices you purchase. Plan automatically renews until cancelled. Restrictions and other terms (Opens in a new window)  apply.\nApple Pay is a service provided by Apple Payments Services LLC, a subsidiary of Apple Inc. Neither Apple Inc. nor Apple Payments Services LLC is a bank. Any card used in Apple Pay is offered by the card issuer.\n We approximate your location from your internet IP address by matching it to a geographic region or from the location entered during your previous visit to Apple.",
          "image": "https://as-images.apple.com/is/og-default?wid=1200&hei=630&fmt=jpeg&qlt=95&.v=1525370171638"
        },
        {
          "id": "https://www.apple.com/mac/",
          "url": "https://www.apple.com/mac/",
          "title": "Mac",
          "author": "",
          "publishedDate": "2024-05-07T20:24:00.000Z",
          "text": "Answer calls or messages from your iPhone directly on your Mac. See and control what’s on your iPhone from your Mac with iPhone Mirroring. Use Universal Clipboard to copy images, video, or text from your iPhone, then paste into another app on your nearby Mac. And thanks to iCloud, you can access your files from either your iPhone or your Mac. And so much more.\nSketch on your iPad and have it appear instantly on your Mac. Or use your iPad as a second display, so you can work on one screen while you reference the other. You can even start a Final Cut Pro project on your iPad and continue it on your Mac.\nAutomatically log in to your Mac when you’re wearing your Apple Watch with Auto Unlock. No password typing required.",
          "image": "https://www.apple.com/v/mac/home/cb/images/meta/mac__c3zv0c86zu0y_og.png?202410291046"
        },
        {
          "id": "https://www.apple.com/ipad/",
          "url": "https://www.apple.com/ipad/",
          "title": "iPad",
          "author": "",
          "publishedDate": "2024-05-07T20:24:00.000Z",
          "text": "Get 3% Daily Cash back with Apple Card. And pay for your new iPad over 12 months, interest‑free when you choose to check out with Apple Card Monthly Installments. ◊  Learn more \niPad is perfect for taking the content you capture on iPhone and bringing it to life on an immersive canvas. You can shoot videos and photos on your iPhone and use the large display of your iPad to edit, add animations, and more. You can also pick up wherever you left off with Handoff.\niPad and Mac are designed to work together to form the ultimate creative setup. Sketch on your iPad and have it appear instantly on your Mac with Sidecar. Then use your iPad for drawing or editing with Apple Pencil or as a second display. Extend your workflow to new places, and when you return to your desk, Universal Control allows you to use one mouse or trackpad seamlessly across both devices.\niPad is a great way to optimize your workouts while tracking your progress on Apple Watch. See personal metrics from Apple Watch integrated on the screen of your iPad in real time. The sensors in Apple Watch combine with advanced algorithms to provide data that keeps you motivated. And see it all come together on your Health app on iPad.",
          "image": "https://www.apple.com/v/ipad/home/cm/images/meta/ipad__f350v51yy3am_og.png?202410241440"
        },
        {
          "id": "https://www.apple.com/iphone/",
          "url": "https://www.apple.com/iphone/",
          "title": "iPhone",
          "author": "",
          "publishedDate": "2024-05-07T20:24:00.000Z",
          "text": "Get credit toward iPhone 16 or iPhone 16 Pro when you trade in an eligible smartphone.    Get credit toward iPhone 16 or iPhone 16 Pro when you trade in an eligible smartphone.  *  Shop iPhone \nWith iPhone Mirroring, you can view your iPhone screen on your Mac and control it without picking up your phone. Continuity features also let you answer calls or messages right from your Mac. You can even copy images, video, or text from your iPhone and paste it all into a different app on your Mac. And with iCloud, you can access your files from either device.\nMisplaced your iPhone? The latest Apple Watch models can show you its approximate distance and direction. 14  To set up a group photo on your iPhone, join the group and use Apple Watch as a viewfinder to snap the shot. And when you take a call on your Apple Watch, just tap your iPhone to continue the conversation there.\nSet up AirPods on iPhone with just a tap. You’ll love Adaptive Audio, which automatically tailors the noise control for you to provide the best listening experience across different environments and interactions throughout the day.",
          "image": "https://www.apple.com/v/iphone/home/bx/images/meta/iphone__kqge21l9n26q_og.png?202410241440"
        },
        {
          "id": "https://www.apple.com/watch/",
          "url": "https://www.apple.com/watch/",
          "title": "Apple Watch",
          "author": "",
          "publishedDate": "2024-05-07T20:24:00.000Z",
          "text": "Combining Apple Watch and iPhone opens up a world of features that make each device better. You can do things like create a custom route with Maps on your iPhone, then download it to your watch to use any time. Or start a cycling workout on your watch and see your metrics automatically appear as a Live Activity on your iPhone.\nApple Watch supercharges your Fitness+ experience with real‑time, personalized metrics onscreen, like your heart rate, calories burned, and Activity rings. 26  And you get the freedom of audio‑guided walks, runs, and meditations with just your watch and AirPods.\nYou can do so much with just Apple Watch and AirPods — all without your iPhone. Take calls, stream music and podcasts, hear incoming notifications. Even respond to messages with Siri.",
          "image": "https://www.apple.com/v/watch/bo/images/meta/apple-watch__f6h72tjlgx26_og.png?202410031527"
        },
        {
          "id": "https://www.apple.com/apple-vision-pro/",
          "url": "https://www.apple.com/apple-vision-pro/",
          "title": "Apple Vision Pro",
          "author": "",
          "publishedDate": "2024-05-07T20:24:00.000Z",
          "text": "Apple Vision Pro seamlessly blends digital content with your physical space.\nSo you can work, watch, relive memories, and connect in ways never before possible.\nThe era of spatial computing is here.\n    Watch the film   \n \nExplore Apple Vision Pro\n \n \nFront\nCameras and sensors\nAudio Straps\nHead bands\nDisplays\nLight Seal\nDigital Crown\nTop button\nPower\nA singular piece of three-dimensionally formed laminated glass flows into an aluminum alloy frame that gently curves to wrap around your face.\nAn array of advanced cameras and sensors work together to let you see the world clearly, understand your environment, and detect hand input.\nSpeakers are positioned close to your ears, delivering rich Spatial Audio that seamlessly blends with real-world sounds.\nTwo head bands are included. The Solo Knit Band provides cushioning, breathability, and stretch, and a Fit Dial lets you adjust Apple Vision Pro to your head. The Dual Loop Band features a pair of adjustable upper and lower straps for a precise fit.\nA pair of custom micro‑OLED displays deliver more pixels than a 4K TV to each eye — for stunning clarity.\nThe Light Seal gently conforms to your face, delivering a precise fit while blocking out stray light.\nPress the Digital Crown to bring up the Home View, and turn it to control your level of immersion while using Environments.\nPress the top button to take spatial videos and spatial photos in the moment.\nThe external battery supports up to 2 hours of general use and up to 2.5 hours of video playback. 1  \nClick or tap each tab to explore Apple Vision Pro.\nEntertainment\nThe ultimate theater.Wherever you are.\nA new dimension for entertainment.\nApple Vision Pro can transform any room into your own personal theater. Expand your movies, shows, and games to your perfect size and experience them in Spatial Audio. Apple Immersive Video puts you in the center of the action with mind‑blowing immersion. And with more pixels than a 4K TV for each eye, you can enjoy stunning content wherever you are — on a long flight or the couch at home.\nProductivity\nA workspace with infinite space.\nDiscover new ways to work.\nApple Vision Pro gives you limitless space to get things done. Organize everything you need anywhere around you, in any way you like. Seamlessly bring in your Mac workflows using Mac Virtual Display. Connect a Magic Keyboard, a Magic Trackpad, and other Bluetooth accessories to expand how you navigate. And with SharePlay in FaceTime, you can collaborate with colleagues using apps together in real time.\nPhotos and Videos\nBe in the moment.All over again.\nYour memories come alive.\nApple Vision Pro is Apple’s first 3D camera. You can capture magical spatial photos and spatial videos in 3D, then relive those cherished moments like never before with immersive Spatial Audio. Your existing library of photos and videos looks incredible at remarkable scale — and now you can transform your 2D photos into spatial photos with just a tap. Even panoramas wrap around you — making you feel like you’re standing right where you took them. You can also take spatial videos with iPhone 16 Pro, iPhone 16, or iPhone 15 Pro, as well as spatial photos with iPhone 16 Pro or iPhone 16, then view them on Apple Vision Pro.\nConnection\nShare quality time.And space.\nA more engaging way to get together.\nApple Vision Pro makes it easy to collaborate and connect wherever you are. You can see FaceTime participants in life-size video tiles, or you can choose to use your spatial Persona and feel like you are sharing the same space with others. And use SharePlay to watch, listen, and play together with your favorite people.\nApps\nDo what you love. Reimagine how you do it.\nA world of apps. A world of discovery.\nApple Vision Pro expands the experience of your go‑to apps and opens up new possibilities in entertainment, productivity, gaming, and more. Browse the web in Safari, create a to‑do list in Notes, chat in Messages, and seamlessly move between them with a glance. And explore the App Store to discover an ever-expanding collection of awe-inspiring spatial apps designed for Apple Vision Pro.\n Visit the App Store    \nvisionOS\nAn operating system designed for spatial.\nNavigate spatial experiences. Naturally.\nBuilt on the foundation of macOS, iOS, and iPadOS, visionOS enables powerful spatial experiences. Control Apple Vision Pro with your eyes, hands, and voice — interactions feel intuitive and magical. Simply look at an element, tap your fingers together to select, and use the virtual keyboard or dictation to type. And visionOS 2 delivers even more ways to enhance work, entertainment, and connecting with friends and family using Apple Vision Pro.\n Learn more about visionOS 2    \nDesign\nDesigned by Apple.\nApple Vision Pro is the result of decades of experience designing high‑performance, mobile, and wearable devices — culminating in the most ambitious product Apple has ever created. Apple Vision Pro integrates incredibly advanced technology into an elegant, compact form, resulting in an amazing experience every time you put it on.\nTechnology\nInnovation you can see, hear, and feel.\nPushing boundaries from the inside out.  Spatial experiences on Apple Vision Pro are only possible through groundbreaking Apple technology. Displays the size of a postage stamp that deliver more pixels than a 4K TV to each eye. Incredible advances in Spatial Audio. A revolutionary dual‑chip design featuring custom Apple silicon. A sophisticated array of cameras and sensors. All the elements work together to create an unprecedented experience you have to see to believe.  \nMore pixels than a 4K TV. For each eye.\nThe custom micro‑OLED display system features 23 million pixels, delivering stunning resolution and colors. And a specially designed three‑element lens creates the feeling of a display that’s everywhere you look.\nOur most advanced Spatial Audio system ever.\nDual-driver audio pods positioned next to each ear deliver personalized sound while letting you hear what’s around you. Spatial Audio makes sounds feel like they’re coming from your surroundings. Audio ray tracing analyzes your room’s acoustic properties to adapt and match sound to your space. And if you want to use headphones with Apple Vision Pro, AirPods Pro 2 with USB‑C and AirPods 4 offer the perfect experience — featuring Lossless Audio with ultra-low latency, supported by the H2‑to‑H2 connection across devices.\nResponsive, precision eye tracking.\nA high‑performance eye‑tracking system of LEDs and infrared cameras projects invisible light patterns onto each eye. This advanced system provides ultraprecise input without your needing to hold any controllers, so you can accurately select elements just by looking at them.\nA sophisticated sensor array.\nA pair of high-resolution cameras transmit over one billion pixels per second to the displays so you can see the world around you clearly. The system also helps deliver precise head and hand tracking and real‑time 3D mapping, all while understanding your hand gestures from a wide range of positions.\nRevolutionary dual‑chip performance.\nA unique dual‑chip design enables the spatial experiences on Apple Vision Pro. The powerful M2 chip simultaneously runs visionOS, executes advanced computer vision algorithms, and delivers stunning graphics, all with incredible efficiency. And the brand-new R1 chip is specifically dedicated to process input from the cameras, sensors, and microphones, streaming images to the displays within 12 milliseconds — for a virtually lag-free, real-time view of the world.\nValues\nDesigned to make a difference.\nOur values lead the way.  Apple Vision Pro was designed to help protect your privacy and keep you in control of your data. Its built‑in accessibility features are designed to work the way you do.  \nUse AR to view Apple Vision Pro.\nOpen this page using Safari on your iPhone or iPad.\nExplore Apple Vision Pro accessories.\n  Shop    \nAn all‑new platform.An all‑new world for developers.\nThe possibilities for what developers can dream up and build for Apple Vision Pro are endless. And with familiar tools and frameworks like Xcode, SwiftUI, RealityKit, and ARKit, as well as support for Unity and the 3D-content preparation app Reality Composer Pro, developers have everything they need to create amazing spatial experiences.\n  Learn more about developing for visionOS",
          "image": "https://www.apple.com/v/apple-vision-pro/e/images/meta/apple-vision-pro-us__f28gp8ey4vam_og.png?202409261242"
        },
        {
          "id": "https://www.apple.com/airpods/",
          "url": "https://www.apple.com/airpods/",
          "title": "AirPods",
          "author": "",
          "publishedDate": "2024-09-27T17:22:17.000Z",
          "text": "AirPods Pro 2 now feature a scientifically validated Hearing Test and clinical‑grade Hearing Aid capability. 1  Learn more \nAirPods 4\nThe next evolution of sound and comfort.\nAirPods Pro 2\nThe world’s first all-in-one hearing health experience.\nFeatures available with a free software update.\nAirPods Max\nFive fresh colors. Bold sound.\nWhich AirPods areright for you?\n    Active Noise Cancellation, Adaptive Audio, and Transparency mode unavailable\n \n \n    Personalized Spatial Audio with dynamic head tracking  ◊◊◊   \n  \n    Hearing Test, Hearing Aid feature, and Hearing Protection unavailable\n \n \n     Voice Isolation,  ΔΔΔ   Hey Siri,and Siri Interactions  ΔΔΔ   \n  \n \n     Charging Case (USB‑C)  ΔΔΔΔ   \n  \n New  AirPods 4\nActive Noise Cancellation\nThe next evolution of sound, comfort, and noise control.\nBuy\n Learn more    \nView in AR\n \n    Active Noise Cancellation, Adaptive Audio, and Transparency mode  ◊   \n  \n \n    Personalized Spatial Audio with dynamic head tracking  ◊◊◊   \n  \n    Hearing Test, Hearing Aid feature, and Hearing Protection unavailable\n \n \n     Voice Isolation,  ΔΔΔ   Hey Siri,and Siri Interactions  ΔΔΔ   \n  \n \n    Wireless Charging Case (USB‑C)  ΔΔΔΔ   with speaker for Find My\n  \n \n    Up to 2x more Active Noise Cancellation,  ◊◊   with Adaptive Audio and Transparency mode  ◊   \n  \n \n    Personalized Spatial Audio with dynamic head tracking  ◊◊◊   \n  \n \n     Hearing Test,  Δ   Hearing Aid,  Δ   and Hearing Protection  ΔΔ   features\n  \n \n     Voice Isolation,  ΔΔΔ   Hey Siri,and Siri Interactions  ΔΔΔ   \n  \n \n    Wireless Charging Case (USB‑C) with MagSafe,  ΔΔΔΔ   speaker for Find My with Precision Finding,  ΔΔΔΔΔ   and lanyard loop\n  \n \n    Up to 2x more Active Noise Cancellation,  ◊◊   with Transparency mode\n  \n \n    Personalized Spatial Audio with dynamic head tracking  ◊◊◊   \n  \n    Hearing Test, Hearing Aid feature, and Hearing Protection unavailable\n \n Compare all AirPods models    \n Get to know AirPods.\n \nApple Music\nGet 3 months of Apple Music free with your AirPods. * \n  Learn more",
          "image": "https://www.apple.com/v/airpods/x/images/meta/airpods__dh7xkbort402_og.png?202410241631"
        },
        {
          "id": "https://www.apple.com/tv-home/",
          "url": "https://www.apple.com/tv-home/",
          "title": "TV & Home",
          "author": "",
          "publishedDate": "2024-05-07T20:24:00.000Z",
          "text": "The future hits home.\nSimply connect your favorite devices and transform your house into a remarkably smart, convenient, and entertaining home. Elevate movie night with theater-like picture and sound. Play any song, in any room, from anywhere. And control lights, locks, and thermostats using Siri. All with the security and privacy of Apple.\nHomePod mini\nSurprising sound for its size.\nThe Apple experience.Cinematic in every sense.\nHome app\nThe foundation for a smarter home.\n Every reason to turn your house into a smart home.\n \nHey Siri, set my bedtime scene\nHey Siri, make it warmer\nHey Siri, turn off the lights downstairs\nNanoleaf A19 Bulb\nLogitech Circle View Wired Doorbell\nComfort\nTurn up the heat or keep your cool with temperature controls and fans.\n  Shop Thermostats     \necobee Smart Thermostat Premium with Siri and Built‑In Air Quality Monitor\nLevel Lock+ with Home Key Support\n Watch, sing, play, and work out. On the big screen.",
          "image": "https://www.apple.com/v/tv-home/n/images/meta/tv-home__fedwm0ly3mqi_og.png?202409151638"
        }
      ]
    }
  ],
  "requestId": "17e8a79ff11bcb73115ef3efcb8e0457"
}

results = exa.get_contents(["https://medium.com"], subpages=5, subpage_target=["blog", "articles"], livecrawl='always')

result = exa.get_contents(
  ["https://www.apple.com/"],
  livecrawl="always",
  subpage_target=["news", "product"],
  subpages=10
)