How to scrape H
acker News
with AgentQL

Looking for a better way to scrape Hacker News? Say goodbye to fragile XPath or DOM selectors that easily break with website updates. AI-powered AgentQL ensures consistent web scraping across various platforms, from Hacker News to any other website, regardless of UI changes.

Not just for scraping Hacker News

Smart selectors work anywhere

https://thehackernews.com

URL

Input any webpage.

{
  articles[] {
    title
    author
    date
    summary
  }
}

Query

Describe data in natural language.

{
  "articles": [
    {
      "title": "New Vulnerability in Zoom Allows Attackers to Bypass Security",
      "author": "The Hacker News",
      "date": "2024-04-26",
      "summary": "A recently discovered vulnerability in Zoom allows attackers to bypass security measures and gain access to user data."
    },
    {
      "title": "Critical Flaw Found in Widely Used IoT Devices",
      "author": "Jane Doe",
      "date": "2024-04-25",
      "summary": "Researchers have identified a critical flaw in a popular line of IoT devices that could allow attackers to take control."
    },
    {
      "title": "Major Data Breach Exposes Millions of User Records",
      "author": "John Smith",
      "date": "2024-04-24",
      "summary": "A major data breach has exposed millions of user records, including sensitive personal information."
    },
    {
      "title": "New Malware Targeting Financial Institutions",
      "author": "Alice Johnson",
      "date": "2024-04-23",
      "summary": "Cybersecurity experts warn of a new malware variant specifically targeting financial institutions and their customers."
    }
  ]
}

Returns

Receive accurate output in seconds.

How to use AgentQL on Hacker News

A dotted lineA blue lineA blue line
1

Install the SDK

Install code for JS and Python

npm install agentql

pip3 install agentql

2

Test and refine

Use the query debugger

3

Run your script

Install code for both JS and Python

agentql init

python example.py

Get started

Holds no opinions on what’s and how’s. Build whatever makes sense to you.