huakunshen/google-trends-crawler

By huakunshen

Updated 2 months ago

A simple rest server for scraping google trends table.

Image

5

Google Trends Scraper

GitHub: https://github.com/HuakunShen/GoogleTrendsScraper.git

This is a simple Hono server with a google trends crawler.

Send any google trends url to the / endpoint with url in the post request body, data in array of rows will be returned.

docker build --platform=linux/amd64 . -t huakunshen/google-trends-crawler # Build the Image
docker run --rm -p 3000:3000 huakunshen/google-trends-crawler # Sample Deploy

Sample Post Request

curl --location 'http://localhost:3000' \
--header 'Content-Type: application/json' \
--data '{
    "url": "https://trends.google.com/trending?geo=CA&sort=title"
}' | jq

Each request result takes ~8s, so this is not an API to run frequently.

Ideal use case: host this in some event-triggered containers like GCP Cloud Run.

Every time you get a response from the crawler, cache the response for a while (e.g. 1 day).

Docker Pull Command

docker pull huakunshen/google-trends-crawler