huakunshen/google-trends-crawler
A simple rest server for scraping google trends table.
5
GitHub: https://github.com/HuakunShen/GoogleTrendsScraper.git
This is a simple Hono server with a google trends crawler.
Send any google trends url to the /
endpoint with url
in the post request body, data in array of rows will be returned.
docker build --platform=linux/amd64 . -t huakunshen/google-trends-crawler # Build the Image
docker run --rm -p 3000:3000 huakunshen/google-trends-crawler # Sample Deploy
Sample Post Request
curl --location 'http://localhost:3000' \
--header 'Content-Type: application/json' \
--data '{
"url": "https://trends.google.com/trending?geo=CA&sort=title"
}' | jq
Each request result takes ~8s, so this is not an API to run frequently.
Ideal use case: host this in some event-triggered containers like GCP Cloud Run.
Every time you get a response from the crawler, cache the response for a while (e.g. 1 day).
docker pull huakunshen/google-trends-crawler