jeeva420/mcp-web-search

By jeeva420

•Updated 7 months ago

MCP server for web search (SearXNG) and scraping/extraction (Crawl4AI) over Streamable HTTP.

Image

Machine learning & AI

620

Overview Tags

jeeva420/mcp-web-search repository overview

⁠mcp-web-search

MCP server for web search, scraping, and LLM-driven data extraction over Streamable HTTP.

Provides three tools to any MCP-compatible client (Claude Desktop, Cursor, etc.):

Tool	Description
`web_search`	Search the web via a SearXNG instance
`web_scrape`	Scrape any URL to clean Markdown using headless Chromium
`web_extract`	Extract structured data from a URL using an LLM

⁠Prerequisites

SearXNG is not included in this image. You must run your own SearXNG⁠ instance separately and provide its URL via the SEARXNG_URL environment variable. Without it, the web_search tool will not work. The web_scrape and web_extract tools work independently of SearXNG.

⁠Quick Start

docker run -p 8100:8100 \
  -e SEARXNG_URL=https://your-searxng-instance.com \
  jeeva420/mcp-web-search

The MCP endpoint is available at http://localhost:8100/mcp.

⁠Environment Variables

Variable	Required	Default	Description
`SEARXNG_URL`	Yes		URL of your SearXNG instance
`WEB_MCP_PORT`	No	`8100`	Server port
`WEB_MCP_HOST`	No	`0.0.0.0`	Server bind address
`WEB_MCP_API_KEY`	No		API key for bearer token auth (disabled when unset)
`LLM_MODEL`	For `web_extract`	`openai/gpt-4o-mini`	LLM model identifier
`LLM_API_KEY`	For `web_extract`		LLM API key
`LLM_BASE_URL`	No		Custom OpenAI-compatible API base URL

⁠Examples

⁠Basic (search + scrape only)

docker run -p 8100:8100 \
  -e SEARXNG_URL=https://your-searxng-instance.com \
  jeeva420/mcp-web-search

⁠With authentication

docker run -p 8100:8100 \
  -e SEARXNG_URL=https://your-searxng-instance.com \
  -e WEB_MCP_API_KEY=my-secret \
  jeeva420/mcp-web-search

⁠With LLM extraction support

docker run -p 8100:8100 \
  -e SEARXNG_URL=https://your-searxng-instance.com \
  -e LLM_MODEL=openai/gpt-4o-mini \
  -e LLM_API_KEY=sk-... \
  jeeva420/mcp-web-search

⁠Docker Compose

services:
  web-mcp:
    image: jeeva420/mcp-web-search
    ports:
      - "8100:8100"
    environment:
      - SEARXNG_URL=https://your-searxng-instance.com
      - WEB_MCP_API_KEY=my-secret
    restart: unless-stopped

⁠MCP Client Configuration

⁠Claude Desktop / Cursor

{
  "mcpServers": {
    "web-search": {
      "type": "streamable-http",
      "url": "http://localhost:8100/mcp"
    }
  }
}

⁠With authentication

{
  "mcpServers": {
    "web-search": {
      "type": "streamable-http",
      "url": "http://localhost:8100/mcp",
      "headers": {
        "Authorization": "Bearer my-secret"
      }
    }
  }
}

⁠Verify It Works

# List available tools
curl -s http://localhost:8100/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | python -m json.tool

# Test search
curl -s http://localhost:8100/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"web_search","arguments":{"query":"hello world"}}}'

⁠Architecture

Transport: Streamable HTTP (stateless) — no stdio, works over the network
Search: Delegates to a self-hosted SearXNG⁠ instance
Scraping: Headless Chromium via Crawl4AI⁠ — handles JavaScript-rendered pages
Extraction: LLM-driven via any OpenAI-compatible API
Platforms: linux/amd64, linux/arm64

Tag summary

Recent tags

Content type

Image

Digest

sha256:1b2f30033…

Size

655.4 MB

Last updated

7 months ago

docker pull jeeva420/mcp-web-search