Gofetch

An SSE Go implementation of the gofetch MCP server that retrieves web content.

Features

• Web Content Retrieval: gofetches URLs and extracts textual content
• Content Extraction: Extract main content from web pages
• Robots.txt Compliance: Respects robots.txt rules (can be disabled)
• Configurable: Supports custom user agents and proxy settings

Why this fetch and not mcp/fetch?

This Go implementation provides similar functionality as the original Python MCP fetch server but has the following benefits:

• Lower memory usage
• Faster startup time / shutdown time
• Single binary deployment - making tool poisoning attacks harder than in interpreted languages
• Better concurrent request handling
• Better container security than original
  • Non-root user
  • Distroless / minimal image
  • Container signing with build provenance
• Uses StreamableHTTP and SSE (deprecated) transport instead of STDIO
• More test coverage

Prerequisites

• Go 1.24 or later
• Task for running tasks

Installation

1. Clone the repository:

   git clone https://github.com/StacklokLabs/gofetch.git
   cd gofetch
   

2. Install dependencies:

   task install
   

3. Build the server:

   task build
   

Usage

Running the server

To run the server with the default StreamableHTTP transport:

./build/gofetch

The server will start and expose endpoints as mandated by the MCP specification:

Streamable HTTP Transport (default):

• MCP endpoint: http://localhost:8080/mcp (for streaming responses and commands)

SSE Transport:

• SSE endpoint: http://localhost:8080/sse
• Messages endpoint: http://localhost:8080/messages

Command Line Options

• --transport: Transport type: sse or streamable-http (default)
• --port: Port number for HTTP-based transports (default: 8080)
• --user-agent: Custom User-Agent string (default: "Mozilla/5.0 (compatible; MCPGoFetchBot/1.0)")
• --ignore-robots-txt: Ignore robots.txt rules
• --proxy-url: Proxy URL for requests

Examples

# Basic server on port 8080
./build/gofetch --transport=streamable-http --port 8080

# Custom port with user agent
./build/gofetch --port 8080 --user-agent "MyBot/1.0"

# Ignore robots.txt on custom port
./build/gofetch --port 8080 --ignore-robots-txt

# Use proxy
./build/gofetch --port 8080 --proxy-url "http://proxy.example.com:8080"

# Use environment variable for port
MCP_PORT=9090 ./build/gofetch

Using the Server

For using the gofetch server, you can follow the usage guide for curl commands:

• SSE
• StreamableHTTP

MCP Tools

The server provides a single tool called fetch with the following parameters:

Tool: fetch

Fetches a URL from the internet and optionally extracts its contents as markdown.

Parameters

• url (required): The URL to fetch
• max_length (optional): Maximum number of characters to return (default: 5000, max: 1000000)
• start_index (optional): Starting character index for content extraction (default: 0)
• raw (optional): Return raw HTML content without simplification (default: false)

Examples

{
  "name": "fetch",
  "arguments": {
    "url": "https://example.com"
  }
}

{
  "name": "fetch",
  "arguments": {
    "url": "https://example.com",
    "max_length": 1000,
    "raw": false
  }
}

Development

Running tests

Requires yardstick client to be installed for integration tests.

task test

Formatting code

task fmt

Linting code

task lint

Updating dependencies

task deps

Running as an MCP Server with ToolHive

gofetch can be run as a Model Context Protocol (MCP) server using ToolHive, which simplifies the deployment and management of MCP servers.

Prerequisites

1. Install ToolHive by following the installation instructions.
2. Ensure you have Docker or Podman installed on your system.

Running fetch with ToolHive (Recommended)

The easiest way to run gofetch is using the packaged version available in ToolHive's registry:

# Register a supported client so ToolHive can auto-configure your environment
thv client setup

# Run the fetch server
thv run fetch --transport streamable-http --target-port 8080

# List running servers
thv list

# Get detailed information about the server
thv registry info fetch

Advanced Usage with Custom Configuration

For advanced users who need custom configuration, you can also run fetch using the container image directly:

# Run the fetch server using the published container image
thv run --name fetch --transport streamable-http --target-port 8080 ghcr.io/stackloklabs/gofetch/server:latest

This command:

• Names the server instance "fetch"
• Uses the SSE transport protocol
• Uses the latest published fetch image from GitHub Container Registry

To use a specific version instead of the latest:

thv run --name fetch --transport sse --target-port 8080 ghcr.io/stackloklabs/gofetch/server:v0.0.1

Managing the gofetch Server

To verify that the gofetch server is running:

thv list

This will show all running MCP servers managed by ToolHive, including the gofetch server.

To stop the gofetch server:

# For custom named version
thv stop fetch

To remove the server instance completely:

# For custom named version
thv rm fetch

Contributing

We welcome contributions to this MCP server! If you'd like to contribute, please review the CONTRIBUTING guide for details on how to get started.

If you run into a bug or have a feature request, please open an issue in the repository or join us in the #mcp-servers channel on our community Discord server.

License

This project is licensed under the Apache v2 License - see the LICENSE file for details.