1. Conduid
  2. Scraping
  3. Researchpowerpack Http
MCP server · Scraping

Researchpowerpack Http

The ultimate research MCP toolkit: Reddit mining, web search with CTR aggregation, and intelligent web scraping - all in one modular package

37Low

Scored 4 months ago · breakdown

About Researchpowerpack Http

Researchpowerpack Http is an MCP server in the Scraping category: the ultimate research MCP toolkit: Reddit mining, web search with CTR aggregation, and intelligent web scraping - all in one modular package. It has been installed 0 times through Conduid.

Install

Claude Code
claude mcp add researchpowerpack-http -- npx -y mcp-researchpowerpack-http
npx
npx -y mcp-researchpowerpack-http

This server has no ConduID identity, so agent calls to it are not receipted. Pin the version you install and review the source before granting it credentials.

Ask AI

Ask AI about Researchpowerpack Http

Powered by Claude · Grounded in docs

I know everything about Researchpowerpack Http. Ask me about installation, configuration, usage, or troubleshooting.

Security checks

  • ·README presentNot checked yet.
  • ·License declaredNot checked yet.
  • ·Tests presentNot checked yet.
  • ·Dependencies pinnedNot checked yet.
  • ·No dynamic code executionNot checked yet.
  • ·Scoped permissionsNot checked yet.

README

MseeP.ai Security Assessment Badge

mcp-researchpowerpack

http mcp server for research. five tools, orientation-first, built for agents that run multi-pass research loops.

ships on mcp-use. tool-level provider and llm orchestration flows through effect ts service layers for typed concurrency, typed errors, scoped kernel cleanup, and step timeouts. provider clients remain small promise/sdk adapters behind those layers. no stdio — http only.

tools

tool what it does needs
start-research returns a goal-tailored brief: primary_branch (reddit / web / both), exact first_call_sequence, 25–50 keyword seeds, iteration hints, gaps to watch, stop criteria. call first every session. LLM_API_KEY + LLM_BASE_URL + LLM_MODEL for the non-degraded brief (optional — server falls back to a static playbook)
raw-web-search parallel search, up to 50 keywords per call. serper is primary; jina search is the fallback when serper is missing, fails, or returns empty. returns the raw ranked markdown list directly — no llm pass. use for broad discovery, audit trails, and reddit permalink probes via explicit site:reddit.com/r/.../comments. SERPER_API_KEY or JINA_API_KEY
smart-web-search parallel search, up to 50 keywords per call, plus required extract. same provider order as raw. always runs llm classification and returns tiered markdown (HIGHLY_RELEVANT / MAYBE_RELEVANT / OTHER) + grounded synthesis + gaps + refine suggestions. supports scope: "web" | "reddit" | "both". SERPER_API_KEY or JINA_API_KEY + llm env triple
raw-scrape-links fetch urls in parallel, return full markdown directly. reddit post permalinks route through the reddit api with threaded comments. non-reddit urls hit jina reader first, then jina reader through scrape.do proxy mode, then optional kernel browser rendering for web pages. pdf / docx / pptx / xlsx urls go straight through jina reader. optional REDDIT_CLIENT_ID / REDDIT_CLIENT_SECRET, SCRAPEDO_API_KEY, JINA_API_KEY, KERNEL_API_KEY
smart-scrape-links same fetch stack as raw scrape, then per-url llm extraction with required extract. returns focused evidence packs with ## Source, ## Matches, ## Not found, and ## Follow-up signals. raw scrape providers + llm env triple

also exposes /health (simplified for proxies) and health://status (full json: planner/extractor reachability, consecutive failure counters, uptime, active sessions).

workflow

call start-research once per session with your goal. the server returns a brief that names the first tool to fire (reddit-first for sentiment/migration, web-first for spec/bug/pricing, both when opinion-heavy and you also need official sources), the keyword seeds to fan out, and stop criteria.

for search fan-out, think bad → better before calling raw-web-search or smart-web-search. turn broad phrases like <feature> support, <product> pricing, <library> bug fix, or <tool> reviews into source-aware probes like:

  • site:<official-docs-domain> "<feature>" "<platform-or-version>"
  • site:<vendor-domain> "<product>" pricing "enterprise" OR "free tier"
  • "<exact error text>" "<library-or-package>" "<version>" site:github.com
  • site:reddit.com/r/<community>/comments "<tool>" "migration" OR "regression"

pair the server with the run-research skill for the full agentic playbook:

npx -y skills add -y -g https://github.com/yigitkonur/skills-by-yigitkonur --skill /run-research

quickstart

# from npm
HOST=127.0.0.1 PORT=3000 npx -y mcp-researchpowerpack

# from source
git clone https://github.com/yigitkonur/mcp-researchpowerpack.git
cd mcp-researchpowerpack
pnpm install && pnpm dev

point your client at http://localhost:3000/mcp:

{
  "mcpServers": {
    "research-powerpack": {
      "url": "http://localhost:3000/mcp"
    }
  }
}

or skip the install entirely and hit the hosted deployment at https://research-mcp.yigitkonur.com/mcp.

config

copy .env.example, set only what you need. blank/whitespace keys are treated as absent. missing keys don't crash the server — they disable the affected capability with a clear error at call time.

server

var default
PORT 3000 http port
HOST 127.0.0.1 bind address; cloud runtimes that set PORT auto-switch to 0.0.0.0. public binds require ALLOWED_ORIGINS, MCP_URL, CSP_URLS, or FLY_APP_NAME
ALLOWED_ORIGINS unset comma-separated origins for host validation / cors; merged with MCP_URL and platform CSP_URLS when present
MCP_URL unset public mcp url; contributes its origin to host validation and well-known resource urls
CSP_URLS unset platform-provided comma-separated public origins; also contributes to host validation, including the derived mcp-use --br-main host
FLY_APP_NAME unset Fly runtime app name; when present, https://<app>.fly.dev is added to host validation for Manufact's deploy verifier
MCP_DISABLE_OAUTH_METADATA unset local loopback only; set to 1, true, yes, or on to skip .well-known/oauth-protected-resource routes for anonymous local mcpc --no-profile tests
NODE_ENV unset production also requires ALLOWED_ORIGINS, MCP_URL, CSP_URLS, or FLY_APP_NAME, even on a local bind
DEBUG unset 1 or 2 to bump mcp-use debug verbosity

providers

var enables
SERPER_API_KEY primary raw/smart web search provider
SCRAPEDO_API_KEY scrape.do proxy-mode retry for jina reader (X-Proxy-Url)
REDDIT_CLIENT_ID + REDDIT_CLIENT_SECRET raw/smart scrape for reddit.com permalinks (threaded post + comments)
JINA_API_KEY jina search fallback and authenticated jina reader requests
KERNEL_API_KEY optional kernel browser-render fallback after jina direct + proxy fail
KERNEL_PROJECT optional kernel project scoping header for org-wide api keys
LLM_API_KEY + LLM_BASE_URL + LLM_MODEL goal-tailored brief, smart-web-search, smart-scrape-links

llm

any openai-compatible endpoint. LLM_API_KEY, LLM_BASE_URL, and LLM_MODEL are required together. reasoning effort is hardcoded to low.

var required?
LLM_API_KEY yes api key for the endpoint
LLM_BASE_URL yes base url for the openai-compatible endpoint (e.g. https://server.up.railway.app/v1)
LLM_MODEL yes primary model (e.g. gpt-5.4-mini)
LLM_FALLBACK_MODEL no model to use after primary exhausts retries — gets 3 more attempts (e.g. gpt-5.4). also receives oversized inputs that exceed the primary's context window

concurrency

all optional. provider limits are clamped 1–200; kernel is clamped 1–20. malformed numeric values are rejected instead of partially parsed.

var default controls
CONCURRENCY_SEARCH 50 parallel serper / jina search queries
CONCURRENCY_SCRAPER 50 parallel scrape.do (proxy mode) requests
CONCURRENCY_JINA_READER 50 parallel jina reader fetches
CONCURRENCY_REDDIT 50 parallel reddit api fetches
CONCURRENCY_KERNEL 3 parallel kernel browser-render fallbacks
LLM_CONCURRENCY 50 parallel llm extraction / classification calls

evals

pnpm test:evals writes a json artifact to test-results/eval-runs/<timestamp>.json. when an openai api key is present, it runs a live responses-api + remote-mcp eval. without one, it exits successfully in explicit skip mode and records the skip in the artifact.

useful env vars:

  • EVAL_MCP_URL
  • EVAL_MODEL
  • EVAL_API_KEY or OPENAI_API_KEY

mcpc smoke tests

pnpm test:mcpc:contract builds the package, starts the compiled server on a free loopback port, connects with the pinned @apify/mcpc CLI, and verifies the deterministic client-visible MCP contract: tools, prompts, resources, degraded start-research, and validation failures. It does not need provider credentials.

pnpm test:mcpc runs the deterministic mcpc contract smoke.

See docs/testing/mcpc-e2e.md for the harness lifecycle, environment isolation contract, and the deterministic MCP contract coverage model.

dev

pnpm install
pnpm dev          # watch mode, serves :3000/mcp
pnpm typecheck    # tsc --noEmit
pnpm test:unit    # deterministic unit tests
pnpm test:http    # below-mcp server startup / http checks
pnpm test:mcpc:contract  # deterministic built-server mcp contract smoke
pnpm test:mcpc    # deterministic built-server mcp contract smoke
pnpm test         # unit + http + deterministic mcpc contract tests
pnpm build        # compile to dist/
pnpm inspect      # mcp-use inspector

prepublishOnly runs pnpm typecheck, pnpm test:unit, pnpm test:http, pnpm test:mcpc:contract, and pnpm build, matching the required publish gate without requiring provider credentials.

deploy

deploy to manufact cloud via the mcp-use cli (github-backed):

pnpm deploy       # runs the package script: mcp-use deploy

the canonical hosted endpoint is https://research-mcp.yigitkonur.com/mcp. the raw manufact server slug is calm-wave-3gtvb.

or self-host anywhere with node 20.19+ / 22.12+:

HOST=0.0.0.0 ALLOWED_ORIGINS=https://app.example.com pnpm start
# or derive production origin protection from the public mcp url:
NODE_ENV=production MCP_URL=https://research.example.com/mcp pnpm start

architecture

index.ts                 server startup, cors, health, shutdown
src/
  config/                env parsing, capability detection, lazy proxy config
  effect/                typed service tags + Live layers; runExternalEffect()
                         only accepts fully-provided programs at the async
                         tool boundary
  clients/               provider api clients (serper, jina, kernel, reddit,
                         scrapedo) — wrapped by Live layers in src/effect/
  tools/
    registry.ts          registerAllTools() — wires the five tools
    start-research.ts    goal-tailored brief + static playbook + planner
                         circuit-breaker
    search.ts            raw/smart search handlers (ctr ranking + optional
                         llm classification)
    scrape.ts            raw/smart scrape handlers (reddit api, jina reader,
                         scrape.do proxy retry, optional kernel, optional
                         llm extraction)
    mcp-helpers.ts       markdown response builders
  services/
    llm-processor.ts     llm extraction, classification, brief generation —
                         primary + fallback model, always low reasoning,
                         oversized inputs route straight to fallback
    markdown-cleaner.ts  html/markdown cleanup (readability + turndown)
  schemas/               zod v4 input validation per tool
  utils/                 errors, retry, ctr aggregator, response builders,
                         logger (stderr-only, mcp-safe)

key patterns: capability detection at startup, description-led tool routing (no bootstrap gate), markdown-only mcp tool output for search/scrape, raw/smart tool split, tiered classified output in smart-web-search, reddit api routing in scrape tools, jina reader first for non-reddit urls, scrape.do proxy-mode retry through X-Proxy-Url, optional kernel browser-render fallback with scoped session cleanup, bounded concurrency via Effect.forEach, ctr-based url ranking, tools never throw (always return toolFailure), and structured errors with retry classification.

license

MIT

README mirrored from the source repository 4 months ago. The original is authoritative.

Questions

About Researchpowerpack Http

How do I install Researchpowerpack Http?

Run claude mcp add researchpowerpack-http -- npx -y mcp-researchpowerpack-http, then add the server to your MCP client's configuration. Conduid has recorded 0 installs, so the command is known to work with current clients.

Is Researchpowerpack Http safe to use with an AI agent?

Its trust score is 37 out of 100 (low). Conduid hasn't run static security checks on this repository yet, so review the source yourself before granting it credentials. It has no ConduID identity yet, so agent calls to it are not receipted.

Is Researchpowerpack Http still maintained?

Conduid hasn't recorded a commit date for this repository yet. Check the repository directly for recent activity.