Spark History MCP Server

Quick start

1. Start Spark history server and modify spark server url in application.yaml
2. Run it directly
3. MCP SSE endpoint：http://localhost:18888/sse

Tools Provided

• Tool name: compare_job_environments, Description: Compare Spark environment configurations between two jobs
• Tool name: compare_job_performance, Description: Compare performance metrics between two Spark jobs
• Tool name: get_application, Description: Get detailed information about a specific Spark application
• Tool name: get_environment, Description: Get the environment information for a Spark application
• Tool name: get_executor, Description: Get information about a specific executor
• Tool name: get_executor_summary, Description: Get a summary of executors about a application
• Tool name: get_job_bottlenecks, Description: Identify performance bottlenecks in a Spark job
• Tool name: get_resource_usage_timeline, Description: Get resource usage timeline for a Spark application
• Tool name: get_sql_list, Description: Get a list of SQL queries for a Spark application
• Tool name: get_stage, Description: Get information about a specific stage
• Tool name: get_stage_task_summary, Description: Get a summary metrics of all tasks in the given stage attempt
• Tool name: list_applications, Description: Get a list of applications from the Spark History Server
• Tool name: list_executors, Description: Get a list of all executors for a Spark application
• Tool name: list_jobs, Description: Get a list of all jobs for a Spark application
• Tool name: list_slowest_jobs, Description: Get the N slowest jobs for a Spark application
• Tool name: list_slowest_sql_queries, Description: Get the N slowest SQL queries for a Spark application
• Tool name: list_slowest_stages, Description: Get the N slowest stages for a Spark application
• Tool name: list_stages, Description: Get a list of all stages for a Spark application

Applications Envisioned in agent

1. Spark-sql task diagnosis
2. Spark-sql task optimization
3. Review of historical task execution trends