Nika

✨ Features · 🤖 Overview · 📦 Installation · 🚀 Quick start · 📖 Learn more · 📄 Docs · 🌐 Website · 📚 Cite

❓ What is NIKA?

Think about SWE-Bench, but for network troubleshooting. NIKA, Network Incident Benchmark for AI Agents, is an open benchmark for agentic evals on network troubleshooting tasks. NIKA reproduces network incidents across data center and campus networks, enterprise WAN overlays, ISP backbones, SDN and P4 fabrics, Kubernetes clusters, and vendor routing environments. It connects any agent directly to a live network stack while the incident is ongoing, evaluating the ability of the AI agent to troubleshoot the network using network diagnostic tools, switch CLIs, and network telemetry data. You don't need physical hardware to run the benchmark, NIKA is powered by state-of-the-art network emulation backends like Kathará and Containerlab, so you can run it on your laptop or in the cloud.

✨ Features

• Network emulators: Use NIKA with Kathará or Containerlab to run live networks without physical hardware.
• Pre-built tasks: Troubleshoot network incidents without building the setup yourself. NIKA deploys the network and injects the faults.
• Bring your AI agent: Use the included Claude Code, Codex, or LangGraph agents, or connect your own harness.
• Agent sandboxing: Run supported agents in isolated sandboxes, with controlled access to the network, filesystem, and tools.
• Fair evaluation: Evaluate incident detection, localization, and diagnosis under reproducible conditions, and submit your results to the leaderboard.
• MCP network telemetry: Give agents access to network diagnostic tools, telemetry, and router and switch CLIs.
• Parallel and remote execution: Run isolated sessions in parallel to speed up evaluations, or deploy the network on a remote server.
• Session inspection: Review agent actions, logs, and scores in the browser with nika inspect.
• NIKA SDK: Create benchmark tasks with new network environments, traffic patterns, and faults.

🙋 Why NIKA?

NIKA lets you plug in any LLM or agent framework and measure its operational capability under identical, reproducible conditions.

It helps different users answer questions like:

• 💬 Network Manager "A vendor is pitching me an AI solution for network operations. It passes all the standard telecom benchmarks (TeleQnA, TeleLogs, TeleMath, 3GPP-TSG), but I need objective evidence it can handle real incidents before I sign off."
• 💬 Network SRE "I respond to network incidents every day. I want an AI agent to help, but I'm not sure it will understand my topology or make things worse."
• 💬 AI Researcher "I'm designing a new harness for long-horizon network tasks. I need a benchmark to ablate components, measure reproducibly, and compare against published baselines."
• 💬 Applied ML Engineer "I want to fine-tune an open-source model on network troubleshooting and need a structured dataset paired with a rigorous evaluation framework."
• 💬 Contributor "I want to contribute a new network environment or fault type to the community and have it evaluated systematically."

🤖 Overview

NIKA combines two components:

1. NIKA Benchmark — a set of tasks, each combining a network environment, a network incident, and a network traffic pattern.
2. NIKA Orchestrator — a modular platform that deploys network environments, injects faults, connects agents to MCP tools, and evaluates their submissions.

Network types

Choose the network environment you want to troubleshoot. See the environment reference for deployment details.

Network incidents

Explore the network incidents included in the working benchmark:

See the failure reference for individual faults and injection options.

Pluggable agents

NIKA includes agents built with LangGraph, mcp-agent, and AutoGen; sandboxed Codex and Claude Code agents; and community integrations such as SADE. Choose an agent or integrate your own.

Network inspection via MCP

Depending on the network environment, agents can use MCP tools to inspect hosts, routers, switches, and Kubernetes clusters, capture packets, and query Pingmesh and INT telemetry. See MCP servers for available tools.

📦 Installation

Prerequisites

• Linux with Python 3.12+, curl, git, and sudo.
• Docker Engine.
• The host vrf kernel module. On Ubuntu, install it with sudo apt-get install linux-modules-extra-$(uname -r).

Install NIKA

git clone https://github.com/sands-lab/nika.git
cd nika
./scripts/install.sh

The installer keeps NIKA's tools in .venv/ and .nika_cache/ and does not install system packages.

More install options: Installation. If containers lose network access or environments fail to start, see Troubleshooting. To run networks on another server, see remote execution.

Agent sandboxing

Host agents such as LangGraph need no sandbox. For sandboxed agents (cli.*, sdk.*, community.sade), install sbx with KVM access and sign in:

./scripts/install.sh --with-sbx
sbx login

If the installer adds you to the kvm group, open a new login shell and run sbx daemon stop before the first sandboxed run. More detail: agent sandboxing.

🚀 Quick start

Choose and configure an agent

NIKA ships preconfigured agents with templates in config/. The example below sets up the LangGraph agent with an OpenAI model.

Step 1. Pick an agent. List the available agents:

uv run nika agent list

Step 2. Start from its template. The installer creates config/nika.yaml for settings. Replace it with the LangGraph template:

cp config/byo.langgraph.example.yaml config/nika.yaml

Step 3. Select the provider and model:

uv run nika config set agent.provider=openai agent.model=gpt-5-mini

Step 4. Add your API key. The installer also creates .env for credentials. Add your key to it:

OPENAI_API_KEY=<your key>

Step 5. Check the active configuration:

uv run nika config show

For other agents or a custom model endpoint, see choose and run an agent. For configuration options, see the configuration reference.

Run a task

Browse the available tasks (Ctrl+C to leave the browser), then try one with the agent you configured:

uv run nika case browse
uv run nika agent run --problem dc_clos_s_link_down

NIKA deploys the environment, injects the fault, runs the agent, closes the session, and saves the results.

Review the run in the NIKA Inspect dashboard:

uv run nika inspect

To evaluate the same agent on a benchmark release:

uv run nika benchmark run --release 0.2.0 --split test --result_dir results/my-run
uv run nika eval summary --result_dir results/my-run

Interactive shell

NIKA also provides a terminal UI for browsing tasks and running agents:

uv run nika

• /tasks — browse and search benchmark tasks.
• /run — choose a task and an agent, then start a run.

Type / to see all commands. See the interactive shell guide for more.

For environment and session management, see the CLI reference.

📖 Learn more

Pick the path that matches what you're trying to do:

🏁 I want to run the benchmark, any agent

1. Quick start — run a task or a benchmark release.
2. Run configuration: YAML settings, credentials, defaults, and migration.
3. CLI reference: nika commands, sessions, and result paths.
4. Browse sessions with nika inspect: local UI over result directories for agent timelines and scores.
5. Leaderboard submission (GitHub scores + Hugging Face trajectories)

🔌 I want to connect my own agent

1. Choose and run an agent: agent choices and hosted/self-hosted walkthroughs; agent reference for configuration and limits.
2. Agent integration workflow: agent contract and integration workflow.
3. Agent skills: reusable troubleshooting knowledge you can attach to an agent.
4. Agent sandboxing: isolated microVM execution.

🌐 I want to create a new network environment

1. Creating benchmark tasks
2. Network environment reference
3. Failure reference
4. Testing guide

💬 I want to ask a question or contribute

1. NIKA Slack community
2. GitHub issues

Network management benchmarks

NIKA is part of a growing ecosystem. The table below compares NIKA with other benchmarks in terms of their focus, agent interactivity, variety, scale, and realism. While the best benchmark depends on your use case, NIKA currently outstands for realistic agentic evaluations in online environments**.

Notes: Type=Online indicates that agents can observe, modify and interact with a live network environment while running. Offline benchmarks evaluate pre-collected (or generated) samples.

📚 Citation

If you use NIKA in your research, please cite:

@misc{nika25long,
  title          = {A Network Arena for Benchmarking AI Agents on Network Troubleshooting},
  author         = {Zhihao Wang and Alessandro Cornacchia and Alessio Sacco and Franco Galante and Marco Canini and Dingde Jiang},
  year           = {2025},
  eprint         = {2512.16381},
  archivePrefix  = {arXiv},
  primaryClass   = {cs.NI},
  url            = {https://arxiv.org/abs/2512.16381}
}

Please also cite our NGNO '25 paper:

@inproceedings{nika25ngno,
  title        = {Towards a Playground to Democratize Experimentation and Benchmarking of AI Agents for Network Troubleshooting},
  author       = {Wang, Zhihao and Cornacchia, Alessandro and Galante, Franco and Centofanti, Carlo and Sacco, Alessio and Jiang, Dingde},
  year         = {2025},
  publisher    = {Association for Computing Machinery},
  url          = {https://doi.org/10.1145/3748496.3748990},
  booktitle    = {Proceedings of the 1st Workshop on Next-Generation Network Observability},
  location     = {Coimbra, Portugal},
  series       = {NGNO '25}
}

🙏 Acknowledgement

We thank the authors of AIOpsLab for their useful feedbacks.

📄 License

NIKA is released under the MIT License.