LLMs - Configuration Management System

Centralized LLM configuration and documentation management system. Tools for building skills, commands, agents, prompts, and managing MCP servers. Multi-LLM support (Claude Code, Codex, OpenCode).

---

🎯 Vision

Build a unified system for managing LLM configurations, documentation, and tooling across multiple LLM providers (Claude Code, Codex, OpenCode, etc.). Enable developers to:

• Fetch and maintain up-to-date documentation from LLM providers
• Build and manage skills, commands, agents, and prompts
• Package and distribute plugins for team sharing
• Manage MCPs (Model Context Protocol servers)
• Work across LLMs with a single toolset

---

✨ Features

Current (Sprint 1-4: Claude Code Focus)

• ✅ Documentation Fetcher: Automatically fetch and update docs from Anthropic, OpenAI, etc.
• ✅ Scope Intelligence: Auto-detect global/project/local configurations
• ✅ Skill Builder: Generate Claude Code skills with templates
• ✅ Command Builder: Create slash commands for automation
• ✅ Agent Builder: Build sub-agents for specialized tasks
• ✅ Prompt Builder: Generate and validate master prompts
• ✅ Plugin Builder: Package skills/commands/agents for distribution
• ✅ MCP Manager: Manage Model Context Protocol servers
• ✅ Hook Builder: Create hook configurations

Future (Sprint 5+: Multi-LLM)

• 🔮 Codex Support: Adapt tools for OpenAI Codex
• 🔮 OpenCode Support: Extend to OpenCode
• 🔮 Universal Format: LLM-agnostic configuration format
• 🔮 RAG Integration: Personal documentation knowledge base

---

🤖 Feature-Implementer v2 Architecture

Status: ✅ Production Ready (v1.0.0)

The Feature-Implementer v2 is a sophisticated multi-agent system that orchestrates the complete software development lifecycle from requirements analysis to deployment. Built with 14 specialized agents, 37 production skills, and intelligent hooks, it provides a structured, automated approach to implementing features from GitHub issues.

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│              FEATURE-IMPLEMENTER (Main Agent)               │
│                  6-Phase Orchestration                      │
└─────────────────────────────────────────────────────────────┘
           │                                       │
    ┌──────┴──────┐                        ┌──────┴──────┐
    │   PHASE 1   │                        │   PHASE 2   │
    │  ANALYSIS   │                        │   DESIGN    │
    └──────┬──────┘                        └──────┬──────┘
           │                                       │
    @analysis-specialist               @design-orchestrator
    ├─ requirements-extractor           ├─ @architecture-designer
    ├─ security-assessor                │  ├─ architecture-planner
    └─ tech-stack-evaluator             │  ├─ data-modeler
                                         │  └─ api-designer
                                         ├─ @documentation-researcher
                                         │  ├─ doc-fetcher
                                         │  └─ doc-analyzer
                                         └─ @dependency-manager
                                            ├─ dependency-analyzer
                                            └─ version-checker

    ┌──────┬──────┐    ┌──────────┬─────────┐    ┌──────┬──────┐
    │ PHASE 3-4   │    │  PHASE 5         │    │  PHASE 6    │
    │ APPROVAL &  │    │  VALIDATION      │    │  DEPLOYMENT │
    │ IMPLEMENT   │    └──────────┬───────┘    └──────┬──────┘
    └─────────────┘               │                    │
    feature-implementer    @validation-orchestrator    @deployment-specialist
    ├─ analysis-skill     ├─ @unit-test-specialist    ├─ documentation-updater
    ├─ design-skill       ├─ @integration-test        ├─ changelog-generator
    └─ implementation     ├─ @test-runner             └─ pr-creator
       - TDD approach     ├─ @code-quality
       - Code review      ├─ @security-specialist
       - Best practices   └─ @e2e-accessibility (frontend)

Six-Phase Workflow

1. Phase 1: Requirements Analysis (@analysis-specialist)

   • Extract requirements from GitHub issues
   • Security assessment (OWASP, data privacy)
   • Tech stack evaluation
   • Output: docs/implementation/analysis/analysis.md

2. Phase 2: Architecture & Design (@design-orchestrator)

   • Parallel sub-agents:
     • Architecture design (components, data models, APIs)
     • Documentation research (fetch latest library docs)
     • Dependency management (compatibility, versions)
   • Design synthesis and validation
   • Output: docs/implementation/prp/prp.md (Problem-Requirements-Plan)

3. Phase 3: User Approval

   • Present analysis and design to user
   • Gather feedback and approval
   • Iterate if needed

4. Phase 4: Implementation (Main agent)

   • Test-Driven Development (TDD)
   • Code according to project standards
   • Follow best practices and patterns
   • Maintain ≤500 lines per file
   • Output: Working code with initial tests

5. Phase 5: Validation (@validation-orchestrator)

   • Sequential specialists (with recursive communication):
     • Unit tests (@unit-test-specialist)
     • Integration tests (@integration-test-specialist)
     • Test execution & coverage (@test-runner-specialist)
     • Code quality checks (@code-quality-specialist)
     • Security scanning (@security-specialist)
     • E2E & accessibility testing (@e2e-accessibility-specialist, frontend only)
   • Recursive loop if validation fails
   • Output: Validation reports, test results, security scan

6. Phase 6: Deployment (@deployment-specialist)

   • Update documentation (README, guides, API docs)
   • Generate CHANGELOG entry
   • Create pull request
   • Output: PR ready for review

All 14 Agents

Agent	Role	Model	Auto-Activated Skills
feature-implementer	Main orchestrator	Sonnet	analysis, design, implementation, validation
analysis-specialist	Requirements analysis	Haiku	requirements-extractor, security-assessor, tech-stack-evaluator
design-orchestrator	Design coordination	Sonnet	design-synthesizer, prp-generator
architecture-designer	Component architecture	Opus + ultrathink	architecture-planner, data-modeler, api-designer
documentation-researcher	Library docs	Haiku + context7	doc-fetcher, doc-analyzer
dependency-manager	Dependency analysis	Haiku	dependency-analyzer, version-checker
validation-orchestrator	Validation coordination	Sonnet	validation-coordinator, recursive-communicator
unit-test-specialist	Unit testing	Haiku	unit-test-writer, pytest-generator, jest-generator
integration-test-specialist	Integration testing	Haiku	integration-test-writer, api-test-generator
test-runner-specialist	Test execution	Haiku	test-executor, coverage-analyzer
code-quality-specialist	Linting & formatting	Haiku	python-quality-checker, typescript-quality-checker, rust-quality-checker
security-specialist	Security scanning	Sonnet	security-scanner, vulnerability-assessor, owasp-checker
e2e-accessibility-specialist	E2E & WCAG 2.1 AA	Sonnet + playwright	e2e-test-writer, accessibility-checker
deployment-specialist	Documentation & PR	Haiku	documentation-updater, changelog-generator, pr-creator

Production Skills (37)

Core Workflow Skills (4):

• analysis/ - Requirements analysis guidance
• design/ - Architecture and API design
• implementation/ - TDD implementation with code standards
• validation/ - Quality validation workflow

Specialized Skills (33) mapped to agents:

• Analysis: requirements-extractor, security-assessor, tech-stack-evaluator
• Design: design-synthesizer, prp-generator, architecture-planner, data-modeler, api-designer
• Documentation: doc-fetcher, doc-analyzer
• Dependencies: dependency-analyzer, version-checker
• Validation: validation-coordinator, recursive-communicator
• Testing: unit-test-writer, pytest-generator, jest-generator, integration-test-writer, api-test-generator, test-executor, coverage-analyzer
• Quality: python-quality-checker, typescript-quality-checker, rust-quality-checker
• Security: security-scanner, vulnerability-assessor, owasp-checker
• E2E: e2e-test-writer, accessibility-checker
• Deployment: documentation-updater, changelog-generator, pr-creator
• On-demand: code-reviewer, test-generator

Hooks Configuration

Pre-commit Hook (.claude/hooks/pre-commit.py):

• Triggers on git commit commands
• Runs: Black → Flake8 → Mypy → Pytest
• Blocking: Exit code 2 prevents commits if checks fail
• 180-second timeout

Post-implementation Hook (.claude/hooks/post-implementation.py):

• Triggers on implementation phase completion
• Detects completion markers in transcript
• Auto-triggers validation workflow
• Non-blocking: Continues normal conversation flow
• 60-second timeout

Usage

# Implement a feature from GitHub issue
@feature-implementer implement issue #123

# The agent will:
# 1. Analyze requirements (Phase 1)
# 2. Design architecture (Phase 2)
# 3. Present design for approval (Phase 3)
# 4. Implement with TDD (Phase 4)
# 5. Validate with specialists (Phase 5)
# 6. Create PR and update docs (Phase 6)

Quality Standards

• Test Coverage: ≥80% required
• Code Quality: Black, Flake8, Mypy must pass
• Security: OWASP Top 10 compliance
• File Size: ≤500 lines per file
• Accessibility: WCAG 2.1 AA (frontend)
• Documentation: Comprehensive API docs, guides, CHANGELOG

Key Features

✅ Multi-Agent Orchestration: 14 specialized agents working in harmony ✅ Progressive Disclosure: Context loaded only when needed ✅ Recursive Validation: Auto-retry validation until all checks pass ✅ Automated Quality Gates: Pre-commit hooks enforce standards ✅ Documentation-First: Always fetch latest library docs ✅ TDD Approach: Tests written before implementation ✅ Security-First: Built-in security scanning and assessment

Documentation

• Architecture: docs/architecture/feature-implementer-v2.md
• User Guide: docs/guides/feature-implementer-v2-guide.md
• Migration Guide: docs/architecture/migration-guide-v1-to-v2.md
• Skills Mapping: docs/architecture/skills-mapping.md

---

🚀 Quick Start

Installation

# Clone the repository
git clone https://github.com/matteocervelli/llms.git
cd llms

# Install dependencies with uv
uv pip install -r requirements.txt

# Install in development mode
uv pip install -e ".[dev]"

Usage

# Fetch documentation
python -m src.tools.doc_fetcher fetch --provider anthropic

# Build a skill
python -m src.tools.skill_builder create --name my-skill --template basic

# Build a command
python -m src.tools.command_builder create --name my-command

# Build an agent
python -m src.tools.agent_builder create --name my-agent

---

🎯 Scope Intelligence System

The project includes a powerful three-tier scope system for managing configurations at different levels:

Scope Tiers

• Global Scope (~/.claude/): User-wide settings that apply to all projects
• Project Scope (.claude/): Project-specific settings shared with the team
• Local Scope (.claude/settings.local.json): Project-local settings not committed to version control

Configuration Precedence: Local > Project > Global

Quick Example

from src.core.scope_manager import ScopeManager

# Auto-detect scope based on current directory
manager = ScopeManager()
scope = manager.detect_scope()
print(f"Detected scope: {scope.value}")

# Get effective scope with CLI flag
scope_config = manager.get_effective_scope('--project')
print(f"Using path: {scope_config.path}")

# Resolve all scopes with precedence
scopes = manager.resolve_all_scopes()
for scope in scopes:
    print(f"{scope.type.value}: {scope.path} (precedence: {scope.precedence})")

Use Cases

• Global: Personal preferences, default templates, user-wide settings
• Project: Team-shared skills/commands, project configuration (committed)
• Local: Personal overrides, API keys, machine-specific config (gitignored)

See src/core/README.md for detailed documentation and ADR-001 for design decisions.

---

📁 Project Structure

~/.claude/llms/
├── commands/              # Slash commands (LLM-agnostic)
├── agents/                # Sub-agents (LLM-agnostic)
├── skills/                # Skills/capabilities (LLM-agnostic)
├── prompts/               # Prompts
├── .claude/               # Claude-specific settings
│   └── settings.json
├── src/                   # Source code
│   ├── tools/             # Builder tools
│   ├── core/              # Core functionality
│   └── utils/             # Utilities
├── templates/             # Templates for creation
│   ├── claude/            # Claude Code templates
│   ├── codex/             # Codex templates (future)
│   └── opencode/          # OpenCode templates (future)
├── docs/                  # Fetched documentation
│   ├── anthropic/
│   ├── openai/
│   └── mcp/
├── manifests/             # Metadata catalogs
└── tests/                 # Test suite

---

🏃 Development

Setup Development Environment

cd ~/.claude/llms

# Install with development dependencies
uv pip install -e ".[dev]"

# Run tests
pytest

# Run tests with coverage
pytest --cov=src --cov-report=html

# Format code
black src/ tests/

# Type checking
mypy src/

# Lint
flake8 src/ tests/

Running Tests

# All tests
pytest

# Specific test file
pytest tests/test_doc_fetcher.py

# With coverage
pytest --cov=src --cov-report=term-missing

# Verbose
pytest -v

---

🤖 Automation

Weekly Documentation Updates

Automatically update LLM provider documentation on a weekly schedule using cron.

Quick Setup

1. Test the script manually:

cd ~/.claude/llms
./scripts/update_docs.sh

2. Add to crontab (Sundays at 2 AM):

crontab -e

Add this line:

# Update LLM documentation weekly (Sundays at 2 AM)
0 2 * * 0 cd ~/.claude/llms && ./scripts/update_docs.sh >> logs/doc_fetcher/cron.log 2>&1

3. Verify cron job:

crontab -l

Enable Email Notifications (Optional)

To receive email alerts on errors, set the environment variable:

# Add to your shell profile (~/.bashrc, ~/.zshrc, etc.)
export DOC_UPDATER_EMAIL="your-email@example.com"

Requirements:

• mail command (install: brew install mailutils on macOS)
• Configured mail server (sendmail, postfix, or SMTP)

Log Management

Log Locations:

• Detailed logs: logs/doc_fetcher/update_YYYYMMDD_HHMMSS.log
• Cron output: logs/doc_fetcher/cron.log

Automatic Rotation:

• Logs older than 30 days are automatically deleted
• Each run creates a new timestamped log file

View Recent Logs:

# List all logs
ls -lh logs/doc_fetcher/

# View latest log
tail -f logs/doc_fetcher/update_*.log | tail -n 50

# View cron output
tail -f logs/doc_fetcher/cron.log

Disable Automation

To temporarily disable automatic updates:

# Comment out the cron job
crontab -e
# Add # at the beginning of the line:
# 0 2 * * 0 cd ~/.claude/llms && ./scripts/update_docs.sh >> logs/doc_fetcher/cron.log 2>&1

To permanently remove:

crontab -e
# Delete the line completely

Troubleshooting

Cron job not running:

1. Check cron is enabled: sudo launchctl list | grep cron (macOS)
2. Check cron logs: grep CRON /var/log/system.log (macOS)
3. Verify script permissions: ls -l scripts/update_docs.sh (should be -rwxr-x---)

Script fails with errors:

1. Run manually to see detailed output: ./scripts/update_docs.sh
2. Check Python installation: python --version (should be 3.11+)
3. Verify dependencies: pip list | grep -E "(click|requests|pydantic|crawl4ai)"
4. Check manifest exists: ls -l manifests/docs.json

Email notifications not working:

1. Check mail command: which mail
2. Test email manually: echo "test" | mail -s "Test" your-email@example.com
3. Check environment variable: echo $DOC_UPDATER_EMAIL
4. Verify mail server configuration

Logs filling up disk:

• Logs are automatically rotated (30-day retention)
• Check disk usage: du -sh logs/doc_fetcher/
• Manually delete old logs: rm logs/doc_fetcher/update_*.log

Advanced Configuration

Custom Schedule:

# Daily at 3 AM
0 3 * * * cd ~/.claude/llms && ./scripts/update_docs.sh >> logs/doc_fetcher/cron.log 2>&1

# Twice weekly (Monday and Thursday at 1 AM)
0 1 * * 1,4 cd ~/.claude/llms && ./scripts/update_docs.sh >> logs/doc_fetcher/cron.log 2>&1

# Monthly (first Sunday at 2 AM)
0 2 1-7 * 0 cd ~/.claude/llms && ./scripts/update_docs.sh >> logs/doc_fetcher/cron.log 2>&1

Custom Python Command:

# Use specific Python interpreter
0 2 * * 0 cd ~/.claude/llms && PYTHON_CMD=python3.11 ./scripts/update_docs.sh >> logs/doc_fetcher/cron.log 2>&1

# Use virtual environment
0 2 * * 0 cd ~/.claude/llms && PYTHON_CMD=.venv/bin/python ./scripts/update_docs.sh >> logs/doc_fetcher/cron.log 2>&1

Custom Log Retention:

Edit scripts/update_docs.sh and change:

LOG_RETENTION_DAYS=30  # Change to desired number of days

---

📚 Documentation

• CLAUDE.md - Project instructions for Claude Code
• AGENTS.md - Generic LLM instructions
• TASK.md - Sprint tracking and GitHub issues
• CONTRIBUTING.md - Contribution guidelines

Tools Documentation

• Scope Manager - src/core/README.md - Scope intelligence system
• Documentation Fetcher - src/tools/doc_fetcher/README.md - Automated doc fetching
• Skill Builder - src/tools/skill_builder/README.md (Sprint 2)
• Command Builder - src/tools/command_builder/README.md (Sprint 2)
• Agent Builder - src/tools/agent_builder/README.md (Sprint 2)

---

🗓️ Roadmap

Sprint 1: Foundation (Current)

• Initialize project structure
• Build scope intelligence system
• Build LLM adapter architecture
• Build documentation fetcher
• Fetch Anthropic/Claude Code documentation
• Set up weekly documentation updates

Sprint 2: Core Builders

• Build skill builder tool
• Build command builder tool
• Build agent builder tool
• Create templates library
• Build catalog manifest system

Sprint 3: Advanced Builders

• Build hook builder tool
• Build plugin builder tool
• Build prompt builder tool
• Build MCP manager tool

Sprint 4: Polish & Documentation

• Build utilities and validators
• Create comprehensive documentation
• End-to-end testing
• Prepare migration to ~/dev/projects/llms

Sprint 5+: Multi-LLM Support

• Add Codex support
• Add OpenCode support
• Universal configuration format
• RAG integration

---

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for details on:

• Development setup
• Code style and standards
• Testing requirements
• Pull request process

---

📋 Project Status

Current Release: v1.0.0 - Feature-Implementer v2 Architecture (Production Ready) Progress: See TASK.md and GitHub Issues

Recent Milestones

✅ Feature-Implementer v2 Architecture (Issues #40-53)

• 14 specialized agents implemented
• 37 production skills created
• Hooks configuration with quality gates
• Complete 6-phase workflow orchestration
• Comprehensive documentation and guides

✅ Sprint 1-4: Foundation tools and builders

• Scope intelligence system
• Documentation fetcher
• Skill, command, and agent builders
• Plugin and prompt builders
• MCP manager

---

📝 License

MIT License - see LICENSE file for details.

---

👤 Author

Matteo Cervelli

• Business Scalability Engineer
• Website: matteocervelli.com
• GitHub: @matteocervelli
• LinkedIn: matteocervelli

---

🔗 Links

• GitHub Repository: https://github.com/matteocervelli/llms
• Issues: https://github.com/matteocervelli/llms/issues
• Milestones: https://github.com/matteocervelli/llms/milestones
• Anthropic Documentation: https://docs.anthropic.com
• Claude Code Documentation: https://docs.claude.com/en/docs/claude-code

---

Built with ❤️ in Tuscany, Italy