Universal Intelligence

Welcome! Before jumping into what this project is, let's start with the basics.

What is an agentic app?

Agentics apps are applications which use AI. They typically use pretrained models, or agents, to interact with the user and/or achieve tasks.

What is a model?

Models are artificial brains, or neural networks in coding terms. 🧠

They can think, but they can't act without being given the appropriate tools for the job. They are trained to produce a specific output, given a specific input. These can be of any type (often called modalities —eg. text, audio, image, video).

What is a tool?

Tools are scripted tasks, or functions in coding terms. 🔧

They can't think, but they can be used to achieve a pre-defined task (eg. executing a script, making an API call, interacting with a database).

What is an agent?

Agents are robots, or simply put, models and tools connected together. 🤖

🤖 = 🧠 + [🔧, 🔧,..]

They can think and act. They typically use a model to decompose a task into a list of actions, and use the appropriate tools to perform these actions.

What is ⚪ Universal Intelligence?

UIN is a protocol aiming to standardize, simplify and modularize these fundamental AI components (ie. models, tools and agents), for them to be accessible to any developer, and distributed on any platform.

It provides three specifications: Universal Model, Universal Tool, and Universal Agent.

UIN also provides a set of ready-made components and playgrounds for you to get familiar with the protocol and start building in seconds.

Universal Intelligence can be used across all platforms (cloud, desktop, web, mobile).

See next: How to make a simple AI application 📖

Universal Intelligence standardizes, simplifies and modularizes the usage and distribution of artifical intelligence, for it to be accessible by any developers, and distributed on any platform.

It aims to be a framework-less agentic protocol, removing the need for proprietary frameworks (eg. Langchain, Google ADK, Autogen, CrewAI) to build simple, portable and composable intelligent applications.

It does so by standardizing the fundamental building blocks used to make an intelligent application (models, tools, agents), which agentic frameworks typically (re)define and build upon —and by ensuring these blocks can communicate and run on any hardware (model, size, and precision dynamically set; agents share resources).

It provides three specifications: Universal Model, Universal Tool, and Universal Agent.

This project also provides a set of community-built components and playgrounds, implementing the UIN specification, for you to get familiar with the protocol and start building in seconds.

Universal Intelligence protocols and components can be used across all platforms (cloud, desktop, web, mobile).

Agentic Framework vs. Agentic Protocol

How do they compare?

Agent frameworks (like Langchain, Google ADK, Autogen, CrewAI), each orchestrate their own versions of so-called building blocks. Some of them implement the building blocks themselves, others have them built by the community.

UIN hopes to standardize those building blocks and remove the need for a framework to run/orchestrate them. It also adds a few cool features to these blocks like portability. For example, UIN models are designed to automatically detect the current hardware (cuda, mps, webgpu), its available memory, and run the appropriate quantization and engine for it (eg. transformers, llama.cpp, mlx, web-llm). It allows developers not to have to implement different stacks to support different devices when running models locally, and (maybe more importantly) not to have to know or care about hardware compatibility, so long as they don't try to run a rocket on a gameboy 🙂

Protocol Specifications

Universal Model

A ⚪ Universal Model is a standardized, self-contained and configurable interface able to run a given model, irrespective of the consumer hardware and without requiring domain expertise.

It embeddeds a model (i.e. hosted, fetched, or local), one or more engines (e.g. transformers, lama.cpp, mlx-lm, web-llm), runtime dependencies for each device type (e.g. CUDA, MPS), and exposes a standard interface.

While configurable, every aspect is preset for the user, based on automatic device detection and dynamic model precision, in order to abstract complexity and provide a simplified and portable interface.

Providers: In the intent of preseting a Universal Model for non-technical mass adoption, we recommend defaulting to 4 bit quantization.

Universal Tool

A ⚪ Universal Tool is a standardized tool interface, usable by any Universal Agent.

Tools allow interacting with other systems (e.g. API, database) or performing scripted tasks.

When Universal Tools require accessing remote services, we recommend standardizing those remote interfaces as well using MCP Servers, for greater portability. Many MCP servers have already been shared with the community and are ready to use, see available MCP servers for details.

Universal Agent

A ⚪ Universal Agent is a standardized, configurable and composable agent, powered by a Universal Model, Universal Tools and other Universal Agents.

While configurable, every aspect is preset for the user, in order to abstract complexity and provide a simplified and portable interface.

Through standardization, Universal Agent can seemlessly and dynamically integrate with other Universal Intelligence components to achieve any task, and/or share hardware recources (i.e. sharing a common Universal Model) —allowing it to generalize and scale at virtually no cost.

When Universal Agents require accessing remote agents, we recommend leveraging Google's A2A Protocols, for greater compatibility.

In simple terms:

Universal Model = 🧠

Universal Tool = 🔧

Universal Agent = 🤖

🤖 = 🧠 + [🔧, 🔧,..] + [🤖, 🤖,..]

Usage

Universal Model

from <provider> import UniversalModel as Model

model = Model()
output, logs = model.process('How are you today?') # 'Feeling great! How about you?'

Automatically optimized for any supported device 🔥

Customization Options

Simple does not mean limited. Most advanted configuration options remain available.

Those are defined by and specific to the universal model provider.

We encourage providers to use industry standard Hugging Face Transformers specifications, irrespective of the backend internally used for the detected device and translated accordingly, allowing for greater portability and adoption.

Optional Parameters

from <provider> import UniversalModel as Model

model = Model(
  credentials='<token>', # (or) object containing credentials eg. { id: 'example', passkey: 'example' }
  engine='transformers', # (or) ordered by priority ['transformers', 'llama.cpp']
  quantization='BNB_4', # (or) ordered by priority ['Q4_K_M', 'Q8_0'] (or) auto in range {'default': 'Q4_K_M', 'min_precision': '4bit', 'max_precision': '8bit'}
  max_memory_allocation=0.8, # maximum allowed memory allocation in percentage
  configuration={
    # (example)
    # "processor": {
    #     e.g. Tokenizer https://huggingface.co/docs/transformers/fast_tokenizers
    #     
    #     model_max_length: 4096,
    #     model_input_names: ['token_type_ids', 'attention_mask']
    #     ...
    # },
    # "model": {
    #     e.g. AutoModel https://huggingface.co/docs/transformers/models
    #    
    #     torch_dtype: "auto"
    #     device_map: "auto"
    #     ...
    # }
  },
  verbose=True # or string describing log level
)


output, logs = model.process(
  input=[
    {
      "role": "system",
      "content": "You are a helpful model to recall schedules."
    },
    {
      "role": "user",
      "content": "What did I do in May?"
    },
  ], # multimodal
  context=["May: Went to the Cinema", "June: Listened to Music"],  # multimodal
  configuration={
    # (example)
    # e.g. AutoModel Generate https://huggingface.co/docs/transformers/llm_tutorial
    # 
    # max_new_tokens: 2000, 
    # use_cache: True,
    # temperature: 1.0
    # ...
  },
  remember=True, # remember this interaction
  stream=False, # stream output asynchronously
  keep_alive=True # keep model loaded after processing the request
) # 'In May, you went to the Cinema.'

Optional Methods

from <provider> import UniversalModel as Model
model = Model()

# Optional 
model.load() # loads the model in memory (otherwise automatically loaded/unloaded on execution of `.process()`)
model.loaded() # checks if model is loaded
model.unload() # unloads the model from memory (otherwise automatically loaded/unloaded on execution of `.process()`)
model.reset() # resets remembered chat history
model.configuration() # gets current model configuration

# Class Optional
Model.contract()  # Contract 
Model.compatibility()  # Compatibility 

Universal Tool

from <provider> import UniversalTool as Tool

tool = Tool(
  # configuration={ "any": "configuration" },
  # verbose=False
)
result, logs = tool.example_task(example_argument=data)

Optional Methods

from <provider> import UniversalTool as Tool

# Class Optional
Tool.contract()  # Contract 
Tool.requirements()  # Configuration Requirements 

Universal Agent

from <provider> import UniversalAgent as Agent

agent = Agent(
  # (optionally composable)
  #
  # model=Model(),
  # expand_tools=[Tool()],
  # expand_team=[OtherAgent()]
)
output, logs = agent.process('What happened on Friday?') # > (tool call) > 'Friday was your birthday!'

Modular, and automatically optimized for any supported device 🔥

Customization Options

Most advanted configuration options remain available.

Those are defined by and specific to the universal model provider.

We encourage providers to use industry standard Hugging Face Transformers specifications, irrespective of the backend internally used for the detected device and translated accordingly, allowing for greater portability and adoption.

Optional Parameters

from <provider.agent> import UniversalAgent as Agent
from <provider.other_agent> import UniversalAgent as OtherAgent
from <provider.model> import UniversalModel as Model
from <provider.tool> import UniversalTool as Tool # e.g. API, database

# This is where the magic happens ✨
# Standardization of all layers make agents composable and generalized.
# They can now utilize any 3rd party tools or agents on the fly to achieve any tasks.
# Additionally, the models powering each agent can now be hot-swapped so that 
# a team of agents shares the same intelligence(s), thus removing hardware overhead, 
# and scaling at virtually no cost.
agent = Agent(
  credentials='<token>', # (or) object containing credentials eg. { id: 'example', passkey: 'example' }
  model=Model(), # see Universal Model API for customizations
  expand_tools=[Tool()], # see Universal Tool API for customizations
  expand_team=[OtherAgent()],  # see Universal Agent API for customizations
  configuration={
    # agent configuration (eg. guardrails, behavior, tracing)
  },
  verbose=True # or string describing log level
)

output, logs = agent.process(
  input=[
    {
      "role": "system",
      "content": "You are a helpful model to recall schedules and set events."
    },
    {
      "role": "user",
      "content": "Can you schedule what we did in May again for the next month?"
    },
  ], # multimodal
  context=['May: Went to the Cinema', 'June: Listened to Music'],  # multimodal
  configuration={
    # (example)
    # e.g. AutoModel Generate https://huggingface.co/docs/transformers/llm_tutorial
    # 
    # max_new_tokens: 2000, 
    # use_cache: True,
    # temperature: 1.0
    # ...
  },
  remember=True, # remember this interaction
  stream=False, # stream output asynchronously
  extra_tools=[Tool()], # extra tools available for this inference; call `agent.connect()` link during initiation to persist them
  extra_team=[OtherAgent()],  # extra agents available for this inference; call `agent.connect()` link during initiation to persist them
  keep_alive=True # keep model loaded after processing the request
) 
# > "In May, you went to the Cinema. Let me check the location for you." 
# > (tool call: database) 
# > "It was in Hollywood. Let me schedule a reminder for next month."
# > (agent call: scheduler)
# > "Alright you are all set! Hollywood cinema is now scheduled again in July."

Optional Methods

from <provider.agent> import UniversalAgent as Agent
from <provider.other_agent> import UniversalAgent as OtherAgent
from <provider.model> import UniversalModel as Model
from <provider.tool> import UniversalTool as Tool # e.g. API, database
agent = Agent()
other_agent = OtherAgent()
tool = Tool()

# Optional 
agent.load() # loads the agent's model in memory (otherwise automatically loaded/unloaded on execution of `.process()`)
agent.loaded() # checks if agent is loaded
agent.unload() # unloads the agent's model from memory (otherwise automatically loaded/unloaded on execution of `.process()`)
agent.reset() # resets remembered chat history
agent.connect(tools=[tool], agents=[other_agent]) # connects additionnal tools/agents
agent.disconnect(tools=[tool], agents=[other_agent]) # disconnects tools/agents

# Class Optional
Agent.contract()  # Contract 
Agent.requirements()  # Configuration Requirements 
Agent.compatibility()  # Compatibility 

API

Universal Model

A self-contained environment for running AI models with standardized interfaces.

Universal Tool

A standardized interface for tools that can be used by models and agents.

Additional methods are defined by the specific tool implementation and documented in the tool's contract.

Any tool specific method must return a tuple[Any, dict], respectively (result, logs).

Universal Agent

An AI agent powered by Universal Models and Tools with standardized interfaces.

Data Structures

Message

Schema

Expandable as needed

Argument

Output

Method

Contract

When describing a Universal Model, we encourage providers to document core information such as parameter counts and context sizes.

Requirement

Compatibility

QuantizationSettings

Expandable as needed

Development

Abstract classes and types for Universal Intelligence components are made available by the package if you wish to develop and publish your own.

# Install abstracts
pip install universal-intelligence

from universal_intelligence.core import AbstractUniversalModel, AbstractUniversalTool, AbstractUniversalAgent, types

class UniversalModel(AbstractUniversalModel):
  # ...
  pass

class UniversalTool(AbstractUniversalTool):
  # ...
  pass

class UniversalAgent(AbstractUniversalAgent):
  # ...
  pass

If you wish to contribute to community based components, mixins are made available to allow quickly bootstrapping new Universal Models.

See Community>Development section below for additional information.

Community Components

The universal-intelligence package provides several community-built models, agents, and tools that you can use out of the box.

Installation

# Install with device optimizations
pip install "universal-intelligence[community,mps]" # Apple
pip install "universal-intelligence[community,cuda]" # NVIDIA

Some components may require additional dependencies. These can be installed on demand.

# Install MCP specific dependencies
pip install "universal-intelligence[community,mps,mcp]" # Apple
pip install "universal-intelligence[community,cuda,mcp]" # NVIDIA

Some of the community components interface with gated models, in which case you may have to accept the model's terms on Hugging Face and log into that approved account.

You may do so in your terminal using huggingface-cli login

or in your code:

from huggingface_hub import login
login()

Playground

You can get familiar with the library using our ready-made playground

python -m playground.example 

or online using our Google Colab Playground!

Usage

Local Models

from universal_intelligence.community.models.local.default import UniversalModel as Model

model = Model()
output, logs = model.process("How are you doing today?")

View Universal Intelligence Protocols for additional information.

Remote Models

from universal_intelligence.community.models.remote.default import UniversalModel as Model # (or) .default__free

model = Model(credentials='your-openrouter-api-key-here')
output, logs = model.process("How are you doing today?")

View Universal Intelligence Protocols for additional information.

Tools

from universal_intelligence.community.tools.simple_printer import UniversalTool as Tool

tool = Tool()
result, logs = tool.print_text("This needs to be printed")

View Universal Intelligence Protocols for additional information.

Agents

from universal_intelligence.community.agents.default import UniversalAgent as Agent

agent = Agent(
  # model=Model(),                  # customize or share 🧠 across [🤖,🤖,🤖,..]
  # expand_tools=[Tool()],          # expand 🔧 set
  # expand_team=[OtherAgent()]      # expand 🤖 team
)
result, logs = agent.process("Please print 'Hello World' to the console", extra_tools=[Tool()])

View Universal Intelligence Protocols for additional information.

Supported Components

Models

Local Models

Import path: universal_intelligence.community.models.local.<import>

(eg. universal_intelligence.community.models.local.default)

Remote Models

Import path: universal_intelligence.community.models.remote.<import>

(eg. universal_intelligence.community.models.remote.default)

Disclaimer: Descriptions provided by OpenRouter.

Tools

Import path: universal_intelligence.community.tools.<import>

(eg. universal_intelligence.community.tools.mcp_client)

Agents

Import path: universal_intelligence.community.agents.<import>

(eg. universal_intelligence.community.agents.default)

Development

You are welcome to contribute to community components. Please find some introductory information below.

Project Structure

universal-intelligence/
├── playground/           # Playground code directory
│   ├── web/              # Example web playground
│   └── example.py               # Example playground
├── universal_intelligence/      # Source code directory
│   ├── core/             # Core library for the Universal Intelligence specification
│   │   ├── universal_model.py   # Universal Model base implementation
│   │   ├── universal_agent.py   # Universal Agent base implementation
│   │   ├── universal_tool.py    # Universal Tool base implementation
│   │   └── utils/              # Utility functions and helpers
│   ├── community/       # Community components
│   │   ├── models/        # Community-contributed models
│   │   ├── agents/        # Community-contributed agents
│   │   └── tools/         # Community-contributed tools
│   └── www/         # Web Implementation
│       ├── core/               # Core library for the Universal Intelligence web specification
│       │   ├── universalModel.ts   # Universal Model web base implementation
│       │   ├── universalAgent.ts   # Universal Agent web base implementation
│       │   ├── universalTool.ts    # Universal Tool web base implementation
│       │   └── types.ts             # Universal Intelligence web types
│       └── community/       # Web community components
│           ├── models/         # Web community-contributed models
│           ├── agents/         # Web community-contributed agents
│           └── tools/          # Web community-contributed tools
├── requirements*.txt             # Project dependencies
├── *.{yaml,toml,json,*rc,ts}     # Project configuration
├── CODE_OF_CONDUCT.md     # Community rules information
├── SECURITY.md            # Vulnerability report information
├── LICENSE             # License information
├── README_WEB.md       # Project web documentation
└── README.md           # Project documentation

Creating New Components

For faster deployment and easier maintenance, we recommend using/enhancing shared mixins to bootstrap new Universal Intelligence components. Those are made available at ./universal_intelligence/community/<component>/__utils__/mixins. Mixins let components provide their own configurations and while levering a shared implementation. You can find an example here: ./universal_intelligence/community/models/qwen2_5_7b_instruct/model.py.

Model weights can be found here: https://huggingface.co

Testing

Each Universal Intelligence component comes with its own test suite.

Running Tests

Installation:

# (optional) Create dedicated python environment using miniconda
conda init zsh
conda create -n universal-intelligence python=3.10.16 -y
conda activate universal-intelligence

# Install base dependencies
pip install -r requirements.txt

# Install community components dependencies
pip install -r requirements-community.txt

# Install optimizations for your currect device
pip install -r requirements-mps.txt # Apple devices
pip install -r requirements-cuda.txt # NVIDIA devices

# Install development dependencies
pip install -r requirements-dev.txt

# Install commit hook
pre-commit install

# (optional) if using the MCP tool, install dedicated MCP specific dependencies
pip install -r requirements-mcp.txt

Some of the community components interface with gated models, in which case you may have to accept the model's terms on Hugging Face and log into that approved account.

You may do so in your terminal using huggingface-cli login

or in your code:

from huggingface_hub import login
login()

Testing:

# python -m universal_intelligence.community.<component>.<name>.test

# examples
python -m universal_intelligence.community.models.default.test
python -m universal_intelligence.community.tools.default.test
python -m universal_intelligence.community.agents.default.test

Please note that running tests may require downloading multiple configurations of the same components, and temporarily use storage space. Tests will be automatically filtered based on hardware requirements.

Clear downloaded models from storage:

# pip install huggingface_hub["cli"] # Install CLI
huggingface-cli delete-cache # Clear cache

Writing Tests

Test utilities provide shared test suites for each component type.

Model test examples:

• Test Suite: universal_intelligence/community/models/__utils__/test.py
• Usage: universal_intelligence/community/models/qwen2_5_7b_instruct/test.py

Agent test examples:

• Test Suite: universal_intelligence/community/agents/__utils__/test.py
• Usage: universal_intelligence/community/agents/simple_agent/test.py

Tool test examples:

• Test Suite: universal_intelligence/community/tools/__utils__/test.py
• Usage: universal_intelligence/community/tools/simple_printer/test.py

Linting

Linting will run as part of the pre-commit hook, however you may also run it manully using pre-commit run --all-files