LLM and AI-Infused Applications with Java & Spring AI

Samples showing how to build Java applications powered by Generative AI and Large Language Models (LLMs) using Spring AI.

🛠️ Pre-Requisites

• Java 25
• Podman/Docker

💡 Use Cases

• Chatbot Chatbot using LLMs via Ollama.

• Question Answering Question answering with documents (RAG) using LLMs via Ollama and PGVector.

• Semantic Search Semantic search using LLMs via Ollama and PGVector.

• Structured Data Extraction
  Structured data extraction using LLMs via Ollama.

• Text Classification Text classification using LLMs via Ollama.

🧠 Models

Chat Models

Chat completion with LLMs via different model providers:

• Mistral AI
• Ollama
• OpenAI
• Multiple Providers

Embedding Models

Vector transformation (embeddings) with LLMs via different model providers:

• Mistral AI
• Ollama
• OpenAI
• ONNX Transformers

Image Models

Image generation with LLMs via different model providers:

• OpenAI

Audio Models

Speech generation with LLMs via different model providers:

• OpenAI

Speech transcription with LLMs via different model providers:

• OpenAI

Moderation Models

Coming soon

📐 Patterns

Prompts, Messages, and Templates

Prompting using simple text:

• Ollama
• OpenAI

Prompting using structured messages and roles:

• Ollama
• OpenAI

Prompting using templates:

• Ollama
• OpenAI

Structured Output

Converting LLM output to structured JSON and Java objects:

• Ollama
• OpenAI

Multimodality

Including various media in prompts with LLMs:

• Mistral AI
• Ollama
• OpenAI

Tool Calling

Tool calling with LLMs via different model providers:

• Mistral AI
• Ollama
• OpenAI

Memory

Using chat memory to preserve context in conversations with LLMs:

• Basic Chat Memory
• JDBC Chat Memory
• Spring Security Chat Memory
• Vector Store Chat Memory

Guardrails

Guardrails for input and output with LLMs via different model providers:

• Input Guardrails
• Output Guardrails

📥 Data Ingestion

Document Readers

Reading and vectorizing documents with LLMs via Ollama:

• JSON
• Markdown
• PDF
• Text
• Tika

Document Transformers

Document transformation with LLMs via Ollama:

• Metadata
  Enrich documents with keywords and summary metadata for enhanced retrieval.
• Splitters
  Divide documents into chunks to fit the LLM context window.

🔢 Vector Stores

Coming soon

🔄 Retrieval Augmented Generation (RAG)

Question answering with documents using different RAG flows (with Ollama and PGVector):

Sequential RAG

• Naive
• Advanced

Branching RAG

• Branching

Conditional RAG

• Conditional

📊 Observability

LLM Observability

LLM Observability for different model providers:

• Mistral AI
• Ollama
• OpenAI

Vector Store Observability

Vector Store Observability for different vector stores:

• PGVector

⚙️ Model Context Protocol

Integrations with MCP Servers for providing contexts to LLMs.

• Brave Search API

📋 Evaluation

Coming soon

🤖 Agents

Coming soon

📚 References and Additional Resources

• Spring AI Reference Documentation

Conferences

• Introducing Spring AI by Christian Tzolov and Mark Pollack (Spring I/O 2024)
• Spring AI Is All You Need by Christian Tzolov (GOTO Amsterdam 2024)
• Concerto for Java and AI - Building Production-Ready LLM Applications by Thomas Vitale (Devoxx UK 2025)
• Modular RAG Architectures with Java and Spring AI by Thomas Vitale (Spring I/O 2025)

Videos

• Building Intelligent Applications With Spring AI by Dan Vega (JetBrains Live Stream)
• Spring AI Series by Dan Vega
• Spring AI Series by Craig Walls
• Spring AI Series by Josh Long

Demos

• Airline Customer Support (Marcus Hellberg)
• Composer Assistant (Thomas Vitale)
• Document Assistant (Marcus Hellberg)
• Flight Booking (Christian Tzolov)

Workshops

• Spring AI - Zero to Hero (Adib Saikali, Christian Tzolov)
• AI Applications with Java and Spring AI (Thomas Vitale)