Kotlin project with a simple HTMX UI, demonstrating Spring AI with Ollama, Open AI and Neo4j. Shows:
- Mixing LLMs in a single application. Use the right LLM for each requirement.
- The power of Spring AI advisors to instrument chats in a reusable way
- The power of integrating LLM calls within a Spring application.
This project features the following custom advisors:
CaptureMemoryAdvisor
: Simple take on the ChatGPT concept of capturing memories. Uses a small model (gemma2:2b
by default) to try to find useful memories in the latest user message. When it finds one it saves a Document to theVectorStore
so it can be brought into the context in this or future chats. So if you tell the bot your favorite color is green, it will remember in future chats. Memory extraction runs asynchronously, so it doesn't slow responding to the user.NoteMentionsAdvisor
: Detects when a topic is mentioned in a chat and raises an application event
This project illustrates two best practices:
- Externalize your prompts. Prompts should not be in Kotlin, Java, Python/whatever programming language. They should be externalized so they can be edited easily and potentially shared.
- Favor explicit configuration of Spring AI vs relying on starters. This project uses only the Neo vector store
starter, as we want only one vector store. But it explicitly configures the Ollama and Open AI models in an
@Configuration
file. This allows us to mix and switch models easily.
This is a standard Spring Boot project, built with Maven and written in Kotlin.
Set the OPEN_AI_API_KEY
environment variable
to your Open AI token, or edit ChatConfiguration.kt
to switch to a different premium chat model.
Use the Docker Compose file in this project to run Neo,
or otherwise change the Neo credentials in application.properties
to use your own database.
Run Ollama on your machine.
Make sure you've pulled the gemma2:2b
model as follows:
docker pull ollama/gemma2:2b
Edit ChatConfiguration.kt
to use a different Ollama model if you prefer.
- Start the server, either in your IDE or with
mvn spring-boot:run
- Go to
https://localhost:8080
to see the simple chat interface
This is a demo to illustrate the power of Spring AI advisors, so it's simplistic.
In particular:
- The
CaptureMemoryAdvisor
works off the latest user message only (although this is extracted into a strategy function) - The
NoteMentionsAdvisor
looks for a literal string. This could easily be improved to work with a local model and exhibit deeper understanding (e.g. "the user is talking about auto service") - The UI is very basic
Contributions welcome.