Lanarky is a FastAPI framework to build production-grade LLM applications.
Table of Contents
- 🌐 multi-mode token streaming
- 💬 simple gradio chatbot UI for fast prototyping
- 🔗 supports LangChain applications
- 🗄️ multiple LLM caching strategies
See Roadmap for upcoming features.
There are great low-code/no-code solutions in the open source to deploy your LLM projects. However, most of them are opinionated in terms of cloud or deployment code. This project aims to provide users with a cloud-agnostic and deployment-agnostic solution which can be easily integrated into existing backend infrastructures.
The library is available on PyPI and can be installed via pip.
pip install lanarkyYou can find the full documentation at https://lanarky.readthedocs.io/en/latest/.
from dotenv import load_dotenv
from fastapi import FastAPI
from langchain import ConversationChain
from langchain.chat_models import ChatOpenAI
from lanarky import LangchainRouter
load_dotenv()
app = FastAPI()
langchain_router = LangchainRouter(
langchain_url="/chat",
langchain_object=ConversationChain(
llm=ChatOpenAI(temperature=0), verbose=True
),
streaming_mode=0
)
app.include_router(langchain_router)See examples/
for list of available demo examples.
Create a .env file using .env.sample and add your OpenAI API key to it
before running the examples.
- Add support for LangChain
- Add Gradio UI for fast prototyping
- Add support for in-memory, Redis and GPTCache LLM caching
- Add support for LlamaIndex
- Add support for Guidance
- Add SQL database integration
- Add support for Rebuff
Leave a ⭐ if you find this project useful.
Contributions are more than welcome! If you have an idea for a new feature or want to help improve lanarky, please create an issue or submit a pull request on GitHub.
See CONTRIBUTING.md for more information.
The library is released under the MIT License.
Are you interested in building LLM applications with us? We would love to hear from you! Reach out to us on Twitter @lanarky_io.
Let's connect and explore the possibilities of working together to create amazing LLM applications with Lanarky!