• Oct 2023:
  • 0.1.102: DocChatAgentConfig.n_neighbor_chunks > 0 allows returning context chunks around match.
  • 0.1.99: Convenience batch functions to run tasks, agent methods on a list of inputs concurrently in async mode. See examples in test_batch.py.
  • 0.1.95: Added support for Momento Serverless Vector Index
  • 0.1.94: Added support for LanceDB vector-store -- allows vector, Full-text, SQL search.
  • 0.1.84: Added LiteLLM, so now Langroid can be used with over 100 LLM providers (remote or local)! See guide here.
• Sep 2023:
  • 0.1.78: Async versions of several Task, Agent and LLM methods; Nested Pydantic classes are now supported for LLM Function-calling, Tools, Structured Output.
  • 0.1.76: DocChatAgent: support for loading docx files (preliminary).
  • 0.1.72: Many improvements to DocChatAgent: better embedding model, hybrid search to improve retrieval, better pdf parsing, re-ranking retrieved results with cross-encoders.
  • Use with local LLama Models: see tutorial here
  • Langroid Blog/Newsletter Launched!: First post is here -- Please subscribe to stay updated.
  • 0.1.56: Support Azure OpenAI.
  • 0.1.55: Improved SQLChatAgent that efficiently retrieves relevant schema info when translating natural language to SQL.
• Aug 2023:
  • Hierarchical computation example using Langroid agents and task orchestration.
  • 0.1.51: Support for global state, see test_global_state.py.
  • 🐳 Langroid Docker image, available, see instructions below.
  • RecipientTool enables (+ enforces) LLM to specify an intended recipient when talking to 2 or more agents. See this test for example usage.
  • Example: Answer questions using Google Search + vecdb-retrieval from URL contents.
  • 0.1.39: GoogleSearchTool to enable Agents (their LLM) to do Google searches via function-calling/tools. See this chat example for how easy it is to add this tool to an agent.
  • Colab notebook to try the quick-start examples:
  • 0.1.37: Added SQLChatAgent -- thanks to our latest contributor Rithwik Babu!
  • Multi-agent Example: Autocorrect chat
• July 2023:
  • 0.1.30: Added TableChatAgent to chat with tabular datasets (dataframes, files, URLs): LLM generates Pandas code, and code is executed using Langroid's tool/function-call mechanism.
  • Demo: 3-agent system for Audience Targeting.
  • 0.1.27: Added support for Momento Serverless Cache as an alternative to Redis.
  • 0.1.24: DocChatAgent now accepts PDF files or URLs.

If you are using SQLChatAgent (e.g. the script examples/data-qa/sql-chat/sql_chat.py), with a postgres db, you will need to:

• Install PostgreSQL dev libraries for your platform, e.g.
  • sudo apt-get install libpq-dev on Ubuntu,
  • brew install postgresql on Mac, etc.
• Install langroid with the postgres extra, e.g. pip install langroid[postgres] or poetry add langroid[postgres] or poetry install -E postgres. If this gives you an error, try pip install psycopg2-binary in your virtualenv.

All of the following environment variable settings are optional, and some are only needed to use specific features (as noted below).

• Qdrant Vector Store API Key, URL. This is only required if you want to use Qdrant cloud. You can sign up for a free 1GB account at Qdrant cloud. If you skip setting up these, Langroid will use Qdrant in local-storage mode. Alternatively Chroma is also currently supported. We use the local-storage version of Chroma, so there is no need for an API key. Langroid uses Qdrant by default.
• Redis Password, host, port: This is optional, and only needed to cache LLM API responses using Redis Cloud. Redis offers a free 30MB Redis account which is more than sufficient to try out Langroid and even beyond. If you don't set up these, Langroid will use a pure-python Redis in-memory cache via the Fakeredis library.
• Momento Serverless Caching of LLM API responses (as an alternative to Redis). To use Momento instead of Redis:
  • enter your Momento Token in the .env file, as the value of MOMENTO_AUTH_TOKEN (see example file below),
  • in the .env file set CACHE_TYPE=momento (instead of CACHE_TYPE=redis which is the default).
• GitHub Personal Access Token (required for apps that need to analyze git repos; token-based API calls are less rate-limited). See this GitHub page.
• Google Custom Search API Credentials: Only needed to enable an Agent to use the GoogleSearchTool. To use Google Search as an LLM Tool/Plugin/function-call, you'll need to set up a Google API key, then setup a Google Custom Search Engine (CSE) and get the CSE ID. (Documentation for these can be challenging, we suggest asking GPT4 for a step-by-step guide.) After obtaining these credentials, store them as values of GOOGLE_API_KEY and GOOGLE_CSE_ID in your .env file. Full documentation on using this (and other such "stateless" tools) is coming soon, but in the meantime take a peek at this chat example, which shows how you can easily equip an Agent with a GoogleSearchtool.

If you add all of these optional variables, your .env file should look like this:

OPENAI_API_KEY=your-key-here-without-quotes
GITHUB_ACCESS_TOKEN=your-personal-access-token-no-quotes
CACHE_TYPE=redis # or momento
REDIS_PASSWORD=your-redis-password-no-quotes
REDIS_HOST=your-redis-hostname-no-quotes
REDIS_PORT=your-redis-port-no-quotes
MOMENTO_AUTH_TOKEN=your-momento-token-no-quotes # instead of REDIS* variables
QDRANT_API_KEY=your-key
QDRANT_API_URL=https://your.url.here:6333 # note port number must be included
GOOGLE_API_KEY=your-key
GOOGLE_CSE_ID=your-cse-id

When using Azure OpenAI, additional environment variables are required in the .env file. This page Microsoft Azure OpenAI provides more information, and you can set each environment variable as follows:

• AZURE_API_KEY, from the value of API_KEY
• AZURE_OPENAI_API_BASE from the value of ENDPOINT, typically looks like https://your.domain.azure.com.
• For AZURE_OPENAI_API_VERSION, you can use the default value in .env-template, and latest version can be found here
• AZURE_OPENAI_DEPLOYMENT_NAME is the name of the deployed model, which is defined by the user during the model setup
• AZURE_GPT_MODEL_NAME GPT-3.5-Turbo or GPT-4 model names that you chose when you setup your Azure OpenAI account.

cfg = OpenAIGPTConfig(
  chat_model="local/localhost:8000", 
  chat_context_length=4096
)
mdl = OpenAIGPT(cfg)
# now interact with it as above, or create an Agent + Task as shown below.

If the model is supported by liteLLM, then no need to launch the proxy server. Just set the chat_model param above to litellm/[provider]/[model], e.g. litellm/anthropic/claude-instant-1 and use the config object as above. For remote models, you will typically need to set API Keys etc as environment variables. You can set those based on the LiteLLM docs. If any required environment variables are missing, Langroid gives a helpful error message indicating which ones are needed.

A toy numbers game, where when given a number n:

• repeater_agent's LLM simply returns n,
• even_agent's LLM returns n/2 if n is even, else says "DO-NOT-KNOW"
• odd_agent's LLM returns 3*n+1 if n is odd, else says "DO-NOT-KNOW"

First define the 3 agents, and set up their tasks with instructions:

from langroid.utils.constants import NO_ANSWER
from langroid.agent.chat_agent import ChatAgent, ChatAgentConfig
from langroid.agent.task import Task
from langroid.language_models.openai_gpt import OpenAIChatModel, OpenAIGPTConfig
config = ChatAgentConfig(
    llm = OpenAIGPTConfig(
        chat_model=OpenAIChatModel.GPT4,
    ),
    vecdb = None,
)
repeater_agent = ChatAgent(config)
repeater_task = Task(
    repeater_agent,
    name = "Repeater",
    system_message="""
    Your job is to repeat whatever number you receive.
    """,
    llm_delegate=True, # LLM takes charge of task
    single_round=False, 
)
even_agent = ChatAgent(config)
even_task = Task(
    even_agent,
    name = "EvenHandler",
    system_message=f"""
    You will be given a number. 
    If it is even, divide by 2 and say the result, nothing else.
    If it is odd, say {NO_ANSWER}
    """,
    single_round=True,  # task done after 1 step() with valid response
)

odd_agent = ChatAgent(config)
odd_task = Task(
    odd_agent,
    name = "OddHandler",
    system_message=f"""
    You will be given a number n. 
    If it is odd, return (n*3+1), say nothing else. 
    If it is even, say {NO_ANSWER}
    """,
    single_round=True,  # task done after 1 step() with valid response
)

Then add the even_task and odd_task as sub-tasks of repeater_task, and run the repeater_task, kicking it off with a number as input:

repeater_task.add_sub_task([even_task, odd_task])
repeater_task.run("3")

Langroid leverages Pydantic to support OpenAI's Function-calling API as well as its own native tools. The benefits are that you don't have to write any JSON to specify the schema, and also if the LLM hallucinates a malformed tool syntax, Langroid sends the Pydantic validation error (suitably sanitized) to the LLM so it can fix it!

Simple example: Say the agent has a secret list of numbers, and we want the LLM to find the smallest number in the list. We want to give the LLM a probe tool/function which takes a single number n as argument. The tool handler method in the agent returns how many numbers in its list are at most n.

First define the tool using Langroid's ToolMessage class:

from langroid.agent.tool_message import ToolMessage
class ProbeTool(ToolMessage):
  request: str = "probe" # specifies which agent method handles this tool
  purpose: str = """
        To find how many numbers in my list are less than or equal to  
        the <number> you specify.
        """ # description used to instruct the LLM on when/how to use the tool
  number: int  # required argument to the tool

Then define a SpyGameAgent as a subclass of ChatAgent, with a method probe that handles this tool:

from langroid.agent.chat_agent import ChatAgent, ChatAgentConfig
class SpyGameAgent(ChatAgent):
  def __init__(self, config: ChatAgentConfig):
    super().__init__(config)
    self.numbers = [3, 4, 8, 11, 15, 25, 40, 80, 90]

  def probe(self, msg: ProbeTool) -> str:
    # return how many numbers in self.numbers are less or equal to msg.number
    return str(len([n for n in self.numbers if n <= msg.number]))

We then instantiate the agent and enable it to use and respond to the tool:

from langroid.language_models.openai_gpt import OpenAIChatModel, OpenAIGPTConfig
spy_game_agent = SpyGameAgent(
    ChatAgentConfig(
        name="Spy",
        llm = OpenAIGPTConfig(
            chat_model=OpenAIChatModel.GPT4,
        ),
        vecdb=None,
        use_tools=False, #  don't use Langroid native tool
        use_functions_api=True, # use OpenAI function-call API
    )
)
spy_game_agent.enable_message(ProbeTool)

For a full working example see the chat-agent-tool.py script in the langroid-examples repo.

Suppose you want an agent to extract the key terms of a lease, from a lease document, as a nested JSON structure. First define the desired structure via Pydantic models:

from pydantic import BaseModel
class LeasePeriod(BaseModel):
    start_date: str
    end_date: str


class LeaseFinancials(BaseModel):
    monthly_rent: str
    deposit: str

class Lease(BaseModel):
    period: LeasePeriod
    financials: LeaseFinancials
    address: str

Then define the LeaseMessage tool as a subclass of Langroid's ToolMessage. Note the tool has a required argument terms of type Lease:

class LeaseMessage(ToolMessage):
    request: str = "lease_info"
    purpose: str = """
        Collect information about a Commercial Lease.
        """
    terms: Lease

Then define a LeaseExtractorAgent with a method lease_info that handles this tool, instantiate the agent, and enable it to use and respond to this tool:

class LeaseExtractorAgent(ChatAgent):
    def lease_info(self, message: LeaseMessage) -> str:
        print(
            f"""
        DONE! Successfully extracted Lease Info:
        {message.terms}
        """
        )
        return json.dumps(message.terms.dict())
    
lease_extractor_agent = LeaseExtractorAgent(
  ChatAgentConfig(
    llm=OpenAIGPTConfig(),
    use_functions_api=False,
    use_tools=True,
  )
)
lease_extractor_agent.enable_message(LeaseMessage)

See the chat_multi_extract.py script in the langroid-examples repo for a full working example.

Langroid provides a specialized agent class DocChatAgent for this purpose. It incorporates document sharding, embedding, storage in a vector-DB, and retrieval-augmented query-answer generation. Using this class to chat with a collection of documents is easy. First create a DocChatAgentConfig instance, with a doc_paths field that specifies the documents to chat with.

from langroid.agent.doc_chat_agent import DocChatAgentConfig
from langroid.vector_store.qdrantdb import QdrantDBConfig
config = DocChatAgentConfig(
  doc_paths = [
    "https://en.wikipedia.org/wiki/Language_model",
    "https://en.wikipedia.org/wiki/N-gram_language_model",
    "/path/to/my/notes-on-language-models.txt",
  ]
  llm = OpenAIGPTConfig(
    chat_model=OpenAIChatModel.GPT4,
  ),
  vecdb=QdrantDBConfig()
)

Then instantiate the DocChatAgent (this ingests the docs into the vector-store):

agent = DocChatAgent(config)

Then we can either ask the agent one-off questions,

agent.chat("What is a language model?")

or wrap it in a Task and run an interactive loop with the user:

from langroid.task import Task
task = Task(agent)
task.run()

See full working scripts in the docqa folder of the langroid-examples repo.

Using Langroid you can set up a TableChatAgent with a dataset (file path, URL or dataframe), and query it. The Agent's LLM generates Pandas code to answer the query, via function-calling (or tool/plugin), and the Agent's function-handling method executes the code and returns the answer.

Here is how you can do this:

from langroid.agent.special.table_chat_agent import TableChatAgent, TableChatAgentConfig
from langroid.agent.task import Task
from langroid.language_models.openai_gpt import OpenAIChatModel, OpenAIGPTConfig

Set up a TableChatAgent for a data file, URL or dataframe (Ensure the data table has a header row; the delimiter/separator is auto-detected):

dataset =  "https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv"
# or dataset = "/path/to/my/data.csv"
# or dataset = pd.read_csv("/path/to/my/data.csv")
agent = TableChatAgent(
    config=TableChatAgentConfig(
        data=dataset,  
        llm=OpenAIGPTConfig(
            chat_model=OpenAIChatModel.GPT4,
        ),
    )
)

Set up a task, and ask one-off questions like this:

task = Task(
  agent, 
  name = "DataAssistant",
  default_human_response="", # to avoid waiting for user input
)
result = task.run(
  "What is the average alcohol content of wines with a quality rating above 7?",
  turns=2 # return after user question, LLM fun-call/tool response, Agent code-exec result
) 
print(result.content)

Or alternatively, set up a task and run it in an interactive loop with the user:

task = Task(agent, name="DataAssistant")
task.run()

For a full working example see the table_chat.py script in the langroid-examples repo.