The fastest way to bring multi-agent workflows to production.

For start, FastAgency is not yet another agentic AI framework. There are many such frameworks available today, the most popular open-source ones being AutoGen, CrewAI, Swarm and LangGraph. FastAgency provides you with a unified programming interface for deploying agentic workflows written in above agentic frameworks in both development and productional settings (current version supports AutoGen only, but other frameworks will be supported very soon). With only a few lines of code, you can create a web chat application or REST API service interacting with agents of your choice. If you need to scale-up your workloads, FastAgency can help you deploy a fully distributed system using internal message brokers coordinating multiple machines in multiple datacenters with just a few lines of code changed from your local development setup.

FastAgency is an open-source framework designed to accelerate the transition from prototype to production for multi-agent AI workflows. For developers who use the AutoGen framework, FastAgency enables you to seamlessly scale Jupyter notebook prototypes into a fully functional, production-ready applications. With multi-framework support, a unified programming interface, and powerful API integration capabilities, FastAgency streamlines the deployment process, saving time and effort while maintaining flexibility and performance.

Whether you're orchestrating complex AI agents or integrating external APIs into workflows, FastAgency provides the tools necessary to quickly transition from concept to production, reducing development cycles and allowing you to focus on optimizing your multi-agent systems.

• Multi-Runtime Support: FastAgency supports multiple agentic runtimes to provide maximum flexibility. Currently, it supports AutoGen and plans to extend support to CrewAI. This ensures that as the AI ecosystem evolves, FastAgency remains a reliable and adaptable framework, capable of leveraging emerging agentic technologies. Developers can easily switch between frameworks, choosing the best one for their project's specific needs.

• Unified Programming Interface Across UIs: FastAgency features a common programming interface that enables you to develop your core workflows once and reuse them across various user interfaces without rewriting code. This includes support for both console-based applications via ConsoleUI and web-based applications via MesopUI. Whether you need a command-line tool or a fully interactive web app, FastAgency allows you to deploy the same underlying workflows across environments, saving development time and ensuring consistency.

• Seamless External API Integration: One of FastAgency's standout features is its ability to easily integrate external APIs into your agent workflows. With just a few lines of code, you can import an OpenAPI specification, and in only one more line, you can connect it to your agents. This dramatically simplifies the process of enhancing AI agents with real-time data, external processing, or third-party services. For example, you can easily integrate a weather API to provide dynamic, real-time weather updates to your users, making your application more interactive and useful with minimal effort.

• Tester Class for Continuous Integration: FastAgency also provides a Tester Class that enables developers to write and execute tests for their multi-agent workflows. This feature is crucial for maintaining the reliability and robustness of your application, allowing you to automatically verify agent behavior and interactions. The Tester Class is designed to integrate smoothly with continuous integration (CI) pipelines, helping you catch bugs early and ensure that your workflows remain functional as they scale into production.

• Command-Line Interface (CLI) for Orchestration: FastAgency includes a powerful command-line interface (CLI) for orchestrating and managing multi-agent applications directly from the terminal. The CLI allows developers to quickly run workflows, pass parameters, and monitor agent interactions without needing a full GUI. This is especially useful for automating deployments and integrating workflows into broader DevOps pipelines, enabling developers to maintain control and flexibility in how they manage AI-driven applications.

FastAgency bridges the gap between rapid prototyping and production-ready deployment, empowering developers to bring their multi-agent systems to life quickly and efficiently. By integrating familiar frameworks like AutoGen, providing powerful API integration, and offering robust CI testing tools, FastAgency reduces the complexity and overhead typically associated with deploying AI agents in real-world applications.

Whether you’re building interactive console tools, developing fully-featured web apps, or orchestrating large-scale multi-agent systems, FastAgency is built to help you deploy faster, more reliably, and with greater flexibility.

Currently, the only supported runtime is AutoGen, with support for CrewAI, Swarm and LangGraph coming soon.

FastAgency currently supports workflows defined using AutoGen and provides options for different types of applications:

• Console: Use the ConsoleUI interface for command-line based interaction. This is ideal for developing and testing workflows in a text-based environment.

• Mesop: Utilize MesopUI for web-based applications. This interface is suitable for creating web applications with a user-friendly interface.

FastAgency can use chainable network adapters that can be used to easily create scalable, production ready architectures for serving your workflows. Currently, we support the following network adapters:

• REST API via FastAPI: Use the FastAPIAdapter to serve your workflow using FastAPI server. This setup allows you to work your workflows in multiple workers and serve them using the highly extensible and stable ASGI server.

• NATS.io via FastStream: Utilize the NatsAdapter to use NATS.io MQ message broker for highly-scalable, production-ready setup. This interface is suitable for setups in VPN-s or, in combination with the FastAPIAdapter to serve public workflows in an authenticated, secure manner.

We strongly recommend using Cookiecutter for setting up a FastAgency project. It creates the project folder structure, default workflow, automatically installs all the necessary requirements, and creates a devcontainer that can be used with Visual Studio Code for development.

1. Install Cookiecutter with the following command:

   pip install cookiecutter

2. Run the cookiecutter command:

   cookiecutter https://github.com/airtai/cookiecutter-fastagency.git

3. Assuming that you used the default values, you should get the following output:

   [1/4] project_name (My FastAgency App):
   [2/4] project_slug (my_fastagency_app):
   [3/4] Select app_type
       1 - fastapi+mesop
       2 - mesop
       3 - nats+fastapi+mesop
       Choose from [1/2/3] (1): 1
   [4/4] Select python_version
       1 - 3.12
       2 - 3.11
       3 - 3.10
       Choose from [1/2/3] (1):
   [5/5] Select authentication
       1 - none
       2 - google
       Choose from [1/2] (1):

4. To run LLM-based applications, you need an API key for the LLM used. The most commonly used LLM is OpenAI. To use it, create an OpenAI API Key and set it as an environment variable in the terminal using the following command:

   export OPENAI_API_KEY=openai_api_key_here

5. Open the generated project in Visual Studio Code with the following command:

   code my_fastagency_app

6. Once the project is opened, you will get the following option to reopen it in a devcontainer:

   

7. After reopening the project in devcontainer, you can verify that the setup is correct by running the provided tests with the following command:

   pytest -s

   You should get the following output if everything is correctly setup.

   =================================== test session starts ===================================
   platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
   rootdir: /workspaces/my_fastagency_app
   configfile: pyproject.toml
   plugins: asyncio-0.24.0, anyio-4.6.2.post1
   asyncio: mode=Mode.STRICT, default_loop_scope=None
   collected 1 item
   
   tests/test_workflow.py .                                                            [100%]
   
   ==================================== 1 passed in 1.02s ====================================

You need to define the workflow that your application will use. This is where you specify how the agents interact and what they do. Here's a simple example of a workflow definition as it is generated by the cookie cutter under my_fastagency_app/workflow.py:

import os
from typing import Any

from autogen.agentchat import ConversableAgent
from fastagency import UI
from fastagency.runtimes.autogen import AutoGenWorkflows

llm_config = {
    "config_list": [
        {
            "model": "gpt-4o-mini",
            "api_key": os.getenv("OPENAI_API_KEY"),
        }
    ],
    "temperature": 0.8,
}

wf = AutoGenWorkflows()


@wf.register(name="simple_learning", description="Student and teacher learning chat")  # type: ignore[misc]
def simple_workflow(ui: UI, params: dict[str, Any]) -> str:
    initial_message = ui.text_input(
        sender="Workflow",
        recipient="User",
        prompt="I can help you learn about mathematics. What subject you would like to explore?",
    )

    student_agent = ConversableAgent(
        name="Student_Agent",
        system_message="You are a student willing to learn.",
        llm_config=llm_config,
    )
    teacher_agent = ConversableAgent(
        name="Teacher_Agent",
        system_message="You are a math teacher.",
        llm_config=llm_config,
    )

    chat_result = student_agent.initiate_chat(
        teacher_agent,
        message=initial_message,
        summary_method="reflection_with_llm",
        max_turns=3,
    )

    return str(chat_result.summary)

This code snippet sets up a simple learning chat between a student and a teacher. It defines the agents and how they should interact and specify how the conversation should be summarized.

To ensure that the workflow we have defined is working properly, we can test it locally using MesopUI. The code below can be found under my_fastagency_app/local/main_mesop.py and imports the defined workflow and sets up MesopUI.

You can run the Mesop application locally with the following command on Linux and MacOS:

gunicorn my_fastagency_app.local.main_mesop:app

On Windows, please use the following command:

waitress-serve --listen=0.0.0.0:8000 my_fastagency_app.local.main_mesop:app

Open the MesopUI URL https://localhost:8000 in your browser. You can now use the graphical user interface to start, run, test and debug the autogen workflow manually.

If you created the project using Cookiecutter, then building the Docker image is as simple as running the provided script, as shown below:

./scripts/build_docker.sh

Similarly, running the Docker container is as simple as running the provided script, as shown below:

./scripts/run_docker.sh

If you created the project using Cookiecutter, there are built-in scripts to deploy your workflow to Fly.io. In Fly.io, the application namespace is global, so the application name you chose might already be taken. To check your application's name availability and to reserve it, you can run the following script:

./scripts/register_to_fly_io.sh

Once you have reserved your application name, you can test whether you can deploy your application to Fly.io using the following script:

./scripts/deploy_to_fly_io.sh

This is only for testing purposes. You should deploy using GitHub Actions{target="_blank"} as explained below.

Cookiecutter generated all the necessary files to deploy your application to Fly.io using GitHub Actions. Simply push your code to your github repository's main branch and GitHub Actions will automatically deploy your application to Fly.io. For this, you need to set the following secrets in your GitHub repository:

• FLY_API_TOKEN
• OPENAI_API_KEY

To learn how to create keys and add them as secrets, use the following links:

• Creating a Fly.io API token
• Creating an OpenAI API key
• Adding secrets to your GitHub repository

We are actively working on expanding FastAgency’s capabilities. In addition to supporting AutoGen, we plan to integrate support for other frameworks, other network provider and other UI frameworks.

Stay up to date with new features and integrations by following our documentation and community updates on our Discord server. FastAgency is continually evolving to support new frameworks, APIs, and deployment strategies, ensuring you remain at the forefront of AI-driven development.

Last but not least, show us your support by giving a star to our GitHub repository.

Your support helps us to stay in touch with you and encourages us to continue developing and improving the framework. Thank you for your support!

Thanks to all of these amazing people who made the project better!