Phospho is the text analytics platform for LLM apps. Detect issues and extract insights from text messages of your users or your app. Gather user feedback and measure success. Iterate on your app to create the best conversational experience for your users.

Ship your LLM app in production with confidence, and iterate on it with insights from your users.

• Flexible logging
• Automatic evaluation
• Insights extraction
• Data visualization
• Collaboration

The phospho lab is the core analytics component of phospho. The phospho lab helps you run batched evaluations and event detections on your messages.

pip install "phospho[lab]"

We will run a simple Event detection job that use OpenAI to read a message and figure out whether an Event happened or not.

export OPENAI_API_KEY=your_openai_api_key

Phospho helps you define a bunch of jobs to efficiently run in a workload on users messages.

from phospho import lab

# A workload is a set of jobs to run on messages
# A job is a Python function that leverages LLM and ML models to extract insights from text
workload = lab.Workload()
workload.add_job(
   lab.Job(
         id="question_answering",
         job_function=lab.job_library.event_detection,
         config=lab.EventConfig(
            event_name="question_answering",
            event_description="The user asks a question to the assistant",
         ),
   )
)

# The workload runs the jobs in parallel on all the messages to perform analytics
await workload.async_run(
   messages=[
      lab.Message(
            id="my_message_id",
            role="User",
            content="What is the weather today in Paris?",
         )
   ],
)

message_results = workload.results["my_message_id"]
print(f"The event question_answering was detected: {message_results['question_answering'].value}")

The event question_answering was detected: True

This event detection pipeline can then be linked back to your product or alerting solution (eg. Slack) through webhooks or used to notice patterns in the user data.

You can use other jobs from the phospho library or create your own jobs to run on your messages.

See more examples of what you can do with phospho in the examples folder.

This repository contains the implementation of the platform frontend, the API backend, and the insights extraction pipeline.

• phospho-python: Python client with analytics engine
• extractor: FastAPI analytics service wrapping the analytics engine
• backend: FastAPI backend
• platform: NextJS frontend
• internal-tools: Platform management tools

1. Clone the repo:

git clone [email protected]:phospho-app/monorepo.git

2. Register to the core external services:

   • OpenAI (or another OpenAI-compatible model provider)
   • Cohere
   • MongoDB Atlas (Alternative: self host a MongoDB)

3. (Optional) Extend the platform capabilities by registering to additional services:

   • Resend (emails)
   • Sentry (performance analytics)
   • Propelauth (authentication)
   • Stripe (payment)

4. Follow the deployment instructions in backend/README.md and platform/README.md

5. Enjoy !

To manage the phospho lab evaluations on a collaborative platform, the easiest way is to register to the hosted version.

1. Create a phospho account
2. Install a phospho client: pip install phospho or npm i phospho
3. Create environment variables for PHOSPHO_API_KEY and PHOSPHO_PROJECT_ID
4. Initialize phospho: phospho.init()
5. Log to phospho with phospho.log(input="question", output="answer")

Follow this guide to get started.

This project is licensed under the Apache 2.0 License - see the LICENSE file for details

• phospho Javascript client
• phospho UI React components
• phospho fastassert constrained inference