Lets build the LLM OS proposed by Andrej Karpathy in this tweet, this tweet and this video.

Also checkout my video on building the LLM OS for more information.

• LLMs are the kernel process of an emerging operating system.
• This process (LLM) can solve problems by coordinating other resources (memory, computation tools).
• The LLM OS:
  • Can read/generate text
  • Has more knowledge than any single human about all subjects
  • Can browse the internet
  • Can use existing software infra (calculator, python, mouse/keyboard)
  • Can see and generate images and video
  • Can hear and speak, and generate music
  • Can think for a long time using a system 2
  • Can “self-improve” in domains
  • Can be customized and fine-tuned for specific tasks
  • Can communicate with other LLMs

[x] indicates functionality that is implemented in this LLM OS app

Note: Fork and clone this repository if needed

python3 -m venv ~/.venvs/aienv
source ~/.venvs/aienv/bin/activate

pip install -r cookbook/llm_os/requirements.txt

• Our initial implementation uses GPT-4, so export your OpenAI API Key

export OPENAI_API_KEY=***

• To use Exa for research, export your EXA_API_KEY (get it from here)

export EXA_API_KEY=xxx

We use PgVector to provide long-term memory and knowledge to the LLM OS. Please install docker desktop and run PgVector using either the helper script or the docker run command.

• Run using a helper script

./cookbook/run_pgvector.sh

• OR run using the docker run command

docker run -d \
  -e POSTGRES_DB=ai \
  -e POSTGRES_USER=ai \
  -e POSTGRES_PASSWORD=ai \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -v pgvolume:/var/lib/postgresql/data \
  -p 5532:5432 \
  --name pgvector \
  phidata/pgvector:16

streamlit run cookbook/llm_os/app.py

• Open localhost:8501 to view your LLM OS.
• Add a blog post to knowledge base: https://blog.samaltman.com/gpt-4o
• Ask: What is gpt-4o?
• Web search: Whats happening in france?
• Calculator: Whats 10!
• Enable shell tools and ask: is docker running?
• Enable the Research Assistant and ask: write a report on the ibm hashicorp acquisition
• Enable the Investment Assistant and ask: shall i invest in nvda?