If you get command not found (Linux/MacOS), or a CommandNotFoundException (Windows), the directory where pip installs scripts is not in your PATH. You can either add that directory to your path (pip show pip | grep Scripts) or instead just run:

python -m memgpt

Clone this repo: git clone https://github.com/cpacker/MemGPT.git

Using poetry:

1. Install poetry: pip install poetry
2. Run poetry install
3. Run poetry run memgpt

Using pip:

1. Run pip install -e .
2. Run python3 main.py

MemGPT's archival memory let's you load your database and talk to it! To motivate this use-case, we have included a toy example.

Consider the test.db already included in the repository.

To talk to this database, run:

memgpt --archival_storage_sqldb=memgpt/personas/examples/sqldb/test.db

And then you can input the path to your database, and your query.

Please enter the path to the database. test.db
...
Enter your message: How old is Bob?
...
🤖 Bob is 25 years old.

To run our example where you can search over the SEC 10-K filings of Uber, Lyft, and Airbnb,

1. Download the .txt files from Hugging Face and place them in memgpt/personas/examples/preload_archival.

2. In the root MemGPT directory, run

   memgpt --archival_storage_files="memgpt/personas/examples/preload_archival/*.txt" --persona=memgpt_doc --human=basic

If you would like to load your own local files into MemGPT's archival memory, run the command above but replace --archival_storage_files="memgpt/personas/examples/preload_archival/*.txt" with your own file glob expression (enclosed in quotes).

In the root MemGPT directory, run

memgpt main.py --archival_storage_files_compute_embeddings="<GLOB_PATTERN>" --persona=memgpt_doc --human=basic

This will generate embeddings, stick them into a FAISS index, and write the index to a directory, and then output:

  To avoid computing embeddings next time, replace --archival_storage_files_compute_embeddings=<GLOB_PATTERN> with
    --archival_storage_faiss_path=<DIRECTORY_WITH_EMBEDDINGS> (if your files haven't changed).

If you want to reuse these embeddings, run

memgpt --archival_storage_faiss_path="<DIRECTORY_WITH_EMBEDDINGS>" --persona=memgpt_doc --human=basic

MemGPT also enables you to chat with docs -- try running this example to talk to the LlamaIndex API docs!

1. a. Download LlamaIndex API docs and FAISS index from Hugging Face.

   # Make sure you have git-lfs installed (https://git-lfs.com)
   git lfs install
   git clone https://huggingface.co/datasets/MemGPT/llamaindex-api-docs
   mv llamaindex-api-docs

   -- OR --

   b. Build the index:

   1. Build llama_index API docs with make text. Instructions here. Copy over the generated _build/text folder to memgpt/personas/docqa.
   2. Generate embeddings and FAISS index.

      cd memgpt/personas/docqa
      python3 scrape_docs.py
      python3 generate_embeddings_for_docs.py all_docs.jsonl
      python3 build_index.py --embedding_files all_docs.embeddings.jsonl --output_index_file all_docs.index
      

2. In the root MemGPT directory, run

   memgpt --archival_storage_faiss_path=<ARCHIVAL_STORAGE_FAISS_PATH> --persona=memgpt_doc --human=basic

   where ARCHIVAL_STORAGE_FAISS_PATH is the directory where all_docs.jsonl and all_docs.index are located. If you downloaded from Hugging Face, it will be memgpt/personas/docqa/llamaindex-api-docs. If you built the index yourself, it will be memgpt/personas/docqa.