SambaNova AI Starter Kits are a collection of open-source examples and guides to facilitate the deployment of AI-driven use cases in the enterprise.

To run these examples, you need access to a SambaStudio environment with your models deployed to endpoints. Most code examples are written in Python, though the concepts can be applied in any language.

If you have any issues with the examples or to provide feedback, create an issue in GitHub.

Begin by creating a Sambaverse account, then get your API key from the username button. Use the available models.

Begin by deploying your LLM of choice (e.g. Llama 2 13B chat, etc) to an endpoint for inference in SambaStudio. Use either the GUI or CLI, as described in the SambaStudio endpoint documentation.

Integrate your LLM deployed on SambaStudio with this AI starter kit in two simple steps:

  git clone https://github.com/sambanova/ai-starter-kit.git

These are represented as configurable variables in the environment variables file in sn-ai-starter-kit/.env.

For example, enter an endpoint with the URL "https://api-stage.sambanova.net/api/predict/nlp/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef0123" in the env file (with no spaces) as:

BASE_URL="https://api-stage.sambanova.net"
PROJECT_ID="12345678-9abc-def0-1234-56789abcdef0"
ENDPOINT_ID="456789ab-cdef-0123-4567-89abcdef0123"
API_KEY="89abcdef-0123-4567-89ab-cdef01234567"

Enter a Sambaverse API key, for example "456789ab-cdef-0123-4567-89abcdef0123", in the env file (with no spaces) as:

SAMBAVERSE_API_KEY="456789ab-cdef-0123-4567-89abcdef0123"

You can use SambaStudio E5 embedding model endpoint instead of using default in cpu HugginFace embeddings to increase inference speed, follow this guide to deploy your SambaStudio embedding model

be sure to set batch size model parameter to 32

Update API information for the SambaNova embedding endpoint. These are represented as configurable variables in the environment variables file in the root repo directory sn-ai-starter-kit/.env. For example, an endpoint with the URL "https://api-stage.sambanova.net/api/predict/nlp/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef0123" would be entered in the env file (with no spaces) as:

EMBED_BASE_URL="https://api-stage.sambanova.net"
EMBED_PROJECT_ID="12345678-9abc-def0-1234-56789abcdef0"
EMBED_ENDPOINT_ID="456789ab-cdef-0123-4567-89abcdef0123"
EMBED_API_KEY="89abcdef-0123-4567-89ab-cdef01234567"

Note that using different embedding models (cpu or sambastudio) may change the results, and change the way they are set and their parameters

with CPU Huggingface embeddings:

           embeddings = HuggingFaceInstructEmbeddings(
               model_name="hkunlp/instructor-large",
               embed_instruction="",
               query_instruction="Represent this sentence for searching relevant passages:",
               encode_kwargs={"normalize_embeddings": True},
           )

with Sambastudio embeddings:

embeddings = SambaNovaEmbeddingModel()

Go to the README.md of the starter kit you want to use and follow the instructions. See Available AI Starter Kits.

Set your environment as shown in integrate your model.

1. Import the samabanova_endpoint langchain wrapper in your project and define your SambaverseEndpoint LLM:

from utils.sambanova_endpoint import SambaverseEndpoint

load_dotenv('.env')

llm = SambaverseEndpoint(
    sambaverse_model_name="Meta/llama-2-7b-chat-hf",
    model_kwargs={
      "do_sample": False,
      "temperature": 0.0,
      "max_tokens_to_generate": 512,
      "select_expert": "llama-2-7b-chat-hf"
      },
)

2. Use the model

llm.invoke("your prompt")

1. Import the samabanova_endpoint langchain wrapper in your project and define your SambaNovaEndpoint LLM:

from utils.sambanova_endpoint import SambaNovaEndpoint

load_dotenv('.env')

llm = SambaNovaEndpoint(
    model_kwargs={
      "do_sample": False,
      "max_tokens_to_generate": 512,
      "temperature": 0.0
      },
)

2. Use the model

llm.invoke("your prompt")

See utils/usage.ipynb for an example.

1. Import the samabanova_endpoint langchain wrapper in your project and define your SambaNovaEmbeddingModel embedding:

from utils.sambanova_endpoint import SambaNovaEndpoint

load_dotenv('.env')

embedding = SambaNovaEmbeddingModel()

2. Use your embedding model in your langchain pipeline

See utils/usage.ipynb for an example.

Note: These AI Starter Kit code samples are provided "as-is," and are not production-ready or supported code. Bugfix/support will be on a best-effort basis only. Code may use third-party open-source software. You are responsible for performing due diligence per your organization policies for use in your applications.