![SambaNova logo](https://github.com/rodrigo-92/ai-starter-kit/raw/main/images/SambaNova-dark-logo-1.png)
SambaNova AI Starter Kits are a collection of open-source examples and guides to facilitate the deployment of AI-driven use cases in the enterprise.
To run these examples, you need access to a SambaStudio environment with your models deployed to endpoints. Most code examples are written in Python, though the concepts can be applied in any language.
If you have any issues with the examples or to provide feedback, create an issue in GitHub.
CoE jump start | This kit demonstrates how to call SambaNova CoE models using the Langchain framework. The script offers different approaches for calling CoE models, including using Sambaverse, using SambaStudio with a named expert, and using SambaStudio with routing. |
Data Extraction | Series of notebooks that demonstrate methods for extracting text from documents in different input formats. |
EDGAR Q&A | Example workflow that uses the SambaNova platform to answer questions about organizations using their 10-K annual reports. Includes a runnable local demo and a Docker container to simplify remote deployment. |
Enterprise Knowledge Retrieval | Sample implementation of the semantic search workflow using the SambaNova platform to get answers to questions about your documents. Includes a runnable demo. |
Fine tuning embeddings | Example workflow for fine-tuning embeddings from unstructured data, leveraging Large Language Models (LLMs) and open-source embedding models to enhance NLP task performance. |
Fine tuning SQL | Example workflow for fine-tuning an SQL model for Question-Answering purposes, leveraging Large Language Models (LLMs) and open-source embedding models to enhance SQL generation task performance. |
Image Search | This example workflow shows a simple approach to image search by image description or image similarity. All workflows are built using the SambaNova platform. |
Post Call Analysis | Example workflow that shows a systematic approach to post-call analysis including Automatic Speech Recognition (ASR), diarization, large language model analysis, and retrieval augmented generation (RAG) workflows. All workflows are built using the SambaNova platform. |
Prompt Engineering | Starting point demo for prompt engineering using SambaNova's API to experiment with different use case templates. Provides useful resources to improve prompt crafting, making it an ideal entry point for those new to this AISK. |
Search Assistant | Sample implementation of the semantic search workflow built using the SambaNova platform to get answers to your questions using search engine snippets, and website crawled information as the source. Includes a runnable demo. |
Web Crawled Data Retrieval | Sample implementation of a semantic search workflow built using the SambaNova platform to get answers to your questions using website crawled information as the source. Includes a runnable demo. |
YoDA: Your Data Your model | Sample training recipe to train a Language Model (LLM) using a customer's private data. |
Begin by creating a Sambaverse account, then get your API key from the username button. Use the available models.
Begin by deploying your LLM of choice (e.g. Llama 2 13B chat, etc) to an endpoint for inference in SambaStudio. Use either the GUI or CLI, as described in the SambaStudio endpoint documentation.
Integrate your LLM deployed on SambaStudio with this AI starter kit in two simple steps:
git clone https://github.com/sambanova/ai-starter-kit.git
These are represented as configurable variables in the environment variables file in sn-ai-starter-kit/.env
.
For example, enter an endpoint with the URL "https://api-stage.sambanova.net/api/predict/nlp/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef0123" in the env file (with no spaces) as:
BASE_URL="https://api-stage.sambanova.net"
PROJECT_ID="12345678-9abc-def0-1234-56789abcdef0"
ENDPOINT_ID="456789ab-cdef-0123-4567-89abcdef0123"
API_KEY="89abcdef-0123-4567-89ab-cdef01234567"
Enter a Sambaverse API key, for example "456789ab-cdef-0123-4567-89abcdef0123", in the env file (with no spaces) as:
SAMBAVERSE_API_KEY="456789ab-cdef-0123-4567-89abcdef0123"
You can use SambaStudio E5 embedding model endpoint instead of using default in cpu HugginFace embeddings to increase inference speed, follow this guide to deploy your SambaStudio embedding model
be sure to set batch size model parameter to 32
Update API information for the SambaNova embedding endpoint. These are represented as configurable variables in the environment variables file in the root repo directory sn-ai-starter-kit/.env
. For example, an endpoint with the URL
"https://api-stage.sambanova.net/api/predict/nlp/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef0123"
would be entered in the env file (with no spaces) as:
EMBED_BASE_URL="https://api-stage.sambanova.net"
EMBED_PROJECT_ID="12345678-9abc-def0-1234-56789abcdef0"
EMBED_ENDPOINT_ID="456789ab-cdef-0123-4567-89abcdef0123"
EMBED_API_KEY="89abcdef-0123-4567-89ab-cdef01234567"
Note that using different embedding models (cpu or sambastudio) may change the results, and change the way they are set and their parameters
with CPU Huggingface embeddings:
embeddings = HuggingFaceInstructEmbeddings( model_name="hkunlp/instructor-large", embed_instruction="", query_instruction="Represent this sentence for searching relevant passages:", encode_kwargs={"normalize_embeddings": True}, )with Sambastudio embeddings:
embeddings = SambaNovaEmbeddingModel()
Go to the README.md
of the starter kit you want to use and follow the instructions. See Available AI Starter Kits.
Set your environment as shown in integrate your model.
- Import the samabanova_endpoint langchain wrapper in your project and define your SambaverseEndpoint LLM:
from utils.sambanova_endpoint import SambaverseEndpoint
load_dotenv('.env')
llm = SambaverseEndpoint(
sambaverse_model_name="Meta/llama-2-7b-chat-hf",
model_kwargs={
"do_sample": False,
"temperature": 0.0,
"max_tokens_to_generate": 512,
"select_expert": "llama-2-7b-chat-hf"
},
)
- Use the model
llm.invoke("your prompt")
- Import the samabanova_endpoint langchain wrapper in your project and define your SambaNovaEndpoint LLM:
from utils.sambanova_endpoint import SambaNovaEndpoint
load_dotenv('.env')
llm = SambaNovaEndpoint(
model_kwargs={
"do_sample": False,
"max_tokens_to_generate": 512,
"temperature": 0.0
},
)
- Use the model
llm.invoke("your prompt")
See utils/usage.ipynb for an example.
- Import the samabanova_endpoint langchain wrapper in your project and define your SambaNovaEmbeddingModel embedding:
from utils.sambanova_endpoint import SambaNovaEmbeddingModel
load_dotenv('.env')
embedding = SambaNovaEmbeddingModel()
- Use your embedding model in your langchain pipeline
See utils/usage.ipynb for an example.
Note: These AI Starter Kit code samples are provided "as-is," and are not production-ready or supported code. Bugfix/support will be on a best-effort basis only. Code may use third-party open-source software. You are responsible for performing due diligence per your organization policies for use in your applications.