-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question]: Getting ValueError Invalid literal for base 10 while vectorizing #14535
Comments
To resolve the Here is the corrected code snippet: embed_model = llm_class(api_key=os.getenv("OPENAI_API_KEY"), deployment=model_type, model=model_type, **kwargs)
Settings.embed_model = embed_model
def save_llama(content, metadata, storage_context):
document = Document(text=content, metadata=metadata)
# Ensure the vector store is properly loaded or initialized
vector_store = SomeVectorStore.load_from_disk('vector_store.json') # Replace with actual vector store loading method
VectorStoreIndex.from_documents(
documents=[document],
storage_context=storage_context,
vector_store=vector_store, # Pass the vector store to the query
show_progress=True
) Make sure to replace |
After debuggin I got that no with self._callback_manager.as_trace("index_construction"):
if index_struct is None:
nodes = nodes or []
index_struct = self.build_index_from_nodes(
nodes + objects # type: ignore
) And that's why VectoreStoreIndex is not being initialized class VectorStoreIndex(BaseIndex[IndexDict]):
"""Vector Store Index.
Args:
use_async (bool): Whether to use asynchronous calls. Defaults to False.
show_progress (bool): Whether to show tqdm progress bars. Defaults to False.
store_nodes_override (bool): set to True to always store Node objects in index
store and document store even if vector store keeps text. Defaults to False
"""
index_struct_cls = IndexDict
def __init__(
self,
nodes: Optional[Sequence[BaseNode]] = None,
# vector store index params
use_async: bool = False,
store_nodes_override: bool = False,
embed_model: Optional[EmbedType] = None,
insert_batch_size: int = 2048,
# parent class params
objects: Optional[Sequence[IndexNode]] = None,
index_struct: Optional[IndexDict] = None,
storage_context: Optional[StorageContext] = None,
callback_manager: Optional[CallbackManager] = None,
transformations: Optional[List[TransformComponent]] = None,
show_progress: bool = False,
# deprecated
service_context: Optional[ServiceContext] = None,
**kwargs: Any,
) -> None:
"""Initialize params."""
self._use_async = use_async
self._store_nodes_override = store_nodes_override
self._embed_model = (
resolve_embed_model(embed_model, callback_manager=callback_manager)
if embed_model
else embed_model_from_settings_or_context(Settings, service_context)
)
self._insert_batch_size = insert_batch_size
super().__init__(
nodes=nodes,
**index_struct=index_struct,**
service_context=service_context,
storage_context=storage_context,
show_progress=show_progress,
objects=objects,
callback_manager=callback_manager,
transformations=transformations,
**kwargs,
) |
I don't think this is related to anything with VectoreStoreIndex -- I think your embedding model is not returning raw floats for embedding values. I don't know what You can easily check with |
Question Validation
Question
I am vectorizing using the below code
The text was updated successfully, but these errors were encountered: