A fast and lightweight pure Python library for splitting text into semantically meaningful chunks.
-
Updated
Jul 12, 2024 - Python
A fast and lightweight pure Python library for splitting text into semantically meaningful chunks.
🍱 semantic-chunking ⇢ semantically create chunks from large document for passing to LLM workflows
A recursive text chunker that attempts to break the text on meaningful boundaries.
This project is designed to extract text from documents and prepare it for processing by Large Language Models (LLM).
Add a description, image, and links to the text-chunking topic page so that developers can more easily learn about it.
To associate your repository with the text-chunking topic, visit your repo's landing page and select "manage topics."