Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: added split by page to DocumentSplitter #6753

Merged
merged 5 commits into from
Jan 17, 2024
Merged

feat: added split by page to DocumentSplitter #6753

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c5e84c6
feat-added-split-by-page-to-DocumentSplitter
sahusiddharth Jan 17, 2024
a189397
added test case and the suggested changes
sahusiddharth Jan 17, 2024
6bf9c78
Update document_splitter.py
sahusiddharth Jan 17, 2024
de0bc65
Update haystack/components/preprocessors/document_splitter.py
sjrl Jan 17, 2024
40c5e3e
Update test_document_splitter.py
sjrl Jan 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update document_splitter.py
  • Loading branch information
@sahusiddharth
sahusiddharth committed Jan 17, 2024
commit 6bf9c78e3c18c1cfe1f110f761d3a01cb4f233e7
2 changes: 1 addition & 1 deletion haystack/components/preprocessors/document_splitter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def _split_into_units(self, text: str, split_by: Literal["word", "sentence", "pa
split_at = " "
else:
raise NotImplementedError(
"DocumentSplitter only supports 'word', 'sentence', 'passage' or 'page' split_by options."
"DocumentSplitter only supports 'word', 'sentence', 'page' or 'passage' split_by options."
)
units = text.split(split_at)
# Add the delimiter back to all units except the last one
Expand Down