fix: put HFTokenStreamingHandler in a lazy_import block
#7403
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
anakin87
added
the
ignore-for-release-notes
anakin87
commented
Mar 22, 2024
| @@ -222,8 +222,7 @@ def check_generation_params(kwargs: Optional[Dict[str, Any]], additional_accepte | |||
| with LazyImport(message="Run 'pip install transformers[torch]'") as torch_and_transformers_import: | |||
| from transformers import PreTrainedTokenizer, PreTrainedTokenizerFast, StoppingCriteria, TextStreamer | |||
|
|
|||
| transformers_import.check() | |||
| torch_import.check() | |||
| torch_and_transformers_import.check() | |||
Pull Request Test Coverage Report for Build 8386643773Details
💛 - Coveralls |
|
Yes, this is on me - apologies. |
vblagoje
approved these changes
Mar 22, 2024
|
I still have a small issue regarding this HFTokenStreamingHandler File "/code/haystack_api/pipeline/custom_pipelines.py", line 27, in <module>
2024-03-22T13:27:51.948291417Z from haystack.components.generators.utils import print_streaming_chunk
2024-03-22T13:27:51.948294052Z File "/usr/local/lib/python3.11/site-packages/haystack/components/generators/__init__.py", line 5, in <module>
2024-03-22T13:27:51.948297138Z from haystack.components.generators.hugging_face_local import HuggingFaceLocalGenerator
2024-03-22T13:27:51.948301586Z File "/usr/local/lib/python3.11/site-packages/haystack/components/generators/hugging_face_local.py", line 13, in <module>
2024-03-22T13:27:51.948308570Z from haystack.utils.hf import deserialize_hf_model_kwargs, serialize_hf_model_kwargs
2024-03-22T13:27:51.948311495Z File "/usr/local/lib/python3.11/site-packages/haystack/utils/hf.py", line 227, in <module>
2024-03-22T13:27:51.948314561Z class StopWordsCriteria(StoppingCriteria):
2024-03-22T13:27:51.948324500Z File "/usr/local/lib/python3.11/site-packages/haystack/utils/hf.py", line 243, in StopWordsCriteria
2024-03-22T13:27:51.948327546Z device: Union[str, torch.device] = "cpu",
2024-03-22T13:27:51.948330792Z ^^^^^
2024-03-22T13:27:51.948333477Z NameError: name 'torch' is not definedI don't use Huggingface at all so torch is not installed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
Several core-integrations tests with Haytstack main are failing after #7377
See for example https://github.com/deepset-ai/haystack-core-integrations/actions/runs/8383450235/job/22959180165
This happens because
HFTokenStreamingHandlerimport in HFLocalGenerator is not inside a lazy_import block,even if the class needs
transformersProposed Changes:
put
HFTokenStreamingHandlerin a lazy_import blockHow did you test it?
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.