Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: mypy currently doesn't check numpy types #2351

Closed
tstadel opened this issue Mar 23, 2022 · 1 comment · Fixed by #3856
Closed

CI: mypy currently doesn't check numpy types #2351

tstadel opened this issue Mar 23, 2022 · 1 comment · Fixed by #3856
Labels
P3 Low priority, leave it in the backlog topic:dependencies topic:tests type:bug Something isn't working

Comments

@tstadel
Copy link
Member

tstadel commented Mar 23, 2022

Describe the bug
As we do not install numpy during our type-check CI step, numpy types like ndarray are not checked. If we install it however, several findings are shown:

mypy haystack
haystack/modeling/model/tokenization.py:432: error: Incompatible types in assignment (expression has type "List[str]", variable has type "ndarray[Any, dtype[Any]]")
haystack/modeling/data_handler/data_silo.py:749: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[Any]]", variable has type "List[int]")
haystack/modeling/data_handler/data_silo.py:765: error: No overload variant of "hstack" matches argument type "ndarray[Any, dtype[Any]]"
haystack/modeling/data_handler/data_silo.py:765: note: Possible overload variants:
haystack/modeling/data_handler/data_silo.py:765: note:     def [_SCT <: generic] hstack(tup: Sequence[Union[_SupportsArray[dtype[_SCT]], Sequence[_SupportsArray[dtype[_SCT]]], Sequence[Sequence[_SupportsArray[dtype[_SCT]]]], Sequence[Sequence[Sequence[_SupportsArray[dtype[_SCT]]]]], Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype[_SCT]]]]]]]]) -> ndarray[Any, dtype[_SCT]]
haystack/modeling/data_handler/data_silo.py:765: note:     def hstack(tup: Sequence[Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]]) -> ndarray[Any, dtype[Any]]
haystack/document_stores/memory.py:210: error: "ndarray[Any, Any]" has no attribute "unsqueeze"; maybe "squeeze"?
haystack/document_stores/memory.py:215: error: "ndarray[Any, dtype[Any]]" has no attribute "unsqueeze"; maybe "squeeze"?
haystack/document_stores/memory.py:231: error: "ndarray[Any, dtype[Any]]" has no attribute "to"
haystack/document_stores/memory.py:254: error: "ndarray[Any, dtype[Any]]" has no attribute "unsqueeze"; maybe "squeeze"?
haystack/pipelines/standard_pipelines.py:491: error: Argument "query_emb" to "query_by_embedding" of "BaseDocumentStore" has incompatible type "Optional[ndarray[Any, Any]]"; expected "ndarray[Any, Any]"
Found 8 errors in 4 files (checked 129 source files)

Expected behavior
numpy types are checked too and there are no mypy issues.

Additional context
Just checked the linux ci but windows ci should have the same issue.

To Reproduce

conda create -n mypy python=3.8
conda activate mypy
pip install mypy pydantic types-Markdown types-PyYAML types-requests types-setuptools types-six types-tabulate types-chardet types-emoji types-protobuf

# run mypy without numpy types
mypy haystack

# run mypy with numpy types
pip install numpy
mypy haystack
@ZanSara
Copy link
Contributor

ZanSara commented Mar 25, 2022

This happens for many other dependencies, pytorch being the worst offender along with transformers. We should find a way to tackle these issues progressively and clean up the repo of these errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 Low priority, leave it in the backlog topic:dependencies topic:tests type:bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants