Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Do not start and end the stream prematurely (#135) #136

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

pavelhoral
Copy link

This PR fixes premature start of the glob stream which can lead to closing the resulting Transform stream before it is being properly processed by upstream components. This can lead to very strange race conditions and unfinished pipelines.

@phated
Copy link
Member

phated commented Jun 14, 2024

This is awesome. Thanks for debugging and adding a regression test! I think we can actually leverage the open function in streamx for this.

@pavelhoral
Copy link
Author

pavelhoral commented Jun 15, 2024

This is awesome. Thanks for debugging and adding a regression test! I think we can actually leverage the open function in streamx for this.

Not sure why, but moving the code to open function breaks the pipeline completely. Seems like open state is too soon for data being produced by the stream.

UPDATE: I am dumb... forgot about the callback.

@pavelhoral
Copy link
Author

Pushed new changes (switch to open method).

@pavelhoral
Copy link
Author

Any update on this? Is there anything else I can do? Ty.

@pavelhoral
Copy link
Author

Any update on this? Is there anything else I can do? Ty.

Would love to have this merged and released. If there is anything I can do to help that, let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants