-
-
Notifications
You must be signed in to change notification settings - Fork 610
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added safe_mode
for Idist broadcast
#1839
Conversation
1ef0d40
to
d6b8be4
Compare
@vfdev-5 I’m not sure about the use case. As far I have understood, only one process holds a valid tensor and others have none. So the type is reduced to build a valid empty tensor (same style). Build such empty tensor should be pretty easy for the user, shouldn’t it ? |
@sdesrozis here is the use-case where we are blocked a bit: #1758 Otherwise, the idea is to write the code like that: t = None
if rank == src:
t = ... # can be tensor, number or str
t = idist.broadcast(t, src=src) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vfdev-5 Perfect ! Maybe a matter of taste about the argument name use_none
which is not very expressive but examples provided are.
@sdesrozis any other better ideas on how to name it :) ? I'll add a test to cover one of missing |
Moreover, we could have a specific safe broadcast mode replacing every target tensors to avoid errors. |
we can broadcast not only tensors but also numbers and strings =>
what do you mean by "replacing every target tensors to avoid errors." ? |
Stupid scenario if rank == 0:
t = tensor.empty((1,))
if rank == 1:
t = 0
if rank == src: # src != 0 & src != 1
t = 3
t = idist.broadcast(t, src=src, use_none=True) # here, max is tensor.empty (I suppose) |
use_none
for Idist broadcast safe_mode
for Idist broadcast
Covered the use-case from #1839 (comment)
…into idist-broadcast-use-none
@sdesrozis i renamed Could you please review again, thanks ! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vfdev-5 LGTM ! Safe mode with discard is nice !
Description:
safe_mode
for Idist broadcastCheck list: