Added safe_mode for Idist broadcast

[vfdev-5]

Description:

• Added safe_mode for Idist broadcast

Check list:

• New tests are added (if a new feature is added)
• New doc strings: description and/or example code are in RST format
• Documentation is updated (if required)

[sdesrozis]

@vfdev-5 I’m not sure about the use case.

As far I have understood, only one process holds a valid tensor and others have none. So the type is reduced to build a valid empty tensor (same style).

Build such empty tensor should be pretty easy for the user, shouldn’t it ?

[vfdev-5]

@sdesrozis here is the use-case where we are blocked a bit: #1758

Otherwise, the idea is to write the code like that:

t = None
if rank == src:
    t = ...  # can be tensor, number or str
t = idist.broadcast(t, src=src)

[sdesrozis]

@vfdev-5 Perfect ! Maybe a matter of taste about the argument name use_none which is not very expressive but examples provided are.

[vfdev-5]

@sdesrozis any other better ideas on how to name it :) ?

I'll add a test to cover one of missing raise ValueError and good to go.

[sdesrozis]

@sdesrozis any other better ideas on how to name it :) ?

use_none_target_tensors ?

Moreover, we could have a specific safe broadcast mode replacing every target tensors to avoid errors.

[vfdev-5]

@sdesrozis any other better ideas on how to name it :) ?

use_none_target_tensors ?

we can broadcast not only tensors but also numbers and strings => use_none_target ?

Moreover, we could have a specific safe broadcast mode replacing every target tensors to avoid errors.

what do you mean by "replacing every target tensors to avoid errors." ?

[sdesrozis]

what do you mean by "replacing every target tensors to avoid errors." ?

Stupid scenario

if rank == 0:
    t = tensor.empty((1,))
if rank == 1:
    t = 0
if rank == src: # src != 0 & src != 1
    t = 3
t = idist.broadcast(t, src=src, use_none=True) # here, max is tensor.empty (I suppose)

[vfdev-5]

@sdesrozis i renamed use_none to safe_mode and now handle also the case you talked about: #1839 (comment)

Could you please review again, thanks !

07a8dba

[sdesrozis]

@vfdev-5 LGTM ! Safe mode with discard is nice !