List of missing support from MXNet in DGL project

[jermainewang]

🚀 Feature

We use this issue as a list of required but currently missing features of MXNet. We also need to prioritize them.

• split with sizes_of_sections
• segment_sum / max / mean / ...
• unique (@szha: requires TODO list for supporting dynamic shape apache/mxnet#12732, didn't find issue specific for this)
• nonzero_1d
• sort that returns both sorted array and argsort (@szha: topk has ret_type which needs to be added to sort)
• [zerocopy_to/from_numpy](@szha: https://github.com/dmlc/dgl/blob/master/python/dgl/backend/mxnet/tensor.py#L234) (Feature request: share memory between numpy.array and mxnet.ndarray apache/mxnet#14244)

Performance related:

• CustomOp (ends up with autograd.Function, and it works pretty well)

[zheng-da]

should we create an issue at mxnet repo?

[VoVAllen]

What's the difference between segment_sum and scatter_sum?

[jermainewang]

segment_sum assumes continuous index while scatter_sum not.

[jermainewang]

should we create an issue at mxnet repo?

Is there an on-going issue in mxnet we could refer to?

[szha]

I added related issue from what I know. Please create for the rest

[eric-haibin-lin]

we can create separate issues (one for each item) in mxnet with labels (sparse, numpy, etc) and use this issue to track the progress

[jermainewang]

That's a good idea. I could help create these issues in MXNet and maintain links to this issue if you want.

[zheng-da]

should we list all problems here and create issues in the mxnet repo for each category? i guess it's nice to have everything in one place.

[zheng-da]

Here is a list of other issues in MXNet:

• constructing a CSR matrix from the coo input is slow: constructing a CSR matrix from the coo input is slow. apache/mxnet#14311
• some of the MXNet operators in CPU are very slow. A typical example is concat. Actually, any operators that share the same implementation for CPU and GPU may have a performance issue. Specially, any operators implemented with mshadow or kernel::launch potentially have performance issues.
• _backward_take in GPU is very slow.
• MXNet operators don't support int64. Thus, they may break for large NDArrays (#elements > 2B)

[zheng-da]

actually, split with sizes_of_sections is supported in MXNet now. I forgot to update it in DGL.

[junrushao]

Do you guys really need zero-copy an NDArray from numpy? If so, I can try to implement.

[jermainewang]

@zheng-da I've updated the list. Removed sparse related requirements as DGL now handles all the sparse operations. Could you track the status again and update this thread?

[junrushao]

@jermainewang zero-copy from numpy has been implemented.

[zheng-da]

@junrushao1994 do we need to change anything to use zero-copy from numpy?

[junrushao]

@zheng-da Try this. Note that only non-strided arrays are supported

[zheng-da]

split and zero copy from NumPy is done.
#660