Proposed updated flow field color scheme and Hypermorph layers in PyTorch

[aviziskind]

Hi, I wanted to thank you for putting together this repository. I have been using it for a few months and have found it very useful for my work analyzing CT scans. In my own copy of the repo, I have added two features that I thought might be helpful to contribute in case others might benefit.

(1) I replaced the default coloring for the warp/flow field with the Baker et al. (2007) (http:https://vision.middlebury.edu/flow/flowEval-iccv07.pdf) color-scheme that is commonly used to visualize flow in the optical flow literature. This makes the flow field a little easier to interpret and provides several advantages over the 'winter' colormap that is the current default coloring:
(a) Each direction is a distinctive color (instead of a range from blue-to-green).
(b) The color wheel is cyclic/circularly continuous as a function of direction angle (whereas there is a sharp blue/green discontinuity in the winter colormap)
(c) The 'lightness' of the color increases as the magnitude of the flow decreases, so that zero flow is white, which makes it easier to distinguish where the magnitude of the flow field is high or low. A comparison between the two color schemes is shown below for a sample flow from the Voxelmorph tutorial (oasis 2d MRI) dataset.

Edit: here is a direct comparison of the two color schemes, shown both as RGB images (top) and arrow plot (bottom):

(2) I implemented the HyperConv2D and HyperConv3D modules in PyTorch, enabling the training of Hypermorph models in PyTorch (currently they are only implemented in TensorFlow).

Each of these features require merging in a Pull Request into both the neurite and voxelmorph repos. Let me know if you would be interested in merging in these features to the master or dev branches of neurite and voxelmorph and I can clean up my code a bit and initiate the Pull Requests. If you're interested, I'd also be curious to know if you'd like the new (Baker et al.) color scheme to be the new default or not.

[HBB0517]

I think your new work is great.However,i have a question,How to draw a flow field as beautiful as yours in the pytorch environment？If you have answer or code,plz tell me.Thank you very much!

[aviziskind]

I have modified the functions available in the neurite library (neurite.plot.flow, specifically), which, by the way, is also published by the authors of this (voxelmorph) repo. I can push my changes to my own fork of neurite/voxelmorph so other people can benefit.

[xxy905055476]

I have modified the functions available in the neurite library (neurite.plot.flow, specifically), which, by the way, is also published by the authors of this (voxelmorph) repo. I can push my changes to my own fork of neurite/voxelmorph so other people can benefit.

hello, I want to know how to use your changes to draw such a beautiful field

[aviziskind]

Hi @adalca , I created Pull Requests for the neurite and voxelmorph repos to merge in the capabilities for the new Baker et al. colors scheme and for the HyperConv2D and HyperConv3D layers for use in the HyperMorph networks in PyTorch (separate pull requests for each feature). I hope these prove to be useful!