Nd Pooling

[jmerkow]

Depends on #2049 'ND convolution with im2col'. I'll rebase once that is finalized and merged.

Some of the GPU (gradient and stochastic) tests fail on occasion. Im not sure what the issue is. I've checked the layer output vs master, and the computed values are the same. However the estimated gradients seem to be off... The estimated_gradient values are nan a lot, where as the computed_gradient is reasonable, so I might setting the test up incorrectly. I didn't look too closely at GradientChecker. I am hoping for comments on this.

No issues with CPU Mode. CPU only goes to 3D though. ND,N=2 or 3 computation is much slower than 2D/3D with nested loops. Similar code to GPU loops can be added for num_spatial_axes > 3, but I wanted to wait before I added it since there are issues.

[jmerkow]

Looks like the ND code had an indexing issue. The blob indexes that failed the gradient tests were all edge (lower edges). I believe that it must be an issue with the forward pass.

[dzhwinter]

do you merge cudnn v2 with your branch? Because there is so many conflict beween these two PR when i use #2049 and cudnn v2. @jmerkow

[jmerkow]

This doesn't touch any cudnn code. It only applies to the CAFFE engine.

[dzhwinter]

I have solved my compile error. Thanks for this sharing! I'm wonder to know is it possible to build nd-pooling based on cudnn v2? Because cudnn v2 have the cudnnndpooling and cudnnndconvolution API for N>=3.

[dzhwinter]

check failed in lrnlayertest, test case :LRNLayerTest, testsetupwithinchannel

[faustomilletari]

Hello,
we have extended the code in this pull request. We fixed a bug in the average pooling, we included cuDnn v2 functionality and we wrote some tests in the appropriate test files in order to test 3d convolution and pooling etc. The question is now, how can we push our changes here?

[dzhwinter]

just send a PR and add notes in detail. Then give a ref here

[YOUNGING]

Hello, your work is awesome, but I'm kind of confused about how to use it and how to write a prototxt of 'Nd caffe'. Can you send me some examples?My email address is [email protected] a lot~

[YOUNGING]

Is Channel an axes? I loaded my data in HDF5 format(Size: 28x28x28x1x200),but returned error ‘>4 axes’?What should I do？

[Yeongtae]

@YOUNGING I think you had a mistake. Because I checked that this branch is right. use https://github.com/jmerkow/caffe/tree/nd-pooling-public and set path to /caffe-nd-pooling-public/build/tools on ~/.bashrc.

I use this solver and model and generating_3D_data using hdf5 format.
https://www.dropbox.com/s/i15ktzezah1f21n/3Dcaffe.zip?dl=0

but I am confusing how to use the networks values which are blobs. Because 3D data can't use matcaffe interface for testing that networks is right, and using results. Does anyone have some idea for this?

[jmerkow]

I do everything in Python. Pycaffe interface works fine for blobs.

[shelhamer]

@jmerkow #2049 for vanilla Caffe N-D convolution is merged -- could you rebase this so that pooling can transcend dimensions too? Thanks for your work on this.

[jmerkow]

@shelhamer Sure. I'll get to work on this in the next few days.

[albenoit]

Hi all, any news about this NDpooling contribution ?

[ajtulloch]

What's the perf impact (on CPU and GPU) for fprop/bprop on bvlc_caffenet/bvlc_googlenet with this change?

[ToruHironaka]

@jmerkow, I tried this promotion and ND-Convolution & Pooling worked fine but there is one thing I concerned about that this promotion's python/draw_net.py somehow did not work. It caused below error. I compiled other caffe promotion on the same machine and their python/draw_net.py worked fine.

Error message:

python/draw_net.py ./examples/siamese/mnist_siamese.prototxt ./examples/siamese/mnist_siamese.png
Drawing net to ./examples/siamese/mnist_siamese.png
Traceback (most recent call last):
File "python/draw_net.py", line 44, in
main()
File "python/draw_net.py", line 40, in main
caffe.draw.draw_net_to_file(net, args.output_image_file, args.rankdir)
File "/home/toru/caffe/python/caffe/draw.py", line 165, in draw_net_to_file
fid.write(draw_net(caffe_net, rankdir, ext))
File "/home/toru/caffe/python/caffe/draw.py", line 156, in draw_net
return get_pydot_graph(caffe_net, rankdir).create(format=ext)
File "/home/toru/caffe/python/caffe/draw.py", line 111, in get_pydot_graph
node_label = determine_node_label_by_layertype(layer, layertype, rankdir)
File "/home/toru/caffe/python/caffe/draw.py", line 72, in determine_node_label_by_layertype
layer.convolution_param.pad)
TypeError: %d format: a number is required, not RepeatedScalarFieldContainer

[jmerkow]

Yes, you need it modify the code that generates the strings for drawnet. They used to be single values, now they are lists.

[ToruHironaka]

@jmerkow Thanks for your answer, I have successfully trained ND-conv and ND-pooling with my HDF5 datasets based on CaffeNet Model layers without Local response normalization (LRN) layer in this promotion. Is it possible to ND-LRN? Or, isn't it necessary for me to add ND-LRN?

[jmerkow]

This PR only provides ND pooling support.

[ToruHironaka]

Thanks for your quick response!

[ToruHironaka]

@jmerkow I have been training my network model with my 3D data 32x32x32 with CaffeNet layer but I almost always got high loss values below. I tweaked around my solver parameters to get better results but I have not gotten yet. My training data is about 1842 for training and 201 for testing.

Train net output #0: loss = 87.3365 (* 1 = 87.3365 loss)

I assumed you do the training with your python scripts. I am simply executing caffe train command to train my network. Should I use python interface for my training? Well, I write caffe training script in python like below but I think it is the same.

#!/usr/bin/env python
import caffe
import numpy as np

solver = caffe.SGDSolver("CAD-Train-48/3DCNN-solver-MNIST-Conv2-Pool2.prototxt")
caffe.set_mode_gpu()
caffe.set_device(0)
solver.solve()

The training data is one thing but my data look working with 3D training so I assumed that I have to do something with my loss or accuracy calculations. Do I have to write a python script to run solver like an example "01-learning-lenet.ipynb" in examples folder?

[ToruHironaka]

@jmerkow I have a question about classification. I could not load deploy start data like below.

input: "data"
input_dim: 202
input_dim: 1
input_dim: 32
input_dim: 32
input_dim: 32
Layer {
........
}

This is because it caused the error below so I simply took train dataset layer and leave test dataset in train-val.prototxt and load it as deploy file and load pretrained model to classify data. This seems to be working. Is this correct way to classify input data?

"check failed: param.input_size() * 4 == param.input_dim_size() (4 vs. 5) Incorrect input blob dimension specifications."

[dkoes]

I don't think PoolingLayer::Reshape is correct. It does not update the new shape using bottom and so doesn't actually reshape. I would add these lines:

  channels_ = bottom[0]->shape(channel_axis_);

  // Setup input dimensions (input_shape_).
  vector<int> bottom_dim_blob_shape(1, num_spatial_axes_ + 1);
  input_shape_.Reshape(bottom_dim_blob_shape);
  int* input_shape_data = input_shape_.mutable_cpu_data();
  for (int i = 0; i < num_spatial_axes_ + 1; ++i) {
    input_shape_data[i] = bottom[0]->shape(channel_axis_ + i);
  }

Edit: This fixes failures in NetTest*TestReshape

[elezar]

@jmerkow Since #2049 has been merged, is there any progress on this pull request? I would find it useful for #3732 as well.

[jcpeterson]

Has this PR been abandoned?

[shelhamer]

It does not seem @jmerkow is still working on it, so if someone can step forward and adopt it please do! N-D pooling is still needed to complement N-D conv and this will clear the way for #3983.

[christianpayer]

In #3983 I implemented N-D pooling using cuDNN. I use the same interface (at least at first sight?) for N-D pooling as used in this PR, so it would be nice, if it gets finalized and merged.
I also implemented some 3D pooling tests for cuDNN in #3983. Feel free to use and adapt them for this PR!

[jmerkow]

I can take another look at this. Besides rebase, what does it need? (Tests etc)

[shelhamer]

@jmerkow good to hear! @jmerkow good to hear! Once rebased, I'd look at the N-D convolution tests to have a sense of the expected coverage. There could be slight documentation updates needed in the layer header and layer catalogue too.

If you still have any questions about testing or the gradient checker, please ask here. Thanks.

[antran89]

Any news about status of this PR? Thanks.

[antran89]

Hi @jmerkow, @shelhamer, I want to test Nd-pooling for my problem. However, it seems that the repo has been abandoned. I could not find unknown repository from your Caffe repo @jmerkow. Thanks.