boost --with-python required on osx for pycaffe target #465 #1193

sanjeev87 · 2014-09-30T20:03:02Z

brew install --build-from-source --with-python --fresh -vd boost is not installing boost_python lib files .. Please refer Homebrew/legacy-homebrew#32704 , that explains that boost no longer installs python .. I am still trying to install boost python , any advice helps as this method is not working

shelhamer · 2014-10-01T16:16:23Z

Try boost 1.55 and older homebrew packages instead. boost 1.56 seems to have yet more issues with NVCC and doesn't build for me due to

/usr/local/include/boost/config/suffix.hpp(496): error: identifier "__int128" is undefined
/usr/local/include/boost/config/suffix.hpp(497): error: expected a ";"

shelhamer · 2014-10-01T16:42:39Z

git checkout a252214 /usr/local/Library/Formula/boost.rb in the homebrew clone, that is /usr/loca/Cellar or the like, will restore the boost 1.55 formula that works on OSX.

roseperrone · 2014-10-06T20:26:56Z

That fixed build errors, but I still get the following runtime error. What can I try to fix it?

(lldb) bt
* thread #1: tid = 0x9437e, 0x0000000000000000, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x0000000103f3b37d Python`PyEval_GetGlobals + 23
    frame #2: 0x0000000103f49ea5 Python`PyImport_Import + 131
    frame #3: 0x0000000103f48687 Python`PyImport_ImportModule + 31
    frame #4: 0x0000000102ee9d4d _caffe.so`caffe::init_module__caffe() + 3149
    frame #5: 0x0000000103e5f071 libboost_python.dylib`boost::python::handle_exception_impl(boost::function0<void>) + 81
    frame #6: 0x0000000103e601e6 libboost_python.dylib`boost::python::detail::init_module(char const*, void (*)()) + 118
    frame #7: 0x000000010009fbc6 Python`_PyImport_LoadDynamicModule + 140
    frame #8: 0x000000010009f8d3 Python`import_submodule + 274
    frame #9: 0x000000010009f49d Python`load_next + 280
    frame #10: 0x000000010009d454 Python`PyImport_ImportModuleLevel + 589
    frame #11: 0x0000000100082e3b Python`builtin___import__ + 135
    frame #12: 0x000000010000e60f Python`PyObject_Call + 99
    frame #13: 0x000000010008d697 Python`PyEval_CallObjectWithKeywords + 93
    frame #14: 0x0000000100089eb2 Python`PyEval_EvalFrameEx + 9358
    frame #15: 0x0000000100087893 Python`PyEval_EvalCodeEx + 1608
    frame #16: 0x0000000100087245 Python`PyEval_EvalCode + 54
    frame #17: 0x000000010009c141 Python`PyImport_ExecCodeModuleEx + 236
    frame #18: 0x000000010009ecee Python`load_source_module + 813
    frame #19: 0x000000010009f8d3 Python`import_submodule + 274
    frame #20: 0x000000010009f49d Python`load_next + 280
    frame #21: 0x000000010009d454 Python`PyImport_ImportModuleLevel + 589
    frame #22: 0x0000000100082e3b Python`builtin___import__ + 135
    frame #23: 0x000000010000e60f Python`PyObject_Call + 99
    frame #24: 0x000000010008d697 Python`PyEval_CallObjectWithKeywords + 93
    frame #25: 0x0000000100089eb2 Python`PyEval_EvalFrameEx + 9358
    frame #26: 0x0000000100087893 Python`PyEval_EvalCodeEx + 1608
    frame #27: 0x0000000100087245 Python`PyEval_EvalCode + 54
    frame #28: 0x000000010009c141 Python`PyImport_ExecCodeModuleEx + 236
    frame #29: 0x000000010009ecee Python`load_source_module + 813
    frame #30: 0x000000010009f0f3 Python`load_package + 303
    frame #31: 0x000000010009f8d3 Python`import_submodule + 274
    frame #32: 0x000000010009f49d Python`load_next + 280
    frame #33: 0x000000010009d454 Python`PyImport_ImportModuleLevel + 589
    frame #34: 0x0000000100082e3b Python`builtin___import__ + 135
    frame #35: 0x000000010000e60f Python`PyObject_Call + 99
    frame #36: 0x000000010008d697 Python`PyEval_CallObjectWithKeywords + 93
    frame #37: 0x0000000100089eb2 Python`PyEval_EvalFrameEx + 9358
    frame #38: 0x0000000100087893 Python`PyEval_EvalCodeEx + 1608
    frame #39: 0x0000000100087245 Python`PyEval_EvalCode + 54
    frame #40: 0x00000001000a70c2 Python`run_mod + 53
    frame #41: 0x00000001000a7165 Python`PyRun_FileExFlags + 133
    frame #42: 0x00000001000a6d02 Python`PyRun_SimpleFileExFlags + 769
    frame #43: 0x00000001000b8256 Python`Py_Main + 2970
    frame #44: 0x00007fff85b825fd libdyld.dylib`start + 1

I followed the advice of https://github.com/Homebrew/homebrew/wiki/Common-Issues (in the segfaut section) with no luck.

shuggiefisher · 2014-10-08T09:57:39Z

brew install boost-python fixed it for me on 1.56

roseperrone · 2014-10-08T19:34:36Z

I did that (uninstalled boost 1.55, installed boost 1.56, ran brew install boost-python, but now I see these errors when I run make clean; make all; make runtest:

[----------] Global test environment tear-down
[==========] 457 tests from 98 test cases ran. (17644 ms total)
[  PASSED  ] 454 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU

 3 FAILED TESTS
make: *** [runtest] Error 1

In my Makefile.config, I've set CPU_ONLY := 1.

shelhamer · 2014-10-08T19:43:50Z

There should be more detailed errors where those tests are run. Do
./build/test/test_power_layer.testbin and include the gradient checker
outputs for the listed failures.

On Wed, Oct 8, 2014 at 12:34 PM, Rose Perrone [email protected]
wrote:

I did that (uninstalled boost 1.55, installed boost 1.56, ran brew
install boost-python, but now I see these errors when I run make clean;
make all; make runtest:

[----------] Global test environment tear-down
[==========] 457 tests from 98 test cases ran. (17644 ms total)
[ PASSED ] 454 tests.
[ FAILED ] 3 tests, listed below:
[ FAILED ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU

3 FAILED TESTS
make: *** [runtest] Error 1

—
Reply to this email directly or view it on GitHub
#1193 (comment).

sanjeev87 · 2014-10-08T21:53:28Z

Hi ,

I am facing the same issue as well.
When I run ./build/test/test_power_layer.testbin i get the following errors,

[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU (3 ms)
[ RUN ] PowerLayerTest/1.TestPowerZero
[ OK ] PowerLayerTest/1.TestPowerZero (0 ms)
[ RUN ] PowerLayerTest/1.TestPowerZeroGradient
[ OK ] PowerLayerTest/1.TestPowerZeroGradient (1 ms)
[ RUN ] PowerLayerTest/1.TestPowerOne
[ OK ] PowerLayerTest/1.TestPowerOne (0 ms)
[ RUN ] PowerLayerTest/1.TestPowerOneGradient
[ OK ] PowerLayerTest/1.TestPowerOneGradient (2 ms)
[ RUN ] PowerLayerTest/1.TestPowerTwo
[ OK ] PowerLayerTest/1.TestPowerTwo (0 ms)
[ RUN ] PowerLayerTest/1.TestPowerTwoGradient
[ OK ] PowerLayerTest/1.TestPowerTwoGradient (15 ms)
[ RUN ] PowerLayerTest/1.TestPowerTwoScaleHalfGradient
[ OK ] PowerLayerTest/1.TestPowerTwoScaleHalfGradient (3 ms)
[----------] 10 tests from PowerLayerTest/1 (27 ms total)

[----------] Global test environment tear-down
[==========] 20 tests from 2 test cases ran. (78 ms total)
[ PASSED ] 17 tests.
[ FAILED ] 3 tests, listed below:
[ FAILED ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU

Please help fix this issue.

roseperrone · 2014-10-09T14:17:06Z

I have build/test/test_all.testbin, but not build/test/test_power_layer.testbin. When I ran it, I found this error:

[ RUN      ] BlobSimpleTest/0.TestPointersCPUGPU
F1009 07:15:19.080399 2099200784 syncedmem.cpp:91] CPU-only Mode: cannot make GPU call.
*** Check failure stack trace: ***
    @        0x1074dc88a  google::LogMessage::Fail()
    @        0x1074dbaa8  google::LogMessage::SendToLog()
    @        0x1074dc51a  google::LogMessage::Flush()
    @        0x1074e0008  google::LogMessageFatal::~LogMessageFatal()
    @        0x1074dcd25  google::LogMessageFatal::~LogMessageFatal()
    @        0x1071ec3b6  caffe::SyncedMemory::gpu_data()
    @        0x10717f8a6  caffe::Blob<>::gpu_data()
    @        0x106f9d72c  caffe::BlobSimpleTest_TestPointersCPUGPU_Test<>::TestBody()
    @        0x10713f69c  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @        0x10712f6ba  testing::Test::Run()
    @        0x107130602  testing::TestInfo::Run()
    @        0x107130cd0  testing::TestCase::Run()
    @        0x1071361e5  testing::internal::UnitTestImpl::RunAllTests()
    @        0x10713ff84  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @        0x107135f09  testing::UnitTest::Run()
    @        0x106f75c0a  main
    @     0x7fff8fa795fd  start
Abort trap: 6

shelhamer · 2014-10-10T05:06:47Z

@roseperrone

[ RUN ] BlobSimpleTest/0.TestPointersCPUGPU
F1009 07:15:19.080399 2099200784 syncedmem.cpp:91] CPU-only Mode: cannot make GPU call.

This shouldn't happen -- it indicates that a GPU test is being called from CPU mode. Did you make all once before setting the CPU_ONLY flag? I just did a quick test clone on OS X 10.9 and everything went right so perhaps try a make clean and make && make test && make runtest.

shelhamer · 2014-10-10T05:44:05Z

The installation instructions for have been updated for the boost and boost-python split in cb60a41 with a warning about NVCC and boost 1.56 at least on OS X.

Note to self: check out -D__STRICT_ANSI__ for clang++.

BradNeuberg · 2014-10-12T18:27:27Z

I'm still seeing these same problems, even with the updates you made to the README.md file. I'm on Mac OS X 10.9.5 using brew and anaconda to set everything up.

BradNeuberg · 2014-10-12T18:31:38Z

Note that I could only get it working by setting CPU_ONLY := 1 in the Makefile. I also had the Nvidia CUDA package installed.

BradNeuberg · 2014-10-12T18:32:11Z

All the unit tests passed except these in my setup:

[  FAILED  ] 3 tests, listed below:
[  FAILED  ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU

BradNeuberg · 2014-10-12T18:45:21Z

I lied; it doesn't actually work, even with the GPU support turned off. When I tried to train using the MNIST data it failed as follows:

bradneuberg: caffe (master) ./examples/mnist/train_lenet.sh
I1012 11:44:40.661694 2007946000 caffe.cpp:99] Use GPU with device ID 0
F1012 11:44:40.662529 2007946000 common.cpp:53] CPU-only Mode: cannot make GPU call.
*** Check failure stack trace: ***
@ 0x100d8e88a google::LogMessage::Fail()
@ 0x100d8daa8 google::LogMessage::SendToLog()
@ 0x100d8e51a google::LogMessage::Flush()
@ 0x100d92008 google::LogMessageFatal::~LogMessageFatal()
@ 0x100d8ed25 google::LogMessageFatal::~LogMessageFatal()
@ 0x100c0c516 caffe::Caffe::SetDevice()
@ 0x100bca683 train()
@ 0x100bcc871 main
@ 0x7fff892345fd start
@ 0x3 (unknown)
./examples/mnist/train_lenet.sh: line 3: 46528 Abort trap: 6 ./build/tools/caffe train --solver=examples/mnist/lenet_solver.prototxt

shelhamer · 2014-10-12T19:26:58Z

@BradNeuberg there are a few conflated issues in your comments:

boost compatibility of pycaffe: keeping with 1.55 and building by --with-python or switching to 1.56 and installing boost-python too works as long as the libstdc++ advice is followed
the power layer gradient failures are covered in TestPowerGradientShiftZero, TestPowerGradient fail with certain boost #1252
the LeNet example error CPU-only Mode: cannot make GPU call. is exactly what it it says: that example is configured for GPU mode by default. Change the solver config for CPU mode and you'll be set: https://github.com/BVLC/caffe/blob/master/examples/mnist/lenet_solver.prototxt#L24-L25

Please follow-up with usage questions on the caffe-users mailing list.

sanjeev87 · 2014-10-12T19:31:58Z

The issue of the three tests failing has been reported by multiple users now and it is now being followed in multiple threads..

This is the place in the mailing list that the discussion is on https://groups.google.com/forum/#!searchin/caffe-users/making$20runtest/caffe-users/TcCx-qjTLp8/2WfQHSOrlmUJ

[ FAILED ] 3 tests, listed below:
[ FAILED ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU

BradNeuberg · 2014-10-12T19:52:37Z

Thanks for responding so quickly! I've installed Boost and boost-python 1.56 and made sure that libstdc++ is used; note that in the README it doesn't say to edit the brew config file for boost-python. Not sure if that is a README bug. I've tried editing that file to ensure boost-python doesn't use clang and it doesn't seem to work.

I'll try the other things you've mentioned to see if I can get it working. Thanks for your help!

mprat · 2014-11-14T20:03:32Z

@BradNeuberg I just made it work on my machine using MKL and Boost 1.55. Make sure you are really careful about your BLAS_LIBS and BLAS_INCLUDE in the Makefile.config when switching to MKL.

beejisbrigit · 2014-11-19T08:55:48Z

I am also running into the same issues @BradNeuberg when doing a fresh "newbie" install of caffe. Is there a cleaner way to revert to boost 1.55, other than running "brew edit boost" and changing the url/sha1 for the package version? "brew install boost 1.55" and some variations of 1.55.## didn't work. Switching the package url does appear to be working however.

mprat · 2014-11-19T15:18:22Z

cd /usr/local
git checkout a252214 /usr/local/Library/Formula/boost.rb
brew install --build-from-source --with-python --fresh -vd boost

(I put it here: https://github.com/BVLC/caffe/wiki/Installation-(OSX) for the future)

svanschalkwyk · 2015-01-26T17:03:33Z

Comments about Boost version are red herrings. I ran into the same issues. They disappeared when I included Python into the build.
Runtest now executes cleanly.

houxianxu · 2015-06-25T01:16:10Z

When I run git checkout a252214 /usr/local/Library/Formula/boost.rb, there is an error.
error: pathspec 'a252214' did not match any file(s) known to git.

Any idea about it, thanks!

mprat · 2015-07-09T01:51:51Z

I actually had this problem myself recently - it looks like someone went through the homebrew commit history and the commit has changed. If you go through and look for the commit hash for boost.rb that is for version 1.55, you can find the right hash. Alternatively, you can do the following:

brew edit boost

And change the first two lines (the url and sha256 ones) to:

url "https://downloads.sourceforge.net/project/boost/boost/1.57.0/boost_1_57_0.tar.bz2"
sha1 "e151557ae47afd1b43dc3fac46f8b04a8fe51c12"

The newest version of caffe should work with boost 1.57 in this configuration - no need to revert back to boost 1.55.

ghost · 2019-10-16T15:18:37Z

how to get rid of this problem
LD -o .build_release/lib/libcaffe.so.1.0.0
ld: library not found for -lboost_python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [.build_release/lib/libcaffe.so.1.0.0] Error 1
please someone help, I did everything..I am on Mac Mojave

zimenglan-sysu-512 · 2019-11-07T07:44:36Z

hi @mekong0404
did u solve it?

shelhamer mentioned this issue Oct 1, 2014

unable to install boost with python #1184

Closed

roseperrone mentioned this issue Oct 8, 2014

Runtime error when importing boost #1235

Closed

shelhamer closed this as completed Oct 10, 2014

shelhamer reopened this Oct 10, 2014

shelhamer added the compatibility label Oct 10, 2014

shelhamer closed this as completed Jan 15, 2015

yosinski mentioned this issue Jan 18, 2015

Updated doc to suggest boost 1.57 #1747

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

boost --with-python required on osx for pycaffe target #465 #1193

boost --with-python required on osx for pycaffe target #465 #1193

sanjeev87 commented Sep 30, 2014

shelhamer commented Oct 1, 2014

shelhamer commented Oct 1, 2014

roseperrone commented Oct 6, 2014

shuggiefisher commented Oct 8, 2014

roseperrone commented Oct 8, 2014

shelhamer commented Oct 8, 2014

sanjeev87 commented Oct 8, 2014

roseperrone commented Oct 9, 2014

shelhamer commented Oct 10, 2014

shelhamer commented Oct 10, 2014

BradNeuberg commented Oct 12, 2014

BradNeuberg commented Oct 12, 2014

BradNeuberg commented Oct 12, 2014

BradNeuberg commented Oct 12, 2014

shelhamer commented Oct 12, 2014

sanjeev87 commented Oct 12, 2014

BradNeuberg commented Oct 12, 2014

mprat commented Nov 14, 2014

beejisbrigit commented Nov 19, 2014

mprat commented Nov 19, 2014

svanschalkwyk commented Jan 26, 2015

houxianxu commented Jun 25, 2015

mprat commented Jul 9, 2015

ghost commented Oct 16, 2019

zimenglan-sysu-512 commented Nov 7, 2019