using nbind / node-gyp with lot's of extra libraries - how to tell node-gyp where all sources are? #35
Comments
|
You should probably compile Any kind of notes or a tutorial would be very welcome if you get it working. |
|
If you simply install |
|
Yeah I already compiled it, as was instructed by the I will sort out a detailed description, as soon as it works ;) |
|
I looked into the |
|
So out of every So far so good the theory, but I am not quite getting how I can then (n)bind to these libraries.
#include void sayHey(std::string name) { #include "nbind/nbind.h" NBIND_GLOBAL() { That should work, right? |
|
Looking at vg's It seems like you'll have quite a few classes and methods to bind. Looking at the class In theory you should be able to create a new #include "vg.hpp"
#include "nbind/api.h"
namespace vg {
// This inherits methods from VG for directly binding them when possible.
class JS_VG : public VG {
public:
// Anything taking a callback will have to be wrapped!
for_each_node(nbind::cbFunction lambda) { /* your code here */ }
// Alternatively you can give your new implementation a new name.
js_for_each_node(nbind::cbFunction lambda) { /* your code here */ }
};
} // namespace
#include "nbind/nbind.h"
// You can rename your JS_VG reimplementation back to VG on the JavaScript side.
NBIND_CLASS(vg :: JS_VG, VG) {
// You don't have to reimplement this, just tell nbind that it exists.
method(normalize);
// This C++ method is overloaded so we need to tell nbind which arguments it takes
// (otherwise the C++ compiler cannot distinguish between overloads).
multimethod(has_node, args(Node *));
// If we want to bind several overloads, they need to be renamed uniquely
// because nbind doesn't yet support overloading by type on the JavaScript side.
multimethod(has_node, args(id_t), "has_node_id");
// We've overloaded this with a new implementation.
multimethod(for_each_node, args(nbind::cbFunction));
// Alternatively if we renamed it uniquely, we can just rename it back here
// and have nbind autodetect the arguments. Actually we can use camelCase
// which is more common in JavaScript.
method(js_for_each_node, "forEachNode");
}Note that I didn't really check the syntax of the above very carefully, but hopefully it will get you started... I think as long as you don't need to call your new C++ methods (reimplemented for nbind purposes) from other classes, you can put your new class declaration in the class JS_VG; |
|
It seems You could look for command line flags in
I think you should start with a really simple C++ application that will just call a single method from |
|
Hello, I'm from the vg project and vaguely understand the build system. Unfortunately I know nothing about We pull in a lot of libraries as Git submodules, because our target environment is academic clusters and other places where users don't have access to a package manager. As part of our Makefile, we manually copy library headers into To build its dependencies, vg relies on a bunch of recursive Make calls. Some of the dependencies (SDSL-lite) demand to be installed using their own shell scripts. We've never actually written any code to install vg globally; we've just been adding it to our PATHs manually. Projects that depend on vg generally include it as a Git submodule, build it with recursive Make, and just point to its includes and libraries inside the submodule. Is it possible to use |
|
@adamnovak |
|
@subwaystation It seems you should copy Once that works, try: export CXX=echo
makeNow it should print the necessary |
|
I was able to compile a small example In the attachment one can find the @jjrv However, when trying to build with node-gyp, I ran into the following problem: CXX(target) Release/obj.target/nbind/JS_VG.o The Here is the current version of my |
|
Or ist node-gyp not able to grasp my |
|
It should use whatever compiler is defined in the environment variable "make_global_settings": [
["CXX", "/usr/local/bin/g++"],
["LINK", "/usr/local/bin/g++"]
] |
|
Then I run into the following: Although, I did not even specify this command line option in |
|
Try removing Not sure how Also, I think
|
|
I did comment them out. So I got rid of one line of You are right about the flags, I just removed them from my |
|
Where else could |
|
Did you remove it also from |
|
Yes, I did: "OTHER_CPLUSPLUSFLAGS": [
"-O3",
"-std=c++11",
# "-stdlib=libc++"
],
# "OTHER_LDFLAGS": ["-stdlib=libc++"] |
|
Seems like
#!/bin/sh
skip=0
for arg; do
shift
[ "$skip" = "1" ] && skip=0 && continue
case "$arg" in
-stdlib=*) ;;
*) set -- "$@" "$arg" ;;
esac
done
/usr/local/bin/g++ $@Just make that executable and point It's a dirty hack but should get the job done. I'm sorry it gets so messy... |
|
Well, better messy than not working :) Sorry to bother you again, but now I am stuck here: I thought |
|
I just found out, that |
|
Gyp "relativizes" paths (I can't remember how it handles when there's no space between the The compiler is actually executed in a new |
|
Putting For some reason, no object files for nbind are created. Do I have to adjust something especially for If I do not |
|
With You could try an absolute path to vg/include. But that will only work On Fri, Nov 11, 2016 at 10:02 AM, subwaystation
|
|
Changing all necessary parameters to absolute paths like |
|
And there's definitely a "gssw.h" in /Users/my_user_name/git/vg/include? On Fri, Nov 11, 2016 at 10:19 AM, subwaystation
|
|
Yes there is: Just to make sure, I did not a bad typo or something, here is my current binding.gyp: |
|
@adamnovak In your |
|
Adding the |
|
Another thing, the result you pasted... -DNODE_GYP_MODULE_NAME=nbind -DUSING_UV_SHARED=1 -DUSING_V8_SHARED=1 -DV8_DEPRECATION_WARNINGS=1 -D_DARWIN_USE_64_BIT_INODE=1 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DBUILDING_NODE_EXTENSION -I/Users/heumos/.node-gyp/7.0.0/include/node -I/Users/heumos/.node-gyp/7.0.0/src -I/Users/heumos/.node-gyp/7.0.0/deps/uv/include -I/Users/heumos/.node-gyp/7.0.0/deps/v8/include -I../node_modules/nan -I../node_modules/nbind/include -Os -gdwarf-2 -mmacosx-version-min=10.7 -arch x86_64 -Wall -Wendif-labels -W -Wno-unused-parameter -std=c++11 -stdlib=libc++ -fno-rtti -fno-threadsafe-statics -O3 -std=c++11 -MMD -MF ./Release/.deps/Release/obj.target/nbind/JS_VG.o.d.raw -c -o Release/obj.target/nbind/JS_VG.o ../JS_VG.cppI don't see any |
|
That did the trick ;) Now I am hanging here: |
|
So theoretically snappy is available: |
|
And added under |
|
Try removing the |
|
It's compiling now, thank you very much @jjrv. If I could get the following example working, I would be very happy: #include "../vg/src/vg.hpp"
#include "nbind/api.h"
using namespace vg;
#include "nbind/nbind.h"
NBIND_CLASS(VG) {
// Add an empty constructor
construct<>();
// Check if the graph is empty
method(empty);
// Generate hash out of graph
method(hash);
}And the corresponding var nbind = require('nbind');
var lib = nbind.init().lib;
var emptyGraph = new lib.VG();
console.log(emptyGraph.hash());
console.log(emptyGraph.empty());Output for I would assume it is a binding problem which I can't figure out. Or a C++ problem. As soon as I am able to run your given, more complex example on the more top of this issue, I will make a point by point tutorial for binding |
|
I think it's because |
|
So I tried this surrounding all the library stuff: But I even tried Another theory: Could the upper error (Symbol not found, ...) have been occurred, because I did not specify all the static libraries one by one? Because when you take a look at the |
|
You're missing a "-" on the first "-Wl". I also notice you have no "-lvg" in your list here; that's the library you On Fri, Nov 11, 2016 at 3:24 PM, subwaystation [email protected]
|
|
Yeah, sorry about the However, the |
|
@jjrv I would still like to get your little bit more complex example to run: #include "../vg/src/vg.hpp"
#include "nbind/api.h"
namespace vg {
// This inherits methods from VG for directly binding them when possible.
class JS_VG : public VG {
// Anything taking a callback will have to be wrapped!
static void for_each_node(nbind::cbFunction lambda) { /* your code here */ }
// Alternatively you can give your new implementation a new name.
static void js_for_each_node(nbind::cbFunction lambda) { /* your code here */ }
};
} // namespace
#include "nbind/nbind.h"
// You can rename your JS_VG reimplementation back to VG on the JavaScript side.
NBIND_CLASS(vg :: JS_VG, VG) {
// You don't have to reimplement this, just tell nbind that it exists.
method(normalize);
// This C++ method is overloaded so we need to tell nbind which arguments it takes
// (otherwise the C++ compiler cannot distinguish between overloads).
multimethod(has_node, args(Node *));
// If we want to bind several overloads, they need to be renamed uniquely
// because nbind doesn't yet support overloading by type on the JavaScript side.
multimethod(has_node, args(id_t), "has_node_id");
// We've overloaded this with a new implementation.
multimethod(for_each_node, args(nbind::cbFunction));
// Alternatively if we renamed it uniquely, we can just rename it back here
// and have nbind autodetect the arguments. Actually we can use camelCase
// which is more common in JavaScript.
method(js_for_each_node, "forEachNode");
}But there seem to be some issues: Furthermore, I don't see, what JavaScript callbacks inside my C++ code are useful for. Do I get better performance? Or can I access JavaScript Code from within C++? |
|
I think you may be missing a I think the Javascript callback thing is just a necessary piece of On Fri, Nov 11, 2016 at 4:21 PM, subwaystation [email protected]
|
|
@subwaystation that code was meant to illustrate First of all, to pass namespace vg {
// This may be needed depending on whether the class has members
// that cannot be used without a wrapper.
class JS_Node : public Node {
public:
// Stuff goes here
};
} // namespace
// This lets nbind understand references to Node objects.
// You can put methods here that don't need wrappers,
// but if you need the JS_Node class it might be cleaner to put them all there.
NBIND_CLASS(vg :: Node, _Node) {}
NBIND_CLASS(vg :: JS_Node, Node) {
inherit(vg :: Node);
// More stuff goes here
}You don't have to put the stuff in at first, but might need it to do something meaningful with the Node object from JavaScript. Again haven't tested if the above compiles. |
|
Callbacks currently need handling because std::string myString;
std::vector<std::string> myVector;
lambda.call<int>(myString, myVector)` That passes a string and an array of strings to a JavaScript callback, and casts its return value to It might be possible to automatize this as well, so Another issue that you'll run into is that |
|
@jjrv Shall I add the tutorial here, or where would be the best place to put it? |
|
@subwaystation Easiest is to just copy&paste it here in this issue or do a PR putting it in the |
|
Tutorial Example of In order for this example to work, your project structure should look like the following:
Create #include "../vg/src/vg.hpp"
#include "nbind/api.h"
#include "nbind/nbind.h"
namespace vg {
NBIND_CLASS(VG) {
// Add an empty constructor
construct<>();
// Check if the graph is empty
method(empty);
// Generate hash out of graph
method(hash);
}
}Then we have to add relevant scripts to our `package.json', which should look like the following: {
"scripts": {
"autogypi": "autogypi",
"node-gyp": "node-gyp"
},
"dependencies": {
"autogypi": "^0.2.2",
"nbind": "^0.3.5",
"node-gyp": "^3.4.0"
}
}Although, verify that the program versions do match the ones on your machine. I left emscripten out here, because at it's current state it is not able to handle multithreading well enough in order to be applicable to As
#!/bin/sh
# This script calls g++ (the homebrew C++ compiler on MAC OS) with given command line
# arguments. It removes the "-stdlib=libc++" flag (and its parameter), which node-gyp
# will add on OS X hosts causing a compiler error.
skip=0
for arg; do
shift
[ "$skip" = "1" ] && skip=0 && continue
case "$arg" in
-stdlib=*) ;;
*) set -- "$@" "$arg" ;;
esac
done
/usr/local/bin/g++ $@Don't forget to make it executable via:
Now we are ready for: Among other files a {
"make_global_settings": [
["CXX", "../node_modules/nbind/bin/g++"],
["LINK", "../node_modules/nbind/bin/g++"]
],
"targets": [
{
"includes": [
"auto.gypi"
],
"sources": [
"JS_VG.cpp"
],
"conditions": [
["OS=='mac'", {
"xcode_settings": {
"OTHER_CPLUSPLUSFLAGS": [
"-c",
"-msse4.1",
"-fopenmp",
"-frtti",
"-ggdb",
"-g",
"-I../../vg/",
"-I../../vg/include"
],
"OTHER_LDFLAGS": [
"-msse4.1",
"-fopenmp",
"-ggdb",
"-g",
"-I../../vg/",
"-I../../vg/include",
"-lvg",
"-lsdsl",
"-lxg",
"-lvcflib",
"-lgssw",
"-lrocksdb",
"-lhts",
"-lgcsa2",
"-lprotobuf",
"-lraptor2",
"-lgfakluge",
"-lsupbub",
"-lpinchesandcacti",
"-l3edgeconnected",
"-lsonlib",
"-lm",
"-lpthread",
"-ly",
"-lbz2",
"-lsnappy",
"-ldivsufsort",
"-ldivsufsort64",
"-ljansson",
"-L../vg/src",
"-L../vg/lib"
]
},
}],
]
}
],
"includes": [
"auto-top.gypi"
]
}Feel free to add parameters for any other If everything went fine, then create var nbind = require('nbind');
var lib = nbind.init().lib;
var emptyGraph = new lib.VG();
console.log(emptyGraph.hash());
console.log(emptyGraph.empty());Then run it with node: And it should print out a hash value and |
|
@subwaystation Wow, thank you! I suppose you'll give permission to add it in the repository and later on a project website? How do I credit you, CC-BY license with author "subwaystation" and a link to your Github profile? |
|
@jjrv Well, thank you! Without your help and @adamnovak I wouldn't have come that far ;) |
|
Thanks again, I've put the tutorial in |
Take a C++ framework, e.g. https://github.com/vgteam/vg/tree/master/src having lots of dependencies when building.
Now I would like to e.g. nbind all classes in file https://github.com/vgteam/vg/tree/master/src/vg.cpp. The problem is that when executing
"npm run -- node-gyp configure build" the following happens:
../src/vg.hpp:11:10: fatal error: 'omp.h' file not found
#include <omp.h>
^
1 error generated.
This error does not occur if I build the project as specified in the wiki. Following this issue http:https://stackoverflow.com/questions/25990296/how-to-include-omp-h-in-os-x I have to state that gcc 4.9.3 via homebrew is installed and globally available. Maybe node-gyp does not use gcc / g++?
Is there anyway to pass all the source files necessary for the build to node-gyp or how do I proceed here? (I have a CMakeLists.txt file containing over 3000 lines of sources, can I pass this to node-gyp? Or would node-gyp be able to do this automatically?)
All the best.
The text was updated successfully, but these errors were encountered: