-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build from source if the ta-lib is not installed #135
base: master
Are you sure you want to change the base?
Conversation
setup.py
Outdated
from distutils.dist import Distribution | ||
|
||
display_option_names = Distribution.display_option_names + ['help', 'help-commands'] | ||
query_only = any('--' + opt in sys.argv for opt in display_option_names) or len(sys.argv) < 2 or sys.argv[1] == 'egg_info' | ||
|
||
# Use setuptools for querying the package, normal builds use distutils |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It breaks on python setup.py sdist
if using distutils
here in my environment:
$ python setup.py sdist
setup.py:81: UserWarning: Cannot find ta-lib library, will try to build from source.
'Cannot find ta-lib library, will try to build from source.'
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'setup_requires'
warnings.warn(msg)
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
warnings.warn(msg)
running sdist
running check
error: [Errno 2] No such file or directory: 'venv/lib/python2.7/lib-dynload/lib-dynload'
@mrjbq7 Would you please detail why the following code block for setup
is essential here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I needed it to work for some commands only available in setuptools
and not distutils
, for example python setup.py egg_info
. But more importantly, I wanted to be able to run "query only" commands like python setup.py --help
without having import errors if numpy
and cython
weren't available.
I'd love to stop using setuptools
and just use distutils
if we can refactor the setup.py
to still allow --help
with no numpy or cython.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mrjbq7 The requirement is quite strict. I've left it unchanged. The problem I met should only be happened in my environment and there might be some problem with my virtualenv.
Shouldn't we use a |
@mrjbq7 No, we'd better not. Otherwise the vendor code in |
Oh I missed the
|
4dbbe0a
to
2d04cd1
Compare
Right, how do you imagine we keep up to date with the underlying C library changes? I'm not trying to fork it or have a stale version included here. Also, does this approach work on Windows? |
@mrjbq7 The last commit of TA-Lib in svn is 2013-04-03 . I tried to ask the owner to migrate from SourceForge to GitHub but there's still no reply. I think the library is out of maintain right now. So I suppose the TA-Lib code will not be changed any more. I've migrated the code from svn to github https://github.com/TA-Lib/ta-lib and tagged The latest version released in https://ta-lib.org/hdr_dw.html is I also compare the ta-lib-0.4.0-src.tar.gz from the official site and the tagged version in svn trunk. They're not fully identical but identical in source code files:
But what if the TA-Lib project is back to life in the future? If it's still maintained in svn, we'll mirror the code in svn trunk to https://github.com/TA-Lib/ta-lib and update the
It's not tested yet since I don't have a Windows machine. I'll try to setup a Windows virtual machine first and test it later. |
2d04cd1
to
64d2d31
Compare
I wish this could be preserved automatically but it seems to need manually re-adding everytime we regenerate the C file.
git-subtree-dir: vendor/ta-lib git-subtree-split: 2ee32d04e2b7686dc0931e17c6a420682c46f3a3
64d2d31
to
ce86339
Compare
@mrjbq7 Yes. it's working now. Unfortunately, I was not able to run
and install the package:
Then, I created a directory
|
ping @mrjbq7 |
Hi @mckelvin, thanks for the ping. I meant to look at this in the last few days but didn't get to it. Will try again soon. The idea is very interesting, the implications I wanted to understand (e.g., |
@mrjbq7 Here is the reason why I have to use pip wheel: I'm using the default compiler (Microsoft Visual C++ Compiler for Python 2.7) for Python in Windows. When building the python package, it need to find If you don't like VC++, you may also try MinGW, then setup the |
Hi, @mrjbq7 what's the status of the PR right now? Do you have any advice? |
The status is I haven't had time to get my windows builder to test it locally and I have been traveling (and for next 10 days). I wish I had a co-maintainer so I wasn't single point of failure here! I want to get this one done for sure but needed time. |
I've had a few people give me guidance that forking and maintaining the |
@mrjbq7 Not really. For example, in Ubuntu the ta-lib C library is not included in
|
@mckelvin TA-Lib absolutely is in apt.
~@ |
Look over #142 Windows install is absolutely easy via the installer. There is only a few ta-libs sources in git, the svn-updater, and a semi-maintained branch trying to reimplement the APIs, and to convert it from stateless. So perhaps there may be better TA-Lib in github soon, until then I would keep the source and wrapper separated. |
If we were to implement a install method, it would be better to write a method that handled it from third party sources depending on their OS/ENV than trying to maintain a copy of the source inside the wrapper's repository. |
@a904guy PPA for ta-lib is a good idea for Ubuntu users, but this PPA is not actively maintained, The latest upload date is 2012-02-29. There's even no corresponding package for Ubuntu Trusty (14.04). And what about CentOS/yum ? Another case is: When a user doesn't have root privilege and he just want to use the python ta-lib inside his own local The idea of including the third party source comes from https://github.com/redis/hiredis-py/tree/master/vendor , where hiredis is included as a I think it's not proper to use word maintain since we don't need to and should not do any modification with vendor code.
The code has been tested on Windows, macOS and Linux (Ubuntu Trusty), and it's compatible with all these 3 OSs. |
You're obviously not familiar with the ta-lib. It's EOL, dead. They never made 0.5 or 0.6 from the roadmap. The only progress that has been made since 2013 is the ta-lib-rt. Which isn't even the TA-Lib Team anymore. The TA-Lib hasn't been modified for a stable release since 2012. Even their own SVN trunk is only at 2013. [Problem]: There isn't a {version} of Ubuntu specific ppa. [Answer]: It doesn't matter as long as you have libc6-dev & libc-dev. Any of the deb files from that ppa still work. Those deb files will still work on a trusty environment, and can be installed without superuser permission. A user without superuser privileges can still build the source into their own ~/.local/bin or ~/bin which is part of their $PATH. dpkg-deb -x {ta-lib.deb from ppa}, and move a few files around Flavors: CentOS/yum/RHEL/Fedora/ect And your code doesn't work on the 3 OSs, it failed on Windows 10 Pro. Doesn't have dependency checking prior. As it failed to install because of numpy missing. Then when I installed numpy. It tried to build TA-Lib and failed again. On the other hand, I just ran the windows installer from Ta-Lib and it worked flawlessly... I haven't tested other OSs yet as I'm doing windows testing on a project at the moment. Which brings me to my point, it increases the development scope of the project from just the python module, to maintaining a build system as well. Nobody has time for that, especially the windows version... I'm failing to see the benefit of maintaining a system to build the source inside the wrapper. The wrapper is just that, a wrapper/interface to the library. It isn't the library. If you really want to make a python installer of a relic c project, why don't you make a new repository project, that is called Ta-Lib-Py-Installer, and subtree this repository, and maintain a build system of your own design? |
Sorry man, as someone who has been close to this project since it was just a thought in @mrjbq7 mind. I don't agree with adding the source and compilation to this repository. It might of worked well for Redis, but that is an active development project. TA-LIB is not. So I would vote 👎 on #135 |
As to an RPM based installation, I had to research a bit, but it appears I packaged ta-lib into a Source RPM, which may be trivially rebuild into binaries, back in 2008. File and notes are at: ftp:https://ftp.owlriver.com/pub/mirror/ORC/ta-lib/ Also, Red Hat for their Open Shift Enterprise later released SRPMs which are freely available at: https://ftp.redhat.com/redhat/linux/enterprise/6Server/en/RHOSE/SRPMS/ta-lib-0.4.0-1.el6op.src.rpm for their initial 'Open Shift Enterprise', which goes EOL later this year Red Hat OpenShift Enterprise I just rebuilt my packaging on CentOS 7 without issue |
I started a project to re-implement all the I had many of them done, but not the candlesticks. So there is still a feature gap. Not yet open-sourced. |
I tried that and it works well. I think it's better than this PR for Debian/Ubuntu users. Thanks for the idea that I didn't try before. Build in docker image wget https://ppa.launchpad.net/mario-mariomedina/talib/ubuntu/pool/main/t/ta-lib/libta-lib0_0.4.0-3_amd64.deb
wget https://ppa.launchpad.net/mario-mariomedina/talib/ubuntu/pool/main/t/ta-lib/ta-lib0-dev_0.4.0-3_amd64.deb
echo "1ef8872a2c12fa56346c877e4fbffba892e88367f89db63c57585a2a388790f5 libta-lib0_0.4.0-3_amd64.deb" | sha256sum -c -
echo "41424a778c9a85cac654341002794bb672fb04087a6f8fbcd0da71c932c2b1ee ta-lib0-dev_0.4.0-3_amd64.deb" | sha256sum -c -
dpkg -i libta-lib0_0.4.0-3_amd64.deb # XXX: Have to install this dep first manually
dpkg -i ta-lib0-dev_0.4.0-3_amd64.deb
pip install numpy
pip wheel --wheel-dir=wheelhouse . Runtime in wget https://ppa.launchpad.net/mario-mariomedina/talib/ubuntu/pool/main/t/ta-lib/libta-lib0_0.4.0-3_amd64.deb
echo "1ef8872a2c12fa56346c877e4fbffba892e88367f89db63c57585a2a388790f5 libta-lib0_0.4.0-3_amd64.deb" | sha256sum -c -
dpkg -i libta-lib0_0.4.0-3_amd64.deb
pip install numpy
pip install wheelhouse/TA_Lib-0.4.10-cp27-cp27mu-linux_x86_64.whl
It's irrelevant with the PR. You can also reproduce with the code in upstream that you have to install numpy manually.
Have you ever searched
I have to admit that the development scope increased by adding a new build system for the underling C library. Have you ever read 84fe612 ? It doesn't nothing but include C/C++ source code while compiling the Cython extension. Is this scope hard for you?
I'm not a Windows user. So I'm nobody?
I don't agree neither, but I have to if it comes to the initial question: How can I install the python I known that the TA-Lib C library is out of maintain, and that's the reason why I mirror the source to GitHub to make it easier to add it as a git subtree instead of coping code of the C library. Again it's a fallback feature.
I don't understand the relationship between a) the underlying library is active development project or not, b) it worked well to combine underlying c code with wrapper in the wrapper repo. Both are dirty but works.
Please read 84fe612 first and let me known how can I do it without changing the code in python ta-lib. My personal reason why sending this PR is to solve the question:
And this PR is trying to resolve a more common question (though it's not perfect yet):
For me, it's also acceptable to use deb files provided in mario-mariomedina's PPA. You can close this PR if you don't like this feature. |
@mrjbq7 Awesome! Why not open source it as a WIP project now? I know that @femtotrader also developed a similar project https://github.com/femtotrader/pandas_talib . So what's the difference between the two? |
I don't have a strong test suite but maybe if I can automate random inputs against ta-lib that might work. Needs some time and a good name. |
Also, my version is more numpy-centric. Seems like @femtotrader has more stuff for pandas which is neat too. |
64GBs of ram.
…On Fri, May 5, 2017, 9:37 AM John Benediktsson ***@***.***> wrote:
Also, my version is more numpy-centric. Seems like @femtotrader
<https://github.com/femtotrader> has more stuff for pandas which is neat
too.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#135 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAYqcbUPTAVTHQ-YEnCrwCir03qHvOLZks5r2yYMgaJpZM4L3h9z>
.
|
This PR is a break-out part of #111 (comment) after PR #132 , which will include the TA-Lib C code at compile time if the library is not installed. It will make it easier to install the python ta-lib module by just run
pip install TA-Lib
without worrying about how to setup the TA-Lib dependency.The first 2 commits are generated by
git subtree add --prefix vendor/ta-lib https://github.com/TA-Lib/ta-lib.git v0.4.0 --squash (https://github.com/TA-Lib/ta-lib/releases/tag/v0.4.0)
.This PR has been verified (by running
make test
) on macOS(EI Capitan) and Linux(Ubuntu 14.04). I'll appreciate if anyone could help to test on Windows.