Enable LLVM NVPTX back-end #19323

maleadt · 2016-11-14T20:43:44Z

(Assumed) non-controversial parts of #19302, enables the LLVM NVPTX back-end (1 MB, 3% file size increase for libLLVM) and adds some crucial patches related to JuliaGPU/CUDAnative.jl#13 (all of them merged already):

D9168: segfault in LLVM
D24300: adds lowering for some intrinsics
D23597: dependency of D24300

tkelman · 2016-11-14T23:24:56Z

deps/llvm.mk

@@ -481,6 +485,12 @@ $(eval $(call LLVM_PATCH,llvm-arm-fix-prel31))
 $(eval $(call LLVM_PATCH,llvm-D25865-cmakeshlib))
 endif # LLVM_VER

+ifeq ($(LLVM_VER_SHORT),3.9)


why is this separate from the above list?

Put it there in my branch to avoid merge conflicts, will change.

tkelman · 2016-11-14T23:38:41Z

deps/llvm.mk

@@ -78,7 +78,11 @@ LLVM_CFLAGS += $(CFLAGS)
 LLVM_CXXFLAGS += $(CXXFLAGS)
 LLVM_CPPFLAGS += $(CPPFLAGS)
 LLVM_LDFLAGS += $(LDFLAGS)
-LLVM_TARGETS := host
+ifeq ($(LLVM_USE_CMAKE),1)
+ LLVM_TARGETS := host;NVPTX


we don't usually indent this kind of thing in makefiles

Fair enough.

tkelman · 2016-11-14T23:39:50Z

will need to be tested on the buildbots to make sure it builds everywhere, if it's going to be enabled on all platforms by default

maleadt · 2016-11-16T15:34:58Z

Travis is happy. Any other configurations to manually test this on?

tkelman · 2016-11-16T15:51:24Z

all of the buildbots

maleadt · 2016-11-16T16:09:49Z

Right, but those only run on master and I don't have an account to request builds on this PR's branch.
So how do I make them run?

ViralBShah · 2016-11-17T05:58:47Z

Perhaps @tkelman or @staticfloat can share the creds with you for kicking off builds.

tkelman · 2016-11-17T18:37:01Z

I triggered a set of builds of this branch, they're running now. Something unrelated is wrong on the mac buildbot, it's having python issues with building the docs.

maleadt · 2016-11-17T21:56:27Z

The windows bots are acting up:

llvm-3.7.1/lib/Target/NVPTX/TargetInfo/NVPTXTargetInfo.cpp:10:19: fatal error: NVPTX.h: No such file or directory
 #include "NVPTX.h"
                   ^
compilation terminated.

Not sure how that could happen, it seems like LLVM 3.7 doesn't build if the NVPTX target is enabled?
I'll have a closer look.

ViralBShah · 2016-11-17T22:50:52Z

I guess things would be easier if were already using llvm 3.9.

ViralBShah · 2016-12-19T18:51:13Z

It does seem that llvm 3.9 is awfully close - but are we going to move to it for 0.6? If not, since it is late in the cycle, should we merge this?

Having CUDANative.jl work with Julia 0.6 out of the box would be a nice feature to add, like we did ARM and POWER for 0.5.

tkelman · 2016-12-19T19:29:01Z

LLVM 3.7 doesn't build if the NVPTX target is enabled

that's a no, we shouldn't merge this without upgrading llvm

tkelman · 2016-12-22T08:51:54Z

Actually we should probably get the LLVM patches here onto master before we upgrade, so they go into the homebrew and appveyor binaries (I'll need to rebuild the latter) and the CI build matches what you get in a source build.

maleadt · 2016-12-22T08:55:06Z

OK, I can split those off. I just rebased this branch on top of your LLVM 3.9 update PR to first see if NVPTX will build correctly on Windows.

tkelman · 2016-12-22T08:56:05Z

Sure, worth testing. Do you have a login to the buildbots, or need me to fire that off? (edit: running now)

maleadt · 2016-12-22T09:08:21Z

Shouldn't we also include the NVPTX back-end in the homebrew/appveyor binaries, if we want to ship that as part of Julia 0.6 @ LLVM 3.9? To do so, we could shield the LLVM_TARGETS += NVPTX behind a check against 3.9+ and merge this PR (if it builds correctly on all platforms).

That said, unless I still manage to create a PR for packaging an additional 'dev tools' tarball (with llvm-config, include/llvm/*, etc), users will have to build from source anyway.

tkelman · 2016-12-22T09:11:30Z

users will have to build from source anyway

Sounds like it wouldn't add all that much value to turn on by default as long as this is the case?

maleadt · 2016-12-22T09:19:17Z

Ah yes, this will fail until JuliaCI/julia-buildbot#54 is resolved of course.

-- Configuring incomplete, errors occurred!
CMake Error at CMakeLists.txt:3 (cmake_minimum_required):
  CMake 3.4.3 or higher is required.  You are running version 3.3.2

ViralBShah · 2016-12-22T09:19:26Z

Yes, if it builds on all platforms, this is a great idea.

tkelman · 2016-12-22T12:50:23Z

we could shield the LLVM_TARGETS += NVPTX behind a check against 3.9+ and merge this PR (if it builds correctly on all platforms).

Let's do that then - just your commit, modified to only build the nvptx backend when using llvm >= 3.9. This branch seems to be working everywhere, but let's rearrange it so we can merge it independently. I'll need to rebase #19678 on top of #19681 anyway, and it'll have to wait for the homebrew formula and some benchmarking.

tkelman · 2016-12-22T14:12:12Z

just your commit

~~Take mine out - can't merge this while 7851f77 is in here~~ all set now

tkelman · 2016-12-22T14:22:37Z

deps/llvm.mk

+LLVM_TARGET_STRING := $(subst $(SPACE),;,$(LLVM_TARGETS))
+else
+COMMA := ,
+LLVM_TARGET_STRING := $(subst $(SPACE),$(COMMA),$(LLVM_TARGETS))


do we need to worry about this case? it's not possible to use configure with 3.9 or newer, so I'd simplify this as something like

ifeq ($(LLVM_VER_SHORT),$(filter $(LLVM_VER_SHORT),3.3 3.4 3.5 3.6 3.7 3.8)) LLVM_TARGET_STRING := host else LLVM_TARGET_STRING := host;NVPTX endif

if that works

I figured I'd keep it a little more extensible, but sure that works too. I'll update the PR.

people can always override LLVM_TARGETS (or LLVM_TARGET_STRING or whatever) in their Make.user, right?

pretty much, yeah.

then I guess we could keep it called LLVM_TARGETS just for minimal disruption to anyone who might already be doing this?

Use of the NVPTX back-end is only tested on LLVM 3.9, in combination with the CUDAnative.jl package.

singam-sanjay · 2017-03-19T18:08:14Z

deps/llvm.mk

@@ -73,17 +73,22 @@ LLVM_LIBCXX_TAR:=$(SRCDIR)/srccache/libcxx-$(LLVM_TAR_EXT)
 endif
 endif # LLVM_VER != svn

+# Figure out which targets to build
+ifeq ($(LLVM_VER_SHORT),$(filter $(LLVM_VER_SHORT),3.3 3.4 3.5 3.6 3.7 3.8))


Does this condition evaluate to true when LLVM_VER_SHORT is svn ?

It shouldn't.

How do you enable the NVPTX back-end for svn builds ?

It should be enabled since the false branch enables nvptx backend............

tkelman reviewed Nov 14, 2016

View reviewed changes

maleadt force-pushed the tb/llvm_nvptx branch 2 times, most recently from b103efd to ef5b0e2 Compare November 15, 2016 16:53

ViralBShah added domain:gpu Affects running Julia on a GPU compiler:codegen Generation of LLVM IR and native code labels Dec 19, 2016

maleadt force-pushed the tb/llvm_nvptx branch from ef5b0e2 to 94c8866 Compare December 22, 2016 08:47

ViralBShah added this to the 0.6.0 milestone Dec 22, 2016

maleadt force-pushed the tb/llvm_nvptx branch 2 times, most recently from 8358eee to 0a55584 Compare December 22, 2016 14:10

maleadt force-pushed the tb/llvm_nvptx branch from 0a55584 to 1658320 Compare December 22, 2016 14:17

tkelman reviewed Dec 22, 2016

View reviewed changes

maleadt force-pushed the tb/llvm_nvptx branch from 1658320 to eaf990f Compare December 22, 2016 14:32

LLVM 3.9+: enable the NVPTX LLVM back-end, and add patches fixing bugs.

706c764

Use of the NVPTX back-end is only tested on LLVM 3.9, in combination with the CUDAnative.jl package.

maleadt force-pushed the tb/llvm_nvptx branch from eaf990f to 706c764 Compare December 22, 2016 14:37

tkelman approved these changes Dec 22, 2016

View reviewed changes

tkelman merged commit 370886c into master Dec 22, 2016

tkelman deleted the tb/llvm_nvptx branch December 22, 2016 15:44

tkelman mentioned this pull request Dec 29, 2016

Default build system support for CUDA/PTX #19302

Closed

singam-sanjay reviewed Mar 19, 2017

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable LLVM NVPTX back-end #19323

Enable LLVM NVPTX back-end #19323

maleadt commented Nov 14, 2016

tkelman Nov 14, 2016

maleadt Nov 15, 2016

tkelman Nov 14, 2016

maleadt Nov 15, 2016

tkelman commented Nov 14, 2016

maleadt commented Nov 16, 2016

tkelman commented Nov 16, 2016

maleadt commented Nov 16, 2016

ViralBShah commented Nov 17, 2016

tkelman commented Nov 17, 2016

maleadt commented Nov 17, 2016

ViralBShah commented Nov 17, 2016

ViralBShah commented Dec 19, 2016 •

edited

Loading

tkelman commented Dec 19, 2016

tkelman commented Dec 22, 2016 •

edited

Loading

maleadt commented Dec 22, 2016

tkelman commented Dec 22, 2016 •

edited

Loading

maleadt commented Dec 22, 2016

tkelman commented Dec 22, 2016

maleadt commented Dec 22, 2016

ViralBShah commented Dec 22, 2016

tkelman commented Dec 22, 2016

tkelman commented Dec 22, 2016 •

edited

Loading

tkelman Dec 22, 2016 •

edited

Loading

maleadt Dec 22, 2016

tkelman Dec 22, 2016

maleadt Dec 22, 2016

tkelman Dec 22, 2016

singam-sanjay Mar 19, 2017

yuyichao Mar 19, 2017

singam-sanjay Mar 19, 2017 •

edited

Loading

yuyichao Mar 19, 2017

Enable LLVM NVPTX back-end #19323

Enable LLVM NVPTX back-end #19323

Conversation

maleadt commented Nov 14, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkelman commented Nov 14, 2016

maleadt commented Nov 16, 2016

tkelman commented Nov 16, 2016

maleadt commented Nov 16, 2016

ViralBShah commented Nov 17, 2016

tkelman commented Nov 17, 2016

maleadt commented Nov 17, 2016

ViralBShah commented Nov 17, 2016

ViralBShah commented Dec 19, 2016 • edited Loading

tkelman commented Dec 19, 2016

tkelman commented Dec 22, 2016 • edited Loading

maleadt commented Dec 22, 2016

tkelman commented Dec 22, 2016 • edited Loading

maleadt commented Dec 22, 2016

tkelman commented Dec 22, 2016

maleadt commented Dec 22, 2016

ViralBShah commented Dec 22, 2016

tkelman commented Dec 22, 2016

tkelman commented Dec 22, 2016 • edited Loading

tkelman Dec 22, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

singam-sanjay Mar 19, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ViralBShah commented Dec 19, 2016 •

edited

Loading

tkelman commented Dec 22, 2016 •

edited

Loading

tkelman commented Dec 22, 2016 •

edited

Loading

tkelman commented Dec 22, 2016 •

edited

Loading

tkelman Dec 22, 2016 •

edited

Loading

singam-sanjay Mar 19, 2017 •

edited

Loading