Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support LLVM 13 #202

Merged
merged 9 commits into from
Apr 3, 2023
Merged

Support LLVM 13 #202

merged 9 commits into from
Apr 3, 2023

Conversation

RyanGlScott
Copy link
Contributor

@RyanGlScott RyanGlScott commented Mar 15, 2023

This contains two key fixes needed to support LLVM 13:

Support METADATA_ARG_LIST

Fixes #174.

Support newer inline asm codes

One code (CST_CODE_INLINEASM_OLD3) was introduced in LLVM 13, and another (CST_CODE_INLINEASM) was introduced in LLVM 14. For the most part, they are parsed identically to previous inline asm codes, but with some minor differences. I have consolidated the logic for parsing all inline asm codes into a single parseInlineAsm function.

Fixes #184.


The rest of the commits are needed to make the test case for #174 work, which exercises quite a bit of pretty-printing functionality that was not previously tested. These include fixes for GaloisInc/llvm-pretty#105, GaloisInc/llvm-pretty#107, #211, and #212. This also requires bumping the llvm-pretty submodule to bring in the changes from GaloisInc/llvm-pretty#108.

@RyanGlScott RyanGlScott changed the title Support METADATA_ARG_LIST Support LLVM 13 Mar 16, 2023
@RyanGlScott
Copy link
Contributor Author

RyanGlScott commented Mar 24, 2023

I have good news and I have bad news.

The good news is that I have merged #210, which will allow us to write test cases that require LLVM 13 or later.

The bad news is that while writing a test case for #174, I discovered a slew of metadata-related bugs in llvm-pretty and llvm-pretty-bc-parser:

None of these issues are specifically about LLVM 13—they're quite old bugs that have laid dormant due to disasm-test not having any substantial test cases involving metadata. I expect that it will be straightforward to fix these bugs, although I might have to make this PR a bit more bloated in order to get all of the prerequisite bugfixes in place.

@RyanGlScott
Copy link
Contributor Author

It took me longer than I'd hoped, but I've finally managed to add a regression test for #174—see di-arg-list.at-least-llvm13.ll. Thankfully, the existing callbr.ll test case already works as a regression test for #184.

In order to make all of the metadata in di-arg-list.at-least-llvm13.ll survive a roundtrip through llvm-pretty's pretty-printer, I had to fix quite a few unrelated bugs as preparation. As a result, this PR is a bit more bloated than I'd like, but oh well. I've made an effort to put each change into its own commit, so that is the best way to review this patch.


@kquick, I'm experiencing an issue with the tasty-sugar setup we're using, and I'm wondering if you have any ideas on how to resolve it. As we already have an at-least-llvm12 option in validParams, I went ahead and added an at-least-llvm13 option in this pull request as well. This has an unfortunate side effect when running disasm-test with LLVM 13, however: it doubles the number of tests! Compare when happens when I run the test suite with LLVM 12:

$ PATH=~/Software/clang+llvm-12.0.1/bin:$PATH cabal run disasm-test -fregression -ffuzz -- -l
Disassembly tests.llvm-as 12.0.1.anon-forward-ref.ll.at-least-llvm12.anon-forward-ref
Disassembly tests.llvm-as 12.0.1.atomicrmw.ll.at-least-llvm12.atomicrmw
Disassembly tests.llvm-as 12.0.1.callbr.ll.at-least-llvm12.callbr
Disassembly tests.llvm-as 12.0.1.factorial2.ll.at-least-llvm12.factorial2
Disassembly tests.llvm-as 12.0.1.fcmp-fast-math.ll.at-least-llvm12.fcmp-fast-math
Disassembly tests.llvm-as 12.0.1.fn-metadata.ll.at-least-llvm12.fn-metadata
Disassembly tests.llvm-as 12.0.1.fneg.ll.at-least-llvm12.fneg
Disassembly tests.llvm-as 12.0.1.fun-attrs.ll.at-least-llvm12.fun-attrs
Disassembly tests.llvm-as 12.0.1.global-array.ll.at-least-llvm12.global-array
Disassembly tests.llvm-as 12.0.1.half-float.ll.at-least-llvm12.half-float
Disassembly tests.llvm-as 12.0.1.hello-world.ll.at-least-llvm12.hello-world
Disassembly tests.llvm-as 12.0.1.insertelt.ll.at-least-llvm12.insertelt
Disassembly tests.llvm-as 12.0.1.mutual-struct-rec.ll.at-least-llvm12.mutual-struct-rec
Disassembly tests.llvm-as 12.0.1.phi-anon.ll.at-least-llvm12.phi-anon
Disassembly tests.llvm-as 12.0.1.poison.at-least-llvm12.ll.at-least-llvm12.poison
Disassembly tests.llvm-as 12.0.1.simple-metadata.ll.at-least-llvm12.simple-metadata
Disassembly tests.llvm-as 12.0.1.simple-res.ll.at-least-llvm12.simple-res
Disassembly tests.llvm-as 12.0.1.struct-initializer.ll.at-least-llvm12.struct-initializer
Disassembly tests.llvm-as 12.0.1.switch-big-value.ll.at-least-llvm12.switch-big-value
Disassembly tests.llvm-as 12.0.1.switch-same-target.ll.at-least-llvm12.switch-same-target
Disassembly tests.llvm-as 12.0.1.switch-simple.ll.at-least-llvm12.switch-simple
Disassembly tests.llvm-as 12.0.1.switch.ll.at-least-llvm12.switch

Versus LLVM 13:

$ PATH=~/Software/clang+llvm-13.0.0/bin:$PATH cabal run disasm-test -fregression -ffuzz -- -l
Disassembly tests.llvm-as 13.0.0.anon-forward-ref.ll.at-least-llvm12.anon-forward-ref
Disassembly tests.llvm-as 13.0.0.anon-forward-ref.ll.at-least-llvm13.anon-forward-ref
Disassembly tests.llvm-as 13.0.0.atomicrmw.ll.at-least-llvm12.atomicrmw
Disassembly tests.llvm-as 13.0.0.atomicrmw.ll.at-least-llvm13.atomicrmw
Disassembly tests.llvm-as 13.0.0.callbr.ll.at-least-llvm12.callbr
Disassembly tests.llvm-as 13.0.0.callbr.ll.at-least-llvm13.callbr
Disassembly tests.llvm-as 13.0.0.di-arg-list.at-least-llvm13.ll.at-least-llvm13.di-arg-list
Disassembly tests.llvm-as 13.0.0.factorial2.ll.at-least-llvm12.factorial2
Disassembly tests.llvm-as 13.0.0.factorial2.ll.at-least-llvm13.factorial2
Disassembly tests.llvm-as 13.0.0.fcmp-fast-math.ll.at-least-llvm12.fcmp-fast-math
Disassembly tests.llvm-as 13.0.0.fcmp-fast-math.ll.at-least-llvm13.fcmp-fast-math
Disassembly tests.llvm-as 13.0.0.fn-metadata.ll.at-least-llvm12.fn-metadata
Disassembly tests.llvm-as 13.0.0.fn-metadata.ll.at-least-llvm13.fn-metadata
Disassembly tests.llvm-as 13.0.0.fneg.ll.at-least-llvm12.fneg
Disassembly tests.llvm-as 13.0.0.fneg.ll.at-least-llvm13.fneg
Disassembly tests.llvm-as 13.0.0.fun-attrs.ll.at-least-llvm12.fun-attrs
Disassembly tests.llvm-as 13.0.0.fun-attrs.ll.at-least-llvm13.fun-attrs
Disassembly tests.llvm-as 13.0.0.global-array.ll.at-least-llvm12.global-array
Disassembly tests.llvm-as 13.0.0.global-array.ll.at-least-llvm13.global-array
Disassembly tests.llvm-as 13.0.0.half-float.ll.at-least-llvm12.half-float
Disassembly tests.llvm-as 13.0.0.half-float.ll.at-least-llvm13.half-float
Disassembly tests.llvm-as 13.0.0.hello-world.ll.at-least-llvm12.hello-world
Disassembly tests.llvm-as 13.0.0.hello-world.ll.at-least-llvm13.hello-world
Disassembly tests.llvm-as 13.0.0.insertelt.ll.at-least-llvm12.insertelt
Disassembly tests.llvm-as 13.0.0.insertelt.ll.at-least-llvm13.insertelt
Disassembly tests.llvm-as 13.0.0.mutual-struct-rec.ll.at-least-llvm12.mutual-struct-rec
Disassembly tests.llvm-as 13.0.0.mutual-struct-rec.ll.at-least-llvm13.mutual-struct-rec
Disassembly tests.llvm-as 13.0.0.phi-anon.ll.at-least-llvm12.phi-anon
Disassembly tests.llvm-as 13.0.0.phi-anon.ll.at-least-llvm13.phi-anon
Disassembly tests.llvm-as 13.0.0.poison.at-least-llvm12.ll.at-least-llvm12.poison
Disassembly tests.llvm-as 13.0.0.simple-metadata.ll.at-least-llvm12.simple-metadata
Disassembly tests.llvm-as 13.0.0.simple-metadata.ll.at-least-llvm13.simple-metadata
Disassembly tests.llvm-as 13.0.0.simple-res.ll.at-least-llvm12.simple-res
Disassembly tests.llvm-as 13.0.0.simple-res.ll.at-least-llvm13.simple-res
Disassembly tests.llvm-as 13.0.0.struct-initializer.ll.at-least-llvm12.struct-initializer
Disassembly tests.llvm-as 13.0.0.struct-initializer.ll.at-least-llvm13.struct-initializer
Disassembly tests.llvm-as 13.0.0.switch-big-value.ll.at-least-llvm12.switch-big-value
Disassembly tests.llvm-as 13.0.0.switch-big-value.ll.at-least-llvm13.switch-big-value
Disassembly tests.llvm-as 13.0.0.switch-same-target.ll.at-least-llvm12.switch-same-target
Disassembly tests.llvm-as 13.0.0.switch-same-target.ll.at-least-llvm13.switch-same-target
Disassembly tests.llvm-as 13.0.0.switch-simple.ll.at-least-llvm12.switch-simple
Disassembly tests.llvm-as 13.0.0.switch-simple.ll.at-least-llvm13.switch-simple
Disassembly tests.llvm-as 13.0.0.switch.ll.at-least-llvm12.switch
Disassembly tests.llvm-as 13.0.0.switch.ll.at-least-llvm13.switch

In the latter, we run at-least-llvm12 and at-least-llvm13 configurations for every test case. Ideally, we'd only run the at-least-llvm13 ones, however. Is there a way to configure tasty-sugar to make this possible?

This brings in the changes from GaloisInc/llvm-pretty#103 and
GaloisInc/llvm-pretty#108. The former requires adding an `Arbitrary` instance
for `DIArgList` in `unit-test`.
This adapts to changes on the `llvm-pretty` side in GaloisInc/llvm-pretty#108.
This corrects an inadvertent mistake introduced in #137 where the order of
cases was reversed.

Fixes #211.
This upholds a LLVM invariant that they are always printed inline, never in the
global list of metadata.

Fixes #212.
@kquick
Copy link
Member

kquick commented Mar 31, 2023

In the latter, we run at-least-llvm12 and at-least-llvm13 configurations for every test case. Ideally, we'd only run the at-least-llvm13 ones, however. Is there a way to configure tasty-sugar to make this possible?

This isn't a tasty-sugar issue per-se, but the Explicit v.s. Assumed handling in disasm-test (actually, the issue is mostly the latter).

For background, if you have tasty-sugar parameters, those parameters can be present in a filename (e.g. blah-at-least-llvm12.ll or missing; in the latter case, if there is a matching base filename (e.g. blah.ll) then tasty-sugar will offer that as the expected file and indicate that at-least-llvm12 is assumed. It will generate one of these Assumed for every parameter value that is not explicitly matched but for which there is a base match.

Thus, we didn't duplicated every file: there was only the single match for poison.at-least-llvm12.ll because there is no poison.ll and the same goes for the new di-arg-list.at-least-llvm13.ll. We did get the Explicit match for those, and did the right thing.

So the main issue is in disasm-test/Main.hs where it handles the Assumed case (https://github.com/GaloisInc/llvm-pretty-bc-parser/blob/master/disasm-test/Main.hs#L307). We don't want it to return True for every assumed parameter... only for one of them. The solution here is to call specMatchesInstalled only for the lowest "at-least" value (because specMatchesInstalled will properly filter against the actual version number for an at-least-llvm* parameter). Thus, you should be able to replace that line 307 with something like:

    Just (TS.Assumed v) | v == "at-least-llvm12" -> specMatchesInstalled v
    Just (TS.Assumed _) -> False

@RyanGlScott
Copy link
Contributor Author

Thanks, that explanation makes sense. I had to tweak your original suggestion to make it worked when the assumed parameter is pre-llvm12:

diff --git a/disasm-test/Main.hs b/disasm-test/Main.hs
index faa1dce..0404046 100644
--- a/disasm-test/Main.hs
+++ b/disasm-test/Main.hs
@@ -307,7 +307,11 @@ runTest llvmVer sweet expct
                    ]
           in case lookup "llvm-range" (TS.expParamsMatch expct) of
                Just (TS.Explicit v) -> specMatchesInstalled v
-               Just (TS.Assumed  v) -> specMatchesInstalled v
+               Just (TS.Assumed v)
+                 |  v == "pre-llvm11" || v == "at-least-llvm12"
+                 -> specMatchesInstalled v
+                 |  otherwise
+                 -> False
                _ -> error "llvm-range unknown"
 
 -- | Assemble some llvm assembly, producing a bitcode file in /tmp.

But after doing that, everything works as I would expect on LLVM 11, 12, and 13. Nice! And if I understand correctly, this approach is future-proof in that I shouldn't need to modify the Assumed logic every time we add another at-least-llvmXX version.


I've pushed the fix above, which means that this PR is finally ready for review!

@RyanGlScott RyanGlScott marked this pull request as ready for review March 31, 2023 23:04
@kquick
Copy link
Member

kquick commented Mar 31, 2023

And if I understand correctly, this approach is future-proof

It should be!

Copy link
Member

@kquick kquick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's 13 done... just 14, 15, and 16 to go.

RyanGlScott and others added 4 commits April 3, 2023 07:24
This is necessary to support LLVM 13.

Fixes #174.
One code (`CST_CODE_INLINEASM_OLD3`) was introduced in LLVM 13, and another
(`CST_CODE_INLINEASM`) was introduced in LLVM 14. For the most part, they are
parsed identically to previous inline `asm` codes, but with some minor
differences. I have consolidated the logic for parsing all inline `asm` codes
into a single `parseInlineAsm` function.

The existing `disasm-test/tests/callbr.ll` test case ensures that we handle all
of these different `inline asm` codes correctly. As an added bonus, it's
portable across multiple LLVM versions!

Fixes #184.
This ensures that we do not run test cases with assumed `tasty-sugar`
parameters multiple times with later LLVM versions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support CST_CODE_INLINEASM constant codes added in LLVM 13 and 14 support METADATA_ARG_LIST
3 participants