CI (Buildkite): Add the package_linux64 and doctest builders

[DilumAluthge]

TODO:

• Don't start the tester_linux64 builder until the package_linux64 builder has completed successfully.
• At the end of the package_linux64 builder, upload the build artifacts.
• At the beginning of the tester_linux64 builder, download the build artifacts from package_linux64 (instead of building Julia from source).
• Get all doctests to pass in the doctest builder.

[DilumAluthge]

Test Summary:                       |     Pass  Fail  Error  Broken     Total
  Overall                           | 38938142    12      6  352644  39290804

@staticfloat Take a look at the test failures. It looks like most are due to issues with the sandboxing?

[staticfloat]

Wow, this is a lot of work Dilum! Awesome to have you working on these pieces. A few things to note:

1. We are going to want to coordinate GCC versions across platforms, so that we can, in general, say something like "Julia is built with GCC 9" and have that be true across all platforms. To that end, we are going to want to more tightly control how the package_linux64 image is built. I don't think we should re-use the llvm-passes image, but should instead construct our own, and probably add a PPA/repository that allows us to add a specific version of GCC directly.

2. I've never tried running rr inside of sandbox, inside of sandbox; if that "just works" I will be ecstatic.

3. path.jl test failure is due to $HOME not existing within the sandbox; we just need to set an appropriate environment mapping here.

4. file.jl test failure is due to Julia mistakenly thinking we're root during the build process (because either $USER or $HOME looks like it) and trying to do a chown() that gets denied unless you're root. We should not run as root, so we should change $USER and $HOME to be something less root-like.

5. file.jl warning is also due to the root detection going awry.

A quick glance over the rest of the errors all look like they stem from $HOME not existing. So let's fix these issues (by defining HOME in the yaml to point to some directory that exists within the sandbox, for one) and see if that improves things significantly.

[DilumAluthge]

2. I've never tried running rr inside of sandbox, inside of sandbox; if that "just works" I will be ecstatic.

I wouldn't get your hopes up 😂.

I'm thinking that for this PR, we focus on getting everything working without rr. And then in a future PR, we can add rr support.

[DilumAluthge]

Okay, looks like we are making progress. Originally, we had 12 failures and 6 errors, and now we are down to 10 failures and 0 errors.

[staticfloat]

@Keno We're running Julia tests inside of sandbox here, and chown() is giving EINVAL instead of EPERM when we pass in invalid UIDs. This is presumably because /proc/self/uid_map is 1000 1000 1, meaning there is only one valid UID in the system. What do you propose we do? Should we try to mess with /etc/subuid, or should we detect this and disable the tests within the sandbox?

[Keno]

Seems inconsistent, but that's Linux for you. I'd just allow EINVAL too in that test.

[DilumAluthge]

I was thinking that it would be nice to keep this PR as Buildkite-changes-only. What do y'all think of making a separate PR to modify the tests, and then rebasing this PR once the test-modifying-PR is merged?

[DilumAluthge]

I was thinking that it would be nice to keep this PR as Buildkite-changes-only. What do y'all think of making a separate PR to modify the tests, and then rebasing this PR once the test-modifying-PR is merged?

#41682 (I also made a few changes in that PR to account for the fact that Base._UVError is not actually a type, but a function that constructs and returns a Base.IOError with a certain format.)

[DilumAluthge]

Any idea why these cmdlineargs tests are failing?

[DilumAluthge]

These cmdlineargs tests are really confusing:

From worker 7: ┌ Info:
From worker 7: └ s = "ERROR: could not load library "/cache/build/amdci5-2/julialang/julia-master/julia-artifact/share/julia/test/nonexistent"\n/cache/build/amdci5-2/julialang/julia-master/julia-artifact/share/julia/test/nonexistent.so: cannot open shared object file: No such file or directory\n"
From worker 7: ┌ Info:
From worker 7: └ p = Process(/cache/build/amdci5-2/julialang/julia-master/julia-artifact/bin/julia -Cnative -J/cache/build/amdci5-2/julialang/julia-master/julia-artifact/lib/julia/sys.so --depwarn=error --check-bounds=yes -g1 --startup-file=no --sysimage=/cache/build/amdci5-2/julialang/julia-master/julia-artifact/share/julia/test/nonexistent, ProcessSignaled(11))
From worker 7: ┌ Info:
From worker 7: └ p.exitcode = 0

It's kind of strange that the process p exited with exit code zero, even though the stderr output (the string s) shows that it printed a fatal error.

[DilumAluthge]

Here's an update: the tester_linux64 builder is failing with 7 failures total (6 fails and 1 error). All other jobs are passing.

My plan is:

1. Disable the tester_linux64 builder.
2. Merge this PR.
3. Make a new PR to enable the tester_linux64 builder.

[DilumAluthge]

I took all the TODOs that we did not complete in this PR, and I moved them to #41509.