Compiling buck2 with buck2's remote execution

[allada]

I am the author of TurboCache and I was checking the compatibility of buck2 with TurboCache. Usually for these kinds of projects my "does it work" test is to try and compile the project itself with my tools. In this case it mostly worked, but was getting some rustc specific errors. It appears the errors are around buck2 possibly not setting the assets in the remote execution digests.

After looking into it a bit, I kept thinking it was a problem with maybe not setting up the environment properly, so I decided to do the same test with BuildBarn, but it ended up giving the same errors.

For reference it does compile properly when not running remote. rustc version (on executing host: rustc 1.71.0-nightly (cca7ee581 2023-05-27). I tried both inside docker and outside docker, same results.

Below are some of the errors returned by rustc. My question is, "is buck2 compilable under buck2 with remote execution?"

buck-out/v2/gen/shim/524f8da68ea2a374/third-party/rust/__zstd-sys-2.0.1+zstd.1.5.2.crate__/zstd-sys-2.0.1+zstd.1.5.2.crate/zstd/lib/decompress/zstd_decompress.c:64:10: fatal error: '../common/xxhash.h' file not found
#include "../common/xxhash.h" /* XXH64_reset, XXH64_update, XXH64_digest, XXH64 */
         ^~~~~~~~~~~~~~~~~~~~
1 error generated.
Action failed: shim//third-party/rust:zstd-sys-2.0.1+zstd.1.5.2-zstd (cxx_compile zstd-sys-2.0.1+zstd.1.5.2.crate/zstd/lib/common/pool.c (pic))
Remote command returned non-zero exit code 1

...

error: proc-macro derive panicked
    --> ./buck-out/v2/gen/root/524f8da68ea2a374/app/buck2_data/__buck2_data-proto__/out/buck.data.rs:3779:30
     |
3779 | #[derive(::serde::Serialize, ::serde::Deserialize)]
     |                              ^^^^^^^^^^^^^^^^^^^^
     |
     = help: message: file missing from serde_derive manifest directory during macro expansion: /worker/build/d3e1881e9b2f57c5/root/buck-out/v2/gen/shim/524f8da68ea2a374/third-party/rust/__serde_derive-1.0.183.crate__/serde_derive-1.0.183.crate/serde_derive-x86_64-unknown-linux-gnu

...

error[E0599]: the method `serialize` exists for tuple `(&String, &u32, &u32, &u32, &u32, &HostInternal, &Option<u16>, &u32, &Option<u32>, &Option<u32>)`, but its trait bounds were not satisfied
    --> shim/third-party/rust/url-2.4.0.crate/src/lib.rs:2508:14
     |
2496 | /         (
2497 | |             serialization,
2498 | |             scheme_end,
2499 | |             username_end,
...    |
2507 | |         )
2508 | |             .serialize(serializer)
     | |             -^^^^^^^^^ method cannot be called due to unsatisfied trait bounds
     | |_____________|
     | 
     |
     = note: the full type name has been written to 'buck-out/v2/gen/shim/524f8da68ea2a374/third-party/rust/__url-2.4.0__/rlib-pic-static_pic-link/extras/liburl/url-0c29c132.long-type-14116539382350290467.txt'
     = note: the following trait bounds were not satisfied:
             `&HostInternal: Serialize`
             which is required by `(&std::string::String, &u32, &u32, &u32, &u32, &HostInternal, &std::option::Option<u16>, &u32, &std::option::Option<u32>, &std::option::Option<u32>): Serialize`

...

error[E0599]: no variant or associated item named `deserialize` found for enum `ActionKind` in the current scope
    --> app/buck2_data/src/lib.rs:89:36
     |
89   |         let d = crate::ActionKind::deserialize(deserializer)?;
     |                                    ^^^^^^^^^^^ variant or associated item not found in `ActionKind`
     |
    ::: ./buck-out/v2/gen/root/524f8da68ea2a374/app/buck2_data/__buck2_data-proto__/out/buck.data.rs:3497:1
     |
3497 | pub enum ActionKind {
     | ------------------- variant or associated item `deserialize` not found for this enum
     |
     = help: items from traits can only be used if the trait is implemented and in scope
     = note: the following traits define an item `deserialize`, perhaps you need to implement one of them:
             candidate #1: `Deserialize`
             candidate #2: `DeserializeSeed`

[dtolnay]

For zstd/lib/decompress/../common/xxhash.h: I do see "zstd/lib/common/xxhash.h" here but it's commented out. Hard to tell why. It's worth trying with that uncommented and see what happens.

For serde: this shouldn't happen with a newer serde version. cargo update -Z bindeps in the shim/third-party/rust directory before running reindeer buckify.

[allada]

I think my larger concern is why it is different from building locally vs remote? The setup on the workers vs local machine are near identical, so there shouldn't be any code version differences.

[cjhopman]

Local isn't sandboxed and so it may be succeeding by accessing things that it shouldn't be. There's more discussion about supporting sandboxing locally here: #358

[ndmitchell]

@allada - if the two fixes @dtolnay proposes works, it would be great to know and we can apply them. As @cjhopman says, this probably fails remotely because we haven't declared all the dependencies. Internally we use remote as a solid check that we haven't accidentally done that, but both the two areas you mention are things we do differently internally, so don't have that safety net with.