Reduce differences in user code from Bazel

[alexeagle]

Discussed with @ndmitchell last week, that some Bazel users may want to try Buck2 without having to make serious changes to their project, for example use the same BUILD files. Even better, any serious Bazel user who wants to switch will need their repo to build&test with both tools during a migration window.

A good first experiment would be to take a small Bazel project and add minimal files to make it work with both.

[ndmitchell]

I took the 3 C++ sample projects from https://github.com/bazelbuild/examples/tree/main/cpp-tutorial. I could compile all of them by doing:

• buck2 init in the project (after patching buck2 init, in ways I have submitted a diff for).
• Add to the .buckconfig an alias rules_cc = rules_cc and a [buildfile] name = BUILD.
• Add symlinks to a prelude, a toolchains (with system toolchains) and a rules_cc (a shim I built over our C++ rules - maybe 10 minutes of hacking.

buck2 run then successfully built and ran all 3 examples. The rules_cc shim was:

def _tweak(args):
    if "visibility" in args:
        args["visibility"] = _tweak_visibility(args["visibility"])
    return args
def _tweak_visibility(visibility):
    # Bazel uses __pkg__ to mean anything in a package,
    # Buck2 has no such concept, so we just remap it to PUBLIC
    return ["PUBLIC" if x.endswith(":__pkg__") else x for x in visibility]
def cc_binary(**kwargs):
    native.cxx_binary(**_tweak(kwargs))
def cc_library(hdrs, **kwargs):
    native.cxx_library(exported_headers = hdrs, **_tweak(kwargs))

I also took the rules from https://github.com/bazelbuild/examples/tree/main/rules and gave them a go. I got a few of them working, but there are two major issues:

• We need a better way to shim things. We have good ways to shim in BUILD files, but less good ways to shim in .bzl files. Not hard to fix, lots of feasible designs, but we should do something. For now I renamed rule to bazel_rule in the Bazel examples (ditto about 2 other identifiers).
• In Buck2 our ctx.actions.run uses cmd_args which means there isn't currently a way to get the path from an artifact. I consider that a really good thing - we are able to write all our rules at a higher level, with more compositional pieces, and more safety - win win. But a lot of the Bazel examples use foo.path. We probably need to add that to make it compatible, ideally with a nice and scary name so people don't use it beyond Bazel compatibility.

But with those two caveats, I managed to get about 7 of the examples working with a shim:

def _struct_items(s):
    return [(k, getattr(s, k)) for k in dir(s)]

def bazel_rule(implementation, attrs = {}):
    def impl(ctx):
        actions = {
            "write": lambda output, content: ctx.actions.write(output.value if getattr(output, "type", None) == "File" else output, content)
        }
        ctx2 = {
            "attr": {},
            "outputs": {},
            "files": {},
            "executable": {},
        }
        for name, (_, field, func) in attrs.items():
            ctx2[field][name] = func(ctx, getattr(ctx.attrs, name))
        ctx2 = struct(
            actions = struct(**actions),
            **{k: struct(**v) for k, v in ctx2.items()},
        )
        res = implementation(ctx2)
        if res == None:
            res = []
        if any([type(x) == type(DefaultInfo()) for x in res]):
            return res
        else:
            return [DefaultInfo()] + res

    return rule(impl = impl, attrs = {k: v[0] for k, v in attrs.items()})

def _label_list(allow_files=False, providers = []):
    if allow_files:
        return (attrs.list(attrs.source(), default = []), "files", lambda _ctx, xs: [_File(x) for x in xs])
    else:
        return (attrs.list(attrs.dep(), default = []), "attr", lambda _ctx, xs: [_Target(x) for x in xs])

def _label(default=None, cfg=None, allow_files=False, executable=False, allow_single_file=None, mandatory=False):
    defaulted = {} if default == None else {"default": default}
    if executable:
        return (attrs.exec_dep(**defaulted), "executable", lambda _ctx, x: x)
    elif allow_files or allow_single_file:
        return (attrs.source(**defaulted), "files", lambda _ctx, x: _File(x))
    else:
        return (attrs.dep(**defaulted), "attr", lambda _ctx, x: x)

# For each attr we return the pair of the attribute, and the function
# that wraps it
attr = struct(
    string = lambda: (attrs.string(), "attr", lambda _ctx, x: x),
    int = lambda default: (attrs.int(default = default), "attr", lambda _ctx, x: x),
    output = lambda: (attrs.string(), "outputs", lambda ctx, x: _File(ctx.actions.declare_output(x))),
    label_list = _label_list,
    label = _label,
)


def _Depset(items):
    return struct(type = "Depset", to_list = lambda: items)

def _File(x):
    return struct(type = "File", value = x, path = x.short_path)

def _Target(x):
    return x

def depset(x, transitive):
    res = dedupe(x + [t for ts in transitive for t in ts.to_list()])
    return _Depset(res)

def bazel_provider(arg):
    return provider(fields = {arg: typing.Any})

def BazelDefaultInfo(files):
    outputs = files.to_list() if getattr(files, "type", None) == "Depset" else files
    return DefaultInfo(default_outputs = [x.value for x in outputs])