Use upstream GoogleTest and add related test utils. (carbon-language#876

)

This moves over to the vanilla upstream GoogleTest pulled in the more
expected manner with Bazel. It also adds Abseil and Google Benchmark
libraries in the same fashion (there are cross dependencies here).

As part of this, also introduce a dependency check test that can enforce
basic layering of dependencies. For example, this lets us ensure that
non-test Carbon code only depends on LLVM and Clang despite having other
libraries available. There remains some cleanup to improve the way these
dependency tests work, but this at least ensures we don't regress.

I've also provided workarounds to allow both Carbon code and LLVM code
to freely be used with GoogleTest (and other `std::ostream` based
output code). This is done by extending the code in
`//common/ostream.h`. One downside is that it requires opening the
`llvm` namespace and adding an ADL_found overload there. I think on
balance this is still a win and doesn't make me too nervous.

The new version of GoogleTest requires printing more often from matchers
and so I've also added several printing routines to types that
previously didn't require them. Otherwise, most of the updates are just
using the more conventional upstream style of including the headers and
adding `ostream.h` where it is needed.

I did consider moving code over to use `std::ostream` instead of LLVM's
`raw_ostream`, but the advantages of not doing virtual dispatch still
seem significant, and it also seems good to retain access to LLVM's
formatting utilities built around `raw_ostream` given that we can't pull
arbitrary dependencies into Carbon code outside of test code.

All of this was slightly motivated by requests for newer features in
GoogleTest, but much more-so by my desire to have access to Google
Benchmark and Abseil when writing benchmarks. For example, using
Abseil's random number generator seems extremely helpful when generating
inputs for benchmarks. The growing dependencies between these packages
further motivated me to just pull them all in and ensure they worked
well.

WORKSPACE

@@ -89,6 +89,45 @@ load(
 
 configure_clang_toolchain(name = "bazel_cc_toolchain")
 
+###############################################################################
+# Abseil libraries
+###############################################################################
+
+abseil_version = "4a995b1eaa4a602f0d3a9ff8eac89d4649cd2fe8"
+
+http_archive(
+ name = "com_google_absl",
+ sha256 = "f5021900ff9a6b8f39406d15f460714660ab6b3e727754663d786d75ecad5ee0",
+ strip_prefix = "abseil-cpp-%s" % abseil_version,
+ urls = ["https://github.com/abseil/abseil-cpp/archive/%s.zip" % abseil_version],
+)
+
+###############################################################################
+# GoogleTest libraries
+###############################################################################
+
+googletest_version = "075810f7a20405ea09a93f68847d6e963212fa62"
+
+http_archive(
+ name = "com_google_googletest",
+ sha256 = "19949c33e795197dbb8610672c18bff447dc31faef3257665d69d1bf0884d67b",
+ strip_prefix = "googletest-%s" % googletest_version,
+ urls = ["https://github.com/google/googletest/archive/%s.zip" % googletest_version],
+)
+
+###############################################################################
+# Google Benchmark libraries
+###############################################################################
+
+benchmark_version = "0baacde3618ca617da95375e0af13ce1baadea47"
+
+http_archive(
+ name = "com_github_google_benchmark",
+ sha256 = "19949c33e795197dbb8610672c18bff447dc31faef3257665d69d1bf0884d67b",
+ strip_prefix = "benchmark-%s" % benchmark_version,
+ urls = ["https://github.com/google/benchmark/archive/%s.zip" % benchmark_version],
+)
+
 ###############################################################################
 # LLVM libraries
 ###############################################################################

bazel/check_deps/BUILD

@@ -0,0 +1,87 @@
+# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+# Exceptions. See /LICENSE for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+load("@mypy_integration//:mypy.bzl", "mypy_test")
+
+# This filegroup should contain all the non-test C++ rules in the repository to
+# enable dependency checking. It can be regenerated by running the following
+# query command:
+#
+# ```
+# bazelisk query 'kind("cc.* rule", attr(testonly, 0, //...))'
+# ```
+filegroup(
+ name = "non_test_cc_rules",
+ data = [
+ "//common:check",
+ "//common:indirect_value",
+ "//common:ostream",
+ "//common:string_helpers",
+ "//executable_semantics",
+ "//executable_semantics/ast",
+ "//executable_semantics/ast:class_definition",
+ "//executable_semantics/ast:declaration",
+ "//executable_semantics/ast:expression",
+ "//executable_semantics/ast:library_name",
+ "//executable_semantics/ast:member",
+ "//executable_semantics/ast:paren_contents",
+ "//executable_semantics/ast:pattern",
+ "//executable_semantics/ast:source_location",
+ "//executable_semantics/ast:statement",
+ "//executable_semantics/common:arena",
+ "//executable_semantics/common:error",
+ "//executable_semantics/common:nonnull",
+ "//executable_semantics/interpreter",
+ "//executable_semantics/interpreter:address",
+ "//executable_semantics/interpreter:dictionary",
+ "//executable_semantics/interpreter:exec_program",
+ "//executable_semantics/interpreter:field_path",
+ "//executable_semantics/interpreter:heap",
+ "//executable_semantics/interpreter:stack",
+ "//executable_semantics/interpreter:type_checker",
+ "//executable_semantics/syntax",
+ "//executable_semantics/syntax:bison_wrap",
+ "//migrate_cpp/cpp_refactoring",
+ "//migrate_cpp/cpp_refactoring:fn_inserter",
+ "//migrate_cpp/cpp_refactoring:for_range",
+ "//migrate_cpp/cpp_refactoring:matcher",
+ "//migrate_cpp/cpp_refactoring:var_decl",
+ "//toolchain/diagnostics:diagnostic_emitter",
+ "//toolchain/diagnostics:null_diagnostics",
+ "//toolchain/driver",
+ "//toolchain/driver:carbon",
+ "//toolchain/lexer:character_set",
+ "//toolchain/lexer:numeric_literal",
+ "//toolchain/lexer:string_literal",
+ "//toolchain/lexer:token_kind",
+ "//toolchain/lexer:tokenized_buffer",
+ "//toolchain/parser:parse_node_kind",
+ "//toolchain/parser:parse_tree",
+ "//toolchain/parser:precedence",
+ "//toolchain/source:source_buffer",
+ ],
+)
+
+genquery(
+ name = "non_test_cc_deps.txt",
+ expression = "kind('cc.* rule', deps(//bazel/check_deps:non_test_cc_rules))",
+ opts = [
+ "--notool_deps",
+ "--noimplicit_deps",
+ ],
+ scope = [":non_test_cc_rules"],
+)
+
+py_test(
+ name = "check_non_test_cc_deps",
+ srcs = ["check_non_test_cc_deps.py"],
+ data = [":non_test_cc_deps.txt"],
+ main = "check_non_test_cc_deps.py",
+)
+
+mypy_test(
+ name = "check_non_test_cc_deps_mypy_test",
+ include_imports = True,
+ deps = [":check_non_test_cc_deps"],
+)

bazel/check_deps/check_non_test_cc_deps.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+
+"""Check that non-test C++ rules only depend on Carbon and LLVM.
+
+Carbon works to ensure its user-visible libraries and binaries only depend on
+their code and LLVM. Among other benefits, this provides a single, simple
+license used for the whole project.
+
+However, we frequently use third-party projects and libraries where useful in
+our test code. Here, we verify that the dependencies of non-test C++ rules only
+include Carbon and LLVM code.
+"""
+
+__copyright__ = """
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+"""
+
+import os
+import sys
+from pathlib import Path
+
+runfiles = Path(os.environ["TEST_SRCDIR"])
+deps_path = (
+ runfiles / "carbon" / "bazel" / "check_deps" / "non_test_cc_deps.txt"
+)
+try:
+ with deps_path.open() as deps_file:
+ deps = deps_file.read().splitlines()
+except FileNotFoundError:
+ sys.exit("ERROR: unable to find deps file: %s" % deps_path)
+
+for dep in deps:
+ print("Checking dependency: " + dep)
+ repo, _, rule = dep.partition("//")
+ if repo == "" and not rule.startswith("third_party"):
+ # Carbon code is always allowed.
+ continue
+ if repo == "@llvm-project":
+ package, _, rule = rule.partition(":")
+
+ # Other packages in the LLVM project shouldn't be accidentally used
+ # in Carbon. We can expand the above list if use cases emerge.
+ if package not in ("llvm", "lld", "clang"):
+ sys.exit(
+ "ERROR: unexpected dependency into the LLVM project: %s" % dep
+ )
+
+ # Check for accidentally using the copy of GoogleTest in LLVM.
+ if rule in ("gmock", "gtest", "gtest_main"):
+ sys.exit(
+ "ERROR: dependency on LLVM's GoogleTest from non-test code: %s"
+ % dep
+ )
+
+ # The rest of LLVM, LLD, and Clang themselves are safe to depend on.
+ continue
+ if repo in ("@llvm_terminfo", "@llvm_zlib"):
+ # These are stubs wrapping system libraries for LLVM. They aren't
+ # distributed and so should be fine.
+ continue
+ if repo in (
+ "@com_google_absl",
+ "@com_google_googletest",
+ "@com_github_google_benchmark",
+ ):
+ # This should never be reached from non-test code, but these targets do
+ # exist. Specially diagnose them to try to provide a more helpful
+ # message.
+ sys.exit("ERROR: dependency only allowed in test code: %s" % dep)
+
+ # Conservatively fail if a dependency isn't explicitly allowed above.
+ sys.exit("ERROR: unknown dependency: %s" % dep)

common/BUILD

@@ -18,8 +18,7 @@ cc_test(
  srcs = ["check_test.cpp"],
  deps = [
  ":check",
- "@llvm-project//llvm:gtest",
- "@llvm-project//llvm:gtest_main",
+ "@com_google_googletest//:gtest_main",
  ],
 )
 
@@ -33,8 +32,7 @@ cc_test(
  srcs = ["indirect_value_test.cpp"],
  deps = [
  ":indirect_value",
- "@llvm-project//llvm:gtest",
- "@llvm-project//llvm:gtest_main",
+ "@com_google_googletest//:gtest_main",
  ],
 )
 
@@ -61,7 +59,6 @@ cc_test(
  srcs = ["string_helpers_test.cpp"],
  deps = [
  ":string_helpers",
- "@llvm-project//llvm:gtest",
- "@llvm-project//llvm:gtest_main",
+ "@com_google_googletest//:gtest_main",
  ],
 )

common/check_test.cpp

@@ -4,7 +4,7 @@
 
 #include "common/check.h"
 
-#include "gtest/gtest.h"
+#include <gtest/gtest.h>
 
 namespace Carbon {
 

common/indirect_value_test.cpp

@@ -4,9 +4,9 @@
 
 #include "common/indirect_value.h"
 
-#include <string>
+#include <gtest/gtest.h>
 
-#include "gtest/gtest.h"
+#include <string>
 
 namespace Carbon {
 namespace {

common/ostream.h

@@ -5,11 +5,14 @@
 #ifndef COMMON_OSTREAM_H_
 #define COMMON_OSTREAM_H_
 
+#include <ostream>
+
+#include "llvm/Support/raw_os_ostream.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace Carbon {
 
-// Support ostream << for types which implement:
+// Support raw_ostream << for types which implement:
 // void Print(llvm::raw_ostream& out) const;
 template <typename T, typename std::enable_if<std::is_member_function_pointer<
  decltype(&T::Print)>::value>::type* = nullptr>
@@ -18,14 +21,75 @@ auto operator<<(llvm::raw_ostream& out, const T& obj) -> llvm::raw_ostream& {
  return out;
 }
 
-// Prevents ostream << for pointers to printable types.
+// Prevents raw_ostream << for pointers to printable types.
 template <typename T, typename std::enable_if<std::is_member_function_pointer<
  decltype(&T::Print)>::value>::type* = nullptr>
 __attribute__((unavailable(
  "Received a pointer to a printable type, are you missing a `*`? "
  "To print as a pointer, cast to void*."))) auto
 operator<<(llvm::raw_ostream& out, const T* /*obj*/) -> llvm::raw_ostream&;
 
+// Support std::ostream << for types which implement:
+// void Print(llvm::raw_ostream& out) const;
+template <typename T, typename std::enable_if<std::is_member_function_pointer<
+ decltype(&T::Print)>::value>::type* = nullptr>
+auto operator<<(std::ostream& out, const T& obj) -> std::ostream& {
+ llvm::raw_os_ostream raw_os(out);
+ obj.Print(raw_os);
+ return out;
+}
+
+// Prevents std::ostream << for pointers to printable types.
+template <typename T, typename std::enable_if<std::is_member_function_pointer<
+ decltype(&T::Print)>::value>::type* = nullptr>
+__attribute__((unavailable(
+ "Received a pointer to a printable type, are you missing a `*`? "
+ "To print as a pointer, cast to void*."))) auto
+operator<<(std::ostream& out, const T* /*obj*/) -> std::ostream&;
+
+// Allow GoogleTest and GoogleMock to print even pointers by dereferencing them.
+// This is important to allow automatic printing of arguments of mocked APIs.
+template <typename T, typename std::enable_if<std::is_member_function_pointer<
+ decltype(&T::Print)>::value>::type* = nullptr>
+void PrintTo(const T* p, std::ostream* out) {
+ *out << static_cast<const void*>(p);
+
+ // Also print the object if non-null.
+ if (p) {
+ *out << " pointing to " << *p;
+ }
+}
+
 } // namespace Carbon
 
+namespace llvm {
+
+// Injects an `operator<<` overload into the `llvm` namespace which detects LLVM
+// types with `raw_ostream` overloads and uses that to map to a `std::ostream`
+// overload. This allows LLVM types to be printed to `std::ostream` via their
+// `raw_ostream` operator overloads, which is needed both for logging and
+// testing.
+//
+// To make this overload be unusually low priority, it is designed to take even
+// the `std::ostream` parameter as a template, and SFINAE disable itself unless
+// that template parameter matches `std::ostream`. This ensures that an
+// *explicit* operator will be preferred when provided. Some LLVM types may have
+// this, and so we want to prioritize accordingly.
+//
+// It would be slightly cleaner for LLVM itself to provide this overload in
+// `raw_os_ostream.h` so that we wouldn't need to inject into its namespace, but
+// supporting `std::ostream` isn't a priority for LLVM so we handle it locally
+// instead.
+template <typename S, typename T,
+ typename = std::enable_if_t<std::is_base_of_v<
+ std::ostream, std::remove_reference_t<std::remove_cv_t<S>>>>,
+ typename = std::enable_if_t<!std::is_same_v<
+ std::remove_reference_t<std::remove_cv_t<T>>, raw_ostream>>>
+auto operator<<(S& standard_out, const T& value) -> S& {
+ raw_os_ostream(standard_out) << value;
+ return standard_out;
+}
+
+} // namespace llvm
+
 #endif // COMMON_OSTREAM_H_

common/string_helpers_test.cpp

@@ -4,10 +4,10 @@
 
 #include "common/string_helpers.h"
 
-#include <string>
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
 
-#include "gmock/gmock.h"
-#include "gtest/gtest.h"
+#include <string>
 
 using ::testing::Eq;
 using ::testing::Optional;