Skip to content

Latest commit

 

History

History
139 lines (70 loc) · 30.2 KB

using-swift-from-c++.md

File metadata and controls

139 lines (70 loc) · 30.2 KB

Using Swift from C++

This vision document presents a high level overview of the "reverse" (i.e. Swift-to-C++) half of the C++ interoperability Swift language feature. It highlights the key principles and goals that determine how Swift APIs are exposed to C++ users. It also outlines the evolution process for this feature. This document does not present the final design for how Swift APIs get mapped to C++ language constructs. The final design of this feature will evolve as this feature goes through the Swift evolution process. This document does not cover the "forward" (i.e. using C++ APIs from Swift) aspect of the C++ interoperability, as it’s covered by a sibling document.

This document is an official feature vision document, as described in the draft review management guidelines of the Swift evolution process. The Language Workgroup has endorsed the goals and basic approach laid out in this document. This endorsement is not a pre-approval of any of the concrete proposals that may come out of this document. All proposals will undergo normal evolution review, which may result in rejection or revision from how they appear in this document.

Introduction

Swift currently provides support for interoperability with C and Objective-C. Swift can import C and Objective-C APIs, and it can also expose its @objc types and functions to Objective-C. However, Swift currently does not provide support for interoperability with C++. Supporting bidirectional C++ interoperability in Swift would allow it to call into C++ APIs, and would make Swift APIs accessible to C++. Making Swift APIs accessible to C++ would simplify the process of adoption of Swift in existing C++ codebases, as it would allow them to incrementally adopt Swift. Furthermore, it would make Swift-only libraries available to C++ codebases.

Overview of Swift's interoperability support

Swift uses two very different approaches for interoperability with C and Objective-C. For "forward" interoperability, Swift embeds a copy of the Clang compiler, which it uses to directly load C and Objective-C Clang modules and translate declarations into a native Swift representation. For "reverse" interoperability, Swift does not want to require Clang to embed Swift, and so Swift generates an Objective-C header that can be used to call into Swift APIs that are exposed in that header. Because Objective-C is restricted in what kinds of types it can work with, and because it requires code to be emitted in a special way that doesn't match the regular Swift ABI, methods and types must be marked as being exposed to Objective-C with the @objc attribute.

Swift's support for C++ interoperability is modeled after its support for C and Objective-C interoperability. For "forward" interoperability, the embedded Clang compiler is used to directly load C++ Clang modules and translate declarations into a native Swift representation. The "forward" interoperability vision document provides more details about this model. For "reverse" interoperability, Swift generates a header that uses C++ language constructs to represent Swift APIs that are exposed by the Swift module. Because of the relatively high expressivity of C++ headers, the generated header is able to provide representation for most native Swift functions, methods, initializers, accessors and types without needing any extra code to be generated in the Swift module. This allows C++ programmers to call into Swift APIs using the familiar C++ function and member function call syntax.

C++ interoperability does not require Swift opt in

The user model employed by Swift's C++ interoperability is different than the user model employed by Swift's Objective-C interoperability. All else being equal, it's best if supporting interoperation with a language doesn't require changing source code or emitting extra code eagerly from the compiler. This doesn't work for Objective-C interoperability because Objective-C requires specially-compatible classes and methods, the code for which cannot be emitted from an Objective-C header. Swift chose to require programs to opt in to Objective-C interoperability with the @objc attribute, both to make export more predictable and to avoid emitting extra code and metadata for all classes. In contrast, as long as the C++ compiler supports the Swift calling convention, a C++ header can call native Swift functions directly, and the C++ type system can be used to wrap most Swift types in a safe C++ representation. Because of this, there is no reason to require Swift module authors to opt in into C++ interoperability. Instead, any Swift module that can be imported into Swift can also be imported into C++, and most APIs will come across automatically.

Some API authors will desire explicit control over the C++ API. Swift will provide an annotation such as the proposed @expose attribute to allow precise control over which APIs get exposed to C++. The API authors will be able to specify that their module should only expose the annotated APIs to C++ when they desire to do so.

Goals

The ultimate goal of allowing Swift APIs to be used from C++ is to remove a barrier to writing more code in Swift rather than C++. Some C++ programmers would like to adopt a memory-safe language like Swift in their codebase. These programmers would typically like to adopt Swift gradually instead of relying on rewriting all of their code in Swift at once. For instance, they might write a new component in Swift, which then needs to be made usable from the rest of their C++ codebase. Today this could be done via an Objective-C wrapper that relies on @objc annotations. However, maintaining @objc wrapper code is annoying and error-prone, and it often adds significant performance costs and expressivity restrictions because of the limitations of C and Objective-C. Allowing C++ to directly use Swift APIs with minimal restrictions or performance overhead would lower the burden of adding Swift to an existing C++ codebase. It also makes Swift an even better language for stable system libraries and APIs, as it allows C++ projects to consume most such libraries without restrictions. These libraries would also have the option of not providing an Objective-C interface for Objective-C clients, as these clients could use the Swift APIs from Objective-C++ instead.

The primary goal of Swift-to-C++ interoperability is to expose every language feature of Swift for which a reasonable C++ API can be created to represent that feature. Swift and C++ are both feature rich programming languages. They share many similar features, but Swift has some features that lack close analogues in C++. For example, C++ does not have a concept of argument labels, but the argument labels can be integrated into the base name of a function to get a roughly similar effect. On the other hand, C++ does not have any feature that can represent a Swift feature like Swift macros in a reasonable manner. Therefore, macros must be dropped in the C++ interface to the module.

It is a goal that Swift-to-C++ interoperability can be used by both mixed Swift/C++ language projects that want to interoperate within themselves, and by C++ projects that need to use a Swift library that wasn't developed with C++ interoperability in mind.

It is a goal of Swift-to-C++ interoperability to expose Swift APIs in a safe, performant and ergonomic manner. The Swift compiler and the language specification should follow several key related principles that are presented below.

Safety

Safety is a top priority for the Swift programming language. Swift code expects its callers to adhere to Swift’s type rules and Swift’s memory model, regardless of whether it’s called from Swift or C++. Thus, the C++ code that calls Swift should properly enforce Swift’s expected language contracts. The enforcement should be done automatically in the generated header. This kind of enforcement does not prevent all possible issues though, as it does not change C++'s safety model. C++ is unsafe by default, and the user is able to use regular C++ pointers to write to memory as if they were using Swift's UnsafeMutablePointer type. This means that bugs in C++ code can easily lead to violations of Swift's invariants. For instance, a bug in user’s C++ code could accidentally overwrite a Swift object stored on the heap, which could cause unexpected behavior (such as a segfault) in Swift code. The user is expected to obey Swift's memory model and type rules when calling into Swift from C++, and thus they bear the ultimate responsibility for avoiding bugs like this one. The user can use certain program analysis tools, such as Address Sanitizer, to help them catch bugs that violate Swift's memory model rules.

Swift expects that values of correct types are passed to Swift APIs. For instance, for calls to generic functions, Swift expects the caller to pass a value of type that conforms to all of the required generic requirements. Type safety should be enforced from C++ as well. The C++ compiler should verify that correct types are passed to the Swift APIs as they’re invoked from C++. A program that tries to pass an incorrect Swift type into a Swift function should not compile. Select uses of specific Swift types might be incompatible with static type validation. For example, verification of generic requirements for a Swift opaque type value returned from a Swift function in C++ requires run-time validation. The program should report a fatal error when such run-time type validation fails, to ensure that the type invariants that Swift expects are not violated. The reported fatal error should clearly indicate why type validation failed and point to the location in the user's code which caused the run-time check.

Memory safety is paramount for Swift code. Swift automatically manages the lifetime of value and reference types in Swift. These principles should translate to C++ as well. The lifetime of Swift values that are created or passed around in C++ should be managed automatically. For instance, the generated C++ code should increment the retain count of a Swift class instance when a Swift class type value is copied in C++, and it should decrement the retain count of a Swift class instance when a Swift class type value is destroyed in C++. The default convention for using Swift types in C++ should discourage dangling references and any other patterns that could lead to an invalid use of a Swift value.

Swift’s memory safety model also requires exclusive access to a value in order to modify that value. For instance, the same value can not be passed to two inout parameters in the same function call. Swift enforces exclusivity using both compile-time and run-time checks. The generated run-time checks trap when an exclusivity violation is detected at runtime. Calls into Swift APIs from C++ should verify exclusivity for Swift values as well.

Performance

Swift-to-C++ bridging should be as efficient as possible. The Swift compiler should avoid unnecessary overhead for calling into Swift functions from C++, or using Swift types from C++. Generally, the bridging code should not convert Swift types into their C++ counterparts automatically, as that can add a lot of overhead. For instance, a Swift function returning a String should still return a Swift String in C++, and not a std::string. Some specific "primitive" Swift types should be converted into their C++ counterparts automatically in order to improve their ergonomics in C++. The conversion for such primitive types should be zero-cost as their ABI should match in Swift and C++. For instance, a Swift function returning a Float can return a float in C++ as they use the same underlying primitive LLVM type to represent the floating point value.

Some Swift features require additional overhead to be used in C++. Resilient value types are a good example of this; C++ expects types to have a statically-known layout, but Swift's resilient value types do not satisfy this, and so the generated C++ types representing those types may need to dynamically allocate memory internally. In cases like these, the C++ interface should at least strive to minimize the dynamic overhead, for example by avoiding allocations for sufficiently small types.

Achieving safety with performance in mind

Certain aspects of Swift’s memory model impose certain restrictions that create tension between the goal of achieving safety and the goal of avoiding unnecessary overhead for calling into Swift from C++. Checking for exclusivity violations is a good example of this. The C++ compiler does not have a notion of exclusivity it can verify, so it is difficult to prove that a value is accessed exclusively in the C++ code that calls into Swift. This means that the C++ code that calls into Swift APIs will most likely require more run-time checks to validate exclusivity than similar Swift code that calls the same Swift APIs.

The adherence to Swift’s type and memory safety rules should be prioritized ahead of performance when C++ calls into Swift, even if this means more run-time checks are required. For users seeking maximum performance, Swift provides additional compiler flags that avoid certain run-time checks. Those flags should be taken into account when the C++ header is generated. An example of such flag is -enforce-exclusivity. When -enforce-exclusivity=none is passed to Swift, the Swift compiler does not emit any run-time checks that check for exclusivity violations. A flag like this should also affect the generated C++ header, and the Swift compiler should not emit any run-time checks for exclusivity in the generated C++ header when this flag is used.

Correct modeling of certain aspects of Swift type semantics in C++ requires additional overhead. This issue is most prominent when modeling Swift's consume operation, as it performs a destructive move, which C++ does not support. Thus, the consume operation has to be modeled using a non-destructive move in C++, which requires additional storage and runtime checks when a Swift value type is used in C++. In cases like this one, the design of how Swift language constructs are mapped to C++ should strive to be as ergonomic and as intuitive as possible, provided there's still a way to achieve appropriate performance using compiler optimizations or future extensions to the C++ language standard.

Ergonomics

Swift APIs should be mapped over to C++ language features that have a direct correspondence to the Swift language feature. In cases where a direct correspondence does not exist, the Swift compiler should provide a reasonable approximation to the original Swift language feature using other C++ constructs. For example, Swift’s enum type can contain methods and nested types, and such constructs can’t be represented by a single C++ enum type in an idiomatic manner. Swift’s enum type can be mapped to a C++ class instead that allows both enum-like switch statement behavior and also enables the C++ user to invoke member functions on the Swift enum value and access its nested types from C++.

The C++ representation of certain Swift types should be appropriately enhanced to allow them to be used in an idiomatic manner. For instance, it should be possible to use Swift’s Array type (or any type that conforms to Collection) in a ranged-based for loop in C++. Such enhancements should be done with safety in mind, to ensure that Swift’s memory model is not violated.

There should be no differences on the C++ side between using libraries that opt-in into library evolution and libraries that don’t, except in specific required cases, like checking the unknown default case of a resilient enum.

Clear language mapping rules

C++ is a very expressive language and it can provide representation for a lot of Swift language constructs. Not every Swift language construct will map to its direct C++ counterpart. For instance, Swift initializers might get bridged to static init member functions instead of constructors in C++, to allow C++ code to call failable initializers in a way that’s consistent with other initializer calls. Therefore, it’s important to provide documentation that describes how Swift language constructs get mapped to C++. It is a goal of C++ interoperability to provide a clear and well-defined mapping for how Swift language constructs are mapped to C++. Additionally, it is also a goal to clearly document which language constructs are not bridged to C++. In addition to documentation, compiler diagnostics should inform the user about types or functions that can’t be exposed to C++, when the user wants to expose them explicitly. It is a goal of C++ interoperability to add a set of clear diagnostics that let the user know when a certain Swift declaration is not exposed. It is not a goal to diagnose such cases when the user did not instruct the compiler to expose a declaration explicitly. For example, the Swift compiler might not diagnose when an exposed Swift type does not expose one of its public methods to C++ due to its return type not being exposed, if such method does not have an explicit annotation that instructs the compiler that this method must be exposed to C++.

Some Swift APIs patterns will map to distinct C++ language constructs or patterns. For instance, an empty Swift enum with static members is commonly used in a namespace-like manner in Swift. This kind of enum can be mapped to a C++ namespace. It is a goal of C++ interoperability to provide a clear mapping for how Swift API patterns like this one are bridged to C++.

The semantics of how Swift types behave should be preserved when they’re mapped to C++. For instance, in C++, there should still be a semantic difference between Swift value and reference types. Additionally, Swift’s copy-on-write data types like Array should still obey the copy-on-write semantics in C++.

Swift language evolution and API design should be unaffected

Importing Swift APIs into C++ should not degrade the experience of programming in Swift. That means that Swift as a language should continue to evolve based on its current goals. New additions to Swift should not restrict their expressivity or safety in order to provide a better experience for Swift APIs in C++. Such new features do not have to be exposed to C++ when it doesn't make sense to do so.

Swift API authors should not change the way they write Swift code and design Swift APIs based on how specific Swift language constructs are exposed to C++. They should use the most effective Swift constructs for the task. It is a key goal of C++ interoperability that the exposed C++ interfaces are safe, performant, and ergonomic enough that programmers will not be tempted to make their Swift code worse just to make the C++ interfaces better.

Objective-C support

The existing Swift to Objective-C bridging layer should still be supported even when C++ bindings are generated in the generated header. Furthermore, the generated C++ bindings should use appropriate Objective-C++ types or constructs in the generated C++ declarations where it makes sense to do so. For instance, a Swift function that returns an Objective-C class instance should be usable from an Objective-C++ compilation unit.

The approach

The Swift compiler exposes Swift APIs to C++ by generating a header file that contains C++ declarations that wrap around the underlying calls into Swift functions that use the native Swift calling convention. The header also provides a suitable representation for Swift types. This generation can be done retroactively, starting from a Swift module interface file, and does not need to be requested when the Swift module is built from source.

Currently the generated header file depends on several LLVM and Clang compiler features for the Swift calling convention support, and thus it can only be compiled by Clang. The header does not depend on any other Clang-specific C++ language extensions. The header can use some other optional Clang-only features that improve developer experience for the C++ users, like specific attributes that improve diagnostics. These Clang-only features are enabled only when the Clang that is being used supports them, so older versions of Clang would still be able to consume the generated header.

The generated header file uses advanced C++ features that require a recent C++ language standard. C++20 is the recommended language standard to use for Swift APIs, however, the generated header should also be compatible with C++17 and C++14.

The generated header file also contains the C and the Objective-C declarations that Swift exposes to C and Objective-C on platforms that support C or Objective-C interoperability. Thus a single header file can be used by codebases that mix C, Objective-C and C++.

For the majority of Swift libraries, the generated header file is a build artifact generated by the client. This is different from Objective-C interoperability, where the generated header is always generated by the library owner and distributed with it. However, Swift library owners can also generate the header and distribute it with the library if they want the client to consume the C++ interface directly without having to run the Swift compiler on their end.

On platforms with ABI stability, the generated C++ code for an ABI stable Swift interface is not tied to the Swift compiler that compiled the Swift code for that Swift module, as ABI stability is respected by the C++ code in the header. In all other cases the generated C++ code in the header is assumed to be tied to the Swift compiler that compiled the Swift module, and thus the header should be regenerated when the compiler changes.

The next few sections provide a high level overview of how Swift types and some other language constructs get bridged to C++. The exact details of how Swift language constructs are bridged will be covered by Swift evolution proposals, and additional documentation, such as this preliminary user guide document.

Bridging Swift types

The generated header contains C++ class types that represent the Swift struct, enum, and class types that are exposed by the Swift module. These types provide access to methods, properties and other members using either idiomatic C++ constructs or, in some cases, non-idiomatic C++ constructs that allow C++ to access more functionality in a consistent manner. The basic operations on these types follow the corresponding Swift semantics. For instance, a C++ value for a Swift class type is essentially a shared pointer that owns a reference to a Swift object, whereas a C++ value for a Swift struct type stores a Swift value of that type, and copying or destroying the C++ value copies or destroys the underlying Swift value. The C++ types also support C++ move semantics and translate them as appropriate to Swift's consume operation. This enables high-performance use of Swift types from C++ and allows Swift's non-copyable types to be exposed to C++.

Protocol types also get exposed to C++. They provide access to their protocol interface to C++. The generated header also provides facilities to combine protocol types into a protocol composition type. The protocol composition type provides access to the combined protocol interface in C++.

Bridging generics

Swift generic functions and types get bridged to C++ as C++ function and class templates. A generated C++ template instantiates a type-checked generic interface that uses the underlying polymorphic semantics that generics require when Swift APIs are called from the generated header. Type-checking is performed using the requires clause introduced in C++20. When C++17 and earlier is used, type-checking is performed using other legacy methods, like enable_if and static_assert. The two type-checking methods are compatible with the delayed template parsing compiler feature that Clang uses when building for Windows.

To help achieve the performance goals outlined in the prior section, the generated class templates specialize the storage for the underlying Swift generic type when the Swift API that is exposed to C++ contains such a bounded Swift generic type. This ensures that non-resilient bounded generic values can be stored inline in a C++ type that represents the underlying Swift type, instead of being boxed on the heap.

Standard library support

The Swift standard library contains a lot of useful functions and types that get bridged to C++. The generated standard library bindings enhance various Swift types like Optional and Array to provide idiomatic C++ APIs that allow the user to use such types in an idiomatic manner from C++.

Using Swift types in C++ templates

The generated header is useful in mixed-language projects as C++ sources can include it, allowing C++ to call Swift APIs. However the C++ interoperability project also provides support for calling C++ APIs from Swift. In certain cases such C++ APIs contain function or class templates that need to be instantiated in Swift. Swift gives the user the ability to use Swift types in such cases, so the C++ templates have to be instantiated with Swift types. This means that the Swift types need to be translated into their C++ counterparts, which could then be used inside the instantiated C++ template specialization. This Swift-to-C++ type translation is performed using the same mechanism that’s used by the Swift compiler to generate the header with C++ interface for a Swift module. This means that a C++ template instantiated with a Swift type will see the same C++ representation of that Swift type regardless of whether it was instantiated from C++, or from Swift.

Evolution process

The approach and the goals for how Swift APIs get bridged to C++ are outlined above. Each distinct Swift language construct that’s bridged to C++ will need to be covered by a detailed evolution proposal. These evolution proposals can refer to this vision document as a general context document for how Swift APIs should be bridged to C++. Every Swift API pattern that has a distinct mapping in C++ will also need a detailed and self-contained evolution proposal as well. The design for how each district language feature or API pattern is bridged to C++ is ratified only once its respective proposal goes through the Swift evolution process and is accepted by the Swift community.

The Swift ecosystem

As a supported language feature, C++ and Swift interoperability must work well on every platform supported by Swift. In similar vein, tools in the Swift ecosystem should be updated to support C++ interoperability. The next few sections of this document provide specific recommendations for how various tools in the Swift ecosystem should be adapted to support C++ interoperability. The C++ interoperability workgroup intends to work on supporting most of these recommendations in the tools that are bundled with the Swift toolchain while working on the initial version of interoperability that could be adopted for production use cases.

Build tool support

The Swift package manager (SwiftPM) is one of the most commonly used ways to build Swift code. Swift package manager can also build C and C++ code. SwiftPM should provide good support for bridging Swift APIs to C++ out of the box. It should generate a compatibility header for a Swift package when it determines that the C++ code in the same or dependent package includes such a header, and it should ensure that this header can be found when the C++ code is compiled. SwiftPM already has some support for generating a header file with Objective-C interface for a Swift module, and the C++ interoperability workgroup intends to reuse that support for supporting the generation of C++ bindings as well.

CMake is a widely used tool used for configuring and building C++ code. CMake should provide good support for adding Swift code to C++ CMake targets. Swift’s ecosystem as a whole should ensure that it should be as straightforward as possible to add support for bridging Swift APIs to C++ within the same CMake project. The C++ interoperability workgroup should provide an example CMake project that shows how Swift and C++ can interoperate between each other, once the appropriate support for mixed-language Swift and C++ targets lands in CMake.

Debugging support

Debugging support is critical for great user experience. LLDB should understand that C++ types in the generated header are just wrappers around Swift values. It should be able to display the underlying Swift value in the debugger when the user tries to inspect the C++ value that stores the Swift value in the debugger. In addition to that, the generated compatibility header should be correctly annotated to ensure that C++ inline thunks that call Swift APIs can be skipped when a user steps in or steps out into or from a call when debugging a program.

IDE support

SourceKit-LSP is a language server that provides cross-platform IDE support for Swift code in the Swift ecosystem. It can also act as a language server for C, Objective-C and C++ as it can wrap around and redirect queries to clangd. This in turn allows SourceKit-LSP to provide support for mixed-language IDE queries, like jump-to-definition, that allows the IDE client to jump from an Objective-C method to call to its underlying Swift implementation. The C++ interoperability workgroup intends to reuse the current support for mixed-language Swift and Objective-C queries to add similar functionality for mixed-language Swift and C++ queries. This would allow IDE clients that use SourceKit-LSP to use features that can operate across the Swift/C++ language boundary. For instance, a client IDE would be able to support jump-to-definition from a Swift call expression to the called C++ function using SourceKit-LSP. This mixed-language query support in SourceKit-LSP is powered by the indexing data emitted by Clang. The C++ interoperability workgroup intends to extend Clang's indexing support to represent references to the wrapper C++ declarations from the generated header as references to the underlying Swift declarations.