Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create ConcurrencyRoadmap.md #34517

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

airspeedswift
Copy link
Member

@airspeedswift airspeedswift commented Oct 30, 2020

File describing the roadmap for introduction of concurrency. Living document expected to link to various proposals and cover features as they are designed.

Errors or wording clarification welcome. Please direct questions/discussion about the design to the Swift forum thread here. You may also find more details in the detailed feature pitches that are linked from this doc.

docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
@Brandon-T
Copy link

It seems to me like actor and UIActor is overly complex for what seems like an automatic Mutex or locking mechanism guarding a variable.. Is it the Objective-C version of atomic?

Maybe call it atomic or use @synchronized? Am I missing something?

@airspeedswift
Copy link
Member Author

Hi @Brandon-T – discussions around the design should be held on the swift forums. The PR is primarily for discussion of the wording of the proposal. Note there is a fuller proposal for the Actors component posted here, which is probably the best place to discuss the tradeoffs of different isolation mechamisms.

docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
@mRB0
Copy link

mRB0 commented Oct 30, 2020

Typo: There are a couple spots where Controller is misspelled in PlayerRefreshControler.

I have a couple questions after reading the document. I think there's an opportunity for clarification in the "things to note" sections:

  1. The document states:

You can mark classes and functions as being tied to that actor with an attribute. The compiler will let you reference this class from anywhere, but to actually call this method, you need to be on the UI actor.

Does that mean I need to explicitly call the function from the UI actor, or is that handled automatically when I call the function (from any actor/queue)?

  1. If one async function calls another, is it possible for the first one to be resumed on a different queue than the one it was on when it invoked the second function? Does the behavior vary depending on whether my function is in an actor class?

For example, if I invoke some UI functions and then call an async function, can I expect to still be on the UI queue when it returns so I can continue updating the UI?

Thanks! I'm really excited to see this becoming available :)

Copy link

@pwightman pwightman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Controler -> Controller typo fixes

docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
docs/ConcurrencyRoadmap.md Outdated Show resolved Hide resolved
Co-authored-by: Parker Wightman <[email protected]>
@airspeedswift
Copy link
Member Author

Hi @mRB0 – thanks for the typo spot! Note, questions/discussion of the design itself should be directed to the forum thread. You might find the answer in the more detailed pitches on async and actors as well, that are now linked from the roadmap doc.

Copy link

@nonsensery nonsensery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few little typos (?) that I noticed

// unprotected: global variable not in an actor
racyGlobal += ["Racy access"]

// unprotected: racyProperty is immutable, but it is a reference type

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be "immutableClassReference"?

Suggested change
// unprotected: racyProperty is immutable, but it is a reference type
// unprotected: immutableClassReference is immutable, but it is a reference type

- The explicit `self.` has been eliminated from property accesses, because no escaping closures capturing `self` are required.
- The accesses to the properties `allPlayers` and `players` now cannot have data races.

To understand how the the last point is achieved we must step out a layer and look at how queues are used to protect state.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To understand how the the last point is achieved we must step out a layer and look at how queues are used to protect state.
To understand how the last point is achieved we must step out a layer and look at how queues are used to protect state.

* A **synchronous function** is the kind of function that Swift programmers are already used to: it runs to completion on a single thread, with no interleaving code except for any synchronous functions it calls.
* A **thread** refers to the underlying platform’s concept of threads. Platforms vary, but tend to share basic characteristics: true concurrency requires creating a platform thread, but creating and running platform threads is expensive. C function calls, and ordinary synchronous Swift functions, require the use of a platform thread.
* An **asynchronous function** is a new kind of function which does not have to run to completion without interruption. An interruption causes the function to be **suspended**. The point at which an async function may give up its thread is a **suspension point**.
* A **task** is an operation that runs asynchronously. All asynchronous functions run as part of some task. When an asynchronous function calls another asynchronous function, the call is still part of the the same task, even if the call has to change actors. A task is the analogue of a thread for asynchronous functions.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* A **task** is an operation that runs asynchronously. All asynchronous functions run as part of some task. When an asynchronous function calls another asynchronous function, the call is still part of the the same task, even if the call has to change actors. A task is the analogue of a thread for asynchronous functions.
* A **task** is an operation that runs asynchronously. All asynchronous functions run as part of some task. When an asynchronous function calls another asynchronous function, the call is still part of the same task, even if the call has to change actors. A task is the analogue of a thread for asynchronous functions.

- The `await` keyword appears before the expression calling `allPlayers`, indicating that the `refreshPlayers` function can become _suspended_ at this point.
- `await` works similarly to `try`, in that it need only appear once at the start of an expression that can suspend, not directly in front of every call that can suspend within that expression.
- The explicit `self.` has been eliminated from property accesses, because no escaping closures capturing `self` are required.
- The accesses to the properties `allPlayers` and `players` now cannot have data races.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allPlayers is not a property.

- Because of this, we can use expression composition to call the `map` function directly on the returned value.
- The `await` keyword appears before the expression calling `allPlayers`, indicating that the `refreshPlayers` function can become _suspended_ at this point.
- `await` works similarly to `try`, in that it need only appear once at the start of an expression that can suspend, not directly in front of every call that can suspend within that expression.
- The explicit `self.` has been eliminated from property accesses, because no escaping closures capturing `self` are required.
Copy link

@dimpiax dimpiax Oct 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because no escaping closures capturing self are required.

because in escaping closures capturing self is required.

docs/ConcurrencyRoadmap.md Show resolved Hide resolved
print(other.immutable)

// error: an actor cannot access another's mutable state
otherActor.mutableArray += ["not allowed"]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be just other?

Suggested change
otherActor.mutableArray += ["not allowed"]
other.mutableArray += ["not allowed"]

@phoney
Copy link

phoney commented Oct 30, 2020

Very well written.

Few typos:

with number of features -> with a number of features
To understand how the the last point -> how the last
Functions will opt into to being async, -> opt into being
any code that hold a reference to the class. -> that holds a reference
"actor local". -> "actor local." More than one of these
smart quotes are used in some places and plain old quotes in others
traditional “notification” methods, -> smart quotes; maybe should be italic

print(other.mutableArray.first)

// allowed: async functions can call async functions on other actors
await other.asyncFunction(otherActor: self)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be just other too?

Suggested change
await other.asyncFunction(otherActor: self)
await other.asyncFunction(other: self)

Copy link

@NinoScript NinoScript left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this 👍

This in turn will allow the defaults to be changed:

- Global variables will be required to be protected by a global actor, or marked "actor unsafe".
- Classes (and types that contain class references) will change from being from being "actor unsafe" by default to being "actor local".

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed from being from being

Suggested change
- Classes (and types that contain class references) will change from being from being "actor unsafe" by default to being "actor local".
- Classes (and types that contain class references) will change from being "actor unsafe" by default to being "actor local".

Copy link
Contributor

@stevapple stevapple left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mainly multiple spaces?

Things to note about this example:

- Declaring a class to be an actor is similar to giving a class a private queue and synchronizing all access to its private state through that queue.
- Because this synchronization is now understood by the compiler, you cannot _forget_ to use the queue to protect state: the compiler will ensure that you are running on the queue in the class's methods, and it will prevent you from accessing the state outside those methods.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Because this synchronization is now understood by the compiler, you cannot _forget_ to use the queue to protect state: the compiler will ensure that you are running on the queue in the class's methods, and it will prevent you from accessing the state outside those methods.
- Because this synchronization is now understood by the compiler, you _won’t forget_ to use the queue to protect state: the compiler will ensure that you are running on the queue in the class's methods, and it will prevent you from accessing the state outside those methods.


- **`async`/`await`** introduces a [coroutine-based](https://en.wikipedia.org/wiki/Coroutine) `async`/`await` model to Swift. Functions will opt into to being `async`, and can then `await` the results of other `async` functions, allowing asynchronous code to be expressed in a more natural "straight-line" form. Pitch [here](https://forums.swift.org/t/concurrency-asynchronous-functions/41619), proposal [here](https://github.com/DougGregor/swift-evolution/blob/async-await/proposals/nnnn-async-await.md).

- **`Task` API and Structured Concurrency** introduces the concept of a task to the standard library. This will cover APIs to create detached tasks, task "nurseries" for dynamically creating child tasks, and the mechanics for cancellation and prioritization of tasks. It also introduces scope-based mechanisms for awaiting values from multiple child tasks, based on the principles of [structured concurrency](https://en.wikipedia.org/wiki/Structured_concurrency). Pitch [here](https://forums.swift.org/t/concurrency-structured-concurrency/41622), proposal [here](https://github.com/DougGregor/swift-evolution/blob/structured-concurrency/proposals/nnnn-structured-concurrency.md).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **`Task` API and Structured Concurrency** introduces the concept of a task to the standard library. This will cover APIs to create detached tasks, task "nurseries" for dynamically creating child tasks, and the mechanics for cancellation and prioritization of tasks. It also introduces scope-based mechanisms for awaiting values from multiple child tasks, based on the principles of [structured concurrency](https://en.wikipedia.org/wiki/Structured_concurrency). Pitch [here](https://forums.swift.org/t/concurrency-structured-concurrency/41622), proposal [here](https://github.com/DougGregor/swift-evolution/blob/structured-concurrency/proposals/nnnn-structured-concurrency.md).
- **`Task` API and Structured Concurrency** introduces the concept of a task to the standard library. This will cover APIs to create detached tasks, task "nurseries" for dynamically creating child tasks, and the mechanics for cancellation and prioritization of tasks. It also introduces scope-based mechanisms for awaiting values from multiple child tasks, based on the principles of [structured concurrency](https://en.wikipedia.org/wiki/Structured_concurrency). Pitch [here](https://forums.swift.org/t/concurrency-structured-concurrency/41622), proposal [here](https://github.com/DougGregor/swift-evolution/blob/structured-concurrency/proposals/nnnn-structured-concurrency.md).


* Properties of an actor will be protected by the actor.
* Immutable memory (such as a `let` constant), local memory (such as a local variable that’s never captured), and value component memory (such as a properties of a struct, or an enum case), are already protected from data races.
* Unsafe memory (such as an arbitrary allocation referenced by an `UnsafeMutablePointer`) is associated with an unsafe abstraction. It’s actively undesirable to try to force these abstractions to be used safely, because these abstractions are meant to be usable to bypass safe language rules when necessary. Instead, we have to trust the programmer to use these correctly.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Unsafe memory (such as an arbitrary allocation referenced by an `UnsafeMutablePointer`) is associated with an unsafe abstraction. It’s actively undesirable to try to force these abstractions to be used safely, because these abstractions are meant to be usable to bypass safe language rules when necessary. Instead, we have to trust the programmer to use these correctly.
* Unsafe memory (such as an arbitrary allocation referenced by an `UnsafeMutablePointer`) is associated with an unsafe abstraction. It’s actively undesirable to try to force these abstractions to be used safely, because these abstractions are meant to be usable to bypass safe language rules when necessary. Instead, we have to trust the programmer to use these correctly.


### First Phase: Basic Actor Isolation

The first phase introduces safety benefits. Users will be able to protect global variables with global actors, and to protect class members by converting them to actor classes. Frameworks that require access on a particular queue can define a global actor and default protocols to it.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The first phase introduces safety benefits. Users will be able to protect global variables with global actors, and to protect class members by converting them to actor classes. Frameworks that require access on a particular queue can define a global actor and default protocols to it.
The first phase introduces safety benefits. Users will be able to protect global variables with global actors, and to protect class members by converting them to actor classes. Frameworks that require access on a particular queue can define a global actor and default protocols to it.

This change in default will necessitate a **source break**, and will need to be gated by a language mode. Code that touches a mutable global variable or shares a class reference across actor boundaries fundamentally cannot be shown to be safe from data races, and will need to change to ensure that it (and code written in the future) is safe from data races. It is hoped that this source break will not be onerous:

- it is expected that use of global variables should be sparing, and most global variables can be protected by a global actor;
- as long as classes are not shared across actor boundaries, "actor local" annotation should not affect code within an actor;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- as long as classes are not shared across actor boundaries, "actor local" annotation should not affect code within an actor;
- as long as classes are not shared across actor boundaries, "actor local" annotation should not affect code within an actor;

* A **synchronous function** is the kind of function that Swift programmers are already used to: it runs to completion on a single thread, with no interleaving code except for any synchronous functions it calls.
* A **thread** refers to the underlying platform’s concept of threads. Platforms vary, but tend to share basic characteristics: true concurrency requires creating a platform thread, but creating and running platform threads is expensive. C function calls, and ordinary synchronous Swift functions, require the use of a platform thread.
* An **asynchronous function** is a new kind of function which does not have to run to completion without interruption. An interruption causes the function to be **suspended**. The point at which an async function may give up its thread is a **suspension point**.
* A **task** is an operation that runs asynchronously. All asynchronous functions run as part of some task. When an asynchronous function calls another asynchronous function, the call is still part of the the same task, even if the call has to change actors. A task is the analogue of a thread for asynchronous functions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* A **task** is an operation that runs asynchronously. All asynchronous functions run as part of some task. When an asynchronous function calls another asynchronous function, the call is still part of the the same task, even if the call has to change actors. A task is the analogue of a thread for asynchronous functions.
* A **task** is an operation that runs asynchronously. All asynchronous functions run as part of some task. When an asynchronous function calls another asynchronous function, the call is still part of the the same task, even if the call has to change actors. A task is the analogue of a thread for asynchronous functions.

* A **thread** refers to the underlying platform’s concept of threads. Platforms vary, but tend to share basic characteristics: true concurrency requires creating a platform thread, but creating and running platform threads is expensive. C function calls, and ordinary synchronous Swift functions, require the use of a platform thread.
* An **asynchronous function** is a new kind of function which does not have to run to completion without interruption. An interruption causes the function to be **suspended**. The point at which an async function may give up its thread is a **suspension point**.
* A **task** is an operation that runs asynchronously. All asynchronous functions run as part of some task. When an asynchronous function calls another asynchronous function, the call is still part of the the same task, even if the call has to change actors. A task is the analogue of a thread for asynchronous functions.
* An async function can create a **child task**. Child tasks inherit some of the structure of their parent task, including its priority, but can run concurrently with it. However, this concurrency is bounded: a function that creates a child task must wait for it to end before returning.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* An async function can create a **child task**. Child tasks inherit some of the structure of their parent task, including its priority, but can run concurrently with it. However, this concurrency is bounded: a function that creates a child task must wait for it to end before returning.
* An async function can create a **child task**. Child tasks inherit some of the structure of their parent task, including its priority, but can run concurrently with it. However, this concurrency is bounded: a function that creates a child task must wait for it to end before returning.

* A program wishes to initiate independent concurrent work that can outlast its spawning context uses a **detached task** rather than a bounded child task.
* A **partial task** is a unit of schedulable work. When the currently-executing function in a task is suspended, that is the end of the partial task, and a new partial task is created to continue the overall task’s work.
* An **executor** is a service which accepts the submission of partial tasks and arranges for some thread to run them. An async function that is currently running always knows the executor on which it's running. An executor is called **exclusive** if the partial tasks submitted to it will never be run concurrently.
* An **actor** is an independent part of the program which can run code. It can only run one piece of code at a time — that is, it acts as an exclusive executor — but the code it runs can execute concurrently with the code run by other actors.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* An **actor** is an independent part of the program which can run code. It can only run one piece of code at a time — that is, it acts as an exclusive executor — but the code it runs can execute concurrently with the code run by other actors.
* An **actor** is an independent part of the program which can run code. It can only run one piece of code at a time — that is, it acts as an exclusive executor — but the code it runs can execute concurrently with the code run by other actors.

* A **partial task** is a unit of schedulable work. When the currently-executing function in a task is suspended, that is the end of the partial task, and a new partial task is created to continue the overall task’s work.
* An **executor** is a service which accepts the submission of partial tasks and arranges for some thread to run them. An async function that is currently running always knows the executor on which it's running. An executor is called **exclusive** if the partial tasks submitted to it will never be run concurrently.
* An **actor** is an independent part of the program which can run code. It can only run one piece of code at a time — that is, it acts as an exclusive executor — but the code it runs can execute concurrently with the code run by other actors.
* An actor can have protected state which can only be accessed by that actor. The system through which this is achieved is called **actor isolation**. The long term goal is for Swift to guarantee actor isolation by default.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* An actor can have protected state which can only be accessed by that actor. The system through which this is achieved is called **actor isolation**. The long term goal is for Swift to guarantee actor isolation by default.
* An actor can have protected state which can only be accessed by itself. The system through which this is achieved is called **actor isolation**. The long term goal is for Swift to guarantee actor isolation by default.

* An **actor** is an independent part of the program which can run code. It can only run one piece of code at a time — that is, it acts as an exclusive executor — but the code it runs can execute concurrently with the code run by other actors.
* An actor can have protected state which can only be accessed by that actor. The system through which this is achieved is called **actor isolation**. The long term goal is for Swift to guarantee actor isolation by default.
* An **actor class** is a reference type, each instance of which is a separate actor. Its protected state is its instance properties, and its actor functions are its instance methods.
* A **global actor** is a global object. Its protected state and actor functions may be spread across many different types. They can be marked with an actor-specific attribute, which Swift can infer in many cases.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* A **global actor** is a global object. Its protected state and actor functions may be spread across many different types. They can be marked with an actor-specific attribute, which Swift can infer in many cases.
* A **global actor** is a global object. Its protected state and actor functions may be spread across many different types. They can be marked with an actor-specific attribute, which Swift can infer in many cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet