Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swiftly proxies #155

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e4efbeb
DRAFT: Swiftly proxies
cmcgee1024 Aug 10, 2024
3b5c404
Add a swiftly install workflow where the version comes from the .swif…
cmcgee1024 Aug 10, 2024
956256f
update design to make auto-installation an error instead
cmcgee1024 Aug 20, 2024
375df29
provide a mechanism to find the currently in-use toolchain physical l…
cmcgee1024 Aug 20, 2024
ed68a9e
add more details about the selector prefix, and methods to escape
cmcgee1024 Aug 21, 2024
c745f4c
Restructure the PR to move the selector syntax from the proxies to a …
cmcgee1024 Aug 29, 2024
560236d
Implement proxy mechanism with dynamic toolchain selection
cmcgee1024 Sep 4, 2024
ae29e88
Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design
cmcgee1024 Sep 5, 2024
ce0d27a
Rewrite the select toolchain function with a type for the selection r…
cmcgee1024 Sep 5, 2024
66dd459
Update the documentation
cmcgee1024 Sep 5, 2024
2df7357
Create a swiftly run command
cmcgee1024 Sep 6, 2024
8976bba
Fix empty command case with a single ++++
cmcgee1024 Sep 7, 2024
3caab66
Write run command and proxy tests
cmcgee1024 Sep 9, 2024
31f4327
Regenerate the cli reference documentation
cmcgee1024 Sep 9, 2024
16caf80
Fix design document discrepancies and add install proxy argument tests
cmcgee1024 Sep 9, 2024
b2aa165
Update the list command to decorate default, and in-use toolchains
cmcgee1024 Sep 10, 2024
7504dd3
Update list tests to check for in use and default labels
cmcgee1024 Sep 10, 2024
0e7b661
Make the version argument optional in the install subcommand
cmcgee1024 Sep 11, 2024
43d620a
Fix case of empty bin directory when checking for overwrite
cmcgee1024 Sep 12, 2024
2e3c59d
Remove +install selector option from swift run in favour of regular `…
cmcgee1024 Sep 20, 2024
9dd640c
Import GPG keys on every install to get new signing keys from swift.org
cmcgee1024 Sep 17, 2024
5e615ea
Make recommended documentation changes.
cmcgee1024 Oct 18, 2024
10a0856
Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design
cmcgee1024 Oct 18, 2024
6d6050e
Provide a better error message on swiftly install with no version
cmcgee1024 Oct 19, 2024
6989796
Update README, and add documentation for the new run subcommand
cmcgee1024 Oct 20, 2024
52d081f
Prompt before updating the `.swift-version` file.
cmcgee1024 Oct 23, 2024
2f6d701
Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design
cmcgee1024 Oct 23, 2024
cb0e923
Create proxies on toolchain installation, creating only the necessary…
cmcgee1024 Nov 5, 2024
bb36de0
Fix the design document
cmcgee1024 Nov 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Write run command and proxy tests
  • Loading branch information
cmcgee1024 committed Sep 9, 2024
commit 3caab6679634299a841221a6129f6ee39b743785
87 changes: 51 additions & 36 deletions Sources/Swiftly/Run.swift
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,11 @@ internal struct Run: SwiftlyCommand {
$ swiftly run ./myscript.sh ++abcde

The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix \
that should be escaped you can disable the selection using a '++++' argument, which turns off any \
that should be escaped you can disable the selection using a '++' argument, which turns off any \
selector argument processing for subsequent arguments. This is anologous to the '--' that turns off \
flag and option processing for subsequent arguments in many argument parsers.

$ swiftly run ./myscript.sh ++++ +abcde +xyz
$ swiftly run ./myscript.sh ++ +abcde +xyz

The script will receive the argument '+abcde' followed by '+xyz'.
"""
Expand All @@ -60,40 +60,10 @@ internal struct Run: SwiftlyCommand {
internal mutating func run() async throws {
try validateSwiftly()

var escapedCommand: [String] = []
var selector: ToolchainSelector?
var install = false
var disableEscaping = false
for c in self.command {
if !disableEscaping && c == "++++" {
disableEscaping = true
continue
}

if !disableEscaping && c.hasPrefix("++") {
escapedCommand.append("+\(String(c.dropFirst(2)))")
continue
}

if !disableEscaping && c == "+install" {
install = true
continue
}

if !disableEscaping && c.hasPrefix("+") {
selector = try ToolchainSelector(parsing: String(c.dropFirst()))
continue
}

escapedCommand.append(c)
}

guard escapedCommand.count > 0 else {
throw Error(message: "Provide at least one command to run")
}

var config = try Config.load()

let (command, selector, install) = try extractProxyArguments(command: self.command)

let toolchain: ToolchainVersion?

if let selector = selector {
Expand All @@ -115,7 +85,7 @@ internal struct Run: SwiftlyCommand {
} else {
let matchedToolchain = config.listInstalledToolchains(selector: selector).max()
guard let matchedToolchain = matchedToolchain else {
throw Error(message: "The selected toolchain \(selector.description) didn't matched any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`")
throw Error(message: "The selected toolchain \(selector.description) didn't match any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`")
}

toolchain = matchedToolchain
Expand All @@ -136,11 +106,56 @@ internal struct Run: SwiftlyCommand {
}

do {
try await Swiftly.currentPlatform.proxy(toolchain, escapedCommand[0], [String](escapedCommand[1...]))
if let outputHandler = SwiftlyCore.outputHandler {
if let output = try await Swiftly.currentPlatform.proxyOutput(toolchain, command[0], [String](command[1...])) {
for line in output.split(separator: "\n") {
outputHandler.handleOutputLine(String(line))
}
}
return
}

try await Swiftly.currentPlatform.proxy(toolchain, command[0], [String](command[1...]))
} catch let terminated as RunProgramError {
Foundation.exit(terminated.exitCode)
} catch {
throw error
}
}
}

public func extractProxyArguments(command: [String]) throws -> (command: [String], selector: ToolchainSelector?, install: Bool) {
var args: (command: [String], selector: ToolchainSelector?, install: Bool) = (command: [], nil, false)

var disableEscaping = false

for c in command {
if !disableEscaping && c == "++" {
disableEscaping = true
continue
}

if !disableEscaping && c.hasPrefix("++") {
args.command.append("+\(String(c.dropFirst(2)))")
continue
}

if !disableEscaping && c == "+install" {
args.install = true
continue
}

if !disableEscaping && c.hasPrefix("+") {
args.selector = try ToolchainSelector(parsing: String(c.dropFirst()))
continue
}

args.command.append(c)
}

guard args.command.count > 0 else {
throw Error(message: "Provide at least one command to run.")
}

return args
}
45 changes: 29 additions & 16 deletions Sources/SwiftlyCore/Platform.swift
Original file line number Diff line number Diff line change
Expand Up @@ -129,13 +129,7 @@ extension Platform {
}

#if os(macOS) || os(Linux)
/// Proxy the invocation of the provided command to the chosen toolchain.
///
/// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with
/// the exit code and program information.
///
public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws {
// The toolchain goes to the beginning of the path, and the SWIFTLY_BIN_DIR is removed from it
internal func proxyEnv(_ toolchain: ToolchainVersion) throws -> [String: String] {
let tcPath = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin")
var newEnv = ProcessInfo.processInfo.environment

Expand All @@ -145,21 +139,36 @@ extension Platform {
}
newEnv["SWIFTLY_PROXY_IN_PROGRESS"] = "1"

// The toolchain goes to the beginning of the PATH
var newPath = newEnv["PATH"] ?? ""
if !newPath.hasPrefix(tcPath.path + ":") {
newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }.filter { $0 != swiftlyBinDir.path }).joined(separator: ":")
newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }).joined(separator: ":")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rather than splitting and joining again, can we just concatenate tcPath, ":", and newPath?

}
newEnv["PATH"] = newPath

// Remove traces of swiftly environment variables
newEnv.removeValue(forKey: "SWIFTLY_BIN_DIR")
newEnv.removeValue(forKey: "SWIFTLY_HOME_DIR")

// Add certain common environment variables that can be used to proxy to the toolchain
newEnv["CC"] = tcPath.appendingPathComponent("clang").path
newEnv["CXX"] = tcPath.appendingPathComponent("clang++").path

try self.runProgram([command] + arguments, env: newEnv)
return newEnv
}

/// Proxy the invocation of the provided command to the chosen toolchain.
///
/// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with
/// the exit code and program information.
///
public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws {
try self.runProgram([command] + arguments, env: self.proxyEnv(toolchain))
}

/// Proxy the invocation of the provided command to the chosen toolchain and capture the output.
///
/// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with
/// the exit code and program information.
///
public func proxyOutput(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws -> String? {
try await self.runProgramOutput(command, arguments, env: self.proxyEnv(toolchain))
}

/// Run a program.
Expand Down Expand Up @@ -208,20 +217,24 @@ extension Platform {
/// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with
/// the exit code and program information.
///
public func runProgramOutput(_ program: String, _ args: String...) async throws -> String? {
try await self.runProgramOutput(program, [String](args))
public func runProgramOutput(_ program: String, _ args: String..., env: [String: String]? = nil) async throws -> String? {
try await self.runProgramOutput(program, [String](args), env: env)
}

/// Run a program and capture its output.
///
/// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with
/// the exit code and program information.
///
public func runProgramOutput(_ program: String, _ args: [String]) async throws -> String? {
public func runProgramOutput(_ program: String, _ args: [String], env: [String: String]? = nil) async throws -> String? {
let process = Process()
process.executableURL = URL(fileURLWithPath: "/usr/bin/env")
process.arguments = [program] + args

if let env = env {
process.environment = env
}

let outPipe = Pipe()
process.standardInput = FileHandle.nullDevice
process.standardError = FileHandle.nullDevice
Expand Down
101 changes: 101 additions & 0 deletions Tests/SwiftlyTests/RunTests.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
import Foundation
@testable import Swiftly
@testable import SwiftlyCore
import XCTest

final class RunTests: SwiftlyTests {
static let homeName = "runTests"

/// Tests that the `run` command can switch between installed toolchains.
func testRunSelection() async throws {
try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) {
// GIVEN: a set of installed toolchains
// WHEN: invoking the run command with a selector argument for that toolchain
var run = try self.parseCommand(Run.self, ["run", "swift", "--version", "+\(Self.newStable.name)"])
var output = try await run.runWithMockedIO()
// THEN: the output confirms that it ran with the selected toolchain
XCTAssert(output.contains(Self.newStable.name))

// GIVEN: a set of installed toolchains and one is selected with a .swift-version file
let versionFile = URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(".swift-version")
try Self.oldStable.name.write(to: versionFile, atomically: true, encoding: .utf8)
// WHEN: invoking the run command without any selector arguments for toolchains
run = try self.parseCommand(Run.self, ["run", "swift", "--version"])
output = try await run.runWithMockedIO()
// THEN: the output confirms that it ran with the selected toolchain
XCTAssert(output.contains(Self.oldStable.name))

// GIVEN: a set of installed toolchains
// WHEN: invoking the run command with a selector argument for a toolchain that isn't installed
run = try self.parseCommand(Run.self, ["run", "swift", "+1.2.3", "--version"])
do {
try await run.run()
XCTAssert(false)
} catch let e as Error {
XCTAssert(e.message.contains("didn't match any of the installed toolchains"))
}
// THEN: an error is shown because there is no matching toolchain that is installed
}
}

/// Tests the `run` command verifying that the environment is as expected
func testRunEnvironment() async throws {
try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) {
// The toolchains directory should be the fist entry on the path
var run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $PATH"])
var output = try await run.runWithMockedIO()
XCTAssert(output[0].contains(Swiftly.currentPlatform.swiftlyToolchainsDir.path))

// The CC and CXX variables should be set to clang/clang++ in the toolchains
run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $CC; echo $CXX"])
output = try await run.runWithMockedIO()
XCTAssert(output[0].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path))
XCTAssert(output[0].hasSuffix("clang"))
XCTAssert(output[1].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path))
XCTAssert(output[1].hasSuffix("clang++"))
}
}

/// Tests the extraction of proxy arguments from the run command arguments.
func testExtractProxyArguments() throws {
var (command, selector, install) = try extractProxyArguments(command: ["swift", "build"])
XCTAssertEqual(["swift", "build"], command)
XCTAssertEqual(false, install)
XCTAssertEqual(nil, selector)

(command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"])
XCTAssertEqual(["swift", "build"], command)
XCTAssertEqual(false, install)
XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector)

(command, selector, install) = try extractProxyArguments(command: ["swift", "build", "+latest"])
XCTAssertEqual(["swift", "build"], command)
XCTAssertEqual(false, install)
XCTAssertEqual(try! ToolchainSelector(parsing: "latest"), selector)

(command, selector, install) = try extractProxyArguments(command: ["+5.6", "swift", "build"])
XCTAssertEqual(["swift", "build"], command)
XCTAssertEqual(false, install)
XCTAssertEqual(try! ToolchainSelector(parsing: "5.6"), selector)

(command, selector, install) = try extractProxyArguments(command: ["swift", "++1.2.3", "build"])
XCTAssertEqual(["swift", "+1.2.3", "build"], command)
XCTAssertEqual(false, install)
XCTAssertEqual(nil, selector)

(command, selector, install) = try extractProxyArguments(command: ["swift", "++", "+1.2.3", "build"])
XCTAssertEqual(["swift", "+1.2.3", "build"], command)
XCTAssertEqual(false, install)
XCTAssertEqual(nil, selector)

do {
let _ = try extractProxyArguments(command: ["+1.2.3"])
XCTAssert(false)
} catch {}

do {
let _ = try extractProxyArguments(command: [])
XCTAssert(false)
} catch {}
}
}