tuist library proposal #192
Comments
This is exactly what motivated me to build Tuist. Back when I was working at SoundCloud (@RomainBoulay can tell you more about this) we modularized the codebase. As a result, we got more than 10 Xcode projects, with a similar structure with the only difference of having different link settings, sources, and resources. I think reached that point, maintaining those projects is a huge burden, and a declarative interface facilitate things a lot. I agree with you that Buck and Bazel is too much to tackle this problem. It'd be nice if they'd provide just the declarative layer but without replacing the build system. Unfortunately, it does.
I'm glad that you found the workflow promising to explore further. I wish I had more free time to add some of the workflows and ideas that I had in mind. Most of them help with the common pain areas that developers need to go through when building iOS apps, like dealing with signing, publishing of new versions to the store. I'm also happy that you are giving Tuist and challenging the assumptions that we made. Happy to iterate the tool with all that feedback.
This is a cost that we are willing to take if we find the right setup for Tuist that helps other projects codify their conventions and use cases into their own tools. As long as the refactoring of the core doesn't add any friction to Tuist as a command line tool, it's all fine.
XcodeGen is wonderful alternative that you might consider. They took a different approach where the manifest format is more aligned with the Xcode project's. Moreover, they added some useful features on top of the format that facilitate defining and reusing the projects. They sticked to yaml and json as formats, which we deprecated because we plan to add features that leverage the Swift compiler. With the refactor of the core though, we could decouple the manifest models from the definition source, so anyone could implement a parser for their own format. |
|
From your proposal, I drawed the diagram below:
I think we could have modules with strong conventions such as a the CLI, the support module (i.e the current I think structuring the project like that is feasible. The challenge is going to be designing the API for those modules with weak conventions to be flexible enough for Tuist and other third-party projects. For example, having the up section in the manifest is something that is specific to Tuist but that needs to be part of the manifest model. What do you think @kwridan ? |
|
Answering to the alternatives section:
I'd rather not do it. I've seen the Buck project being forked by Uber and never merged back. It also makes developers more confusing because there'd be two development threads diverging overtime.
You described the downsides very well. There'd a lot of duplicated logic with this project. When I embarked on building Tuist, I considered reusing XcodeGen's generation logic, but given that I was very proficient on xcodeproj, I decided not to introce another indirection layer and rather use xcodeproj directly. |
|
Thanks for your quick response!
Awesome, glad you're open to this!
I actually gave
This is great! It's very close to what we had in mind!
Agreed, this will be the most challenging part.
The up command is an interesting case, I'm curious why it's defined at the project level as opposed to the workspace level, or perhaps something even higher level (e.g. Will explore this further with a few of my colleagues and post back some suggestions on the API for us to iterate on. |
|
@pepibumur here are some suggestions I've been exploring with @marciniwanicki - we'd be keen on your thoughts / feedback 🙂 - thanks for taking the time to hear us out! ApproachA change like this will likely cause a lot of code to move around, as such to help minimize impact we can consider the following:
This will help minimize the changes required for review and allow us to verify that the generated projects are identical before and after the changes. Additionally, it will (hopefully) make the iteration process on the API easier as we'd have a working base! We can start a feature branch Initially on the feature branch we can consider keeping the existing LibraryAny thoughts on the name, The two commands that would use the
To distinguish them from their APIEntry Pointtry tuistGenerator.generate(basePath: path, at: destination)public class TuistGenerator {
private let graphLoader: GraphLoading
private let graphModelLoader: GraphModelLoading
private let workspaceGenerator: WorkspaceGenerating
public func generate(basePath: AbsolutePath) throws {
// ..
}
public func graph(basePath: AbsolutePath) throws -> Graphing {
// ...
}
}The client code would need to implement public protocol GraphModelLoading {
func loadProject(at path: AbsolutePath) throws -> Project
func loadWorkspace(at path: AbsolutePath) throws -> Workspace
func manifests(at path: AbsolutePath) -> Set<Manifest>
func manifestPath(at path: AbsolutePath, manifest: Manifest) throws -> AbsolutePath
}Where What is your preference for referring to the models here? "graph model" or "manifest model" - perhaps choosing another name to disambiguate between those and the ClassesCandidates for the library: Manifest/Graph ModelsWhile the goal initially isn't to introduce any new features to the exisiting models, some changes are required:
For this, a new Testing
UpBased on the example you provided regarding
Currently (in Option 1) Mantain 2 graphs
Option 2) Abstract the graph modelsAnother option is to introduce some abstractions around the graph models to allow having different implementation that contain additional properties class TuistProject: Project {
// Project
var name: String
//...
// Additional properties
var up: [Up]
}The graph would need to be exposed to make this technique work and some casting may be involved, additionally would require all clients to provide their own types or subclass. Option 3) Use a single graph from the library but extend it to store custom data (metadata)Setting up and maintaining two graphs (one in the generator and second in the client code - i.e. TuistKit) can introduce quite a lot of boilerplate. We can expose public protocol GraphModelLoading {
associatedtype Metadata
func loadProject(at path: AbsolutePath) throws -> Project
func loadWorkspace(at path: AbsolutePath) throws -> Workspace
func manifests(at path: AbsolutePath) -> Set<Manifest>
func manifestPath(at path: AbsolutePath, manifest: Manifest) throws -> AbsolutePath
func loadMetadata(at path: AbsolutePath) throws -> Metadata
}Requires further investigation / prototyping but is something that could work. One downside could be the complexity generics add to the classes within the library. Option 4) Model ExtensionsAnother options is to store some additional metadata into the library models that can later be decoded. // TuistGenerator
class Project {
var metadata: String?
}
// TuistKit
extension Project {
var up: [Up] {
// decode metadata
}
}This mitigates the generics complexity, a downside would be the need to decode data twice, once to form the graph in the first place, and then again for the metadata. Option 5) (Initially) include it in the libraryTo minimize impact, perhaps initially have SummaryPerhaps a hybrid of the options above would be possible. Do you have any further thoughts or ideas how to split the responsibilities of the modules and how to integrate them? Initially (even when it's landed on master) we could state |
|
Thanks @kwridan and @marciniwanicki for elaborating a proposal for introducing those changes. Here's my feedback on your proposal:
That plan sounds good to me. What I suggest before doing any change is adding some integration tests that given a project manifest that contains all supported options, it generates the expected Xcode project. I've opened an issue to tackle that.
I'd avoid having a branch that we merge new branches into. Instead, I'd find the way to make incremental changes that keep master shippable all the time. With those tests in place, which will bring the necessary confidence to introduce changes, I think it's ok to merge directly into master.
I'm actually thinking we can get rid of
I like the idea of the func manifests(at path: AbsolutePath) -> Set<Manifest>
func manifestPath(at path: AbsolutePath, manifest: Manifest) throws -> AbsolutePath
You are right regarding the ambiguity. I'd keep the names in the
Looks good 👍. What about using another name for
Do you mind if I add a 5th option that simplifies things a lot 😜? What about extracting the let setup = Setup([
.carthage(...)
] That way,
The approach looks good to me. Only one more thing. What do you think if modify the API that you suggested to support a closure that gets called before each project gets written? public func generate(basePath: AbsolutePath, beforeWrite: (XcodeProj, Path) -> Void) throwsThat way consumers of the API could hook into the process and introduce any desired tweaks into the project.
We should definitively state that. We'll need some internal iterations until we reach an API we are happy with for both internal and external usages. |
|
Feel free to create issues based on the feedback @kwridan. Feel free to drive the work and let me know if you need any help before making a decision. I'm very excited about this happening and seeing Tuist supporting your project's needs. |
|
Thanks for the detailed feedback, it is much appreciated!
makes a lot sense! 👍
Agreed
Funny story, while testing we accidentally added We found some related issues for Swift PM:
func manifests(at path: AbsolutePath) -> Set<Manifest>
func manifestPath(at path: AbsolutePath, manifest: Manifest) throws -> AbsolutePathoops! too much copy and paste during our prototyping :) you are right these won't be needed.
I was mostly referring to how we should refer to these models for discussions and documentation. I was confusing which of these are manifests (I assume the classes in Technically we’d be able to support having a func platform(from description: ProjectDescription.Platform) -> TuistGenerator.Platform {
switch description {
case .iOS: return .iOS
}
}This would only exist in a thin layer (the model loader) that translates between both and wouldn’t spread further. I guess it will be easier to iterate on once we get started.
This is the best one 👍 let's go with that. We weren't sure about the requirements of up and if it needed to remain in the graph.
Yes this is a great suggestion, this would allow a lot of flexibility for clients of the API. We could introduce it after the initial version as we’ll no doubt need it.
🙌
Awesome! Here’s a summary of follow ups:
We’re excited to get started on this! We’ll kick off over the next week or so. |
😅, let's keep the
I usually refer to manifest models the ones in the Let's keep their names in the frameworks. |
I'm also excited about this 🎉. Please, add me as a reviewer on those PRs, and I'll gladly give you feedback and help you get those changes merged into the project. |
|
👍 thanks @pepibumur! - will close this proposal. |
Introduce new TuistGenerator module (based on the proposal tuist#192). a# modified: Sources/TuistKit/Models/UpCustom.swift
As promised, following up from the feedback in #172 re: the library suggestion.
Overview
For one of the (work) projects I’ve been working on that has several Xcode projects, we’ve reached a stage where we can no longer sustainable manage Xcode projects manually, however aren’t yet at a scale to adopt a whole new different build system and toolchain (like
buckandbazel) just yet. As such we’ve been experimenting with a few different tools that can assist us with Xcode project generation.tuisthas the most promising workflow which we’d like to explore further. However having a large project and team introduces some interesting cases that need further consideration. Some of which were highlighted in my earlier feedback, which I’m glad to see was well received and many of the items made it for consideration and some have already been resolved!Some of those items (like custom definitions, conventions) may be only useful for our use cases, and would perhaps be harmful to try and push them onto
tuist. As such the idea of somehow leveragingtuistas a library is quite appealing.In theory that would allow
tuistto continue enforcing its conventions without compromise, whilst at the same time allowing 3rd parties to leveragetuistas a library to support their use cases and conventions. Of course, this will come at a cost, it would complicate the internals oftuistand as you rightfully highlighted, would introduce an additional support burden. However this is something we'd be happy to contribute towards.Actually, if I’ve done my research right,
XcodeGenis another potential candidate for such a library. It seems there’s a long standing feature request to support multiple projects and workspaces, somethingtuistalready supports. Additionally, I seetuistis deprecatingjsonandyamlsupport, as such this does create an opportunity forXcodeGento continue support for those formats.Proposal
tuistcombines the following functionality:tuistfeaturesThe proposal, is to separate out some of those to their own libraries within the same swift package (at least initially until they are proven).
TuistKitSwiftFileInterpreterto obtainProjectDescriptionProjectDescriptionTuistGentuistlogic like theUpcommand supportTuistGen(naming is hard 😛)XcodeGen)SwiftFileInterpreterswiftccalls on a given file and returns the outputSplitting is possibly the easy part, the harder part would be figuring out the API
TuistGenwould need to expose to cater fortuistand other 3rd parties. Can post some more on this if there’s interest.Alternatives
For 3rd party developers with a customisation need, there are a few options:
tuistand apply customisationsxcodeprojdirectlyOption 1) provides quick wins short term, but has long term drawbacks. Developers would soon lose the ability to contribute back any general improvements and vice versa pull in any improvements made upstream.
Option 2) is probably the most flexible and powerful, but realistically, some if not most of the generator logic of
tuistwould be replicated to some degree. Additionally like option 1, contributions would be hard to make upstream and it would be a shame to have a divergent yet similar tool out there.The text was updated successfully, but these errors were encountered: