🚀 Feature: Reproducible Projects

[ZeroAurora]

🔖 Feature description

Projects could be managed as simple as managing codes, just like Terraform. By achiving "reproducible", open-source project owner can feel easier to put their backend projects online.

🎤 Pitch

Current CLI is not reproducible, i.e. can not copy-and-paste from dev to production and work. It must be configured over Web UI. This is not that convenient when deploying self-hostable (especially open-source) applications.
So my idea is:

• Provide a CLI command to "export" and "import" all of the project, with schemas and settings, and without data.
• Provide a manual for interested developers to work with. For example, naming project/collections/buckets/... with provided names might be more "portable" than randomly generated IDs.

👀 Have you spent some time to check if this issue has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

[stnguyen90]

Current CLI is not reproducible, i.e. can not copy-and-paste from dev to production and work. It must be configured over Web UI.

Can you clarify what can't be configured? I know a project can't be created via the CLI. From your experience, what else is missing exactly?

[ZeroAurora]

Can you clarify what can't be configured? I know a project can't be created via the CLI. From your experience, what else is missing exactly?

1. Turning on and off modules (cloud functions, users, database...) and the functions of each module (auth methods...).
2. Specify IDs when creating projects, functions, collections, etc in CLI (AFAIK they can be configured in appwrite.json, but it needs recreating one with the same ID in WebUI)
3. Changes in the WebUI won't be synchronized with local projects.

[stnguyen90]

1. Turning on and off modules (cloud functions, users, database...) and the functions of each module (auth methods...).

You can't set this in the appwrite.json, but it should be doable via a CLI command: https://github.com/appwrite/sdk-for-cli/blob/8fe40a8c7471ab783c8bbabbda9e8d8744324d47/lib/commands/projects.js#L1001.

2. Specify IDs when creating projects, functions, collections, etc in CLI (AFAIK they can be configured in appwrite.json, but it needs recreating one with the same ID in WebUI)

Would PR appwrite/sdk-generator#484 cover this?

3. Changes in the WebUI won't be synchronized with local projects.

You don't want the appwrite.json to update after updating something in the Appwrite Console, right?

[stnguyen90]

This seems like it may be related to #491

[ZeroAurora]

Would PR appwrite/sdk-generator#484 cover this?

Thanks! I may test it some day.

You don't want the appwrite.json to update after updating something in the Appwrite Console, right?

Well, maybe just allowing to export an appwrite.json in the UI/CLI is enough.

[stnguyen90]

• Fix: CLI collection init&deploy (databases) sdk-generator#561
• Feat: CLI buckets init&deploy sdk-generator#562
• Feat: CLI teams init&deploy sdk-generator#563
• Feat: CLI settings init&deploy sdk-generator#564

[johannesschobel]

i think it would be benefitial to have a "single source of truth" (i.e., the appwrite.json file) that contains all relevant information to copy one project from one server to another one. That should, for example, also include the database names, and so on (currently not available)

[zdanl]

as per mention, there are many plugins, "packaged projects" as @stnguyen90 called them, clearly with a need to be reproducible, some of which appwrite or the authors potentially can monetize on, even. are there any plans in that direction, Appwrite Premium, or that kind of appstore / plugin store?

off the top of my head, i have the following roadmap with my project:

• integrate web3 connect (non-custodial, ethers.js, ethereum) with appwrite sdk; notify the backend when a client-side wallet connection was established
• custodial wallets (this could be a plugin, and since there may be more than 1 wallet per Account, it may be better off packaged with a serverless function and a database collection)
• stronger and more specific support for oauth providers, especially github, which allows to interface with Repos in a way that DigitalOcean, Vercel, etc. display very well
• a generic form of "projects" in a way that "teams" generalizes the "role" model of authentication systems. almost every website provides some sort of a workspace, either paid (github codespaces) or unpaid (basic organizational domains) that are named and relational to the items they contain / are described by (tags, description) in the most basic form, with user memberships in a more advanced (Github Projects, Github repos).

[bhagn]

I'm not sure of the current progress on this issue. We have been using Appwrite for some time now and we had this issue of (re)creating dev. environments when a new developer joins, or someone wishes to setup the project from scratch. There is considerable amount of time that's spent in just setting up the project, API keys, OAuth etc; The latest CLI does help with syncing the collections and functions but doesn't go beyond that.

We ended up creating our own CLI written on top of the Appwrite APIs to manage pretty much every resource Appwrite provides. But the most important feature being able to create a snapshot of the entire config and restoring/syncing it. This has helped us to reproduce the same env. on different machines very quickly - ~60s :)

For those of you interested in giving it a try check out
Pypi: https://pypi.org/project/appw/

Github: https://github.com/TheGallliNetwork/appwrite-cli

This snapshots are just json files that are created in the directory where you run the command. It tries not to store any of the API keys that get generated in your system, but uses placeholders and re-creates them on newer systems. The only key that gets stored in the snapshot (in plain text) are the auth keys (Oauth) - so please make sure you are aware of this and don't check-in to public repositories.