0001-cortex-structure-rfc

[EiffelFly]

Summary

To sync between the cloud version app and the open-source app, we set up a monorepo to store necessary components, toolkits, and views across these two apps. It named is Cortex.

Motivation

We have dozen of duplicated views and components between the cloud version app and the open-source app. During the process of development, it becomes cumbersome to sync between these two repositories. We need to find a way to maintain the reusable components, views, and toolkits (helpers) across these two apps.

Detailed design

design-system, toolkit, and view

The cortex contains three main packages: design-system, toolkit, and view. In the dependency graph, the design-system and toolkit are independent but view will be dependent on both the design-system and toolkit.

• design-system: the basic level components in the instill-ai like Select, Button, Input, and more.
• toolkit: the tools to build the proper functionality of our user interface, like the SDK of the VDP API, the react-query wrapper, and the Zustand stores. It won't have any kind of components.
• view: combined the help of the design-system and the toolkit. The view will store the high-level components or views like ConfigureProfileForm or other resources' tables.

About the release flow

The release flow is constructed by google release-please and contains two steps:

1. The merge of the feature PR: once it is merged, the release-please action will be triggered, the release-please CLI will read through the config of our manifest, compares the difference between the HEAD, and then decide with packages need to bump up the version.
2. The merge of the release-please PR: once it merges, the release-please action will be triggered. The release-please action will update the tags and the releases of the Cortex. The JS-DevTools/npm-publish action will be triggered. If the version is the same as the newest version in the npm, the action will abandon the release procedure. If the version is new, this action will publish it.

About the usage

Normally, the cloud version app and the open-source app will use the design-system, toolkit, and view at the same time. But in some circumstances like chrome extension or other user interfaces, we can only use the design-system to build the UI and use the toolkit to access the VDP API.

Drawbacks

• At the first glance, a user may feel unnecessary to separate the single design-system into three packages.
• The code splitting part will be harder to optimize.
• The toolkit and view package's size will grow overtime

Alternatives

Submodules

In the cloud version app, we can use submodules to access the open-source app to sync between these two apps. But this lead to some greater drawbacks like

• It's hard to configure typescript, especially since we use path alias in both the cloud version app and the open-source app. This will lead to unnecessary configuration overhead.
• It's hard to version the version of the submodule. Every time the open-source console has changed, the cloud version needs to adapt. It may not be a problem right now but maybe a deployment overhead when we want to proceed with rolling updates in our production.

Caveats

• Because the release-please will add the PR title as the release note in the changelog. Please be careful of your PR title, try to be as specific as possible.

Unsolved questions

• How to better code splitting the packages?
• How to manage stories across view and design-system

[EiffelFly]

Here is my second thought:

During the development, I discovered that the develop and release of the toolkit and view are usually dependent. But right now they are separated into two independent package which disrupt the usage of the toolkit and view.

I think we should make them as single package.

In short, if the packages are being used and released altogether in almost every scenario, we should consider release and use them as single package

[EiffelFly]

If we combine toolkit and view into single package. We should set up the separate entry point for each child folder. Like

import {..} from "@instill-ai/toolkit/vdp-sdk"
import {..} from "@instill-ai/toolkit/view"