Maintaining core extensions separately

[qiangxue]

In order to enable faster version releases and make core extensions more scalable, we are thinking to maintain each core extension and application template separately in individual github projects. Below is a preliminary plan. Before we execute it, we would like to have it reviewed by everyone to make sure we do not omit anything important. Thank you!

Tasks

• stop automatic subsplit
• delete /extensions and /apps from yii2 repo
• move/adjust unit tests and documentation specific for each extension to the corresponding extension repo
• adjust yii2-dev to keep BC (solved by dev command)
• adjust build scripts to support major/minor releases
• adjust apidoc tool for generating api/guide docs for extensions
• enhance build scripts to support automatic patch releases
• adjust travis configs. Each extension or app will need its own travis setup
• adjust internal docs
  • update https://github.com/yiisoft/yii2/blob/master/docs/internals/versions.md
  • adjust git workflow for extensions https://github.com/yiisoft/yii2/blob/master/docs/internals/git-workflow.md
  • adjust getting started https://github.com/yiisoft/yii2/blob/master/docs/internals/getting-started.md

Project Organization

1. Maintain individual core extensions and application templates in separate independent GitHub projects. Each extension/app project will
   • keep using its current project name to ensure BC. For example, the yii2-gii extension will be maintained in yiisoft/yii2-gii project.
   • maintain its tutorial doc in its "docs" folder. The API doc will be generated on-the-fly when the extension/app is being released.
   • maintain its own test code in its "tests" folder.
   • maintain its own message translations and all other relevant meta code.
   • track issues via the corresponding GitHub project.
   • be released independently as needed. See versioning policy below.
2. Continue using "yii2" as the main project to maintain the core framework. "yii2" should contain the following stuff
   • the core framework code
   • the definitive guide. Extension-specific guide should be put in the corresponding extension projects. API doc should be generated on-the-fly when releasing the core framework.
   • the test code for the core framework only
   • build tools and internal docs
3. Continue doing automatic subsplit from "yii2" to "yii2-framework" to ensure BC.
4. Track the core framework issues in the "yii2" project. Extension issues should be reported to the corresponding extension projects.

Release and Versioning Policy

1. Version numbers are in the format of 2.x.y.z.
   • 2.x: major releases, containing major features. May be BC-breaking. Release cycle is around 6 months. With news announcements. Project site will be updated.
   • 2.x.y: minor releases, containing minor features and bug fixes. Must be BC with prior 2.x.* releases. Release cycle is around 1 to 2 months. With news announcements. Project site will be updated.
   • 2.x.y.z: patch releases, containing bug fixes only. Must be BC with prior 2.x.*.* releases. Release cycle is around 1 to 2 weeks. No news announcement or project site update (unless it contains major/security issue fixes). The release process will be made mostly automatic.
2. Branching policy
   • master branch is the dev branch for the current major release. Once a new minor release is ready, create a maintenance branch named 2.x.y off master.
   • Create tags 2.x.y.z on 2.x.y branch to mark patch releases (including 2.x.y.0 release).
   • Changes on 2.x.y will be merged back to master constantly.
   • For the next major release, create a dev branch named 2.t-dev. Once the next major release is ready, it will be merged to master, and then create a 2.t.0 branch off master (the old 2.t-dev branch will be deleted to avoid confusion).
3. yii2 and core extension/application projects are released independently of each other.
   • Both yii2 and core extension projects follow the above versioning and branching policies.
   • The release cycles described above only apply to the core framework.
   • Extensions and application templates are released on demand. It is likely that an extension has no new releases for a very long time because it does not receive any bug fixes or enhancements.
   • Extensions and application templates may have version numbers that are different from that of the core framework. For example, yii2-gii may have the latest version number as 2.0.5 while yii2 is already at 2.1.3, even though currently they are all synchronized at 2.0.3.

[Ragazzo]

should be done already long time ago :)

[mtangoo]

"API doc should be generated on-the-fly when releasing the core framework"
How does this affect us who were generating Documentations locally? Will it need extra gym or it will just work?
I can't see any problem to the proposed plan so far!

[nicklasos]

+1

[i-sevostyanov]

good idea

[nineinchnick]

So it's only about versioning? I mostly care about being able to clone everything at once to hack it.

Looks good to me.

[lynicidn]

👍

[Ryadnov]

👍

[farmani]

it should be done earlier but its nice to hear you and I believe that some new developer can join to developers team to focus on special extension and so new idea brings up by focusing on some specific part of a big project

[SDKiller]

Remove bower-asset/yii2-pjax from require section of framework composer.json and move framework/widgets/PjaxAsset.php and framework/widgets/Pjax.php to separate extension like bootstrap and jui.

Reasons:

• it is not mandatory for framework functionality
• someone may not even need it
• someone may want to use another push state implementation
• it depends much on upstream and may need instant fixes like gridview pjax search stopped working #7629
• each bower-asset adds overhead on composer update

But now it is even not possible to avoid bower-asset/yii2-pjax installation as it is required by framework's composer.json

[harry-flipkod]

+1

[ghost]

+1

[zmilan]

Good one 👍

[creocoder]

@SDKiller

Remove bower-asset/yii2-pjax from require section of framework composer.json and move framework/widgets/PjaxAsset.php and framework/widgets/Pjax.php to separate extension like bootstrap and jui.

👍 But... its not enough, all JS stuff should be removed to extensions include client side from validators, etc. But yes, currently main pain is fxp/composer-asset-plugin. And there is nothing in first post about that. So in general everything looks like simple cosmetic changes which will solve only small portion of troubles.

[voskobovich]

+1

[lichunqiang]

+1 for @creocoder

[samdark]

@SDKiller, @creocoder while I understand why you want to avoid using fxp/composer-asset-plugin, it can't be easily done in 2.0.x since it breaks backwards compatibility significantly.

[tecnologiaterabyte]

Excellent allow deploy fixes and enhancements fastest way. 👍

[adamaltman]

This proposal sounds good at first, from a code/test organization perspective. But if the goal is to really speed up releases, this only works if this structure empowers more people to contribute in more meaningful ways than currently possible, including reviewing and accepting pull requests.

Also, I wonder is it possible that core changes break some extension compatibility? (As I understand the core may not depend on an extension, but an extension may depend on the core. Unless I'm wrong there.) And who is responsible in that scenario for regaining compatibility? And how would they be aware? What would be the behavior of the CI for a pull request? Does it mean that when someone changes an extension that relies on a particular core api, that they must also submit tests to the core to ensure that api persists in future versions of the core?

[dynasource]

Good decision. Eventually it had to be done, so starting now is wize.

With respect to external parts like Pjax. I advocate to put these in separate extensions too because of either limitations / or not using it / or the use of alternative packages like i.e Intercooler.

About the fxp/composer-asset-plugin. This significant limitation on proper behavior of composer is also something that deserves attention on short term. If I could choose one thing to do first, it would be this.

[samdark]

@adamaltman the current issues are

1. git subsplit from main repo to read only copies. It is extremely slow.
2. we can't release extensions independently.

Proposed change solves it.

Compatibility between extension / framework versions could be achieved if extension will specify framework version in its requirements. We're going to do it as well.

CI will work for extensions i.e. extension will be tested with framework version specified in its composer.json. It's nowadays rarely required to change core code in order to fix something in extensions afterwards. If it's the case, extension release has to wait for framework release.

[samdark]

@dynasource I'm afraid it could not be done in 2.0.x... Or probably I don't see how to do it w/o breaking changes. Anyway, it's a separate issue and it's better to discuss it separately.

[adamaltman]

Yes, @samdark for issues 1 and 2, it's good. 👍

[qiangxue]

@SDKiller Thanks for pointing out the issue with pjax. I think we handle this in a separate issue because it involves breaking BC. In this issue, we will focus on resolving this subsplit issue and make sure no BC is broken.

The following changes will need to be made for this issue:

• stop automatic subsplit
• move/adjust unit tests and documentation specific for each extension to the corresponding extension repo
• adjust yii2-dev to keep BC
• adjust build scripts to support major/minor releases
• enhance build scripts to support automatic patch releases
• what else?

[samdark]

1. Decide on how to build/display docs for individual extensions.
2. Decide on how to build/display API docs for extensions.

[lynicidn]

@qiangxue minimizing dependencies

[samdark]

@lynicidn that's separate issue.

[lynicidn]

how u want check requirments? in all extensions put requirments.php ?
https://github.com/yiisoft/yii2/blob/master/apps/advanced/requirements.php#L124

[qiangxue]

I think requirement checker should be made in application templates only. Or perhaps we should simply rely on composer dependency to check them.

[lynicidn]

may be need add interface for checker and check before put in vendor/autoload.php ?
If i don't use mailer or imagine - why i should have it in template ? Logicaly check it before configure

[amnah]

I installed yii2-app-basic using composer create-project, but it excluded the .gitignore file (I assume because of the newly added .gitattributes file)

Is that supposed to happen? I found it weird when I noticed that my vendor dir got committed.

[samdark]

Right. .gitignore should not be in .gitattributes... At least for apps.

[samdark]

Fixed.

[pana1990]

i think that Translation status should should take into due consideration status docs from extensions.

what do you think about it?

[samdark]

What do you mean?

[pana1990]

here : https://github.com/yiisoft/yii2/blob/master/docs/internals/translation-status.md

It should have the status docs from extensions.

sorry for my english.

[samdark]

Yes. Should be updated...

[dynasource]

is it possible to further minimize yii's composer.json, ie. by making composer libraries such as cebe/markdown as suggestion instead of a requirement?

[samdark]

@dynasource only by throwing out Markdown and HtmlPurifier helpers, MaskedInput etc.

[RusAlex]

My 2 cents here. Yes it would be great to have packages not only as separate repositories, but in future as separate components which can be used without core. At moment, when your project based on Yii 1.x , and when it's big enough, you wont have time to re-write whole project at once. And this situation makes me more careful in selecting packages I will use. One main ides now is to start replacing app parts which are depends on Yii 1.x with stand alone components where it could. And leave Yii only several global responsibilities: Handle request, render result. I see this the only way possible in projects with big code base grow and being rewriten seamlesly not in a "full rewrite" way. And I hope Yii will get rid of this problem in future. Thanks

[creocoder]

@RusAlex

My 2 cents here. Yes it would be great to have packages not only as
separate repositories, but in future as separate components which can
be used without core.

As I understand it completely contradicts the philosophy of Yii. This framework of tightly coupled components with lots of conscious violation of many OOP principles (like SOLID) for practice reasons. If short independent components from Yii will not be interested to anyone who build systems with independent components. Yii is interesting as full stack framework and only as full stack.

And I hope Yii will get rid of this problem in future.

I bet that this never will happen.

[RusAlex]

Anyway I could use it as I wrote for Handling Request and Response. Then I will be free in selecting other App components. Yii allows it.

[creocoder]

@RusAlex

Anyway I could use it as I wrote for Handling Request and
Response.

I could do so to, but never will do that. Just check http:https://stackphp.com/ . If you want free set of components you usually want microframework. Like Silex. Yii is full stack. Its different. And you know, choosing Yii for handling Request and Response only is really not smart option because there is much more better tools for that. Every tool have it niche.

[RusAlex]

While there are good built in features to make simple and nice CRUDs fast it will make sense =) Agree Yii is a tool and it must be used wisely as any other.

[vbsessa]

+1
Actually, i'm really looking forward the changes proposed in the wiki, titled "ideas for 2.1". They seem nice and ambicious.

[samdark]

@cebe anything left for this one?

[cebe]

@samdark automation of releases. I'd like to keep this open until done.

[cebe]

also docs are not 100% fine. E.g. guide for extension is not rendered anywhere.

[cebe]

closing this now. extension docs generation will be part of the new website.

[cebe]

docs about project structure, versioning etc are now in https://github.com/yiisoft/yii2/blob/master/docs/internals/README.md