Most of the changes to Haystack, including bug fixes and small improvements, are implemented through the normal Pull Request workflow, according to our contribution guidelines.

Some changes, though, are "substantial", and these are the ones we want to put through a bit of a design process to make sure we're all on the same page before we invest the time into the actual implementation of a new feature or a deep refactoring.

We've introduced the "Proposals design process" to provide a consistent and controlled path for such changes to Haystack.

We will apply the same level of rigor to both core developers' and Community's proposals. The primary difference between them is in the design phase: core developers proposals tend to be submitted at the end of the design process whereas the Community ones tend to be submitted at the beginning, as a way to kickstart it.

Follow the process if you intend to make "substantial" changes to Haystack, rest_api or the process itself. What is defined as a "substantial" change is evolving based on community norms and on what part of the project you are proposing to change, but it may include the following:

• A new feature that creates new API surface areas.
• A new component (Nodes, Pipelines, Document Stores).
• Removing features that already shipped in the current minor version.
• A deep refactoring that would require new tests or introduce new dependencies.
• A change that's complex enough to require multiple steps to be delivered.

Some changes don't require a proposal, for example:

• Minor bug fixes.
• Rephrasing, reorganizing, or otherwise "changing shape does not change meaning".
• Addition and removal of warnings or other error messages.
• Additions only likely to be noticed by other contributors, invisible to Haystack users.

In any case, the core developers might politely ask you to submit a proposal before merging a new feature when they see fit.

Laying some groundwork ahead of the proposal can make the process smoother.

Although there is no single way to prepare for submitting a proposal, it is generally a good idea to collect feedback from other project developers first, to make sure that the change is is actually needed. As we're an open source community where everyone can impact the project, we all need to make an effort to build consensus.

When you're preparing for writing and submitting a proposal, talk the idea over on our official Discord server and in a Github issue or discussion in the Haystack repository.

To get a major feature added to Haystack, you first merge the proposal into the Haystack repo as a Markdown file. At that point, the proposal can be implemented and eventually included into the codebase.

There are several people involved in the process:

• Decision Driver: the person creating the proposal. If the Decision Driver is not a core contributor themselves, one will be assigned to the PR and will take care of facilitating the process.
• Input Givers: anybody reviewing or commenting the PR.
• Approvers: the core contributors approving the PR.

During its lifecycle, a proposal can transition between the following states:

• Review: proposal is getting feedback.
• Final Comment: proposal received approval from 3 core contributors; this state must be kept for a grace period of 3 calendar days.
• Active: proposal was approved and merged and can be implemented if not already.
• Stale: proposal didn't get any update in the last 30 days and will be closed after a grace period of 10 days.
• Rejected: proposal was actively rejected and the reasons explained.

To create a proposal:

1. Copy 0000-template.md to text/0000-my-feature.md, where 'my-feature' is a descriptive name of the feature you're proposing. Don't assign an identification number yet.
2. Fill in the proposal. Pay attention to details. Proposals that present convincing motivation, demonstrate an understanding of the feature impact, and honestly present the drawbacks and alternatives tend to be received well.
3. Submit a pull request. This ensures the document receives design feedback from a larger community, and as the Decision Driver, you should be prepared to revise it in response.
4. Rename the file using the PR number, for example from text/0000-my-feature.md to text/4242-my-feature.md. The proposal is now in Review state.
5. Reach an agreement with the Input Givers and integrate the feedback you got. Proposals that have broad support are much more likely to make progress than those that don't receive any comments.
6. Now it's time for the Approvers to decide whether the proposal is a candidate for inclusion in Haystack. Note that a review from the core contributors may take a long time, and getting early feedback from members of the Community can ease the process.
7. When the proposal enters the Final Comment state (see above), the PR will be marked accordingly, entering a grace period lasting 3 calendar days during which a proposal can be modified based on feedback from core developers or the Community. Big changes may trigger a new final comment period.
8. Approvers may reject a proposal once the public discussion and adding comments are over, adding the reason for rejection. A core developer then closes the related PR. The proposal gets the Rejected state.
9. When the final comment period ends, the PR is merged and the proposal becomes Active.

Once a proposal becomes active, the authors are free to implement it and submit the feature as one or more pull requests to the Haystack repo. Becoming 'active' is not a rubber stamp, and in particular still doesn't mean the feature will ultimately be merged; it does mean that the core team has agreed to it in principle and is open to merging it if the implementation reflects the contents of the proposal.

The fact that a given proposal has been accepted and is 'active' doesn't imply it has a priority assigned or somebody's currently working on it.

To change an active proposal, open follow-up PRs. Our goal is to write each proposal so that it reflects the final design of the feature, but the nature of the process means that we cannot expect every merged proposal to actually reflect what the end result will be at the time of the next release. That's why we try to keep each proposal document somewhat in sync with the feature as planned, tracking such changes through follow-up pull requests to the document.

As the author of a proposal, you're not obligated to implement it. Of course, the author (like any other developer) is welcome to post an implementation for review after the proposal has been accepted.

Haystack's proposals design process process owes its inspiration to the React and Rust RFC processes. We're open to changing it if needed.