Skip to content

Commit

Permalink
This closes apache/beam-site#217
Browse files Browse the repository at this point in the history
  • Loading branch information
davorbonaci committed Apr 19, 2017
2 parents 542892a + 32044ff commit f63eeed
Show file tree
Hide file tree
Showing 13 changed files with 24 additions and 24 deletions.
4 changes: 2 additions & 2 deletions website/src/_posts/2016-03-17-capability-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -579,9 +579,9 @@ With initial code drops complete ([Dataflow SDK and Runner](https://github.com/a

While we’d love to have a world where all runners support the full suite of semantics included in the Beam Model (formerly referred to as the [Dataflow Model](http:https://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf)), practically speaking, there will always be certain features that some runners can’t provide. For example, a Hadoop-based runner would be inherently batch-based and may be unable to (easily) implement support for unbounded collections. However, that doesn’t prevent it from being extremely useful for a large set of uses. In other cases, the implementations provided by one runner may have slightly different semantics that those provided by another (e.g. even though the current suite of runners all support exactly-once delivery guarantees, an [Apache Samza](http:https://samza.apache.org/) runner, which would be a welcome addition, would currently only support at-least-once).

To help clarify things, we’ve been working on enumerating the key features of the Beam model in a [capability matrix]({{ site.baseurl }}/learn/runners/capability-matrix/) for all existing runners, categorized around the four key questions addressed by the model: <span class="wwwh-what-dark">What</span> / <span class="wwwh-where-dark">Where</span> / <span class="wwwh-when-dark">When</span> / <span class="wwwh-how-dark">How</span> (if you’re not familiar with those questions, you might want to read through [Streaming 102](http:https://oreilly.com/ideas/the-world-beyond-batch-streaming-102) for an overview). This table will be maintained over time as the model evolves, our understanding grows, and runners are created or features added.
To help clarify things, we’ve been working on enumerating the key features of the Beam model in a [capability matrix]({{ site.baseurl }}/documentation/runners/capability-matrix/) for all existing runners, categorized around the four key questions addressed by the model: <span class="wwwh-what-dark">What</span> / <span class="wwwh-where-dark">Where</span> / <span class="wwwh-when-dark">When</span> / <span class="wwwh-how-dark">How</span> (if you’re not familiar with those questions, you might want to read through [Streaming 102](http:https://oreilly.com/ideas/the-world-beyond-batch-streaming-102) for an overview). This table will be maintained over time as the model evolves, our understanding grows, and runners are created or features added.

Included below is a summary snapshot of our current understanding of the capabilities of the existing runners (see the [live version]({{ site.baseurl }}/learn/runners/capability-matrix/) for full details, descriptions, and Jira links); since integration is still under way, the system as whole isn’t yet in a completely stable, usable state. But that should be changing in the near future, and we’ll be updating loud and clear on this blog when the first supported Beam 1.0 release happens.
Included below is a summary snapshot of our current understanding of the capabilities of the existing runners (see the [live version]({{ site.baseurl }}/documentation/runners/capability-matrix/) for full details, descriptions, and Jira links); since integration is still under way, the system as whole isn’t yet in a completely stable, usable state. But that should be changing in the near future, and we’ll be updating loud and clear on this blog when the first supported Beam 1.0 release happens.

In the meantime, these tables should help clarify where we expect to be in the very near term, and help guide expectations about what existing runners are capable of, and what features runner implementers will be tackling next.

Expand Down
4 changes: 2 additions & 2 deletions website/src/_posts/2016-04-03-presentation-materials.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ authors:
- takidau
---

Are you interested in giving a presentation about Apache Beam? Perhaps you want to talk about Apache Beam at a local Meetup or a convention. Excellent! The Apache Beam community is excited to expand and grow the community. To help kickstart this process, we are excited to announce an initial set of [Apache Beam presentation materials]({{ site.baseurl }}/learn/presentation-materials/) which anyone can use to give a presentation about Apache Beam.
Are you interested in giving a presentation about Apache Beam? Perhaps you want to talk about Apache Beam at a local Meetup or a convention. Excellent! The Apache Beam community is excited to expand and grow the community. To help kickstart this process, we are excited to announce an initial set of [Apache Beam presentation materials]({{ site.baseurl }}/contribute/presentation-materials/) which anyone can use to give a presentation about Apache Beam.

<!--more-->

Expand All @@ -21,4 +21,4 @@ As a community, we want to build a shared collection of high quality presentatio
* Using Apache Beam with runners
* [Google Cloud Dataflow](https://goo.gl/2ay8mi)

As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the [Apache Beam presentation materials]({{ site.baseurl }}/learn/presentation-materials/) page or email the [`[email protected]`](mailto:[email protected]) mailing list with your ideas or questions.
As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the [Apache Beam presentation materials]({{ site.baseurl }}/contribute/presentation-materials/) page or email the [`[email protected]`](mailto:[email protected]) mailing list with your ideas or questions.
4 changes: 2 additions & 2 deletions website/src/_posts/2016-06-15-first-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ making them readily available for our users. The initial release includes the
SDK for Java, along with three runners: Apache Flink, Apache Spark and Google
Cloud Dataflow, a fully-managed cloud service. The release is available both
in the [Maven Central Repository](http:https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.beam%22),
as well as a download from the [project’s website]({{ site.baseurl }}/use/releases/).
as well as a download from the [project’s website]({{ site.baseurl }}/get-started/downloads/).

The goal of this release was process-oriented. In particular, the Beam
community wanted to release existing functionality to our users, build and
Expand All @@ -35,5 +35,5 @@ anticipated, perhaps one every 1-2 months.
As always, the Beam community welcomes feedback. Stabilization, usability and
the developer experience will be our focus for the next several months. If you
have any comments or discover any issues, I’d like to invite you to reach out
to us via [user’s mailing list]({{ site.baseurl }}/use/mailing-lists/) or the
to us via [user’s mailing list]({{ site.baseurl }}/get-started/support/) or the
[Apache JIRA issue tracker](https://issues.apache.org/jira/browse/BEAM/).
2 changes: 1 addition & 1 deletion website/src/_posts/2016-08-03-six-months.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,5 +39,5 @@ But perhaps most importantly, we’re committed to building an involved, welcomi
* Had multiple talks on Beam at venues including ApacheCon, Hadoop Summit, Kafka Summit, JBCN Barcelona, and Strata.
* Presented at multiple existing meetups and are starting to organize some of our own.

While it’s nice to reflect back on all we’ve done, we’re working full _stream_ ahead towards a stable release and graduation from incubator. And we’d love your help -- join the [mailing lists]({{ site.baseurl }}/use/mailing-lists/), check out the [contribution guide]({{ site.baseurl }}/contribute/contribution-guide/), and grab a [starter task](https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)) from Jira!
While it’s nice to reflect back on all we’ve done, we’re working full _stream_ ahead towards a stable release and graduation from incubator. And we’d love your help -- join the [mailing lists]({{ site.baseurl }}/get-started/support/), check out the [contribution guide]({{ site.baseurl }}/contribute/contribution-guide/), and grab a [starter task](https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)) from Jira!

Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,5 @@ On management and thought leader side, Beam went from “what’s Beam?” at pr

We rounded out the tutorial with live demonstrations of Beam running on Apache Spark, Apache Flink, the local runner, and DataFlow runner. Then, we brought in the big brainpower and had a Q and A session.

If you’re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we’re happy to help you. In addition to this tutorial, we have [other presentation materials]({{ site.baseurl }}/learn/presentation-materials/). You can reach out to us on the [user mailing list]({{ site.baseurl }}/use/mailing-lists/).
If you’re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we’re happy to help you. In addition to this tutorial, we have [other presentation materials]({{ site.baseurl }}/contribute/presentation-materials/). You can reach out to us on the [user mailing list]({{ site.baseurl }}/get-started/support/).

2 changes: 1 addition & 1 deletion website/src/_posts/2016-10-20-test-stream.md
Original file line number Diff line number Diff line change
Expand Up @@ -306,4 +306,4 @@ The addition of TestStream alongside window and pane-specific matchers in PAsser
has enabled the testing of Pipelines which produce speculative and late panes.
This permits tests for all styles of pipeline to be expressed directly within the
Java SDK. If you have questions or comments, we’d love to hear them on the
[mailing lists]({{ site.baseurl }}/use/mailing-lists/).
[mailing lists]({{ site.baseurl }}/get-started/support/).
10 changes: 5 additions & 5 deletions website/src/contribute/contribution-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ We use a review-then-commit workflow in Beam for all contributions.

**For larger contributions or those that affect multiple components:**

1. **Engage**: We encourage you to work with the Beam community on the [Apache JIRA issue tracker](https://issues.apache.org/jira/browse/BEAM) and [developer’s mailing list]({{ site.baseurl }}/use/mailing-lists/) to identify good areas for contribution.
1. **Engage**: We encourage you to work with the Beam community on the [Apache JIRA issue tracker](https://issues.apache.org/jira/browse/BEAM) and [developer’s mailing list]({{ site.baseurl }}/get-started/support/) to identify good areas for contribution.
1. **Design:** More complicated contributions will likely benefit from some early discussion in order to scope and design them well.

**For all contributions:**
Expand All @@ -34,7 +34,7 @@ We look forward to working with you!
### Mailing list(s)
We discuss design and implementation issues on the [email protected] mailing list, which is archived [here](https://lists.apache.org/[email protected]). Join by emailing [`[email protected]`](mailto:[email protected]).

If interested, you can also join the other [mailing lists]({{ site.baseurl }}/use/mailing-lists/).
If interested, you can also join the other [mailing lists]({{ site.baseurl }}/get-started/support/).

### JIRA issue tracker
We use the Apache Software Foundation's [JIRA](https://issues.apache.org/jira/browse/BEAM) as an issue tracking and project management tool, as well as a way to communicate among a very diverse and distributed set of contributors. To be able to gather feedback, avoid frustration, and avoid duplicated efforts all Beam-related work should be tracked there.
Expand Down Expand Up @@ -380,7 +380,7 @@ Some larger features are developed on a feature branch before being merged into
To contribute code on a feature branch, use the same process as above, but replace `master` with the [name of the branch]({{ site.baseurl }}/contribute/work-in-progress/#feature-branches).

Since feature branches are often used for new components, you may find that
there is no [committer]({{ site.baseurl }}/project/team/) familiar with all the details of the
there is no [committer]({{ site.baseurl }}/contribute/team/) familiar with all the details of the
new language or runner. In that case, consider asking someone else familiar
with the technology to do an initial review before looping in a committer for a
final review and merge.
Expand All @@ -407,15 +407,15 @@ A new runner should:

1. Be able to handle a subset of the model that address a significant set of use cases (aka. ‘traditional batch’ or ‘processing time streaming’)
1. Update the capability matrix with the current status
1. Add a webpage under `learn/runners`
1. Add a webpage under `documentation/runners`

A new SDK should:

1. Provide the ability to construct graphs with all the basic building blocks of the model (ParDo, GroupByKey, Window, Trigger, etc)
1. Begin fleshing out the common composite transforms (Count, Join, etc) and IO connectors (Text, Kafka, etc)
1. Have at least one runner that can execute the complete model (may be a direct runner)
1. Provide integration tests for executing against current and future runners
1. Add a webpage under `learn/sdks`
1. Add a webpage under `documentation/sdks`


### Website
Expand Down
4 changes: 2 additions & 2 deletions website/src/contribute/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ Learn about the best practices for developing new PTransforms.
#### Resources
Resources for contributing to Beam, including mailing lists, the main GitHub repo, and the issues tracker.

* [Mailing Lists]({{ site.baseurl }}/use/mailing-lists/)
* [Mailing Lists]({{ site.baseurl }}/get-started/support/)
* [Source Repository]({{ site.baseurl }}/contribute/source-repository)
* [Issue Tracking]({{ site.baseurl }}/use/issue-tracking/)
* [Issue Tracking]({{ site.baseurl }}/get-started/support/)
8 changes: 4 additions & 4 deletions website/src/contribute/release-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The Apache Beam project periodically declares and publishes releases. A release

The Beam community treats releases with great importance. They are a public face of the project and most users interact with the project only through the releases. Releases are signed off by the entire Beam community in a public vote.

Each release is executed by a *Release Manager*, who is selected among the [Beam committers]({{ site.baseurl }}/project/team). This document describes the process that the Release Manager follows to perform a release. Any changes to this process should be discussed and adopted on the [dev@ mailing list]({{ site.baseurl }}/use/mailing-lists).
Each release is executed by a *Release Manager*, who is selected among the [Beam committers]({{ site.baseurl }}/contribute/team). This document describes the process that the Release Manager follows to perform a release. Any changes to this process should be discussed and adopted on the [dev@ mailing list]({{ site.baseurl }}/get-started/support/).

Please remember that publishing software has legal consequences. This guide complements the foundation-wide [Product Release Policy](http:https://www.apache.org/dev/release.html) and [Release Distribution Policy](http:https://www.apache.org/dev/release-distribution).

Expand Down Expand Up @@ -344,7 +344,7 @@ Finally, propose a pull request with these changes. (Don’t merge before finali

1. Maven artifacts deployed to the staging repository of [repository.apache.org](https://repository.apache.org/content/repositories/)
1. Source distribution deployed to the dev repository of [dist.apache.org](https://dist.apache.org/repos/dist/dev/beam/)
1. Website pull request proposed to list the [release]({{ site.baseurl }}/use/releases/), publish the [Java API reference manual]({{ site.baseurl }}/documentation/sdks/javadoc/), and publish the [Python API reference manual]({{ site.baseurl }}/documentation/sdks/pydoc/).
1. Website pull request proposed to list the [release]({{ site.baseurl }}/get-started/downloads/), publish the [Java API reference manual]({{ site.baseurl }}/documentation/sdks/javadoc/), and publish the [Python API reference manual]({{ site.baseurl }}/documentation/sdks/pydoc/).

You can (optionally) also do additional verification by:
1. Check that Python zip file contains the `README.md`, `NOTICE`, and `LICENSE` files.
Expand Down Expand Up @@ -458,7 +458,7 @@ Create a new Git tag for the released version by copying the tag for the final r

### Merge website pull request

Merge the website pull request to [list the release]({{ site.baseurl }}/use/releases/), publish the [Python API reference manual]({{ site.baseurl }}/documentation/sdks/pydoc/), and the [Java API reference manual]({{ site.baseurl }}/documentation/sdks/javadoc/) created earlier.
Merge the website pull request to [list the release]({{ site.baseurl }}/get-started/downloads/), publish the [Python API reference manual]({{ site.baseurl }}/documentation/sdks/pydoc/), and the [Java API reference manual]({{ site.baseurl }}/documentation/sdks/javadoc/) created earlier.

### Mark the version as released in JIRA

Expand All @@ -469,7 +469,7 @@ In JIRA, inside [version management](https://issues.apache.org/jira/plugins/serv
* Maven artifacts released and indexed in the [Maven Central Repository](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.beam%22)
* Source distribution available in the release repository of [dist.apache.org](https://dist.apache.org/repos/dist/release/beam/)
* Source distribution removed from the dev repository of [dist.apache.org](https://dist.apache.org/repos/dist/dev/beam/)
* Website pull request to [list the release]({{ site.baseurl }}/use/releases/) and publish the [API reference manual]({{ site.baseurl }}/documentation/sdks/javadoc/) merged
* Website pull request to [list the release]({{ site.baseurl }}/get-started/downloads/) and publish the [API reference manual]({{ site.baseurl }}/documentation/sdks/javadoc/) merged
* Release tagged in the source code repository
* Release version finalized in JIRA. (Note: Not all committers have administrator access to JIRA. If you end up getting permissions errors ask on the mailing list for assistance.)

Expand Down
2 changes: 1 addition & 1 deletion website/src/documentation/sdks/java.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The Java SDK for Apache Beam provides a simple, powerful API for building both b

## Get Started with the Java SDK

Get started with the [Beam Programming Model]({{ site.baseurl }}/learn/programming-guide/) to learn the basic concepts that apply to all SDKs in Beam.
Get started with the [Beam Programming Model]({{ site.baseurl }}/documentation/programming-guide/) to learn the basic concepts that apply to all SDKs in Beam.

See the [Java API Reference]({{ site.baseurl }}/documentation/sdks/javadoc/) for more information on individual APIs.

Expand Down
2 changes: 1 addition & 1 deletion website/src/documentation/sdks/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The Python SDK for Apache Beam provides a simple, powerful API for building batc

## Get Started with the Python SDK

Get started with the [Beam Programming Guide]({{ site.baseurl }}/learn/programming-guide) to learn the basic concepts that apply to all SDKs in Beam. Then, follow the [Beam Python SDK Quickstart]({{ site.baseurl }}/get-started/quickstart-py) to set up your Python development environment, get the Beam SDK for Python, and run an example pipeline.
Get started with the [Beam Programming Guide]({{ site.baseurl }}/documentation/programming-guide) to learn the basic concepts that apply to all SDKs in Beam. Then, follow the [Beam Python SDK Quickstart]({{ site.baseurl }}/get-started/quickstart-py) to set up your Python development environment, get the Beam SDK for Python, and run an example pipeline.

See the [Python API Reference]({{ site.baseurl }}/documentation/sdks/pydoc/) for more information on individual APIs.

Expand Down
Loading

0 comments on commit f63eeed

Please sign in to comment.