Skip to content

Commit

Permalink
[FLINK-15999][doc] Move time and windows section from programming-mod…
Browse files Browse the repository at this point in the history
…el.md to timely-stream-processing.md
  • Loading branch information
aljoscha committed Feb 21, 2020
1 parent 4382399 commit 8d7c0db
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 38 deletions.
38 changes: 0 additions & 38 deletions docs/concepts/programming-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,44 +62,6 @@ Flink offers different levels of abstraction to develop streaming/batch applicat
expressiveness, but represents programs as SQL query expressions.
The [SQL](../dev/table/index.html#sql) abstraction closely interacts with the Table API, and SQL queries can be executed over tables defined in the *Table API*.


## Windows

Aggregating events (e.g., counts, sums) works differently on streams than in batch processing.
For example, it is impossible to count all elements in a stream,
because streams are in general infinite (unbounded). Instead, aggregates on streams (counts, sums, etc),
are scoped by **windows**, such as *"count over the last 5 minutes"*, or *"sum of the last 100 elements"*.

Windows can be *time driven* (example: every 30 seconds) or *data driven* (example: every 100 elements).
One typically distinguishes different types of windows, such as *tumbling windows* (no overlap),
*sliding windows* (with overlap), and *session windows* (punctuated by a gap of inactivity).

<img src="{{ site.baseurl }}/fig/windows.svg" alt="Time- and Count Windows" class="offset" width="80%" />

More window examples can be found in this [blog post](https://flink.apache.org/news/2015/12/04/Introducing-windows.html).
More details are in the [window docs](../dev/stream/operators/windows.html).

{% top %}

## Time

When referring to time in a streaming program (for example to define windows), one can refer to different notions
of time:

- **Event Time** is the time when an event was created. It is usually described by a timestamp in the events,
for example attached by the producing sensor, or the producing service. Flink accesses event timestamps
via [timestamp assigners]({{ site.baseurl }}/dev/event_timestamps_watermarks.html).

- **Ingestion time** is the time when an event enters the Flink dataflow at the source operator.

- **Processing Time** is the local time at each operator that performs a time-based operation.

<img src="{{ site.baseurl }}/fig/event_ingestion_processing_time.svg" alt="Event Time, Ingestion Time, and Processing Time" class="offset" width="80%" />

More details on how to handle time are in the [event time docs]({{ site.baseurl }}/dev/event_time.html).

{% top %}

## Next Steps

Continue with the basic concepts in Flink's [Distributed Runtime](runtime.html).
37 changes: 37 additions & 0 deletions docs/concepts/timely-stream-processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,43 @@ under the License.
* This will be replaced by the TOC
{:toc}

## Time

When referring to time in a streaming program (for example to define windows), one can refer to different notions
of time:

- **Event Time** is the time when an event was created. It is usually described by a timestamp in the events,
for example attached by the producing sensor, or the producing service. Flink accesses event timestamps
via [timestamp assigners]({{ site.baseurl }}/dev/event_timestamps_watermarks.html).

- **Ingestion time** is the time when an event enters the Flink dataflow at the source operator.

- **Processing Time** is the local time at each operator that performs a time-based operation.

<img src="{{ site.baseurl }}/fig/event_ingestion_processing_time.svg" alt="Event Time, Ingestion Time, and Processing Time" class="offset" width="80%" />

More details on how to handle time are in the [event time docs]({{ site.baseurl }}/dev/event_time.html).

{% top %}

## Windows

Aggregating events (e.g., counts, sums) works differently on streams than in batch processing.
For example, it is impossible to count all elements in a stream,
because streams are in general infinite (unbounded). Instead, aggregates on streams (counts, sums, etc),
are scoped by **windows**, such as *"count over the last 5 minutes"*, or *"sum of the last 100 elements"*.

Windows can be *time driven* (example: every 30 seconds) or *data driven* (example: every 100 elements).
One typically distinguishes different types of windows, such as *tumbling windows* (no overlap),
*sliding windows* (with overlap), and *session windows* (punctuated by a gap of inactivity).

<img src="{{ site.baseurl }}/fig/windows.svg" alt="Time- and Count Windows" class="offset" width="80%" />

More window examples can be found in this [blog post](https://flink.apache.org/news/2015/12/04/Introducing-windows.html).
More details are in the [window docs](../dev/stream/operators/windows.html).

{% top %}

## Latency & Completeness

### Latency vs. Completeness in Batch & Stream Processing
Expand Down

0 comments on commit 8d7c0db

Please sign in to comment.