Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a 'how to populate ECS fields' section to readme... #345

Merged
merged 2 commits into from
Mar 5, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -608,7 +608,7 @@ in the future. Please avoid using them:

**Writing fields**

* All fields must be lower case
* All field names must be lower case
* Combine words using underscore
* No special characters except `_`

Expand All @@ -621,6 +621,33 @@ in the future. Please avoid using them:
* *Use prefixes.* Fields must be prefixed except for the base fields. For example all `host` fields are prefixed with `host.`. See `dot` notation in FAQ for more details.
* Do not use abbreviations. (A few exceptions like `ip` exist.)

**How to populate ECS fields**

In order to maximize the level of interoperability of your data with future analysis content
(e.g., saved searches, visualizations, dashboards, alerts, machine learning jobs,
reports, apps, etc.); whether this content is provided by Elastic
or shared by the community, please keep the following guidelines in mind.


* Start by populating ECS Core fields. These fields are the most common across
all use cases. Your implementation should attempt to populate as many of
the ECS Core fields as practical.
* ...even if these fields or values are metadata that are not present in your
original event. For example, a firewall event shipped to the Elastic Stack
over syslog may not actually include a field with value ”syslog,” but the
ECS `agent.type` field should still be set to “syslog."
* ...even if the data has already been mapped to one ECS field. For example,
if you’ve mapped your event’s client IP address to ECS `client.ip`, you can
also copy the same value to ECS `source.ip` and append it to `related.ip`.
* Populate as many ECS Extended fields as apply to the subject and use case of
your event. ECS Extended fields generally apply to more narrow use cases,
so feel free to skip ECS Extended fields that are not applicable to your use case.
* You're free to choose whether to copy your event fields to ECS fields,
leaving your original fields intact as custom fields, or to rename your existing
event fields to ECS, essentially migrating them to the ECS field names.
Future ECS content and applications will depend only upon the presence of
the ECS fields, and will ignore any custom fields.

## Normalization

In order to be help allow for correlation across different sources, ECS must sometimes
Expand Down
29 changes: 28 additions & 1 deletion docs/implementing.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ in the future. Please avoid using them:

**Writing fields**

* All fields must be lower case
* All field names must be lower case
* Combine words using underscore
* No special characters except `_`

Expand All @@ -34,6 +34,33 @@ in the future. Please avoid using them:
* *Use prefixes.* Fields must be prefixed except for the base fields. For example all `host` fields are prefixed with `host.`. See `dot` notation in FAQ for more details.
* Do not use abbreviations. (A few exceptions like `ip` exist.)

**How to populate ECS fields**

In order to maximize the level of interoperability of your data with future analysis content
(e.g., saved searches, visualizations, dashboards, alerts, machine learning jobs,
reports, apps, etc.); whether this content is provided by Elastic
or shared by the community, please keep the following guidelines in mind.


* Start by populating ECS Core fields. These fields are the most common across
all use cases. Your implementation should attempt to populate as many of
the ECS Core fields as practical.
* ...even if these fields or values are metadata that are not present in your
original event. For example, a firewall event shipped to the Elastic Stack
over syslog may not actually include a field with value ”syslog,” but the
ECS `agent.type` field should still be set to “syslog."
* ...even if the data has already been mapped to one ECS field. For example,
if you’ve mapped your event’s client IP address to ECS `client.ip`, you can
also copy the same value to ECS `source.ip` and append it to `related.ip`.
* Populate as many ECS Extended fields as apply to the subject and use case of
your event. ECS Extended fields generally apply to more narrow use cases,
so feel free to skip ECS Extended fields that are not applicable to your use case.
* You're free to choose whether to copy your event fields to ECS fields,
leaving your original fields intact as custom fields, or to rename your existing
event fields to ECS, essentially migrating them to the ECS field names.
Future ECS content and applications will depend only upon the presence of
the ECS fields, and will ignore any custom fields.

## Normalization

In order to be help allow for correlation across different sources, ECS must sometimes
Expand Down