Skip to content

Commit

Permalink
Add a 'how to populate ECS fields' section to readme (elastic#345)
Browse files Browse the repository at this point in the history
  • Loading branch information
webmat authored and ruflin committed Mar 5, 2019
1 parent fc136a5 commit 29524de
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 2 deletions.
29 changes: 28 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -562,7 +562,7 @@ in the future. Please avoid using them:

**Writing fields**

* All fields must be lower case
* All field names must be lower case
* Combine words using underscore
* No special characters except `_`

Expand All @@ -575,6 +575,33 @@ in the future. Please avoid using them:
* *Use prefixes.* Fields must be prefixed except for the base fields. For example all `host` fields are prefixed with `host.`. See `dot` notation in FAQ for more details.
* Do not use abbreviations. (A few exceptions like `ip` exist.)

**How to populate ECS fields**

In order to maximize the level of interoperability of your data with future analysis content
(e.g., saved searches, visualizations, dashboards, alerts, machine learning jobs,
reports, apps, etc.); whether this content is provided by Elastic
or shared by the community, please keep the following guidelines in mind.


* Start by populating ECS Core fields. These fields are the most common across
all use cases. Your implementation should attempt to populate as many of
the ECS Core fields as practical.
* ...even if these fields or values are metadata that are not present in your
original event. For example, a firewall event shipped to the Elastic Stack
over syslog may not actually include a field with value ”syslog,” but the
ECS `agent.type` field should still be set to “syslog."
* ...even if the data has already been mapped to one ECS field. For example,
if you’ve mapped your event’s client IP address to ECS `client.ip`, you can
also copy the same value to ECS `source.ip` and append it to `related.ip`.
* Populate as many ECS Extended fields as apply to the subject and use case of
your event. ECS Extended fields generally apply to more narrow use cases,
so feel free to skip ECS Extended fields that are not applicable to your use case.
* You're free to choose whether to copy your event fields to ECS fields,
leaving your original fields intact as custom fields, or to rename your existing
event fields to ECS, essentially migrating them to the ECS field names.
Future ECS content and applications will depend only upon the presence of
the ECS fields, and will ignore any custom fields.

## Normalization

In order to be help allow for correlation across different sources, ECS must sometimes
Expand Down
29 changes: 28 additions & 1 deletion docs/implementing.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ in the future. Please avoid using them:

**Writing fields**

* All fields must be lower case
* All field names must be lower case
* Combine words using underscore
* No special characters except `_`

Expand All @@ -33,6 +33,33 @@ in the future. Please avoid using them:
* *Use prefixes.* Fields must be prefixed except for the base fields. For example all `host` fields are prefixed with `host.`. See `dot` notation in FAQ for more details.
* Do not use abbreviations. (A few exceptions like `ip` exist.)

**How to populate ECS fields**

In order to maximize the level of interoperability of your data with future analysis content
(e.g., saved searches, visualizations, dashboards, alerts, machine learning jobs,
reports, apps, etc.); whether this content is provided by Elastic
or shared by the community, please keep the following guidelines in mind.


* Start by populating ECS Core fields. These fields are the most common across
all use cases. Your implementation should attempt to populate as many of
the ECS Core fields as practical.
* ...even if these fields or values are metadata that are not present in your
original event. For example, a firewall event shipped to the Elastic Stack
over syslog may not actually include a field with value ”syslog,” but the
ECS `agent.type` field should still be set to “syslog."
* ...even if the data has already been mapped to one ECS field. For example,
if you’ve mapped your event’s client IP address to ECS `client.ip`, you can
also copy the same value to ECS `source.ip` and append it to `related.ip`.
* Populate as many ECS Extended fields as apply to the subject and use case of
your event. ECS Extended fields generally apply to more narrow use cases,
so feel free to skip ECS Extended fields that are not applicable to your use case.
* You're free to choose whether to copy your event fields to ECS fields,
leaving your original fields intact as custom fields, or to rename your existing
event fields to ECS, essentially migrating them to the ECS field names.
Future ECS content and applications will depend only upon the presence of
the ECS fields, and will ignore any custom fields.

## Normalization

In order to be help allow for correlation across different sources, ECS must sometimes
Expand Down

0 comments on commit 29524de

Please sign in to comment.