Skip to content

ADE detects anomalous time slices and messages in Linux logs (either RFC3164 or RFC5424 format) using statistical learning.

License

Notifications You must be signed in to change notification settings

openmainframeproject/ade

Repository files navigation

Anomaly Detection Engine for Linux Logs (ADE)

ADE can process a large numbers of logs from a large number of Linux systems to create a compact summary of those logs. The summary identifies and consolidates similar text strings into a single message example and assigns it a key (message id).  The summary determines if  the message id are being issued when expected, are being issued at the expected rate during a time slice, and how often during the day are the message or a similar message (same message id) issued.

You can use those results to examine

  • A set of logs to find anomalies which may be helpful when attempting to find the root cause of a problem or incident
  • The currently generated logs to find anomalies which may be helpful when attempting to find the cause of an on-going problem or incident

Please see https://openmainframeproject.github.io/ade/ for documentation on ADE.

Releases

Saw Kill 1.0.4

  • Fix problems with timezone in Junit test
  • Fix problems with year of test data in function test

Fall Kill 1.0.3

CII Best Practices

  • Support for Core Infrastructure Initiative
    • add travis-ci build for every pull request
    • analyze every pull request with Sonarqube
    • store results of Sonarqube analysis at Sonarqube.com
  • Add sample to mask sensitive data within Linux logs to allow sharing of logs
  • Fix problem with train_test.sh
  • Fix additional problems identified by Sonarqube

Poesten Kill 1.0.2

  • Support for changing analytics
    • command to check syntax of model (flowlayout.xml file)
    • command to print out statistical information contained within model file (.bin file) to text file
    • command to print out version of code and data base
  • Multiple SonarQube(TM) issues fixed
  • Fix to problem with regression test
  • Wiki article "Example of reading ADE data into R objects"

Esopus Creek 1.0.1

  • Support for MariaDB(TM)
  • Verify script - determine if sufficient messages are available to create a valid model
  • Multiple SonarQube(TM) issues fixed
  • Wiki article "Hints on how to update XSLT - tailor the output shown in a browser to problem"

Initial release 1.0.0

  • Parsing of Linux Logs in RFC5424 and RFC3164 format
  • Splitting logs into time slices
  • Handling wrapper messages
  • Statistical analysis of logs
  • Creates output

About

ADE detects anomalous time slices and messages in Linux logs (either RFC3164 or RFC5424 format) using statistical learning.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published