.. rst-class:: opening

    This document summarizes installation and use of *BroControl*,
    Bro's interactive shell for operating Bro installations. *BroControl*
    has two modes of operation: a *stand-alone* mode for
    managing a traditional, single-system Bro setup; and a *cluster*
    mode for maintaining a multi-system setup of coordinated Bro
    instances load-balancing the work across a set of independent
    machines.  Once installed, the operation is pretty similar
    for both types; just keep in mind that if this document refers to
    "nodes" and you're in a stand-alone setup, there is only a
    single one and no worker/proxies.

You can find the latest BroControl release for download at http:https://www.bro.org/download.

BroControl's git repository is located at git:https://git.bro.org/broctl. You can browse the repository here.

This document describes BroControl 1.4-96. See the CHANGES file for version history.

Running BroControl requires the following prerequisites:

• A Unix system. FreeBSD, Linux, and Mac OS X are supported and should work out of the box. Other Unix systems will quite likely require some tweaking. Note that in a cluster setup, all systems must be running exactly the same operating system.
• A version of Python >= 2.6 (on FreeBSD, the package "py27-sqlite3" must also be installed).
• A bash (note in particular, that on FreeBSD, bash is not installed by default).
• If sendmail is installed (for a cluster setup, it is needed on the manager only), then BroControl can send mail. Otherwise, BroControl will not attempt to send mail.

For a cluster setup that spans more than one machine, there are additional requirements:

• Every host in the cluster must have rsync installed.
• The manager host must have ssh installed, and every other host in the cluster must have sshd installed and running.
• Decide which user account will be running BroControl, and then make sure this user account is set up on all hosts in your cluster. Also make sure this user can ssh from the manager host to each of the other hosts in your cluster, and this must work without being prompted for a password/passphrase (one way to accomplish this is to use ssh public key authentication).

If you're using a load-balancing method such as PF_RING, then there is additional software to install (for details, see the :doc:`Cluster Configuration <../../configuration/index>` documentation).

Follow the directions to install Bro and BroControl according to the instructions in the :doc:`Installing Bro <../../install/install>` documentation. Note that if you are planning to run Bro in a cluster configuration, then you need to install Bro and BroControl only on the manager host (the BroControl install or deploy commands will install Bro and all required scripts to the other hosts in your cluster).

If you decide to run BroControl as an unprivileged user, there are a few issues that you may encounter.

If you installed Bro and BroControl as the "root" user, then you will need to adjust the ownership or permissions of the "logs" and "spool" directories (and everything in those directories) so that the user running BroControl has write permission.

If you're using a cluster setup that spans multiple machines, and if your BroControl install or deploy commands fail with a permission denied error, then it's most likely due to the user running BroControl not having permission to create the install prefix directory (by default, this is /usr/local/bro) on each remote machine. A simple workaround is to login to each machine in your cluster and manually create the install prefix directory and then set ownership or permissions of this directory so that the user who will run BroControl has write access to it.

Finally, on the worker nodes (or the standalone node), Bro must have access to the target network interface in promiscuous mode. If Bro doesn't have the necessary permissions, then it will fail almost immediately upon startup. A workaround for this is provided in the Bro FAQ.

Before actually running BroControl, you first need to edit the broctl.cfg, node.cfg, and networks.cfg files.

In the broctl.cfg file, you should review the BroControl options and make sure they are set correctly for your environment. Most options have default values that are reasonable for most users (the MailTo option is probably the one that you will most likely want to change), but for a description of every BroControl option, see the Option Reference section below.

Next, edit the node.cfg file and specify the nodes that you will be running. For a description of every option available for nodes, see the Node section below. If you will be using a standalone configuration then most likely the only option you need to change is the interface. If you will be using a cluster configuration, there is a :doc:`Cluster Configuration <../../configuration/index>` guide that provides examples and additional information.

Finally, edit the networks.cfg file and list each network (see the examples in the file for the format to use) that is considered local to the monitored environment.

BroControl is an interactive interface for managing a Bro installation which allows you to, e.g., start/stop the monitoring or update its configuration.

BroControl is started with the broctl script and then expects commands on its command-line (alternatively, broctl can also be started with a single command directly on the shell's command line, such as broctl help):

> broctl
Welcome to BroControl x.y

Type "help" for help.

[BroControl] >

As the message says, type help to see a list of all commands. We will now briefly summarize the most important commands. A full reference follows Command Reference.

The config command gives a list of all BroControl options with their current values. This can be useful when troubleshooting a problem to check if an option has the expected value.

If this is the first time you are running BroControl, then the first command you must run is the BroControl deploy command. The "deploy" command will make sure all of the files needed by BroControl and Bro are brought up-to-date based on the configuration specified in the broctl.cfg, node.cfg, and networks.cfg files. It will also check if there are any syntax errors in your Bro policy scripts. For a cluster setup it will copy all of the required scripts and executables to all the other hosts in your cluster. Then it will successively start manager, proxies, and workers (for a standalone configuration, only one Bro instance will be started).

The status command can be used to check that all nodes are "running". If any nodes have a status of "crashed", then use the diag command to see diagnostic information (you can specify the name of a crashed node as an argument to the diag command to show diagnostics for only that one node).

To stop the monitoring, issue the stop command. After all nodes have stopped, the status command should show all nodes as "stopped". exit leaves the shell (you can also exit BroControl while Bro nodes are running).

Whenever the BroControl or Bro configuration is modified in any way, including changes to configuration files and site-specific policy scripts or upgrading to a new version of Bro, deploy must be run (deploy will check all policy scripts, install all needed files, and restart Bro). No changes will take effect until deploy is run.

The BroControl cron command performs housekeeping tasks, such as checking whether Bro is running or not (and starting or stopping to match the expected state, as needed), checking if there is sufficient free disk space, etc. This command is intended to be run from a cron job, rather than interactively by a user. To setup a cron job that runs once every five minutes, insert the following entry into the crontab of the user running BroControl (change the path to the actual location of broctl on your system):

0-59/5 * * * * /usr/local/bro/bin/broctl cron

If the "broctl cron disable" command is run, then broctl cron will be disabled (i.e., broctl cron won't do anything) until the "broctl cron enable" command is run. To check the status at any time, run "broctl cron ?".

On the manager system (and on the stand-alone system), while Bro is running you can find the current set of (aggregated) logs in logs/current (which is a symlink to the corresponding spool directory).

Bro logs are automatically rotated once per hour by default, or whenever Bro is stopped. A rotated log is renamed to contain a timestamp in the filename. For example, the conn.log might be renamed to conn.2015-01-20-15-23-42.log.

Immediately after a log is rotated, it is archived automatically. When a log is archived, it is moved to a subdirectory of logs/ named by date (such as logs/2015-01-20), then it is renamed again, and gzipped. For example, a rotated log file named conn.2015-01-20-15-23-42.log might be archived to logs/2015-01-20/conn.15:48:23-16:00:00.log.gz. If the archival was successful, then the original (rotated) log file is removed.

If, for some reason, a rotated log file cannot be archived then it will be left in the node's working directory. Next time when BroControl either stops Bro or tries to restart a crashed Bro, it will try to archive such log files again. If this attempt fails, then an email is sent which contains the name of a directory where any such unarchived logs can be found.

When BroControl starts Bro it creates two files "stdout.log" and "stderr.log", which just capture stdout and stderr from Bro. Although these are not actually Bro logs, they might contain useful error or diagnostic information.

Also, whenever logs are rotated, a connection summary report is generated if the trace-summary tool is installed. Although these are not actually Bro logs, they follow the same filename convention as other Bro logs and they have the filename prefix "conn-summary". To prevent these files from being created, set the value of the TraceSummary option to an empty string.

You'll most likely want to adapt the Bro policy to the local environment and generally site-specific tuning requires writing local policy scripts.

Sample local policy scripts (which you can edit) are located in share/bro/site. In the stand-alone setup, a single file called local.bro gets loaded automatically. In the cluster setup, the same local.bro gets loaded, followed by one of three other files: local-manager.bro, local-worker.bro, and local-proxy.bro are loaded by the manager, workers, and proxy, respectively.

The recommended way to modify the policy is to use only "@load" directives in the local.bro scripts. For example, you can add a "@load" directive to load a Bro policy script that is included with Bro but is not loaded by default. You can also create custom site-specific policy scripts in the same directory as the local.bro scripts, and "@load" them from one of the local.bro scripts. For example, you could create your own Bro script mypolicy.bro in the share/bro/site directory, and then add a line "@load mypolicy" (without the quotes) to the local.bro script. Note that in a cluster setup, notice filtering should be done only on the manager.

After creating or modifying your local policy scripts, you must install them by using the BroControl "install" or "deploy" command. Next, you can use the BroControl "scripts" command to verify that your new scripts will be loaded when you start Bro.

If you want to change which local policy scripts are loaded by the nodes, you can set SitePolicyStandalone for all Bro instances, SitePolicyManager for the manager, and SitePolicyWorker for the workers. To change the directory where local policy scripts are located, set the option SitePolicyPath to a different path. These options can be changed in the broctl.cfg file.

When writing custom site-specific policy scripts, it can sometimes be useful to know in which order the scripts are loaded (the BroControl "scripts" command shows the load order of every script loaded by Bro). For example, if more than one script sets a value for the same global variable, then the value that takes effect is the one set by the last such script loaded.

When BroControl starts Bro, the first script loaded is init-bare.bro, followed by init-default.bro (keep in mind that each of these scripts loads many other scripts).

Next, the local.bro script is loaded. This provides for a common set of loaded scripts for all nodes.

Next, the "broctl" script package is loaded. This consists of some standard settings that BroControl needs.

In a cluster setup, one of the following scripts are loaded: local-manager.bro, local-proxy.bro, or local-worker.bro.

The next scripts loaded are local-networks.bro and broctl-config.bro. These scripts are automatically generated by BroControl based on the contents of the networks.cfg and broctl.cfg files.

The last scripts loaded are any node-specific scripts specified with the option aux_scripts in node.cfg. This option can be used to load additional scripts to individual nodes only. For example, one could add a script experimental.bro to a single worker for trying out new experimental code.

There are several situations when BroControl sends mail to the address given in MailTo (note that BroControl will not be able to send any mail when the value of the SendMail option is an empty string):

1. When the broctl cron command runs it performs various tasks (such as checking available disk space, expiring old log files, etc.). If any problems occur, a mail will be sent containing a list of those issues. Setting MailHostUpDown=0 will disable some of this output. Also, setting StatsLogEnable=0 will disable some functionality involving writing to stats.log (which could also reduce the amount of email).
2. When BroControl tries to start or stop (via any of these commands: start, stop, restart, deploy, or cron) a node that has crashed, a crash report is mailed (one for each crashed node). The crash report is essentially just the output of the broctl diag command. When broctl cron is run with the --no-watch option, then it will not attempt to start or stop Bro.
3. When BroControl stops Bro or restarts a crashed Bro, if any log files could not be archived, then an email will be sent. This can be disabled by setting MailArchiveLogFail=0.
4. If trace-summary is installed, a traffic summary is mailed each rotation interval. This can be disabled by setting MailConnectionSummary=0.

The following summary lists all commands supported by BroControl. All commands may be either entered interactively or specified on the shell's command line. If not specified otherwise, commands taking [<nodes>] as arguments apply their action either to the given set of nodes, to the manager node if "manager" is given, to all proxy nodes if "proxies" is given, to all worker nodes if "workers" is given, or to all nodes if none are given.

capstats [<nodes>] [<interval>]

Determines the current load on the network interfaces monitored by each of the given worker nodes. The load is measured over the specified interval (in seconds), or by default over 10 seconds. This command uses the :doc:`capstats<../../components/capstats/README>` tool, which is installed along with broctl.

check [<nodes>]

Verifies a modified configuration in terms of syntactical correctness (most importantly correct syntax in policy scripts). This command should be executed for each configuration change before install is used to put the change into place. The check command uses the policy files as found in SitePolicyPath to make sure they compile correctly. If they do, install will then copy them over to an internal place from where the nodes will read them at the next start. This approach ensures that new errors in a policy script will not affect currently running nodes, even when one or more of them needs to be restarted.

cleanup [--all] [<nodes>]

Clears the nodes' spool directories (if they are not running currently). This implies that their persistent state is flushed. Nodes that were crashed are reset into stopped state. If --all is specified, this command also removes the content of the node's TmpDir, in particular deleteing any data potentially saved there for reference from previous crashes. Generally, if you want to reset the installation back into a clean state, you can first stop all nodes, then execute cleanup --all, and finally start all nodes again.

config

Prints all configuration options with their current values.

cron [enable|disable|?] | [--no-watch]

This command has two modes of operation. Without arguments (or just --no-watch), it performs a set of maintenance tasks, including the logging of various statistical information, expiring old log files, checking for dead hosts, and restarting nodes which terminated unexpectedly (the latter can be suppressed with the --no-watch option if no auto-restart is desired). This mode is intended to be executed regularly via cron, as described in the installation instructions. While not intended for interactive use, no harm will be caused by executing the command manually: all the maintenance tasks will then just be performed one more time.

The second mode is for interactive usage and determines if the regular tasks are indeed performed when broctl cron is executed. In other words, even with broctl cron in your crontab, you can still temporarily disable it by running cron disable, and then later reenable with cron enable. This can be helpful while working, e.g., on the BroControl configuration and cron would interfere with that. cron ? can be used to query the current state.

deploy

Checks for errors in Bro policy scripts, then does an install followed by a restart on all nodes. This command should be run after any changes to Bro policy scripts or the broctl configuration, and after Bro is upgraded or even just recompiled.

This command is equivalent to running the check, install, and restart commands, in that order.

df [<nodes>]

Reports the amount of disk space available on the nodes. Shows only paths relevant to the broctl installation.

diag [<nodes>]

If a node has terminated unexpectedly, this command prints a (somewhat cryptic) summary of its final state including excerpts of any stdout/stderr output, resource usage, and also a stack backtrace if a core dump is found. The same information is sent out via mail when a node is found to have crashed (the "crash report"). While the information is mainly intended for debugging, it can also help to find misconfigurations (which are usually, but not always, caught by the check command).

exec <command line>

Executes the given Unix shell command line on all hosts configured to run at least one Bro instance. This is handy to quickly perform an action across all systems.

exit

Terminates the shell.

help

Prints a brief summary of all commands understood by the shell.

install [--local]

Reinstalls on all nodes (unless the --local option is given, in which case nothing will be propagated to other nodes), including all configuration files and local policy scripts. Usually all nodes should be reinstalled at the same time, as any inconsistencies between them will lead to strange effects. This command must be executed after all changes to any part of the broctl configuration (and after upgrading to a new version of Bro or BroControl), otherwise the modifications will not take effect. Before executing install, it is recommended to verify the configuration with check.

netstats [<nodes>]

Queries each of the nodes for their current counts of captured and dropped packets.

nodes

Prints a list of all configured nodes.

peerstatus [<nodes>]

Primarily for debugging, peerstatus reports statistics about the network connections cluster nodes are using to communicate with other nodes.

print <id> [<nodes>]

Reports the current live value of the given Bro script ID on all of the specified nodes (which obviously must be running). This can for example be useful to (1) check that policy scripts are working as expected, or (2) confirm that configuration changes have in fact been applied. Note that IDs defined inside a Bro namespace must be prefixed with <namespace>:: (e.g., print HTTP::mime_types_extensions to print the corresponding table from file-ident.bro).

process <trace> [options] [-- <scripts>]

Runs Bro offline on a given trace file using the same configuration as when running live. It does, however, use the potentially not-yet-installed policy files in SitePolicyPath and disables log rotation. Additional Bro command line flags and scripts can be given (each argument after a -- argument is interpreted as a script).

Upon completion, the command prints a path where the log files can be found. Subsequent runs of this command may delete these logs.

In cluster mode, Bro is run with both manager and worker scripts loaded into a single instance. While that doesn't fully reproduce the live setup, it is often sufficient for debugging analysis scripts.

quit

Terminates the shell.

restart [--clean] [<nodes>]

Restarts the given nodes, or all nodes if none are specified. The effect is the same as first executing stop followed by a start, giving the same nodes in both cases.

If --clean is given, the installation is reset into a clean state before restarting. More precisely, a restart --clean turns into the command sequence stop, cleanup, check, install, and start.

scripts [-c] [<nodes>]

Primarily for debugging Bro configurations, the scripts command lists all the Bro scripts loaded by each of the nodes in the order they will be parsed by the node at startup. If -c is given, the command operates as check does: it reads the policy files from their original location, not the copies installed by install. The latter option is useful to check a not yet installed configuration.

start [<nodes>]

Starts the given nodes, or all nodes if none are specified. Nodes already running are left untouched.

status [<nodes>]

Prints the current status of the given nodes.

stop [<nodes>]

Stops the given nodes, or all nodes if none are specified. Nodes not running are left untouched.

top [<nodes>]

For each of the nodes, prints the status of the two Bro processes (parent process and child process) in a top-like format, including CPU usage and memory consumption. If executed interactively, the display is updated frequently until key q is pressed. If invoked non-interactively, the status is printed only once.

update [<nodes>]

After a change to Bro policy scripts, this command updates the Bro processes on the given nodes while they are running (i.e., without requiring a restart). However, such dynamic updates work only for a subset of Bro's full configuration. The following changes can be applied on the fly: The value of all const variables defined with the &redef attribute can be changed. More extensive script changes are not possible during runtime and always require a restart; if you change more than just the values of &redef-able consts and still issue update, the results are undefined and can lead to crashes. Also note that before running update, you still need to do an install (preferably after check), as otherwise update will not see the changes and it will resend the old configuration.

This section summarizes the options that can be set in broctl.cfg for customizing the behavior of BroControl (the option names are case-insensitive). Usually, one only needs to change the "user options", which are listed first. The "internal options" are, as the name suggests, primarily used internally and set automatically. They are documented here only for reference.

BroArgs (string, default _empty_)

Additional arguments to pass to Bro on the command-line (e.g. broargs=-f "tcp port 80").

BroPort (int, default 47760)

The TCP port number that Bro will listen on. For a cluster configuration, each node in the cluster will automatically be assigned a subsequent port to listen on.

CommTimeout (int, default 10)

The number of seconds to wait before assuming Broccoli communication events have timed out.

CommandTimeout (int, default 60)

The number of seconds to wait for a command to return results.

CompressCmd (string, default "gzip -9")

If archived logs will be compressed, the command to use for that. The specified command must compress its standard input to standard output.

CompressExtension (string, default "gz")

If archived logs will be compressed, the file extension to use on compressed log files. When specifying a file extension, don't include the period character (e.g., specify 'gz' instead of '.gz').

CompressLogs (bool, default 1)

True to compress archived log files.

CronCmd (string, default _empty_)

A custom command to run everytime the cron command has finished.

Debug (bool, default 0)

Enable extensive debugging output in spool/debug.log.

Env_Vars (string, default _empty_)

A comma-separated list of environment variables (e.g. env_vars=VAR1=123, VAR2=456) to set on all nodes immediately before starting Bro. Node-specific values (specified in the node configuration file) override these global values.

HaveNFS (bool, default 0)

True if shared files are mounted across all nodes via NFS (see the FAQ).

IPv6Comm (bool, default 1)

Enable IPv6 communication between cluster nodes (and also between them and BroControl). This overrides the Bro script variable Communication::listen_ipv6.

KeepLogs (string, default _empty_)

A space-separated list of filename shell patterns of expired log files to keep (empty string means don't keep any expired log files). The filename shell patterns are not regular expressions and do not include any directories. For example, specifying 'conn.* dns*' will prevent any expired log files with filenames starting with 'conn.' or 'dns' from being removed. Finally, note that this option is ignored if log files never expire.

LogDir (string, default "${BroBase}/logs")

Directory for archived log files.

LogExpireInterval (string, default "0")

Time interval that archived log files are kept (a value of 0 means log files never expire). The time interval is expressed as an integer followed by one of the following time units: day, hr, min.

LogRotationInterval (int, default 3600)

The frequency of log rotation in seconds for the manager/standalone node (zero to disable rotation). This overrides the Bro script variable Log::default_rotation_interval.

MailAlarmsInterval (int, default 86400)

The frequency (in seconds) of sending alarm summary mails (zero to disable). This overrides the Bro script variable Log::default_mail_alarms_interval.

MailAlarmsTo (string, default "${MailTo}")

Destination address for alarm summary mails. Default is to use the same address as MailTo. This overrides the Bro script variable Notice::mail_dest_pretty_printed.

MailArchiveLogFail (bool, default 1)

True to enable sending mail when log files fail to be archived.

MailConnectionSummary (bool, default 1)

True to mail connection summary reports each log rotation interval (if false, then connection summary reports will still be generated and archived, but they will not be mailed). However, this option has no effect if the trace-summary script is not available.

MailFrom (string, default "Big Brother <bro@localhost>")

Originator address for mails. This overrides the Bro script variable Notice::mail_from.

MailHostUpDown (bool, default 1)

True to enable sending mail when broctl cron notices the availability of a host in the cluster to have changed.

MailReplyTo (string, default _empty_)

Reply-to address for broctl-generated mails.

MailSubjectPrefix (string, default "[Bro]")

General Subject prefix for mails. This overrides the Bro script variable Notice::mail_subject_prefix.

MailTo (string, default "<user>")

Destination address for non-alarm mails. This overrides the Bro script variable Notice::mail_dest.

MakeArchiveName (string, default "${BroBase}/share/broctl/scripts/make-archive-name")

Script to generate filenames for archived log files.

MemLimit (string, default "unlimited")

Maximum amount of memory for Bro processes to use (in KB, or the string 'unlimited').

MinDiskSpace (int, default 5)

Percentage of minimum disk space available before warning is mailed.

PFRINGClusterID (int, default 0)

If PF_RING flow-based load balancing is desired, this is where the PF_RING cluster id is defined. The default value is configuration-dependent and determined automatically by CMake at configure-time based upon whether PF_RING's enhanced libpcap is available. Bro must be linked with PF_RING's libpcap wrapper for this option to work.

PFRINGClusterType (string, default "4-tuple")

If PF_RING flow-based load balancing is desired, this is where the PF_RING cluster type is defined. Allowed values are: 2-tuple, 4-tuple, 5-tuple, tcp-5-tuple, 6-tuple, or round-robin. Bro must be linked with PF_RING's libpcap wrapper and PFRINGClusterID must be non-zero for this option to work.

PFRINGFirstAppInstance (int, default 0)

The first application instance for a PF_RING dnacluster interface to use. Broctl will start at this application instance number and increment for each new process running on that DNA cluster. Bro must be linked with PF_RING's libpcap wrapper, PFRINGClusterID must be non-zero, and you must be using PF_RING+DNA and libzero for this option to work.

Prefixes (string, default "local")

Additional script prefixes for Bro, separated by colons. Use this instead of @prefix.

SaveTraces (bool, default 0)

True to let backends capture short-term traces via '-w'. These are not archived but might be helpful for debugging.

SendMail (string, default "@SENDMAIL@")

Location of the sendmail binary. Make this string blank to prevent email from being sent. The default value is configuration-dependent and determined automatically by CMake at configure-time. This overrides the Bro script variable Notice::sendmail.

SitePluginPath (string, default _empty_)

Directories to search for custom plugins (i.e., plugins that are not included with broctl), separated by colons.

SitePolicyManager (string, default "local-manager.bro")

Space-separated list of local policy files for manager.

SitePolicyPath (string, default "${PolicyDir}/site")

Directories to search for local (i.e., site-specific) policy files, separated by colons. For each such directory, all files and subdirectories are copied to PolicyDirSiteInstall during broctl 'install' or 'deploy' (however, if the same file or subdirectory is found in more than one such directory, then only the first one encountered will be used).

SitePolicyStandalone (string, default "local.bro")

Space-separated list of local policy files for all Bro instances.

SitePolicyWorker (string, default "local-worker.bro")

Space-separated list of local policy files for workers.

StatsLogEnable (bool, default 1)

True to enable BroControl to write statistics to the stats.log file.

StatsLogExpireInterval (int, default 0)

Number of days entries in the stats.log file are kept (zero means never expire).

StatusCmdShowAll (bool, default 1)

True to have the status command show all output, or False to show only some of the output (peer information will not be collected or shown, so the command will run faster).

StopTimeout (int, default 60)

The number of seconds to wait before sending a SIGKILL to a node which was previously issued the 'stop' command but did not terminate gracefully.

TimeFmt (string, default "%d %b %H:%M:%S")

Format string to print date/time specifications (see 'man strftime').

TimeMachineHost (string, default _empty_)

If the manager should connect to a Time Machine, the address of the host it is running on.

TimeMachinePort (string, default "47757/tcp")

If the manager should connect to a Time Machine, the port it is running on (in Bro syntax, e.g., 47757/tcp).

ZoneID (string, default _empty_)

If the host running BroControl is managing a cluster comprised of nodes with non-global IPv6 addresses, this option indicates what RFC 4007 zone_id to append to node addresses when communicating with them.

BinDir (string, default "${BroBase}/bin")

Directory for executable files.

Bro (string, default "${BinDir}/bro")

Path to Bro binary.

BroBase (string, default _empty_)

Base path of broctl installation on all nodes.

CapstatsPath (string, default "${bindir}/capstats")

Path to capstats binary; empty if not available.

CfgDir (string, default "${BroBase}/etc")

Directory for configuration files.

DebugLog (string, default "${SpoolDir}/debug.log")

Log file for debugging information.

HelperDir (string, default "${BroBase}/share/broctl/scripts/helpers")

Directory for broctl helper scripts.

LibDir (string, default "${BroBase}/lib")

Directory for library files.

LibDirInternal (string, default "${BroBase}/lib/broctl")

Directory for broctl-specific library files.

LocalNetsCfg (string, default "${CfgDir}/networks.cfg")

File defining the local networks.

LockFile (string, default "${SpoolDir}/lock")

Lock file preventing concurrent shell operations.

LogExpireMinutes (int, default 0)

Time interval (in minutes) that archived log files are kept (0 means they never expire). Users should never modify this value (see the LogExpireInterval option).

NodeCfg (string, default "${CfgDir}/node.cfg")

Node configuration file.

OS (string, default _empty_)

Name of operating system as reported by uname.

PcapBufsize (int, default 128)

Number of Mbytes to provide as buffer space when capturing from live interfaces via libpcap.

PcapSnaplen (int, default 8192)

Number of bytes per packet to capture from live interfaces via libpcap.

PluginDir (string, default "${LibDirInternal}/plugins")

Directory where standard plugins are located.

PolicyDir (string, default "${BroScriptDir}")

Directory for standard policy files.

PolicyDirSiteInstall (string, default "${SpoolDir}/installed-scripts-do-not-touch/site")

Directory where the shell copies local (i.e., site-specific) policy scripts when installing.

PolicyDirSiteInstallAuto (string, default "${SpoolDir}/installed-scripts-do-not-touch/auto")

Directory where the shell copies auto-generated local policy scripts when installing.

PostProcDir (string, default "${BroBase}/share/broctl/scripts/postprocessors")

Directory for log postprocessors.

ScriptsDir (string, default "${BroBase}/share/broctl/scripts")

Directory for executable scripts shipping as part of broctl.

SpoolDir (string, default "${BroBase}/spool")

Directory for run-time data.

StandAlone (bool, default 0)

True if running in stand-alone mode (see elsewhere).

StateFile (string, default "${SpoolDir}/state.db")

File storing the current broctl state.

StaticDir (string, default "${BroBase}/share/broctl")

Directory for static, arch-independent files.

StatsDir (string, default "${LogDir}/stats")

Directory where statistics are kept.

StatsLog (string, default "${SpoolDir}/stats.log")

Log file for statistics.

Time (string, default _empty_)

Path to time binary.

TmpDir (string, default "${SpoolDir}/tmp")

Directory for temporary data.

TmpExecDir (string, default "${SpoolDir}/tmp")

Directory where binaries are copied before execution. This option is ignored if HaveNFS is 0.

TraceSummary (string, default "${bindir}/trace-summary")

Path to trace-summary script (empty if not available). Make this string blank to disable the connection summary reports.

Version (string, default _empty_)

Version of the broctl.

BroControl provides a plugin interface to extend its functionality. A plugin is written in Python and can do any, or all, of the following:

• Perform actions before or after any of the standard BroControl commands is executed. When running before the actual command, it can filter which nodes to operate or stop the execution altogether. When running after the command, it gets access to the commands success status on a per-node basis (where applicable).
• Add custom commands to BroControl.
• Add custom options to BroControl defined in broctl.cfg.
• Add custom keys to nodes defined in node.cfg.

A plugin is written by deriving a new class from BroControl class Plugin. The Python script with the new plugin is then copied into a plugin directory searched by BroControl at startup. By default, BroControl searches <prefix>/lib/broctl/plugins; additional directories may be configured by setting the SitePluginPath option. Note that any plugin script must end in *.py to be found. BroControl comes with some example plugins that can be used as a starting point; see the <prefix>/lib/broctl/plugins directory.

In the following, we document the API that is available to plugins. A plugin must be derived from the Plugin class, and can use its methods as well as those of the Node class.

class Plugin

The class Plugin is the base class for all BroControl plugins.

The class has a number of methods for plugins to override, and every plugin must at least override name() and pluginVersion().

For each BroControl command foo, there are two methods, cmd_foo_pre and cmd_foo_post, that are called just before the command is executed and just after it has finished, respectively. The arguments these methods receive correspond to their command-line parameters, and are further documented below.

The cmd_<XXX>_pre methods have the ability to prevent the command's execution, either completely or partially for those commands that take nodes as parameters. In the latter case, the method receives a list of nodes that the command is to be run on, and it can filter that list and returns modified version of nodes to actually use. The standard case would be returning simply the unmodified nodes parameter. To completely block the command's execution, return an empty list. To just not execute the command for a subset, remove the affected ones. For commands that do not receive nodes as arguments, the return value is interpreted as boolean indicating whether command execution should proceed (True) or not (False).

The cmd_<XXX>_post methods likewise receive the commands arguments as their parameter, as documented below. For commands taking nodes, the list corresponds to those nodes for which the command was actually executed (i.e., after any cmd_<XXX>_pre filtering).

Note that if a plugin prevents a command from executing either completely or partially, it should report its reason via the message() or error() methods.

If multiple plugins hook into the same command, all their cmd_<XXX>_{pre,post} are executed in undefined order. The command is executed on the intersection of all cmd_<XXX>_pre results.

Finally, note that the restart command is just a combination of other commands and thus their callbacks are run in addition to the callbacks for restart.

debug (self, msg)

Logs a debug message in BroControl's debug log if enabled.

error (self, msg)

Reports an error to the user.

execute (self, node, cmd)

Executes a command on the host for the given node of type Node. Returns a tuple (success, output) in which success is True if the command ran successfully and output is the combined stdout/stderr output.

executeParallel (self, cmds)

Executes a set of commands in parallel on multiple hosts. cmds is a list of tuples (node, cmd), in which the node is a Node instance and cmd is a string with the command to execute for it. The method returns a list of tuples (node, success, output), in which success is True if the command ran successfully and output is the combined stdout/stderr output for the corresponding node.

getGlobalOption (self, name)

Returns the value of the global BroControl option or state attribute name. If the user has not set the options, its default value is returned. See the output of broctl config for a complete list.

getOption (self, name)

Returns the value of one of the plugin's options, name.

An option has a default value (see options()), which can be overridden by a user in broctl.cfg. An option's value cannot be changed by the plugin.

getState (self, name)

Returns the current value of one of the plugin's state variables, name. The returned value will always be a string. If it has not yet been set, an empty string will be returned.

Different from options, state variables can be set by the plugin. They are persistent across restarts.

Note that a plugin cannot query any global BroControl state variables.

hosts (self, nodes)

Returns a list of Node objects which is a subset of the list in nodes, such that only one node per host will be chosen. If nodes is empty, then the returned list will be a subset of the entire list of configured nodes.

message (self, msg)

Reports a message to the user.

nodes (self)

Returns a list of all configured Node objects.

parseNodes (self, names)

Returns a tuple which contains two lists. The first list is a list of Node objects for a string of space-separated node names. If a name does not correspond to a known node, then the name is added to the second list in the returned tuple.

setState (self, name, value)

Sets one of the plugin's state variables, name, to value. value must be a string. The change is permanent and will be recorded to disk.

Note that a plugin cannot change any global BroControl state variables.

broProcessDied (self, node)

Called when BroControl finds the Bro process for Node node to have terminated unexpectedly. This method will be called just before BroControl prepares the node's "crash report" and before it cleans up the node's spool directory.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_capstats_post (self, nodes, interval)

Called just after the capstats command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_capstats_pre (self, nodes, interval)

Called just before the capstats command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command. interval is an integer with the measurement interval in seconds.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_check_post (self, results)

Called just after the check command has finished. It receives the list of 2-tuples (node, bool) indicating the nodes the command was executed for, along with their success status.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_check_pre (self, nodes)

Called just before the check command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_cleanup_post (self, nodes, all)

Called just after the cleanup command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_cleanup_pre (self, nodes, all)

Called just before the cleanup command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command. all is boolean indicating whether the --all argument has been given.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_config_post (self)

Called just after the config command has finished.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_config_pre (self)

Called just before the config command is run. Returns a boolean indicating whether or not the command should run.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_cron_post (self, arg, watch)

Called just after the cron command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_cron_pre (self, arg, watch)

Called just before the cron command is run. arg is an empty string if the command is executed without arguments. Otherwise, it is one of the strings: enable, disable, ?. watch is a boolean indicating whether the cron command should restart abnormally terminated Bro processes; it's only valid if arg is empty.

Returns a boolean indicating whether or not the cron command should run.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_custom (self, cmd, args, cmdout)

Called when a command defined by the commands method is executed. cmd is the command (without the plugin's prefix), and args is a single string with all arguments. It returns a CmdResult object containing the command results.

If the arguments are actually node names, parseNodes can be used to get the Node objects.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_deploy_post (self)

Called just after the deploy command has finished.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_deploy_pre (self)

Called just before the deploy command is run. Returns a boolean indicating whether or not the command should run.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_df_post (self, nodes)

Called just after the df command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_df_pre (self, nodes)

Called just before the df command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_diag_post (self, nodes)

Called just after the diag command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_diag_pre (self, nodes)

Called just before the diag command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_exec_post (self, cmdline)

Called just after the exec command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_exec_pre (self, cmdline)

Called just before the exec command is run. cmdline is a string with the command line to execute.

Returns a boolean indicating whether or not the exec command should run.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_install_post (self)

Called just after the install command has finished.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_install_pre (self)

Called just before the install command is run. Returns a boolean indicating whether or not the command should run.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_netstats_post (self, nodes)

Called just after the netstats command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_netstats_pre (self, nodes)

Called just before the netstats command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_nodes_post (self)

Called just after the nodes command has finished.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_nodes_pre (self)

Called just before the nodes command is run. Returns a boolean indicating whether or not the command should run.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_peerstatus_post (self, nodes)

Called just after the peerstatus command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_peerstatus_pre (self, nodes)

Called just before the peerstatus command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_print_post (self, nodes, id)

Called just after the print command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_print_pre (self, nodes, id)

Called just before the print command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command. id is a string with the name of the ID to be printed.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_process_post (self, trace, options, scripts, success)

Called just after the process command has finished. Arguments are as with the pre method, plus an additional boolean success indicating whether Bro terminated normally.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_process_pre (self, trace, options, scripts)

Called just before the process command is run. It receives the trace to read from as a string, a list of additional Bro options, and a list of additional Bro scripts.

Returns a boolean indicating whether or not the process command should run.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_restart_post (self, nodes)

Called just after the restart command has finished. It receives a list of nodes indicating the nodes on which the command was executed.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_restart_pre (self, nodes, clean)

Called just before the restart command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command. clean is boolean indicating whether the --clean argument has been given.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_scripts_post (self, nodes, check)

Called just after the scripts command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_scripts_pre (self, nodes, check)

Called just before the scripts command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command. check is boolean indicating whether the -c option was given.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_start_post (self, results)

Called just after the start command has finished. It receives the list of 2-tuples (node, bool) indicating the nodes the command was executed for, along with their success status.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_start_pre (self, nodes)

Called just before the start command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_status_post (self, nodes)

Called just after the status command has finished. Arguments are as with the pre method.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_status_pre (self, nodes)

Called just before the status command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_stop_post (self, results)

Called just after the stop command has finished. It receives the list of 2-tuples (node, bool) indicating the nodes the command was executed for, along with their success status.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_stop_pre (self, nodes)

Called just before the stop command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_top_post (self, nodes)

Called just after the top command has finished. Arguments are as with the pre method. Note that when top is run interactively to auto-refresh continuously, this method will be called once after each update.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_top_pre (self, nodes)

Called just before the top command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command. Note that when top is run interactively to auto-refresh continuously, this method will be called once before each update.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_update_post (self, results)

Called just after the update command has finished. It receives the list of 2-tuples (node, bool) indicating the nodes the command was executed for, along with their success status.

This method can be overridden by derived classes. The default implementation does nothing.

cmd_update_pre (self, nodes)

Called just before the update command is run. It receives the list of nodes, and returns the list of nodes that should proceed with the command.

This method can be overridden by derived classes. The default implementation does nothing.

commands (self)

Returns a set of custom commands provided by the plugin.

The return value is a list of 3-tuples each having the following elements:

command

A string with the command's name. Note that the command name exposed to the user will be prefixed with the plugin's prefix as returned by prefix() (e.g., myplugin.mycommand).

arguments

A string describing the command's arguments in a textual form suitable for use in the help command summary (e.g., [<nodes>] for a command taking an optional list of nodes). Empty if no arguments are expected.

description

A string with a description of the command's semantics suitable for use in the help command summary.

This method can be overridden by derived classes. The implementation must not call the parent class' implementation. The default implementation returns an empty list.

done (self)

Called once just before BroControl terminates. This method can do any cleanup the plugin may require.

This method can be overridden by derived classes. The default implementation does nothing.

hostStatusChanged (self, host, status)

Called when BroControl's cron command finds the availability of a cluster system to have changed. Initially, all systems are assumed to be up and running. Once BroControl notices that a system isn't responding (defined as not accepting SSH sessions), it calls this method, passing in a string with the name of the host and a boolean status set to False. Once the host becomes available again, the method will be called again for the same host with status now set to True.

Note that BroControl's cron tracks a host's availability across execution, so if the next time it's run the host is still down, this method will not be called again.

This method can be overridden by derived classes. The default implementation does nothing.

init (self)

Called once just before BroControl starts executing any commands. This method can do any initialization that the plugin may require.

Note that when this method executes, BroControl guarantees that all internals are fully set up (e.g., user-defined options are available). This may not be the case when the class __init__ method runs.

Returns a boolean, indicating whether the plugin should be used. If it returns False, the plugin will be removed and no other methods called.

This method can be overridden by derived classes. The default implementation always returns True.

name (self)

Returns a string with a descriptive name for the plugin (e.g., "TestPlugin"). The name must not contain any whitespace.

This method must be overridden by derived classes. The implementation must not call the parent class' implementation.

nodeKeys (self)

Returns a list of names of custom keys (the value of a key can be specified in node.cfg for any node defined there). The value for a key will be available from the Node object as attribute <prefix>_<key> (e.g., node.myplugin_mykey). If not set, the attribute will be set to an empty string.

This method can be overridden by derived classes. The implementation must not call the parent class' implementation. The default implementation returns an empty list.

options (self)

Returns a set of local configuration options provided by the plugin.

The return value is a list of 4-tuples each having the following elements:

name

A string with name of the option (e.g., Path). Option names are case-insensitive. Note that the option name exposed to the user will be prefixed with your plugin's prefix as returned by prefix() (e.g., myplugin.Path).

type

A string with type of the option, which must be one of "bool", "string", or "int".

default

The option's default value. Note that this value must be enclosed in quotes if the type is "string", and must not be enclosed in quotes if the type is not "string".

description

A string with a description of the option semantics.

This method can be overridden by derived classes. The implementation must not call the parent class' implementation. The default implementation returns an empty list.

pluginVersion (self)

Returns an integer with a version number for the plugin. Plugins should increase their version number with any significant change.

This method must be overridden by derived classes. The implementation must not call the parent class' implementation.

prefix (self)

Returns a string with a prefix for the plugin's options and commands names (e.g., "myplugin").

This method can be overridden by derived classes. The implementation must not call the parent class' implementation. The default implementation returns a lower-cased version of name().

class Node

Class representing one node of the BroControl maintained setup. In standalone mode, there's always exactly one node of type standalone. In a cluster setup, there is exactly one of type manager, one or more of type proxy, and zero or more of type worker.

A Node object has a number of keys with values that are set via the node.cfg file and can be accessed directly (from a plugin) via corresponding Python attributes (e.g., node.name):

name (string)

The name of the node, which corresponds to the [<name>] section in node.cfg.

type (string)

The type of the node, which will be one of standalone, manager, proxy, and worker.

env_vars (string)

A comma-separated list of environment variables to set when running Bro (e.g., env_vars=VAR1=1,VAR2=2). These node-specific values override any global values specified in the broctl.cfg file.

host (string)

The hostname (or the IP address) of the system the node is running on.

interface (string)

The network interface for Bro to use; empty if not set.

lb_procs (integer)

The number of clustered Bro workers you'd like to start up. This number must be greater than zero.

lb_method (string)

The load balancing method to distribute packets to all of the processes. This must be one of: pf_ring, myricom, custom, or interfaces.

lb_interfaces (string)

If the load balancing method is interfaces, then this is a comma-separated list of network interface names to use.

pin_cpus (string)

A comma-separated list of CPU numbers to which the node's Bro processes will be pinned (if not specified, then CPU pinning will not be used for this node). This option is only supported on Linux and FreeBSD (it is ignored on all other platforms). CPU numbering starts at zero (e.g., the only valid CPU numbers for a machine with one dual-core processor would be 0 and 1). If the length of this list does not match the number of Bro processes for this node, then some CPUs could have zero (if too many CPU numbers are specified) or more than one (if not enough CPU numbers are specified) Bro processes pinned to them. Only the specified CPU numbers will be used, regardless of whether additional CPU cores exist.

aux_scripts (string)

Any node-specific Bro script configured for this node.

zone_id (string)

If BroControl is managing a cluster comprised of nodes using non-global IPv6 addresses, then this configures the RFC 4007 zone_id string that the node associates with the common zone that all cluster nodes are a part of. This identifier may differ between nodes.

Any attribute that is not defined in node.cfg will be empty.

In addition, plugins can override Plugin.nodeKeys to define their own node keys, which can then be likewise set in node.cfg. The key names will be prepended with the plugin's Plugin.prefix (e.g., for the plugin test, the node key foo is set by adding test.foo=value to node.cfg).

Finally, a Node object has the following methods that can be called from a plugin:

cwd (self)

Returns a string with the node's working directory.

describe (self)

Returns an extended string representation of the node including all its keys with values (sorted by key).

getPID (self)

Returns the process ID of the node's Bro process if running, and None otherwise.

getPort (self)

Returns an integer with the port that this node's communication system is listening on for incoming connections, or -1 if no such port has been set yet.

hasCrashed (self)

Returns True if the node's Bro process has exited abnormally.

Can I use an NFS-mounted partition as the cluster's base directory to avoid the ``rsync``'ing?

Yes. BroBase can be on an NFS partition. Configure and install the shell as usual with --prefix=<BroBase>. Then add HaveNFS=1 and SpoolDir=<spath> to broctl.cfg, where <spath> is a path on the local disks of the nodes; <spath> will be used for all non-shared data (make sure that the parent directory exists and is writable on all nodes!). Then run make install again. Finally, you can remove <BroBase>/spool (or link it to <spath>). In addition, you might want to keep the log files locally on the nodes as well by setting LogDir to a non-NFS directory. (Only the manager's logs will be kept permanently, the logs of workers/proxies are discarded upon rotation.)

What do I need to do when something in the Bro distribution changes?

After pulling from the main Bro git repository, just re-run make install inside your build directory. It will reinstall all the files from the distribution that are not up-to-date. Then do broctl deploy to make sure everything gets pushed out.

Can I change the naming scheme that BroControl uses for archived log files?

Yes, set MakeArchiveName to a script that outputs the desired destination file name for an archived log file. The default script for that task is <BroBase>/share/broctl/scripts/make-archive-name, which you can use as a template for creating your own version. See the beginning of that script for instructions.

Can BroControl manage a cluster of nodes over non-global IPv6 scope (e.g. link-local)?

Yes, set ZoneID in broctl.cfg to the zone identifier that the BroControl node uses to identify the scope zone (the ifconfig command output is usually helpful, if it doesn't show the zone identifier appended to the address with a '%' character, then it may just be the interface name). Then in node.cfg, add a zone_id key to each node section representing that particular node's zone identifier and set the host key to the IPv6 address assigned to the node within the scope zone. Most nodes probably have the same zone_id, but may not if their interface configuration differs. See RFC 4007 for more information on IPv6 scoped addresses and zones.