Key Features • Installation • Coverage Report • Advanced Config • License • Change Log / Roadmap • Code of Conduct
A gem to measure production code usage, showing a counter for the number of times each line of code that is executed. Coverband allows easy configuration to collect and report on production code usage. It reports in the background via a thread or can be used as Rack middleware, or manually configured to meet any need.
Note: Coverband is not intended for test code coverage, for that we recommended using SimpleCov.
The primary goal of Coverband is giving deep insight into your production runtime usage of your application code, while having the least impact on performance possible.
- Low performance overhead
- Simple setup and configuration
- Out of the box support for all standard code execution paths (web, cron, background jobs, rake tasks, etc)
- Splits load time (Rails eager load) and Run time metrics
- Easy to understand actionable insights from the report
- Tracks Gem usage (still in experimental stages and not recommended for production)
- Development mode, offers deep insight of code usage details (number of LOC execution during single request, etc) during development.
- Mountable web interface to easily share reports
Coverband stores coverage data in Redis. The Redis endpoint is looked for in this order:
ENV['COVERBAND_REDIS_URL']
ENV['REDIS_URL']
localhost
The redis store can also be explicitly defined within the coverband.rb. See advanced config.
Add this line to your application's Gemfile
, remember to bundle install
after updating:
gem 'coverband'
If tracking gem usage, be sure to include coverband before other gems you would like to track.
With older versions of coverband, projects would report to redis using rack or sidekiq middleware. After coverband 4.0, this should no longer be required and could cause performance issues. Reporting to redis is now automatically done within a background thread with no custom code needed.
See changelog.
The Railtie integration means you shouldn't need to do anything else other than ensure coverband is required after rails within your Gemfile. The only exception to this is gem tracking of Bundle.require
which depends on requiring coverband within the application.rb. See Collecting Gem / Library Usage.
For the best coverage you want this loaded as early as possible. I have been putting it directly in my config.ru
but you could use an initializer, though you may end up missing some boot up coverage. To start collection require Coverband as early as possible.
require 'coverband'
require File.dirname(__FILE__) + '/config/environment'
use Coverband::BackgroundMiddleware
run ActionController::Dispatcher.new
Coverband comes with a mountable rack app for viewing reports. For Rails this can be done in config/routes.rb
with:
Rails.application.routes.draw do
mount Coverband::Reporters::Web.new, at: '/coverage'
end
But don't forget to protect your source code with proper authentication. Something like this when using devise:
Rails.application.routes.draw do
authenticate :user, lambda { |u| u.admin? } do
mount Coverband::Reporters::Web.new, at: '/coverage'
end
end
or you can enable basic auth by setting ENV['COVERBAND_PASSWORD']
or via your configuration config.password = 'my_pass'
The web endpoint is a barebones endpoint that you can either expose direct (after authentication) or you can just link to the actions you wish to expose. The index is intended as a example to showcase all the features.
The web index as available on the Coverband Demo site
- force coverage collection: This triggers coverage collection on the current webserver process
- clear coverage report: This will clear the coverage data. This wipes out all collected data (dangerous)
- View individual file details
- clear individual file coverage: This will clear the details of the file you are looking at. This is helpful if you don't want to lose all Coverage data but made a change that you expect would impact a particular file.
The rake task generates a report locally and opens a browser pointing to coverage/index.html
.
rake coverband:coverage
This is mostly useful in your local development environment.
Since Coverband is Simplecov output compatible it should work with any of the SimpleCov::Formatter
's available. The output below is produced using the default Simplecov HTML formatter.
Details on an example Sinatra app
Take Coverband for a spin on the live Heroku deployed Coverband Demo. The full source code for the demo is available to help with installation, configuration, and understanding of basic usage.
If you need to configure coverband, this can be done by creating a config/coverband.rb
file relative to your project root.
- See lib/coverband/configuration.rb for all options
- By default Coverband will try to stored data to Redis * Redis endpoint is looked for in this order:
ENV['COVERBAND_REDIS_URL']
,ENV['REDIS_URL']
, orlocalhost
Below is an example config file for a Rails 5 app:
#config/coverband.rb
Coverband.configure do |config|
config.store = Coverband::Adapters::RedisStore.new(Redis.new(url: ENV['MY_REDIS_URL']))
config.logger = Rails.logger
# configure S3 integration
config.s3_bucket = 'coverband-demo'
config.s3_region = 'us-east-1'
config.s3_access_key_id = ENV['AWS_ACCESS_KEY_ID']
config.s3_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
# config options false, true. (defaults to false)
# true and debug can give helpful and interesting code usage information
# and is safe to use if one is investigating issues in production, but it will slightly
# hit perf.
config.verbose = false
# default false. button at the top of the web interface which clears all data
config.web_enable_clear = true
# default false. Experimental support for tracking view layer tracking.
# Does not track line-level usage, only indicates if an entire file
# is used or not.
config.track_views = true
end
Do you use figaro, mc-settings, dotenv or something else to inject environment variables into your app? If so ensure you have that done BEFORE coverband is required.
For example if you use dotenv, you need to do this, see https://github.com/bkeepers/dotenv#note-on-load-order
gem 'dotenv-rails', require: 'dotenv/rails-now'
gem 'coverband'
gem 'other-gem-that-requires-env-variables'
Sometimes you have files that are known to be valuable perhaps in other environments or something that is just run very infrequently. Opposed to having to mentally filter them out of the report, you can just have them ignored in the Coverband reporting by using config.ignore
as shown below. Ignore takes a string but can also match with regex rules see how below ignores all rake tasks as an example.
config.ignore += ['config/application.rb',
'config/boot.rb',
'config/puma.rb',
'config/schedule.rb',
'bin/*',
'config/environments/*',
'lib/tasks/*']
Ignoring Custom Gem Locations: Note, if you have your gems in a custom location under your app folder you likely want to add them to config.ignore
. For example, if you have your gems not in a default ignored location of app/vendor
but have them in app/gems
you would need to add gems/*
to your ignore list.
Coverband allows an optional feature to track all view files that are used by an application.
To opt-in to this feature... enable the feature in your Coverband config.
config.track_views = true
If you add some additional Coverband configuration your coverage html report will be written directly to S3, update config/coverband.rb
like below.
# configure S3 integration
config.s3_bucket = 'coverband-demo'
config.s3_region = 'us-east-1'
config.s3_access_key_id = ENV['AWS_ACCESS_KEY_ID']
config.s3_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
Alternatively, Coverband if you don't set via the config.s3_*
accessor methods will look for the standard S3 environment variables.
ENV['AWS_BUCKET']
ENV['AWS_REGION']
ENV['AWS_ACCESS_KEY_ID']
ENV['AWS_SECRET_ACCESS_KEY']
If all your coverage is being counted as loading or eager_loading coverage, and nothing is showing as runtime Coverage the initialization hook failed for some reason. The most likely reason for this issue is manually calling eager_load!
on some Plugin/Gem. If you or a plugin is altering the Rails initialization process, you can manually flip Coverband to runtime coverage by calling these two lines, in an after_initialize
block, in application.rb
.
config.after_initialize do
unless Coverband.tasks_to_ignore?
Coverband.report_coverage # record the last of the loading coverage
Coverband.runtime_coverage! # set all future coverage to runtime
end
end
or if you know you are manually calling eager load anywhere in your initialization process immediately adfter call those two lines. A user reported an issue after calling ResqueWeb::Engine.eager_load!
for example.
Rails.application.routes.draw do
ResqueWeb::Engine.eager_load!
Coverband.report_coverage
Coverband.runtime_coverage!
end
If you have many servers and they all hit Redis at the same time you can see spikes in your Redis CPU, and memory. This is do to a concept called cache stampede. It is better to spread out the reporting across your servers. A simple way to do this is to add a random wiggle on your background reporting. This configuration option allows a wiggle. The right amount of wiggle depends on the numbers of servers you have and how willing you are to have delays in your coverage reporting. I would recommend at least 1 second per server.
Add a wiggle (in seconds) to the background thread to avoid all your servers reporting at the same time:
config.reporting_wiggle = 30
Coverband on very high volume sites with many server processes reporting can have a race condition. To resolve the race condition and reduce Ruby memory overhead we have introduced a new Redis storage option. This moves the some of the work from the Ruby processes to Redis. It is worth noting because of this, it has a larger demands on the Redis server. So adjust your Redis instance accordingly. To help reduce the extra redis load you can also change the background reporting time period.
- set the new Redis store:
config.store = Coverband::Adapters::HashRedisStore.new(Redis.new(url: redis_url))
- adjust from default 30s reporting
config.background_reporting_sleep_seconds = 120
- reminder it is recommended to have a unique Redis per workload (background jobs, caching, Coverband), for this store, it may be more important to have a dedicated Redis.
Now that Coverband uses MD5 hashes there should be no reason to manually clear coverage unless one is testing, changing versions, possibly debugging Coverband itself.
rake coverband:clear
Between the release of 4.0 and 4.1 our data format changed. This resets all your coverage data. If you want to restore your previous coverage data, feel free to migrate.
rake coverband:migrate
- We will be working to support migrations going forward, when possible
Rails apps should automaticallly include the tasks via the Railtie.
For non Rails apps, either add the below to your Rakefile
or to a file included in your Rakefile
such as lib/tasks/coverband.rake
if you want to break it up that way.
require 'coverband'
Coverband.configure
require 'coverband/utils/tasks'
Verify it works
rake -T coverband
rake coverband:clear # reset coverband coverage data
rake coverband:coverage # report runtime coverband code coverage
WARNING: Gem Tracking is still in experimental stages and not recommended for production. We have some performance issues when view reports on large applications. Gem tracing also during background thread data collection has HIGH memory requirements, during report merging (seemingly around 128mb of extra memory, which is crazy). We recommend deploying WITHOUT track_gems
first and only enabling it after confirming that Coverband is working and performing well.
Gem usage can be tracked by enabling the track_gems
config.
Coverband.configure do |config|
config.track_gems = true
end
The track_gems
feature exposes a Gems tab in the report which prints out the percentage usage of each Gem. See demo here.
When tracking gems, it is important that Coverband#start
is called before the gems to be tracked are required. The best way to do this is to require coverband before Bundle.require is called. Within rails, require coverband within the application.rb like so:
require 'coverband'
Bundler.require(*Rails.groups)
If you are using the resque integration, resque needs to be required before coverband since the integration will not run unless resque is loaded. Within the application.rb just require resque before coverband.
require 'resque'
require 'coverband'
Bundler.require(*Rails.groups)
The track_gems config only exposes the overall usage of a gem. In order to see the detail of each file, enable the gem_details
flag.
Coverband.configure do |config|
config.track_gems = true
config.gem_details = true
end
This flag exposes line by line usage of gem files. Unfortunately due to the way the coverband report is currently rendered, enabling gem_details
slows down viewing of the coverage report in the browser and is not yet recommended.
Coverband starts on require of the the library which is usually done within the Gemfile. This can be disabled by setting the COVERBAND_DISABLE_AUTO_START
environment variable. This environment variable can be useful to toggle coverband on and off in certain environments.
In order to start coverband manually yourself when this flag is enabled, call Coverband.configure
followed by Coverband.start
.
Coverband.configure
Coverband.start
Note: To debug issues getting Coverband working. I recommend running in development mode, by turning verbose logging on config.verbose = true
and passing in the Rails.logger config.logger = Rails.logger
to the Coverband config. We respect the log level, and I would recommend log level info generally, but if you are investigating a prolbem Coverband logs additional data at the debug
level. This makes it easy to follow in development mode. Be careful to not leave these on in production as they will affect performance.
If you are trying to debug locally wondering what code is being run during a request. The verbose modes config.verbose = true
&& Rails.logger.level = :debug
. With true set it will output the number of lines executed per file, to the passed in log.
- Coverband 3.0.X+ requires Ruby 2.3+
- Coverband currently requires Redis for production usage
We will match Heroku & Ruby's support lifetime, supporting the last 3 major Ruby releases. For details see supported runtimes.
For Rails, we will follow the policy of the Rails team maintenance policy. We officially support the last two major release versions, while providing minimal support (major bugs / security fixes) for an additional version. This means at the moment we primaryly target Rails 6.x, 5.x, and will try to keep current functionality working for Rails 4.x but may release new features that do not work on that target.
If you are working on adding features, PRs, or bugfixes to Coverband this section should help get you going.
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Make sure all tests are passing (run
bundle install
, make sure Redis is running, and then executerake test
) - Create new Pull Request
If you submit a change please make sure the tests and benchmarks are passing.
- run tests:
bundle exec rake
BUNDLE_GEMFILE=Gemfile.rails4 bundle exec rake
(Same tests using rails 4 instead of 5)
- view test coverage:
open coverage/index.html
- run the benchmarks before and after your change to see impact
rake benchmarks
- run a single test by line number like rspec:
bundle exec m test/coverband/reporters/html_test.rb:29
- total fail on front end code, for line for line coverage, because of the precompiled template step basically coverage doesn't work well for
erb
,slim
, and the like.- related it will try to report something, but the line numbers reported for
ERB
files are often off and aren't considered useful. I recommend filtering out .erb using theconfig.ignore
option. The default configuration excludes these files - NOTE: We now have file level coverage for view files, but don't support line level detail
- The view file detection doesn't workf or mailers at the moment only for web related views / JSON templates. This is due to how Rails active mailer notifications work.
- related it will try to report something, but the line numbers reported for
What files have been synced to Redis?
Coverband.configuration.store.covered_files
What is the coverage data in Redis?
Coverband.configuration.store.coverage
The Coverband logo was created by Dave Woodall. Thanks Dave!
This is a MIT License project... See the file license.txt for copying permission.