A test suite for Sass. The test cases are all in the /spec
folder.
Sass spec is written in ruby, so you will need to have ruby and bundler installed in order to run it.
Run tests against Ruby Sass with the sass-spec.rb
file in the root directory.
$ git clone https://github.com/sass/sass-spec.git
$ cd sass-spec
$ bundle install
$ bundle exec sass-spec.rb
To run tests against Ruby Sass in development, edit the Gemfile and add:
gem 'sass', :path => "/path/to/sass/directory"
Then run bundle install
and bundle exec sass-spec.rb
will run
against your development version of ruby sass.
Conversely, if you edit the Sass Gemfile
and set gem 'sass-spec', :path => "..."
then the Sass unit tests will run against your
development version of sass-spec.
Full text help is available if you run that w/ the -h
option.
Tests are stored in the spec
directory. The structure of the specs is
being worked on right now, however, most directory names should be
fairly self explanatory.
Spec tests each apply to their own range of Sass Language versions. Each folder in the spec directory can have start and end language versions set and it will apply to all of the tests contained in that folder and below. Individual tests can override these settings.
Use the annotate subcommand to annotate and report on annotations
applied to individual test cases. Run ./sass-spec.rb annotate -h
to
see all available options.
Examples:
$ ./sass-spec.rb annotate --end-version 3.5 spec/basic
spec/basic:
* setting end_version to 3.5...done
$ ./sass-spec.rb annotate --end-version 4.0 spec/basic/40_pseudo_class_identifier_starting_with_n
spec/basic:
* setting end_version to 4.0...done
$ ./sass-spec.rb annotate --report spec/basic
+-------------------------------------------------------+-----+-----+-----+
| Test Case | 3.4 | 3.5 | 4.0 |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/00_empty | ✓ | ✓ | |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/01_simple_css | ✓ | ✓ | |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/02_simple_nesting | ✓ | ✓ | |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/03_simple_variable | ✓ | ✓ | |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/04_basic_variables | ✓ | ✓ | |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/... | ✓ | ✓ | |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/40_pseudo_class_identifier_starting_with_n | ✓ | ✓ | ✓ |
+-------------------------------------------------------+-----+-----+-----+
| spec/basic/... | ✓ | ✓ | |
+-------------------------------------------------------+-----+-----+-----+
When running the tests, it is important to specify which language version subset of the tests should be used. When not specified, the latest version is used.
$ ./sass-spec.rb -V 3.4 ...
- Set up sass spec if you haven't yet.
- Add an
input.s[ac]ss
file in an appropriate folder. - Optionally, annotate it with sass-spec.rb annotate ... <path_to_folder>
- Run
sass-spec.rb -g <path_to_folder>
to generate the expected output files. - Verify the generated output is what you expected.
- Run
sass-spec.rb <path_to_folder>
just to make sure it passes. - Commit and send Pull Request. Be sure to include the reason for the new spec in the commit message.
A lot of the management tasks for specs is centered around how to handle
specs as the language changes. Many common fixes for failing tests can
be found by running tests with the --interactive
command line option.
When a test would fail, it first stops, lets you see what's failing and choose a fix for it or you can let it fail and fix it later.
Sometimes, many tests are all failing and you know they need to be
updated en masse and interactive mode would be very cumbersome in this
context. In these situations the --migrate
option or the --generate
option are very useful.
The --generate
option causes all tests that are being ran to have
their expected output, error and exit statuses updated to match the
current results. For passing tests, this operation is a net result of
not changing files.
In --interactive
mode, a common option is to regenerate the expected
outputs just like --generate
does, but on a case-by-case basis.
The migrate option only works on tests that are failing.
- Make a copy of the current test named "<folder>-<current_version>"
- Mark the original test as having an
end_version
as the version that comes before the version being tested right now. - Set the copy as having a
start_version
as the version being tested right now. - Regenerate the expected output for the new test so that it passes.
In --interactive
mode, a common option is to migrate the spec just
like --migrate
does, but on a case-by-case basis.
If a test or folder of tests is pending for a particular implementation, you can mark that test as pending for just that implementation.
$ ./sass-spec.rb annotate --pending libsass spec/awesome_new_feature/
Then those tests will be marked as skipped if you run sass-spec and pass
the --impl NAME
option (E.g. in this case --impl libsass
)
The --interactive
mode will provide marking a test as pending for the
current implementation as a remedy for many types of spec failures.
- Ruby 2.1.0 contained a regression that changed the order of some selectors, causing test failures in sass-spec. That was fixed in Ruby 2.1.1. If you're running sass-spec against a Ruby Sass, please be sure not to use Ruby 2.1.0.
- Some of our spec files have UTF-8 characters in their filenames. If
you are on OSX, you may encounter issues with git showing files that are
not checked in but actually are which can cause issued which doing
rebase or changing branches. If so, run
git config core.precomposeunicode false
and it should clear things up.
After installing a libsass dev enviroment (see libsass readme... sassc, this spec, and libsass), the tests are run by going to the libsass folder and running ./script/spec.
This project needs maintainers! There will be an ongoing process of simplifying test cases, reporting new issues and testing them here, and managing mergers of official test cases.
This project requires help with the Ruby test drivers (better output, detection modes, etc) AND just with managing the issues and writing test cases.