The name of this talk is Elixir Testing, I know that the title sounds grand, Testing as a whole is extremely broad - I know that I won’t be able to scratch the surface on what ExUnit is capable of doing but for the most part the goal of the talk is get you generally acquainted with the library's capabilities and share some tips from the frontlines.

Disclaimer This not a Talk on TDD but I may draw on some concepts accidentally.

Like a lot of developers testing was not something I would think to do right off the bat when I approach building a new application or learning a new language. But after much painful learning here are some reasons why I feel its important think of testing upfront.

1. Avoiding regression.

Breaking changes happen. You need something to keep you in check.

2. Ease of Refactoring.

If you do decide to change some already working code to something more elegant. Your tests are your guide to success.

3. Building Confidence in your code and as a developer.

While your trying to get to a rough draft on a project if you have tests you will find your self feeling good about improving any code without breaking your mvp. You will feel confident about stopping points. Who cares how the code looks its tested.

As stated in the docs - It is Elixir’s Unit Testing framework - but what about Acceptance , Integration tests ? We’ll get to that.

ExUnit includes everything we need to thoroughly test our code.

Test Scripts

These are the mininum requirements to get ExUnit up and running.

In your script you need: You create a file it must end with _test.exs

ExUnit.start() - starts the process

ExUnit.configure() - Loads in configuration.

Defining your test scenario usually follows this pattern.

defmodule MyTestModuleTest do 
	use ExUnit.Case
	test “name of test”  do 
		logic and assertions
	end
end

Elixir Script example:

you can run this file via $> elixir path/to/filename_test.exs

When ExUnit.start() is called.

• The main ExUnit Process is started

• The process will read in the file or files _test.exs and start create an individual struct - ExUnit.TestModule for each testmodule you have defined and populate the tests attribute on that module with an ExUnit.Test struct for eac test defined in your module.

• ExUnit.Test ExUnit will convert the struct into an function. What this function returns - specifies whether a test is a passing test or failing test.

ExUnit.start Application

In the world of Testing there is a lot of terminology. Generally speaking.

A Test Scenario is any functionality that can be tested. This consists of a detailed Test procedure. Made up of TestCases.

A TestCases are sets of steps which are performed on a system to verify the expected output.

How are those ExUnit.Test structs created at runtime - for that we have to take a look at the test macro.

Lets look at the ExUnit definitions for commonly used macros

register_test a function called internally

The gist of what's happening is the test macro defines a function on the module the macro was used.

The registering of that test - this function does a couple things it checks that a function by that name was not already implemented and builds the environment in which that function should run.

  # Place this before the last end in your TestModule 
  # tests are converted to functions.
  
  IO.inspect Module.definitions_in(HelloWorldTest, :def)

Every describe block receives a name which is used as prefix for upcoming tests. Inside a block, ExUnit.Callbacks.setup/1 may be invoked and it will define a setup callback to run only for the current block. The describe name is also added as a tag, allowing developers to run tests for specific blocks.

The world surrounding your Test.

Tagged test Example

Context data helps each test run in specific scope. We can use tags or callbacks to help get data required to test how our functionality works when the variables have changed.

@tag attributes are placed just above the implementation the test macro. They are used to pass data from the test to the callback.

@moduletag attribute is placed at the top of the file within your TestModule. They are used to define some arbitrary data that you would want already set for all tests or a describe block.

setup callback - is run in the same process as the test itself.

setup_all callback - is run in a seperate process per module.

In order to understand callbacks it might be good to understand what's happening. Here is a run down of the life-cycle of the test process:

• the test process is spawned
• it runs setup/2 callbacks
• it runs the test itself
• it stops all supervised processes
• the test process exits with reason :shutdown
• on_exit/2 callbacks are executed in a separate process

These macros help ExUnit mimic conditions under which functionality is called to test for desired result.

ExUnit Context Tests * Note that ExUnit itself is tested with ExUnit.

Great examples and Introduction

assert

refute

• run test once again.

This not how majority of elixir apps are setup - they are all some form of mix application.

mix is Elixir's go to build tool for app management. It comes pre built with testing functionality. Its not tied specifically to ExUnit. If there is any other testing framework you can simply drop it in and write your tests in it.

When you create an new app - A test folder already setup with a test_helper.exs file in it. You can put some logic in this file but mostly it's there for mission critical directives such as

ExUnit.start() or Application.ensure_all_started(:your_fav_testing_helper_app)

In a new mix app. Everything is already preconfigured.

mix test

mix help test

Quick write up with mix testing tips

mix test

mix test path_to_your_file.exs:10

mix test path_to_your_file.exs

@tag runnable: true
test ....

mix test --include runnable:true

mix test --only describe:"Block A"

mix test --only describe:"PMap"

mix test --slowest <N> Added in Elixir 1.6 - * find the slowest tests.

# mix.exs

  ...
  
  defp deps do
    [{:test_package, only: [:dev, :test]}]
  end

  defp aliases do
    [
      test: ["ecto.drop", "ecto.create", "ecto.migrate", "test"],
      foobar: ["run -e 'IO.puts(\"One\")'", "run -e 'Mix.Task.reenable(:run)'"]
    ]
  end

In summary Mix is a elixir's build has a lot of niceties that help you manage your tests. I encourage you to read up more - just look at the docs.

How do we define Boundaries? - feedback.

Slides

Talk

I would encourage watching it. From my experience developing phoenix 1.3 apps I can attest the benefits of Contexts and how they have influenced my testing.

Doing Context Mapping generally leads to the creation of better tests. Context mapping helps can identify the system boundaries, which intern help you create better API’s for integration testing. Creating other forms of tests benefit from this practice as well.

What we have access in Elixir by virtue Erlang is the ability to build a lot of isolated functional cores that communicate with each other. We encapsulate this functionality in processes. How do we ensure these processes we create are behaving correctly. How do we test them.

Here is an example of testing a module with GenServer behaviour.

GenServer Test

Module with GenServer Behaviour

Disclaimer: Testing in Phoenix a topic that warrants its own talk. This is just an overview.

Guess what - Phoenix is a just another mix application. Phoenix includes modules that make use of ExUnit.CaseTemplate to group functionality for testing things built from different parts of the framework.

ExUnit.CaseTemplate

This module allows a developer to define a test case template to be used throughout their tests. This is useful when there are a set of functions that should be shared between tests or a set of setup callbacks.

Because of the nature of Phoenix and the different moving parts - Contexts, Controllers, and Views. Phoenix makes use of this module to section off test groups with shared functionality.

ConnCase Example

For Example if your using fixtures. Your tests for your Phoenix Controllers dont need to include those functions. The test modules you define that test your Phoenix Contexts probably do.

Great Fixture Ecto writeup

Example of a Context Case pulled from an app I worked on.

--

Integration Testing and Phoenix.

Phoenix already makes it easy to write tests for your controllers. Hound makes writing Integration tests easy by runnning phoenix that run in a headless browser.

You can put this in your mix.exs {:hound, "~> 1.0", only: [:dev, :test]}

Ensure that the application starts when by putting Application.ensure_all_started(:hound) in test_helper.exs

Excellent Hound Tutorial

Readup on Phoenix.ChannelTest

Readup on Phoenix.ConnTest

Elixir LDN - Ecto Shared Process explanation Tagged Youtube Video at mark where speaker begins explaining shared processes. Issues with Ecto - are solely when you dont set things up correctly.

An additional explanation on the same topic of Shared processes. Making Sense of Ecto 2 SQL.Sandbox and Connection Ownership Modes

setup do
   :ok = Ecto.Adapters.SQL.Sandbox.checkout(Repo)
    Ecto.Adapters.SQL.Sandbox.mode(Repo, {:shared, self()})
end

Honorable Mentions

Bypass provides a quick way to create a custom plug that can be put in place instead of an actual HTTP server to return prebaked responses to client requests. This is most useful in tests, when you want to create a mock HTTP server and test how your HTTP client handles different types of responses from the server.

It elimenates the need for Adapter Pattern / Testing - the need to define condtionals in your code based on the environment your running in.

Example Bypass Tests

ExVCR

• Doc Tests - pretty straight forward.

• Static Types with regards to Testing.

Testing Mix Tasks

Hex Package: Mix Test Watch

Setup: Mix Test Watch

mix test.watch

semaphoreci ExUnit intro

semaphoreci testing streams

Conclusion

You may be new to a language - but you may not be new to a domain. Knowing how to write tests should be one of the initial things you learn while gettin your mvp off the ground.

One of the main goals of testing your code is gaining assurances. Now me coming from the the ruby world where things can get mutated on the fly. Assurance was key to my sanity. In Elixir I found that I did not have that problem not to say that you can’t get things wrong but its much easier to follow how your passing the data along to see where you the issue lies.

#Always Read the Docs