The following story shows the importance of smoke tests when writing an app using the Ember+Rails stack. It covers the most basic example, and even then things go awry. In addition, it shows an example of an end-to-end test and some of the difficulty in setting up an environment for smoke tests. Finally, it mentions exhaust, a library that can ease this pain.
Imagine, you’re ready to start writing your next great application. It’s like TodoMVC meets Twitter meets eBay: Users will write down their To Do lists and auction off tasks to other users in 140 characters or less. Investors are very excited. You’re very excited. High-fives all around. You’re going to change the world, and you’re going to do that using all the right technologies: Ember for the client side and Rails for the backend!
Here is your first user story:
cucumber Given I am a user When I go to the todos page Then I should see a list of all todos
Getting Started with Ember Rails
Piece of cake! You start by installing the latest Ember:
> npm install -g ember bower > ember new todo-frontend > cd todo-frontend
“I don’t care what @dhh says,” you think out loud. “I’m a rockstar developer, so I’m gonna test-drive it like I stole it!” You don’t have an API to test yet, so you start by mocking it. You install pretender and generate your first acceptance test:
> ember install ember-cli-pretender > ember generate acceptance-test todos
// tests/acceptance/todos.js
import Ember from 'ember';
import { module, test } from 'qunit';
import startApp from 'emberbook/tests/helpers/start-app';
import Pretender from 'pretender';
let TODOS = [
{id: 1, title: "write a blog post"},
{id: 2, title: "let people read it"},
{id: 3, title: "... profit"}
];
module('Acceptance | todos', {
beforeEach: function() {
this.application = startApp();
this.server = new Pretender(function() {
this.get('/todos', function(){
var json = {
todos: TODOS
};
return [200, {}, JSON.stringify(json)];
});
});
},
afterEach: function() {
Ember.run(this.application, 'destroy');
this.server.shutdown();
}
});
test('visiting /todos', function(assert) {
visit('/todos');
andThen(function() {
var title = find('h1');
assert.equal(title.text(), 'Todo List');
var todos = find('.todo-item');
assert.equal(todos.length, TODOS.length);
assert.equal(currentURL(), '/todos');
});
});Time to fire up the test runner and watch it all burn:
ember test --serve
Yep, it’s red like the devil. First error, no route.
// app/router.js
import Ember from 'ember';
import config from './config/environment';
var Router = Ember.Router.extend({
location: config.locationType
});
Router.map(function() {
this.route("todos");
});
export default Router;Now the test can’t find the text “Todo List.” You fix this as well:
{{!- app/templates/todos.hbs }}
# Todo List
{{#each model as |todo|}}
<p class='todo-item'>
{{todo.title}}
</p>
{{/each}}Now the test breaks because there are no To Do items on the page. You move forward:
// app/routes/todos.js
import Ember from 'ember';
export default Ember.Route.extend({
model() {
return this.store.findAll("todo");
}
});Can’t find the “todo” model. One more fix:
// app/models/todo.js
import DS from 'ember-data';
export default DS.Model.extend({
title: DS.attr(),
complete: DS.attr("boolean")
});Whoa, it’s green! You pat yourself on the back. But you can’t actually deliver the story until there’s an API to back it up. Time to grab the latest version of Rails.
Testing an Ember Rails Application
This is going to be an API rather than a full-fledged application, so you only need a few gems. You spend several minutes pondering a post you just read about building Rails API apps and deciding whether to use the Rails 5 --api flag. However, you want this product to hit the market “yesterday,” so you just roll with what’s stable:
> gem install rails > rails new todo_api --skip-test-unit --skip-bundle > cd todo_api
You also skipped test-unit because you’d rather use Rspec’s eloquent DSL for this project’s tests.
Time to boil the Gemfile down to the few gems you really need. This API only needs to serve JSON, so out with anything related to the asset pipeline:
# Gemfile source 'https://rubygems.org' gem 'rails', '4.2.3' gem 'sqlite3' gem 'active_model_serializers' group :development, :test do gem 'rspec-rails' end
Bundle it and get ready to write some specs!
> bundle install > rails generate rspec:install
Time to bang out the first request spec:
# spec/requests/todos_spec.rb
require "rails_helper"
RSpec.describe "Todos", :type => :request do
# letBANG because `before {expected_todos}` looks funny
let!(:expected_todos) do
3.times.map do |i|
Todo.create(title: "Todo #{i+1}")
end
end
it "lists all todos" do
get "/todos"
todos = extract_key_from_response("todos", response)
expect(todos.count).to eq expected_todos.count
end
def extract_key_from_response(key, response)
json_response = JSON.parse(response.body)
expect(json_response).to have_key key
json_response[key]
end
endYou run the test. Watch it go red! First gripe: no model. You fix that:
class Todo < ActiveRecord::Base end
It’s migration time!
<code class="language-bash rails generate migration CreateTodos title:string</code>
# db/migrate/#{timestamp}\_create\_todos.rb
class CreateTodos < ActiveRecord::Migration
def change
create_table :todos do |t|
t.string :title
t.timestamps null: false
end
end
end> rake db:migrate
And of course, it needs a route.
# config/routes.rb Rails.application.routes.draw do resources :todos, only: :index end
The test is still red, because there is no controller. Next fix:
# controllers/todos_controller.rb
class TodosController < ApplicationController
def index
render json: todos
end
private
def todos
Todo.all
end
endAll green! Time to boot up the app(s) and bask in your own creative genius. You run the following commands in different terminals:
> rails server > ember server
You can’t wait to see your baby running. You open “http://local:4200/todos”… and you get a blank screen. Now that was disappointing. Both test suites are green. What could have gone wrong?
After some inspection into the problem, you notice a GET http://localhost:4200/todos 404 (Not Found) in your logs. Well, crap. Ember doesn’t know the API is on a different domain. Well, that’s an easy fix. You just need to specify a host in the application adapter. And since you are attractive and intelligent, you know better than to hardcode the host. After all, it will change for staging and production. So you open config/environment.js in your Ember app, and you add the following:
// config/environment.js if (environment === 'test') {
ENV.API_HOST = ''
} else {
ENV.API_HOST = (process.env.API_HOST || 'http://localhost:3000')
}You set the test environment’s API_HOST to an empty string, so that the current tests keep working. All other environments use the Rails’ default http://localhost:3000 unless an API_HOST environment variable is set. Now you can create an application adapter and fix the bug:
// app/adapters/application.js
import DS from 'ember-data';
import config from '../config/environment';
export default DS.RESTAdapter.extend({
host: config.API_HOST
});W00T! Fixed that problem. Time to run the tests. Once again, you start the Rails and Ember apps in two different terminals.
> rails server > ember server
Dang it. The app still doesn’t work. You didn’t set up CORS so that modern browsers can talk to the Rails API. You add rack-cors to the Gemfile, bundle it, and add the simplest policy you can think of to get things working. Allow everything!
# Gemfile gem 'rack-cors', :require => 'rack/cors'
bundle install
# config/application.rb
config.middleware.insert_before 0, "Rack::Cors", :debug => true, :logger => (-> { Rails.logger }) do
allow do
origins '*'
resource '*',
:headers => :any,
:methods => [:get, :post, :delete, :put, :options, :head],
:max_age => 0
end
endYou cross your fingers, run the tests, and start the servers. When you visit the todos route, you see your app is working. Phew! Even though your app is working now, you aren’t 100 percent happy. You spent the last hour test driving an application that didn’t work, even though all tests were green. And you didn’t even focus unit tests — all of your tests were integration tests. Shouldn’t integration tests prevent things like this?
Writing Smoke Tests for an Ember Rails App
At this point, you remember a post on the Hashrocket blog about test driving elixir apps with cucumber. You decide to try these techniques in practice and write some smoke tests. The type of integration tests that were just written are great for testing application(s) in isolation. These isolated integration tests run very fast. And because they run fast, the majority of your tests should be either request specs or Ember integration. There is no reason to have full coverage (testing every possibility) with smoke tests.
However, in order to ensure the app actually works, there should be at least one smoke test for every API endpoint. Knowing that cucumber-rails already loads your test environment, you think of the simplest solution you can to try writing a smoke test. Simply add cucumber-rails, manually start Ember and Rails, then overwrite how capybara works. This is the solution you come up with. You add cucumber to the Gemfile and install it.
# Gemfile group :development, :test do gem 'cucumber-rails', require: false gem 'database_cleaner' gem 'capybara-webkit' # ... end
> bundle install > rails generate cucumber:install
Then you overwrite capybara’s behavior.
# features/support/env.rb Capybara.configure do |config| # Don't start rails config.run_server = false # Set capybara's driver. Use your own favorite config.default_driver = :webkit # Make all requests to the Ember app config.app_host = 'http://localhost:4200' end
And you write the first feature and step definitions.
# features/todo.feature
Feature: Todos
When a user visits "/todos", they should see all todos
Scenario: User views todos
Given 4 todos
When I visit "/todos"
Then I should see "Todo List"
And I see 4 todos# features/steps/todo.rb
Given(/^(\d+) todos$/) do |num|
num = num.to_i
num.times do |i|
Todo.create(title: "Title #{i}")
end
end
When(/^I visit "(.*?)"$/) do |path|
visit path
end
Then(/^I should see "(.*?)"$/) do |text|
expect(page).to have_text(text)
end
Then(/^I see (\d+) todos$/) do |num|
expect(all(".todo-item").length).to eq num.to_i
endNow in separate terminals you run both Rails and Ember. This time Rails is started in the test environment so that Ember requests hit the Rails test database.
> rails server --environment test > ember server
You run the test suite with rake, and voilà! You see that the test suite passes. You know that both Ember and Rails are communicating end-to-end. And even though you are super proud of yourself, you’re not happy. “The servers should start automatically. Why can’t I just type rake?” you think to yourself. Here is the first solution you come up with.
# features/support/env.rb
ember_pid = fork do
puts "Starting Ember App"
Dir.chdir("/your/user/directory/todo-frontend") do
exec({"API_HOST" => "http://localhost:3001"}, "ember server --port 4201 --live-reload false")
end
end
rails_pid = fork do
puts "Starting Rails App"
Dir.chdir(Rails.root) do
exec("rails server --port 3001 --environment test")
end
end
sleep 2
at_exit do #kill the Ember and Rails apps
puts("Shutting down Ember and Rails App")
Process.kill "KILL", rails_pid
Process.kill "KILL", ember_pid
end
Capybara.configure do |config|
config.run_server = false
config.default_driver = :webkit
config.app_host = 'http://localhost:4201'
endThis solution works okay. Rails and Ember run on separate ports, so the development versions of the two servers can keep running. The only problem is the sleep 2. As the Rails and Ember apps grow, so will the time needed to wait for the servers to start. Also, this number is magic; it might take longer on another machine. How long will it take on the CI server?
What you really want to do is halt the tests until you know that Rails and Ember are running. However, after some investigation, you realize there is no way to know if Ember is running successfully. But then you notice how EmberCLI smoke tests itself.
it('ember new foo, server, SIGINT clears tmp/', function() {
return runCommand(path.join('.', 'node_modules', 'ember-cli', 'bin', 'ember'), 'server', '--port=54323','--live-reload=false', {
onOutput: function(string, child) {
if (string.match(/Build successful/)) {
killCliProcess(child);
}
}
})
.catch(function() {
// just eat the rejection as we are testing what happens
});
});Now you know that Ember has booted when it outputs “Build successful” and have some insight as to how you might wait for Rails.
# features/support/env.rb
begin
DatabaseCleaner.strategy = :truncation
rescue NameError
raise "You need to add database_cleaner to your Gemfile (in the :test group) if you wish to use it."
end
ember_server = nil
rails_server = nil
Dir.chdir("/Users/mwoods/hashrocket/todo-frontend") do
ember_server = IO.popen([{"API_HOST" => "http://localhost:3001"}, "ember", "server", "--port", "4201", "--live-reload", "false", :err => [:child, :out]])
end
Dir.chdir(Rails.root) do
rails_server = IO.popen(['rails', 'server', '--port', '3001', '--environment', 'test', :err => [:child, :out]])
end
# Before timeout loop to prevent orphaning processes
at_exit do
Process.kill 9, rails_server.pid, ember_server.pid
end
# if it takes longer than 30 seconds to boot throw an error
Timeout::timeout(30) do
# wait for the magic words from ember
while running = ember_server.gets
if running =~ /build successful/i
break
end
end
# when rails starts logging, it's running
while running = rails_server.gets
if running =~ /info/i
break
end
end
end
Capybara.configure do |config|
config.run_server = false
config.default_driver = :webkit
config.app_host = 'http://localhost:4201'
endThis is not a perfect solution, but it’s good enough for day one. On day two, you might decide to move the smoke tests to their own repository. After all, they aren’t Rails-specific, so they probably should be a separate project.
Problems like these and many others have happened to me. That’s the reason I’m introducing exhaust, a new gem that hopefully alleviates some of the headache associated with smoke testing an Ember+Rails stack. Enjoy!
| Reference: | How to Write Smoke Tests for an Ember Rails Stack from our WCG partner Florian Motlik at the Codeship Blog blog. |
