Building Cross Model Search with Ember, Part II
In Part One of this two part series, we covered how to integrate Elasticsearch and Rails to query against multiple models. To illustrate this, we started a Rails app for an ecommerce store named Tember. It includes an Elasticsearch index made up of four different models using Toptal’s Chewy gem.
Today we’re going to pick up where we left off and build a cross model autocompleting search feature for the Tember store. We left off last time in the Rails console interacting with our new indexes, so the first thing we need to do is get our Rails app accepting HTTP requests and sending back data.
Let’s get started!
Serving Data from Rails and Elasticsearch Over HTTP
We are strictly building an API for this search endpoint, so let’s organize the controllers under the api/v1
namespace.
$ rails g controller api/v1/searches create app/controllers/api/v1/searches_controller.rb invoke rspec create spec/controllers/api/v1/searches_controller_spec.rb
I breathe easier when I’m confident in the test coverage, so let’s start start with some rspec specs for our search controller. Open up the searches_controller_spec.rb
, and let’s add in some tests for the controller:
# spec/controllers/api/v1/searches_controller_spec.rb require 'rails_helper' RSpec.describe Api::V1::SearchesController, :type => :controller do describe 'GET #index - /v1/searches' do before do get :index, q: 'an example query string' end it { is_expected.to respond_with :ok } it { is_expected.to respond_with_content_type :json } end end
Let’s run the tests and start moving from red to green:
$ rspec spec/controllers/api/v1/searches_controller_spec.rb 1) Api::V1::SearchesController GET #index - /v1/searches Failure/Error: get :index, q: 'an example query string' ActionController::UrlGenerationError: No route matches {:action=>"index", :controller=>"api/v1/searches", :q=>"a example query string"}
As you probably noticed, we don’t have any routes defined. Let’s open up config/routes.rb
and add the appropriate routes.
As I mentioned earlier, organization is crucial at any point in a project, but especially at the start of one. Organizing the routes specified for the API into an api
and v1
namespace helps keep your project tidy, adds logical separation of responsibilities, and if you add constraints in the declaration, it can be used to add subdomains.
Subdomains allow you to load balance at the DNS level if necessary. Carlos Souza from Code School covers this material in depth in his Surviving APIs with Rails course.
# config/routes.rb Rails.application.routes.draw do # use the constraints option to add a subdomain to the uri namespace :api, path: '' do # constraints: { subdomain: 'api' } namespace :v1 do resources :searches, only: [:index] end end end
Let’s run the tests and see where we stand:
$ rspec spec/controllers/api/v1/searches_controller_spec.rb 1) Api::V1::SearchesController GET #index - /v1/searches Failure/Error: get :index, q: 'an example query string' AbstractController::ActionNotFound: The action 'index' could not be found for Api::V1::SearchesController
We covered our No routes match
error, but now we’re seeing another error complaining about the lack of an index
action in our controller. Here’s how we fix this:
# controllers/api/v1/searches_controller.rb class Api::V1::SearchesController < ApplicationController # v1/searches def index render json: [] end end
Now when we run rspec spec
, it gives us a green light:
. Finished in 0.01418 seconds (files took 2.24 seconds to load) 2 examples, 0 failures
Loading Elasticsearch Documents Inside Rails Controllers
This controller doesn’t gives us any results, so let’s add in some of the Chewy commands we used in the previous post to pull in data from Elasticsearch. If you recall, we had a class called StoreIndex
which represented the integration of our Postgres stored data and our Elasticsearch documents.
Yes, I said “documents.” Elasticsearch is actually a NoSQL document oriented datastore. Let’s pull in the command we used last time and dissect it:
StoreIndex.query(term: {name: 'oak'})
This particular command is looking at the index as a whole and asking for any document where the name field matches oak
. If we were simply interested in the name
field, this command would be sufficient. Unfortunately, we’re interested in more fields and want our search to query against names, descriptions, and even the review bodies. Let’s alter the way we interface with our StoreIndex
:
StoreIndex.query(query_string: {fields: [:name, :description, :body], query: ‘oak’, default_operator: 'or'})
This command gives us more control over the fields we are querying against. You also might notice the default_operator: ‘or’
. This makes the search query inclusive. Other filter mode options include: :and
, :or
, :must
, :should
.
If we open up the console to test this, you’ll see additional classes are loaded by the query:
$ rails c 2.2.2 :005 > StoreIndex.query(query_string: {fields: [:name, :description, :body], query: 'oak', default_operator: 'or'}).load.to_a StoreIndex Search (38.9ms) {:body=>{:query=>{:query_string=>{:fields=>[:name, :description, :body], :query=>"oak", :default_operator=>"or"}}}, :index=>["store"], :type=>[]} Product Load (0.4ms) SELECT "products".* FROM "products" WHERE "products"."id" IN (1, 2, 3) Review Load (28.5ms) SELECT "reviews".* FROM "reviews" WHERE "reviews"."id" IN (2, 1, 3) Vendor Load (0.3ms) SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" IN (1, 3)
Let’s adjust the SearchesController to include our new query commands:
class Api::V1::SearchesController < ApplicationController # v1/searches def index render json: StoreIndex.query(query_string: {fields: [:name, :description, :body], query: params[:q], default_operator: 'or'}).load.to_a, each_serializer: SearchesSerializer end end
We are querying across many models, so it’s important to have solid test coverage on the components of this feature. Let’s open up our controller spec and start adding more cases:
require 'rails_helper' RSpec.describe Api::V1::SearchesController, :type => :controller do let(:vendor) { Vendor.new(id: 1, name: 'Example Vendor Name', description: 'Example vendor description.') } let(:product) { Product.new(id: 1, name: 'Example Product Name', description: 'Example product description.') } let(:json) { JSON.parse(response.body) } it { should route(:get, 'v1/searches').to(action: :index) } describe 'GET #index - /v1/searches' do before do StoreIndex.stub_chain(:query, :load, :to_a).and_return([vendor, product]) get :index, q: 'oak' end it { is_expected.to respond_with :ok } it 'returns array of records' do expect(json["searches"]).to be_a Array end it 'should have `searches` as a root key' do expect(json).to have_key("searches") end end end
We have added quite a few things, so let’s take a moment to review.
First, we added a route test in the controller test. Second, we’re stubbing the StoreIndex
to return an empty array.
We stub our the index because this test should not be responsible for testing the indexes. Instead, it should be responsible for testing what happens with the return value. We’ll refactor this test later to add models in the return value.
Finally, we added in tests to ensure the root key is searches
and the value of it is an array. Adjusting the controller to include a root key option with the value of “searches” makes this test pass.
class Api::V1::SearchesController < ApplicationController # v1/searches def index render json: StoreIndex.query(query_string: {fields: [:name, :description, :body], query: params[:q], default_operator: 'or'}).load.to_a, each_serializer: SearchesSerializer, root: "searches" end end
In order for the search API to interface with Ember, we need to normalize the results we get back. This is easier explained by examining sample data responses.
If we have four different models — Vendor, ReviewAuthor, Review, Product — we want the results to show the same data for each one like so:
{ "searches":[ { "id":1, "type":"Product", "title":"Some product title", "description":"Some product description" }, { "id":1, "type":"Vendor", "title":"Some product title", "description":"Some vendor description" }, { "id":1, "type":"Review", "title":"Some review title", "description":"Some review description" }, { "id":1, "type":"ReviewAuthor", "title":"Some author name", "description":"Some author bio" } ] }
Each of the models have different attributes, but we map them to display fields for UI purposes. In order to render JSON, we will need a serializer. We’ll call it SearchesSerializer
.
$ rails g serializer searches create app/serializers/searches_serializer.rb
Now that we have a serializer, let’s add in more tests to our controller spec:
it 'should respond with one product' do expect(json["searches"].map(&["title"])).to include(product.name) end it 'should respond with one vendor' do expect(json["searches"].map(&:["title"])).to include(product.name) end
We want to ensure that the results have title attributes which include our sample data. In order to do this, we’ll create aliases on each model mapping title and description. For example, the vendor model now looks like this:
class Vendor < ActiveRecord::Base # es indexes update_index('store#vendor') { self } # specifying index, type and back-reference # for updating after user save or destroy # associations has_many :products alias_attribute :title, :name end
After we have aliased the appropriate methods on each model, we can use their respective attributes in the serializer like so:
class SearchesSerializer < ActiveModel::Serializer attributes :id, :title, :description, :type def id object.class.to_s.downcase + '|' + object.id.to_s end def type object.class.to_s.camelize end end
I want you to notice two things. First, we have added in the type string. This will be used for display purposes on the front end.
Second, we’ve overwritten the ID attribute to increase the uniqueness of the value. We do this because in ember-data
, two records with the same ID will cause issues with the local caching. A simple CURL command will test out new searches endpoint for our Tember store.
$ curl -X GET 'http://localhost:3000/v1/searches?q=oak' {"searches":[{"id":"vendor|1","title":"Oakmont Oaks","description":"Oakmont provides choice oak from Orlando, Louisville, and Seattle","type":"Vendor"},{"id":"vendor|3","title":"Big Hill","description":"Big Hill provides oak, northern ash, and pine from various regions of Canada.","type":"Vendor"}]}
Our data is looking great! Now, it’s time to hop into the front-end of this ecommerce store and start coding up some Ember.js goodness.
Building an Autocomplete Search in Ember
Ember is a front-end framework for building ambitious web applications. I started with Ember almost exactly one year ago today and have continually been impressed by the community, the framework’s maturity, and the tooling Ember is shipping with (ember-cli
).
Without further delay, let’s dive in! We’ll start by creating a new Ember app with the cli; if you have not used ember-cli before, check it out!
ember new tember-ember
If we cd
into tember-ember
, we can start setting up our Ember project. I’m going to start by editing the bower.json
to ensure we’re on the correct Ember and ember-data
versions.
At the time of this article, Ember is on v2.2.0 and ember-data
is on v2.2.1. Let’s boot up the app and see what we get:
A bit boring if you ask me. Let’s clean it up a bit.
Since we’ve been riding the edge with Rails 5, we shouldn’t stop there! Let’s install Bootstrap 4 and give ourselves a fancier UI:
bower install bootstrap#v4.0.0-alpha.2
With some standard and some new bootstrap commands dropped into templates/application.hbs
, we’re beginning to look a bit better:
<nav class="navbar navbar-full navbar-dark bg-danger"> <a class="navbar-brand" href="#"> Tember, <small>a store for high quality wood workers.</small> </a> </nav> {{outlet}}
Creating a service-fed component in Ember 2.x style
At this point, we’re ready to start searching our store and displaying our results. To do this, we’ll create a store-search
component.
Ember has moved away from the traditional MVC in favor of a component-driven model. Components make it much easier to isolate pieces of UI, making them easier to understand, test, and manage.
If you’re familiar with the 1.x methodology, you might be asking yourself, “How do components get their data without a controller?” Valid question. In the past, Ember has used the Route
to load data which could then be sent into a component through the controller. Routes are still a thing in 2.x, except the new norm will be routable components and components fed via services. We’ll touch on this shortly.
In the meantime, let’s generate our component:
$ ember g component store-search installing component create app/components/store-search.js create app/templates/components/store-search.hbs installing component-test create tests/integration/components/store-search-test.js
Now, make the component look like this:
templates/components/store-search.hbs {{input name="query" value=query class="form-control" placeholder="Search for anything..."}} {{#if search.noResults}} <li class='no-results list-unstyled'> No results for "<span>{{query}}</span>". </li> {{/if}} <div class='row'> {{#each search.results as |result|}} <div class='col-xs-6 col-sm-4'> <a href="{{result.link}}"> <div class="card"> <div class="card-header"> {{result.type}} </div> <div class="card-block"> <h4 class="card-title">{{result.title}}</h4> <p class="card-text">{{result.description}}</p> <p class="card-text"><small class="text-muted">{{result.lastUpdated}}</small></p> </div> </div> </a> </div> {{/each}} </div>
You might be noticing the familiar attributes we created in our Rails serializers; they will be injected into this search component.
The component will be responsible for telling the user if there are no results, and if there are results, the user will begin to see them as Bootstrap 4 cards. The card component of Bootstrap 4 replaces the panel/well of previous Bootstrap versions.
The data being pushed into the component will be from the search
model. It will also have a link
property which will be used to link to the resource. Let’s create that now:
// app/models/search.js import DS from 'ember-data'; export default DS.Model.extend({ type: DS.attr('string'), description: DS.attr('string'), title: DS.attr('string'), link: Ember.computed('type', function() { switch (this.get('type')) { case 'Product': return `product/${this.get('id')}`; case 'Vendor': return `vendor/${this.get('id')}`; case 'ReviewAuthor': return `review-author/${this.get('id')}`; case 'Review': return `review/${this.get('id')}`; } }) });
As I mentioned above, there are more ways than routes to load data. Another strategy to bring data into Ember is through services. Services are essentially a singleton object in Ember with a slick API to inject them anywhere you need — components, in our case.
Let’s create an Ember service to bring data into this search component.
ember g service search installing service create app/services/search.js installing service-test create tests/unit/services/search-test.js
As mentioned previously, we want the search component to let us know if there are zero results for a given query. In order to do this, we need to cache the current search query and results.
We’ll handle all of this functionality inside the service by simply updating the searchValue
attribute. Inside the service, we’ll add a few methods:
// app/services/search.js import Ember from 'ember'; export default Ember.Service.extend({ store: Ember.inject.service(), searchValue: null, searchResults: [], resultsEmpty: false, noResults: Ember.computed('resultsEmpty', 'searchValue', function() { // ensure there is a search query and the results // are empty to prevent "No results for ''". if (this.get('resultsEmpty') && Ember.isPresent(this.get('searchValue')) && this.get('searchValue').length > 0) { return true; } else { return false; } }), results: Ember.computed('searchResults.[]', function() { return this.get('searchResults'); }), _fetchSearchResults: Ember.observer('searchValue', function() { // exit without making a reqeust if value cast is empty if (Ember.isBlank(this.get('searchValue'))) { return []; } this.get('store').query('search', { q: this.get('searchValue') }).then((results) => { // check if the query results are empty if (results.get('length') > 0) { this.set('resultsEmpty', false); } else { this.set('resultsEmpty', true); } this.set('searchResults', results); }); }) });
The noResults
method is responsible for letting the component know a search has occurred, but it returned no results. The other methods are self explanatory: One fetches results from Rails, and the other is a computed property returning the results.
As you can see, we rely on the service to do almost all the work. The component just needs to be aware of how to interact with the service:
// app/components/store-search.js import Ember from 'ember'; export default Ember.Component.extend({ search: Ember.inject.service(), query: null, _valueChanged: Ember.observer('query', function() { // use the run loop to add a debounce Ember.run.debounce(this, function() { // check if the query is at least 2 chars if (this.get('query').length > 2) { this.set('search.searchValue', this.get('query')); } else if (this.get('query').length === 0) { this.set('search.searchResults', []); } }, 200); }), });
This component starts off by injecting the search-service
and setting the query
attribute to null
.
Lastly, we set an observer to monitor the query
value. It interfaces with the search-service
every time it’s updated by changing the service’s current searchValue
updating the results.
Now that we have the service and component wired up, let’s check it out:
And there you have it! An autocompleting cross model search component built with Ember.js, Rails 5, and Elasticsearch.
Conclusion
I hope this post has helped you become more confident in your ability to integrate Rails and Elasticsearch to create a scalable search solution for any use case. There’s certainly much to be desired when it comes to the accuracy of the search, but every use case needs to be dialed in with the appropriate Elasticsearch techniques.
Don’t forget to check out the Ember and Rails repos:
Happy coding!
Reference: | Building Cross Model Search with Ember, Part II from our WCG partner Florian Motlik at the Codeship Blog blog. |