🚀 Intelligent search made easy
Searchkick learns what your users are looking for. As more people search, it gets smarter and the results get better. It’s friendly for developers - and magical for your users.
Searchkick handles:
- stemming -
tomatoes
matchestomato
- special characters -
jalapeno
matchesjalapeño
- extra whitespace -
dishwasher
matchesdish washer
- misspellings -
zuchini
matcheszucchini
- custom synonyms -
qtip
matchescotton swab
Plus:
- query like SQL - no need to learn a new query language
- reindex without downtime
- easily personalize results for each user
- autocomplete
- “Did you mean” suggestions
- supports many languages
- works with ActiveRecord, Mongoid, and NoBrainer
💬 Get handcrafted updates for new features
🍊 Battle-tested at Instacart
Does your startup use Searchkick? Want a free hour of advising? Fill out this application. I’ll reach out to a few companies.
- Getting Started
- Querying
- Indexing
- Instant Search / Autocomplete
- Aggregations
- Deployment
- Performance
- Elasticsearch DSL
- Reference
Searchkick 3.0 was recently released! See how to upgrade
Thinking of upgrading from Elasticsearch 5 to 6? Read this first
Install Elasticsearch. For Homebrew, use:
brew install elasticsearch
brew services start elasticsearch
Add this line to your application’s Gemfile:
gem 'searchkick'
The latest version works with Elasticsearch 5 and 6. For Elasticsearch 2, use version 2.5.0 and this readme.
Add searchkick to models you want to search.
class Product < ApplicationRecord
searchkick
end
Add data to the search index.
Product.reindex
And to query, use:
products = Product.search("apples")
products.each do |product|
puts product.name
end
Searchkick supports the complete Elasticsearch Search API. As your search becomes more advanced, we recommend you use the Elasticsearch DSL for maximum flexibility.
Query like SQL
Product.search "apples", where: {in_stock: true}, limit: 10, offset: 50
Search specific fields
fields: [:name, :brand]
Where
where: {
expires_at: {gt: Time.now}, # lt, gte, lte also available
orders_count: 1..10, # equivalent to {gte: 1, lte: 10}
aisle_id: [25, 30], # in
store_id: {not: 2}, # not
aisle_id: {not: [25, 30]}, # not in
user_ids: {all: [1, 3]}, # all elements in array
category: /frozen .+/, # regexp
_or: [{in_stock: true}, {backordered: true}]
}
Order
order: {_score: :desc} # most relevant first - default
All of these sort options are supported
Limit / offset
limit: 20, offset: 40
Select
select: [:name]
These source filtering options are supported
Searches return a Searchkick::Results
object. This responds like an array to most methods.
results = Product.search("milk")
results.size
results.any?
results.each { |result| ... }
By default, ids are fetched from Elasticsearch and records are fetched from your database. To fetch everything from Elasticsearch, use:
Product.search("apples", load: false)
Get total results
results.total_count
Get the time the search took (in milliseconds)
results.took
Get the full response from Elasticsearch
results.response
Boost important fields
fields: ["title^10", "description"]
Boost by the value of a field (field must be numeric)
boost_by: [:orders_count] # give popular documents a little boost
boost_by: {orders_count: {factor: 10}} # default factor is 1
Boost matching documents
boost_where: {user_id: 1}
boost_where: {user_id: {value: 1, factor: 100}} # default factor is 1000
boost_where: {user_id: [{value: 1, factor: 100}, {value: 2, factor: 200}]}
Boost by recency
boost_by_recency: {created_at: {scale: "7d", decay: 0.5}}
You can also boost by:
Use a *
for the query.
Product.search "*"
Plays nicely with kaminari and will_paginate.
# controller
@products = Product.search "milk", page: params[:page], per_page: 20
View with kaminari
<%= paginate @products %>
View with will_paginate
<%= will_paginate @products %>
By default, results must match all words in the query.
Product.search "fresh honey" # fresh AND honey
To change this, use:
Product.search "fresh honey", operator: "or" # fresh OR honey
By default, results must match the entire word - back
will not match backpack
. You can change this behavior with:
class Product < ApplicationRecord
searchkick word_start: [:name]
end
And to search (after you reindex):
Product.search "back", fields: [:name], match: :word_start
Available options are:
:word # default
:word_start
:word_middle
:word_end
:text_start
:text_middle
:text_end
To match a field exactly (case-sensitive), use:
User.search query, fields: [{email: :exact}, :name]
To only match the exact order, use:
User.search "fresh honey", match: :phrase
Searchkick defaults to English for stemming. To change this, use:
class Product < ApplicationRecord
searchkick language: "german"
end
A few languages require plugins:
chinese
- analysis-ik pluginjapanese
- analysis-kuromoji pluginkorean
- analysis-openkoreantext pluginpolish
- analysis-stempel pluginukrainian
- analysis-ukrainian pluginvietnamese
- analysis-vietnamese plugin
class Product < ApplicationRecord
searchkick synonyms: [["scallion", "green onion"], ["qtip", "cotton swab"]]
end
Call Product.reindex
after changing synonyms.
Synonyms cannot be more than two words at the moment.
To read synonyms from a file, use:
synonyms: -> { CSV.read("/some/path/synonyms.csv") }
For directional synonyms, use:
synonyms: ["lightbulb => halogenlamp"]
The above approach works well when your synonym list is static, but in practice, this is often not the case. When you analyze search conversions, you often want to add new synonyms or tags without a full reindex. You can use a library like ActsAsTaggableOn and do:
class Product < ApplicationRecord
acts_as_taggable
scope :search_import, -> { includes(:tags) }
def search_data
{
name_tagged: "#{name} #{tags.map(&:name).join(" ")}"
}
end
end
Search with:
Product.search query, fields: [:name_tagged]
Prepopulate English synonyms with the WordNet database.
Download WordNet 3.0 to each Elasticsearch server and move wn_s.pl
to the /var/lib
directory.
cd /tmp
curl -o wordnet.tar.gz http:https://wordnetcode.princeton.edu/3.0/WNprolog-3.0.tar.gz
tar -zxvf wordnet.tar.gz
mv prolog/wn_s.pl /var/lib
Tell each model to use it:
class Product < ApplicationRecord
searchkick wordnet: true
end
By default, Searchkick handles misspelled queries by returning results with an edit distance of one.
You can change this with:
Product.search "zucini", misspellings: {edit_distance: 2} # zucchini
To prevent poor precision and improve performance for correctly spelled queries (which should be a majority for most applications), Searchkick can first perform a search without misspellings, and if there are too few results, perform another with them.
Product.search "zuchini", misspellings: {below: 5}
If there are fewer than 5 results, a 2nd search is performed with misspellings enabled. The result of this query is returned.
Turn off misspellings with:
Product.search "zuchini", misspellings: false # no zucchini
If a user searches butter
, they may also get results for peanut butter
. To prevent this, use:
Product.search "butter", exclude: ["peanut butter"]
You can map queries and terms to exclude with:
exclude_queries = {
"butter" => ["peanut butter"],
"cream" => ["ice cream", "whipped cream"]
}
Product.search query, exclude: exclude_queries[query]
Search 🍨🍰 and get ice cream cake
!
Add this line to your application’s Gemfile:
gem 'gemoji-parser'
And use:
Product.search "🍨🍰", emoji: true
Control what data is indexed with the search_data
method. Call Product.reindex
after changing this method.
class Product < ApplicationRecord
belongs_to :department
def search_data
{
name: name,
department_name: department.name,
on_sale: sale_price.present?
}
end
end
Searchkick uses find_in_batches
to import documents. To eager load associations, use the search_import
scope.
class Product < ApplicationRecord
scope :search_import, -> { includes(:department) }
end
By default, all records are indexed. To control which records are indexed, use the should_index?
method together with the search_import
scope.
class Product < ApplicationRecord
scope :search_import, -> { where(active: true) }
def should_index?
active # only index active records
end
end
If a reindex is interrupted, you can resume it with:
Product.reindex(resume: true)
For large data sets, try parallel reindexing.
- when you install or upgrade searchkick
- change the
search_data
method - change the
searchkick
method
- app starts
There are four strategies for keeping the index synced with your database.
- Inline (default)
Anytime a record is inserted, updated, or deleted
- Asynchronous
Use background jobs for better performance
class Product < ApplicationRecord
searchkick callbacks: :async
end
Jobs are added to a queue named searchkick
.
- Queuing
Push ids of records that need updated to a queue and reindex in the background in batches. This is more performant than the asynchronous method, which updates records individually. See how to set up.
- Manual
Turn off automatic syncing
class Product < ApplicationRecord
searchkick callbacks: false
end
You can also do bulk updates.
Searchkick.callbacks(:bulk) do
User.find_each(&:update_fields)
end
Or temporarily skip updates.
Searchkick.callbacks(false) do
User.find_each(&:update_fields)
end
Data is not automatically synced when an association is updated. If this is desired, add a callback to reindex:
class Image < ApplicationRecord
belongs_to :product
after_commit :reindex_product
def reindex_product
product.reindex
end
end
The best starting point to improve your search by far is to track searches and conversions.
Searchjoy makes it easy.
Product.search "apple", track: {user_id: current_user.id}
See the docs for how to install and use.
Focus on:
- top searches with low conversions
- top searches with no results
Searchkick can use conversion data to learn what users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (our conversion metric at Instacart), that item gets a little more weight for similar searches.
The first step is to define your conversion metric and start tracking conversions. The database works well for low volume, but feel free to use Redis or another datastore.
Searchkick automatically treats apple
and APPLE
the same.
Next, add conversions to the index.
class Product < ApplicationRecord
has_many :searches, class_name: "Searchjoy::Search", as: :convertable
searchkick conversions: [:conversions] # name of field
def search_data
{
name: name,
conversions: searches.group(:query).uniq.count(:user_id)
# {"ice cream" => 234, "chocolate" => 67, "cream" => 2}
}
end
end
Reindex and set up a cron job to add new conversions daily.
rake searchkick:reindex CLASS=Product
Note: For a more performant (but more advanced) approach, check out performant conversions.
Order results differently for each user. For example, show a user’s previously purchased products before other results.
class Product < ApplicationRecord
def search_data
{
name: name,
orderer_ids: orders.pluck(:user_id) # boost this product for these users
}
end
end
Reindex and search with:
Product.search "milk", boost_where: {orderer_ids: current_user.id}
Autocomplete predicts what a user will type, making the search experience faster and easier.
Note: To autocomplete on general categories (like cereal
rather than product names), check out Autosuggest.
Note 2: If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s much faster to load all records into JavaScript and autocomplete there (eliminates network requests).
First, specify which fields use this feature. This is necessary since autocomplete can increase the index size significantly, but don’t worry - this gives you blazing faster queries.
class Movie < ApplicationRecord
searchkick word_start: [:title, :director]
end
Reindex and search with:
Movie.search "jurassic pa", fields: [:title], match: :word_start
Typically, you want to use a JavaScript library like typeahead.js or jQuery UI.
First, add a route and controller action.
class MoviesController < ApplicationController
def autocomplete
render json: Movie.search(params[:query], {
fields: ["title^5", "director"],
match: :word_start,
limit: 10,
load: false,
misspellings: {below: 5}
}).map(&:title)
end
end
Note: Use load: false
and misspellings: {below: n}
(or misspellings: false
) for best performance.
Then add the search box and JavaScript code to a view.
<input type="text" id="query" name="query" />
<script src="jquery.js"></script>
<script src="typeahead.bundle.js"></script>
<script>
var movies = new Bloodhound({
datumTokenizer: Bloodhound.tokenizers.whitespace,
queryTokenizer: Bloodhound.tokenizers.whitespace,
remote: {
url: '/movies/autocomplete?query=%QUERY',
wildcard: '%QUERY'
}
});
$('#query').typeahead(null, {
source: movies
});
</script>
class Product < ApplicationRecord
searchkick suggest: [:name] # fields to generate suggestions
end
Reindex and search with:
products = Product.search "peantu butta", suggest: true
products.suggestions # ["peanut butter"]
Aggregations provide aggregated search data.
products = Product.search "chuck taylor", aggs: [:product_type, :gender, :brand]
products.aggs
By default, where
conditions apply to aggregations.
Product.search "wingtips", where: {color: "brandy"}, aggs: [:size]
# aggregations for brandy wingtips are returned
Change this with:
Product.search "wingtips", where: {color: "brandy"}, aggs: [:size], smart_aggs: false
# aggregations for all wingtips are returned
Set where
conditions for each aggregation separately with:
Product.search "wingtips", aggs: {size: {where: {color: "brandy"}}}
Limit
Product.search "apples", aggs: {store_id: {limit: 10}}
Order
Product.search "wingtips", aggs: {color: {order: {"_term" => "asc"}}} # alphabetically
All of these options are supported
Ranges
price_ranges = [{to: 20}, {from: 20, to: 50}, {from: 50}]
Product.search "*", aggs: {price: {ranges: price_ranges}}
Minimum document count
Product.search "apples", aggs: {store_id: {min_doc_count: 2}}
Date histogram
Product.search "pear", aggs: {products_per_year: {date_histogram: {field: :created_at, interval: :year}}}
For other aggregation types, including sub-aggregations, use body_options
:
Product.search "orange", body_options: {aggs: {price: {histogram: {field: :price, interval: 10}}}
Specify which fields to index with highlighting.
class Product < ApplicationRecord
searchkick highlight: [:name]
end
Highlight the search query in the results.
bands = Band.search "cinema", highlight: true
View the highlighted fields with:
bands.with_highlights.each do |band, highlights|
highlights[:name] # "Two Door <em>Cinema</em> Club"
end
To change the tag, use:
Band.search "cinema", highlight: {tag: "<strong>"}
To highlight and search different fields, use:
Band.search "cinema", fields: [:name], highlight: {fields: [:description]}
By default, the entire field is highlighted. To get small snippets instead, use:
bands = Band.search "cinema", highlight: {fragment_size: 20}
bands.with_highlights(multiple: true).each do |band, highlights|
highlights[:name].join(" and ")
end
Additional options can be specified for each field:
Band.search "cinema", fields: [:name], highlight: {fields: {name: {fragment_size: 200}}}
You can find available highlight options in the Elasticsearch reference.
Find similar items.
product = Product.first
product.similar(fields: [:name], where: {size: "12 oz"})
class Restaurant < ApplicationRecord
searchkick locations: [:location]
def search_data
attributes.merge(location: {lat: latitude, lon: longitude})
end
end
Reindex and search with:
Restaurant.search "pizza", where: {location: {near: {lat: 37, lon: -114}, within: "100mi"}} # or 160km
Bounded by a box
Restaurant.search "sushi", where: {location: {top_left: {lat: 38, lon: -123}, bottom_right: {lat: 37, lon: -122}}}
Note: top_right
and bottom_left
also work
Bounded by a polygon
Restaurant.search "dessert", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123}, {lat: 39, lon: -123}, {lat: 37, lon: 122}]}}}
Boost results by distance - closer results are boosted more
Restaurant.search "noodles", boost_by_distance: {location: {origin: {lat: 37, lon: -122}}}
Also supports additional options
Restaurant.search "wings", boost_by_distance: {location: {origin: {lat: 37, lon: -122}, function: "linear", scale: "30mi", decay: 0.5}}
You can also index and search geo shapes.
class Restaurant < ApplicationRecord
searchkick geo_shape: {
bounds: {tree: "geohash", precision: "1km"}
}
def search_data
attributes.merge(
bounds: {
type: "envelope",
coordinates: [{lat: 4, lon: 1}, {lat: 2, lon: 3}]
}
)
end
end
See the Elasticsearch documentation for details.
Find shapes intersecting with the query shape
Restaurant.search "soup", where: {bounds: {geo_shape: {type: "polygon", coordinates: [[{lat: 38, lon: -123}, ...]]}}}
Falling entirely within the query shape
Restaurant.search "salad", where: {bounds: {geo_shape: {type: "circle", relation: "within", coordinates: [{lat: 38, lon: -123}], radius: "1km"}}}
Not touching the query shape
Restaurant.search "burger", where: {bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
Containing the query shape
Restaurant.search "fries", where: {bounds: {geo_shape: {type: "envelope", relation: "contains", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
Searchkick supports single table inheritance.
class Dog < Animal
end
In your parent model, set:
class Animal < ApplicationRecord
searchkick inheritance: true
end
The parent and child model can both reindex.
Animal.reindex
Dog.reindex # equivalent, all animals reindexed
And to search, use:
Animal.search "*" # all animals
Dog.search "*" # just dogs
Animal.search "*", type: [Dog, Cat] # just cats and dogs
Note: The suggest
option retrieves suggestions from the parent at the moment.
Dog.search "airbudd", suggest: true # suggestions for all animals
To help with debugging queries, you can use:
Product.search("soap", debug: true)
This prints useful info to stdout
.
See how Elasticsearch scores your queries with:
Product.search("soap", explain: true).response
See how Elasticsearch tokenizes your queries with:
Product.search_index.tokens("Dish Washer Soap", analyzer: "searchkick_index")
# ["dish", "dishwash", "washer", "washersoap", "soap"]
Product.search_index.tokens("dishwasher soap", analyzer: "searchkick_search")
# ["dishwashersoap"] - no match
Product.search_index.tokens("dishwasher soap", analyzer: "searchkick_search2")
# ["dishwash", "soap"] - match!!
Partial matches
Product.search_index.tokens("San Diego", analyzer: "searchkick_word_start_index")
# ["s", "sa", "san", "d", "di", "die", "dieg", "diego"]
Product.search_index.tokens("dieg", analyzer: "searchkick_word_search")
# ["dieg"] - match!!
See the complete list of analyzers.
Searchkick uses ENV["ELASTICSEARCH_URL"]
for the Elasticsearch server. This defaults to http:https://localhost:9200
.
Choose an add-on: Bonsai or Elastic Cloud. SearchBox does not work at the moment.
For Bonsai:
heroku addons:create bonsai
heroku config:set ELASTICSEARCH_URL=`heroku config:get BONSAI_URL`
For Found:
heroku addons:create foundelasticsearch
heroku addons:open foundelasticsearch
Visit the Shield page and reset your password. You’ll need to add the username and password to your url. Get the existing url with:
heroku config:get FOUNDELASTICSEARCH_URL
And add elastic:password@
right after https://
:
heroku config:set ELASTICSEARCH_URL=https://elastic:[email protected]
Then deploy and reindex:
heroku run rake searchkick:reindex CLASS=Product
Create an initializer config/initializers/elasticsearch.rb
with:
ENV["ELASTICSEARCH_URL"] = "https://es-domain-1234.us-east-1.es.amazonaws.com"
To use signed request, include in your Gemfile:
gem 'faraday_middleware-aws-sigv4'
and add to your initializer:
Searchkick.aws_credentials = {
access_key_id: ENV["AWS_ACCESS_KEY_ID"],
secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
region: "us-east-1"
}
Then deploy and reindex:
rake searchkick:reindex CLASS=Product
Create an initializer config/initializers/elasticsearch.rb
with:
ENV["ELASTICSEARCH_URL"] = "http:https://username:[email protected]"
Then deploy and reindex:
rake searchkick:reindex CLASS=Product
Create an initializer config/initializers/elasticsearch.rb
with multiple hosts:
ENV["ELASTICSEARCH_URL"] = "http:https://localhost:9200,http:https://localhost:9201"
See elasticsearch-transport for a complete list of options.
Add the following to config/environments/production.rb
:
config.lograge.custom_options = lambda do |event|
options = {}
options[:search] = event.payload[:searchkick_runtime] if event.payload[:searchkick_runtime].to_f > 0
options
end
See Production Rails for other good practices.
Significantly increase performance with faster JSON generation. Add Oj to your Gemfile.
gem 'oj'
This speeds up all JSON generation and parsing in your application (automatically!)
Significantly increase performance with persistent HTTP connections. Add Typhoeus to your Gemfile and it’ll automatically be used.
gem 'typhoeus'
To reduce log noise, create an initializer with:
Ethon.logger = Logger.new(nil)
If you run into issues on Windows, check out this post.
By default, all string fields are searchable (can be used in fields
option). Speed up indexing and reduce index size by only making some fields searchable. This disables the _all
field unless it’s listed.
class Product < ApplicationRecord
searchkick searchable: [:name]
end
By default, all string fields are filterable (can be used in where
option). Speed up indexing and reduce index size by only making some fields filterable.
class Product < ApplicationRecord
searchkick filterable: [:brand]
end
Note: Non-string fields are always filterable and should not be passed to this option.
For large data sets, you can use background jobs to parallelize reindexing.
Product.reindex(async: true)
# {index_name: "products_production_20170111210018065"}
Once the jobs complete, promote the new index with:
Product.search_index.promote(index_name)
You can optionally track the status with Redis:
Searchkick.redis = Redis.new
And use:
Searchkick.reindex_status(index_name)
You can also have Searchkick wait for reindexing to complete
Searchkick.reindex(async: {wait: true})
You can use ActiveJob::TrafficControl to control concurrency. Install the gem:
gem 'activejob-traffic_control', '>= 0.1.3'
And create an initializer with:
ActiveJob::TrafficControl.client = Searchkick.redis
class Searchkick::BulkReindexJob
concurrency 3
end
This will allow only 3 jobs to run at once.
You can specify a longer refresh interval while reindexing to increase performance.
Product.reindex(async: true, refresh_interval: "30s")
Note: This only makes a noticable difference with parallel reindexing.
When promoting, have it restored to the value in your mapping (defaults to 1s
).
Product.search_index.promote(index_name, update_refresh_interval: true)
Push ids of records needing reindexed to a queue and reindex in bulk for better performance. First, set up Redis in an initializer. We recommend using connection_pool.
Searchkick.redis = ConnectionPool.new { Redis.new }
And ask your models to queue updates.
class Product < ApplicationRecord
searchkick callbacks: :queue
end
Then, set up a background job to run.
Searchkick::ProcessQueueJob.perform_later(class_name: "Product")
You can check the queue length with:
Product.search_index.reindex_queue.length
For more tips, check out Keeping Elasticsearch in Sync.
Searchkick supports Elasticsearch’s routing feature, which can significantly speed up searches.
class Business < ApplicationRecord
searchkick routing: true
def search_routing
city_id
end
end
Reindex and search with:
Business.search "ice cream", routing: params[:city_id]
Reindex a subset of attributes to reduce time spent generating search data and cut down on network traffic.
class Product < ApplicationRecord
def search_data
{
name: name
}.merge(search_prices)
end
def search_prices
{
price: price,
sale_price: sale_price
}
end
end
And use:
Product.reindex(:search_prices)
Split out conversions into a separate method so you can use partial reindexing, and cache conversions to prevent N+1 queries. Be sure to use a centralized cache store like Memcached or Redis.
class Product < ApplicationRecord
def search_data
{
name: name
}.merge(search_conversions)
end
def search_conversions
{
conversions: Rails.cache.read("search_conversions:#{self.class.name}:#{id}") || {}
}
end
end
Create a job to update the cache and reindex records with new conversions.
class ReindexConversionsJob < ApplicationJob
def perform(class_name)
# get records that have a recent conversion
recently_converted_ids =
Searchjoy::Search.where("convertable_type = ? AND converted_at > ?", class_name, 1.day.ago)
.order(:convertable_id).uniq.pluck(:convertable_id)
# split into groups
recently_converted_ids.in_groups_of(1000, false) do |ids|
# fetch conversions
conversions =
Searchjoy::Search.where(convertable_id: ids, convertable_type: class_name)
.group(:convertable_id, :query).uniq.count(:user_id)
# group conversions by record
conversions_by_record = {}
conversions.each do |(id, query), count|
(conversions_by_record[id] ||= {})[query] = count
end
# write to cache
conversions_by_record.each do |id, conversions|
Rails.cache.write("search_conversions:#{class_name}:#{id}", conversions)
end
# partial reindex
class_name.constantize.where(id: ids).reindex(:search_conversions)
end
end
end
Run the job with:
ReindexConversionsJob.perform_later("Product")
Searchkick makes it easy to use the Elasticsearch DSL on its own.
Create a custom mapping:
class Product < ApplicationRecord
searchkick mappings: {
product: {
properties: {
name: {type: "keyword"}
}
}
}
end
Note: If you use a custom mapping, you'll need to use custom searching as well.
To keep the mappings and settings generated by Searchkick, use:
class Product < ApplicationRecord
searchkick merge_mappings: true, mappings: {...}
end
And use the body
option to search:
products = Product.search body: {match: {name: "milk"}}
Note: This replaces the entire body, so other options are ignored.
View the response with:
products.response
To modify the query generated by Searchkick, use:
products = Product.search "milk", body_options: {min_score: 1}
or
products =
Product.search "apples" do |body|
body[:min_score] = 1
end
Searchkick is built on top of the elasticsearch gem. To access the client directly, use:
Searchkick.client
To batch search requests for performance, use:
fresh_products = Product.search("fresh", execute: false)
frozen_products = Product.search("frozen", execute: false)
Searchkick.multi_search([fresh_products, frozen_products])
Then use fresh_products
and frozen_products
as typical results.
Note: Errors are not raised as with single requests. Use the error
method on each query to check for errors. Also, if you use the below
option for misspellings, misspellings will be disabled.
Search across multiple indices with:
Searchkick.search "milk", index_name: [Product, Category]
Boost specific indices with:
indices_boost: {Category => 2, Product => 1}
To query nested data, use dot notation.
User.search "san", fields: ["address.city"], where: {"address.zip_code" => 12345}
Reindex one record
product = Product.find(1)
product.reindex
Reindex multiple records
Product.where(store_id: 1).reindex
Reindex associations
store.products.reindex
Remove old indices
Product.search_index.clean_indices
Use custom settings
class Product < ApplicationRecord
searchkick settings: {number_of_shards: 3}
end
Use a different index name
class Product < ApplicationRecord
searchkick index_name: "products_v2"
end
Use a dynamic index name
class Product < ApplicationRecord
searchkick index_name: -> { "#{name.tableize}-#{I18n.locale}" }
end
Prefix the index name
class Product < ApplicationRecord
searchkick index_prefix: "datakick"
end
For all models
Searchkick.index_prefix = "datakick"
Use a different term for boosting by conversions
Product.search("banana", conversions_term: "organic banana")
Multiple conversion fields
class Product < ApplicationRecord
has_many :searches, class_name: "Searchjoy::Search"
# searchkick also supports multiple "conversions" fields
searchkick conversions: ["unique_user_conversions", "total_conversions"]
def search_data
{
name: name,
unique_user_conversions: searches.group(:query).uniq.count(:user_id),
# {"ice cream" => 234, "chocolate" => 67, "cream" => 2}
total_conversions: searches.group(:query).count
# {"ice cream" => 412, "chocolate" => 117, "cream" => 6}
}
end
end
and during query time:
Product.search("banana") # boost by both fields (default)
Product.search("banana", conversions: "total_conversions") # only boost by total_conversions
Product.search("banana", conversions: false) # no conversion boosting
Change timeout
Searchkick.timeout = 15 # defaults to 10
Set a lower timeout for searches
Searchkick.search_timeout = 3
Change the search method name
Searchkick.search_method_name = :lookup
Change search queue name
Searchkick.queue_name = :search_reindex
Eager load associations
Product.search "milk", includes: [:brand, :stores]
Eager load different associations by model
Searchkick.search("*", index_name: [Product, Store], model_includes: {Product => [:store], Store => [:product]})
Run additional scopes on results
Product.search "milk", scope_results: ->(r) { r.with_attached_images }
Specify default fields to search
class Product < ApplicationRecord
searchkick default_fields: [:name]
end
Turn off special characters
class Product < ApplicationRecord
# A will not match Ä
searchkick special_characters: false
end
Use a different similarity algorithm for scoring
class Product < ApplicationRecord
searchkick similarity: "classic"
end
Change import batch size
class Product < ApplicationRecord
searchkick batch_size: 200 # defaults to 1000
end
Create index without importing
Product.reindex(import: false)
Lazy searching
products = Product.search("carrots", execute: false)
products.each { ... } # search not executed until here
Add request parameters, like search_type
and query_cache
Product.search("carrots", request_params: {search_type: "dfs_query_then_fetch"})
Reindex conditionally
class Product < ApplicationRecord
searchkick callbacks: false
# add the callbacks manually
after_commit :reindex, if: -> (model) { model.previous_changes.key?("name") } # use your own condition
end
Reindex all models - Rails only
rake searchkick:reindex:all
Turn on misspellings after a certain number of characters
Product.search "api", misspellings: {prefix_length: 2} # api, apt, no ahi
Note: With this option, if the query length is the same as prefix_length
, misspellings are turned off
Product.search "ah", misspellings: {prefix_length: 2} # ah, no aha
For performance, only enable Searchkick callbacks for the tests that need it.
Add to your test/test_helper.rb
:
# reindex models
Product.reindex
# and disable callbacks
Searchkick.disable_callbacks
And use:
class ProductTest < Minitest::Test
def setup
Searchkick.enable_callbacks
end
def teardown
Searchkick.disable_callbacks
end
def test_search
Product.create!(name: "Apple")
Product.search_index.refresh
assert_equal ["Apple"], Product.search("apple").map(&:name)
end
end
Add to your spec/spec_helper.rb
:
RSpec.configure do |config|
config.before(:suite) do
# reindex models
Product.reindex
# and disable callbacks
Searchkick.disable_callbacks
end
config.around(:each, search: true) do |example|
Searchkick.callbacks(true) do
example.run
end
end
end
And use:
describe Product, search: true do
it "searches" do
Product.create!(name: "Apple")
Product.search_index.refresh
assert_equal ["Apple"], Product.search("apple").map(&:name)
end
end
Use a trait and an after create
hook for each indexed model:
FactoryBot.define do
factory :product do
# ...
# Note: This should be the last trait in the list so `reindex` is called
# after all the other callbacks complete.
trait :reindex do
after(:create) do |product, _evaluator|
product.reindex(refresh: true)
end
end
end
end
# use it
FactoryBot.create(:product, :some_trait, :reindex, some_attribute: "foo")
Set:
Searchkick.index_suffix = ENV["TEST_ENV_NUMBER"]
Check out this great post on the Apartment gem. Follow a similar pattern if you use another gem.
Elasticsearch 6 removes the ability to reindex with the _all
field. Before you upgrade, we recommend disabling this field manually and specifying default fields on your models.
class Product < ApplicationRecord
searchkick _all: false, default_fields: [:name]
end
If you need search across multiple fields, we recommend creating a similar field in your search data.
class Product < ApplicationRecord
def search_data
{
all: [name, size, quantity].join(" ")
}
end
end
Elasticsearch is eventually consistent, meaning it can take up to a second for a change to reflect in search. You can use the refresh
method to have it show up immediately.
product.save!
Product.search_index.refresh
Due to the distributed nature of Elasticsearch, you can get incorrect results when the number of documents in the index is low. You can read more about it here. To fix this, do:
class Product < ApplicationRecord
searchkick settings: {number_of_shards: 1}
end
For convenience, this is set by default in the test environment.
View the changelog.
Thanks to Karel Minarik for Elasticsearch Ruby and Tire, Jaroslav Kalistsuk for zero downtime reindexing, and Alex Leschenko for Elasticsearch autocomplete.
- Reindex API
- Incorporate human eval
Everyone is encouraged to help improve this project. Here are a few ways you can help:
- Report bugs
- Fix bugs and submit pull requests
- Write, clarify, or fix documentation
- Suggest or add new features
If you’re looking for ideas, try here.
To get started with development and testing:
git clone https://github.com/ankane/searchkick.git
cd searchkick
bundle install
rake test