Mount a Solr search engine with Rails using Docker Compose

One of the most interesting challenges while developing an application is indexing and searching for data efficiently. Solr is a structured-document database which uses index and searches for technologies to perform its features, some of them are full-text-search, faceted search, real-time indexing, etc.

In a development team, it could be hard, and it could take a while to set up the environment for every computer, and even more if each developer has different environments (different operative systems) which hinder the process.

In order to make the process faster, and to avoid configuring all the stuff, we could use docker images.

For test purposes of this post, we will set up an application with Rails and Solr using Docker Compose. We will use Postgresql as the database engine.

So, let's set the application up.

Used Versions

• docker 18.09.2.
• docker-compose 1.23.2.

Defining The Project

To define the build dependencies of the app we are going to use a file called Dockerfile with the content below.

  FROM ruby:2.4.1
  RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs

  RUN mkdir /app
  WORKDIR /app
  ADD Gemfile /app/Gemfile
  ADD Gemfile.lock /app/Gemfile.lock
  RUN bundle install
  ADD . /app

Now we need to create an initial Gemfile which just loads Rails

  # frozen_string_literal: true

  source 'https://rubygems.org'
  ruby '2.4.1'

  gem 'rails', '~> 5.1', '>= 5.1.6'

Create an empty Gemfile.lock to build our Dockerfile.

Finally. let's create our docker-compose.yml file, this is where the services are defined. To make a simple web application structure, we are going to use a Postgres docker image and build the web image with the Dockerfile that we previously defined, to do so we will add to the web service the property build ..

  version: '3'
  services:
    db:
      image: postgres
      volumes:
        - 'postgres:/var/lib/postgresql/data'

    web:
      build: .
      command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
      volumes:
        - '.:/app'
      ports:
        - '3000:3000'
      depends_on:
        - db
      stdin_open: true
      tty: true

  volumes:
    postgres:

We made the web service dependent on the db service to wait for the start of the last one. Also, we configured the volume of the db service (postgres), so it is visible for all the services.

Building The Project

Now we have the necessary files to build the project and generate the Rails skeleton. First, we need to generate the rails skeleton by running the command below.

  docker-compose run web rails new . --force --no-deps --database=postgresql

Once we have the rails skeleton, let's build the images.

  docker-compose up --build

For now, let's stop the services by pressing the ctrl + c combination keys, and then we run the command below.

  docker-compose down

Connecting The Database

Replace the content of the config/database.yml with the following.

  default: &default
    adapter: postgresql
    encoding: unicode
    host: db
    username: postgres
    password:
    pool: 5

  development:
    <<: *default
    database: myapp_development

  test:
    <<: *default
    database: myapp_test

  production:
    <<: *default
    database: <%= ENV['DATABASE_URL'] %>

Notice that the host is db which is the name of the Postgres service in the docker-compose file. Now you can create the Database

  docker-compose run web rake db:create

Creating A Simple Application

So far we have a rails infrastructure with Postgres as database engine mounted on docker. It's time to use it in order to create an application. We will create a classic application of author-books.

Migrations

Let's create the migrations for every model by running the commands below.

  bash$ docker-compose run web rails generate model Author name:string
  bash$ docker-compose run web rails generate model Book name:string synopsis:text

After that, we need to add a reference to the author directly in the book migration db/migrate/...create_books.rb.

  . . .
  t.text :synopsis
  t.references :author, null: false, index: true
  . . .

Now we can run the migrations in the console.

  docker-compose run web rails db:migrate

Models

Establishing the associations in the models.

Book

  # frozen_string_literal: true

  class Book < ApplicationRecord
    belongs_to :author
  end

Author

  # frozen_string_literal: true

  class Author < ApplicationRecord
    has_many :books
  end

Controllers

Books

  # frozen_string_literal: true

  class BooksController < ApplicationController
    def index
      @books = Book.all
    end
  end

With this, you can make the design of your book's index view.

Adding Solr

Solr Service on the docker-compose file

docker-compose.yml.

  version: '3'
  services:
    db:
      image: postgres
      volumes:
        - 'postgres:/var/lib/postgresql/data'

    solr:
      image: solr:7.0.1
      ports:
        - "8983:8983"
      volumes:
        - solr_data:/opt/solr/server/solr/mycores
      entrypoint:
        - docker-entrypoint.sh
        - solr-precreate
        - development
      depends_on:
        - db

    web:
      build: .
      command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
      volumes:
        - '.:/app'
      ports:
        - '3000:3000'
      depends_on:
        - db
        - solr
      stdin_open: true
      tty: true

  volumes:
    postgres:
    solr_data:

The new service was added to the docker compose from the image: Solr 7.0.1. This service was added as a dependency of the web service and its volume in a global scope.

Add the sunspot gem

Gemfile

  . . .
  gem 'sunspot_rails'
  . . .

In order to install the docker service and the sunspot gem we will re-build the images by running the command below.

  docker-compose up --build

At this point we can access to the Solr admin console on the URL: http://localhost:8983.On this console go to the sidebar at the bottom, select the core development, and then select Schema. This action will change the view of the body. Here click on the Add Field button., and in the popup that will appear, fill the name field with "type", select the "String" option in the field type, and make sure that only the indexed and multiValued options are checked. Finally, click on the Add Field button at the bottom of the popup.

The type field is a field which is required by the sunspot gem.

Now we need to generate the sunspot config files with the command below.

  docker-compose run web rails generate sunspot_rails:install

Change the config in the config/sunspot.yml file. In the development section, change the hostname to solr and the port to 8983.

  . . .
  development:
    solr:
      hostname: solr
      port: 8983
      log_level: INFO
      path: /solr/development
  . . .

Add the index

To establish the index, set it up in the models. app/models/book.rb.

  . . .
  searchable do
    text :name, :synopsis
    text :author do
      author.name
    end
  end
  . . .

Let's reindex for all the models.

  docker-compse run web rake sunspot:solr:reindex

Change the index method in the controller in order to perform the search action.

  . . .
  def index
    @search_result = Sunspot.search(Book) do
      fulltext params[:keyword] if params[:keyword].present?
    end

    @hits = @search_result.hits
    @books = @hits.map(&:instance)
  end
  . . .

Now you can search for books by their names, synopsis and the author’s name.