How to Prevent Sequence Race Conditions when Using Rails ActiveJob

This article describes using the Ruby on Rails web application framework along with an external service. It is designed to help prevent race conditions from being present while using a sequence of requests in ActiveJob.

Ruby on Rails

Ruby On Rails is a web application framework that includes everything needed to create database-backed web applications according to the Model-View-Controller (MVC) pattern. It is very popular and is used by many big companies, such as GitHub, Airbnb, Twitch, SoundCloud, Hulu, Zendesk, Square, Cookpad, and others. Many startups use it too. It is open source and its source code is available on GitHub.

Ruby on Rails is written in the Ruby programming language. If you want to get a good job as a web developer, you should definitely learn it. There are many free online resources for learning Ruby, such as The Odin Project, The Ruby on Rails Tutorial, among others. There is also a big library of courses online on platforms like coursera, codecamp, and several others.

On the MVC pattern, the model is the data layer that is responsible for handling the application’s data, logic and rules. The view is the user interface. It displays the data (the model) in a human-readable format. The controller is the glue between the model and the view. It handles user input and performs interactions on the data model objects. It then selects a view to render and returns it to the user. One key aspect of a performant Rails application is that the responses from the controller need to be fast.

When dealing with a slow process on a controller, the first thing to do is to check if the process is really slow or if it is just taking a long time to respond. You can do this by using the time command in the terminal. For example when requesting data from another service, you cannot control the response time or any of its variations. So, a common strategy is to execute the task in the background.

Active Job

Active Job is a framework for declaring jobs and making them run on a variety of queuing backends. These jobs can be everything from regularly scheduled clean-ups, to billing charges, to mailings. Anything that can be chopped up into small units of work and run in parallel, really. Active Job makes it easy to switch between different queuing systems by simply changing a few lines of code. It also provides a unified interface to those queuing systems, so that each one can be seamlessly interchanged. This means that you can start with one system in development and another in production.

If you are using Rails 4.2 or higher, you can use Active Job. If you are using Rails 4.1 or lower, you can use Delayed Job. Delayed Job is a simple, flexible, and reliable background processing plugin for Ruby. It is designed to be simple, yet powerful. It is a good choice for Rails 4.1 or lower.

Declaring a job looks like this:

class MyJob < ApplicationJob
    def perform(*args)
        # Do something later
    end
end

To run a job, you can use theperform_later method. For example:

MyJob.perform_later('bob', 5)

You can also use theperform_nowmethod to run the job immediately. For example:

MyJob.perform_now('bob', 5)

These two methods will be important when using Active Job and structuring your code.

Now, let’s assume that you are using Active Job and you have a slow process on your controller. You can use the perform_later method to run the slow process in the background. This way the controller will respond fast and the slow process will run in the background. This is a good strategy for dealing with slow processes on the controller. and for the sake of our example let’s fetch data from an external api like a CRM client.

CRM Client

CRM.create_lead is a method that creates a lead on the CRM client. This method is slow and takes a long time to respond. So we want to run this method in the background.

class CrmCreateLeadJob < ApplicationJob
    queue_as :default

    def perform(name, email)
        id = CRM.create_lead(name, email)
        Customer.create(name: name, email: email, crm_id: id)
    end
end

CRM.set_status is a method that sends data to an external api. This is a common scenario when dealing with external apis. You cannot control the response time or any of its variations. So again, executing the task in the background is the way to go.

Now let’s assume that we want to fetch data from the CRM client on the controller and send data to the CRM client on the controller. We can do this by using the perform_later method. This way the controller will respond fast and the slow process will run in the background. This is a good strategy for dealing with slow processes on the controller.

class CrmSetStatusJob < ApplicationJob
    queue_as :default

    def perform(id, status)
        CRM.set_status(id, status)
    end
end

Customer Controller

class CustomerController < ApplicationController
    def create_lead
        CrmCreateLeadJob.perform_later(params[:name], params[:email])
    end
end

Race Condition

A race condition in programming is a situation when two processes are being executed at the same time but have either constraints on access to the same data or one depends on the other process forming a sequence. This can lead to unexpected results. For example, if two processes are trying to access the same data at the same time, one of the processes may overwrite the other process’s data. This can lead to unexpected results. In the case of a sequence the second process may try to run before the first process has finished. This can lead to unexpected results.

Creating a Sequence Race Condition

In order to keep this situation as simple as possible, we are defining a constraint on the lead API, you cannot set the status when creating the lead. So the status definition needs to be done as a separate request to the API.

When we get a new lead on our Rails application we want to create a lead on the CRM client.

We evaluate some data about the user and then we set the status of the lead on the CRM client.

class CustomerController < ApplicationController
    def create_lead
        customer = Customer.create(name: params[:name], email: params[:email])
        CrmCreateLeadJob.perform_later(params[:name], params[:email])
        status = Status.compute_status_for(customer.id)
        CrmSetStatusJob.perform_later(params[:id], status)
    end
end

Even though we are using the perform_later method, we are not actually executing the code because the jobs will run at a later time so this code will not work as expected. Let's change the code on the set_status method to get the status by itself.

class CustomerController < ApplicationController
    def create_lead
        customer = Customer.create(name: params[:name], email: params[:email])
        CrmCreateLeadJob.perform_later(customer.name, customer.email)
        CrmSetStatusJob.perform_later(params[:id])
    end
end

class CrmSetStatusJob < ApplicationJob
    queue_as :default

    def perform(id)
        crm_id = Customer.find(id)
        status = Status.compute_status_for(id)
        CRM.set_status(crm_id , status)
    end
end

Now, this code is better because the set_status job will compute its own status and will not depend on the create_lead job. But we still have a problem. The set_status job will run at a later time and the create_lead job will run at a later time. So there is a chance that the create_lead job will not have finished yet. So we need to make sure that the create_lead job will finish before the set_status job runs. We can do this by using the perform_now method.

Using Synchronous and Asynchronous Jobs

class CustomerController < ApplicationController
    def create_lead
        customer = Customer.create(name: params[:name], email: params[:email])
        CrmCreateLeadJob.perform_now(customer.name, customer.email)
        CrmSetStatusJob.perform_later(customer.id)
    end
end

This code will work as expected. But we have a problem. The create_lead job will run synchronously and the set_status job will run asynchronously. So the create_lead job will block the controller and the set_status job will run in the background. This is not a good strategy because the controller will be blocked and the user will have to wait for the create_lead job to finish. So we need to find a way to run the create_lead job asynchronously and the set_status job asynchronously.

Encapsulating Jobs

So we create another job that will run the create_lead job synchronously and the set_status job synchronously.

class createLeadAndSetStatusJob < ApplicationJob
    queue_as :default

    def perform(id)
        customer = Customer.find(id)
        CrmCreateLeadJob.perform_now(customer.name, customer.email)
        CrmSetStatusJob.perform_now(customer.id)
    end
end

class CustomerController < ApplicationController
    def create_lead
        customer = Customer.create(name: params[:name], email: params[:email])
        createLeadAndSetStatusJob.perform_later(customer.id)
    end
end