Zero Downtime Delayed Jobs

Here is a script I created that will do Zero Downtime Delayed Job restarts. This script will spin up a new set of delayed jobs workers and send a signal to the currently running workers to shut down after they are finished working on current work. The current script/delayed_job restart would only wait up to some timeout value and then would kill the job, potentially losing any work that it was working on.

I store this in a file called delayed_job_restart.sh. Make sure you “chmod +x delayed_job_restart.sh”. Feedback is welcome!

##############
CONFIGURATION
##############

# Set the rails environment.
RAILS_ENV=development

# This is the location of your rails application. (Rails.root)
APP_DIR=/path/to/app

# Whatever queues you want to handle.
QUEUES=queue1,queue2

# Number of delayed job instances.
NUMJOBS=6

########################
DONT EDIT BELOW HERE
UNLESS YOU UNDERSTAND IT
########################

# This is the file that will hold the location of
# the new round of delayed job pids.
NEW_PID_DIR_FILE=$APP_DIR/pids/delayed_job_pid_dir.new

# This is the directory where the new delayed job pids will live. We
# create a random directory.
NEW_PID_DIR=$APP_DIR/pids/delayed_job/`cat /dev/urandom | env LC_CTYPE=C tr -cd 'a-f0-9' | head -c 32`

# Make the directory and create the
# pid file.
mkdir -p $NEW_PID_DIR
touch $NEW_PID_DIR_FILE

# Put the pid dir into the pid file.
echo $NEW_PID_DIR > $NEW_PID_DIR_FILE

echo "Starting new delayed job processes..."
cd $APP_DIR
RAILS_ENV=$RAILS_ENV bundle exec script/delayed_job start --queues=$QUEUES --pid-dir=`cat $NEW_PID_DIR_FILE` -n $NUMJOBS
echo "Done."

# This is the current pid dir.
PID_DIR_FILE=$APP_DIR/pids/delayed_job_pid_dir

# If the pid dir exists, let tell them to shut down
# when they are done.
if [ -f $PID_DIR_FILE ]; then
  echo "Sending signal to stop old delayed job processes..."
  PID_DIR=`cat $PID_DIR_FILE`
  for f in $PID_DIR/*.pid
  do
    PID=`cat $f`
    echo " TERM to $PID"
    kill -TERM $PID
  done
  echo "Done."
  echo "Removing old PID directory."
  rm -rf $PID_DIR

fi

# Since we have a set of new jobs running,
# lets move the pid dir to the "current" dir.
mv $NEW_PID_DIR_FILE $PID_DIR_FILE

Faster Rails Tests

Anyone that has worked on a large rails application knows how long it takes for tests to run. I have this same problem as well. I use the build in testing framework packaged with Rails. One thing I noticed was that the Rails environment seemed to be loading between running unit, functional and integration tests. I felt this was unnecessary if I wanted to run the whole test suite, say for CI purposes, so I created a custom rake task that I use:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# lib/tasks/test.rake
require 'rake/testtask'
 
Rake::TestTask.new do |t|
 t.libs << "lib"
 t.libs << "test"
 t.name = "test:ci"
 t.warning = false
 t.verbose = false
 t.test_files = FileList[
 "test/unit/**/*_test.rb",
 "test/functional/**/*_test.rb",
 "test/integration/**/*_test.rb"
 ]
end

This allows me to run all my tests in a single application boot and shave a small amount of time off the full run:

1
bundle exec rake test:ci

Validating uniqueness of nested model on create

I recently ran into the issue where I setup a uniqueness validator on a field, with a scope on the parent’s ID. This was fine for updating with an existing parent, but if I ever created two or more new fields, the validation would fail because there was no ID on the parent model yet for the child to validate against.

This is apparently still an open issue with rails, as found by this url: https://rails.lighthouseapp.com/projects/8994/tickets/2160-nested_attributes-validates_uniqueness_of-fails.

There is a solution in that link, but I have came up with a similar method, with less code that seems to work fine and passes all my tests. I have also added to my solution the ability to add an error to the individual nested items that duplicate. This is how I fixed this (note that I make a lot of assumptions with code, just showing the example):

class Child < ActiveRecord::Base
  belongs_to :parent
  validates :value, :uniqueness => { :scope => :parent_id }
end
 
class Parent < ActiveRecord::Base
  has_many :children
  accepted_nested_attributes_for :children
 
  validate :uniqueness_of_children
 
  private
 
  def uniqueness_of_children
    hash = {}
    children.each do |child|
      if hash[child.value]
        # This line is needed to form the parent to error out, otherwise the save would still happen
        errors.add(:"children.value", "duplicate error") if errors[:"children.value"].blank?
        # This line adds the error to the child to view in your fields_for
        child.errors.add(:value, "has already been taken")
      end
      hash[child.value] = true
    end
  end
end

Let me know if you try this out and it works for you, or if it needs any improvements. Thanks!

How to use Delayed Job to handle your Carrierwave processing


This tutorial builds on my previous post about how to add FFMPEG processing to Carrierwave. Here I will show you my attempt at being able to utilize Delayed::Job to do the heavy lifting of processing when uploading files using Carrierwave. Remember, this could probably use some improvement, but it is a great starting point. So lets begin.

The first thing you will need to do is add Delayed::Job to your application:

# Gemfile
gem 'delayed_job'

Next you need to create the migration and migrate the database:

rails generate delayed_job
rake db:migrate

Now we get to the good part. Lets create a module to include into Carrierwave that will support holding off on doing the processing until Delayed::Job gets around to it:

# lib/carrier_wave/delayed_job.rb
module CarrierWave
  module Delayed
    module Job
      module ActiveRecordInterface
        def delay_carrierwave
          @delay_carrierwave ||= true
        end
 
        def delay_carrierwave=(delay)
          @delay_carrierwave = delay
        end
 
        def perform
          asset_name = self.class.uploader_options.keys.first
          self.send(asset_name).versions.each_pair do |key, value|
            value.process_without_delay!
          end
        end
 
        private
 
        def enqueue
          ::Delayed::Job.enqueue self
        end
      end
 
      def self.included(base)
        base.extend ClassMethods
      end
 
      module ClassMethods
        def self.extended(base)
          base.send(:include, InstanceMethods)
          base.alias_method_chain :process!, :delay
          ::ActiveRecord::Base.send(:include, CarrierWave::Delayed::Job::ActiveRecordInterface)
        end
 
        module InstanceMethods
          def process_with_delay!(new_file)
            process_without_delay!(new_file) unless model.delay_carrierwave
          end
        end
      end
    end
  end
end

Awesome! Now we need to tie this into our Uploader:

# app/uploaders/asset_uploader.rb
require File.join(Rails.root, "lib", "carrier_wave", "ffmpeg")
require File.join(Rails.root, "lib", "carrier_wave", "delayed_job") # New
 
class AssetUploader < CarrierWave::Uploader::Base
  include CarrierWave::Delayed::Job # New
  include CarrierWave::FFMPEG
 
  # Choose what kind of storage to use for this uploader:
  storage :file
 
  # Override the directory where uploaded files will be stored.
  # This is a sensible default for uploaders that are meant to be mounted:
  def store_dir
    "#{Rails.root}/uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end
 
  # Add a version, utilizing our processor
  version :bitrate_128k do
    process :resample => "128k"
  end
end

The last thing we have to do is update our model to queue up delayed job:

# app/models/asset.rb
class Asset < ActiveRecord::Base
  mount_uploader :asset, AssetUploader
 
  after_save :enqueue # New
end

There you have it. Now when you create a new Asset, associate a file, and save it, it shouldn’t run the processes, but instead create a Delayed::Job record. Then Delayed::Job should pick it up and run the processors on it. This may not be perfect, but at least its a start!

Thanks for reading!

Create FFMPEG processor for Carrierwave in Rails 3

I have had the pleasure of working with the carrierwave gem recently (as opposed to paperclip), and I must say, I am quite the fan. Once major thing I missed however, was the available list of custom user plugins for it, unlike paperclip. I believe this is mostly due to how new and recent carrierwave is. That being said, I put together a simple example of a FFMPEG process that will allow me to resample the bitrate of a file. This should lay the ground work for other features as well. This example is using Rails 3, but should be easily adaptable for 2. Also, make sure you already have FFMPEG installed and running properly. So lets get started:

First things first…we need to add the appropriate gems to our Gemfile:

# Gemfile
gem 'carrierwave'
gem 'streamio-ffmpeg'

Next is the meat and potatoes of this..the actual FFMPEG process for carrierwave. I choose to keep my plugin files in the directory lib/carrierwave. Make sure you have this path included in your application.rb file if you are using rails 3. Here is the code:

# lib/carrierwave/ffmpeg.rb
require 'streamio-ffmpeg'
module CarrierWave
  module FFMPEG
    module ClassMethods
      def resample( bitrate )
        process :resample => bitrate
      end
    end
 
    def resample( bitrate )
      directory = File.dirname( current_path )
      tmpfile   = File.join( directory, "tmpfile" )
 
      File.move( current_path, tmpfile )
 
      file = ::FFMPEG::Movie.new(tmpfile)
      file.transcode( current_path, :audio_bitrate => bitrate)
 
      File.delete( tmpfile )
    end
  end
end

Good. Now that we have the plugin coded up, we need to include it into our uploader. I already have one mounted to my Asset model. Here is what my AssetUploader now looks like:

# app/uploaders/asset_uploader.rb
require File.join(Rails.root, "lib", "carrier_wave", "ffmpeg")
 
class AssetUploader < CarrierWave::Uploader::Base
  include CarrierWave::FFMPEG # <= include the plugin
 
  # Choose what kind of storage to use for this uploader:
  storage :file
 
  # Override the directory where uploaded files will be stored.
  # This is a sensible default for uploaders that are meant to be mounted:
  def store_dir
    "#{Rails.root}/uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end
 
  # Add a version, utilizing our processor
  version :bitrate_128k do
    process :resample => "128k"
  end
end

There! Now whenever you add a new file, it should fire off the processor and create a new version. I hope this help anyone still up in the air about how to put together their own plugin/process for carrierwave. Next I will demonstrate how to incorporate Delayed::Job to move these intensive tasks to the background!

How to create PDF’s and Images from your website in Rails

I am going to show you how to generate both a pdf and image from a single action in a controller using the awesome, wkhtmltopdf library. This also uses PDFKit and WebSnap gems available on GitHub.

This example assumes the following:

  • wkhtmltopdf and wkhtmltoimage are already installed and accessible on in the PATH.
  • You have an html page setup to display the record.
  • You have created a pdf CSS file to help display the pdf, if you so choose.
  # config/initializers/mime_types.rb
  Mime::Type.register "application/pdf", :pdf
  Mime::Type.register "image/png", :png
 
  # app/controllers/items_controller.rb
  def show
    @item = Item.find(params[:id])
 
    respond_to do |format|
      format.html { }
      format.pdf {
        html = render :action => "show.html.erb"
        kit  = PDFKit.new( html, :zoom => 0.75 )
        kit.stylesheets << File.join( RAILS_ROOT, "public", "stylesheets", "pdf.css" )
 
        send_data kit.to_pdf, :filename => "item.pdf", :type => 'application/pdf', :disposition => 'inline'
      }
      format.png {
        html = render :action => "show.html.erb", :layout => "application.html.erb"
 
        # I am nil'ing these options out because my version of wkhtmltoimage does
        # not support the scale options and I do not want to crop the image at all.
        snap = WebSnap.new(html, :format => 'png', :'scale-h' => nil, :'scale-w' => nil,
          :'crop-h' => nil, :'crop-w' => nil, :quality => 100, :'crop-x' => nil, :'crop-y' => nil)
 
        send_data snap.to_bytes, :filename => "item.png", :type => "image/png", :disposition => 'inline'
      }
    end

Now you should be able to access three distinct views, each producing a different result

  http://example.com/items/1 # => Generates an html page.
  http://example.com/items/1.pdf # => Generates a pdf of the html page.
  http://example.com/items/1.png # => Generates a png of the html page.

You could easily also add more image types by just created another block for each format, and
changing the :format to whatever one you would like.

Request formats, filters, and functional tests…

I recently had to write some tests against a controller that was filtering based on the requesting format. In this case, I wanted to allow xml requests only, and redirect to login on everything else. This was fine when browsing or using curl by doing a simple:

skip_before_filter :login_required, :only => [:create], :if => Proc.new {|c| c.request.format.xml?}

My problem came when I was trying to create tests to verify that both html and xml requests did in fact produce the correct response. After many hours of messing around, I came up with a simple solution. First I skip the filters for every request, then I have another filter to re-enable them on anything but the xml request:

skip_before_filter :login_required, :only => [:create]
before_fiilter :only => [:create] do |c|
  c.send(:login_required) unless c.request.format.xml?
end

Voila!!! This allowed me to continue with my rails testing and browser and curl work appropriately. (I use curl to test the xml request).

def test_not_logged_in_normal_post
  post :create, :login => "test@test.com", :password => "test"
  assert_response :redirect
end
 
def test_not_logged_in_xml_post
  post :create, :format => 'xml', :login => "test@test.com", :password => "test"
  assert_response :success
end

Including methods and associations in a JSON Data set with Rails

I was poking around while working with creating an application specifically for web services. We decided to use JSON as the methods of transportation of data, but the problem came when I wanted to include custom methods, or associations in my data set. The solution was fairly simple, using the to_json method.

Suppose you have the following classes:

class Client < ActiveRecord::Base
  has_many :employees
end
 
class Employee < ActiveRecord::Base
  belongs_to :client
 
  def full_name
    "#{first_name} #{last_name}"
  end
end

We want the controller to return a client with association employees and the full name in the database. Here is how we would go about doing that:

def show
  @client = Client.find(params[:id])
 
  respond_to do |format|
    format.json { render :json => @client.to_json(
      :include => {
        :employee => {
          :only => :email,
          :methods => [ :full_name ]
        }
      }
    ) }
  end
end

You will end up with the following data set:

{ client: { name: "Some client", employee: { email: "test@test.com", full_name: "John Doe" } } }

Forgive me if I messed up the json output…doing it from memory :) There are of course way easier uses for this too, but I just decided to spit out a more complex one.

Will paginate – how to pass custom parameters

I came across and issue today that involved my pagination not working correctly. What was happening was a user would do a search with various criteria and the result would show with pagination links. Once they clicked on a new page, it would return them to a list of results, but that wasn’t using any criteria. The solution was fairly simple, but not obviously documented. (Unless you look in the source files)

<%= will_paginate @object, :params => { :custom_param => @custom_param_value } %>

Here are some additional custom params you can use:

Display options

  • :previous_label — default: “<< Previous" (this parameter is called :prev_label in versions 2.3.2 and older!)
  • :next_label — default: “Next >>”
  • :page_links — when false, only previous/next links are rendered (default: true)
  • :inner_window — how many links are shown around the current page (default: 4)
  • :outer_window — how many links are around the first and the last page (default: 1)
  • :separator — string separator for page HTML elements (default: single space)

HTML options

  • :class — CSS class name for the generated DIV (default: “pagination”)
  • :container — toggles rendering of the DIV container for pagination links, set to false only when you are rendering your own pagination markup (default: true)
  • :id — HTML ID for the container (default: nil). Pass +true+ to have the ID automatically generated from the class name of objects in collection: for example, paginating ArticleComment models would yield an ID of “article_comments_pagination”.

Advanced options

  • :param_name — parameter name for page number in URLs (default: :page)
  • :params — additional parameters when generating pagination links (eg. :controller => “foo”, :action => nil)
  • :renderer — class name, class or instance of a link renderer (default: WillPaginate::LinkRenderer)