Freezzo.com

How to use Delayed Job to handle your Carrierwave processing

Randy — Thu, 06 Jan 2011 14:26:49 +0000

This tutorial builds on my previous post about how to add FFMPEG processing to Carrierwave. Here I will show you my attempt at being able to utilize Delayed::Job to do the heavy lifting of processing when uploading files using Carrierwave. Remember, this could probably use some improvement, but it is a great starting point. So lets begin.

The first thing you will need to do is add Delayed::Job to your application:

# Gemfile
gem 'delayed_job'

Next you need to create the migration and migrate the database:

rails generate delayed_job
rake db:migrate

Now we get to the good part. Lets create a module to include into Carrierwave that will support holding off on doing the processing until Delayed::Job gets around to it:

# lib/carrier_wave/delayed_job.rb
module CarrierWave
  module Delayed
    module Job
      module ActiveRecordInterface
        def delay_carrierwave
          @delay_carrierwave ||= true
        end
 
        def delay_carrierwave=(delay)
          @delay_carrierwave = delay
        end
 
        def perform
          asset_name = self.class.uploader_options.keys.first
          self.send(asset_name).versions.each_pair do |key, value|
            value.process_without_delay!
          end
        end
 
        private
 
        def enqueue
          ::Delayed::Job.enqueue self
        end
      end
 
      def self.included(base)
        base.extend ClassMethods
      end
 
      module ClassMethods
        def self.extended(base)
          base.send(:include, InstanceMethods)
          base.alias_method_chain :process!, :delay
          ::ActiveRecord::Base.send(:include, CarrierWave::Delayed::Job::ActiveRecordInterface)
        end
 
        module InstanceMethods
          def process_with_delay!(new_file)
            process_without_delay!(new_file) unless model.delay_carrierwave
          end
        end
      end
    end
  end
end

Awesome! Now we need to tie this into our Uploader:

# app/uploaders/asset_uploader.rb
require File.join(Rails.root, "lib", "carrier_wave", "ffmpeg")
require File.join(Rails.root, "lib", "carrier_wave", "delayed_job") # New
 
class AssetUploader < CarrierWave::Uploader::Base
  include CarrierWave::Delayed::Job # New
  include CarrierWave::FFMPEG
 
  # Choose what kind of storage to use for this uploader:
  storage :file
 
  # Override the directory where uploaded files will be stored.
  # This is a sensible default for uploaders that are meant to be mounted:
  def store_dir
    "#{Rails.root}/uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end
 
  # Add a version, utilizing our processor
  version :bitrate_128k do
    process :resample => "128k"
  end
end

The last thing we have to do is update our model to queue up delayed job:

# app/models/asset.rb
class Asset < ActiveRecord::Base
  mount_uploader :asset, AssetUploader
 
  after_save :enqueue # New
end

There you have it. Now when you create a new Asset, associate a file, and save it, it shouldn’t run the processes, but instead create a Delayed::Job record. Then Delayed::Job should pick it up and run the processors on it. This may not be perfect, but at least its a start!

Thanks for reading!

Create FFMPEG processor for Carrierwave in Rails 3

Randy — Thu, 23 Dec 2010 16:20:13 +0000

I have had the pleasure of working with the carrierwave gem recently (as opposed to paperclip), and I must say, I am quite the fan. Once major thing I missed however, was the available list of custom user plugins for it, unlike paperclip. I believe this is mostly due to how new and recent carrierwave is. That being said, I put together a simple example of a FFMPEG process that will allow me to resample the bitrate of a file. This should lay the ground work for other features as well. This example is using Rails 3, but should be easily adaptable for 2. Also, make sure you already have FFMPEG installed and running properly. So lets get started:

First things first…we need to add the appropriate gems to our Gemfile:

# Gemfile
gem 'carrierwave'
gem 'streamio-ffmpeg'

Next is the meat and potatoes of this..the actual FFMPEG process for carrierwave. I choose to keep my plugin files in the directory lib/carrierwave. Make sure you have this path included in your application.rb file if you are using rails 3. Here is the code:

# lib/carrierwave/ffmpeg.rb
require 'streamio-ffmpeg'
module CarrierWave
  module FFMPEG
    module ClassMethods
      def resample( bitrate )
        process :resample => bitrate
      end
    end
 
    def resample( bitrate )
      directory = File.dirname( current_path )
      tmpfile   = File.join( directory, "tmpfile" )
 
      File.move( current_path, tmpfile )
 
      file = ::FFMPEG::Movie.new(tmpfile)
      file.transcode( current_path, :audio_bitrate => bitrate)
 
      File.delete( tmpfile )
    end
  end
end

Good. Now that we have the plugin coded up, we need to include it into our uploader. I already have one mounted to my Asset model. Here is what my AssetUploader now looks like:

# app/uploaders/asset_uploader.rb
require File.join(Rails.root, "lib", "carrier_wave", "ffmpeg")
 
class AssetUploader < CarrierWave::Uploader::Base
  include CarrierWave::FFMPEG # <= include the plugin
 
  # Choose what kind of storage to use for this uploader:
  storage :file
 
  # Override the directory where uploaded files will be stored.
  # This is a sensible default for uploaders that are meant to be mounted:
  def store_dir
    "#{Rails.root}/uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end
 
  # Add a version, utilizing our processor
  version :bitrate_128k do
    process :resample => "128k"
  end
end

There! Now whenever you add a new file, it should fire off the processor and create a new version. I hope this help anyone still up in the air about how to put together their own plugin/process for carrierwave. Next I will demonstrate how to incorporate Delayed::Job to move these intensive tasks to the background!

Moved to a new host!

Randy — Sat, 09 Oct 2010 18:38:38 +0000

I have spend the last week setting up a new VPS host at Linode. This site and my youtube video playlist site, http://www.jamzee.com, are both currently running on the new host. Everything has been great so far and this will give me more control in the future!

If you are interested in joining Linode, check out my referral link.

How to create PDF’s and Images from your website in Rails

Randy — Thu, 29 Jul 2010 15:03:07 +0000

I am going to show you how to generate both a pdf and image from a single action in a controller using the awesome, wkhtmltopdf library. This also uses PDFKit and WebSnap gems available on GitHub.

This example assumes the following:

wkhtmltopdf and wkhtmltoimage are already installed and accessible on in the PATH.
You have an html page setup to display the record.
You have created a pdf CSS file to help display the pdf, if you so choose.

  # config/initializers/mime_types.rb
  Mime::Type.register "application/pdf", :pdf
  Mime::Type.register "image/png", :png
 
  # app/controllers/items_controller.rb
  def show
    @item = Item.find(params[:id])
 
    respond_to do |format|
      format.html { }
      format.pdf {
        html = render :action => "show.html.erb"
        kit  = PDFKit.new( html, :zoom => 0.75 )
        kit.stylesheets << File.join( RAILS_ROOT, "public", "stylesheets", "pdf.css" )
 
        send_data kit.to_pdf, :filename => "item.pdf", :type => 'application/pdf', :disposition => 'inline'
      }
      format.png {
        html = render :action => "show.html.erb", :layout => "application.html.erb"
 
        # I am nil'ing these options out because my version of wkhtmltoimage does
        # not support the scale options and I do not want to crop the image at all.
        snap = WebSnap.new(html, :format => 'png', :'scale-h' => nil, :'scale-w' => nil,
          :'crop-h' => nil, :'crop-w' => nil, :quality => 100, :'crop-x' => nil, :'crop-y' => nil)
 
        send_data snap.to_bytes, :filename => "item.png", :type => "image/png", :disposition => 'inline'
      }
    end

Now you should be able to access three distinct views, each producing a different result

  http://example.com/items/1 # => Generates an html page.
  http://example.com/items/1.pdf # => Generates a pdf of the html page.
  http://example.com/items/1.png # => Generates a png of the html page.

You could easily also add more image types by just created another block for each format, and
changing the :format to whatever one you would like.

Request formats, filters, and functional tests…

Randy — Thu, 04 Mar 2010 19:09:01 +0000

I recently had to write some tests against a controller that was filtering based on the requesting format. In this case, I wanted to allow xml requests only, and redirect to login on everything else. This was fine when browsing or using curl by doing a simple:

skip_before_filter :login_required, :only => [:create], :if => Proc.new {|c| c.request.format.xml?}

My problem came when I was trying to create tests to verify that both html and xml requests did in fact produce the correct response. After many hours of messing around, I came up with a simple solution. First I skip the filters for every request, then I have another filter to re-enable them on anything but the xml request:

skip_before_filter :login_required, :only => [:create]
before_fiilter :only => [:create] do |c|
  c.send(:login_required) unless c.request.format.xml?
end

Voila!!! This allowed me to continue with my rails testing and browser and curl work appropriately. (I use curl to test the xml request).

def test_not_logged_in_normal_post
  post :create, :login => "test@test.com", :password => "test"
  assert_response :redirect
end
 
def test_not_logged_in_xml_post
  post :create, :format => 'xml', :login => "test@test.com", :password => "test"
  assert_response :success
end

Including methods and associations in a JSON Data set with Rails

Randy — Fri, 02 Oct 2009 13:58:11 +0000

I was poking around while working with creating an application specifically for web services. We decided to use JSON as the methods of transportation of data, but the problem came when I wanted to include custom methods, or associations in my data set. The solution was fairly simple, using the to_json method.

Suppose you have the following classes:

class Client < ActiveRecord::Base
  has_many :employees
end
 
class Employee < ActiveRecord::Base
  belongs_to :client
 
  def full_name
    "#{first_name} #{last_name}"
  end
end

We want the controller to return a client with association employees and the full name in the database. Here is how we would go about doing that:

def show
  @client = Client.find(params[:id])
 
  respond_to do |format|
    format.json { render :json => @client.to_json(
      :include => {
        :employee => {
          :only => :email,
          :methods => [ :full_name ]
        }
      }
    ) }
  end
end

You will end up with the following data set:

{ client: { name: "Some client", employee: { email: "test@test.com", full_name: "John Doe" } } }

Forgive me if I messed up the json output…doing it from memory There are of course way easier uses for this too, but I just decided to spit out a more complex one.

Will paginate – how to pass custom parameters

Randy — Fri, 31 Jul 2009 18:06:42 +0000

I came across and issue today that involved my pagination not working correctly. What was happening was a user would do a search with various criteria and the result would show with pagination links. Once they clicked on a new page, it would return them to a list of results, but that wasn’t using any criteria. The solution was fairly simple, but not obviously documented. (Unless you look in the source files)

<%= will_paginate @object, :params => { :custom_param => @custom_param_value } %>

Here are some additional custom params you can use:

Display options

:previous_label — default: “<< Previous" (this parameter is called :prev_label in versions 2.3.2 and older!)
:next_label — default: “Next >>”
:page_links — when false, only previous/next links are rendered (default: true)
:inner_window — how many links are shown around the current page (default: 4)
uter_window — how many links are around the first and the last page (default: 1)
:separator — string separator for page HTML elements (default: single space)

HTML options

:class — CSS class name for the generated DIV (default: “pagination”)
:container — toggles rendering of the DIV container for pagination links, set to false only when you are rendering your own pagination markup (default: true)
:id — HTML ID for the container (default: nil). Pass +true+ to have the ID automatically generated from the class name of objects in collection: for example, paginating ArticleComment models would yield an ID of “article_comments_pagination”.

Advanced options

:param_name — parameter name for page number in URLs (default: :page)
:params — additional parameters when generating pagination links (eg. :controller => “foo”, :action => nil)
:renderer — class name, class or instance of a link renderer (default: WillPaginate::LinkRenderer)

How to trim a string and fill remaining space in Ruby

Randy — Sun, 14 Jun 2009 19:53:36 +0000

So I needed a way to take a string, and depending on its size, shorten it and fill the remaining space with some arbitrary sequence of characters. Ljust would work, except for the fact that it will fill a string up to a given length, but I only needed to do this when over a certain size. Here is an example:

str = "Somereallylongstring"
if str.length > 10
  puts str[0..7] + '...'
else
  puts str
end
# => "Somerea..."

This is the basic idea of what I wanted to do. I decide to make it cleaner and override the String class like so:

class String
  def lfill(len = self.length, fill = '.')
    tmp = self[0..(len - 3)] + '...' if self.length - 3 > len
    return tmp || self
  end
end
 
str = "Somereallylongstring"
puts str.lfill(10)
 
#=> "Somerea..."

acts_as_similar – A basic similarity activerecord plugin

Randy — Thu, 04 Jun 2009 17:34:32 +0000

I had a need to find items that were similar to an item. I looked at acts_as_recommendable, and while this was very nice plugin, it was too slow when working with large data sets. I also decided that the results I needed didn’t need to be scientifically accurate, just a rough match. So I created this plugin to allow for a very elementary way to find items that are similar to the object you are working with. This works best when used with a has_many :through relationship.

Basically all it is doing is looking for other objects of the same class, that have some related value.

Git it here: http://github.com/freezzo/acts_as_similar/tree/master

Example 1
=========
Look for similar playlists, using the videos as what defines similarity.
This will look for all playlists that have a similar video as the playlist you are looking against.

class Playlist
  has_many :playlists_videos
  has_many :videos, :through => :playlists_videos
 
  acts_as_similar :videos
end

Example 2
=========
Look for similar playlists, using the title of the playlist as the similarity item.
This will look for all playlists that have a similar title as the playlist you are looking against.

class Playlist
  has_many :playlists_videos
  has_many :videos, :through => :playlists_videos
 
  acts_as_similar :field => :title
end

Example 3
=========
Look for similar playlists, using the video_id of the playlists_videos as the similarity item.
This will look for all playlists that have a similar video_id as the playlist you are looking against.

class Playlist
  has_many :playlists_videos
  has_many :videos, :through => :playlists_videos
 
  acts_as_similar :playlists_videos, :field => :video_id
end

Execute
=========

@playlist = Playlist.first
@playlist.similar

Note: This is a work in progress.

ActiveRecord – Making :include and :select play nice!

Randy — Wed, 20 May 2009 15:00:45 +0000

So I have been using a nice plug-in that will allow me to using :select when using :include, and not have it pull the entire data set. You can add the plug-in to your app like this:

script/plugin install git://github.com/blythedunham/eload-select.git

Here are some ways to use the plug-in:

Employee.find :all,
    :select => 'addresses.city, address.state, employees.*',
    :include => :address

Employee.find :first,
    :select => 'now() as current_time, addresses.city, DATE(addresses.created_at) as addresses.created_at, employee.*',
    :include => :address

Employee.find :all,
    :select => 'addresses.city, employees.name, employees.start_date',
    :include => :address

Examples taken from:
http://www.snowgiraffe.com/tech/329/eager-loading-select-plugin-when-select-plays-nice-with-include/