Setting up Background Job (BJ) Properly

23 June 2008

One of the features of Bj is that is signals the runner via a HUP when a new job is ready to collect, breaking the standard 42 second sleep between checking for jobs and processing the job immediately, this did not seem to be the case for this customer however.

The first place I checked /bj-1.0.1/lib/bj/api.rb where the magic job submit command action happens, glancing at the submit method I saw the following...

1
2
3
4
5
6
7
8

def submit jobs, options = {}, &block
      options.to_options!
      Bj.transaction(options) do
        table.job.submit jobs, options, &block
      end
    ensure
      Bj.runner.tickle unless options[:no_tickle]
    end

Uh-oh, looks like we turned tickle off in order to move the runner out of the control of our app, we should probably check what that method does...

/bj-1.0.1/lib/bj/runner.rb

1
2
3
4

def tickle
     return nil if Bj.config[Runner.no_tickle_key]
     ping or start
  end

Again, looks like we're checking if the tickle option is turned off before allowing the tickle method to continue further and then calling ping or start.

The ping method has what we're looking for, down on line 85 of runner.rb

1

process.kill Runner.hup_signal, pid

So it definitely looks like when the runner is not under the direct control of the app we lose our auto start and job signalling abilities - this is probably wise, if we didn't start the runner directly then its pid would be an educated guess and not signalling seems to fit the model of running the job runner outside of the influence of the submitting application.

Now, how would you go about cutting down on the potential 42 second delay before jobs being picked up if you use cron to start the runner. It's simple really, with a bit more digging it turns out that there is an option which doesn't pop up on the ./bin/bj run command syntax which lets you over-ride the 42 second sleep time and avoid job delay.

1
2

def run
    wait = options[:wait] || 42

Bingo! We set that option to 2 second on our cronjob line (2 seconds isn't an unreasonably short amount of time to do a single sql query on the job table) and the problem was fixed!

*/2 * * * * bj run --forever --wait=2 --rails_env=production --rails_root=/data/user/current

Comments

Sorry, comments are closed for this article.