Curiouser and Curiouser!

An app in a week

Today I released version 1.0.9 of Spike my Rails log viewer app for Mac OS X. If you want to see what Spike does I have put a new screencast online today.

In a little over a week it went from being an idea, to being a rough, but functional, sketch of an app, to being a tidy little app that works quite well. Adding reload today was the last major step I had in mind.

Bindings is a lot of what makes it possible. There is actually very little code holding Spike together (see for yourself on GitHub). Most of my effort went into the LogParser class.

Bindings are a little hard to diagnose when they go wrong, fussy to setup in Interface Builder (so much so that, in Elysium where I have vast, vast, numbers of bindings I do them all in code), but they really do make life simple.

I was really left with one main problems to solve: How to parsing the Ruby hash of parameters passed to each request. The other stuff I was lifting out of the request using Cocoa's NSScanner and looking for prefixes. Parsing potentially deeply nested hash structures required something more sophisticated.

My first reaction was to turn to Ragel again. But I hadn't used Ragel for some time and it seemed a daunting proposition to relearn it and figure out the integration with Objective-C and Xcode. I started Googling for something, anything, easier and quite by chance came up against [Todd Ditchendorf]'s(http://ditchnet.org/) (he of Fluid fame) pet project TDParseKit.

TDParseKit turns out to be a rather neat Objective-C toolkit for assembling tokenizers and parsers and Todd has done an amazing amount of work both creating the kit and creating sample parsers for a range of languages. If you're looking to parse, for example JSON, Todd already has you covered. I still have some work to do on the parameter parser and some requests don't parse properly because of file references (for uploads). But Todd's already been a big help with that. TDParseKit is a very cool project.

The only other major hurdle I hit was a very strange problem that occurred intermittently, more often with some files for reasons I could not deduce, where the results table wouldn't display a row until you clicked on it first. It was driving me nuts.

Fortunately I was able to talk it over with Rainer Brockerhoff author, most recently, of Quay and Klicko. I'm very lucky to be able to call on him as he's both great to chat too (even though we have differening opinions on syntax sugar!) and has a wealth of experience with Mac programming that few I've met can equal.

After talking the problem over with Rainer he pointed out that I was updating a bound property from a background thread. The inconsistency of the problem should have pointed to timing issues & threading but I couldn't see the wood for the trees since I'd already decided it was a bindings/NSTableView problem. Problem solved!

I've learned a bit about table rendering, NSDocument, NSSearchFields, and the joy of using NSPredicate.

All in all it's been fun.

I see v1.0.9 of Spike as relatively feature complete. It does a job and, I think, does it pretty well. I'm now hoping to turn my attention back to Elysium and get a new release of that out the door because I have some cool stuff for it.

But I've setup a GetSatisfaction product page for Spike so if you've any ideas for how to improve it please let me know or send me a patch!

Spike

I quite often find myself digging through Rails log files while developing and having to page through all that crap looking for one particular request I want, and then to get the right information out of it, is a pain. Several times I've thought something like "I wish I had an app that would let me sift through the request log more easily."

I decided to bite the bullet and see if I could write one. The result is spike 1.0. It's slow (there is some kind of GC compatibility issue with a frameworks I have used that's causing a particular problem), doesn't do everything you might hope, and only tested on the two log files I have to hand. But it solves my immediate problem:

I can load up a log file and see the requests in a table with all the pertinent details visible in one place. Then I can filter by controller, action, client, or session and dig out some of the details of any requests that look interesting.

I guess I am finding it a lot easier to write Cocoa apps now. This kind of task, although still relatively simple, would have been out of my league a year ago and would certainly have taken more than a days effort to get to a working application.

Nearly all the things

"Nearly all the things I do that are of any merit at all start off just being good fun, and I think I'm sort of building up to doing something else quite soon." - Brian Eno

Great quote I found on Synthtopia.

Expiring Rails cache from a background task

One of the great things about switching to Delayed Job for background processing in reeplay.it is that it's made it trivial to take tasks out of the request pipeline and move them into the background.

One such came up today. Because Stefano makes friends with just about every user in the system a particular update reached the point where it was taking so long to run that we were getting gateway timeouts... bad news.

But we knew the answer almost immediately. Move that part of the request into a DJ task. All too easy...

... except that part of the task used a cache sweeper to. Oops!

Rails caching is neat but it seems to be, and for reasons that escape me, inextricably linked to the ActionController context. I've hit this problem before and the usual compromises are:

• Create a special controller action just to expire the cache
• Use your knowledge of where the cache is and how it's stored to delete it by hand

For my money both of the above suck. What I really want is to be able to access the cache from anywhere in the Rails stack. Today I snapped and figured out how to do that.

Here's a cut down version of what my DJ job class looked like:

class VideoJob < Job

    def job
          video.do_something_awesome
          sweep_video_cache( video )
    end

    def do_something_awesome
      # apply secret sauce
    end

    def sweep_video_cache( video )
      # err, what do we do here?
    end

end

now I add just enough ActionPack "goodness" to make the Fragments module feel at home:

class UpdateVideoStatus < Job
  include ActionController::Caching::Fragments

  def job
    video.do_something_awesome
    sweep_video_cache( video )
  end

  def do_something_awesome
      # apply secret sauce
    end

  def sweep_video_cache( video )
    content = video.content
    expire_fragment( "user/#{content.user.id}/content/#{content.id}" )
    expire_fragment( "user/#{content.user.id}/content_tiled/#{content.id}" )
    expire_fragment( "user/#{content.user.id}/dashboard/recent_video" )
    expire_fragment( "user/#{content.user.id}/dashboard/river_of_video" )
  end

  private

  # The following methods are defined to fake out the ActionController
  # requirements of the Rails cache

  def cache_store
    ActionController::Base.cache_store
  end

  def self.benchmark( *params )
    yield
  end

  def cache_configured?
    true
  end

end

By adding three methods we're able to call expire_fragment just like we would from a cache sweeper. This is working for me using the FileStore but I don't see any reason it shouldn't work using other kinds of store.

My original hope had been that I could use our existing cache sweeper but it looked more complicated to get that working outside of the controller. This was the compromise. I'll refactor to have them share code later.

I don't think I'm doing anything too heinous here but if there's a better way I'd love to hear it.

The "I"

"Compartmentalization, splitting, and projection are ways in which the ego continues to pretend that it is completely in control at all times when, in reality, human experience is one of shifting beingness, instinctual or territorial reactiveness, and emotional motives, for which the "I" is not always complicit."

[Source Wikipedia]

Steampunk keyboards

Some wonderful steam era keyboards for programming your computation engine.