Archives for March 2007

How to make friends and influence species

Dave Winer suggests sending a copy of Windows Vista to Alpha Centuri:

If I were Bill Gates, I might send a copy of Windows Vista to Alpha Centauri (of course with a computer to run it on)

Hrmm... they might not appreciate such a gesture. With all the problems we have at home do we really want to provoke war with Alpha Centuri?

29/03/2007 21:35 by Matt Mower | Permalink | comments:
More about:

Apparently it's pure beauty

Ever since I learned about the existence of savant's I wondered what being a math savant would be like. The idea that you could be given any kind of complex problem and immediately come up with the answer but have no idea why or how... my feeling was that it might be the kind of thing that would drive you mad:

One of the researchers tried to throw him off by presenting him with a section of the decimals of Pi which was wrong, with some digits in the wrong place. And, whereas the real series of decimals is pure beauty to him, the false series gave him a strong reaction of being wrong and disharmonious.

Apparently not.

29/03/2007 20:41 by Matt Mower | Permalink | comments:
More about:

Why we'll all stay poor

According to an article from medialens sent to me by a friend, in 2006 the United Kingdom spent 39,776,000,000 pounds on defence. That is the worlds second largest defence budget (trailing the US) to defend (according to Wikipedia) the 22nd largest population. Does this make sense?

According to the same medialens article replacing the Trident nuclear missile system will cost, all told, some 76 billion pounds. This to replace a system that was never used, never likely to be used, and that we could not use anyway without the complicity of the United States. Does this make sense?

When I talk about a taxless society I am, usually, deliberately taking a somewhat extreme stand. But this is because I find that so many people don't even think about how their taxes are spent. They talk about the government spending more on this or that, but dont' seem to connect that to it raising and spending all of our money.

There are a good many people employed in the defence business and I am sure that they are made uncomfortable by the idea of defence cuts. But in the same way that bread is a bad way of spending points on Weight Watchers (you get a lot of calories per slice), defence is a very bad way of spending taxes.

If we were to stop being paranoid for a moment and consider the real and likely threats to Britain (the world is not quite as it was in 1907) I would like to think that a few more people might agree that we don't require the worlds second largest defence budget.

The present government love to set (and then miss) targets. For example the eradication of child poverty:

In March 2006, the government had been forced to announce that it had failed - by a significant margin - to meet the first target in that project. It had boasted it would reduce the number of children living in poverty by 25 per cent - approximately one million - and missed by 300,000.

The cost of eradicating child poverty: 4 billion pounds.

So those figures again:

  • Defence 39 billion. No problem.
  • Poverty 4 billion. Can't be done.

In my view the less the government does, the less taxes it needs to raise. In many peoples view that makes me a heartless capitalist. If not wanting a huge leviathan of a government spending 39 billion pounds of our money on defence, and wanting us to have that money to spend on our priorities makes me a heartless capitalist then so I am.

If we could have back some (and clearly I am not suggesting that we should spend nothing on defence) of what we spend on defence then I would be happy to see some of the remainder spent on hospitals, social services, relief of the poor.

What I don't want to see is a government raising taxes to keep us all poor.

29/03/2007 20:36 by Matt Mower | Permalink | comments:

That which does not kill us

So last night I went to my first yoga session. I enjoyed it but it was damn tiring and showed up how poor my balance is. Today my car broke down and I ended up walking 3 miles to karate this evening. The karate session began with a warm-up that damn near put me in the ground, and then we had an hours practice! Then I got to walk home again.

I look forward to the time where I can do all this without breaking a sweat.

27/03/2007 22:03 by Matt Mower | Permalink | comments:
More about:

The Perfect Man

The perfect man is an honest man who's lost all his illusions. If he's got a mind, he's excellent company; since he attaches no importance to anything, he can't be a pedant; he's forbearing because he remembers that, like those still suffering from illusions, he once had them too. Being carefee, he's decisive in his dealings; he never quibbles or insists. If other people do it to him, he ignores them or brushes them aside. He's bound to be more cheerful than anyone else because he can always find the appropriate epigram to apply to his fellow man. He's on the right track and can afford to laugh at those who aren't. He's in a well-lit area watching the foolish antics of people stumbling around in the dark. He can demolish with a laugh the false standards and judgements which others apply to people and things.

Chamfort - From his reflections on life, love, and society.

22/03/2007 08:38 by Matt Mower | Permalink | comments:
More about:

Recommendations for Mice and Keyboards

My friend Martin Hall has transitioned to the Mac and is looking for recommended applications as well as mice & keyboards:

I just got an Apple MacBook (13" Black) after a lengthy hiatus from using Macs -- what are the programs and/or gadgets that make the most sense to use -- and have. I think that I need a good mouse and keyboard. Any ideas? What are some of the best utilities?

He already knows about TextMate and, I hope, Quicksilver which are two indespensible apps. I continue to recommend the Logitech MX-R mouse but I can't, in all honesty, recommend the Apple external keyboard I am using.

I'd be interested in recommendations of external keyboards (and why you think they are good).

22/03/2007 08:21 by Matt Mower | Permalink | comments:
More about:

Collusion comes in many guises

I was interested to read Gear Diary talking about Clear; A new service in the U.S. that travellers can pay for ("Clear's first year price is $99.95 (includes a $28.00 TSA vetting fee). You can lock-in this rate by purchasing a two year membership for $199.90 or three years for $299.85.") to avoid the queing system put in place by the TSA.

If you think the TSA are bad now, watch out! Now that there is money to be made in making travellers lives miserable (rather than just doing it for the ignorant, cretinous, pleasure of it) you can expect the TSA to behave worse and worse towards the ordinary, non-paying, travelling public.

I would be very surprised if in 5-10 years it isn't proven that there is collusion at the top levels of the TSA and Clear (and the other TSA franchisees that will spring to life if this takes root).

22/03/2007 08:13 by Matt Mower | Permalink | comments:
More about:

It was all fine until step #2

I am one of those wussy cat owners who cannot give their cats tablets. Well, in fact, only one of my cats has ever needed to take tablets and we decided, for the good of my health, that we'd find another route because it was more likely that I was going to take the tablet than she was. If only I could remember to ask the vet to share her secret powers with me!

Because of this I always read "how to get your cat to take tablets" posts with interest, like this one "How to give your cat a pill in 7 steps". It starts well enough:

1) Catch the cat from behind the face, making sure that your thumb and index are placed where the maxilars joint. It is good if you can do this with the cat sitting on a high place, so you do not need to bend over.

but all goes horribly wrong on step 2:

2) Slowly, shift the cat's head backwards, gently raising her mouth. She does not like it, but she does not object, either. Apply a slight pressure on her jaws, with your thumb and index (or middle finger). The mouth will still be closed.

Note the part I have emphasized... the writer of this post was clearly not talking about a torty. My torty is prepared to object quite a bit... with claws. Oh and don't get me started on the time I tried to use a 'pill-giver'...

21/03/2007 17:43 by Matt Mower | Permalink | comments:
More about:

Interactive fixtures

Several times recently I have found myself slightly uncertain why a test wasn't working the way I expected. Our fixtures are beginning to get complicated as we add more, and more complex, associations. I kept finding myself wanting to go to

RAILS_ENV=test ./script/console
>> some_model(:fixture_name).association.something

and see the data, but of course it's not that simple. I've been able to do this with ruby-debug by setting a breakpoint in a test and dropping to an IRB session from there but it's hardly convenient. However knowing that ruby-debug could do it, I reasoned it should be simple enough to do. A few minutes studying the ruby-debug source code and I saw how to get what I wanted by lifting ruby-debugs IRB command and plunking into a test context.

Here is my ./script/testcon:

#!/usr/bin/env ruby
require 'irb'
require File.dirname(__FILE__) + '/../config/boot'
require File.join( RAILS_ROOT, 'test', 'test_helper' )

module IRB
  def self.start_session( target_binding )
    IRB.setup( nil )

    workspace = WorkSpace.new( target_binding )
    irb = Irb.new( workspace )

    @CONF[:IRB_RC].call( irb.context ) if @CONF[:IRB_RC]
    @CONF[:MAIN_CONTEXT] = irb.context

    # trap("SIGINT") do
    #   irb.signal_handle
    # end

    eval "puts self.class.to_s", target_binding

    catch( :IRB_EXIT ) do
      irb.eval_input
    end
  end
end

class InteractiveFixtures < Test::Unit::TestCase
  all_fixtures

  def test_interactive
    IRB.start_session( binding )
  end
end

When run, testcon drops you into an IRB session within the context of a test case where you can use fixtures as you would in a test and even call assertions by hand. Not pretty but it gets the job done.

I later went back and added this same code to my test_helper putting the call to start_session(..) inside a method called irb_here() so that I could break actually in the test and evaluate expressions. Now I can investigate models & fixtures in the abstract or in specific cases.

My thanks to Kent once again for ruby-debug.

21/03/2007 11:07 by Matt Mower | Permalink | comments:
More about:

Kiai

Just back from my second lesson in Shotokan karate. Where last week I had no idea what I was doing at any point, this week I was only baffled about 85% of the time and, as we practiced the first kata, i started getting into it and enjoying myself. It feels weird to shout kiai but Sensei Richard says you get over that. Also last week I was completely exhausted by the end of the training session and this week I felt energized. I wish I'd started this a year ago :-)

20/03/2007 21:12 by Matt Mower | Permalink | comments:
More about:

Not just a pretty logo

I am very pleased to report that my good friend Terry Frazier has launched the beta of Nearline Publishers, Inc. which aim to help consultants, speakers, and knowledge professionals publish themselves using the new blended media enabled by the web.

I've already read one of the e-books produced by Nearlines first author, Mitch Byers, titled Salary Negotiations RX and I thought it was a knock-out. After reading that book I felt prepared and confident when it came to negotiating my remuneration with Cominded. If you're in the market for a new job I can whole-heartedly recommend reading it and I bet the interview book is also worthwhile.

Nearline has been a dream of Terry's for several years and I've watched him battle and overcome many challenges in that time. It takes a lot for one man to bootstrap a new business concept alone but he's done it. Well done man! I'm looking forward, with confidence, to Nearline being a great success!

16/03/2007 18:33 by Matt Mower | Permalink | comments:

A better mousetrap

Kent Sibilev has released ruby-debug version 0.8. I've been thankful for ruby-debug a number of times and often use it to spelunk rails app code. The major development in version 0.8 is that the debugger backend has been separated from the CLI so that it is possible to implement a new interface on top.

A long time ago, when I first started messing with Cocoa application programming, I tried to implement a ruby debugger. Although I managed to embed the ruby interpreter and even single-step through some code my knowledge of both Cocoa programming and the operation of the Ruby debugger wasn't well enough developed to go any further. I think Kent just made the idea of a functional, Aqua interfaced, Ruby debugger a much more managable proposition.

Thanks Kent!

16/03/2007 10:39 by Matt Mower | Permalink | comments:
More about:

When did side-effects become a good thing?

You can tell there aren't many functional programmers in government because even a cursory glance at the outcome of most government policy show that side-effects abound. Ming writes:

Policy makers are often very bad at understanding systems. They will often traffic in fragmented campaigns that will demonstrate that they take a certain issue seriously. Sort of, "We have to at least try to ...". Try to limit polution, lower crime rates, protect children, or whatever. And if they can present a study that says that their new law produced a 5% drop in whatever it was, it would be considered a success. Even though the whole system isn't working any better, and possibly might be working a lot worse in many other ways.

His article is ostensibly about a movement to put speed-limits on the autobahn (the theory being that this may reduce environmental impact due to fuel consumption) but his real point is about the way in which so much policy bears only tangentially on the desired effect and is often laden with problematic side-effects.

16/03/2007 09:00 by Matt Mower | Permalink | comments:
More about:

Just in from Amazon

Well it took two weeks but my Amazon order finally arrived. I'm really going to have to resist the urge to buy any more books for a while as I am beginning to drown in a sea of must-reads which now includes:

I was going to put Programming Haskell straight on the shelf since I've diverted into Erlang instead but I figure a lot of what I read there will still be applicable in Erlang. It's all functional programming, right?

Anyway I've started cataloging my book collection at LibraryThing.

15/03/2007 14:27 by Matt Mower | Permalink | comments:

The future will be smaller and more elegant than we imagined

In my last post about my first experiences with Erlang I outlined a module I came up with to implement futures. Futures provide a useful means of calling a function without waiting for it to finish and being able to obtain the resulting value at some later point. While my implementation worked some of the comments I have received showed it could be implemented much more simply even than the couple of dozen lines I had.

Julian Fondren (#ayrnieu) offered the following enhancement:

new(F) ->
  Caller = self(),
  spawn( fun () -> Caller ! { self(), { ok, F() } } end ).

which neatly removes the need for the oneshot/0 that was the receiver and proxy for the future'd function.

This still left the issue of the return value being the pid of a generic Erlang process. In comments I discussed how this leaky abstraction bothered me a little. I wasn't hopping up and down about it but it seemed to lack something compared to an equivalent solution with classes.

In comments Bob Ippolito offered a neat, functional, solution:

fun () -> value(Pid) end.

i.e. to wrap the pid in an anonymous function that returns the result value. Hence, to the caller, the future becomes just another function that can be asked for the result of the future'd function. Quite stylish. It's this sort of functional insight I was looking for when I took up Erlang. I expect to use this trick often now.

The resulting futures module becomes so trivial it's hardly worth calling it a module, yet it neatly and elegantly (to my eye) solves a real problem:

-module(futures).
-export([new/1,value/1]).

new( F ) ->
  Caller = self(),
  Pid = spawn( fun () -> Caller ! { self(), { ok, F() } } end ),
  fun() -> value( Pid ) end.

value( Pid ) ->
  receive
    { Pid, { ok, Result } } -> Result
  end.

In #erlang we discussed a little about how this might be modified to handle cases where there was a lot of concurrency that might, for example, overwhelm a file-system with open files. This could be as simple a modification as:

spawn( fun() -> Caller ! { self(), { ok, ( throttle( F ) )() } end ).

where throttle/1 is some function that ensures that only N processes are active before executing the wrapped function. Another suggestion was to think more in terms of message passing and use an additional supervisor process that keeps N processes alive and passes futures to them for evaluation in turn.

Given my duplicate finder might well run into futher instances of {error,emfile} I will probably experiment with one or both approaches to throttling.

14/03/2007 22:16 by Matt Mower | Permalink | comments:

A first concurrency in Erlang

I've recently started learning the Erlang language using Joe Armstrong's new, in-beta, book. In the past I learned some Lisp but I ended up admiring Lisp more in the abstract than I liked it in practice and so I didn't stick at it. Lately I've had a yen to return to functional programming because so much of what I find elegant about Ruby seems to have it's roots in the FP world and I feel I've so much more to learn yet.

Getting started with a new programming language for me starts with picking a problem to solve that is useful, not too difficult, and exercises a reasonable spectrum of the languages features. In this case, because Erlang is so strong in it's support for concurrency, it seemed natural to look for a problem whose solution would be enhanced by a parallel approach. Since I have a very large collection of PDF's which includes quite a lot of duplicates, and since I don't like any of the de-dupe solutions I've tried, I thought a good first problem would be scanning for and finding duplicate PDF's across all my disks.

I spent a bit of time looking at the library that comes with Erlang and was pleased to discover the fold_files function in the filelib module. At a stroke this takes care of most of the actual work of iterating the file-system and filtering the required files. What remained was learning how to build the duplicate database in functional style and bearing in mind Erlang immutable data structures and single-assignment variables.

My approach to finding duplicates was to calculate an MD5 hash of the file size along with the first & last 16 bytes of the file which looks like this:

get_file_hash( FilePath, Size ) ->
  {ok,[Hunk1,Hunk2]} = myfile:open(
    FilePath, [read], fun( File ) -> file:pread( File, [{0,16},{Size-16,16}] ) end
  ),
  hex:hex(
    binary_to_list(
      erlang:md5(
        lists:append( integer_to_list( Size ), lists:append( Hunk1, Hunk2 ) )
      )
    )
  ).

As an aside I really like the way Erlang functions are declared, e.g.

length( [] ) -> 0;
length( [H|T] ) -> 1 + length( T ).

which uses pattern matching on the arguments to find the right clause of the function to evaluate. It's an elegant approach although I did get caught out by not realising that clauses of a function with the same name but different arity do no belong together, e.g.

find( Path ) -> expr;
find( Path, [] ) -> expr.

does not work because the two are not clauses of the same function (being instead clauses of find/1 and find/2 respectively). If you find yourself with perplexing head mismatch errors try looking for this pattern and separate the clauses properly (using a .).

My definition of the function myfile:open(), above, reflects my failure to close any files during the first attempts at iterating a large filesystem leading to emfile errors. I realised I had forgotten that it was even possible to call Ruby's File#open without passing a block because it's so natural and effective to do so, it's something I think Matz got dead right, hence:

myfile:open( Filename, Modes, Block ) when is_function( Block ) ->
  {ok,File} = file:open( Filename, Modes ),
  RetVal = Block( File ),
  file:close( File ),
  RetVal.

that demonstrates another neat aspect of Erlang function clauses, guard expressions.

Another aside is that, in Erlang, the = operator is not "equals" but "matches". This relates to the fact that Erlang, in a manner somewhat akin to Prolog, uses unification in expressions so that matching unbound variables will bind them and thereafter attempt to unify them, a simple example:

X = 2.

where X is an unbound variable will bind X to the value 2. If later you were to evaluate,

X = 3.

you would get an error because 3 does not match the value, 2, that is already bound to X. This, I think, can take some getting used to but is pretty neat. For example, if File is a filespec record then:

#filespec{hash=Key,size=Size} = File

will bind the variables Key and Size (in parallel) to the hash and size attributes of the record bound to File.

I was surprised at how quickly I arrived at a solution to my basic problem. It was almost as fast I could imagine doing it in Ruby. However what the basic solution lacked was that it took no advantage of Erlangs famous concurrency primitives, spawn & co. Initially I was quite unsure about how to parallelize my solution since so much of the work was going on in fold_files. After some pondering I concluded that what I was looking for was futures. I looked through the library that comes with Erlang but couldn't recognize was I was looking for, so I cooked up the following:

-module(futures).
-export([new/1,value/1]).

new( Fun ) ->
  Pid = spawn( fun oneshot/0 ),
  Pid ! { self(), Fun },
  Pid.

value( Pid ) ->
  receive
    { Pid, { ok, Result } } -> Result
  end.

oneshot() ->
  receive
    { Sender, Fun } ->
      Sender ! { self(), { ok, Fun() } };
    Any ->
      io:format( "Oneshot(~p) received unknown message ~p~n", [self(),Any] )
  end.

Which, it turns out, isn't a shocking travesty ;-) I did get some useful suggestions from the fine folks in #erlang including looking at Joe Armstrong's definition of pmap. Now I was able to make essentially a two line change to have a fully concurrent solution:

scan( Path ) ->
  lists:map(
    fun( Future ) -> futures:value( Future ) end,
    filelib:fold_files(
      Path,
      ".*.pdf",
      true,
      fun( F, Acc ) -> [
        futures:new( fun() ->
          file_info:file_info( F )
        end ) | Acc ] end,
      []
    )
  ).

Creating the future spawns an Erlang process that runs the embedded function (that does all the nasty mucking about with files and MD5 hashing). Later the future is referenced for it's value which will block until a value is available (but by which time all the processes have been spawned off anyway). Crucial to this being efficient is Erlangs ability to cope with hundreds of thousands of processes (I was able to run tests creating millions of processes on my MacBook Pro CD2 with 2GB memory). For more about that take a look here.

The way I have defined the future module highlights what is for me, so far, the only thing I don't like about Erlang (and probably all functional languages). I miss the ability to wrap the future into a nicely packaged class. Calling:

Pid = futures:new( fun .... end ).
futures:value( Pid ).

seems less elegant than (hypothetically):

future = Future.new do ... end
future.value()

Something about having to treat the generic process id as a future... maybe it's a small point but modules do expose the details more and feel somewhat clunky after using Ruby classes for such a long time. This last is apparent in things like:

lists:map( fun ... end, List )

vs.

list.map do ... end

as well. There have to be tradeoff's I suppose.

I've still got a few bugs to work through (the odd future call seemingly to returnin a pid instead of the expected result which is a bit weird) but otherwise I have quickly arived at 80% of a fast neat solution to my problem. The last step of which will be to have it create a customisable script for handling the duplicate files.

I'm beginning to like Erlang a lot.

13/03/2007 13:09 by Matt Mower | Permalink | comments:

Just left of Attila the Hun

Went out last night for dinner with two of my dear friends, Mike and Megan, from my days at the University of North London. It was a great evening catching up and, as often happens getting together, a clash of political ideals. By the end of the evening we had established that for all my liberatarian pretensions I am a closet and Tory and that Mike is just an armchair socialist with an expensive (if problematic) fridge.

However between the trading of vile slurs on our respective characters we did manage to establish that we both think that - however much tax is being raised right now - it's being grossly misspent. I'm not sure how much the 30 billion odd pounds a year we know (and how much don't we know?) is spent on defence amounts to per tax-payer but I'm willing to bet it's a lot.

13/03/2007 10:38 by Matt Mower | Permalink | comments:

Ahead one third at Amazon

I put an Amazon order in on March 2nd for 5 items, all of which were in stock. At the time I didn't pay too much attention to the delivery date and it really only occurred to me today that it's a long way off. In fact Amazon tell me the items won't be dispatched until March 14th at the earliest.

How can it take 8 working days to assemble and dispatch an order of 5 in-stock items?

Update: Amazon responded telling me there were waiting for one item from a supplier. When I checked it does say 5-8 days for delivery so that sounds about right. I guess I just got unlucky; I'm reasonably sure it was "in stock" when I started the ordering process.

05/03/2007 22:30 by Matt Mower | Permalink | comments:
More about:

More functional goodness

When I found out yesterday that the Prag's are publishing a book on Erlang I couldn't have ordered it fast enough. I've been through the Erlang tutorial a couple of times and found much both to like and to confuse me. Just reading the first couple of chapters of the beta book has helped me clarify some things and it holds out the promise of so much more.

I still love Ruby and will probably continue to write the bulk of my code in it (especially since I work in Ruby on Rails) but I have a few things in mind for which Ruby is not especially well suited and a compiled, multi-processing savvy, language might be a better fit. I'd already earmarked Erlang as a possible choice for implementing the backend of those projects (with a Rails front-end).

The other language I am considering is Haskell. I've been reading quite a few Haskell related blogs lately and am getting more and more intrigued with it. My concern is that I am spreading myself very thin right now and that Haskell (given it's legendary ability to baffle) may be a bridge too far. Still, since I am now actively experimenting with Scheme, Erlang and maybe Haskell I am hoping that I am on the cusp of some kind of functional epiphany.

Since I also love Ruby and Objective-C I'm not quite sure what that means. Probably that i'm in danger of blowing a lot of neurons.

05/03/2007 12:43 by Matt Mower | Permalink | comments:
More about:

Social planning is like this

I just heard an item on Radio 4 about the governments subsidy of people who want to add wind turbines or solar panels to their home. The program gets about 28 million pounds a year to distribute (less, according to the chap being interviewed, than the cost of a small section of main A-road) and is distributed, on a lottery basis, monthly.

This is crazy even for government. We all get taxed and then the government drip feeds our own money back to us using some daft scheme that probably costs more to administer than it gives away.

If we accept the notion that the government should be setting (or encouraging) social priorities (like responsible energy usage) then I think the correct way to achieve this without creating more problems is to offer a tax allowance. Instead of petitioning the government for the 50% subsidy and installing the wind turbine if and when I get lucky, I do it when it makes sense and deduct the subsidy direct from my next tax bill.

01/03/2007 08:50 by Matt Mower | Permalink | comments:

Two for the price of one

I've made no secret of the fact that I have won the lottery at least twice a week for the last couple of months. I know, some people have all the luck.

In a new development today I find that I have not only won the UK NATIONAL LOTTERY again but also won "One Million Great Britain Pounds" from Microsoft and AOL... and all announced in the same email:

Dear winner,

The prestigious Microsoft and AOL has set out and successfully organized a promo marking the end of year anniversary and over 100,000.000.00 (One Hundred Million >Great Britain Pounds) for our end of year

immediately followed by

We announce to you the draw of the British Lottery International programs held on the 15th of Febuary 2007 in London. Your e-mail address attached to ticket number: 564 75600545-188 with serial number 5388/02 drew the lucky numbers: 31-6-26-13-35-7, which subsequently won you the lottery in the 2nd category.

If we could just get all the spammers in the world to combine and send out a single email we might be getting somewhere.

And, no, the drinks are not on me... not until I've managed to get all those Great Britain Pounds into a currency you can spend.

01/03/2007 00:01 by Matt Mower | Permalink | comments:
More about: