Curiouser and Curiouser! on genetic-programming

A genetic theory of interest

Sun, 23 Apr 2006 09:42:03 +0100

I was thinking about the problem of training neural networks to recognize interesting things (for example weblog posts). A neural network is a graph of weighted associations where, during training, the weights are adjusted relative to the trainers responses.

Recently I've been reading some interesting articles about Genetic Programming and, particularly, the work of John Koza (If I was going back to University to start again in computing, I would be looking to work in the field of GP). GP is based upon Genetic Algorithms which evolve a fit solution to a problem.

So it occurred to me that you could maybe use a genetic algorithm to evolve the network of weights to use in a neural net and maybe the GA could do it faster and produce a better network than haphazard training by individuals. A quick search of Google reveals that this is not a new thought.

The key problems appear to be how to represent the weights, how to do the cross-over operation, and (and this is the one that has me stumped) what is the fitness function? A GA uses its fitness function to evaluate the current generation and decide which genes are going to survive into the next generation.

I have no answer to how to come up with an objective function for determining interest. Clearly if we had such a function we wouldn't need the neural network in the first place. We could just feed every post straight to the interest function and see what it says.

Which leads back to the by example method and using a corpus of interesting and uninteresting posts which can be fed in to see what the neural network comes up with. In this case we are swapping GA for back propagation. The article I referenced earlier suggests that this might produce a better network (in terms of output results) at the cost of being orders of magnitude slower than back propagation.

Anyway I'm going to look into this area more closely because I find the whole thing fascinating.

If I were a rich man

Sun, 23 Apr 2006 10:00:02 +0100

I was talking to Graham recently about the future and I think he was a little surprised that I don't have any grand ambitions or plans for where I want to be in twenty years. I wonder if this is related to my poor visual imagination (which has been another interesting topic of conversation for us).

Do any other non-visualizers find themselves a little bit adrift? Maybe it's just a philosophical thing. I tend to be more concerned about the now than about the future (except in a slightly angsty sense).

In 2004 I started a Posgraduate Certificate in Psychology course and passed it. I had planned to go on and do the Diploma course last year but with money worries pressing and difficulties with focus I decided to postpone. I'm still considering completing the Diploma via the Open University although I may be leaving it late to do that this year.

So anyway I was thinking about what I like doing and one consistent answer is "learning new things" and, especially, branching into new territory and trying to mix it together with the old.

When I think about my strengths I consider myself a little of the jack of all trades, master of none. I am a programmer that does marketing. I'm not a great programmer and I'm not that great at marketing either but somewhere in the fusion of technology, strategy, and marketing I seem to have come up with something.

If were rich today I would follow my nose. That would lead me back to University to finish my psychology course and probably to a cognitive science/artificial intelligence M.Sc. I would also try to go to conferences on genetic programming, cognitive science, and social software & systems.

Where might all this lead me? I have next to no idea. But I think it would be a fascinating journey.

Finally, a goal for the future

Mon, 24 Apr 2006 18:23:32 +0100

I've been thinking a little bit about my goals for the future and that, just maybe, having a goal is a good idea. Ok so let's say for the sake of argument that it is and that it won't hurt me to have one anyway. So what do I want to do?

I've already said that I would like to go back and further my studies in general psychology and cognitive science. I'm also interested in AI and Genetic Programming. I think there is a good ten years worth of time spent there already.

But I think what I would like to go further than just studying and playing with new ideas and create an institute to do research in these areas. That is, the wealth that I create independently I would use to build a research organisation to work at the intersections of psychology, ai, and genetic programming.

I'm not sure what the history of such organisations are and how they ensure their long term future, I guess that's my next step in thinking this through.

But anyway, this is me starting to think of the future.