A genetic theory of interest

Sun, 23 Apr 2006 09:42:03 +0100

I was thinking about the problem of training neural networks to recognize interesting things (for example weblog posts). A neural network is a graph of weighted associations where, during training, the weights are adjusted relative to the trainers responses.

Recently I've been reading some interesting articles about Genetic Programming and, particularly, the work of John Koza (If I was going back to University to start again in computing, I would be looking to work in the field of GP). GP is based upon Genetic Algorithms which evolve a fit solution to a problem.

So it occurred to me that you could maybe use a genetic algorithm to evolve the network of weights to use in a neural net and maybe the GA could do it faster and produce a better network than haphazard training by individuals. A quick search of Google reveals that this is not a new thought.

The key problems appear to be how to represent the weights, how to do the cross-over operation, and (and this is the one that has me stumped) what is the fitness function? A GA uses its fitness function to evaluate the current generation and decide which genes are going to survive into the next generation.

I have no answer to how to come up with an objective function for determining interest. Clearly if we had such a function we wouldn't need the neural network in the first place. We could just feed every post straight to the interest function and see what it says.

Which leads back to the by example method and using a corpus of interesting and uninteresting posts which can be fed in to see what the neural network comes up with. In this case we are swapping GA for back propagation. The article I referenced earlier suggests that this might produce a better network (in terms of output results) at the cost of being orders of magnitude slower than back propagation.

Anyway I'm going to look into this area more closely because I find the whole thing fascinating.

Curiouser and Curiouser! on genetic-algorithms

A genetic theory of interest