The broad topic of my honours research at present is that of feature selection. This may sound horribly obscure and esoteric (and in many ways it is) but it’s also quite an everyday concept if we consider some loose analogies.

Feature selection is related to machine learning, where the goal is essentially to design algorithms that will allow a computer to learn concepts based on input data. (Why? This can be applied to anything from spam filtering to medical diagnosis to astronomy to any other number of fields.) A close analogy to machine learning is the process of teaching humans, especially children: by exposing a child to multiple examples of the same type of “thing” (e.g. dogs) and also contrasting that with multiple examples of a different (but related) type of thing (e.g. cats), the child begins to develop concepts that relate to each type of thing, allowing him or her to then associate new instances with these concepts. Dogs and cats are no longer mysterious entities but rather very specific types of things where each individual animal can now be understood as belonging to one of two groups.

But what makes a dog a dog and not a cat? Both have four legs, two eyes, a mouth, a nose and so on — these features are irrelevant when considering the differences between dogs and cats, and so we can safely tune them out without adversely affecting the learning outcome. This is the essence of feature selection.

In the case of machine learning, feature selection is often a necessity. The more irrelevant and/or redundant features that are included in each instance, the more time is spent analysing features that don’t contribute anything meaningful to the process. In some domains (e.g. bioinformatics), the relevant features may be only a tiny fraction of the entire feature set, and processing all features becomes computationally infeasible.

The trick, then, is in deciding which features to keep and which to throw out. There are many, many approaches to this, some more complex than others. One technique, for example, uses a variation of Darwinian natural selection known as a genetic algorithm, where feature subsets breed and mutate, with those subsets that positively contribute to the learning process being more likely to have their “genes” passed on to the next generation; as the population evolves, we get closer and closer to an optimal solution — the Übermensch of feature subsets, so to speak.

In general, the more effective a feature selection method is, the more complex it’s going to be. Can we create more effective methods without an associated increase in complexity? This is the challenge, and in a general sense is the fundamental question of computer science as well.

If brainpower were all that was needed, computers could solve these problems themselves. No, more than anything else, creative thought is the key — that knack for looking at things at odd angles in order to see what others have missed. But don’t worry: computational creativity is in the pipeline as well.

The intention of this blog is threefold:

1. to process my thoughts on the relationship between science and religion;
2. to document my ideas on computer and cognitive science; and
3. to rave about a few of my other favourite things.

These three themes will, of course, intertwine on most occasions.

Anyone who has been reading this blog for any length of time knows that this is not actually the first post. But the old incarnation was too unfocused and self-indulgent — I want this blog to be an exploration of ideas, not a LiveJournal-style personal diary.

In any case, I intend to post at least twice a week. So, stay tuned…