Abstractions #2 Intelligence as an Abstraction Optimisation Process
Are some abstractions 'better' than others?
I’m finally getting back to this series of posts I start last year on abstractions. I became too daunted by how much I wanted to write, so I’m going to try and break it down into more and shorter posts to encourage me to continue and finish it!
In the last post, I defined abstractions as maps from one domain to a second domain which help us answer questions in the first domain by answering the corresponding question in the second domain.
Now I want to argue that ‘intelligence’ basically is abstractions and that we can measure ‘intelligence’ in some hand-wavy way by how good the abstractions that it uses are.
One way we can think of intelligence is as the ability to answer questions correctly. These could be prediction questions about the world. You often see intelligence defined as the ability to achieve one’s goals i.e. to predict which actions will lead to one’s goals being achieved. But I think we also have a more general idea that somebody is intelligent if they answer mathematical questions say or other questions that are less directly related to prediction.
And abstractions are also essentially ways to answer questions. I have seen learning defined as the compression of information and we can think of an abstraction as being that compression. Now, if we had a look-up table with the answer for every single question we could face, then we would have perfect intelligence, but we’d need an infinite look-up table and an infinite amount of compute power to generate it. The nearest non-infinite thing we have is the AIXI agent which is uncomputable.
However if we find ‘good’ abstractions then we can reduce the compute power that we need to answer certain questions. For example the abstractions that form Newtonian Mechanics allow us to answer a huge number of questions with far less compute power than we otherwise could on a more case-by-case basis.
So I like to think of ‘good’ abstractions as way to get more intelligence for less compute power. With AI, having more compute power in effect allows us to get away with not having as good abstractions. We have seen the impact of just throwing more compute power at training AI. However we have also seen how transformers allow AI to generate abstractions that it couldn’t before the advent of transformers.
So one way of thinking of training AI is as an ‘abstraction optimisation process’ of some sort. We want our training process to generate the ‘best’ set of abstractions we can and the neural network architecture can impact how well it does that.
Ordering Abstractions: What is a ‘good’ abstraction?
However, an abstraction optimisation process needs to optimise for something. So it needs to have a way of knowing if one abstraction is ‘better’ than another. Mathematicians would refer to this as putting an ordering (or more precisely a partial ordering) on the set of abstractions.
I have been deliberating using inverted commas when talking about ‘good’ abstractions and if we think about it, we can see there are various different ways that we could regard one abstraction as being better than another:
The ‘amount’ of information compression that the abstraction gives us i.e. how easy is it to answer the question in the second domain?
The complexity of the map that comprises the abstraction (with the example I gave in the previous post of predicting whether it would snow tomorrow, we can see that there might be a choice to make as to whether just to go for a less precise abstraction with far lower complexity)
Our epistemic confidence in the abstraction
The statistical likelihood of the answer given by the abstraction being correct
How powerful the abstraction is (i.e. does it answer a wide range of questions, or equivalently a very ‘general’ question)
We could use any of these or some combination of these to give us an ordering of abstractions. In John Wentworth’s work, he defines an ordering based on the first of these criteria. If all maps have acceptable complexity, high epistemic confidence and high likelihood of the answer being correct, then this is an intuitive and reasonable way to compare abstractions. However it is not the only one.
There are trade-offs when it comes to combining these criteria. We could even do some sleight-of-hand and insist that we embed these trade-offs in our questions. We could only consider questions like ‘what is the best abstraction given that we want such-and-such a level of epistemic confidence, such-and-such a level of statistical certainty and these constraints on the complexity’?. But this feels like cheating to me. It feels simpler to me to think of the trade-off in terms of how we decide to order abstractions. I think they are worth keeping explicit because in these trade-offs, we hit decision theory. Ordering abstractions is about choosing an abstraction that maximises our ‘expected reward’, where we can the ‘reward’ is having our question answered correctly. That reward function may not be uniform for all questions and will influence the ordering we choose.
One of the most interesting ways to look at ordering abstractions is by how ‘powerful’ they are. Humans are drawn to powerful abstractions that answer lots of different questions. It seems to be a really important ordering criteria that humans use in the ‘abstraction optimisation process’ that forms their intelligence. We don’t seem to like having too many too many abstractions if we can help it. Obviously, our brains are still full of millions of different abstractions but we seem to have limits. The languages we speak have words that are vague and overlap, for instance. We like theories that don’t have too many special cases. We form beliefs and can fix onto them remarkably rigidly. Confirmation bias helps keep the number of abstractions we have to hold in check.
Thinking about the whole set of abstractions belonging to an intelligence is interesting because it suggests that instead of just thinking about an optimisation process for individual abstractions, we need to think about optimisation processes for sets of abstractions instead. So we might actually want a partial ordering of sets of abstractions not just individual abstractions. This could of course be based on a partial ordering for individual abstractions, but adds another dimension.
Obviously AI doesn’t have the memory and computational limits of the human brain. This preference for powerful abstractions isn’t a mandatory one and for humans, may have come about because of those limits. On the other hand, if powerful abstractions are ‘correct’ then an intelligence that has them, will need less compute power for the same intelligence than one that doesn’t. So in ordering abstractions, I don’t think we should ignore ‘powerfulness’ as an ordering criteria.
So now we have this idea of intelligence as an abstraction optimisation process, next time I will talk about how I see this fitting in with interpretability work i.e. the task of understanding what is going on inside neural networks.