Ville V. Kokko

Could an artificial intelligence system be conscious? How would we know if it was? Some of my colleagues have written on this blog that it couldn’t. I’m somewhat in disagreement, but it’s not that I think we can know for certain that it’s possible, either. In this post, I examine what happens when we try to study this question seriously.

To briefly note: When I speak of consciousness, what I mean is the conscious thing having a subjective experience, that it’s like something to be that thing. This should not be confused with a number of other things, such as intelligence in the sense of being able to reason and act effectively. Speaking of consciousness in this sense has some unique problems.

It all goes back to the so-called problem of other minds. We can only directly access our own conscious states, so how can we know anyone else has a conscious mind? The answer here is induction: as we can see that other people act and look just like ourselves, and now even that the same kind of things happen in their brains, we can confidently guess that they feel the same things too.

The same applies to other animals, to varying degrees. We can see that they have versions of behaviours associated with consciousness in ourselves, and similar neural structures. It’s much easier to say about animals closer to us in structure, like mammals, of course. Something like an insect is similar in some ways and different in others, and it gets more complicated to study whether it has its own form of consciousness anyway.

Now, what about an AI system? We know that it’s like us in some ways: consciousness seems to be heavily involved in processing information, and an AI system processes information. The way it processes it is in some ways the same as how we do, especially on the surface, and in some ways different, especially on the lower level it’s all built on. Which part of this is important for consciousness?

For all we know, it could be of defining importance that a computer is not built out of biological matter, and thus can’t possibly have consciousness. Or it could be that this doesn’t matter at all, and all it takes is processing information the right way.

The reason we don’t have a better idea of what’s required is that the subjective nature of consciousness makes it hard to pin it down to anything else. We know some things are conscious from our own experience of being conscious, and we know that consciousness correlates with certain physical things in us. But what physical things are required for it?

The so-called hard problem of consciousness is to explain how and why anything is conscious. As I said, consciousness seems to involve processing information. It would be hard to imagine consciousness without information. However, information without consciousness seems easy enough to imagine. We don’t know what role consciousness plays in our internal processing of information; there seems to be no logical reason those same things couldn’t happen without conscious feeling.

Consciousness kind of just is there, observed directly, unlike other things that we need words and concepts to describe. This is exactly why some philosophers regard it as mysterious and others regard the question as bogus; both are reacting to the fact that it seems to be something going outside our normal ways of explaining things.

Well, regardless, we are able to observe that some things correlate with consciousness, particularly in ourselves and our brains. This allows us to make theories about when we can expect to get consciousness, even regardless of its mysterious, transcendental-seeming nature. There’s still the problem that it’s very hard to test these theories.

If we were creating a theory about something objective like magnetism that everyone can observe, we could run tests on the theory by observing what happens when we do certain experiments with magnetism. But what about when we make a theory about how, outside of just humans, consciousness emerges? Only the being that is conscious can observe the presence of its own consciousness.

We can do some tests on consciousness on humans, who can report their conscious experiences verbally. But if we go over to AIs that can report things verbally (LLMs), then the most obvious reason that they will report having consciousness, or not having it, is just that they are imitating humans. There’s no remotely direct evidence that we could use.

As an example of an attempted method of “measuring” AI consciousness, consider the Digital Consciousness Model. The model is built to assess the probability of different systems – it could be applied to things other than AI – of being conscious. It does this by collecting different possible indicators of consciousness and seeing how likely experts think a particular system possesses those qualities, leading to an overall probability of the particular system being conscious.

This model is built in quite a sophisticated fashion, and it might be among the better attempts we can make to study the question at this time. However, it’s still very much groping in the dark! The whole reason it has a large set of possible indicators of consciousness is that there are different theories about how consciousness works, and the creators of the model have little chance of telling which of these theories are correct, if any.

So, when the researchers are calculating the probability for a particular system being conscious, they are having to factor in the uncertainty of which indicators of consciousness have anything to do with it. No wonder they admit they could get many different kinds of probabilities based on different assumptions.

One last thing worth noting here is that I’ve been talking about the difficulties of scientists and philosophers who are being very objective in their work – how even so, it’s difficult to tell whether consciousness is present. The question is even more difficult when you consider all the factors that can affect people (including the scientists and philosophers) when they are not being as objective as possible.

For psychological reasons, LLMs that imitate human speech production can give us the impression that they are conscious, even if it’s only because they are so good at faking being human. This doesn’t have to have anything to do with objective evidence that they are. We may also be affected by all the often uncritical hype in the media that assumes that an AI is already far more of a mind than we know it to be.

Thus, we should all be wary of drawing conclusions about AI consciousness, even if we get the impression that it’s real. The best answer is that we don’t currently know at what point AI could become conscious, or even what that would really mean.