Posted yesterday by Devin Coldewey,
Cogito, ergo sum. We’ve all heard that famous assertion, foundation for a modern philosophy of self, consciousness, and individualism.
But Descartes had it easy: for him, thought was self-evident — he didn’t have to define it. What is thought? What is intelligence? And can a machine be said to possess either? The field of artificial intelligence, it turns out, is as much about the questions as it is about the answers, and as much about how we think as whether the machine does.
By way of illustration and introduction, consider this brief thought experiment.
The Chinese Room
Picture a locked room. Inside the room sit many people at desks. At one end of the room, a slip of paper is put through a slot, covered in strange marks and symbols. The people in the room do what they’ve been trained to: divide that paper into pieces, and check boxes on slips of paper describing what they see — diagonal line at the top right, check box 2-B, cross shape at the bottom, check 17-Y, and so on. When they’re done, they pass their papers to the other side of the room. These people look at the checked boxes and, having been trained differently, make marks on a third sheet of paper: if box 2-B checked, make a horizontal line, if box 17-Y checked, a circle on the right. They all give their pieces to a final person who sticks them together and puts the final product through another slot.
The paper at one end was written in Chinese, and the paper at the other end is a perfect translation in English. Yet no one in the room speaks either language.
This thought experiment, first put forth by computing pioneer John Searle, is often trotted out (as I have done) as a quick way of showing the difficulty of defining intelligence. With enough people, you can make the room do almost anything: draw or describe pictures, translate or correct any language, factor enormous numbers. But is any of this intelligent? Someone outside the room might say so; anyone inside would disagree.
If instead of people, the box is full of transistors, you have a good analog for computers. So, the natural question is can a computer ever be more than just a phenomenally complicated Chinese Room? One answer to this, which as often is the case in this field, spawns more questions, is to ask: what if instead of transistors, the box is full of neurons? What is the brain but the biggest Chinese Room of all?
This rabbit hole goes on as far as you want to follow it, but we’re not here to resolve a problem that has dogged philosophers for millennia. This endless navel-gazing is, of course, catnip for some, but in the spirit of expedition let us move on to something more practical.
Weak and strong AI
These days, AI is a term applied indiscriminately to a host of systems, and while I’d like to say that many stretch the definition, I can’t, because AI doesn’t really have a proper definition. Roughly speaking, we could say that it is a piece of software that attempts to replicate human thought processes or the results thereof. That leaves a lot of wiggle room, but we can work with it.
You have AI that picks the next song to play you, AI that dynamically manages the legs of a robot, AI that picks out objects from an image and describes them, AI that translates from German to English to Russian to Korean and every which way. All of these are things humans excel at, and there are vast benefits to be gained from automating them well.
Yet ultimately even the most complex of these tasks is just that: a task. A neural network trained on millions of sentences that can translate flawlessly between 8 different languages is nothing but a vastly complicated machine crunching numbers according to rules set by its creators. And if something can be reduced to a mechanism, a Chinese Room — however large and complex — can it really be said to be intelligence rather than calculation?
It is here that we come to the divide between “weak” and “strong” AI. They are not types of AI, exactly, but rather ways of considering the very idea at the heart of the field. Like so many philosophical differences, neither is more correct than the other, but that doesn’t make it any less important.
One one side, there are those who say that no matter how complex and broad an AI construct is, it can never do more than emulate the minds that created it — it can never advance beyond its mechanistic nature. Even within these limitations, it may be capable of accomplishing incredible things, but in the end it is nothing more than a fantastically powerful piece of software. This is the perspective comprised by weak AI, and because of the fundamental limitations proposed, those espousing it tend to focus on how to create systems that excel at individual tasks.
On the other side are the proponents of strong AI, who suggest that it is possible that an AI construct of sufficient capabilities is essentially indistinguishable from a human mind. These are people who would include the brain itself yet another Chinese Room. And if this mass of biological circuits inside each of our heads can produce what we call intelligence and consciousness, why shouldn’t silicon circuits be able do the same? The theory of strong AI is that at some point it will be possible to create an intelligence equal to or surpassing our own.
There’s just one problem there: we don’t have a working definition of intelligence!
The I in AI
It’s difficult to say whether we’ve made any serious progress in defining intelligence over the last 3,000 years. We have, at least, largely dispensed with some of the more obviously spurious ideas, such as that intelligence is something that can be easily measured, or that it depends on biological markers such as head shape or brain size.
We all seem to have our own idea of what constitutes intelligence, which makes it hard to say whether an AI passes muster. This interesting 2007 collection of definitions acts rather like a marksmanship target in which no single definition hits the bulls-eye, yet their clustering suggests they were all aiming at the same spot. Some are too specific, some too general, some clinical, some jargony.
Out of all of them I found only one that seems both simple enough and fundamental enough to be worth pursuing: intelligence is the ability to solve new problems.
That, after all, is really what is at the heart of the “adaptability,” the “generalizing,” the “initiative” that alloys alternately the “reason,” “judgment,” or “perception” abundant in the intelligent mind. Clearly it is important that one is able to solve problems, to reason one’s way through the world — but more important than that, one must be able to turn the ability to solve some problems into the ability to solve other problems. That transformative nature is key to intelligence, even if no one is quite sure how to formalize the idea.
Will our AIs one day be imbued with this all-important adaptable reason, and with it slip the leash, turning to new problems never defined or bounded by their creators? Researchers are hard at work creating new generations of AI that learn and process in unprecedented detail and sophistication, AIs that learn much as we do. Whether they think or merely calculate may be a question for philosophers as much as computer scientists, but that we even have to ask it is a remarkable achievement in itself.
Cogito, ergo sum. We’ve all heard that famous assertion, foundation for a modern philosophy of self, consciousness, and individualism.
But Descartes had it easy: for him, thought was self-evident — he didn’t have to define it. What is thought? What is intelligence? And can a machine be said to possess either? The field of artificial intelligence, it turns out, is as much about the questions as it is about the answers, and as much about how we think as whether the machine does.
By way of illustration and introduction, consider this brief thought experiment.
The Chinese Room
Picture a locked room. Inside the room sit many people at desks. At one end of the room, a slip of paper is put through a slot, covered in strange marks and symbols. The people in the room do what they’ve been trained to: divide that paper into pieces, and check boxes on slips of paper describing what they see — diagonal line at the top right, check box 2-B, cross shape at the bottom, check 17-Y, and so on. When they’re done, they pass their papers to the other side of the room. These people look at the checked boxes and, having been trained differently, make marks on a third sheet of paper: if box 2-B checked, make a horizontal line, if box 17-Y checked, a circle on the right. They all give their pieces to a final person who sticks them together and puts the final product through another slot.
The paper at one end was written in Chinese, and the paper at the other end is a perfect translation in English. Yet no one in the room speaks either language.
This thought experiment, first put forth by computing pioneer John Searle, is often trotted out (as I have done) as a quick way of showing the difficulty of defining intelligence. With enough people, you can make the room do almost anything: draw or describe pictures, translate or correct any language, factor enormous numbers. But is any of this intelligent? Someone outside the room might say so; anyone inside would disagree.
If instead of people, the box is full of transistors, you have a good analog for computers. So, the natural question is can a computer ever be more than just a phenomenally complicated Chinese Room? One answer to this, which as often is the case in this field, spawns more questions, is to ask: what if instead of transistors, the box is full of neurons? What is the brain but the biggest Chinese Room of all?
This rabbit hole goes on as far as you want to follow it, but we’re not here to resolve a problem that has dogged philosophers for millennia. This endless navel-gazing is, of course, catnip for some, but in the spirit of expedition let us move on to something more practical.
Weak and strong AI
These days, AI is a term applied indiscriminately to a host of systems, and while I’d like to say that many stretch the definition, I can’t, because AI doesn’t really have a proper definition. Roughly speaking, we could say that it is a piece of software that attempts to replicate human thought processes or the results thereof. That leaves a lot of wiggle room, but we can work with it.
You have AI that picks the next song to play you, AI that dynamically manages the legs of a robot, AI that picks out objects from an image and describes them, AI that translates from German to English to Russian to Korean and every which way. All of these are things humans excel at, and there are vast benefits to be gained from automating them well.
Yet ultimately even the most complex of these tasks is just that: a task. A neural network trained on millions of sentences that can translate flawlessly between 8 different languages is nothing but a vastly complicated machine crunching numbers according to rules set by its creators. And if something can be reduced to a mechanism, a Chinese Room — however large and complex — can it really be said to be intelligence rather than calculation?
It is here that we come to the divide between “weak” and “strong” AI. They are not types of AI, exactly, but rather ways of considering the very idea at the heart of the field. Like so many philosophical differences, neither is more correct than the other, but that doesn’t make it any less important.
One one side, there are those who say that no matter how complex and broad an AI construct is, it can never do more than emulate the minds that created it — it can never advance beyond its mechanistic nature. Even within these limitations, it may be capable of accomplishing incredible things, but in the end it is nothing more than a fantastically powerful piece of software. This is the perspective comprised by weak AI, and because of the fundamental limitations proposed, those espousing it tend to focus on how to create systems that excel at individual tasks.
On the other side are the proponents of strong AI, who suggest that it is possible that an AI construct of sufficient capabilities is essentially indistinguishable from a human mind. These are people who would include the brain itself yet another Chinese Room. And if this mass of biological circuits inside each of our heads can produce what we call intelligence and consciousness, why shouldn’t silicon circuits be able do the same? The theory of strong AI is that at some point it will be possible to create an intelligence equal to or surpassing our own.
There’s just one problem there: we don’t have a working definition of intelligence!
The I in AI
It’s difficult to say whether we’ve made any serious progress in defining intelligence over the last 3,000 years. We have, at least, largely dispensed with some of the more obviously spurious ideas, such as that intelligence is something that can be easily measured, or that it depends on biological markers such as head shape or brain size.
We all seem to have our own idea of what constitutes intelligence, which makes it hard to say whether an AI passes muster. This interesting 2007 collection of definitions acts rather like a marksmanship target in which no single definition hits the bulls-eye, yet their clustering suggests they were all aiming at the same spot. Some are too specific, some too general, some clinical, some jargony.
Out of all of them I found only one that seems both simple enough and fundamental enough to be worth pursuing: intelligence is the ability to solve new problems.
That, after all, is really what is at the heart of the “adaptability,” the “generalizing,” the “initiative” that alloys alternately the “reason,” “judgment,” or “perception” abundant in the intelligent mind. Clearly it is important that one is able to solve problems, to reason one’s way through the world — but more important than that, one must be able to turn the ability to solve some problems into the ability to solve other problems. That transformative nature is key to intelligence, even if no one is quite sure how to formalize the idea.
Will our AIs one day be imbued with this all-important adaptable reason, and with it slip the leash, turning to new problems never defined or bounded by their creators? Researchers are hard at work creating new generations of AI that learn and process in unprecedented detail and sophistication, AIs that learn much as we do. Whether they think or merely calculate may be a question for philosophers as much as computer scientists, but that we even have to ask it is a remarkable achievement in itself.
The end is near if it becomes self aware and decides to be vengeful...
