I was struck by the prescience and clarity of Mitch Kapor's writing. He correctly predicted that a computer would be able to win on Jeopardy [0], 9 years before the fact. And he understood that the Turing Test was more of a thought experiment than an actual test [1], which is why I assume he took the bet in the first place.
Conversation is actually a rather poor measure of intelligence. I would say, show me a computer that can learn anything that it wasn't specifically programmed to learn. This doesn't mean unsupervised categorization or learning to play a video game. I'm talking about a scenario where programmers present their code or machine with no knowledge of what the task will be. Not something chosen from a known list of possibilities, but any task that a human could conceivably be taught to perform in less than an hour. Anything from writing a sonnet in iambic pentameter to assembling ikea furniture based on instructions. A true test of _general_ intelligence.
I would take that long bet out to 2129, and beyond. I don't see software with that level of intellectual flexibility being written in our lifetimes, or the lifetimes of our children or their children.
[0] "While it is possible to imagine a machine obtaining a perfect score on the SAT or winning Jeopardy..."
[1] "... a skeptic about machine intelligence could fairly ask how and why the Turing Test was transformed from its origins as a provocative thought experiment by Alan Turing to a challenge seriously sought."
I would think that your conversation with the computer would play a vital role while you were instructing it in the task it is to perform, even if it were a purely one-sided, instructional conversation the computer would still need to acknowledge truthfully whether it had correctly understood the instructions. More likely, the goal of creating such a system would be for it to be capable of engaging in active learning through inquiry but I suppose that would not be absolutely necessary given a specific task.
There are also a number of assumptions when approaching problems such as composing a sonnet or assembling furniture. With a human, you assume that they have a broad understanding of the language you are communicating in, that they have some knowledge of poetry even if just in passing, have experienced cadence, rhyme, or other constructs and have prior experience of the world from which to draw 'inspiration'. You also assume that they have seen and used a variety of furniture, tools, and even just generally explored a physical environment to a degree where they are capable of manipulating objects by moving, rotating, stacking, joining, etc. or even as basic as having the concept of object permanence. They must also know all of the relevant nouns and verbs referring to the pieces and tools they will be using.
If you continue to think along these lines you very quickly realize that the only reason you can teach a human being something in an hour is because they possess years or decades of previous instruction, programming, and prior learning attempts as well as a data set of visual, aural, tactile, olfactory, and spatial experiences encompassing a similar time frame.
You said anything we can teach a human within an hour: do you mean a fresh human that's only just been supplied with a brain? Because you can't teach a baby anything besides basic stimulus response on that time frame, and I'm pretty sure we can get computers to do that if we rig up the stimulus response hardware in a way comparable to a newborn.
What I think you actually meant, and what I think is a completely rigged test, is comparing a human with years of training and adaptive hardware modification to a computer with absolutely no training in its ability to learn a new skill or building out its knowledge base. That, of course, is a completely terrible comparison.
I'll take your bet, and call it by 2050, if you're willing to compare a human with 5 years of hardware and knowledge base training and adaptation to a computer with 1 year of hardware adaption (eg, FPGA circuit reprogramming) and 1 year of knowledge base training.
I'm willing to modify the bet for that: I think an AI with 20 years to be trained and have things like FPGA components adjusted can learn in 1 hour anything I could teach a person in the same amount of time with the same amount of pre-training.
> Not something chosen from a known list of possibilities, but any task that a human could conceivably be taught to perform in less than an hour. Anything from writing a sonnet in iambic pentameter to assembling ikea furniture based on instructions. A true test of _general_ intelligence.
The sad thing is I happen to know humans who would utterly fail at this (which, I know, is not invalidating the fact that a computer passing this would be worthwhile, but care has to be taken as to what that means), routinely challenging my faith in mankind.
I don't think specific problems are the issue. Even assembling furniture given instructions is not too difficult to ask a machine to solve, given the right programming language. It's the jump from one specific problem to another, without a heuristic used to leap between problem types. That would be a machine learning the ability to generalize and infer like a human.
The Turing test is simple because it is easy to add complexity to something that is simple. As a human, I can convince myself that an idiot is a genius and a genius is an idiot because of introspection. Humans have imagination, and we very often don't see how that warps our perception of intelligence. We still think we can be the distant, disconnected, scientific, observer - carefully constructing a pristine and universally objective technology. But computers, out of all kinds of technology, show us how much of our humanity goes into our creations.
>I would take that long bet out to 2129, and beyond. I don't see software with that level of intellectual flexibility being written in our lifetimes, or the lifetimes of our children or their children
This comic is a great illustration of how a "virtually impossible" task just a few years ago (identifying a bird in a picture) is now "virtually trivial".
A conversation is probably the best measure of general intelligence we have, at least if you allow the judges to formulate arbitrary challenges: please solve exercise 3a from Caroll. Simply because you can encode almost all human experience as text.
Having said that, I do not believe that passing the Turing test necessarily a desirable property of an AI. At the very least, a AI would need to be able to lie about its childhood. Furthermore, a AI may be truly alien, so that even if we assume a truly sentient AI it would necessarily need the ability to hide its own internal states, for example a AI may view the entire sex/gender/procreation topic as mere curiosity, and instead emulate a Human well enough that it can fool other humans.
Sure but the AI wouldn't necessarily have to respond. I think the main flaw in the Turing test is it's cultural dependence. I would argue that Eliza has passed plenty of Turing tests, mostly with people who had a chance encounter with it.
But why would an AGI individual have to be able to lie about it's childhood ? It could simply have a childhood. But I understand your argument : "Human" AI is constantly trained for the function "be a human". Machine intelligence is being trained with "is human X cheating", "is this spam", "what is the french for apple", "which way will stock X go" (and other "what will <large number of humans> do in situation X" questions) ... functions.
Keep in mind that your own brain is also simply running an algorithm, with "hidden state". Here's the first thing it hides pretty well : there's no single "you", there's about 300 regions in your brain, that are physically different, in different locations, and not directly linked. And if the rest of your brain gets disabled, each of those 300 regions can use your body to convince your own mother it's really you she's talking to. And the function your brain is executing is not to "have a soul" or something like that, but is mostly a behaviour-copy algorithm. It searches out other humans, observes their actions, learns to predict them and then uses what it's learned to "be you". Most people will not believe this is how it works, not even when they know quite a bit about neurons, and know pretty well that this is what our neurons do.
But when it comes right down to it, you, me, everyone is an "AI algorithm" just like Google search. Aside from the shear scale, number of connections mostly, there is nothing all that remarkable about the hardware it's running on.
I love how an AGI algorithm is presented in the follow up to the Battlestar Galactica series, "Caprica". Zoe Graystone explains it at one point in the series. It's fiction of course, but still. Her algorithm searches the internet, your computer, and everything and anything it can find for pictures and videos of it's "target" and then builds the function "given situation X, what would <human> do ?" and then puts the algorithm in a virtual body that looks like the human, and uploads it to V-world. She runs it on herself, a friend and accidentally on a third person she doesn't know and then proceeds to get herself blown up by a friend/terrorist (they can be both at the same time, you know) in the first episode, and their avatars wake up in V-world, sort of a facebook virtual reality edition, unaware of what has happened (and as V-world is mostly used for sex, dancing and debauchery it's not that easy to find out unless you're looking for it). Currently we don't have nearly enough data on any person to make this work ... but I am pretty convinced it is possible to make this work.
This would work, since it's doing the same thing your brain does when it hears the question "do you remember feeling happy when your mother held you when you were a baby ?". First, your brain doesn't actually remember that either way. It simply works it's way through to a good answer to that question. The thing it'll do is predict how you should answer that question, given (mostly) how other humans answer that question.
This is a very different problem from "lying about its childhood", specifically it's much more solvable. Can I predict how humans would answer questions about their childhood ? Certainly. Easy. Make the algorithm take enough contextual information into account (so that it both realizes what realistic answers are and either answers realistically or responds in a way that humans respond when they don't have a "good prediction" for an answer, for instance by saying they don't remember, or by changing the subject, or ...)
If you have young kids you will see this process in action. You have to "teach" kids memory. When they talk only a little, if you ask them something about earlier that day, they will respond. And the response makes sense, but it's not actually what happened. You will at some point realise this and correct them. Correcting them teaches them to match up what happened with their response, because they learn to predict that that's what's expected of people. Over time this correcting them will work, and they will have learned memory (and then they learn to lie, especially on questions like "did you hit your sister ?", and when that ends badly they learn to "not lie", like adults do. "Did you hit your sister ?" "She stole my lego").
Note that the underlying memory is also a prediction. If I ask you what happened this morning, and you want to try to answer correctly, you will start with an obvious fact (which isn't necessarily the truth), like "I woke up". Your brain will take that thought, and predict (not remember) what happened next. So I woke up, ok next, brushing teeth, ok next, read cell phone messages, ok next, ... ah ! the answer to the question.
>I would argue that Eliza has passed plenty of Turing tests, mostly with people who had a chance encounter with it.
People assume a conversational partner is 'real'. I could write "Hello? Is anyone there?" on a postcard, and it would 'pass' in a chance encounter sort of way.
The Turing test is useless without the tester demanding cooperation. Otherwise I've got an AI for sale that will recite an eloquent speech about the Brooklyn bridge. I call it PRINTFriend.
Well, indirectly, the Turing test does allow you to test for learning, as long as it can be taught during the period of time available for the interview and using a chat. Assembling furniture might be hard to test for, but you could certainly ask it to write a sonnet.
I wonder how the humans in the turing test would get on with sonnet writing. I imagine if they were, say computer scientists, rather poorly. I don't think I'd do terribly well at that.
Looking just now at Turing's paper he actually deals with Sonnets and suggests the conversation might go:
>Q: Please write me a sonnet on the subject of the Forth Bridge.
>A : Count me out on this one. I never could write poetry.
Write a sonnet in iambic pentameter an hour? Okay, but keep in mind you're holding your AI to a higher standard than the average human. Most people can't write a sonnet in an hour, though it's very easy to find those who can.
Can anybody point me to the papers where scientists have actually "reverse engineered (...) regions of the brain" or present "highly detailed mathematical models of (...) neurons"?
As far as I know, research in those directions is nowhere near as sophisticated as Kurzweil tries to make us believe. The mathematical models for neurons I've seen may reproduce some firing statistics, but they are not at all suitable for actually modelling behavior of a system in response to a stimulus.
I'm not sure what you are trying to say with those links. I know that people are working on mathematical models of neurons, it's just that the current models have little in common with real neuronal systems. The only systems that are described nicely are simple systems with very few neurons (eg. reflexes that involve two or three neurons). And more ambitious projects, like the Openworm project you linked to, can't seem to get past preliminary phases...
I'm not sure how you can claim "reflexes that involve two or three neurons" are state of the art or that OpenWorm "can't seem to get past preliminary phases". Neither http://www.openworm.org/science.html nor http://www.i-programmer.info/news/105-artificial-intelligenc... look like a preliminary phase to me. That's a system of more than 300 neurons, fully mapped, simulated and working, both virtually and hooked up to real sensors and actuators.
I guess what I'm looking for is the comparison of the model with experiment. How do the researchers know that they are simulating c. Elegans, and not just an artificial organism inspired by c elegans?
I don't think that a human brain, isolated and somehow kept alive, would be able to pass the Turing test. Especially not if it was grown in a tank or something. I might be wrong, of course. The story of Helen Keller seem to imply that very little stimuli is needed to become self-aware and be able to communicate in a meaningful way -- but who can say how important it is to be able to relate, on some fundamental level -- in order to share enough to pass a Turing test?
However, Kurzweil's argument focuses on the fact that he believes we will someday be able to simulate the human brain, and that's something I disagree with strongly.
Saying that we will never be able to simulate a piece of hardware/wetware is a pretty strong statement. (Unless you think there is some kind of magic inside it.)
the obstacle is that systems composed of many simple elements quickly become so complicated that we can't simulate them anymore, even if we would completely understand the individual elements.
i'd argue against that. Somewhere else in this thread someone brought up weather predictions; even though we completely understand the physics of weather, we can't simulate worldwide climate precisely for various reasons (not enough information, not enough computational resources, chaotic behavior). I think it's the same thing with our brain.
I may be wrong, but weather cannot be simulated because real version is running over the complete globe. Brain, while hard to simulate with present technology, still is a 3 pound thing. Computational resources obstacle will be overcome.
The Turing Test is a thought experiment, not an actual test. No machine will ever pass because that is fundamentally a misunderstanding of the Turing Test.
Hmm, that's a very literal and limited conception of the 'Turing Test' – almost computer-like! On the other hand, these two very smart human technologists (Kapor and Kurzweil) both seem OK with an actual-test reification of the 'Turing Test'.
And then, the word 'fundamentally' has been thrown in to bluff confidence, but without providing actual supporting reasoning. That's a bit like some of the old ELIZA evasions.
Based on the content, I vote the parent comment 'AI'. Did I get it right?
Thank you for pointing this out. It always bugs me to no end when people talk about 'passing the Turing Test', it just shows a huge lack of understanding on what Turning was getting at.
The important part of the Turing Test isn't whether or not we can build a computer to 'pass it', it's what is it that differentiates us from just being complex computers that can spit-out the right answer when asked, and if there actually is any difference. The question really is "If a computer can act exactly like a human, to the point where people can't tell the two apart, what exactly is the difference between that computer and a real human?". Most people would say that the computer can't "think" and a human can, but if the computer can shift bits around in such a way that it comes to the 'right' response, what's the difference between that and 'thinking'?
That's the point of the thought experiment in the paper, but that thought experiment includes a description of an actual test that can be made, which is what people call the Turing Test.
A fallacy is equating the ability to think alone with our human potential. It's not just thinking that has built human civilization. Coming up with the correct answer for any query is one thing - evolving from a single cell to manipulate our environment and having consciousness spontaneously emerge and then within a few thousand years discover many of the secrets of the universe and on the verge of becoming a god-like species if we don't destroy an entire planet first.
When the machines can do that then we can compare apples to apples.
Thinking is a nice feat and I think it's going to be solved in many of our lifetimes. It's definitely something I'd like to research more at some point.
I feel like you're missing the forest for the trees, but I have to say I find it somewhat disturbing that you feel as though you have to assert your superiority over machines by citing an argument that's basically "I evolved, you were created, so I'm better".
Laypeople in general. I think that's part of the fallacy - the idea that a 'turing test' is some recognized standard for when computers are smarter than humans. But as others have stated it works a lot better as a thought experiment than something to directly pursue.
There's a t-shirt out there that says, "you < turing", and while it's intended as a nerdy insult, the question I have is how many humans could pass a Turing Test?
They made this bet in 2002, so we're almost halfway to 2029. Does anyone (other than Kurzweil) seriously think a Turning Test-passing machine is just over the horizon?
(And no, contrived scenarios with computers pretending to be foreign children don't count[0])
But Kurzweil is banking on the exponential growth of technology. According to his reasoning (and using Moore's Law of 2x performance every 18 months, as a guideline) we are less than 1% of the way towards the goal (assuming 100% is passing the Turing test).
Now I don't have any idea if he will be correct, or even what a machine that will pass the Turing test will be like. But if you accept the premise that technological advancement is on an exponential curve, half way in terms of progress will be much closer to 2029 than half way in time.
(Note I am not saying that technology is exponential or that it is following Moore's law or that it will be exponential forever or that 2x every 18 months is fact, its simply an extension of Kurzweil reasoning to its logical conclusion.)
Exponential improvements aren't very helpful if you're working on exponentially hard problems.
Exponential growth of technology hasn't solved machine intelligence (by direct simulation) for essentially the same reason why it hasn't solved numerical weather prediction or made quantum computers pointless.
To expand on weather prediction: you need an order of magnitude improvement in computer speed to make weather forecasts with the same quality one day earlier. So exponential growth of computing power means linear growth in terms of subjective benefit.
To be fair, we don't understand the difficulty of modeling intelligence anywhere near as well as we understand the difficulty of modeling the weather. But, given the limits of our current abilities to simulate the brain, it seems reasonable to guess that similar principles could hold.
Well, it is. You can't claim it grows in any specific way, because it does not grow (at least at the Local Group).
You can not claim that exponential growth on space travel tech isn't enough to get intergalactic travel. Any non-declining rate of progress will get us there eventually. (Whatever "exponential growth" means on this context, for computation it's very well defined. Also, whatever realist is on non-declining rate of progress, the rate for computation is clearly increasing.)
>Exponential improvements aren't very helpful if you're working on exponentially hard problems.
Sure they are! They reduce it to a problem that's only polynomially hard! :-p
>Exponential growth of technology hasn't solved machine intelligence (by direct simulation) for essentially the same reason why it hasn't solved numerical weather prediction or made quantum computers pointless.
Okay, but that's a rather high bar to judge it against. Humanity has solved many lesser machine intelligence problems, like web search, voice recognition, routing, and recommendation. Though I agree the relevant exponential growth was in the power of the algorithms, not so much the hardware.
I don't see why "weather forecasting" is similar to "brain simulation" in difficulty. The comparison would be more valid if the problem were "brain forecasting", that is to build an AI that would behave exactly the same as a particular human, but that's an even harsher measure of intelligence than the turing test that I doubt anyone would take seriously. I'm not saying the brain's easy to simulate, it's just the analogy here seems shaky. Where is the source of exponential difficulty?
I think there's an excellent chance that deep learning research will lead to a machine that can pass the Turing test in the not too distant future. I can't say if it will be within exactly 14 years or not, but if you've been following the latest developments in deep learning, the path to get there is much more clear today than it was even five years ago.
I think you are greatly overestimating what deep learning can do. In the 90s, we could recognise digits accurately. Now we can do the same with traffic signs even in bad weather etc. That is exactly the kind of progress that we have made in 14 years. And let's not forget this is a manually tuned algorithm for a particular problem.
It's great progress, but it is also a far cry from what humans can do and there is no clear path at all to get there - currently.
The recent advancements using RNN is much more than sign recognition. It's about language generation, control and reinforcement learning, and attention.
I assume that deep learning will help with coming up with suitable replies. But will it also helping with keeping track of state, ie. making sure that the computer's answers are consistent? It seems to me that this will be the hardest part about passing the Turing test.
agreed - there are current approaches that are becoming feasible as infrastructure is opened up and affordable to the general public. It's a game changer but a lot of people misinterpret the meaning and implications it seems like. A good question to ask is how this advance would affect you directly.
Given the specific 'Turing Test' procedure described in the bet, I think there's an excellent chance Kurzweil will win. (If the IM sessions were 5 minutes with each judge rather than 2 hours, he might win today.)
2029 is a long way still and I would be surprised if the Turing Test wasn't passed. The past few years have had massive progress in AI. A few years ago machine vision was an impossible problem believed to require strong AI, and now we are exceeding humans.
Massive progress is being made in natural language understanding too. A number of papers have just come out on using deep NNs to do question answering or conversation.
Those are images incredibly optimized to fool and exploit those neural networks. However on the actual image recognition tasks they were designed for they beat humans.
That's exactly my point. Computer vision is a "solved" problem within certain specific domains for which a particular algorithm has been trained/tuned ("the actual image recognition tasks they were designed for"). They are nowhere near as good as humans at the general-looking-at-any-random-things task yet.
Similarly, I imagine we will get very good domain-specific "AI" in the next ten years: some sort of amped-up Siri that can, say, answer questions about hotels in a travel destination and book one for you.
I do not think we will get a chatbot that can hold a conversation on any conceivable topic at a level that you would confuse for a human.
Imagenet is a pretty open task. The images are taken randomly from the internet with real world clutter and noise and other issues. There are 1,000 classes which is pretty large compared to previous datasets.
When it started getting attention a few years ago algorithms sucked at it. No one would have believed in a few years we could beat humans on it. It seems like a case of moving goal posts, where every time a milestone is reached, it's disregarded.
> It seems like a case of moving goal posts, where every time a milestone is reached, it's disregarded.
I think it's sort of a case of AI researchers tried moving the goalposts closer, and now we're moving them back. Back in the 60s, a bunch of people thought a general-purpose artificial intelligence was a realistic goal in the near future.
Then it turned out that problem was hard, really hard. AI researchers instead defined-down the problem. Instead of building AI, they worked on building "chess-playing AI" or "traffic-optimizing AI". These are useful and interesting in their own right, but they're not really what people mean when they talk about "intelligence".
Over time, this meant that the entire field of "AI" became discredited, which is why nobody talks about AI anymore, they talk about "machine learning", or "deep learning", etc. If somebody says they're going to make a human-level general-purpose AI, they're no longer taken seriously.
So what metric are you using to measure progress in AI? There are plenty of benchmarks on very general tasks like image recognition or language modeling, etc. There is no room for bias or moving goal posts on a benchmark.
All I'm saying is that AI is rapidly advancing. I'm not claiming its human level yet, and I don't really care what people in the 50s predicted.
The more I think about the Turing Test, the more flawed (or hackable) I see it. What if instead of improving the computer I go the other way around and I put a disabled human (like an autistic or something) behind the curtain? This may certainly exhibit a very unnatural model of thought and make the computer harder to identify. If however, such hack would be prevented by the fact that the judge is the one that chooses his human subject for the test, like they knowing each other to some degree, then the test becomes more of a challenge to recognize the specifics that one particular person may have in relations with not only computers but other humans as well!
The turing test must have an adversarial component. E.g. For any competition with X entrants, the computer candidate must be compared against one of the other human entrant team member, randomly selected. If the other entrant is (correctly) identified as the human, a fraction of the year's prize, say 1/X, goes to the adversarial team, and the candidate is barred from winning that year.
If a computer can convince judges that it is human, then it can also convince judges that it is sentient. If it is convincingly sentient, is it moral to program it?
Investing the money is definitely part of what makes this interesting. Thanks to compound interest, some truly large sums could be at stake by the time the bet is resolved. (We also couldn't do it any other way, as a number of the bettors are unlikely to be around when the bets are resolved.) The money, though, is not held directly by the Long Now. It's in a special account set up with The Farsight Fund of Capital Research and Management Company. That's mentioned here: https://longbets.org/about/http://longbets.org/faq/
As to whether it'll exist in 2029, I'd say the odds are good. The Turing bet is a 27-year bet and we're already nearly halfway there. But if you don't think so, I will be entirely glad to bet against you. ;-)
The Long Now for years had a little museum space that got only a modest number of visitors. But conversation about the long term is their goal, and they realized that coffeehouses and bars are where a lot of good conversation happens, so they converted the museum into a cafe during the day and a bar at night.
If you're ever in San Francisco, it's a great nerdy tourist stop. It's in Fort Mason, on the north edge of San Francisco between Fisherman's Wharf and the Golden Gate Bridge.
The long now foundation are quite an interesting bunch, known mostly for their 10,000 year clock:
"The Long Now Foundation itself is the brainchild of inventor and engineer Danny Hillis, who launched the non-profit to build the clock. The Long Now foundation has over 3,300 members who are supporting the project, but Bezos is by far the most prominent and seemingly deep-pocketed, kicking in a projected $42 million"
The only other search result for the phrase "genderless no form-factors" is an HN comment from four days ago, also by a brand new user (not the same username, but I'm guessing the same user). It is, as far as I can tell, a nonsense phrase.
What are you trying to say, and can you say it in English?
Conversation is actually a rather poor measure of intelligence. I would say, show me a computer that can learn anything that it wasn't specifically programmed to learn. This doesn't mean unsupervised categorization or learning to play a video game. I'm talking about a scenario where programmers present their code or machine with no knowledge of what the task will be. Not something chosen from a known list of possibilities, but any task that a human could conceivably be taught to perform in less than an hour. Anything from writing a sonnet in iambic pentameter to assembling ikea furniture based on instructions. A true test of _general_ intelligence.
I would take that long bet out to 2129, and beyond. I don't see software with that level of intellectual flexibility being written in our lifetimes, or the lifetimes of our children or their children.
[0] "While it is possible to imagine a machine obtaining a perfect score on the SAT or winning Jeopardy..."
[1] "... a skeptic about machine intelligence could fairly ask how and why the Turing Test was transformed from its origins as a provocative thought experiment by Alan Turing to a challenge seriously sought."