Numenta - Has Artificial Intelligence Arrived?
Jeff
Hawkins made a name for himself in the tech industry as the founder of Palm Computing and inventor of the Palm Pilot. He later founded Handspring, where he invented the Treo. If you were a fan of his work then, you are going to love what Jeff is
up to now. He is currently pursuing his life-long passions, neuroscience and
intelligence. His latest work made quite a splash a few years ago when he published On
Intelligence. In this thin volume Jeff Hawkins elegantly summarized his
theory of how the brain gave rise to intelligence. Disputing conventional
wisdom
that the brain is complex, or that intelligence is inseparable from other human
qualities such as emotions, Jeff put forward a proof that human intelligence
is a function of the neocortex and that it is temporal in nature.
To prove his theory, Jeff founded Numenta - a company dedicated to developing algorithms and software based on the ideas put forward in the book. This spring Numenta released its first product, an experimental software aimed at researchers and advanced developers which embodies the algorithms and techniques pioneered by Jeff and his crew. Numenta is presenting here at ETech today and so it's a great opportunity to familiarize you with these exciting new developments. Has the age of Artificial Intelligence arrived? Is it what we thought it would be? Read on to find out.
Hierarchical Temporal Memory (HTM)
One of the key insights that Jeff had was based on the fact that life has a spacio-temporal quality. This is a fancy way of saying that things happen in space and time. It is of course basic physics, but Jeff concluded that the structure in our brain that models reality, should also have spacio-temporal characteristics. After all, a good model is an approximation of the actual process. With that, Jeff looked for a part of the brain that would fit the description and immediately realized that it is the Neocortex.
Jeff and his colleagues spent a lot of time studying the neocortext and were able to understand its essential operations. Based on their understanding they created the Hierarchical Temporal Memory (HTM) model, which captured the essential computation by constructing tree-like hierarchies. Like its biological forefather, the Neocortext, HTM applies the same algorithm to all inputs. The four basic operations performed by each element are:
- Discover causes in the world
- Infer causes of novel input
- Make predictions
- Direct actions
This model, the scientists claim, simulates what would commonly be classified as intelligence.
1. Discover causes in the world
Similar to the neural networks, HTM does not have any prewired classification of the world. Instead, HTM accepts a sequence of spacio-temporal inputs and 'learns' the patterns in the input stream. In the diagram above, the senses digitize the signal and turn them into bitmaps (or vectors), which are then are processed by a classification system. The system then assigns the likelihood of a particular cause to each symbol. In plain english, you are shown a sequence of pictures of cats and dogs - and each picture you classify as either a cat or a dog. But just like we can't do that when we are born, neither can HTM. In fact HTM needs to go through a training process before it can 'learn' to distinguish things.
2. Infer causes of novel input
A trained HTM is able to assign the likelihood of a particular cause. Given a new input, the system then uses its previous knowledge to classify it. People actually do the same thing; given a sequence of pictures of cats and dogs, there is a chance (small) that we will make a mistake. What is particularly interesting is how HTM deals with novel input - it is used to continue the learning process. Each new input, along with its temporal aspect, is processed by the system and causes the system to change. As an example, think of the process involved in recognizing an object via sensory input - we move our hands around it in order to recognize the object. Jeff Hawkins explains that this ability to handle continuously variable input is one of the keys to make the whole system work.
3. Make predictions
The ability to predict or to imagine things is one of the most basic human abilities. Forecasting, mental modeling, imagination and planning - these are powerful attributes of intelligent behavior in humans, which each find place in HTM. Each node in the HTM network combines its memory with incoming signals, to predict what is going to happen next. This prediction can actually serve as an input itself, mimicking the process of imagination in humans. The entire network is able to compute a series of future states - so for example, like people, it is able to anticipate bad or dangerous situations before they actually take place.
4. Direct actions
Probably the most important thing that people do after they think (most of the time) is act. The ability to calculate the sum of all inputs, conclude and do something, has been wired into HTM. Since the model itself has no way of interacting with the external world, its actions need to essentially go through a translator before being implemented (think how brain controls movement for example). So in its raw version, HTM actions are just internal commands that can be interpreted in various ways. For example, they can be hooked up to the motor generator to power physical behavior. In this first version of the model, the set of basic behaviors is pre-wired. However even in this early stage, the model is capable of generating complex responses by combining the basic building blocks.
Hal, are you there?
So what are we to make of this? Have Jeff Hawkins and his researchers at Numenta invented Artificial Intelligence? The answer is yes and no. It is likely that some future version of their system is going to be able to pass the famous Turing Test, but hardly anyone would mistake the Numeta creation for a human being. In fact the very beauty of this creation is that it decouples intelligence from other human qualities. Jeff and his colleagues invented an algorithm that mimics typical computation which occurs in our brains, but it is far from being a complete artificial intelligence.
So in terms of moral and ethical implications, right now there are no issues. Could there be in the future? Yes. The future generation of this algorithm, if implemented in advanced robots, could become closer to what Arnold Schwarzenegger so elegantly portrayed in the Terminator series. But seriously, as with any technology care must be taken as to how and where it is used.
In the meantime, we are excited to report on this breakthrough. Jeff's invention has paved the road to a new, brain-like computing paradigm. It is possible that the long-awaited promise of neural networks and cellular automata is finally being delivered. This means that computers will be able to tackle problems that come so easy to us, like recognizing faces or seeing patterns in music. But since computers are much faster than humans when it comes to computation, we also hope that new frontiers will be broken - enabling us to solve the problems that were unreachable before.
This post is based on the white paper on Numenta's web site. We highly recommend it, as it has a lot enlightening details about the architecture of HTM. Please take a look and let us know what you think about this exciting development.
Share3 TrackBacks
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/2081
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
Things like this are so exciting. Nice article.
As I was writing yesterday, you don't create AI by having a rigid structure you need to let the system draw its own conclusions (right or wrong) and this is what it's all about.
Sounds pretty exciting, yes.. But there are tens or maybe even hundreds of similar theories that try to explain how the brain works. Most of them sound really exciting and interesting but there has been no way of replicating this mechanism on man made machines yet. So I keep my breath till they find a way to make it really work, but I'll definitely check this out more.
As an autistic person and the parent of an autistic son, I've been interested in (and writing/speaking about) the characterization of autism - ways in which our functioning/processing seems to differ from the (putative) norm. The conceptual models posited in this article broaden the range of possibilities to consider. Thanks! :-)
Will this also rely on human input? For example, give the computer a picture of a cat, how does it know it's a cat? I really like the "grid" concept; that is, computers can learn from what other computers are doing.
Very interesting.
Robert,
It does not rely on human input in that sense. It knows in the similar way that you and I know before we call it a cat. So you form an idea of a thing, you can distinguish it from other things, but you do not need to call it anything. Of course to be able to communicate with people eventually, such system would need to have the natural language mapping.
Alex
Ah, okay... Thanks for the clarification, Alex - that makes perfect sense.
This sounds like a modern view of french philosopher Henri Bergson about perceptions. See : Matter and Memory 1896
http://en.wikipedia.org/wiki/Henri_Bergson
I love that as people are exploring AI, Semantics, NLP all those kinds of technologies it inevitably involves psychology and philosophy before you can create a working product.
Bot programming is one of my favorite things to do..creating a mind online to think for you. I would love to read up more on this. Great work!
I often think of what my wife says about men when it comes to this kind of stuff - "you're not really listening to what I am saying". Computers and sophisticated algorithms by themselves don't really convey the type of "AI" people are looking to get from their online experience. Google can scan all the books it wants, but their massive data farms will never feel the emotion from the words it captures. It will never digest a story about a sports personality and cheer for them inside their digital brains, or cry when a hero dies, or feel that rush of excitement when it comes to the climax in a good thriller. The Googleplex can never believe in dragons like my little girl because she read about them.
For that reason, computers cannot ever really truly understand the language. They can compare things, make words easier to find, even give you information about a shiny red car you see in a video. Until a digital brain can understand how a person has passion for something or someone or how we can care about a fictional character as much as someone we know (Rachel and Ross), it cannot truly be consdered having "beliefs". It has contructs and maybe even a little bit of understanding, but only at the most basic level (similar to the worm understanding there is something in front of it, but not knowing what it is or why it is.)
Until computers can feel (which is a very long way off), the best we can do is hope they can approximate us at a superficial level. Everyone is working to make that approximation better and better, but the best anyone can do is getter better sense of what is in front of the worm.
i am sure this modellimg can provide some further input to the attempts to create human-like intelligence but I think the writer got a bit too excited and perhaps comfused marketing-speak with reality. This model is hardly a breakthrough, it is interesting but as others have mentioned it is very similar to hundreds other such models.
Numenta's technology has huge potential in the fields of pattern recognition and data mining. That's big business, and with Jeff's marketing skills they can't fail.
But the technology has yet to show that it's capable of more than the old "neural network" architectures that failed in the past few decades... and whether HTM can ever do deliberative reasoning is another question!
alexjc
"It will never digest a story about a sports personality and cheer for them inside their digital brains, or cry when a hero dies, or feel that rush of excitement when it comes to the climax in a good thriller. The Googleplex can never believe in dragons like my little girl because she read about them."
Not true. It could be programmed to do that, just as your brain is. It's all just data in one form or another...it's just very difficult for an individual to understand that what they "feel" as emotions or pain is the feedback mechanism internal to the processing of this "data".
Finally, HTM is really just another learning algorithm very similar to what already exists in the field of data mining.
Data mining programs have been able to infer causes and relationships for a long time...this isn't the breakthrough that the company wants you to think it is.
Can someone point me to any peer reviewed scientific publications on HTM? I haven't read the guy's book and from your description (and the one on their website) it does not sound all that exciting or different than many other approaches that try to model temporal and spatial hierarchies. It is very easy to develop a general theory that says abstract this and that and you can solve the biggest problems, but once you start looking at the actual complexity of hierarchical solutions, you realize that the models are still highly intractable. I wouldn't get too excited about this.
Really interesting.
PS: I think that elephant image has been taken from my 4th/5th grade CBSE English text book:)
I think the essence of what Bob Ellsworth is getting at a few posts back is that we can create a machine capable of human-like intelligence, but that does not mean it will be capable of developing a human-like personality.
There is a lot of philosophy out there on this topic. I think some of the best of it has to do with the marriage of mind with matter. Some of the things that make us distinctly human are rooted in the physical. Particularly, the senses that are available to us, our sense of pain and desire for self-preservation, our sense of age and mortality, our sense of pleasure, our ability to physically produce offspring, and the linkage of pleasure to physical activities and non-physical abstractions such as wealth, power, camaraderie, etc.
Here are the steps I think we would need to take _after_ we develop a machine capable of human-like intelligence, to achieve something close to an artificial human-like personality:
1) Place the intelligence in a body that has human-like senses: vision and hearing are most important. Touch is important as well, but could come later. Smell and taste could come even further down the line.
2) Place the intelligence in a body that has a roughly human-like form. o
I don't mean to be a negatron but this isn't nearly as academically exciting as the title implies and the article is a bit misleading. The 4 step operations remind me of Brooks's 3 step dynamic approach to AI but with more real world knowledge applied. As for the quote about passing the Turing test, sorry but not through this approach. "capable of human-like intelligence, but that does not mean it will be capable of developing a human-like personality." != being able to pass the Turing test. Without an integrated approach to perception and understanding like that being tackled at FARG, you can keep dreaming about Alan's famous standard. It's interesting work nonetheless, and if you combined this with cutting edge bayesian models you could really have something, I mean Google's worth billions from far worse algorithms....hint hint....(*thinks about investing in Numenta)
Jordan
If I could run a little program on my computer that traded my stock portfolio as well or better than any human has ever done, I wouldn't really care whether it could pass a Turing test or had emotions.
Maybe I'll live long enough to see machines who believe they have consciousness and feelings. For now, I'd be delighted to settle for one that could do the simple useful things. Like laundry, cooking, cleaning, or even just piloting a Furby in amusing ways.
"It is likely that some future version of their system is going to be able to pass the famous Turing Test"
And exactly why is this likely? I didn't see this prediction substantiated? Just like others say, why is this so much different from Neural Networks? Nonetheless, I like to read more about these kinds of developments. Keep the interesting news coming!
OK - I was all for AI then I started thinking..
If they can make AI understand input that is not of proper grammar, spelling and punctuation that would be great. An AI that can really get the data I want to see. For example, if I ask for schematics for a low noise guitar pre amp and it comes back with something close to being 100% valid, that would be good. If the AI can look at the schematic pictures or ASCII text - break it down, look at the components, find the specs and do a SPICE check on it and come back with the top 10 schematics, that would be great - but a little scary. If it can do that and then come up with it's own schematics and SPICE results and even possible PCB designs, that's where I would be scared.
Why would I be scared? Well I'm a professional programmer. If I were to ask the AI "I would like to have an eCommerce program that did this, that and some other things in PHP" and it came back with a whole system that it programmed with all the features I wanted and more (including making it secure and optimized) then I would be out of job along with a lot of other people in other industries.
WOW..but it's a far cry until we can put this technology into use to work even slightly reliably.
I should probably carefully read the white paper before saying this, but on a first sight I wonder what's the novelity of this "algorithm" ?? As a so called "expert" I cannot really see a decisive difference between this model and other existing models.
To alex:
"...which are then are processed by a classification system" . The classification system is crucial for the perfomance of a neural system, so it's a little irritating that you don't lose a word about it...
"The answer is yes and no. It is likely that some future version of their system is going to be able to pass the famous Turing Test".
I would be very careful about this kind of conclusions. Really, some researchers might be quite pissed now ;)
@22 Eray, your need to look at the exact algorithms to see the difference, but main difference between HTM and NN is that HTM takes into consideration temporal patterns and that it thrives on processing different inputs.
Re: Turing Test. I am pretty careful ;) I have masters in computer science and spent 8 years studying complex systems, information theory and computability. Obviously any conclusion that I draw is subjective, but my point of view that this approach in the future might lead to a system that can pass Turing Test.
Alex
From the this writeup (and some of the numenta marketing copy), one could think that Mr Hawkins has just single-handedly invented the field of machine learning.
A slightly more grounded comparison between this and existing ML work is on their pg here.
If you are interested in actually reading up on this sort of thing, you could check out the wikipedia stuff (which is actually pretty good), or recent papers from these conferences/journals:
NIPS
JMLR
ICML
oops those journal links got messed up
NIPS
JMLR
ICML
There are some interesting ideas here, but the hard part of AI research is making the ideas concrete and running experiments to find the relative strengths and weaknesses between your approach and other related (there are MANY) approaches.
If this is so revolutionary, why not publish the work in peer-reviewed conferences or journals (like the ones listed above)? Until then, I say it's too hard to tell how worthwhile this is.
Very nice article.
My excitement will be peaked when any kind of algoritm can do the following: tell the difference between "love" - like "I love you" and "I would love for you to die"
Entirely different things, but the same word. Even a 5 year old can tell the difference. But not the giant brain at Google. It could if we put in all the rules about what we are searching, but you can't say "show me all the emotionally positive results with the word LOVE". Maybe you can program in this kind of emotional discretion, but I don't see it happening anytime soon.
Everything else is simply refining the search algoritm. What I am talking about is changing the structure of search from words to feelings/emotions/visuals (exciting car, cool shoes, etc.)
HTM, once developed further, will prove to be an asset to mankind. It will allow us to be removed from mundane to dangerous situations that do not require certain variables as well as values that “only " the human mind can fully comprehend.
“I think therefore I am “is a quote that exemplifies the realization of man as a cognitive self-aware being. Any form of AI could only hope to mimic the minds reactions, but will never capture the true essence of “I am......"
www.lost.eu/38606
^^ thought it was relevant.