Jeff
Hawkins made a name for himself in the tech industry as the founder of Palm Computing and inventor of the Palm Pilot. He later founded Handspring, where he invented the Treo. If you were a fan of his work then, you are going to love what Jeff is
up to now. He is currently pursuing his life-long passions, neuroscience and
intelligence. His latest work made quite a splash a few years ago when he published On
Intelligence. In this thin volume Jeff Hawkins elegantly summarized his
theory of how the brain gave rise to intelligence. Disputing conventional
wisdom
that the brain is complex, or that intelligence is inseparable from other human
qualities such as emotions, Jeff put forward a proof that human intelligence
is a function of the neocortex and that it is temporal in nature.
To prove his theory, Jeff founded Numenta - a company dedicated to developing algorithms and software based on the ideas put forward in the book. This spring Numenta released its first product, an experimental software aimed at researchers and advanced developers which embodies the algorithms and techniques pioneered by Jeff and his crew. Numenta is presenting here at ETech today and so it's a great opportunity to familiarize you with these exciting new developments. Has the age of Artificial Intelligence arrived? Is it what we thought it would be? Read on to find out.
One of the key insights that Jeff had was based on the fact that life has a spacio-temporal quality. This is a fancy way of saying that things happen in space and time. It is of course basic physics, but Jeff concluded that the structure in our brain that models reality, should also have spacio-temporal characteristics. After all, a good model is an approximation of the actual process. With that, Jeff looked for a part of the brain that would fit the description and immediately realized that it is the Neocortex.
Jeff and his colleagues spent a lot of time studying the neocortext and were able to understand its essential operations. Based on their understanding they created the Hierarchical Temporal Memory (HTM) model, which captured the essential computation by constructing tree-like hierarchies. Like its biological forefather, the Neocortext, HTM applies the same algorithm to all inputs. The four basic operations performed by each element are:
- Discover causes in the world
- Infer causes of novel input
- Make predictions
- Direct actions
This model, the scientists claim, simulates what would commonly be classified as intelligence.
1. Discover causes in the world
Similar to the neural networks, HTM does not have any prewired classification of the world. Instead, HTM accepts a sequence of spacio-temporal inputs and 'learns' the patterns in the input stream. In the diagram above, the senses digitize the signal and turn them into bitmaps (or vectors), which are then are processed by a classification system. The system then assigns the likelihood of a particular cause to each symbol. In plain english, you are shown a sequence of pictures of cats and dogs - and each picture you classify as either a cat or a dog. But just like we can't do that when we are born, neither can HTM. In fact HTM needs to go through a training process before it can 'learn' to distinguish things.
2. Infer causes of novel input
A trained HTM is able to assign the likelihood of a particular cause. Given a new input, the system then uses its previous knowledge to classify it. People actually do the same thing; given a sequence of pictures of cats and dogs, there is a chance (small) that we will make a mistake. What is particularly interesting is how HTM deals with novel input - it is used to continue the learning process. Each new input, along with its temporal aspect, is processed by the system and causes the system to change. As an example, think of the process involved in recognizing an object via sensory input - we move our hands around it in order to recognize the object. Jeff Hawkins explains that this ability to handle continuously variable input is one of the keys to make the whole system work.
3. Make predictions
The ability to predict or to imagine things is one of the most basic human abilities. Forecasting, mental modeling, imagination and planning - these are powerful attributes of intelligent behavior in humans, which each find place in HTM. Each node in the HTM network combines its memory with incoming signals, to predict what is going to happen next. This prediction can actually serve as an input itself, mimicking the process of imagination in humans. The entire network is able to compute a series of future states - so for example, like people, it is able to anticipate bad or dangerous situations before they actually take place.
4. Direct actions
Probably the most important thing that people do after they think (most of the time) is act. The ability to calculate the sum of all inputs, conclude and do something, has been wired into HTM. Since the model itself has no way of interacting with the external world, its actions need to essentially go through a translator before being implemented (think how brain controls movement for example). So in its raw version, HTM actions are just internal commands that can be interpreted in various ways. For example, they can be hooked up to the motor generator to power physical behavior. In this first version of the model, the set of basic behaviors is pre-wired. However even in this early stage, the model is capable of generating complex responses by combining the basic building blocks.
Hal, are you there?
So what are we to make of this? Have Jeff Hawkins and his researchers at Numenta invented Artificial Intelligence? The answer is yes and no. It is likely that some future version of their system is going to be able to pass the famous Turing Test, but hardly anyone would mistake the Numeta creation for a human being. In fact the very beauty of this creation is that it decouples intelligence from other human qualities. Jeff and his colleagues invented an algorithm that mimics typical computation which occurs in our brains, but it is far from being a complete artificial intelligence.
So in terms of moral and ethical implications, right now there are no issues. Could there be in the future? Yes. The future generation of this algorithm, if implemented in advanced robots, could become closer to what Arnold Schwarzenegger so elegantly portrayed in the Terminator series. But seriously, as with any technology care must be taken as to how and where it is used.
In the meantime, we are excited to report on this breakthrough. Jeff's invention has paved the road to a new, brain-like computing paradigm. It is possible that the long-awaited promise of neural networks and cellular automata is finally being delivered. This means that computers will be able to tackle problems that come so easy to us, like recognizing faces or seeing patterns in music. But since computers are much faster than humans when it comes to computation, we also hope that new frontiers will be broken - enabling us to solve the problems that were unreachable before.
This post is based on the white paper on Numenta's web site. We highly recommend it, as it has a lot enlightening details about the architecture of HTM. Please take a look and let us know what you think about this exciting development.
]]>