Hyperlinking the Real World
European researchers working on the MOBVIS project have developed a new system that will allow camera phone users to hyperlink the real world. After taking a picture of a streetscape in an urban area, the MOBVIS technology identifies objects like buildings, infrastructure, monuments, cars, and even logos and banners. It then renders relevant information on the screen using icons that deliver text-based details about the object when clicked.
This project goes beyond today's mapping applications like Google's Street View, for example, which first identifies your location either via GPS or triangulation and then shows you pictures of that area. Instead, MOBVIS actually lets you "see" the world through your mobile phone. This is computer vision, or rather, mobile vision.
There are obviously numerous potential applications for such a technology. On the MOBVIS homepage, they offer up some scenarios for how their application could be used, including the following:
Tourism/Augmented City Maps: The MOBVIS technology could be used to inform visitors about the objects in an area be them buildings or landmarks. The images could also be annotated with additional information like history, event information, or information about nearby shops.
Visual Localization: For phones without GPS technology, triangulation could be combined with the computer vision technology to locate a user's position and orientation in a manner that would be comparable to GPS and just as accurate.
Motion Estimation: Also comparable to GPS, MOBVIS could enable continuous position updates to determine the location of objects in motion as well as their speed.
Incremental Map Updates: MOBVIS supports incremental updating of maps which would allow for the automated authoring of the urban infrastructure. No longer would Google need to send their vans around taking pictures of streets - the data could be uploaded from users' phones as they took their photos.
Picture-Driven Search Engine: Because the mobile phone could now "see" the surrounding landscape, the world - reality - becomes the backdrop for a sort of picture-driven search engine in which the objects in the world are all hyperlinked and annotated like a real-life semantic web.
How It Works
The MOBVIS system begins with a pre-populated database of geo-referenced panoramas (such as Google's Street View, perhaps). The objects in the images are then manually annotated with information. Once that's complete, the system is ready for search queries from mobile users. After a user takes a picture, MOBVIS compares the photo to the photos in its database and returns the relevant links.
The challenge here is getting a mobile phone picture to match up with the more pristine photos found in the database. The database photos would likely be clear, crisp, and detailed, but a user's photo could be grainy, taken on a dark and cloudy day, or taken from an odd angle.
The MOBVIS system's main strength comes from its feature-matching algorithm developed by the University of Ljubljana in Slovenia, one of the partners of the project. This algorithm can very accurately detect minute differences between similar objects. In real-world tests, it's reported that this system was highly accurate, detecting the right building 80 percent of the time.
Aleš Leonardis, head of the Ljubljana team, believes that number can be improved, too. He also notes that that the system, though not always right, was never wrong. "It was remarkable that there were no false positives," he says. "Sometimes the system couldn't identify a building, but it never put the incorrect link on a building."
You can read more about the research here on the MOBVIS project's homepage.
ShareRelated Entries
1 TrackBacks
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/9638
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
And then the machines take over.
A video demonstration of the MOBVIS system is available at http://vicos.fri.uni-lj.si/HypR/
Posted by: dusano.myopenid.com
|
January 2, 2009 8:43 AM
This would be amazing, almost like having your own personal tour guide everywhere you go. I know there were companies working on this type of technology, didn't think they were up to it just yet with the phones. Will take time to perfect and I'm sure there are a lot of questions to it but looking forward to learning more about it.
I am going to file all of this under N for Never Gonna Happen.
Seriously, a phone is *not* going to have a GPS but instead you'll just take a bunch of photos and "that will be as good as a real GPS"? In what way?
@Jeff if by 'N for never gonna happen' you mean 'I didn't know it already happened', then sure. :)
There are primitive apps that let you point a webcam at a picture and will load another piece of data over it on screen. I'll dig up a video I shot, where I held up a picture of a car printed on a piece of paper in front of a webcam, and on the display, it displayed a video.
There are also a few console video games on the market that do this with various bar code derivatives.
Now, with google doing indexing of buildings, satellite imagery, mapping data, 3D repositories growing of every city on earth, they probably have everything they need to index against.
The app I used in that scenario would be simply cloud based. It sees a pattern and displays something over it on the display (which is currently a screen on a computer or phone-- we aren't at the comfortably, un-nerdy headsup display era yet heh).
We are much closer than we think, because it is happening.
This sounds a lot like a more open version of Nokia's Point and Find technology ... http://www.nokia.com/pointandfind
Mobvis guys better watch out, because TonchiDot says "We Have Patent!" ;)
(also: "Join Us!" and "Look Up, Don't Look Down!")
see http://www.techcrunch.com/2008/09/17/tonchidot-madness-the-video/
Very cool to see this!
This guy was my professor at the university. The course he teaches (computer vision) definitely was one of more interesting ones!u
I hope prof. Leonardis and his team find a way to move this into production!
(oh, and great to see so many images from Ljbljana, Slovenia at RWW :)
Andraz Tori, Zemanta
love this concept!! can't wait for 3d HUD (glasses?) + human tag/linking.
ciao,
'enzo
--
http://twitter.com/_enzo
This is great. We are at the beginning of a true information revolution. A picture is worth a thousand words, literally. Soon, computing will be a much more visually interactive experience. The tipping point will come when it becomes something you don't even need to think about.
Well, they were funded by EU money and this year they had to show something.
The problem is that this technology is not ready for consumer and business. Why?
- you need to take panoramic pictures before you even start using this (they mention google street view, but even google street view doesn't have all the places covered and they never showed that it works with google street too), expecting that people will take pictures and upload them is just naive
- it takes to long to process the picture
- this tech in the next years will be surpassed by similar but yet better tech: instead that you process image, you just use orientation device/gyro (similar to iphone) that will be in the phones, this will give you 99,9% accuracy all the time
- simple always win
Well, they received EU funding and they COULD show something ;-) This represents a proof of concept and is ready to be marketed.
- orientation device/gyro will NOT be able to work properly in urban environments, be sure about this .. think about signal reflections from buildings, cars, roads, etc. furthermore, vision technology is able to interpret images of dynamic objects that have not been mapped before, and enable to register the device with the environment accurately (vision based).
- usability studies show that people enjoy using the service and do not bother about some delay. However, there will be much faster technology available in the near future.
- Panoramic pictures are not a must, one could even use consumer photography under some contraints, to mass market the technology
Heh. I got my hands on some NRC videos, and one of them was titled HyperLinking Reality.
http://www.youtube.com/watch?v=G6L7q8FTRB8