A few months back, I heard something about Photosynth – a new method of modelling scenes using photographic images to build up a 3D representation and yesterday I got the chance to have a look at it myself. At first, I just didn’t get it, but after having seen a few synths, I now think that this is really cool technology with a lot of potential for real world applications.

It’s difficult to describe Photosynth but it’s essentially a collage of two dimensional images used together to create a larger canvas through which it’s possible to navigate in three dimensions (four actually). It began life as a research project on “photo tourism” at the University of Washington, after which Microsoft Research and Windows Live Labs took it on to produce Photosynth, using technology gained with Microsoft’s acquisition of Seadragon Software and the first live version of Photosynth was launched yesterday.

Clearly this is not a straightforward application – it’s taken two years of development with an average team size of 10 people just to bring the original research project to the stage it’s at today – so I’ll quote the Photosynth website for a description of how it works:

“Photosynth is a potent mixture of two independent breakthroughs: the ability to reconstruct the scene or object from a bunch of flat photographs, and the technology to bring that experience to virtually anyone over the Internet.

Using techniques from the field of computer vision, Photosynth examines images for similarities to each other and uses that information to estimate the shape of the subject and the vantage point the photos were taken from. With this information, we recreate the space and use it as a canvas to display and navigate through the photos.

Providing that experience requires viewing a LOT of data though—much more than you generally get at any one time by surfing someone’s photo album on the web. That’s where our Seadragonâ„¢ technology comes in: delivering just the pixels you need, exactly when you need them. It allows you to browse through dozens of 5, 10, or 100(!) mega-pixel photos effortlessly, without fiddling with a bunch of thumbnails and waiting around for everything to load.”

I decided to try Photosynth out for myself and the first thing I found was that I needed to install some software. On my Windows computer it installed a local application to create synths and an ActiveX control to view them. Creating a synth from 18 sample images of my home office desk took just a few minutes (each of the images I supplied was a 6.1 mega-pixel JPG taken on my Nikon D70) and I was also able to provide copyright/Creative Commons licensing information for the images in the synth:

Once it had uploaded to the Photosynth site, I could add a description, view other people’s comments, get the links to e-mail/embed the synth, and provide location information. I have to say that I am truly amazed how well it worked. Navigate around to the webcam above my laptop and see how you can go around it and see the magnet on the board behind!

It’s worth pointing out that I have not read the Photosynth Photography Guide yet – this was just a set of test photos looking at different things on and around the desk. If you view the image in grid view you can see that there are three images it didn’t know what to do with – I suspect that if I had supplied more images around those areas then they could have worked just fine.

You may also notice a lack of the usual office artifacts (family photos) etc. – they were removed before I created the synth, for privacy reasons, at the request of one of my family members.

My desk might is not the best example of this technology, so here’s another synth that is pretty cool:

In this synth, called Climbing Aegialis (by J.P.Peter) you can see a climber making his way up the rock face – not just in three dimensions – but in four. Using the . and , keys it’s possible to navigate through the images according to the order in which they were taken.

Potting Shed is another good example – taken by Rick Szeliski, a member of the team that put this product together:

Hover over the image to see a doughnut-shaped ring called a halo and click this to navigate around the image in 3D. If you use the normal navigation controls (including in/out with the mouse scrollwheel) it is possible to go through the door and enter the potting shed for a look inside!

There are also some tiny pixel-sized pin-pricks visible as you navigate around the image. These are the points that were identified whilst the 3D matching algorithm was running. They can be toggled on an off with the p key and in this example they are so dense in places that the image can actually be made out from just the pixel cloud.

Now that the first release of Photosynth is up and running, the development team will transition from Windows Live Labs into Microsoft’s MSN business unit where they will work on using the technology for real and integrating it with other services – like Virtual Earth, where synths could be displayed to illustrate a particular point on a map. Aside from photo tourism, other potential applications for the technology include real estate, art and science – anywhere where visualising an item in three or four dimensions could be of use.

The current version of Photosynth is available without charge to anyone with a Windows LiveID and the service includes 20GB of space for images. The synths themselves can take up quite a bit of space and, at least in this first version of the software, all synths are uploaded (a broadband Internet connection will be required). It’s also worth noting that all synths are public so photos will be visible to everyone on the Internet.

If you couldn’t see the synths I embedded in this post, then you need to install an ActiveX control (Internet Explorer) or plugin (Firefox). Direct3D support is also required so Photosynth is only available for Windows (XP or later) at the moment but I’m told that a Mac version is on the way – even Microsoft appreciates that many of the people who will be interested in this technology use a Mac. On the hardware side an integrated graphics card is fine but the number of images in a synth will be limited by the amount of available RAM.

Finally, I wanted to write this post yesterday but, following the launch, the Photosynth website went into meltdown – or as Microsoft described it “The Photosynth site is a little overwhelmed just now” – clearly there is a lot of interest in this technology. For more news on the development of Photosynth, check out the Photosynth blog.