All of the components for creating fully immersive virtual worlds have suddenly become ubiquitous and cheap, often built into devices that we have already in our pockets and on our desktops. These devices have everything they need to become state-of-the-art platforms for immersive games and virtual reality: powerful graphics, high-resolution displays, and precision sensors. Just add some optics, and now a responsive, fully immersive virtual reality platform can be built for next to nothing. And once we can do that, there's nothing to stop us from unleashing a flood of alternate and augmented worlds that can be colocated with our physical surroundings, anywhere and any time. More than just information or annotation, we can begin to imagine a multiplicity of inhabitable, immersive, interactive, networked environments that can be coordinated with our everyday lives. The coming proliferation of virtual and augmented worlds will make manifest the idea that there is far more to reality than we are normally aware of, and that there are countless virtual realms that can now be brought into conscious experience.

For most researchers outside of the field, Human Computer Interaction (or HCI) is the study and evaluation of interactive systems and techniques. While this is an important part of our discipline, nowadays HCI is as much about _building_ the underlying technologies and systems as it is studying their use. In this talk I will demonstrate why it is an exciting time to be a computer science researcher in this discipline. You can play with the newest technologies, such as exotic cameras, displays and sensing hardware; readily embrace approaches outside of your discipline (e.g. within computer vision, machine learning, signal processing, or computer graphics); and even invent technologies and algorithms along the way. However, ultimately, you'll build working systems that are grounded by real-world problems that have direct impact on users.

The Interactive 3D Technologies Group at Microsoft Research Cambridge, UK, has been born out of this new approach to HCI research. I will demonstrate examples of projects within the group, which are all motivated by pushing the boundaries of how people can interact with computers, but doing this through technical and systems innovation. These projects move computing beyond the mouse and keyboard into the physical world around us. A key part in this work is blurring the lines between the digital and physical worlds. The group is multi-discipline in nature, allowing us to not only to embrace but invent technologies and techniques from fields such as optics, computer vision, robotics and computer graphics. The talk will also hint at exciting new technologies around the corner – some of which our group is working on – which will be critical building blocks for like-minded researchers in the future.

Clearly there are many parallels between being an HCI researcher and an AR researcher. We both celebrate building working systems, embrace new technologies, focus on real-time techniques, and systems that have direct user impact. My talk will argue that by bridging the gap between disciplines such as AR and HCI, we can solve grander challenges, such as making augmented reality a reality, and moving closer to one day building the HoloDeck (note: being a Star-Trek fan is not a requisite for this talk!).

This will be an interactive session, with demos, videos, and hopefully not too many explosions (unless one of the GPUs in my ridiculously big laptop overheats).

As technology continually advances our ability to merge real and virtual worlds, how will we reconcile new forms of interaction with the imperatives of art and literature? Perhaps we need a common place for our old and new virtual realities to enhance each other.

We will look at some new ways of merging divergent paradigms into a coherent experience, from Jane Austen's "Pride and Prejudice" to collaborative live programming to Vernor Vinge's vision of a digitally transformed world where everybody is "wearing".

Higher frame-rates promise better tracking of rapid motion, but advanced real-time vision systems rarely exceed the standard 10ñ 60Hz range, arguing that the computation required would be too great.

Actually, increasing frame-rate is mitigated by reduced computational cost per frame in trackers which take advantage of prediction. Addition- ally, when we consider the physics of image formation, high frame-rate implies that the upper bound on shutter time is reduced, leading to less motion blur but more noise. So, putting these factors together, how are application-dependent performance requirements of accuracy, robustness and computational cost optimised as frame-rate varies? Using 3D camera tracking as our test problem, and analysing a fundamental dense whole image alignment approach, we open up a route to a systematic investigation via the careful synthesis of photorealistic video using ray-tracing of a detailed 3D scene, experimentally obtained photometric response and noise models, and rapid camera motions. Our multi-frame-rate, multi-resolution, multi-light-level dataset is based on tens of thousands of hours of CPU rendering time. Our experiments lead to quantitative conclusions about frame-rate selection and highlight the crucial role of full consideration of physical image formation in pushing tracking performance.

In recent years photo and video sharing web sites like Flickr and Youtube have become increasingly popular. Nowadays, every day millions of photos are uploaded. These photos survey the world. Given the scale of data we are facing significant challenges to process them within a short time frame given limited resources. In my talk I will present our work on the highly efficient organization and reconstruction of 3D models from city scale photo collections (millions of images per city) on a single PC in the span of a day. The approaches address a variety of the current challenges to achieve a concurrent 3D model from these data. For registration and reconstruction from photo collections these challenges are: selecting the data of interest from the noisy datasets, and efficient robust camera motion estimation. Shared challenges of photo collection based 3D modeling and 3D reconstruction from video are: high performance stereo estimation from multiple views, as well as image based location recognition for topology detection. In the talk I will discuss the details of our image organization method, our efficient stereo fusion technique for determining the scene depths from photo collection images.