x
Imagine you’re in an office with colleagues,
brainstorming about the next conference you’re hosting. “How about Vancouver?” you ask. Your voice activates, as if by magic, a transparent browser window on a wall, listing the results of a search on “Vancouver.” “I don’t think so,” says your colleague from across the table. She calls up another search and more windows appear in the air, this time with information on London venues. She reaches out and picks one with a pinching motion, then flicks it down onto the table in front of you. Your 3D holographic avatars?—?because you’re working from different cities?—?look at each other and nod in agreement.
This is Spatial?—?a software platform that lets collaborators beam in from anywhere in the world to collaborate in a single 3D augmented reality space. Think Hangouts with all participants physically free to move around, while playing with the cool Minority Report gesture-based interface?—?except it, too, has escaped the 2D screen.
Created by TED Fellow and interface designer Jinha Lee and his colleague (and TED speaker) Anand Agarawala, Spatial runs on AR headsets to let users collaborate, search, brainstorm and share content in a virtual AR space. While the technology is now being piloted as an enterprise tool by the likes of Ford Motor Company, Lee reckons that the rest of us will be hanging out together in virtual 3D spaces within a few years. We asked him to tell us more.
What does Spatial do?
Spatial transforms any space you have into a 3D collaborative virtual workspace, allowing people who are to join from anywhere. Participants put on AR glasses like Microsoft HoloLens, then run Spatial to invite remote participants. Everyone is represented to each other in the AR space as a 3D avatar, and works together as if they’re sitting with each other, face to face.
Once participants have teleported into Spatial, they can use the entire 3D space around them to visualize and immerse themselves in any topic. You can pull up objects and, grabbing them with a pinching gesture, throw them out anywhere into the “room”?—?whether they’re documents, photos, browser windows, 3D models. You can write a sticky note, for example, and send it out from your phone onto a wall or a desk. It can even just hang in the air. Participants can move freely around these virtual objects, which stay put in the virtual space. Everyone in the session can see the process as it happens, and can then interact with your objects as they wish. All this lets you brainstorm in a group setting very quickly. The content is saved so that it can be reloaded later. Some might call it Slack for 3D space.
Can you give an example of how it might work?
Spatial allows people working in any industry or creative endeavor to quickly externalize thoughts, making them into objects that other people can see and interact with. Let’s say I want to host a discussion of a new office design. My collaborator, who’s joining me as a 3D avatar, shares his sketches from his laptop, and organizes them on the wall we’ve mapped out in our virtual space?—?a common point of reference. I can lift up my fingers and say, “Boutique design.” As the words pop out of my mouth, inspirational images and 3D models of boutique interior designs appear before our eyes. Other participants can leave sticky notes on selected images, or grab those images and move them into piles, or even toss them across the room to you. Biomedical companies can invite experts from all around the globe to a room filled with patients’ medical charts and X-rays, for example. Artists can mock up an exhibition; filmmakers can storyboard. Engineers can review and simulate their data models together, and so on.
Say I’m here in Cambridge, and you’re in New York. If I host the call, does that mean you will see my space in Cambridge?
I won’t actually see your real-life space. You’d scan a big wall or table as a point of reference, and then when I teleport into your AR room, I’d see your virtual wall, and it’d be matched to mine, so we’d be looking at the same “wall.” It doesn’t have to be wall-centric, though. Tables or even the full 3D VR space can be used as a point of reference. We envision that in the future, we will be able to see each other’s real-life spaces, too, as the hardware evolves.Above, a demonstration of how Spatial works and what it looks like from within the augmented reality program.
We already meet virtually using video chat apps like Hangouts, Skype and even Facebook Messenger. Why not just keep using these tools for long-distance communication?
Those applications are fine for one-to-one exchange of information, but they’re still not great for creating information interactively, especially in groups. For instance, two people can get on a Skype chat and simultaneously work on a Google doc in another window, but this breaks down when there are more people, or if there’s more complex work to do than taking notes or editing a document.
Meanwhile, a larger group can meet virtually using Hangouts, but those conversations only allow one person at a time to dominate the conversation, while the others sit passively. Real interactivity is very limited in either scenario. Spatial creates a virtual environment that allows for more active group interaction.Participants who don’t have AR headsets can still participate in Spatial meetings, represented in the AR space as a floating webcam window.
How does Spatial generate such realistic-looking 3D avatars?
Spatial automatically generates a holographic version of you. You just connect and link your Facebook, where Spatial picks up a photograph of your face, and generates a static 3D shape from the 2D image using machine-learning APIs. Then it animates the avatar with realistic movement based on your gestures, eye blinks and voice. Combined with Spatial Audio, this gives a compelling presence of each user.
Does Spatial work if you don’t have AR goggles?
Yes. We have a web app, so even if you don’t have a headset, you can still join in with your computer or phone browser. What you get is kind of like a first-person-shooter, 3D-game style scene on your browser, with little people walking and talking in your window. From the perspective of the other participants, you’ll be represented as a floating webcam. You can still navigate and upload content. We believe that in the near future, the majority of people will work with AR goggles, but until we get there, we’d like to allow people without headsets to participate in Spatial meetings, too.
Before Spatial, Jinha Lee developed SpaceTop, a 3D computing interface that allowed users to virtually reach inside the screen to type, draw and directly manipulate interfaces that appeared to float in 3D space above the keyboard. Growing up as a child in love with arts and crafts, I enjoyed building and expressing my ideas using my hands. When I first learned to use the computer, I remember feeling extremely frustrated that 2D screens and mouse cursors?—?essentially a small dot?—?were all I could use to navigate. I thought there was a far more potential in human-computer symbiosis, which can only be realized by increasing the communication bandwidth between the two.
During my time at MIT and Microsoft, I began to develop 3D interfaces like SpaceTop using transparent screens and Kinect cameras. Then, at Samsung, I created MediaSquare, a user interface that allows multiple people to easily toss content onto big screens with their individual devices, turning TVs into a social hub for the living room. Throughout these projects, I was essentially trying to address the question “What would people would do when computing is liberated from screens and lives in 3D space surrounding us?” However, I emulated such immersive environments using the screen-based technologies that happened to be available at the time.
One of Jinha Lee’s former projects, MediaSquare, allowed multiple users to share and manipulate content simultaneously on a single big screen via their personal devices. This represented a step towards his vision of collective screen-free computing but was still confined to a screen.Then, when I saw the HoloLens in 2015, which was then still very new, I realized that fully immersive 3D computing environments might not be so far away after all. Along the way, I met Spatial’s co-founder Anand Agarawala. Anand has been a pioneer in the user-interface space, having created BumpTop?—?a multitouch and physics-based 3D user interface. We shared our vision that computing should live in 3D space, decided that now would be a good time to bring this vision into reality. We founded Spatial at the end of 2016.
Do you think this is the future of computing?
Yes. I believe that phones and computers will be replaced with 3D AR user interfaces in the near future. Eventually this will happen on the street, but we’ll see this transition first at the workplace. That’s why we’re focusing on enterprise applications to start with. Companies in a variety of industries?—?manufacturing, biomedical, design, e-commerce?—?are currently piloting Spatial as a solution for connecting their global workforce.
It will be interesting to see what happens when AR computing becomes the norm, freeing technology from screens. In the past ten years, computing has evolved to become increasingly personal and intimate: we actually carry individual worlds in our pockets. It’s been very successful, but I think there’s big potential for connecting multiple minds from different spaces and allowing them to collaborate seamlessly and fluently. Essentially, Spatial is designing a transition from the era of personal computing to collective computing.
Article Produced By
Karen Frances
organic unidirectional time machine
