Microsoft Mesh aims to bring holographic virtual collaboration to everyone

Last week, I sat around a table with fellow journalists as Greg Sullivan, head of mixed reality at Microsoft, detailed the company’s vision for the future of virtual collaboration. No one wore a mask or stood aside. We were not afraid of getting sick. Instead, we all wore HoloLens 2 headsets and sat in different parts of the world. The holographic table was right next to my desk, and my media friends floated around my desk as we chatted with our cartoonish avatars. For a second, I felt like I was mingling with real life during The Before Times.

We were experimenting with one of the first apps powered by Microsoft Mesh, the company’s ambitious new attempt to unify holographic virtual collaboration across multiple devices, be it VR headsets, AR (like HoloLens), laptops or smartphones. Powered by Microsoft’s Azure cloud, Mesh isn’t just an app, it’s a platform other developers can use to collaborate remotely on their own software. If remote work is here to stay – and by most accounts are – Microsoft wants to be the company that takes us beyond Zoom video chats, to holographic experiences that anyone can participate in.

“Not only will we be able to share holograms, but we’ll be able to do so in a way that gives us action and presence,” Sullivan said in our virtual meeting. “We can create those experiences, where even though we are physically separate, we feel like we are in the same room, sharing an experience and collaborating on a project.”

While we saw a solid blow to virtual collaboration of Spatial, Microsoft is trying something even more complex. Sullivan compares Mesh to the launch of Xbox Live in 2002, a service that dramatically simplified online multiplayer gaming for consoles. This has made it easier for developers to connect their games to the Internet and has led to a boom in online multiplayer titles for the Xbox and Xbox 360. This has given Microsoft a head start over Sony and Nintendo, who all have two took years to catch up. .

Microsoft is using the opening keynote from today’s Ignite conference to showcase the capabilities of Mesh. Alex Kipman, the company technician behind HoloLens and Kinect, will take the stage as a real-time hologram (what Microsoft calls “holoportation”). Think of it a bit like the holographic messages we saw in Star wars and other science fiction stories. It’s not photo realistic, but if you’re wearing a VR headset, it’s almost like he’s in the room with you. On a standard monitor or phone screen, this may just appear as a hokey special effect. But it’s not hard to imagine possibly slipping on an AR headset like the HoloLens 2 and watching a hologram presentation right in your living room, as if you were sitting in the front row during a show.

My Microsoft Mesh demo, to be clear, was nowhere near as impressive. Our avatars were simplistic, with detached arms and limited facial movements. It was like being surrounded by a group of Nintendo Miis. But there was still a decent sense of immersion: I could tell exactly where everyone was even when my eyes were closed, thanks to realistic audio processing. And we were able to collaborate with 3D models, pass them around the table and resize them as we liked.

As we looked at some pretty basic 3D figures, Sullivan pointed out that Mesh can serve high-quality models from the cloud as well (it’s powered by Azure, after all). This would allow designers and engineers to collaborate with the same resources they use on their workstations from anywhere in the world. This is what director and producer James Cameron is aiming for his next series, OceanXplorers. The nonprofit behind this show, OceanX, plans to create a Mesh-compatible “holographic lab” on its advanced vessel, allowing scientists on board and remotely to collaborate around 3D models.

“The idea is to take all this amazing scientific data that we collect and put it in a holographic frame and use it as a way to guide science missions in real time,” said Vincent Pieribone, vice president of OceanX, in a statement. This would allow researchers to huddle around the data and chat like they would in real life, regardless of the distance between them.