In early 2018, my team at Microsoft was tasked with exploring how enterprise software could leverage immersive, new technologies like mixed reality.
With VR gaming and entertainment on the rise, we wanted to figure out more practical use cases for this platform.
In our industry, we often hear the phrase "data is everywhere". With Power BI and Dynamics 365 on HoloLens we can make this statement a (virtual) reality.
As a primer, the HoloLens is a head-mounted display (HMD) that holds an array of sensors, optics, and processors to seamlessly overlay digital images onto the real world. We call this environment, mixed reality.
A key affordance of the headset is voice and hand gestures as the primary input, which releases you from a traditional mouse and keyboard at your desk.
This is ideal for first line task workers and service engineers, who typically do precision work with tools in their hands or are mobile deployed into the field.
From our research we discovered a steep learning curve for frontline workers entering new jobs.
Frontline workers are people who do the actual production, sales and repair work for many companies.
These jobs are procedural tasks and require a human touch, but with an aging, retiring workforce much of that skill is going away.
For new employees, there's only so much training you can get from, practice modules and shadowing someone behind their back.
We have all this information locked up in silos. Things like live readouts from machinery, process knowledge from field experts, and serial numbers for parts stuck in a book.
We needed to come up with a solution to get all this data in front of the eyes of frontline workers and allow them their jobs in a more seamless way.
minimum viable product:
Our earliest version of this concept was simply shipping Power BI for HoloLens.
The list of tasks is pretty basic:
1) Pinning dashboards and reports to surfaces
2) QR code scanning to quickly access of data
3) Docking content for easy mobility
4) Voice commands for hands-free interaction
This software coupled with some QR stickers allowed a factory to quickly get up and running with live data on location.
A stepping stone
Power BI for HoloLens is a simple consumption experience, with data going one way to the user.
With the camera, sensors, Azure cloud computing and the rest of the Microsoft stack, we can do more than teaching.
Object recognition enables quality assurance and validation that the tasks are done correctly.
Technicians out in the field can get live help from remote experts for hard tasks.
My team and I worked on these and various other solutions in this space. Due to the sensitive nature of the work, I'm unable to show them here.