The trend of moving from mobile computing to spatial computing

Talal A G Shaikh, Director of Undergraduate Studies and Programme Director, Software Engineering, School of Mathematical and Computer Sciences, Heriot-Watt University Dubai.

Augmented Reality is the use of computer technology to overlay digital objects such as images, 3D models, videos, and animations on top of a physical, real-world environment which helps creates rich, interactive experiences. In simple words, augmented reality enhances the real world with digital objects. Augmented reality can be displayed on various hardware devices such as screens, mobile phones, head-mounted displays and smart glasses which include spectacles by Snapchat and Google Glass. Augmented reality utilises technologies like SLAM, simultaneous localisation and mapping and depth tracking, a sensor data calculating the distance to the objects.

Mixed Reality is a combination of virtual reality and augmented reality, which helps create an environment where both physical and virtual elements can exist and interact in real-time. The digital content in mixed reality is spatially aware and responsive. It works by adding virtual digital content to the real world or by adding elements of the real world into the virtual digital one. 

Mixed reality depends on an interaction between headset hardware, mixed reality software and digital content on cloud-based servers. Popular mixed reality hardware devices include Magic Leap, Meta 2 and Microsoft HoloLens. 

With the constant evolution of immersive technologies, augmented reality and mixed reality-endowed products are already making great strides in wide array of industries such as construction, oil and gas, defence, automotive, education, healthcare, gaming, live events, and retail. 

One of the greatest examples of augmented reality applications has to be Pokémon GO – the game that took the world by storm in 2017 as it allows players to catch their favorite Pokémon characters by looking through their smartphones at the real world, but with superimposed visuals. 

In the retail world, IKEA set the precedent with its augmented reality-based app which allows shoppers to visualise how each item of furniture will look like once they place the virtual version onto their living spaces. The goal of IKEA’s augmented reality-aided experience is to help customers make informed purchasing decisions that make them feel happier and more satisfied – an ingenious customer loyalty masterstroke. 

Hospitals and other healthcare facilities are recruiting augmented reality diagnostic tools to enable healthcare providers visually communicate new or complex treatments to patients. Augmented reality can also show how new drugs and medical devices interact with the body, and is going to become more indispensable to surgeons and critical healthcare providers in the years to come. 

As mixed reality, becomes more accessible, the technology is seeing more applications in industries such as education, automotive and construction. For instance, car manufacturer Ford is using Microsoft HoloLens mixed reality headset to prototype vehicles which allows for rapid prototyping in a virtual environment, instead of having to create new physical models to see what new materials and angles will look like on a final design, therefore reducing the time and cost of iteration. 

Mixed reality technologies are being used in the education sector to enhance the student’s ability to learn. Using 3D Projections and simulations, medical students can interact and alter the virtual objects in order to study them in the most suitable manner.

The advent and the rise of mixed reality- and augmented reality-based innovations in the last decade have truly brought some of the most ground-breaking applications to the forefront that has positively impacted the lives of many around the world. 

The fundamental problem with the augmented reality mixed reality headset concept is that most people are only willing to wear something on their face that is about the size and weight of sunglasses or regular prescription glasses. 

As consumer devices such as smartphones and smartglasses become more compatible with augmented reality, it will naturally increase the user adoption and guarantee higher engagement, in turn enabling developers to reach a bigger audience. 

As hardware devices become more accessible, the onus then lies on the developer community to make augmented reality and mixed reality apps that are effortless to use. Whether it is a smartphone or a headset, the user’s device is their window to an immersive experience, and so it is important to ensure that the user is able to interact with the fabricated elements with ease and as intuitively as possible.

The best augmented reality apps deliver practical value to their users. Let us take the example of IKEA again; the augmented reality elements do not embody the entire app, but rather support the main goal of the app, which is to help customers buy furniture and décor by looking at 3D renderings of said items superimposed onto their homes. 

Several studies have proven that people prefer spending more time with their screens with increasing leisure time. The more time users spend on an augmented reality or mixed reality app, the more data there is about their interactions, which makes personalised offerings more possible and therefore helps such apps to increase and improve their user engagement.

Leading tech players such as Apple, Google, Facebook and Snapchat are already pushing the envelope with augmented reality and mixed reality. It is only a matter of time before augmented reality and mixed reality becomes more mainstream and accessible beyond the current niches. 

Although majority of the apps are currently focused on games, one can already see increasing applications in niches such as retail and shopping, utility and entertainment.

A major constraint developers face is the constant evolution of such SDKs, which sometimes makes it difficult for them to keep up with the rapid developments. As the world moves from a mobile computing to a spatial computing era, this transformation comes with a great learning curve. As a developer, this is the time to get on board to learn and build frameworks that will serve as the foundation for the next generation to come. 

As discussed earlier, the increasing consumption of advanced smartphones is an incentive for developers to build augmented reality apps that will reach a wider audience. Moreover, the advent of 5G will make augmented reality and mixed reality more real, and will help them go mainstream. In order to get truly life-like image quality, these augmented reality mixed reality viewers and companion smartphones also rely on a cloud computing connection. 

Talal A G Shaikh, Director of Undergraduate Studies and Programme Director, Software Engineering, School of Mathematical and Computer Sciences, Heriot-Watt University Dubai.
Talal A G Shaikh, Director of Undergraduate Studies and Programme Director, Software Engineering, School of Mathematical and Computer Sciences, Heriot-Watt University Dubai.

Key takeaways 

  • The digital content in mixed reality is spatially aware and responsive.
  • Mixed reality depends on interaction between headset hardware, mixed reality software and digital content on cloud-based servers. 
  • Popular mixed reality hardware devices include Magic Leap, Meta 2 and Microsoft HoloLens. 
  • Augmented reality can be displayed on hardware devices such as screens, mobile phones, head-mounted displays.
  • One of the greatest examples of augmented reality applications has to be Pokémon GO.
  • The problem with headset concept is most people are only willing to wear something like sunglasses.
  • Healthcare facilities are recruiting augmented reality tools to visually communicate complex treatments to patients.