How smart glass and augmented reality are helping law enforcement and safety

NNTC is a UAE based company, working in the field of emerging technologies. NNTC is significantly focused on computer vision for different use cases like public safety and industrial safety. The vendor has developed a unique vision solution designed for law enforcement on patrol and is integrated with smart glasses. An officer just wears the glasses and the built-in camera looks at people. If there is match with the database of suspects, it gives an alert and law enforcement can take action about it.

Says Dmitry Doshaniy, Head of NNTC, “This is a normal 2D technology, which works with a video stream created by a generic camera. The software looks for faces in the video stream. As soon as it identifies the face, it is converted to a number like a hash number which is compared to similar numbers stored in the database.” The user can have various lists: white list, black list of wanted people, missing people, access denied, access allowed, and if there is a match with one of these lists, then an alert is displayed in the smart glasses, as well as in the control center.

The wireless device and the wearable server are part of the solution. The glasses have Wi-fi, which is connected to the mobile wearable server. The database and most of the application software is running on it. The video stream from the glasses is processed, the faces are cut out, and then matched to the database. And then if there is a match the result is sent back and displayed in the built-in screen of the glasses

The camera that is built inside the glasses generates a video stream, which is sent for processing. The software looks at each individual frame and checks if there are objects like human faces in the frame. If there are objects, it picks them up and saves them as pictures. The pictures are converted into numbers and these numbers are compared to other numeric representations of people from the database of the watch lists.

According to NNTC’s Doshaniy, “This particular part is not very complex because we use standard protocols.” The glasses are connected through Wi-fi to the wearable server. There is an application running here, using the Android operating system in the Vuzix glasses. This is an application built by NNTC, that serves as a user interface to see alerts in the application allowing the end user to take actions.

Doshaniy points to other challenges. “I would say the biggest challenge is packaging the solution in a wearable form factor.”

The recognition has to happen instantly, so NNTC cannot rely on the 3G-4G networks. It is for this reason that the complete processing and access to the database sits locally on the officer’s belt, continuously finding the right components. NNTC did also have to involve Intel in finding the solution. “It was challenging,” says Doshaniy.

The end to end solution provided by NNTC uses multiple vendor partners. While the smart glasses are provided by Vuzix, the algorithm for number crunching used in facial recognition is from VisionLabs. The mobile server is a custom-built product from Intel, while the application and database are proprietary from NNTC. The range of the glasses is the same as any Wi-Fi network. The server can be mounted on a patrol bike or inside a vehicle, while the camera is mounted as a body cam or as wearable glasses.

In urban locations, there are tens of thousands of cameras at work collecting data. While there can be a large number of cameras available, getting real-time, onsite information from them, on-the fly, is often challenging for law enforcement. However, if facial recognition and alerts are not enabled, then law and order will not receive any benefits from this large-scale investment. “If the city does not have advanced technology, there is still a crime problem. What do you do? Do you wait for ten years? We offer a solution to this problem. You equip the mobile patrol officers with wearable facial recognition technology,” explains Doshaniy.

In its regional go-to-market activities, NNTC relies on local partners who have competency in surveillance and physical security. “Naturally law enforcement and public safety agencies would not buy from everyone. So, they need a trusted partner and good technology. We do the technology part but we also need somebody who is capable to deliver,” points out Doshaniy. Channel partners would also add value to the solution from NNTC and do the actual implementation.

With regard to the legal restrictions around the use of facial recognition using smart glasses, Doshaniy explains that it all depends on the use case of the solution, in the particular country. Other than the technology, there are other issues around the data being generated and data being accessed using the mobile server. By default, the localisation of data, transmission of data, and sharing of data follow the GDPR regulations.

According to Joachim Aertebjerg, Director Technical Solution Sales EMEA, Intel, the vendor is supporting NNTC in its project to develop solutions for industrial safety and crowd control. It is based on a visual image that is projected inside the smart glass and uses computation from a variable PC using Intel processors with new memory technologies.

One such new technology, persistent memory, allows faster data processing, can manage large volumes of data and works well for databases, typically those containing look up images for face recognition. Persistent memory from Intel is available in the data center and it is increasingly coming in laptops and will also find its way into variable computers. While Intel is predominantly a hardware vendor, its software tools are made available to the ecosystem including partners like NNTC, so that they can build their applications on top of the latest hardware.

“Applications are becoming increasingly abstracted from the hardware, and programming language are also at a higher level than they were in the past. The more optimisation you get for the entire solution stack down to the hardware the better the performance. In a variable solution where you have power constraints and therefore also computation hardware constraints, you do not want headset to be too clunky. You really need smart programming and that is where local software development comes into play,” says Intel’s Aertebjerg.

iFalcon Face Control Mobile

This is an autonomous face recognition system, that uses artificial intelligence, and is integrated with wearable smart glasses. The bodycam or smart glasses screen crowds to match faces against a database of violators, missing people or suspects. Once there is a match, the solution retrieves relevant information from the database and instantly sends an alert to the smart glasses. iFalcon Face Control Mobile brings face recognition capabilities, even without the network of CCTV cameras that cover every inch of a city.

iFalcon runs on an algorithm, which is in the top three, according to global ratings such as the Face Recognition Vendor Test compiled by the US National Institute of Standards and Technology and Labelled Faces in the Wild. The smart glasses or bodycam detect faces with a built-in camera and transmits information to a wearable portable computer over wireless channel.

Key takeaways

• Applications are becoming increasingly abstracted from the hardware.
• Programming language are at a higher level than they were in the past.
• The more optimisation you get for the solution down to hardware, better the performance.
• Where you have power and computation constraints, you do not want the headset to be too clunky.
• You need smart programming and that is where local software development comes into play.
• Law enforcement and public safety agencies would not buy from everyone.
• The server can be mounted inside a vehicle, while the camera is mounted as a body cam.