Human Life In Artificial Intelligence World
Cameras are watching us almost continuously, but there’s little point in this. If a person does not analyze the picture, the camera remains just a device that produces terabytes of the unsuitable stream of hours. The alternative is to provide the camera with AI tools. And such a video surveillance system will be able to replace the security guard sleeping in front of the monitor, the boss in the office and the marketer in the supermarket. We tell exactly how.
A “smart” camera is a conditional concept, and in most cases, the mind itself is not embedded in the camera itself but installed on a server, where the video stream from the camera is analyzed using artificial intelligence technology. The camera does not have the computing power to perform complex image analysis. Hereinafter, we will talk about such gadgets, so the “smart” camera is only the “eye” of a truly smart computer, which accounts for all the intellectual workload.
Scout room: the useful “Big Brother” at home
In 2014, Google announced the Nest Cam, a small camera designed to monitor home security. She knew how to recognize people’s faces, see in the dark, hear voices, and also convey her impressions of what is happening in the mobile application of the hosts. With the advent of Google’s voice control system in 2016, Google’s opportunity to integrate a smart camera into the ecosystem of other smart gadgets from the Good Corporation.
And at the turn of 2017 and 2018. small startups say Lighthouse, created specifically for the development and promotion of smart cameras, which are now several dozen, have entered the market. How do these devices work?
A smart camera creates a dynamic 3D model of the room and all the objects in it. Any change in the situation (movement, movement, the appearance of a new object) is recorded and reflected in the model. Machine learning allows the camera to classify both objects and their actions. The longer the camera runs, the better she understands who is in front of her, recognizes her and others, learns to distinguish between a child and, say, a dog. Special attention to people – the camera recognizes faces, creates a base of visitors with their photos in high resolution and from different angles, compares them and numbers them. Sometimes the camera asks the owner through the application to clarify the status of a person, showing a photo of people embarrassing her. The camera analyzes what is happening non-stop, transfers data to the cloud service and to the client application.
Such cameras, both in the “smart” home and outside it, make it possible to implement a number of new features and scenarios. If your pet dog woke up before you and began to wander around the house, the smart camera will not wake you up with good morning wishes, because it distinguishes you from the animal. If the wind shook a tree outside the window, the camera does not call the police, believing that the burglar is breaking, because the cars or natural objects are also identified by it. The usual motion sensor could easily react to such events, causing unnecessary trouble.
If you are not at home, and a courier must come to you, for example, you can remotely open the door for him, follow his movement with the help of Nest, tell through the speaker were to leave the parcel, and then politely say goodbye. Source: Google Nest YouTube Channel
But the “smart” camera pays attention to the new neighbor who has come to borrow salt. Seeing the person for the first time, she remembers his face, and also sends a notification of a stranger to the dwelling. Of course, if the neighbor in your absence borrowed salt or more, the camera would give an alarm. If a neighbor later swears to her mother that she was not going to steal anything, the camera will still remember her, like all other guests who have distinguished themselves with unusual behavior – especially advanced gadgets distinguish between normal and abnormal housing situations.
After returning home from work, you wanted to play the Xbox, but you discover that there is no prefix in place. Again, the neighbor? Courier? “No,” the camera will say. “This is your beloved mother, during yesterday’s visit, put the gadget in the closet.” The camera can report the disappearance of the object from the familiar environment. Leaving the house, you ask the camera to notify about the visit of the mother in your absence. As soon as this happens, you will receive a message and call your mother asking you not to touch the Xbox anymore.
The above-described functionality is already the real capabilities of smart cameras such as Nest, Wyze Cam, Arlo, Simplisafe, etc. In 2018, the market for such devices reached $ 7 billion, according to analyst firm Strategy Analytics. By 2023, this figure could grow to $ 9.7 billion, and in “pieces” their sales could increase from the current 57 million to 120 million units over the same period.
Everything will be fine, wherever you go: smart cameras on the streets
Back in the 1980s, a camera that distinguished license plates helped to find a stolen car for the first time, but only in the 2010s, smart cameras learned to analyze not one, but thousands of characteristics of observed objects. So, in August 2017, the Chinese search engine Baidu announced that he was able to recognize various actions of people – from walking with a dog and washing windows to cutting down trees, etc. – 300 thousand videos with an accuracy of 88%.
- However, home “smart” cameras are able to do this, but they observe a stationary situation, while on the street people constantly move, and take their different lenses. How to collect from this a single picture for analysis?
Toshiba developed SATLYS technology. It identifies a specific person who has fallen under the lenses of different cameras. For this, artificial intelligence identifies a small number of distinctive features of an individual and compares them with signs of other passersby who could get into the stream of neighboring cameras.
The computer does not take into account all the characteristic features in order not to overload itself with comparisons of millions of features of appearance but chooses one or several essential ones. That is what he is looking for on other records. Moreover, the system is able to search for a person based on an external feature that can be entered into the system, like in the Google line, say, “red backpack”, “white dress”, “girl”, etc.
The main difficulty is that different cameras give a different image of a person due to the mismatch of the quality of shooting, viewing angle, lighting location, etc. Toshiba has created a technology that is able to highlight a specific feature of an individual and look for a match with it in other video broadcasts. Source: Toshiba
So, street “smart” cameras can recognize us, understand what we are doing and where we are going. What does this give? For example, the video surveillance system, of which it is a part) has already transferred information about him to the police, because it knows how to identify rogue people. You smoke a cigarette and throw it on the sidewalk, – you have to pay a fine for this sin because the systems are able to distinguish between offenses, even minor ones.
- You decide to go to the office by city train. At the train station, you are waiting for the train and you see a man who is swaying along the tracks. Maybe he was drunk or he became ill – the video surveillance system, which determines inadequate human behavior in high-risk areas, also reports to the station staff.
In the train, you nicely talked to a beautiful stranger, and parting, they found that your watch evaporated with her. The only thing you remember is a red backpack on the back of a girl. A policeman at the station makes a request to the video system “red backpack”, “girl” and finds all the beautiful strangers with red backpacks, which today shone on the records. You point a finger at yours — the police will look for her.
Hitachi has created a video surveillance system for large public spaces — shopping centers or stadiums — that can track the movement of several people at once and analyze their appearance (hair length, the color of clothes, etc.). Source: CGTN YouTube Channel
From the station to your favorite work some two kilometers, and you take a bike. You drive yourself and still do not know what the Gelendwagen runs under the control of a very busy bearded guy who ignores traffic lights. Smart cameras at the intersection have long noticed you and him, and they sent you and other passers-by (and the police) a warning about a dangerous driver in a mobile application. Traffic lights also hold a red signal for pedestrians, and everything safely escapes trouble, which cannot be said about the driver of the Gelendvagen. And here is your favorite office. But first about the pictures of the beautiful cities of the future and reality.
- In the harsh reality, the Chinese, who are implementing three projects in this area, have achieved the greatest progress in social systems of smart video surveillance. Camera networks under the sinister name “Sharp Eyes” (more than 180 million devices), as well as Sky Net (no, this is not a joke; more than 20 million lenses) and “Safe Cities” (no, this is not irony; more than 2 million cameras) only regularly monitor the inhabitants of the PRC, but they already know how to recognize and search for criminals Moreover, the Chinese Big Brother is even capable of punishing citizens automatically. Switching to the red light entails photographing the offender, recognizing his face and automatically publishing on the electronic board the shame in the city.
But the integration of video surveillance into the urban Internet of things is only evolving – it is more expensive and more difficult. So, in Detroit, one of the streets is equipped with a smart surveillance system associated with traffic lights and a special mobile application for citizens. It identifies unwary pedestrians and notifies them to drivers. It is also able to extend the green signal of the traffic light for the rapidly approaching cyclists.
The one-eyed boss: the pros and cons of smart cameras at work
Cameras in the workplace have long been part of the office space, but in many ways, they remain decorative if the boss does not sit on the other side of the 24/7 lens. However, it can now be replaced by artificial intelligence.
In 2017, Microsoft presented a comprehensive smart video surveillance system for the workspace. Cameras, computers, and peripherals are connected to a remote “intelligent cloud”, which uses AI to analyze what is happening in terms of safety and productivity. In this case, smart cameras work as part of the Internet of Things ecosystem. In addition to analyzing more than 27 million different events in the picture, AI tools in the cloud receive signals from workstations, machines, and other equipment.
How does it feel to work in such an office? The scenario can be like this. You pass to work without any passes because the corporate camera knows you by sight. A colleague comes to meet you: he never greets, puts his mug on your documents and jokes about your mother. The smart camera considers the sour expression of your face and will transmit information about a possible conflict to the HR department. Immersed in the thoughts of a new neighbor, you completely forget to attach a badge with a name and a photo to a prominent place, as required by corporate rules. Pass carelessly to the workplace, not knowing that you already ran into a fine, because the smart camera recorded your violation and transmitted information about it to the head of the department, who, a few minutes later, will remind you of the importance of following the company’s rules manner), because he received a notification on the smartphone about an employee without a badge. But the cameras are also watching him and they know: lately, he has been sleeping too often in front of the monitor off, and he himself forgets where he left his badge, although the “smart” camera later helps him to find it.
A smart boss can be useful not only in the office but also in the hospital: a patient with a sick heart decided to walk around the department. The smart camera captures this. A device measuring its heart rate also transmits information to the “intelligent cloud”. As soon as the patient’s heart begins to tire, the alarm comes to the post, and one of the nurses approaches the patient. Source: Microsoft YouTube Channel
Leaving work, you get a report on your performance during the day: on the basis of video data, the AI compiled activity graphs, as well as a list of mistakes made.
We are far from a comprehensive workflow analysis system in which cameras would play a leading role. While they mostly monitor compliance with safety regulations. For example, in the Australian branch of the international construction company Laing O’Rourke, a system for monitoring and warning about dangerous situations at the construction site has been implemented. As soon as smart cameras see a possible danger to workers, they send threat messages to their smartphones or smart watches.
But the analysis of the productivity of workers is mainly based not on direct observation of them, but on the monitoring of specific work operations. Such data is easier to “shoot” from computers and other working equipment than to observe an employee who has only a hand controlling a mouse for a whole day.
Well, it’s time to move out of the office home, but first, you need to look into the store.
Under the eyes of marketers: why “smart” cameras in the mall
Cameras have long ensured that the thieves didn’t snatch anything from the store. But for this, it is usually enough to have the “natural” intelligence of the guard sitting in front of the monitor during the entire shift. Meanwhile, cameras most often see law-abiding customers, and this video information can be much more useful for retailers.
In particular, it was understood by the American network Walmart, which opened in April 2019 a store equipped with cameras, the image from which is processed by artificial intelligence. Cameras and sensors in the supermarket generate 1.6 TB of information per second, which is processed by the server right in the store.
Video surveillance, in this case, does not imply the identification of the customers’ identity – cameras monitor mainly the actions of customers, as well as the goods on the shelves. In the dark corners (say, in the depth of the shelving) sensors are located that help the system to understand whether the goods are in place or not. In addition, the cameras can recognize the gender, age, and type of client’s figure and display digital signage for him on the screens, by which he passes, special offers and advertising corresponding to these characteristics.
What could be shopping under the camera? Excuse me: only you entered the store, and the “smart” cameras have already determined the sex, age, and even assumed idle status. You are heading to the drinks department for juice, but then you bump into toys and lightly stick on radio-controlled cars, forgetting where you were going. Next month, on the recommendation of the AI, the departments will be swapped, because you and the like always buy juices and never like toys. You get to the shelves with juice and, having found that your favorite apple isn’t there, you are going to leave, but here you see an employee of the store, who drives up to a cart with your favorite juice. Cameras and sensors found a lack of this product on the shelf and made it known to the warehouse.
Now it’s time for the sausage – you’ve already been drawn to your beloved Kraków, but here again, the store employee interferes with your peaceful shopping. He takes the sausage from the shelf because, by its color and shape of the chamber, the expiry date has been determined. You take the doctor’s, go further, and then the store employee brings a basket that you forgot to take at the entrance: the cameras noticed that your hands are full of beer and sausage, and sent a signal to the staff. And you were given a basket, and not a cart, because the cameras saw how you came on foot and did not arrive by car, from which it was concluded that you do not need any products. You are already going to leave, but you see a strange man – he looks around suspiciously at the sides and gazes at a bottle of good whiskey for 100,500 rubles. This is the same stranger who hung around in the morning at the neighbor’s Bentley. But here the guard approaches him and begins a conversation with him. The cameras recognized the suspicious behavior characteristic of petty thieves and alerted the staff before the crime occurred.
To date, not all of the features described have been implemented. The same “smart” hypermarket Walmart mentioned above is still a single example, and retailers use smart cameras primarily to analyze consumer traffic for business optimization. Such solutions are already supplied by companies such as GoodVision. Cameras predicting theft have been developed (Vaak, a Japanese startup), but not widely implemented yet. Finally, the technology of visual quality control of the product and its product range is a matter for the future, since it is still more profitable for retailers to attract relatively inexpensive labor for these purposes.
Conclusion: Big Brother – good or evil?
In May 2019, San Francisco authorities prohibited police and other municipal services from using “smart” cameras with facial recognition. Innovation-Friendly San Francisco was the first city in the United States to impose such a ban. The city council considered that this technology threatens the rights of the population. Meanwhile, in another American city, Boston, it was these systems that helped find the perpetrators of the terrorist attack on the marathon two years earlier, and there is no prohibition there. Who is right? Perhaps, the one who understands that the equipment is impartial, and only the final consumer of its capabilities is important. It is his motives that determine whether the lens of a smart camera is aiming at us, or if he is only carefully watching us.
