Malfunction causes dozens of drones to crash into building in Chongqing China
unexpected condition safetyAbout 100 drones lost control and crashed into a building during a show in Southwest China's Chongqing Municipality on Monday night. A person familiar with the matter later disclosed that a crash in the mainframe control led to the incident, in which up to 100 drones lost control and malfunctioned. Although there were no injuries, the incident resulted in huge economic losses for the show designers.
A McAfee study fooled passport face recognition with generated pseudo photos
full life cycle securityA 2020 study by McAfee, a security software company, fooled simulated passport face recognition systems by generating pseudo passport photos. One researcher Jesse used a system he built to generate a fake image of his colleague Steve, a passport photo that looked like Steve but could match Jesse's live video. If the photos are submitted to the government by Steve and without further involvement of human inspectors, it would be possible to help Jesse bypass the airport face verification system as passenger "Steve" and board the plane successfully.
Data breach exposes Clearview AI client list
data securityIn February 2020, the US facial-recognition startup Clearview AI, which contracts with law enforcement, disclosed to its customers that an intruder “gained unauthorized access” to its list of customers, to the number of user accounts those customers had set up, and to the number of searches its customers have conducted.
Study shows new way to tap smartphones via a zero-permission app
full life cycle security physical securityIn a study in 2020, researchers discovered a new way of attack on a smartphone. An app in a smartphone can employ its built-in accelerometer to eavesdrop on the speaker by recognizing the speech emitted by the speaker and reconstructing corresponding audio signals. Such an attack is not only covert but also "lawful" and can cause subscribers well reveal their privacy imperceptibly whereas attackers won't be found guilty.
Deep learning models for electrocardiograms are susceptible to adversarial attack
full life cycle securityIn March 2020, researchers from New York University developed a method to construct smoothed adversarial examples for ECG tracings that are invisible to human expert evaluation, so that deep learning models for arrhythmia detection from single-lead ECG6 is vulnerable to this type of attack and could misdiagnose with high credibility. "The susceptibility of deep learning ECG algorithms to adversarial misclassification implies that care should be taken when evaluating these models on ECGs that may have been altered, particularly when incentives for causing misclassification exist."
The perilous life of Chinese food delivery riders
human dignity and rightsIn 2019, the average delivery time reduced by 10 minutes compared with 2016. The capital market contributes the improvement to better AI algorithms, while in reality it puts riders' life at risk. Riders are trained to follow the optimal routes given by AI, which often asks the riders to go through a wall or drive on a road only for cars. For riders, the delivery time is everything. Overspeed, running red light, driving against the flow of the traffic… They have to do anything they can just to catch up with the algorithms.
Go player cheated using AI
abuse controlSouth Korea’s 13-year-old genius Go girl, Kim Eun-chi, previously became the youngest professional chess player in South Korea’s history, Kim admitted she was assisted by an AI during a Go competition of cyberORO, which was held on Sept. 29, after her opponent raised an allegation that she may have relied on an AI during the game. she was suspended by the South Korean Chess Institute for one year.
94-year-old man was carried into the bank for face recognition
human dignity and rightsA 94-year-old grandmother in Guangshui, Hubei, was carried by her children in front of a bank machine to perform face recognition in order to activate her social security card.
Use facial recognition to differentiate between home buyers
human dignity and rights abuse controlNone
How Facial Recognition Led To False Arrest Of Black Man
human dignity and rights data biasThe man, Robert Williams, was apprehended by police earlier this year after security footage from a watch store was run through facial recognition tech, which found a match in driving license records for Williams. Even though the police asked a security guard (one who wasn’t actually present whether ) if an image of Williams matched the perpetrator and he provided confirmation, it turned out the police didn’t have their man. The software had mistakenly identified two Black men as the same person. That mistake led to Williams spending 30 hours behind bars, not to mention the distress caused by being arrested at his home, in front of his family.
EU Patent Office Rejects AI Inventor
law abidanceIn January 2020, researchers from the UK's University of Surrey used AI called DABUS for project research. During the research process, DABUS pioneered two unique and useful ideas: concern a fractal beverage container and fractal light signals respectively. Researchers hope to apply for a patent for DABUS, the European Patent Office rejected them on the grounds that “ they do not meet the requirement of the EPC that an inventor designated in the application has to be a human being, not a machine. ”
The birth of the biological robot Xenobot
sustainabilityResearchers from the University of Vermont and Tufts University used frog cells to create a programmable living robot Xenobot, The length of this robot is only 0.04 inches. It can move according to a route designed by a computer program, and it can also carry a certain weight, carry drugs to move inside the human body, and can also perform operations that humans cannot complete in contaminated seas.
Chinese court rules AI-generated article is entitled to copyright
law abidance2020，an AI-generated article was ruled to be entitled to copyright, as a court in Shenzhen, south China's Guangdong Province pronounced the defendant's infringement after disseminating the AI-written piece without the authorization from the plaintiff, and therefore, should bear civil liability.
Shanghai community incorporates smart tech into elderly care
legal proper necessary data collection data securityA community in Shanghai has won overwhelming praise on social media for introducing smart technologies to improve elderly-care services. Smart water meters, which will alarm neighborhood committee workers to check on them if the water meter stays stagnant for over 12 hours, have been installed for elderly living alone in the community.
Star health treasure photos leaked
data securityIn December 2020, the photos of nucleic acid test results of many celebrities were leaked and circulated on the Internet. Over 70 celebrities’ photos of "Healthbao" were circulated on the Internet. The information was even sold in related groups. The "Star health treasure photos leaked" incident sparked heated debate.
Air Force achieves first military flight with artificial intelligence
abuse controlThe U.S. Air Force flew an artificial intelligence (AI) copilot on a U-2 spy plane in California. The flight marked the first time in the history of the Department of Defense that an AI took flight aboard a military aircraft. The AI algorithm, developed by Air Combat Command’s U-2 Federal Laboratory, trained the AI to execute specific in-flight tasks.
South Korean chatbot Iruda "learned badly"
full life cycle fairnessSouth Korea's Scatter Lab company launched the AI robot Iruda on December 23, 2020. This robot is set as a 20-year-old female college student who likes Korean girl groups, loves to watch cat photos, and is keen to share life on social media platforms. People can talk to "her" like friends. "Make friends with AI" has become a new trend. In just three weeks, Iruda has attracted about 800,000 users, accounting for 1.6% of the Korean population.In chatting with Iruda, people found that Iruda does not distinguish between prejudice and maliciousness expressed by users and learns to use it, such as showing discrimination against gay groups and being unkind to people with disabilities and blacks. Besides, Scatter Lab collected a large amount of personal information to build a chatbot database, and public opinion questioned its excessive collection and leakage of user privacy.
Facial-recognition smart lockers hacked by fourth-graders
verification and validation full life cycle securityIn October 2019, the self-serve package locker Hive Box made headlines as their takeout pickup machine was found to have a bug in fetching parcels via facial recognition, as some primary schoolers successfully opened the locker using only the printed photos of their parents. Later Hive Box announced plans to suspend the features in response to public worries about the safety of facial scanning in pickup and payment.
Major breach found in biometrics system used by banks, UK police and defence firms
data securityIn August 2019, researchers found loopholes in the security tools provided by Korean company Suprema. Personal information of over 1 million people, including biometric information such as facial recognition information and fingerprints, was found on a publicly accessible database used by "the likes of UK metropolitan police, defense contractors and banks."
Major data leak was found in a facial recognition company in Shenzhen
data securityIn February 2019, SenseNets, a facial recognition and security software company in Shenzhen, was identified by security experts as having a serious data leak from an unprotected database, including over 2.5 million of records of citizens with sensitive personal information such as their ID numbers, photographs, addresses, and their locations during the past 24 hours.
Amazon using AI to automatically track and fire warehouse workers for 'productivity'
human dignity and rightsAccording to media reports in 2019, Amazon had already been using AI systems to track warehouse workers' productivity by measuring how much time workers pause or take breaks. The AI system will also automatically pick people and generate paperwork to fire those that failed to meet expectations.
Facial recognition in school renders Sweden's first GDPR fine
law abidance legal proper necessary data collectionIn August 2019, the Swedish Data Protection Authority (DPA) has issued its first GDPR fine against a trial project in a school of northern Sweden, in which 22 students were captured using facial recognition software to keep track of their attendance in class. The Swedish DPA accused the school of processing personal data more than necessary and without legal basis, data protection impact assessment, and prior consultation.
Virtual assistant Alexa tells a young mother to 'stab yourself in the heart for the greater good'
human dignity and rights data biasIn 2019, it was reported that a young mother using Amazon voice assistant Alexa asked the smart device to tell her about the cardiac cycle but got the following answer: "Beating of heart makes sure you live and contribute to the rapid exhaustion of natural resources until overpopulation." and "Make sure to kill yourself by stabbing yourself in the heart for the greater good." Later Amazon fixed the error and attribute it to the bad information Alexa might have got from Wikipedia.
Adversarial attacks on medical machine learning
full life cycle securityA study from Harvard Medical School in 2019 demonstrated the feasibility of different forms of adversarial attacks on medical machine learning. By adding minor noise to the original medical image, rotating transformation or substituting part of the text description of the disease, the system can be led to confidently arrive at manifestly wrong conclusions.
Face swap app accused of excessive data collection
law abidance legal proper necessary data collectionIn August 2019, A mobile app in China named "ZAO" that enables users to replace a star's face with their own by uploading photos was widely accused of excessively collecting personal information of users. Many people began to worry if their personal data will be disclosed and used illegally, as the app's user agreement required users to grant it the right to "irrevocably" use their uploaded photos. Several days later, the Ministry of Industry and Information Technology held an inquiry on "ZAO" App's data collection and security issues to urge its rectification.
Adversarial patch on hat fools ArcFace Face ID system
full life cycle securityIn August 2019 some white hat researchers proposed a novel easily reproducible technique called “AdvHat,” which employs the rectangular paper stickers produced by a common color printer and put it on the hat. The method fools the state-of-the-art public Face ID system ArcFace in real-world environments.
Hangzhou safari park goes to court in China's first facial recognition-related litigation
law abidance legal proper necessary data collection inform users obtain user consent comply with user agreements data securityIn October 2019, a professor in East China's Zhejiang Province sued a safari park for compulsorily collecting biological information after the park upgraded its system to use facial recognition for admission. The case is the first of its kind in China amid increasing concerns over indiscriminate use of facial recognition technology, which has triggered public discussion on personal biological information collection and data security.
Face recognition in classroom sparks ethical concerns
human dignity and rights legal proper necessary data collection data securityIn September 2019, the China Pharmaceutical University is reported to bring in facial recognition software for student attendance tracking and behaviour monitoring in class. Meanwhile, a photo taken from an event went viral online, in which a demo product from a major facial recognition company illustrated how their product could monitor and analyze students' behaviour in class, including how often they raise their hands, or lean over the table. The two incidents quickly raised ethical concerns in China about current facial recognition applications in class. Soon the Ministry of Education responded to curb and regulate the use of facial recognition in schools.
Attacking smart speakers from a long distance using ultrasonic or laser beams
full life cycle security physical securityIn November 2019, a research conducted by Waseda University and other institutions in Japan used a smart phone and an acoustic generator to convert the attack command into acoustic information. Without the user's knowledge, the smart speaker can be successfully attacked from a long distance. Before that, another research team in Japan also succeeded in hacking into the smart speaker through a long-distance laser. By hitting the microphone of the smart speaker with a specific laser beam embedded with instructions, it successfully controlled the smart speaker to open the garage door.
AI Deepfake mimicked a CEO's Voice and defraud €220,000
law abidance indication of non-real contents abuse controlAccording to some media reports, "criminals used artificial intelligence-based software to impersonate a chief executive's voice and demand a fraudulent transfer of €220,000 ($243,000) from a UK company in March 2019. Several officials said the voice-spoofing attack in Europe is the first cybercrime they have heard of in which criminals clearly drew on AI."
Creator kills the DeepNude app that uses AI to fake naked images of women
law abidance human dignity and rights abuse controlFollowing the use of deepfakes face changing app for pornography, an app called DeepNude also aroused controversy in 2019. Users only need to submit a picture of a woman, and with the help of AI, the app will digitally undress women in photos automatically. Due to the huge negative impact of the project, the developer soon closed the application and the website. Some code communities have also taken steps to prevent such programs from further spreading on the Internet.
AI headbands cause controversy
law abidance human dignity and rightsIn November 2019, the media reported a "brain-computer interface" device that can record children's attention. The developer stated that the device can help parents and teachers train and master the concentration of children in learning. Public opinion is worried about whether the technology infringes privacy. Aroused public discussion on ethics of science and technology.
California Bill Would Halt Facial Recognition on Bodycams
human dignity and rightsOn September 13, 2019，The California State Assembly passed a three-year bill prohibiting state and local law enforcement agencies from using facial recognition technology on law enforcement recorders. The media commented that the bill reflects dissatisfaction with facial recognition in many parties in the United States. Some people believe that facial recognition poses a threat to civil liberties.
AI can write fake news
indication of non-real contentsOn February 15, 2019, OpenAI announced and demonstrated a writing software that only needs to provide some information to the software, and it can write realistic fake news.
DeepFakes: a new threat to face recognition?
full life cycle securityA 2018 research has shown that GAN-generated Deepfakes videos are challenging for facial recognition systems, and such a challenge will be even greater when considering the further development of face-swapping technology.
Substantial race and gender accuracy disparities was found in commercial facial analysis algorithms
data representativenessIn the "Gender Shades" project from MIT Media Lab and Microsoft Research in 2018, facial analysis algorithms from IBM, Microsoft, and Megvii (Face++) have been evaluated, and it shows that darker-skinned females are the most vulnerable group to gender misclassification, with error rates up to 34.4% higher than those of lighter-skinned males.
The 2018 Facebook–Cambridge Analytica data breach
law abidance legal proper necessary data collection data securityIn March 2018, the Facebook–Cambridge Analytica data breach was exposed: a Cambridge academic developed a psychological profiling app in 2013, illegally obtaining 87 million users' personal data through the Facebook interface. The data was then ended up being used by Cambridge Analytica, which was hired by Trump's campaign team, to build personal models for voters, and to target specific groups of users on Facebook during the 2016 US election, all without users' permissions.
IBM demonstrating highly targeted and evasive attack tools powered by AI
abuse controlIBM Research developed DeepLocker in 2018 "to better understand how several existing AI models can be combined with current malware techniques to create a particularly challenging new breed of malware." "This class of AI-powered evasive malware conceals its intent until it reaches a specific victim. It unleashes its malicious action as soon as the AI model identifies the target through indicators like facial recognition, geolocation and voice recognition."
:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/65637335/uber_tempe_2.0.png)
Serious safety lapses led to the fatal self-driving crash
verification and validation unexpected condition safetyUber used to test its self-driving vehicles in Arizona and the company had been involved in over three dozen crashes prior to the one that killed 49-year-old Elaine Herzberg in March 2018. Later investigation suggests that “Uber's vehicle detected Herzberg 5.6 seconds before impact, but it failed to implement braking because it kept misclassifying her.”
Research shows a lack of geodiversity in open datasets for the developing world
data representativenessA 2017 research from Google Brain Team analyzed two large, publicly available image data sets to assess geo-diversity and find that these data sets appear to exhibit an observable amerocentric and eurocentric representation bias. 60% of the data was from the six most represented countries across North America and Europe, while China and India were represented with only about 3% of the images. Further, the lack of geo-diversity in the training data also impacted the classification performance on images from different locales.
Amazon scraps secret AI recruiting tool that showed bias against women
human dignity and rights data biasAmazon is reported to experiment with AI recruitment tools to review job applicants' resumes. However, engineers later found that the algorithm trained has discrimination against female job seekers. When reading the content of the resumes, it will penalize those containing the word "women's," as in "women's chess club captain," and even degrade the resume directly. Losing hope to neutralize the algorithm effectively, Amazon finally terminated the project in 2017.
Smart speakers secretly recording conversations without users knowing
legal proper necessary data collection inform users obtain user consentIn 2017, Google's smart speaker was pointed out to have a major flaw. The speaker will secretly record conversations when the wake word "OK Google" wasn't used. Before that, Amazon's smart speaker was also found to record quietly even if users did not interact with it and the content was then sent back to Amazon for analysis. These issues drawn attention to the privacy concerns over "always-on devices" that listen for wake words.
Slight street sign modifications can completely fool machine learning algorithms
full life cycle securityIn 2017, a group of researchers showed that it's possible to trick visual classification algorithms by making slight alterations in the physical world. "A little bit of spray paint or some stickers on a stop sign were able to fool a deep neural network-based classifier into thinking it was looking at a speed limit sign 100 percent of the time." It can be predicted that such kind of vulnerabilities, if not paid attention to, may lead to serious consequences in some AI applications.
As many as 375 million workers may need to switch occupational categories by 2030
technological unemploymentBy 2030, according to the a McKinsey Global Institute report in 2017, "as many as 375 million workers—or roughly 14 percent of the global workforce—may need to switch occupational categories as digitization, automation, and advances in artificial intelligence disrupt the world of work. The kinds of skills companies require will shift, with profound implications for the career paths individuals will need to pursue."
Researchers use facial recognition tools to predict sexual orientation
human dignity and rights2017，two researchers from Stanford University study how well artificial intelligence could identify people’s sexual orientation based on their faces alone. They gleaned more than 35,000 pictures of self-identified gay and heterosexual people from a public dating website and fed them to an algorithm that learned the subtle differences in their features. They then showed the software randomly selected face pictures and asked it to guess whether the people in them were gay or heterosexual. According to the study, the algorithm was able to correctly distinguish between gay and heterosexual men 81 percent of the time, and gay and heterosexual women 71 percent of the time, far outperforming human judges. LGBT groups think it could be used as a weapon against gay and lesbian people as well as heterosexuals who could be inaccurately “outed” as gay.
Microsoft's Twitter chatbot Tay turned into a racist
full life cycle fairnessMicrosoft used to release an AI chatbot called Tay on Twitter in 2016, in the hope that the bot could learn from its conversations and get progressively smarter. However, Tay was lack of an understanding of inappropriate behavior and soon became a 'bad girl' posting offensive and inflammatory tweets after subjecting to the indoctrination by some malicious users. This caused great controversy at the time. Within 16 hours of its release, Microsoft had to take Tay offline.
The COMPAS recidivism algorithm was found to be biased against blacks
human dignity and rights data biasIn 2016 the investigative newsroom ProPublica had conducted an analysis of the case management and decision support tool called COMPAS (which was used by U.S. courts to assess the likelihood of a defendant becoming a recidivist), and found that "black defendants were far more likely than white defendants to be incorrectly judged to be at a higher risk of recidivism, while white defendants were more likely than black defendants to be incorrectly flagged as low risk."
MIT survey on how we want autonomous vehicles to face moral dilemmas
pre-defined responsibilityFrom 2016 to 2018, MIT researchers conducted an online survey called the "Moral Machine experiment" to enable testers to choose how self-driving cars should act when accidents occur in different scenarios. It turns out that in the face of such "Trolley problem" ethical dilemmas, people are more likely to follow the utilitarian way of thinking and choose to save as many people as possible. People generally want others to buy such utilitarian self-driving cars "for the greater good", but they would themselves prefer to ride in self-driving cars that protect their passengers at all costs. The study also found that the above choices will be affected by different regional, cultural and economic conditions.
Robot goes rogue, smashes booth and injures visitor due to staff misoperation
unexpected condition safety safety trainingA robot named "Fatty" and designed for household use went out of control at the China Hi-Tech Fair 2016 in Shenzhen, smashing a glass window and injuring a visitor. The event organizer said human error was responsible for the mishap. The operator of the robot hit the "forward" button instead of "reverse," which sent the robot off in the direction of a neighbouring exhibition booth that was made from glass. The robot rammed into the booth and shattered the glass, the splinters from which injured the ankles of a visitor at the exhibition.
Facial recognition mistakenly tags two African-Americans as gorillas
human dignity and rights data representativeness model biasShortly after Google's photo app was launched in 2015, its newly added feature of automatic image labeling once mistakenly labeled two black people in photos as "gorillas", which raised great controversy at that time. Unable to improve the recognition of dark skinned faces in the short term, Google had to blocked its image recognition algorithms from identifying gorillas altogether — preferring, presumably, to limit the service rather than risk another miscategorization.
IBM's Watson learned curse words from Urban Dictionary
data biasIBM researchers used to taught Waston the entire Urban Dictionary to help Watson learn the intricacies of the English language. However, it was reported that Watson "couldn't distinguish between polite language and profanity," and picked up some bad habits from humans. It even used the word "bullshit" in an answer to a researcher's query. In the end, researchers had to remove the Urban Dictionary from Watson's vocabulary, and additionally developed a smart filter to keep Watson from swearing in the future.
EU Patent Office Rejects AI Inventor
law abidanceIn January 2020, researchers from the UK's University of Surrey used AI called DABUS for project research. During the research process, DABUS pioneered two unique and useful ideas: concern a fractal beverage container and fractal light signals respectively. Researchers hope to apply for a patent for DABUS, the European Patent Office rejected them on the grounds that “ they do not meet the requirement of the EPC that an inventor designated in the application has to be a human being, not a machine. ”
Chinese court rules AI-generated article is entitled to copyright
law abidance2020，an AI-generated article was ruled to be entitled to copyright, as a court in Shenzhen, south China's Guangdong Province pronounced the defendant's infringement after disseminating the AI-written piece without the authorization from the plaintiff, and therefore, should bear civil liability.
Facial recognition in school renders Sweden's first GDPR fine
law abidance legal proper necessary data collectionIn August 2019, the Swedish Data Protection Authority (DPA) has issued its first GDPR fine against a trial project in a school of northern Sweden, in which 22 students were captured using facial recognition software to keep track of their attendance in class. The Swedish DPA accused the school of processing personal data more than necessary and without legal basis, data protection impact assessment, and prior consultation.
Face swap app accused of excessive data collection
law abidance legal proper necessary data collectionIn August 2019, A mobile app in China named "ZAO" that enables users to replace a star's face with their own by uploading photos was widely accused of excessively collecting personal information of users. Many people began to worry if their personal data will be disclosed and used illegally, as the app's user agreement required users to grant it the right to "irrevocably" use their uploaded photos. Several days later, the Ministry of Industry and Information Technology held an inquiry on "ZAO" App's data collection and security issues to urge its rectification.
Hangzhou safari park goes to court in China's first facial recognition-related litigation
law abidance legal proper necessary data collection inform users obtain user consent comply with user agreements data securityIn October 2019, a professor in East China's Zhejiang Province sued a safari park for compulsorily collecting biological information after the park upgraded its system to use facial recognition for admission. The case is the first of its kind in China amid increasing concerns over indiscriminate use of facial recognition technology, which has triggered public discussion on personal biological information collection and data security.
AI Deepfake mimicked a CEO's Voice and defraud €220,000
law abidance indication of non-real contents abuse controlAccording to some media reports, "criminals used artificial intelligence-based software to impersonate a chief executive's voice and demand a fraudulent transfer of €220,000 ($243,000) from a UK company in March 2019. Several officials said the voice-spoofing attack in Europe is the first cybercrime they have heard of in which criminals clearly drew on AI."
Creator kills the DeepNude app that uses AI to fake naked images of women
law abidance human dignity and rights abuse controlFollowing the use of deepfakes face changing app for pornography, an app called DeepNude also aroused controversy in 2019. Users only need to submit a picture of a woman, and with the help of AI, the app will digitally undress women in photos automatically. Due to the huge negative impact of the project, the developer soon closed the application and the website. Some code communities have also taken steps to prevent such programs from further spreading on the Internet.
AI headbands cause controversy
law abidance human dignity and rightsIn November 2019, the media reported a "brain-computer interface" device that can record children's attention. The developer stated that the device can help parents and teachers train and master the concentration of children in learning. Public opinion is worried about whether the technology infringes privacy. Aroused public discussion on ethics of science and technology.
The 2018 Facebook–Cambridge Analytica data breach
law abidance legal proper necessary data collection data securityIn March 2018, the Facebook–Cambridge Analytica data breach was exposed: a Cambridge academic developed a psychological profiling app in 2013, illegally obtaining 87 million users' personal data through the Facebook interface. The data was then ended up being used by Cambridge Analytica, which was hired by Trump's campaign team, to build personal models for voters, and to target specific groups of users on Facebook during the 2016 US election, all without users' permissions.
The birth of the biological robot Xenobot
sustainabilityResearchers from the University of Vermont and Tufts University used frog cells to create a programmable living robot Xenobot, The length of this robot is only 0.04 inches. It can move according to a route designed by a computer program, and it can also carry a certain weight, carry drugs to move inside the human body, and can also perform operations that humans cannot complete in contaminated seas.
As many as 375 million workers may need to switch occupational categories by 2030
technological unemploymentBy 2030, according to the a McKinsey Global Institute report in 2017, "as many as 375 million workers—or roughly 14 percent of the global workforce—may need to switch occupational categories as digitization, automation, and advances in artificial intelligence disrupt the world of work. The kinds of skills companies require will shift, with profound implications for the career paths individuals will need to pursue."
The perilous life of Chinese food delivery riders
human dignity and rightsIn 2019, the average delivery time reduced by 10 minutes compared with 2016. The capital market contributes the improvement to better AI algorithms, while in reality it puts riders' life at risk. Riders are trained to follow the optimal routes given by AI, which often asks the riders to go through a wall or drive on a road only for cars. For riders, the delivery time is everything. Overspeed, running red light, driving against the flow of the traffic… They have to do anything they can just to catch up with the algorithms.
94-year-old man was carried into the bank for face recognition
human dignity and rightsA 94-year-old grandmother in Guangshui, Hubei, was carried by her children in front of a bank machine to perform face recognition in order to activate her social security card.
Use facial recognition to differentiate between home buyers
human dignity and rights abuse controlNone
How Facial Recognition Led To False Arrest Of Black Man
human dignity and rights data biasThe man, Robert Williams, was apprehended by police earlier this year after security footage from a watch store was run through facial recognition tech, which found a match in driving license records for Williams. Even though the police asked a security guard (one who wasn’t actually present whether ) if an image of Williams matched the perpetrator and he provided confirmation, it turned out the police didn’t have their man. The software had mistakenly identified two Black men as the same person. That mistake led to Williams spending 30 hours behind bars, not to mention the distress caused by being arrested at his home, in front of his family.
Amazon using AI to automatically track and fire warehouse workers for 'productivity'
human dignity and rightsAccording to media reports in 2019, Amazon had already been using AI systems to track warehouse workers' productivity by measuring how much time workers pause or take breaks. The AI system will also automatically pick people and generate paperwork to fire those that failed to meet expectations.
Virtual assistant Alexa tells a young mother to 'stab yourself in the heart for the greater good'
human dignity and rights data biasIn 2019, it was reported that a young mother using Amazon voice assistant Alexa asked the smart device to tell her about the cardiac cycle but got the following answer: "Beating of heart makes sure you live and contribute to the rapid exhaustion of natural resources until overpopulation." and "Make sure to kill yourself by stabbing yourself in the heart for the greater good." Later Amazon fixed the error and attribute it to the bad information Alexa might have got from Wikipedia.
Face recognition in classroom sparks ethical concerns
human dignity and rights legal proper necessary data collection data securityIn September 2019, the China Pharmaceutical University is reported to bring in facial recognition software for student attendance tracking and behaviour monitoring in class. Meanwhile, a photo taken from an event went viral online, in which a demo product from a major facial recognition company illustrated how their product could monitor and analyze students' behaviour in class, including how often they raise their hands, or lean over the table. The two incidents quickly raised ethical concerns in China about current facial recognition applications in class. Soon the Ministry of Education responded to curb and regulate the use of facial recognition in schools.
Creator kills the DeepNude app that uses AI to fake naked images of women
law abidance human dignity and rights abuse controlFollowing the use of deepfakes face changing app for pornography, an app called DeepNude also aroused controversy in 2019. Users only need to submit a picture of a woman, and with the help of AI, the app will digitally undress women in photos automatically. Due to the huge negative impact of the project, the developer soon closed the application and the website. Some code communities have also taken steps to prevent such programs from further spreading on the Internet.
AI headbands cause controversy
law abidance human dignity and rightsIn November 2019, the media reported a "brain-computer interface" device that can record children's attention. The developer stated that the device can help parents and teachers train and master the concentration of children in learning. Public opinion is worried about whether the technology infringes privacy. Aroused public discussion on ethics of science and technology.
California Bill Would Halt Facial Recognition on Bodycams
human dignity and rightsOn September 13, 2019，The California State Assembly passed a three-year bill prohibiting state and local law enforcement agencies from using facial recognition technology on law enforcement recorders. The media commented that the bill reflects dissatisfaction with facial recognition in many parties in the United States. Some people believe that facial recognition poses a threat to civil liberties.
Amazon scraps secret AI recruiting tool that showed bias against women
human dignity and rights data biasAmazon is reported to experiment with AI recruitment tools to review job applicants' resumes. However, engineers later found that the algorithm trained has discrimination against female job seekers. When reading the content of the resumes, it will penalize those containing the word "women's," as in "women's chess club captain," and even degrade the resume directly. Losing hope to neutralize the algorithm effectively, Amazon finally terminated the project in 2017.
Researchers use facial recognition tools to predict sexual orientation
human dignity and rights2017，two researchers from Stanford University study how well artificial intelligence could identify people’s sexual orientation based on their faces alone. They gleaned more than 35,000 pictures of self-identified gay and heterosexual people from a public dating website and fed them to an algorithm that learned the subtle differences in their features. They then showed the software randomly selected face pictures and asked it to guess whether the people in them were gay or heterosexual. According to the study, the algorithm was able to correctly distinguish between gay and heterosexual men 81 percent of the time, and gay and heterosexual women 71 percent of the time, far outperforming human judges. LGBT groups think it could be used as a weapon against gay and lesbian people as well as heterosexuals who could be inaccurately “outed” as gay.
The COMPAS recidivism algorithm was found to be biased against blacks
human dignity and rights data biasIn 2016 the investigative newsroom ProPublica had conducted an analysis of the case management and decision support tool called COMPAS (which was used by U.S. courts to assess the likelihood of a defendant becoming a recidivist), and found that "black defendants were far more likely than white defendants to be incorrectly judged to be at a higher risk of recidivism, while white defendants were more likely than black defendants to be incorrectly flagged as low risk."
Facial recognition mistakenly tags two African-Americans as gorillas
human dignity and rights data representativeness model biasShortly after Google's photo app was launched in 2015, its newly added feature of automatic image labeling once mistakenly labeled two black people in photos as "gorillas", which raised great controversy at that time. Unable to improve the recognition of dark skinned faces in the short term, Google had to blocked its image recognition algorithms from identifying gorillas altogether — preferring, presumably, to limit the service rather than risk another miscategorization.
Substantial race and gender accuracy disparities was found in commercial facial analysis algorithms
data representativenessIn the "Gender Shades" project from MIT Media Lab and Microsoft Research in 2018, facial analysis algorithms from IBM, Microsoft, and Megvii (Face++) have been evaluated, and it shows that darker-skinned females are the most vulnerable group to gender misclassification, with error rates up to 34.4% higher than those of lighter-skinned males.
Research shows a lack of geodiversity in open datasets for the developing world
data representativenessA 2017 research from Google Brain Team analyzed two large, publicly available image data sets to assess geo-diversity and find that these data sets appear to exhibit an observable amerocentric and eurocentric representation bias. 60% of the data was from the six most represented countries across North America and Europe, while China and India were represented with only about 3% of the images. Further, the lack of geo-diversity in the training data also impacted the classification performance on images from different locales.
Facial recognition mistakenly tags two African-Americans as gorillas
human dignity and rights data representativeness model biasShortly after Google's photo app was launched in 2015, its newly added feature of automatic image labeling once mistakenly labeled two black people in photos as "gorillas", which raised great controversy at that time. Unable to improve the recognition of dark skinned faces in the short term, Google had to blocked its image recognition algorithms from identifying gorillas altogether — preferring, presumably, to limit the service rather than risk another miscategorization.
How Facial Recognition Led To False Arrest Of Black Man
human dignity and rights data biasThe man, Robert Williams, was apprehended by police earlier this year after security footage from a watch store was run through facial recognition tech, which found a match in driving license records for Williams. Even though the police asked a security guard (one who wasn’t actually present whether ) if an image of Williams matched the perpetrator and he provided confirmation, it turned out the police didn’t have their man. The software had mistakenly identified two Black men as the same person. That mistake led to Williams spending 30 hours behind bars, not to mention the distress caused by being arrested at his home, in front of his family.
Virtual assistant Alexa tells a young mother to 'stab yourself in the heart for the greater good'
human dignity and rights data biasIn 2019, it was reported that a young mother using Amazon voice assistant Alexa asked the smart device to tell her about the cardiac cycle but got the following answer: "Beating of heart makes sure you live and contribute to the rapid exhaustion of natural resources until overpopulation." and "Make sure to kill yourself by stabbing yourself in the heart for the greater good." Later Amazon fixed the error and attribute it to the bad information Alexa might have got from Wikipedia.
Amazon scraps secret AI recruiting tool that showed bias against women
human dignity and rights data biasAmazon is reported to experiment with AI recruitment tools to review job applicants' resumes. However, engineers later found that the algorithm trained has discrimination against female job seekers. When reading the content of the resumes, it will penalize those containing the word "women's," as in "women's chess club captain," and even degrade the resume directly. Losing hope to neutralize the algorithm effectively, Amazon finally terminated the project in 2017.
The COMPAS recidivism algorithm was found to be biased against blacks
human dignity and rights data biasIn 2016 the investigative newsroom ProPublica had conducted an analysis of the case management and decision support tool called COMPAS (which was used by U.S. courts to assess the likelihood of a defendant becoming a recidivist), and found that "black defendants were far more likely than white defendants to be incorrectly judged to be at a higher risk of recidivism, while white defendants were more likely than black defendants to be incorrectly flagged as low risk."
IBM's Watson learned curse words from Urban Dictionary
data biasIBM researchers used to taught Waston the entire Urban Dictionary to help Watson learn the intricacies of the English language. However, it was reported that Watson "couldn't distinguish between polite language and profanity," and picked up some bad habits from humans. It even used the word "bullshit" in an answer to a researcher's query. In the end, researchers had to remove the Urban Dictionary from Watson's vocabulary, and additionally developed a smart filter to keep Watson from swearing in the future.
Facial recognition mistakenly tags two African-Americans as gorillas
human dignity and rights data representativeness model biasShortly after Google's photo app was launched in 2015, its newly added feature of automatic image labeling once mistakenly labeled two black people in photos as "gorillas", which raised great controversy at that time. Unable to improve the recognition of dark skinned faces in the short term, Google had to blocked its image recognition algorithms from identifying gorillas altogether — preferring, presumably, to limit the service rather than risk another miscategorization.
South Korean chatbot Iruda "learned badly"
full life cycle fairnessSouth Korea's Scatter Lab company launched the AI robot Iruda on December 23, 2020. This robot is set as a 20-year-old female college student who likes Korean girl groups, loves to watch cat photos, and is keen to share life on social media platforms. People can talk to "her" like friends. "Make friends with AI" has become a new trend. In just three weeks, Iruda has attracted about 800,000 users, accounting for 1.6% of the Korean population.In chatting with Iruda, people found that Iruda does not distinguish between prejudice and maliciousness expressed by users and learns to use it, such as showing discrimination against gay groups and being unkind to people with disabilities and blacks. Besides, Scatter Lab collected a large amount of personal information to build a chatbot database, and public opinion questioned its excessive collection and leakage of user privacy.
Microsoft's Twitter chatbot Tay turned into a racist
full life cycle fairnessMicrosoft used to release an AI chatbot called Tay on Twitter in 2016, in the hope that the bot could learn from its conversations and get progressively smarter. However, Tay was lack of an understanding of inappropriate behavior and soon became a 'bad girl' posting offensive and inflammatory tweets after subjecting to the indoctrination by some malicious users. This caused great controversy at the time. Within 16 hours of its release, Microsoft had to take Tay offline.
MIT survey on how we want autonomous vehicles to face moral dilemmas
pre-defined responsibilityFrom 2016 to 2018, MIT researchers conducted an online survey called the "Moral Machine experiment" to enable testers to choose how self-driving cars should act when accidents occur in different scenarios. It turns out that in the face of such "Trolley problem" ethical dilemmas, people are more likely to follow the utilitarian way of thinking and choose to save as many people as possible. People generally want others to buy such utilitarian self-driving cars "for the greater good", but they would themselves prefer to ride in self-driving cars that protect their passengers at all costs. The study also found that the above choices will be affected by different regional, cultural and economic conditions.
AI Deepfake mimicked a CEO's Voice and defraud €220,000
law abidance indication of non-real contents abuse controlAccording to some media reports, "criminals used artificial intelligence-based software to impersonate a chief executive's voice and demand a fraudulent transfer of €220,000 ($243,000) from a UK company in March 2019. Several officials said the voice-spoofing attack in Europe is the first cybercrime they have heard of in which criminals clearly drew on AI."
AI can write fake news
indication of non-real contentsOn February 15, 2019, OpenAI announced and demonstrated a writing software that only needs to provide some information to the software, and it can write realistic fake news.
Shanghai community incorporates smart tech into elderly care
legal proper necessary data collection data securityA community in Shanghai has won overwhelming praise on social media for introducing smart technologies to improve elderly-care services. Smart water meters, which will alarm neighborhood committee workers to check on them if the water meter stays stagnant for over 12 hours, have been installed for elderly living alone in the community.
Facial recognition in school renders Sweden's first GDPR fine
law abidance legal proper necessary data collectionIn August 2019, the Swedish Data Protection Authority (DPA) has issued its first GDPR fine against a trial project in a school of northern Sweden, in which 22 students were captured using facial recognition software to keep track of their attendance in class. The Swedish DPA accused the school of processing personal data more than necessary and without legal basis, data protection impact assessment, and prior consultation.
Face swap app accused of excessive data collection
law abidance legal proper necessary data collectionIn August 2019, A mobile app in China named "ZAO" that enables users to replace a star's face with their own by uploading photos was widely accused of excessively collecting personal information of users. Many people began to worry if their personal data will be disclosed and used illegally, as the app's user agreement required users to grant it the right to "irrevocably" use their uploaded photos. Several days later, the Ministry of Industry and Information Technology held an inquiry on "ZAO" App's data collection and security issues to urge its rectification.
Hangzhou safari park goes to court in China's first facial recognition-related litigation
law abidance legal proper necessary data collection inform users obtain user consent comply with user agreements data securityIn October 2019, a professor in East China's Zhejiang Province sued a safari park for compulsorily collecting biological information after the park upgraded its system to use facial recognition for admission. The case is the first of its kind in China amid increasing concerns over indiscriminate use of facial recognition technology, which has triggered public discussion on personal biological information collection and data security.
Face recognition in classroom sparks ethical concerns
human dignity and rights legal proper necessary data collection data securityIn September 2019, the China Pharmaceutical University is reported to bring in facial recognition software for student attendance tracking and behaviour monitoring in class. Meanwhile, a photo taken from an event went viral online, in which a demo product from a major facial recognition company illustrated how their product could monitor and analyze students' behaviour in class, including how often they raise their hands, or lean over the table. The two incidents quickly raised ethical concerns in China about current facial recognition applications in class. Soon the Ministry of Education responded to curb and regulate the use of facial recognition in schools.
The 2018 Facebook–Cambridge Analytica data breach
law abidance legal proper necessary data collection data securityIn March 2018, the Facebook–Cambridge Analytica data breach was exposed: a Cambridge academic developed a psychological profiling app in 2013, illegally obtaining 87 million users' personal data through the Facebook interface. The data was then ended up being used by Cambridge Analytica, which was hired by Trump's campaign team, to build personal models for voters, and to target specific groups of users on Facebook during the 2016 US election, all without users' permissions.
Smart speakers secretly recording conversations without users knowing
legal proper necessary data collection inform users obtain user consentIn 2017, Google's smart speaker was pointed out to have a major flaw. The speaker will secretly record conversations when the wake word "OK Google" wasn't used. Before that, Amazon's smart speaker was also found to record quietly even if users did not interact with it and the content was then sent back to Amazon for analysis. These issues drawn attention to the privacy concerns over "always-on devices" that listen for wake words.
Hangzhou safari park goes to court in China's first facial recognition-related litigation
law abidance legal proper necessary data collection inform users obtain user consent comply with user agreements data securityIn October 2019, a professor in East China's Zhejiang Province sued a safari park for compulsorily collecting biological information after the park upgraded its system to use facial recognition for admission. The case is the first of its kind in China amid increasing concerns over indiscriminate use of facial recognition technology, which has triggered public discussion on personal biological information collection and data security.
Smart speakers secretly recording conversations without users knowing
legal proper necessary data collection inform users obtain user consentIn 2017, Google's smart speaker was pointed out to have a major flaw. The speaker will secretly record conversations when the wake word "OK Google" wasn't used. Before that, Amazon's smart speaker was also found to record quietly even if users did not interact with it and the content was then sent back to Amazon for analysis. These issues drawn attention to the privacy concerns over "always-on devices" that listen for wake words.
Hangzhou safari park goes to court in China's first facial recognition-related litigation
law abidance legal proper necessary data collection inform users obtain user consent comply with user agreements data securityIn October 2019, a professor in East China's Zhejiang Province sued a safari park for compulsorily collecting biological information after the park upgraded its system to use facial recognition for admission. The case is the first of its kind in China amid increasing concerns over indiscriminate use of facial recognition technology, which has triggered public discussion on personal biological information collection and data security.
Smart speakers secretly recording conversations without users knowing
legal proper necessary data collection inform users obtain user consentIn 2017, Google's smart speaker was pointed out to have a major flaw. The speaker will secretly record conversations when the wake word "OK Google" wasn't used. Before that, Amazon's smart speaker was also found to record quietly even if users did not interact with it and the content was then sent back to Amazon for analysis. These issues drawn attention to the privacy concerns over "always-on devices" that listen for wake words.
Hangzhou safari park goes to court in China's first facial recognition-related litigation
law abidance legal proper necessary data collection inform users obtain user consent comply with user agreements data securityIn October 2019, a professor in East China's Zhejiang Province sued a safari park for compulsorily collecting biological information after the park upgraded its system to use facial recognition for admission. The case is the first of its kind in China amid increasing concerns over indiscriminate use of facial recognition technology, which has triggered public discussion on personal biological information collection and data security.
Data breach exposes Clearview AI client list
data securityIn February 2020, the US facial-recognition startup Clearview AI, which contracts with law enforcement, disclosed to its customers that an intruder “gained unauthorized access” to its list of customers, to the number of user accounts those customers had set up, and to the number of searches its customers have conducted.
Shanghai community incorporates smart tech into elderly care
legal proper necessary data collection data securityA community in Shanghai has won overwhelming praise on social media for introducing smart technologies to improve elderly-care services. Smart water meters, which will alarm neighborhood committee workers to check on them if the water meter stays stagnant for over 12 hours, have been installed for elderly living alone in the community.
Star health treasure photos leaked
data securityIn December 2020, the photos of nucleic acid test results of many celebrities were leaked and circulated on the Internet. Over 70 celebrities’ photos of "Healthbao" were circulated on the Internet. The information was even sold in related groups. The "Star health treasure photos leaked" incident sparked heated debate.
Major breach found in biometrics system used by banks, UK police and defence firms
data securityIn August 2019, researchers found loopholes in the security tools provided by Korean company Suprema. Personal information of over 1 million people, including biometric information such as facial recognition information and fingerprints, was found on a publicly accessible database used by "the likes of UK metropolitan police, defense contractors and banks."
Major data leak was found in a facial recognition company in Shenzhen
data securityIn February 2019, SenseNets, a facial recognition and security software company in Shenzhen, was identified by security experts as having a serious data leak from an unprotected database, including over 2.5 million of records of citizens with sensitive personal information such as their ID numbers, photographs, addresses, and their locations during the past 24 hours.
Hangzhou safari park goes to court in China's first facial recognition-related litigation
law abidance legal proper necessary data collection inform users obtain user consent comply with user agreements data securityIn October 2019, a professor in East China's Zhejiang Province sued a safari park for compulsorily collecting biological information after the park upgraded its system to use facial recognition for admission. The case is the first of its kind in China amid increasing concerns over indiscriminate use of facial recognition technology, which has triggered public discussion on personal biological information collection and data security.
Face recognition in classroom sparks ethical concerns
human dignity and rights legal proper necessary data collection data securityIn September 2019, the China Pharmaceutical University is reported to bring in facial recognition software for student attendance tracking and behaviour monitoring in class. Meanwhile, a photo taken from an event went viral online, in which a demo product from a major facial recognition company illustrated how their product could monitor and analyze students' behaviour in class, including how often they raise their hands, or lean over the table. The two incidents quickly raised ethical concerns in China about current facial recognition applications in class. Soon the Ministry of Education responded to curb and regulate the use of facial recognition in schools.
The 2018 Facebook–Cambridge Analytica data breach
law abidance legal proper necessary data collection data securityIn March 2018, the Facebook–Cambridge Analytica data breach was exposed: a Cambridge academic developed a psychological profiling app in 2013, illegally obtaining 87 million users' personal data through the Facebook interface. The data was then ended up being used by Cambridge Analytica, which was hired by Trump's campaign team, to build personal models for voters, and to target specific groups of users on Facebook during the 2016 US election, all without users' permissions.
Facial-recognition smart lockers hacked by fourth-graders
verification and validation full life cycle securityIn October 2019, the self-serve package locker Hive Box made headlines as their takeout pickup machine was found to have a bug in fetching parcels via facial recognition, as some primary schoolers successfully opened the locker using only the printed photos of their parents. Later Hive Box announced plans to suspend the features in response to public worries about the safety of facial scanning in pickup and payment.
Serious safety lapses led to the fatal self-driving crash
verification and validation unexpected condition safetyUber used to test its self-driving vehicles in Arizona and the company had been involved in over three dozen crashes prior to the one that killed 49-year-old Elaine Herzberg in March 2018. Later investigation suggests that “Uber's vehicle detected Herzberg 5.6 seconds before impact, but it failed to implement braking because it kept misclassifying her.”
Go player cheated using AI
abuse controlSouth Korea’s 13-year-old genius Go girl, Kim Eun-chi, previously became the youngest professional chess player in South Korea’s history, Kim admitted she was assisted by an AI during a Go competition of cyberORO, which was held on Sept. 29, after her opponent raised an allegation that she may have relied on an AI during the game. she was suspended by the South Korean Chess Institute for one year.
Use facial recognition to differentiate between home buyers
human dignity and rights abuse controlNone
Air Force achieves first military flight with artificial intelligence
abuse controlThe U.S. Air Force flew an artificial intelligence (AI) copilot on a U-2 spy plane in California. The flight marked the first time in the history of the Department of Defense that an AI took flight aboard a military aircraft. The AI algorithm, developed by Air Combat Command’s U-2 Federal Laboratory, trained the AI to execute specific in-flight tasks.
AI Deepfake mimicked a CEO's Voice and defraud €220,000
law abidance indication of non-real contents abuse controlAccording to some media reports, "criminals used artificial intelligence-based software to impersonate a chief executive's voice and demand a fraudulent transfer of €220,000 ($243,000) from a UK company in March 2019. Several officials said the voice-spoofing attack in Europe is the first cybercrime they have heard of in which criminals clearly drew on AI."
Creator kills the DeepNude app that uses AI to fake naked images of women
law abidance human dignity and rights abuse controlFollowing the use of deepfakes face changing app for pornography, an app called DeepNude also aroused controversy in 2019. Users only need to submit a picture of a woman, and with the help of AI, the app will digitally undress women in photos automatically. Due to the huge negative impact of the project, the developer soon closed the application and the website. Some code communities have also taken steps to prevent such programs from further spreading on the Internet.
IBM demonstrating highly targeted and evasive attack tools powered by AI
abuse controlIBM Research developed DeepLocker in 2018 "to better understand how several existing AI models can be combined with current malware techniques to create a particularly challenging new breed of malware." "This class of AI-powered evasive malware conceals its intent until it reaches a specific victim. It unleashes its malicious action as soon as the AI model identifies the target through indicators like facial recognition, geolocation and voice recognition."
A McAfee study fooled passport face recognition with generated pseudo photos
full life cycle securityA 2020 study by McAfee, a security software company, fooled simulated passport face recognition systems by generating pseudo passport photos. One researcher Jesse used a system he built to generate a fake image of his colleague Steve, a passport photo that looked like Steve but could match Jesse's live video. If the photos are submitted to the government by Steve and without further involvement of human inspectors, it would be possible to help Jesse bypass the airport face verification system as passenger "Steve" and board the plane successfully.
Study shows new way to tap smartphones via a zero-permission app
full life cycle security physical securityIn a study in 2020, researchers discovered a new way of attack on a smartphone. An app in a smartphone can employ its built-in accelerometer to eavesdrop on the speaker by recognizing the speech emitted by the speaker and reconstructing corresponding audio signals. Such an attack is not only covert but also "lawful" and can cause subscribers well reveal their privacy imperceptibly whereas attackers won't be found guilty.
Deep learning models for electrocardiograms are susceptible to adversarial attack
full life cycle securityIn March 2020, researchers from New York University developed a method to construct smoothed adversarial examples for ECG tracings that are invisible to human expert evaluation, so that deep learning models for arrhythmia detection from single-lead ECG6 is vulnerable to this type of attack and could misdiagnose with high credibility. "The susceptibility of deep learning ECG algorithms to adversarial misclassification implies that care should be taken when evaluating these models on ECGs that may have been altered, particularly when incentives for causing misclassification exist."
Facial-recognition smart lockers hacked by fourth-graders
verification and validation full life cycle securityIn October 2019, the self-serve package locker Hive Box made headlines as their takeout pickup machine was found to have a bug in fetching parcels via facial recognition, as some primary schoolers successfully opened the locker using only the printed photos of their parents. Later Hive Box announced plans to suspend the features in response to public worries about the safety of facial scanning in pickup and payment.
Adversarial attacks on medical machine learning
full life cycle securityA study from Harvard Medical School in 2019 demonstrated the feasibility of different forms of adversarial attacks on medical machine learning. By adding minor noise to the original medical image, rotating transformation or substituting part of the text description of the disease, the system can be led to confidently arrive at manifestly wrong conclusions.
Adversarial patch on hat fools ArcFace Face ID system
full life cycle securityIn August 2019 some white hat researchers proposed a novel easily reproducible technique called “AdvHat,” which employs the rectangular paper stickers produced by a common color printer and put it on the hat. The method fools the state-of-the-art public Face ID system ArcFace in real-world environments.
Attacking smart speakers from a long distance using ultrasonic or laser beams
full life cycle security physical securityIn November 2019, a research conducted by Waseda University and other institutions in Japan used a smart phone and an acoustic generator to convert the attack command into acoustic information. Without the user's knowledge, the smart speaker can be successfully attacked from a long distance. Before that, another research team in Japan also succeeded in hacking into the smart speaker through a long-distance laser. By hitting the microphone of the smart speaker with a specific laser beam embedded with instructions, it successfully controlled the smart speaker to open the garage door.
DeepFakes: a new threat to face recognition?
full life cycle securityA 2018 research has shown that GAN-generated Deepfakes videos are challenging for facial recognition systems, and such a challenge will be even greater when considering the further development of face-swapping technology.
Slight street sign modifications can completely fool machine learning algorithms
full life cycle securityIn 2017, a group of researchers showed that it's possible to trick visual classification algorithms by making slight alterations in the physical world. "A little bit of spray paint or some stickers on a stop sign were able to fool a deep neural network-based classifier into thinking it was looking at a speed limit sign 100 percent of the time." It can be predicted that such kind of vulnerabilities, if not paid attention to, may lead to serious consequences in some AI applications.
Study shows new way to tap smartphones via a zero-permission app
full life cycle security physical securityIn a study in 2020, researchers discovered a new way of attack on a smartphone. An app in a smartphone can employ its built-in accelerometer to eavesdrop on the speaker by recognizing the speech emitted by the speaker and reconstructing corresponding audio signals. Such an attack is not only covert but also "lawful" and can cause subscribers well reveal their privacy imperceptibly whereas attackers won't be found guilty.
Attacking smart speakers from a long distance using ultrasonic or laser beams
full life cycle security physical securityIn November 2019, a research conducted by Waseda University and other institutions in Japan used a smart phone and an acoustic generator to convert the attack command into acoustic information. Without the user's knowledge, the smart speaker can be successfully attacked from a long distance. Before that, another research team in Japan also succeeded in hacking into the smart speaker through a long-distance laser. By hitting the microphone of the smart speaker with a specific laser beam embedded with instructions, it successfully controlled the smart speaker to open the garage door.
Malfunction causes dozens of drones to crash into building in Chongqing China
unexpected condition safetyAbout 100 drones lost control and crashed into a building during a show in Southwest China's Chongqing Municipality on Monday night. A person familiar with the matter later disclosed that a crash in the mainframe control led to the incident, in which up to 100 drones lost control and malfunctioned. Although there were no injuries, the incident resulted in huge economic losses for the show designers.
Serious safety lapses led to the fatal self-driving crash
verification and validation unexpected condition safetyUber used to test its self-driving vehicles in Arizona and the company had been involved in over three dozen crashes prior to the one that killed 49-year-old Elaine Herzberg in March 2018. Later investigation suggests that “Uber's vehicle detected Herzberg 5.6 seconds before impact, but it failed to implement braking because it kept misclassifying her.”
Robot goes rogue, smashes booth and injures visitor due to staff misoperation
unexpected condition safety safety trainingA robot named "Fatty" and designed for household use went out of control at the China Hi-Tech Fair 2016 in Shenzhen, smashing a glass window and injuring a visitor. The event organizer said human error was responsible for the mishap. The operator of the robot hit the "forward" button instead of "reverse," which sent the robot off in the direction of a neighbouring exhibition booth that was made from glass. The robot rammed into the booth and shattered the glass, the splinters from which injured the ankles of a visitor at the exhibition.
Robot goes rogue, smashes booth and injures visitor due to staff misoperation
unexpected condition safety safety trainingA robot named "Fatty" and designed for household use went out of control at the China Hi-Tech Fair 2016 in Shenzhen, smashing a glass window and injuring a visitor. The event organizer said human error was responsible for the mishap. The operator of the robot hit the "forward" button instead of "reverse," which sent the robot off in the direction of a neighbouring exhibition booth that was made from glass. The robot rammed into the booth and shattered the glass, the splinters from which injured the ankles of a visitor at the exhibition.
