Rekognition, Amazon‘s facial recognition tool and one of many AWS cloud services for developers, has been the subject of controversy for its use by law enforcement agencies and a reported sales pitch to Immigration and Customs Enforcement (ICE).
But most people might not know what the service actually does. Companies are using it for far more than facial recognition, especially among media companies, social platforms and in international finance.
In fact, it’s the flexibility of the platform itself that has led to the greatest controversy, likely to continue as companies expand how they use the product.
Rekognition is a developer tool with several different functions, including facial recognition, “pathing” — which involves tracking an object, like a soccer ball, through a video frame — and finding and reading text in images and on video that is hard to see with the naked eye.
Rekognition is also used for what’s known as facial analysis or sentiment analysis, which is meant to tag images as showing people who are smiling or frowning to record their projected emotions. The service uses artificial intelligence to “learn” which faces or objects are important to the individual corporate or agency user.
In the private sector, the service has been perhaps most enthusiastically adopted by companies in the marketing and advertising space, particularly those that have huge catalogs of photos and videos that they want to categorize and tag for easy searching.
According to case studies provided by Amazon, social media companies use the program to weed out “fake followers” and find “micro-influencers” who have strong followings on social networks and could be used to promote brands. The White House has used Rekognition in an application that lets users take a photo and find which first lady they most closely resemble.
Companies like ScrippsNetworks Interactive C-SPAN use the technology to more quickly sift through hours of video and gigabytes of photos in order to correctly tag individuals speaking or featured in pictures.
Financial services companies in countries where bank customers don’t have regular access to physical bank locations and rely on mobile banking, such as payments start-up Paylater, have also used the technology to validate transactions.
The “pathing” functionality allows these companies to closely track an object or player — a typical use has been in sports coverage — throughout a single video shot. The company has said the pathing function cannot be combined with facial recognition, and objects can only be tracked through a single camera-shot, not across multiple cameras.
The program can be adapted by developers for a company’s individual needs. Because this feature also takes it out of the control or overview of Amazon, many people, including some Amazon employees, have taken issue with how the technology may be used.
Rekognition’s biggest recent controversy involved Amazon’s pitch to ICE, which was uncovered by the non-profit Project on Government Oversight. A number of Amazon employees objected to the use of the software because of the immigration revelations.
The American Civil Liberties Union (ACLU) criticized the company for the pitch and for the technology in general.
“People should be free to walk down the street without being watched by the government,” the ACLU said in July. “By automating mass surveillance, facial recognition systems like Rekognition threaten this freedom, posing a particular threat to communities already unjustly targeted in the current political climate.”
AWS CEO Andy Jassy reportedly told employees in November that the company will continue supporting the product. Amazon won’t comment on how or which agencies use the service currently, but some have been public about it.
One of those agencies is the Washington County, Oregon, Sheriff’s Office, which uses Rekognition for assistance in finding “persons of interest” using existing mugshots.
“There are nearly 20,000 different bookings (when a person is processed into the jail) every year,” wrote Chris Adzima, an information systems analyst in the Washington County Sheriff’s Office, in a blog post last year. “As time passes, officers’ memories of individual bookings fade.”
The office also uses Rekognition to investigate Amber alerts and Silver alerts, which respectively aim to help track down missing children or senior citizens, respectively.
In response to some of these controversies, Amazon has also touted the technology’s work for organizations that work with law enforcement to advocate for crime victims. Those include the International Centre for Missing & Exploited Children and anti-human trafficking organization Marinus Analytics, according to an AWS spokesperson.
