The top 6 FAQs about facial recognition


Written by:

How facial recognition works. Via

With the FTC currently focusing on facial recognition, we figured it was a good time to  provide answers to the 6 biggest questions about this technology and how it affects you.

1.  What is facial recognition?

2.  How is facial recognition being used today?

3.  Can I be recognized?  How?

4.  What can I do about this?

5.  What are the risks of facial recognition?

6.  What are the benefits?

A stranger snaps a picture of you with his iPhone camera while you’re walking down the street on your way to work in the morning. You don’t see him. He uses a mobile app to analyze the photo he just took of you. The app scans your face and searches the web for matches. In less than a minute, the stranger knows your name and contact info, is scrolling through your Facebook albums, and is reading your Twitter timeline. Predicting your social security number—and thus stealing your identity–is one small step away.

Your face is your identity:  you should have control over it.

Sounds like science fiction? It’s science fact. Say hello to the faceprint, the facial equivalent of the fingerprint. The identifying characteristics unique to our bodies are called biometrics. Iris patterns, faces, and fingerprints are common examples. Once you have a faceprint, you can compare it to a database of faces and look for a match. This is the world of facial recognition technology, and it’s happening right now.

Frequently Asked Questions about Facial Recognition

1.  What is facial recognition?

When we talk about facial recognition, we’re really talking about two different applications:  basic and advanced.  Basic facial recognition answers the question, “Is this a face?”  You see this kind of technology in photo-editing software.  Apple’s iPhoto is notorious for identifying non-faces, like cookies and animals, as faces.  If it has eyes, a nose, and a mouth, there’s a good chance it’s a face.

Who IS this guy?

Advanced facial recognition builds on these principles to answer the question, “Is this a particular face?”  As anyone who’s ever used a character builder in a video game can tell you, our unique faces are comprised of variations on several main features:  the width of our nose, the wideness of our eyes, the depth of our jaw, the height of our cheekbones, and the distance between our eyes are a few of them.  Facial recognition software takes your features and turns them into a numerical code.  Compare this code, or faceprint, with any database of photos, and you can start making matches and identifications.

2.  How is facial recognition being used today?

Facial recognition is alive and flourishing.  It’s used in many broad areas, including social networking, photo editing, security, law enforcement, casinos, and in odd places that you might not expect.   For example, the dating website based matchmaking around the principle that people with similar facial features are attracted to each other, using facial recognition to match user photos, and matches potential dog owners with canines that look like them.  Face recognition cameras scanned all the fans walking through the turnstiles at Super Bowl XXXV, now referred to as the Snooper Bowl, running the scans against a database of criminal mugshots.  That was a decade ago, when the internet was still in its relative childhood.  We’re in an age now when Facebook collects 100-page dossiers on all of us, when ad networks track everything we do online, when companies buy and sell our contact information:  the street we grew up on, the names of our family members, aerial shots of our homes.

SceneTap’s interface. The app lets you check the gender ratio of a bar before you head over.

Lots of mobile apps use facial recognition, too.  A particularly interesting one, SceneTap, tracks the ratios of males to females and ages at 250 participating U.S. bars.  These bars install face-detection cameras, and the app calculates the number of people at the bar, the male-to-female ratio, and the average age of patrons.  SceneTap doesn’t receive bar patrons’ permission to capture their faces and share demographic information about them.   Another notable app is FACER Celebrity, made by Animetrics Inc., a facial-recognition company based in Conway, N.H., that focuses on the law-enforcement and security industries.  FACER Celebrity is a free iPhone app that allows users to match their face to a star.  The app, which has about 30,000 downloads, uses the same facial-recognition technology deployed by local law enforcement to identify criminal suspects, says Animetrics CEO Paul Schuepp.

Companies give two main reasons for using facial recognition technology:  it helps with security, and it makes photo editing and sharing easier.  On the security side, law enforcement officials have argued that facial recognition can help find missing people, identify criminals in a crowd, preempt terrorists from boarding planes with fake passports.  It’s also used for private security in casinos to identify card counters and kick them out before they can win too much.  Casinos also say their systems identify people with gambling addictions who’ve asked casinos to forcibly remove them if they can’t stop themselves.  Even supermarket security uses facial recognition:  one grocery chain in the UK uses facial recognition to stop underage customers from buying alcohol.

On the photo sharing side, facial recognition can scan albums for faces and either suggest tags or automatically tag people.  You’ll already find it in Apple’s iPhoto, Google’s Picasaweb, Microsoft’s Windows Live Photo Gallery, and other photo editors.  It also collects information on different people’s faces through existing tags:  the more tags, angles, lighting types, hairstyles, and other details in your photos, the better the software’s ability to pick you out in other photos.  It’s one thing to confine this technology to the photos on your own computer, but things get more complicated when the internet gets involved.

Facebook got in trouble with privacy advocates when it rolled out facial recognition by default.  It’s since dialed it back to “Tag Suggestions,” which you can choose to disable.  Even if you disable it, though, Facebook still collects information about your face whenever it’s tagged.  And when you consider that Facebook’s 600 million members upload over 250 million photos every day, you see that they’re building an empire of facial data.  Rumor has it they’re building a way to search for people by picture alone.  And Google’s Goggles app can already identify inanimate objects through photographs.  Add already-existing facial recognition software to that, and you could “identify strangers on the street.”

We recommend disabling tag suggestions in Facebook.

3.  How accurate is it?

The accuracy rate is 99.31% on still frontal face images.  Changes in lighting, face positioning, makeup, hairstyle, facial hair, glasses, and other accessories decreases the accuracy rate.

4.  Can I be recognized?  How?

Yes, as long as you have at least one picture of your face publicly available online.  And yes, social networks count as “public” in most cases.  The more photos there are of you on the internet, the greater the chances of the facial recognition software finding a match.

If someone took a picture of you while you are out on the street, they can scan it and cross-reference it against photo databases to get a match.  You might guess that the U.S. Department of State has one of the biggest facial recognition databases with over 75 million photographs, but it doesn’t come close to the largest of them all:  Facebook, with over 140 billion photos (and it will add 70 billion more this year).  That’s up from 10 billion in 2008, and it makes up 4% of all photos ever taken throughout history.  ImageShack has over 20 billion; PhotoBucket has 7.2 billion, Flickr has 3.4 billion.

Bet you didn’t realize exactly how many photos Facebook has on its servers.

You can also be found through existing photos of yourself with reverse image searches such as TinEye or Google.  Reverse image search works by looking for photo fingerprints, exact matches of existing photos.  Merely similar photos won’t show up, but cropped and differently-sized versions of the same photo will.

Let’s say you’re a member of an online dating site, and although you use a real photo of yourself, you don’t provide your real name.  If you’ve ever used that same photo anywhere else, you may be in trouble:  anyone can save that picture and do a reverse image search with it.  These searches reveal other locations where that photo can be found.  Maybe it’s your personal website, your Facebook profile, a people search website aggregating data about you, and a newspaper article.  Suddenly, that person knows far more about you than you provided in your dating profile’s “about me” section.

If you’re worried about reverse image search, here’s a tip:  use different images for different contexts.  Don’t use the same photo on your employer’s profile page that you have as your Facebook profile picture.  Keep separate photos you use for work, photos you use for family, photos you use for friends, and photos you want to keep entirely private.  That way, someone doing a reverse search with a particular image will only find a limited set of results:  the ones you’ve chosen to associate with that image.

5.  What are the risks of facial recognition?

Take the massive amount of information that Google, Facebook, ad networks, data miners, and people search websites are collecting on all of us; add the info that we voluntarily provide to dating sites, social networks, and blogs; combine that with facial recognition software; and you have a world with reduced security, privacy, anonymity, and freedom.  Carnegie Mellon researchers predict that this is “a world where every stranger in the street could predict quite accurately sensitive information about you (such as your SSN, but also your credit score, or sexual orientation” just by taking a picture.

Risk 1:  Identity theft and security

Think of your personal information—name, photos, birthdate, address, usernames, email addresses, family members, and more—as pieces of a puzzle.  The more pieces a cyber criminal has, the closer he is to solving the puzzle.  Maybe the puzzle is your credit card number.  Maybe it’s the password you use everywhere.  Maybe you’re your social security number.

Identity thieves often use social security numbers to commit fraud. Photo:

Facial recognition software is a tool that can put all these pieces together.  When you combine facial recognition software with the wealth of public data about us online, you have what’s called “augmented reality:”  “the merging of online and offline data that new technologies make possible.”   You also have a devastating blow to personal privacy and an increased risk of identity theft.

Once a cyber criminal figures out your private information, your money and your peace of mind are in danger.  Common identity theft techniques include opening new credit cards in your name and racking up charges, opening bank accounts under your name and writing bad checks, using your good credit history to take out a loan, and draining your bank account.  More personal attacks may include hijacking your social networks while pretending to be you, reading your private messages, and posting unwanted or embarrassing things “as” you.

The research:  how facial recognition can lead to identity theft

Carnegie Mellon researches performed a 2011 facial recognition study using off-the-shelf face recognition software called PittPatt, which was purchased by Google.  By cross-referencing two sets of photos—one taken of participating students walking around campus, and another taken from pseudonymous users of online dating sites—with public Facebook data (things you can see on a search engine without even logging into Facebook), they were able to identify a significant number of people in the photos.  Based on the information they learned through facial recognition, the researchers were then able to predict the social security numbers of some of the participants.

They concluded this merging of our online and offline identities can be a gateway to identity theft:

If an individual’s face in the street can be identified using a face recognizer and identified images from social network sites such as Facebook or LinkedIn, then it becomes possible not just to identify that individual, but also to infer additional, and more sensitive, information about her, once her name has been (probabilistically) inferred.

Some statistics on identity theft from the Identity Theft Assistance Center (ITAC):

  • 8.1 million adults in the U.S. suffered identity theft in 2011
  • Each victim of identity theft loses an average of $4,607
  • Out-of-pocket losses (the amount you actually pay, as opposed to your credit card company) average $631 per victim
  • New account fraud, where thieves open new credit card accounts on behalf of their victims, accounted for $17 billion in fraud
  • Existing account fraud accounted for $14 billion.

Risk 2:  Chilling effects on freedom of speech and action

Facial recognition software threatens to censor what we say and limit what we do, even offline Imagine that you’re known in your community for being an animal rights activist, but you secretly love a good hamburger.  You’re sneaking in a double cheeseburger at a local restaurant when, without your knowledge, someone snaps a picture of you.  It’s perfectly legal for someone to photograph you in a public place, and aside from special rights of publicity for big-time celebrities, you don’t have any rights to control this photo.  This person may not have any ill intentions; he may not even know who you are.  If he uploads it to Facebook, and Facebook automatically tags you in it, you’re in trouble.

Anywhere there’s a camera, there’s the potential that facial recognition is right behind it.

The same goes for the staunch industrialist caught at the grassroots protest; the pro-life female politician caught leaving an abortion clinic; the CEO who has too much to drink at the bar; the straight-laced lawyer who likes to dance at goth clubs.  If anyone with a cell phone can take a picture, and any picture can be tied back to us even when the photographer doesn’t know who we are, we may stop going to these places altogether.  We may avoid doing anything that could be perceived as controversial.  And that would be a pity, because we shouldn’t have to.

Risk 3:  Physical safety and due process

Perhaps most importantly, facial recognition threatens our safety.  It’s yet another tool in stalkers’ and abusers’ arsenals.  See that pretty girl at the bar?  Take her picture; find out everything about her; pay her a visit at home.  It’s dangerous in its simplicity.

There’s a separate set of risks from facial recognition that doesn’t do a good job of identifying targets:  false identifications.  An inaccurate system runs the risk of identifying, and thus detaining or arresting, the wrong people.  Let’s say that an airport scans incoming travelers’ faces to search for known terrorists.  Their systems incorrectly recognize you as a terrorist, and you’re detained, searched, interrogated, and held for hours, maybe even arrested.  This is precisely why Boston’s Logan Airport abandoned its facial recognition trials in 2002:  its systems could only identify volunteers 61.4 percent of the time.

6.  What are the benefits of facial recognition?

Sometimes facial hair throws off facial recognition software. Too bad there’s not mustache recognition software. Photo:

Facial recognition has its benefits, especially for certain groups and activities.

Benefit 1:  Easier photo organization

Facial recognition boasts the end of long tagging sessions.  If you upload all your photos and facial recognition software identifies and tags your friends, most of your work is done.  Even less powerful software that only identifies and suggests tags, rather than tags automatically, can save a lot of time.

Benefit 2:  Greater access to information

If a picture’s worth a thousand words, then a picture associated with a database of internet data is worth an encyclopedia.  You could learn all about someone with a single photo.  This could be helpful for people who need a lot of information in a short period of time:  for example, if you want to learn about a potential date, get some background on a job applicant, or find a long-lost friend or relative.

Facial recognition applications could have special significance for those with prosopagnosia, also known as face blindness.  This condition makes it difficult or impossible to identify people.  You may have met someone a hundred times, but if you have severe prosopagnosia, you won’t recognize him or her.  And for those of us who just have trouble remembering faces, this wouldn’t be a bad thing, either.

People with severe prosopagnosia can’t recognize faces. Photo:

Benefit 3:  Criminal identification

If you have a database of known criminals, you can use facial recognition to cross-reference those faces against other databases of faces. With the right technology, you could scan a public area for wanted felons, for example, or see if any of them have signed up for Facebook.  Even if you can try to hide from a criminal past by changing your name, it’s much harder to change your face.  Thus facial recognition is already a valuable tool in law enforcements’ arsenal, and it’s only becoming more important.

Benefit 4:  Money, money, money

If you’re an advertiser, there’s money in facial recognition.  Photos are the key to unlocking a vault of valuable personal information.  The more they know about a person or a segment, the better they can tailor their ads to target them, and the more successful these ads will be.  Your data is worth its weight in gold.

What do you think about facial recognition?

Is it cool?  Scary?  A little of both?  What would you like to see happen with it in the future?  Leave your thoughts in the comments box below.

About Abine:  Abine, Inc., The Online Privacy Company, is the leading provider of online privacy solutions for consumers.  Abine’s products and services allow regular people to regain control over their personal information while continuing to browse, interact, and shop online.

24 Replies to “The top 6 FAQs about facial recognition”

  1. Easy facial recognition is just one of many technologies becoming available that increase information accessibility. Generally, this is a good thing, but unfortunately most lawmakers (who don’t fully grok our increasingly heterogenous culture) have little understanding of the technology and how it’s applied to make good judgments when it doesn’t simply place that judgment call in a government agency’s hands.

    Scary/exciting times.

    • Sarah Downey says:

      Yeah, it’s got both positive and negative uses. We can’t completely eliminate it, nor should we. It’s just a question of minimizing those scary uses, the ones that would threaten free speech and association, while maximizing helpful ones.

  2. Bung says:

    A little of both. Thank you for a very well written and informative article.

  3. Marie Wilson says:

    I know of someone who got as far as his degree and masters in a different name along with multiple supporting documents. successfully got a learners drivers licenses in fake name and was finally caught when he went for his final driving test. This was a goverment employeee who had been hired in his fake identity. So it is good but it is also scary. I liked the information about using different photos online for different things.

  4. Recent Blogroll Additions…

    I saw this really good post today….

  5. Figure out how to make use of Search engine optimisation WordPress Plugin…

    […]Abine | The Online Privacy Company[…]…

  6. Nicole says:

    Ahh… THIS is why Facebook bought Instagram?! Something wasn’t adding up, but now it does….

  7. webbdesign|kungsbacka|hemsida…

    […]Abine | The Online Privacy Company[…]…

  8. turnstiles says:

    As anyone who’s ever used a character builder in a record game can tell you, our unique faces are comprised of variations on numerous main features: the width of our nose, the wideness of our eyes, the depth of our jaw, the height of our ….

  9. cool says:

    Can anyone tell me how I can download face recognition software to find my face on the internet? Which is a good one to use?

  10. Nava says:

    I want to know that if contact lenses use can the face will be identified?

    • Sarah Downey says:

      Yep–facial recognition tools analyze things like the distance between your facial features, the angle of your jaw, the width of your face, etc. to get faceprints. The color of your eye won’t matter; it’s the shape of your facial features that does.

  11. This is a sad state of affairs. The human race being lured into a world of rampant technology where international goverments epecially the USA can gather information about us. Personally I will not use my face in Face Book or post pics of myself. Why? What did we all do before we hadf all this wonderful technology?

  12. Mrs. Spang says:

    hey i have a question my husband has a daughter by another women years ago and i’m trying to convince him to get a dna test how do i compare to photos to show him that she doesn’t look like him and that she looks like another guys daughter not his? so he will be more willing to get a dna test.

  13. dfgfsdsdfsdfdfs says:

    Let’s say it straight – maybe in USA You have some form of privacy(at least on paper 😉 ), but in European, control-people zones You must often have at least one id card. And They require You to give the photo, which often “must be” computer-ready…

    But if there is a system, there is often a catch – professional masks are not so hard to find/buy and probably most people can hide from the system(or they can’t arrest all)…

  14. divya says:


    How do System Identify a face in image

  15. Me says:

    Well I got a good Idea what I can do against that. I just use my photo, and create lots of fake Profiles with fake Names. Good luck then finding the right one 😉


  16. budyk says:

    what a great reading…..nice

  17. TD says:

    I think you need to leave the poor man and his daughter alone. From woman to woman you don’t tell a kid who loves their father, we found out your daddy isn’t your daddy, and you don’t tell the father who loves his child that it’s not his child, unless the father wants to know himself and is looking for answers. This is his business. It’s usually too late, especially when you say “years” meaning the child is years old. At this point, you’re just ruining lives for your own selfish reasons. Think if that little girl was you. This isn’t like telling her Santa isn’t real… DNA testing needs to be done in the beginning, after childbirth, or otherwise it’s pointless. If the man loves this little girl, then you seriously need to back off. Children get adopted every day and men become fathers that way. It doesn’t take blood to love. If he sees himself as her father, let it be.

  18. We stumbled over here coming from a different website and thokught I should check things out.
    I like what I seee so i am just following you.
    Look forward to finnding out about your web page

  19. akbar says:

    what a great reading…..nice

  20. Jane says:

    When ever I go shopping. I’m constantly being watched by security. I am NOT a criminal. I have never broken the law. It’s makes me feel very uncomfortable. How can I find out why there is a need to surveillance me. I’m a 53 year old veteran not a criminal

  21. Linda says:

    Jane, I am having the same problem. I am also watched when I go in certain restaurants, & other stores. I would like to know who put my face in the system, why, & how do I get it taken off? I am LIVID!!! When you compromise innocent people’s privacy & portray them as criminals, it should be a crime. I want to sue the person who did this to me.

Leave a Reply