How the Merlin Bird Photo ID looks like. Image: Cornell Lab of Ornithology

A group at the Cornell Lab of Ornithology developed a sophisticated facial recognition software specially designed to identify birds for photos. Called the Merlin Bird Photo ID, the software works its magic by employing a combination of image recognition algorithms, deep learning techniques (so it learns from its mistakes and gets better in time) and human collaborators who upload photos and help the software by first identifying the key features that makes a species distinct. The team is now working at turning the software into an app, so that anyone with a smartphone can take photos of a lingering avian wonder then instantly come to know which species it is.

So far, Merlin can successfully identify up to 400 bird species, all native to North America.

“Asking computers to identify bird species is a challenge not only because some species look so alike, but also because their shape varies from moment to moment. On top of that, photographs of birds often include complex backgrounds, and the birds may be far away or blurry,” explains the Cornell Lab.

First, a user uploads a photo of the bird, when and where the photo was taken so the software can filter out species that shouldn’t belong in that place or during that season. The user then has to draw a box around the bird itself to distinguish it from its surrounding, and select the bill, eye, and tail – the most prominent features that professional birdwatchers first look at to recognize a species. Shifting through millions of data points and scouring each pixel – all within seconds – the software retrieves the correct species 90% of the time.

“Computers can process images much more efficiently than humans – they can organize, index, and match vast constellations of visual information such as the colors of the feathers and shapes of the bill,” said Serge Belongie, a professor of Computer Science at Cornell Tech. The state-of-the-art in computer vision is rapidly approaching that of human perception, and with a little help from the user, we can close the remaining gap and deliver a surprisingly accurate solution.

Birdwatching using binoculars might be a lot funner, but using a phone might be a lot easier for newbies. Image: Flickr

To compare records, Merlin uses a massive database of 70 million bird photos uploaded by bird enthusiasts on eBird.org, but also thousands of images supplied by other keen birdwatchers. As more and more data is fed into Merlin, it should not only be able to identify birds more accurately, but also more of them. There are more than 10,000 species of birds in the world, and biologists regularly discover new species. The ultimate goal is to have this turned into an app for your smartphone. It would not only be entertaining, but also educational, maybe prompting awareness (one in eight birds are threatened by pesticides and climate change) and inspiring a new generation of biologists and ornithologists. For plants, image recognition works a lot easier given there are some 21 apps which can be downloaded to identify plants based on pics taken with your phone.