CAPTCHA Puzzles: Bots Perform 15% Better Than Humans

The Incredible Advantage of CAPTCHA Solving by Bots is that they outperform humans by 15%

Some websites have a series of puzzles that require the users to correctly identify traffic lights, buses, or crosswalks to prove that they are indeed human before they log in. These puzzles are called Completely Automated Public Turing tests to tell Computers and Humans Apart (CAPTCHA), and they are designed to protect a website from fraud and abuse without creating friction.

Only authorized individuals and not automated intrusions are meant to be able to access the site thanks to the riddles. However, the team's technical head Aaron Malenfant warned the Verge at the time that the technique would not be practical in ten years because cutting-edge technology would allow the Turing test to run in the background. In 2019, Google replaced CAPTCHA with a more sophisticated tool called reCAPTCHA. He had predicted correctly.

Fast-evolving artificial intelligence (AI) bots are now outperforming the reCAPTCHA technique used to verify the legitimacy and personhood of visitors to various websites. They achieve this by mimicking the functioning of the human brain and eyesight. In several areas, AI bots are competing with humans and even surpassing them.

AI Bots Can Defeat Humans And CAPTCHA, Based on Research.

The effectiveness of AI-automated attacks against multiple CAPTCHA puzzle methods is now reported in a research article (pdf) that was released last month but has not yet undergone peer review. The study, which was carried out by a collaboration of academics from the University of California, Irvine, ETH Zurich, Lawrence Livermore National Laboratory, and Microsoft, revealed that AI bots are now more effective than humans in CAPTCHA tests.  Even more, than the bots that CAPTCHAs are designed to prevent, they give the appearance that people are more like machines. Furthermore, they work considerably more quickly.

To test websites that employed CAPTCHA puzzles, which are present on 120 of the top 200 websites in the world, the researchers enlisted 1,400 volunteers. "The accuracy of the bots is often above 96%, and it ranges from 85 to 100%. This much exceeds the human accuracy range we measured (50-85%)," according to the study. The human solving time for reCAPTCHA is 18 seconds, which is almost identical to the bots' time of 17.5 seconds. Otherwise, the bots' solving durations are much shorter in every scenario. The study was conducted using a variety of CAPTCHAs on MTurk, Amazon's platform for crowdsourcing work, including some requiring users to write warped text and rotate images to recognize chimneys and boats.

In a contextualized situation, human solving time decreases to 22 seconds, according to the report. This indicates that AI bots are quicker than humans in this more realistic setting.

"We can say with certainty that people hate taking the tests. Gene Tsudik, one of the study's researchers, told the New Scientist that the team didn't need to conduct a study to reach that conclusion. "But people don't know whether that effort, that colossal global effort, that is invested into solving CAPTCHAs every day, every year, every month, whether that effort is worthwhile."

The challenge with creating better CAPTCHAs, according to Cengiz Acartürk, a cognitive and computer scientist at Jagiellonian University in Kraków, Poland, is that they already have a limit. People abandon if something is too tough, said Acartürk. Whether or not adding CAPTCHA puzzles to a website is worthwhile may ultimately rely on whether or not the next step is so crucial to the user's experience that a challenging challenge won't deter users while offering an adequate level of security.