Making a privacy-friendly CAPTCHA system
When we want to prevent bots from performing somekind of action on the web, be it authentication, commenting, or what-have-you, CAPTCHA is one of the classic go-to solutions for web masters. Short for Completely Automated Public Turing Test To Tell Computers and Humans Apart, CAPTCHA serves challenges that are easy for humans to solve, but hard for computers.
Classic CAPTCHA systems consisted of simple letters and numbers generated by the server, however computers armed with OCR (Optical Character Recognition). For example, this GitHub repo offers Python libraries for breaking different kinds of classic CAPTCHA: GitHub - nladuo/captcha-break: captcha break based on opencv2, tesseract-ocr and some machine learning algorithm.
Newer CAPTCHA systems like hCAPTCHA and ReCAPTCHA (shown above) are much more resistant to attack. So much so, that attackers would rather pay a few pennies for workers in developing countries to solve CAPTCHAs for them than try to break them computationally. Check out Anti Captcha: Captcha Solving Service. Bypass reCAPTCHA, FunCaptcha Arkose Labs, image captcha, GeeTest, HCaptcha. to see how this works.
But this new generation of CAPTCHA technology comes with 2 drawbacks:
- JS, rather than server-side
Users invested in privacy, such as those using Tor Browser, often disable JS completely. Most modern CAPTCHAs won’t work this way, instead relying on frontend JS for a smooth user experience.
- Dependent upon third-party APIs
Instead of fully integrating on-prem with your server, like classic CAPTCHA, modern CAPTCHAs developed in response to Google’s ReCAPTCHA are third party APIs. Some offer enterprise on-prem solutions, but for us non-enterprise normies we just have to integrate with their 3rd party.
But websites also want to trust their CAPTCHA system, rather than just depending on a third party API which would be spying on their users.
Okay, so for a CAPTCHA system to be trustworthy, frontend JS must not be mandatory (although we can offer it as an option, for users who want it!), and users must be able to host it themselves (again, only if they want to - offering the CAPTCHA as an API is a fine addition though).
We’re still missing one ingredient. Modern CAPTCHA systems following these principles have been developed, most notably by prominent marketplaces on Tor. But they never release the source code, meaning every time a new site comes out, they’re left to implement this from scratch.
We’re gonna make it Free and open-source software - Wikipedia.
Building our CAPTCHA system
What’s something simple we can do on the backend, that would be nontrivial for bots to defeat? Well, I’ve always liked GitHub’s CAPTCHA based on rotating images:
And hey, rotating images should be pretty easy, right? I mean, we can write a function to generate a few rotations of an image, so we can ask the user which one is rotated “correctly”. First, the code to actually do the rotation:
def rotate_img(img_path, angle):
original_img = Image.open(img_path)
rotated = original_img.rotate(angle)
buffered = BytesIO()
rotated.save(buffered, format="PNG")
b64_rot = base64.b64encode(buffered.getvalue())
return b64_rot.decode("utf-8")
Great, with that function, we can create n rotations (as the user requests) and return the images. We use base64, by the way, since that will be the easiest way to load the images into an HTML document for the user to see later on. Anyway, time to generate a bunch of different rotations and also note the correct one!
# Angles we'll rotate the original by
# when we create n rotations
def calculate_angles(n):
return list(range(0, 360, 360 // n))
def captchafy(img_path, n=6):
angles = calculate_angles(n)
# Shuffle the images, marking the correct one as such
rotated_imgs = [[False, rotate_img(img_path, angle)] for angle in angles]
rotated_imgs[0][0] = True
random.shuffle(rotated_imgs)
# Recover the index of the correct image
correct_img = [img[0] for img in rotated_imgs].index(True)
return correct_img, [img['image'] for img in rotated_imgs]
Great! I grabbed a free PNG off the web to see if it works, and, let’s see… first some code to call our new functions…
if __name__ == '__main__':
img_path = random_image(dir='images/')
answer, options = captchafy(img_path)
for img_b64 in options:
print(img_b64)
im = Image.open(BytesIO(base64.b64decode(img_b64)))
im.show()
Run it, and…
Looks like the image is, indeed, rotating. Well that’s we want, so now we can call this step mostly done and worry about how to give this images to a web server which can serve them for humans to solve. And for bots to, hopefully, not solve.
I’ll use Flask, because we’ve already written the code in Python and Django seems like overkill for this proof-of-concept.
Integration with Flask
First, we import all the libraries we’ll need, set up the Flask boilerplate, and set up some storage for solutions to CAPTCHAs users have submitted. The final list (captcha_solved ) is where we’ll store the cookies of users who’ve solved the CAPTCHA so they can pass on to whatever is after the CAPTCHA.
from flask import Flask, render_template, request, make_response, redirect
from numpy import real
from freecaptcha import captcha
import uuid
app = Flask(__name__, static_url_path='', static_folder='static',)
captcha_solutions = {}
captcha_solved = []
Great, so if the user browses to the index of the site and hasn’t completed the CAPTCHA, we’ll want to redirect them to the CAPTCHA page. If they have solved it, we let them in:
@app.route("/", methods=['GET'])
def index():
captcha_cookie = request.cookies.get('freecaptcha_cookie')
if captcha_cookie in captcha_solved:
return render_template('index.html')
else:
return redirect("/captcha", code=302)
Great, so we’ve established the (still unwritten) /captcha endpoint as a gateway to access the / index. It would probably be wiser yet to make the CAPTCHA check a middleware to access any other endpoint, but… one step at a time!
Now let’s try to implement the actual endpoint. First, we’ll create the endpoint itself and respond to a CAPTCHA that’s been submitted by a user:
@app.route("/captcha", methods=['GET', 'POST'])
def captcha_page():
# This means they just submitted a CAPTCHA
# We need to see if they got it right
incorrect_captcha = False
if request.method == 'POST':
captcha_quess = request.form.get('captcha', None)
captcha_cookie = request.cookies.get('freecaptcha_cookie')
real_answer = captcha_solutions.get(captcha_cookie, None)
if real_answer is not None:
if int(captcha_quess) == int(real_answer):
captcha_solved.append(captcha_cookie)
return redirect("/", code=302)
else:
incorrect_captcha = True
Phew! That’s some pretty hefty business logic. But we’ve still got to serve the initial CAPTCHA as a response to the GET request! So let’s add some additional code to our function, which will also respond to POSTs when the CAPTCHA was incorrect:
# Select an image
image_path = captcha.random_image()
# Generate list of rotated versions of image
# and save which one is correct
answer, options = captcha.captchafy(image_path)
# Provide the CAPTCHA options to the web page using the CAPTCHA
resp = make_response(render_template("captcha.html", captcha_options=options, incorrect_captcha=incorrect_captcha))
# Track this user with a cookie and store the correct answer
# by linking the cookie with the answer, we can check their answer
# later
freecaptcha_cookie = str(uuid.uuid4())
resp.set_cookie('freecaptcha_cookie', freecaptcha_cookie)
captcha_solutions[freecaptcha_cookie] = answer
return resp
Wonderful. Well, that’s it, now we just need to make a captcha.html file that will give an error message when they get it wrong…
Frontend with Jinja2
Flask uses Jinja2 for HTML templates. We’ll receive the CAPTCHA options, as well as a warning to supply if the user already got a CAPTCHA wrong (this is the code for captcha.html):
<div id="root">
{% if incorrect_captcha %}
<div class="isa_error">
<i class="fa fa-error">Failed CAPTCHA, try again!</i>
</div>
{% endif %}
<div class="isa_info">
<i class="fa fa-info">Select the version of the image that has not been rotated</i>
</div>
<form method="POST">
<div class='grid'>
{% for captcha_image in captcha_options %}
<label>
<input type="radio" name="captcha" value="{{ loop.index0 }}" checked>
<img src="data:image/png;base64,{{ captcha_image }}"></img>
</label>
{% endfor %}
</div>
<center><input type="submit" value="Submit"></input></center>
</form>
</div>
Great! Let’s take a look now:
And if we select the wrong one, we should receive an error message. Do we? Let’s find out:
Yay! The final step is creating a simple index page for users who pass the CAPTCHA to move on to. Let’s try it!
Beautiful. Well, not yet, but the idea is what we’re after! So is this the answer to privacy anti-bot forever? Probably not. With a lot more work, it could become an acceptible solution for some web masters. But it’s also vulnerable to some nasty attacks.
Defeating our CAPTCHA?
The easiest way would be bruteforce. Afterall, a user can just guess the CAPTCHA and has a 1/6 chance of getting it right. So a script can just try a few times, and by the 3 try has a 50/50 shot of having already gotten the right result.
And since it’s intended to work with Tor, we can’t just ban IP’s who fail a certain number of times. Let’s try writing a script with Python to see if we can get to the index route by bruteforcing the CAPTCHA:
import requests
i = 0
while True:
payload = {'captcha':'2'}
session = requests.Session()
session.get('http://localhost:5000/')
res = session.post('http://localhost:5000/captcha',data=payload)
i += 1
if 'Congrats' in res.text:
break
print(f'CAPTCHA beat in {i} attempts')
And now we run the script:
$ python3 templates/bruteforce.py
CAPTCHA beat in 9 attempts
Yeah, defeated in about a second and a half.
To prevent this attack, we’d want to add more CAPTCHAs into additional activities. So, a CAPTCHA to access the site, then another along with your credentials during log in, and to post a comment, and so on. And we can up the number of image options to 10. And so on.
Another simple attack would be to just start manually saving the upright versions of the images in a DB and see which one matches programmatically.
The recommended mitigation here is obvious: regularly add lots of images.
Final attack worth mentioning? : AI. GitHub - ZJCV/RotNet: Image rotation correction based on DeepLearning is a machine learning library that detects rotated images. Not much we can do about this - an attacker with this can definitely defeat our measley CAPTCHA. Our best hope is to use complex, many-colored images unlike the simple dataset the AI was trained on.
What next?
So how do we continue from here? We may start with the suggestions mentioned earlier, like transferring the CAPTCHA logic to a middleware, and hardening our defenses against obvious attack vectors. But beyond that, we’d want to make the app easier to integrate into any project.
Versions in other popular languages, and with less code that’s so specific to a single platform (like Flask). The end code for this project can be found here, if you’re curious to try it out or contribute: https://git.lain.church/jesusvilla/FreeCAPTCHA
