Browser automation will occur by executing events in the DOM or by calling properties of the page/window. It’s all JavaScript designed for user interaction executed by a bot.
The one event that cannot be automated is cursor movement/position. Put a check into your event handlers that check that the cursor is actually over the event target.
You are right every testing solutions out there push UIEvents to the page rather than clicking with an actual mouse. That's why puppeteer, selenium etc are scraping tools not testing tools
Mobile users will use touch events instead of click events and likely your interface will be different and the screen width will be different. Check for these things along with keywords from the user agent string to determine mobile users from other users.
Return key on a control in a form will fire a submit event. Check for cursor position in your submit handler.
Tell freemium users what is the acceptable rate for requests per second. Publish the allowable rate on the website. Ban freemium user IPs that exceed the allowable rate. This can be done using a proxy.
100 requests/second isn't that much, especially if you're fronting your website with Cloudflare. Do you have some unauthenticated endpoint(s) that eat up a ton of server CPU?
We tried removing "async" -- thinking it would force sequential processing -- but it unexpectedly seemed to cause parallel processing of requests, which caused CUDA memory errors.
Before removing "async", this is the weird behavior we observed:
* Hacker blasts 50-100 requests.
* Our ML model processes each request in normal time and sequentially.
* But instead of returning individual responses immediately, the server holds onto all responses -- sending responses only when the last request finishes (or a bunch of requests finish).
* Normally, request 1 should return in N seconds, request 2 in 2N seconds, but with this, all requests returned in about N50 seconds (assuming batch size of 50).
1. Any suggestions on this?
2. Mind clarifying how sync vs aync works? The FastAPI docs are unclear.
Any chance the entire thing can be offloaded to a task queue (Celery/etc)? This would decouple the HTTP request processing from the actual ML task.
The memory errors you're seeing could suggest that you may not actually be able to run multiple instances of the model, and even if you could it may not actually give you more performance than processing sequentially.
Seems like ultimately your current design can't gracefully handle too many concurrent requests, legitimate or malicious - this is a problem I recommend you address regardless of whether you manage to ban the malicious users.
@headlessvictim2 search for "Asynchronous Request-Reply pattern" if you want more information about this kind of architecture. You will remove any bottleneck from the API server and can easily scale out from the task queue.
You would still have the same bottleneck but the API request would return straight away with some sort of correllation ID. Then the workers that handle the GPU bound tasks would pull jobs when they are ready. If you get a lot of jobs all that will happen is the queue will fill up and the clients will wait longer and hit the status endpoint a few more times.
Python async is co-operative multi-tasking (as opposed to per-emptive)
There is an event loop that goes through all the tasks and runs them.
The issue is the event loop can only move on to the next task when you reach an await. So if you run a lot of code (say an ML model) between awaits no other task can advance during this time.
This is why it is co-operative, it is up to a task to release the event loop, by hitting an await, so other tasks can get work done.
This is fine when you have async libs that often hit awaits at things that are IO related like say db, or http calls.
FastAPI will spawn controllers that are not defined as async functions on a thread pool but it is still a python so GIL and all that.
You should do as the sibling comment says and decouple your http from your ML and feed the ML with something like Celery. This way your server is always there to respond to things (even if just a 429) to hit a cache or whatever else.
Just add the captcha only for requests coming from the problematic ASNs, like AWS.
edit: Actually, since you use CF, just make a firewall rule that forces the captcha for those ASNs before it even gets to your app. They have a field named "ip.geoip.asnum" for that, and an action called "challenge" which will force a captcha.
This could have major GDPR implications if that's something the parent cares about. ReCaptcha is basically Google spyware that happens to provide captcha services.
The freemium service provides access to compute-heavy machine learning models running on GPUs.
Hackers blast 50-100 requests in the same second, which clog the servers and block legitimate users.
We reported IPs to AWS and use Cloudflare "Super Bot Fight Mode" to thwart attacks, but the hackers still break through.
We don't require accounts, but could impose account requirements if this helps.
Any suggestions?