CoderZone.org
Pages: 1 « previous     next »
  Print  
Author Topic: SPAM and bots  (Read 8880 times) Bookmark and Share
Max
Jr. Member
*****
Posts: 75



View Profile WWW
« on: Jan 14, 2011, 09:14:55 am »

Forum spam? You probably won't see it here.  Smiley

We do all sorts of tricky things to frustrate spammers and bots.

First, we use time gating on the registration page- if the page is filled out too fast ( faster than a human is likely able to do), it's probably a bot and the registration fails. If it takes too long (a couple of minutes or more), it fails, because it's probably a bot saving the page so it can come back and submit it later. Time gating is used on some other pages as well.

We use some masked & hidden fields in the registration page and certain other pages. Humans can't see those fields so if they're fields are filled in, it's a bot. Whatever page is being submitted is failed or discarded.

We also use BotScout to pre-screen registrants. If you're in the Botscout database your registration just fails with cryptic errors, over and over and over.  Grin

Some fields in your forum profile aren't able to be saved unless you've made a certain number of posts. Human spammers who successfully sign up will have to make 5 or 10 posts before they can puts links and stuff in their profile. Most of them will never bother; it's just too much work for them.

Also, leaving external links in posts is disallowed until a user reaches a certain number of posts. You can leave links to places here on the site (like a snippet or the sandbox), but posting an link external returns a polite error message. Similar to the forum profile fields, it's just too much work for most spammers to slog through.

There are a few other things we do (special, uber-tricky stuff), but those are the main impediments to spammers and bots. Most of it is designed to be transparent to real users and shouldn't get in the way of using the forum.


Logged
Max
Jr. Member
*****
Posts: 75



View Profile WWW
« Reply #1 on: Jan 16, 2011, 07:52:15 am »

It looks some of our safeguards are working as they're intended...it didn't take long for the bots to find us. Now we're seeing a steady stream of reports from some of the anti-bot code indicating that lots of mysterious users are failing the registration process because (for example) they're filling out the registration form in under a second or two....far too fast for an actual human to be able to do it.  Grin

« Last Edit: Jan 16, 2011, 07:54:59 am by Max » Logged
UnrealEd
Newbie
*
Posts: 22



View Profile
« Reply #2 on: Jan 16, 2011, 11:46:45 am »

Haha, brilliant!

Never thought of the time-gating. Quite interesting...

How do you determine the max and min value? I'm currently working on a contact form for one of my clients, and was thinking of adding this feature. The min value is rather easy, I just type as fast as I can, using autocompletion. But the max time is tricky, atm I have it set to an hour, but that's just a wild guess + if you don't want to contact someone within an hour, just don't contat him/her.
Logged
Max
Jr. Member
*****
Posts: 75



View Profile WWW
« Reply #3 on: Jan 16, 2011, 09:37:06 pm »

The minimum time was set the same way you did, by filling in the form as fast as possible several times and setting it below the average time. Realistically the registration form here should take someone at least 12 to 15 seconds to fill out. Anything less than that is suspicious at best.

The max time was set to a couple of minutes. If they take longer than that, they'll just have to go back and do it again. I think an hour is way too long, the most I'd want to let a form age is maybe 10 minutes or so at the max.

For your form I'd set a meta-refresh time of 5 or 10 minutes and have it forward to the front page of the site, or perhaps a "Please try again" page...if they walk away then after a while it just goes there and they get a nice message telling them to please try again.

Today alone we've gotten about ~30 failed attempts, all for going to fast. The bot herders don't want to waste time so they hammer the form as fast as they can. The next step would be to fail the form but make it look like it succeeded, displaying a fake confirmation page (but not actuallyregistering the user). Maybe I'll add that if I have some time this week. Smiley



Haha, brilliant!

Never thought of the time-gating. Quite interesting...

How do you determine the max and min value? I'm currently working on a contact form for one of my clients, and was thinking of adding this feature. The min value is rather easy, I just type as fast as I can, using autocompletion. But the max time is tricky, atm I have it set to an hour, but that's just a wild guess + if you don't want to contact someone within an hour, just don't contat him/her.
Logged
Max
Jr. Member
*****
Posts: 75



View Profile WWW
« Reply #4 on: Jan 17, 2011, 03:01:30 pm »

My my, the bots sure do want in here, lol. They're failing the time-gating, both long and short as well as filling in the hidden fields. Tsk, tsk. They also apparently don't understand teh Javascripty thingies. And Botscout has killed off about 70 attempts all by itself over the last 2 days.
Logged
bizzar
Newbie
*
Posts: 19



View Profile
« Reply #5 on: Jan 19, 2011, 11:40:11 am »

That time-gating sounds great, definitely something I haven't heard of before.
Logged
Max
Jr. Member
*****
Posts: 75



View Profile WWW
« Reply #6 on: Jan 19, 2011, 02:32:40 pm »

That time-gating sounds great, definitely something I haven't heard of before.

It's definitely effective. Smiley

I'm probably not the first person to come up with the idea, but I was trying to think up creative ways of foiling bots one day and it occurred to me that they probably operate on the same principle as spammers do in general- "as much as they can, as fast as they can".

So I figured if a form is being filled out and submitted too fast to be a human, it's gotta be a bot.

After a little testing on some bait forms I found that bots will either complete and submit a form in a second or two or else there's a delay of an hour or more. Some do appear to save the form and come back later to submit it. I don't know if it's being saved for a human to process or examine, but sometimes there was a looooong delay between the form being generated and the time it was submitted.

After that it was a simple matter to stick a time stamp field in the form and look at it after it's submitted. Too fast or too slow and it just gets failed. A few of the forms I use have the time stamp inserted via javascript, which cuts out a whole class of bots, namely the ones that don't process javascript. All they see is an empty field.

I suppose the day will come when the time stamp will have to be salted and encrypted to prevent bots from inserting their own time stamps, but that's a ways off. They'll have to figure out what field it is and what the minimum time is. So far it seems to have stopped all of them.

The next thing I'm looking at is a dynamic form field name for the time stamp that changes every time the form is generated. The name would be encrypted and then placed in another field so the processing code can decrypt it to figure out what the name of the field is that it should look at to find the time stamp. Kind of convoluted, but it's looking like it'll be necessary eventually. Sad

Of course, even after they manage to register, there's still not much they can do. They can't make 5 or 10 useful posts so they get over the link-posting limit, but they don't know that so they keep trying and trying. I got 6 or 7 notifications already today with a "FAILED: Too Fast" subject line, lol.
Logged
Max
Jr. Member
*****
Posts: 75



View Profile WWW
« Reply #7 on: Jan 20, 2011, 08:07:40 pm »

Lol. Had quite a few of these pouring in today (about 20 total, from a few different IPs):

Registration Fail: Too Slow (250 secs - 121.96.38.55)
Registration Fail: Too Slow (168 secs - 121.96.38.55)

As a bot, 4 minutes is an in-between length of time to play with a form. Usually it's a few seconds or an hour or more. Maybe it's trying something adaptive. (??)

However, as a human, if you take over 4 minutes to fill in the registration form, you're probably too slow to participate meaningfully in the forum. Smiley
Logged
cuberat
Newbie
*
Posts: 40


View Profile
« Reply #8 on: Jan 26, 2011, 02:44:13 pm »

One of my sites got attacked by bots.  I added bot blocking code (http://botscout.com), and a reCAPTCHA extension.

I'd like to write a script that accepts the bounce emails, extracts the information (IP address, username, and email), and creates links that can be used to submit them to a bot service.  It would require human review, since many people misspell their email addresses.
Logged
cuberat
Newbie
*
Posts: 40


View Profile
« Reply #9 on: Jan 31, 2011, 06:19:28 am »

I guess I'm naive - I don't understand why bots attack sites.  Are they trying to gain access to the data?  Are they just trying to waste space?  I get tons of hits on my blogs which are obviously not humans, and they have referral links that are not related topics.  Why bother?

I can understand the attackers that probe for specific vulnerabilities, and then install malicious code - that has a clear purpose.  But why register in an application?
Logged
Max
Jr. Member
*****
Posts: 75



View Profile WWW
« Reply #10 on: Jan 31, 2011, 07:29:23 am »

Bots have a couple of different "goals", for want of a better word:

1) Most of the time they just want to drop their spammy links to get a better rank in the search engines. They'll drop links in posts, but they'll often do "silent spamming" by registering and leaving links in their forum profile. This accounts for 99% of their entire reason for existence- gaming the search engines.

2) Sometimes they want to hunt for vulnerabilities in forms, many of which can't be accessed unless they register (so they can get to forms deeper in the site, like a posting form).

3) Sometimes it's an extended SEO campaign, like those run by software such as Xrumer. From wikipedia:

"XRumer is a Windows blackhat SEO program that is able to successfully register and forum spam with the aim of boosting search engine rankings.  ...  the software can avoid the suspicions of forum administrators by first registering to make a post in the form of a question which mentions the spam product ("Where can I get...?"), before registering another account to post a spam link which mentions the product. The side effect of these innocent-looking posts is that helpful forum visitors may search on a search engine (e.g. Google) for the product and themselves post a link to help out, thus bolstering the product's Google stats without falling afoul of forum posting policies."

Sometimes the "helpful" forum visitor posting a response is a bot too, although often it's an unknowing member just responding normally.

Since most bots are mindlessly stupid, they'll end up trying to post to any form regardless of whether or not it would makes sense. It's why you'll often see a search form hit 500 times with nonsense data. But they're getting better and better. Some of them now recognize the kinds of forms that they "want" and avoid the ones that don't meet their criteria. (This is done mainly to avoid wasting their processing time, not for the sake of courtesy.)

 I get tons of hits on my blogs which are obviously not humans, and they have referral links that are not related topics.  Why bother?

It's all about the links, and how they translate into higher search engine rankings.
« Last Edit: Jan 31, 2011, 07:33:01 am by Max » Logged
Tags: security 
Pages: 1
  Print  
 
Jump to: