Top referrers?

MsMittens · Post by **MsMittens** » Tue Mar 16, 2004 3:59 pm

Hrmm... I've run into something that I'm not sure if it's a bug or perhaps I need to finish configuring something. I noticed that someone managed to put a specific website (not referred to in any of the posts) on the top referrers (they appeared after a single night at 50 in the Top Referrers list). When I checked the logs I found the following (note that my website is www.msmittens.com):

Code: Select all

[b]This is a snippet. There is more than this[/b]

211.152.14.95 - - [15/Mar/2004:18:20:32 -0500] "GET / HTTP/1.1" 200 76526 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:20:47 -0500] "GET /index.php?/feeds/index.rss2 HTTP/1.1" 200 53129 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:20:53 -0500] "GET /index.php?/feeds/atom.xml HTTP/1.1" 200 54078 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:04 -0500] "GET /index.php HTTP/1.1" 200 76526 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:18 -0500] "GET /comment.php HTTP/1.1" 200 8008 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:24 -0500] "GET /categories/3_security_article HTTP/1.1" 404 - "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:25 -0500] "GET /exit.php?url=ahr0cdovl3d3dy5jbwe0mdauy29t&entry_id=27 HTTP/1.1" 302 5 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:28 -0500] "GET /exit.php?url=ahr0cdovl3d3dy50cmfpbnjpz2h0lmnvbq==&entry_id=27 HTTP/1.1" 302 5 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:29 -0500] "GET /exit.php?url=ahr0cdovl3d3dy5iawn5y2xpbmcuy29t&entry_id=27 HTTP/1.1" 302 5 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:29 -0500] "GET /exit.php?url=ahr0cdovl21hcnmuanbslm5hc2euz292lw==&entry_id=25 HTTP/1.1" 302 5 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:30 -0500] "GET /exit.php?url=ahr0cdovl3d3dy5jzwnpbglhemhhbmcub3jnlw==&entry_id=25 HTTP/1.1" 302 5 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:32 -0500] "GET /index.php?serendipity[page]=2 HTTP/1.1" 200 61885 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:46 -0500] "GET /rss.php HTTP/1.1" 200 54730 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:49 -0500] "GET /rss.php HTTP/1.1" 200 54717 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:21:57 -0500] "GET / HTTP/1.1" 200 76571 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:22:13 -0500] "GET /rss.php?version=atom0.3 HTTP/1.1" 200 55847 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.95 - - [15/Mar/2004:18:22:26 -0500] "GET /rss.php?version=2.0 HTTP/1.1" 200 54717 "http://www.mosel.com/pages/01_orte.jsp" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"

What is it they've done?

Post by **garvinhicking** » Tue Mar 16, 2004 6:48 pm

What you are experiencing is called "Referrer Spam". You can't do anything but to

* either patch s9y to filter certain referrers,
* disable the "top referrer" plugin or to
* only count referrers from a single host one per 30 minutes or so (configurable?)

Of course the last version is preferred, as we all could make use of it

MsMittens · Post by **MsMittens** » Tue Mar 16, 2004 9:59 pm

* only count referrers from a single host one per 30 minutes or so (configurable?)

How specifically? I've looked through the admin panel and can't seem to find it. And I've added them to the list of referrers not to be listed but it's still there.

Post by **garvinhicking** » Wed Mar 17, 2004 3:46 pm

Hi!

MsMittens wrote:
* only count referrers from a single host one per 30 minutes or so (configurable?)
How specifically? I've looked through the admin panel and can't seem to find it. And I've added them to the list of referrers not to be listed but it's still there.

Ah, I'm sorry I didn't make myself clear enough. The latest mention is not coded in s9y, I meant that you'd have to code it yourself.

The list of referrers is only for new referrers not to be included, you have to kick them out of your database by hand.

And if I were you, I'd send a complain to mosel.com. They are a german town/city/community council and shouldn't be doing such nasty things to your site.

Regards,
Garvin.

MsMittens · Post by **MsMittens** » Wed Mar 17, 2004 9:03 pm

And if I were you, I'd send a complain to mosel.com. They are a german town/city/community council and shouldn't be doing such nasty things to your site.

I would but the address that doing it (211.152.14.95) traces to China.

So I'm not even sure if it was done with their knowledge.

Stuart Tannehill · Post by **Stuart Tannehill** » Tue Mar 23, 2004 7:31 am

You, Ms. Mittens, are the security expert so I suggest this with some trepidation but it seems you could create a rule for your firewall that blocks that specific IP address. That is the first thing that comes to my mind (I use ipfw).

MsMittens · Post by **MsMittens** » Wed Mar 24, 2004 2:29 am

LOL.. I can make a firewall but kinda useless since I don't host the website.

That'd be up to my hosting service.

daFool · Post by **daFool** » Mon Jul 05, 2004 1:03 pm

We have been filtering out referrer spammers for a while but now we have a new problem. Exit spammers. There have been more than 1000 exits for particular sites that have no links from our blogs and leave no trace in logfiles.

How is this done? Is this a known bug in Serendipity or a simple feature? Some browsing thru serendipity blogs show that we are not only ones having this problem and even more serendipity users are getting referrer spammed (including some of the developers of serendipty).

Our current Blocked Referrers:
http://www.incest-taboo.net;www.bestiality-pics.org;zoo.x-stories.org;violence.x-stories.org;taboo.x-stories.org;gays.fotospornocaseras.com;
http://www.rape-stories.biz;incest.pics--movies.com;rape.pics--movies.com;www.secureroot.org;zoo.pics--movies.com;

Our current cleaner:
delete from serendipity_referrers where host='www.bestiality-pics.org';
delete from serendipity_referrers where host='www.rape-stories.biz';
delete from serendipity_referrers where host='www.incest-taboo.net';
delete from serendipity_referrers where host='zoo.x-stories.org';
delete from serendipity_referrers where host='violence.x-stories.org';
delete from serendipity_referrers where host='taboo.x-stories.org';
delete from serendipity_referrers where host='gays.fotospornocaseras.com';
delete from serendipity_referrers where host='incest.pics--movies.com';
delete from serendipity_referrers where host='rape.pics--movies.com';
delete from serendipity_referrers where host='zoo.pics--movies.com';

delete from serendipity_exits where host='www.bestiality-pics.org';
delete from serendipity_exits where host='www.rape-stories.biz';
delete from serendipity_exits where host='www.incest-taboo.net';
delete from serendipity_exits where host='zoo.x-stories.org';
delete from serendipity_exits where host='violence.x-stories.org';
delete from serendipity_exits where host='taboo.x-stories.org';
delete from serendipity_exits where host='gays.fotospornocaseras.com';
delete from serendipity_exits where host='incest.pics--movies.com';
delete from serendipity_exits where host='rape.pics--movies.com';
delete from serendipity_exits where host='zoo.pics--movies.com';
delete from serendipity_exits where host='www.linuxwaves.net';
delete from serendipity_exits where host='www.macinstruct.net';
delete from serendipity_exits where host='www.secureroot.org';

Post by **garvinhicking** » Mon Jul 05, 2004 1:24 pm

This has just been recently addressed in our CVS development branch. We now no longer only accept every URL within our exit.php script, but only submit the ID with it.

For the time being, you could patch your exit.php like this and put it into an older s9y release

Code: Select all

<?php # $Id: exit.php,v 1.5 2003/07/08 09:13:27 garvinhicking Exp $
include_once 'serendipity_config.inc.php';

$url      = $serendipity['baseURL'];
if (isset($_GET['url']) && !empty($_GET['url']) && isset($_GET['entry_id']) && !empty($_GET['entry_id'])) {
    $url = str_replace('&', '&', base64_decode($_GET['url']));

    // See if the submitted link is in our database and should be tracked
    $links = serendipity_db_query("SELECT id, link FROM {$serendipity['dbPrefix']}references WHERE entry_id = {$_GET['entry_id']}");
    $found_id = false;
    foreach($links AS $idx => $link_row) {
        if ($link_row['link'] == $_GET['url']) {
            $found_id = $link_row['id'];
        }
    }

    if ($found_id) {
        // URL is valid. Track it.
        serendipity_track_url('exits', $url, $_GET['entry_id']);
    }
}

header('Location: ' . $url);
?>

(I haven't tried the code though.

daFool · Post by **daFool** » Mon Jul 05, 2004 2:10 pm

garvinhicking wrote:This has just been recently addressed in our CVS development branch. We now no longer only accept every URL within our exit.php script, but only submit the ID with it.

For the time being, you could patch your exit.php like this and put it into an older s9y release
Code: Select all
(I haven't tried the code though. :-)[/quote]

I am now trying it. Thanks for the very fast response. 
 :D

leeps · Post by **leeps** » Tue Jul 06, 2004 12:51 am

i got this one tested:

Code: Select all

<?php # $Id: exit.php,v 1.5.1 2004/07/06 00:39:00 leeps Exp $
include_once 'serendipity_config.inc.php';

$url      = $serendipity['baseURL'];

if (isset($_GET['url']) && !empty($_GET['url'])) {

    $found_id = false;

    $url = str_replace('&', '&', base64_decode($_GET['url']));
    if ($url == "http://www.wetter.com/home/extern/ex_search.php?ms=1&ss=1&sss=2&search=33098"
     || $url == "http://www.team-iil.de"
     || $url == "http://www.lechte.net"
     || $url == "http://www.leezal.net"
     || $url == "http://www.jako.bi"
     || $url == "http://www.blogchalking.com"
     || $url == "http://www.leatheregg.com/bloggercode/"
     || $url == "http://travis.kroh.net/blogger_decoder/?code=B2%20d%2B%2B%20t%2B%20k-%20s-%20u--%20f%20i%2B%20o%2B%20x%20e%2B%20l%2B%20c"
       ) $found_id = true;

    if (isset($_GET['entry_id']) && !empty($_GET['entry_id'])) {
        // See if the submitted link is in our database and should be tracked
        $links = serendipity_db_query("SELECT id, link FROM {$serendipity['dbPrefix']}references WHERE entry_id = {$_GET['entry_id']}");
        if (is_array($links) && !$found_id) {
            foreach ($links as $idx => $link_row) {
                if ($link_row['link'] == $_GET['url']) {
                    $found_id = $link_row['id'];
                }
            }
        }
    }

    if ($found_id) {
        // URL is valid. Track it.
        serendipity_track_url('exits', $url, $_GET['entry_id']);
    }

}

header("Location: " . $url);
?>

the urls i OR are the ones which i have in plugins, because they don't have an entry_id.
the code of garvin wouldn't respect these, as it didn't fault-tolerate that $links would be no array :)
however, my solution is not perfect... i suspect urls not getting tracked if there's just one url in one entry (which would make $links a one-dim array and therefor something might screw up). i'm a bit tired now and don't want to look at this anymore :)

this should not let spam-urls through. but, as i'm a frequent target, i can tell you about wednesday evening :/

edit: typo in garvin's name :)

markl999 · Post by **markl999** » Tue Jul 06, 2004 7:39 pm

As a temp fix i create a file called refblock.inc.php with a list of all the sites to 'ignore' from the referrer list and then just added this to serendipity_functions.inc.php

Code: Select all

function serendipity_track_url($list, $url, $entry_id = 0) {
    //below added by me to block referrers
    if(file_exists('refblock.inc.php')){
        $refblock = file('refblock.inc.php');
        if(in_array($url, $refblock)){
            return;
        }
    }
    //end of temp blocking stuff

No idea if this is the 'best' way to do it, but it's working for me

daFool · Post by **daFool** » Wed Jul 07, 2004 9:40 pm

garvinhicking wrote:This has just been recently addressed in our CVS development branch. We now no longer only accept every URL within our exit.php script, but only submit the ID with it.

For the time being, you could patch your exit.php like this and put it into an older s9y release

if ($link_row['link'] == $_GET['url']) {
$found_id = $link_row['id'];
}
}

Perhaps the "if" should be ($link_row['link'] ==$url) instead of $_GET['url'])?

If the "if" stays as it is written nothing seems to be accepted...

Top referrers?

Top referrers?

Block them in your firewall

How about exits then?

Re: How about exits then?

Re: How about exits then?

Re: How about exits then?