HomePage | WikiMinion | RecentChanges | Preferences

Discuss WikiMinion here. See also [Various discussions on Thought Storms wiki]

The thought storms wiki seems to be the home page where RichardP is coordinating information about wikiminion. That wiki's temporarily locked at the moment though. -- Halz - 11th Feb 2005

Stupid question comming up: Where did all the spam go? Seems like this wiki was last spammed 4 days ago. That's way below the average. -- Manni (Feb-9-05)

Manni, it is hard to tell without access to the web server logs, but the reduction in spam might be due to WikiMinion's banlist maintenance feature. Nick accepted my offer to have WikiMinion automatically add prolific spam hosts to KayakWiki's ban list, which means if a spammer makes a significant nuisance of himself on one of the other wiki's patrolled by WikiMinion it might well be the case that they end up on KayakWiki's ban list before they have the opportunity to insert their spam here. Another possibility is that it isn't related to a specific feature of WikiMinion, but just like "spam attracts more spam," perhaps a heightened response to spam causes spammers to look for greener pastures elsewhere? On the other hand, the drop in spam might be unrelated to WikiMinion at all. I've noticed that the most active time for the Chinese spammers is Sunday Evening (Pacific time), so maybe an influx of spam will hit this coming Sunday? -- RichardP

Ah! I guess the banlist explains it. I don't think that spammers follow what is happening on a wiki. They are too busy spamming (and often their English consists of the sentence "Edit text of this page" only). Sunday evening Pacific? That would be early Monday morning in China, right? -- Manni - 9th Feb 2005

Yeah I was thinking it seemed a bit less spammy round here. In the past there have been lulls in the frequency of spam attacks on kayakwiki, but the problem had been on the increase. No spam for several days is quite remarkable! This is the way RecentChanges should look. Just legit edits. Must be the banlist trick having this effect I think. -- Halz - 9th Feb 2005

I am very happy with how successful RichardP's anti spam efforts have been. The Banned List seems to be a big help, and the bot is taking care of the rest. Thanks for your help Richard. BTW, I am still working on switching over to OddMuse?. If I can find the source of the HomePage locking problem and users are agreeable I may make the switch.-- Nick Schade

Oooh looks like we spoke too soon. The same evil porno spammer is doing his major attack again. I can see wikiminion simultaneosly reverting the edits. I wonder if it will continue to do so until after the spammer's bot has finished. Or maybe it will stop suddenly, thinking it has finished, and then only clean up the rest 2 hours later. Remains to be seen. The banlist trick would be even more nifty if wikiminion could somehow identify this situation (a particular IP address spamming at the same time as wikiminion's antispam run) and ban them immediately (thereby cutting short their spam run). Of course most spammers don't do such humungous spam runs. Only this asshole. -- Halz - 11th Feb 2005

Halz, WikiMinion works by retrieving a copy of RecentChanges and then checking the recent edits for spam. So you are correct, if WikiMinion visits in the middle of a spam run it might appear to clean up some of the spam and then stop suddenly. However, after finding spam on a particular wiki, WikiMinion revisits that wiki much more frequently for a while. So, in this case, WikiMinion would return to clean up any additional spam much quicker than two hours. WikiMinion does take notice of proflic spammers in the middle of its antispam runs, but it doesn't currently immediately add an IP address to the master ban list without first checking with me for confirmation. This means that if I am not near my computer or doing something else it might be several hours before I give WikiMinion permission to add an IP address to the ban list. Do you think WikiMinion should do it automatically without waiting for permission? The ban list feature of WikiMinion is still new, so, frankly, I am a little reluctant to let WikiMinion ban people completely on its own. -- RichardP

It make sense to have a little human intervention on the ban list. If you find that over time you always agree with a suggested ban, then let it happen automatically. -- Nick Schade

Yeah banning is a bad thing if we are not certain that the IP address is spamming. I suppose the ultimate solution would be to use some kind of fuzzy level of certainty value. The value would be attached to IP addresses and/or spammer domain names. If an IP address added just one bad link, then we have a low level of certainty. i.e. Perhaps they were just discussing the spam problem, and just accidentally linked to them. If you (a trusted human) have confirmed that an IP address is evil then we have a high level of certainty, but maybe the bot just suspects a particular IP address, and as it encounters more and more spam, the certainty level goes up. Then we just choose banning threshold, so that when a case becomes clear-cut (even without human checking) the bot can immediately ban. This is a sophisticated way of doing it (would take a lot of programming and testing), but actually I'm not sure it would work. Just an idea really. Of course the certainty level logic could be tested out (displayed) without actually using the value. -- Halz - 17th Feb 2005

Today we're seeing 'recreate' as the edit summary from the WikiMinion on many pages. What does that mean? I would ask the question [here] but it's locked at the moment -- Halz - 17th Feb 2005

Halz, sorry about the ThoughtStorm? wiki being locked - Phil Jones decided to lock the wiki while he was traveling.

With regards to the WikiMinion's recreate message, it is its response to the problem of page history pollution and UseMod?'s lack of search engine exclusion on old versions of a page. UseMod? doesn't use a HTML meta robots tag to mark old revisions off-limits to search engines, so a spammer doesn't lose much Google page rank if his spam is reverted but his spam is still visible to Google via the page history. To combat this WikiMinion performs a 'recreate' when a page's history consists entirely of useless pollution (spam and corresponding spam reverts by WikiMinion). Note that any edit in the history by anyone other than a spammer or WikiMinion prevents a recreate. To do a 'recreate' WikiMinion retrieves the page contents, deletes the page, and then recreates the page using the original contents. This process denies Google page rank to spammers and cleans up the page history. WikiMinion does recreates only if it knows the wiki admin password and the wiki doesn't support HTML meta robots tag. By the way, the 'recreate' isn't a particularly new feature, but they didn't occur on KayakWiki until recently because page histories go back a couple of weeks so it took a while before any pages had page histories consisting entirely of spam or reverts by WikiMinion. -- RichardP

HomePage | WikiMinion | RecentChanges | Preferences
This page is read-only | View other revisions
Last edited April 10, 2006 12:06 am by Michael Daly (diff)