Repository navigation
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
Splam
=====
Splam is a simple spam scoring plugin. It contains a set of rules that are run on a field
to help you determine the likelihood of that field being spam. It doesn't do anything
other than give a field a score. It's up to you to act on that score.
Check out the tests for instructions on how to use: you'll want to integrate this into
your application's workflow.
It's heavily biased towards the spam I've been seeing in the past two or three hours.
This includes lots of crap with
- bbcode [url=
- lots of links (http://)
- russian text
- links to russian or chinese websites
You can write your own plugins to Splam: simply subclass Splam::Rule. Splam is clever enough
to iterate over all Rule's subclasses and run the 'run' method on the field to be checked.
The other way to do this would be to define Rule.add_rule do ... end but I think the class
form is easier for rubyists to understand and modify.
Splam aggregates the scores from all the rules. From the brief testing I've done, anything over
about 40 is likely to be spam. Real spam will blow out of the scoring stratosphere with over 1,000.
Recommended serving directions:
class Comment
include Splam
splammable :body
end
comment = Comment.new :body => "This is spam!!!1"
comment.splam? # => false
comment.splam_score # => 2
comment.splam_reasons # => []
Add this to a model, check the score, and determine (based on other factors such as logged-in
user, time spent on the page, validity of request headers, length of user's membership on the
site) whether to ban the post or not.
We recommend showing the post to the user (spambox them in) but hide it from everyone else.
Configuration
-------------
Apps with different customers can score differently:
Splam.configure do |c|
c.profile = :lighthouse # a named preset (Splam::Config::PROFILES)
c.weights = { :bad_words => 0.5, :href => 2 } # rule weights, overriding the preset
c.bad_word_score = 12 # BadWords' base score
end
The default profile is Tender's tuning. The :lighthouse profile is the tuning
of splam 0.3.0: more word genres, a bad-word base of 15, lower link-count
thresholds, trailing-tag checks, the user-record checks, and the Fuzz and
LineLength rules (opt-in rules, which only run when a profile or a suite names
them). A suite that names its own weights keeps them.
Results per field:
comment.splam_scores # => { :body => 120, :title => 0 }
comment.splam?(:body) # that field against its suite's threshold
comment.splam_reasons_by_field # => { :body => [...], :title => [...] }
Your own rules
--------------
Subclass Splam::Rule and it joins the default rules (or name it in a suite).
Rules can read the request your model passes in:
class PasteCheck < Splam::Rule
def run
return unless @request && @request[:time]
add_score 300, "Submitted within 2 seconds" if @request[:time] < 2
end
end
class Comment
include Splam
splammable :body, 250 do |s|
s.request = lambda { |comment| { :time => comment.seconds_on_form } }
end
end
Mark a rule opt-in (def self.opt_in?; true; end) to keep it out of the
defaults unless a profile enables it.
Comparing versions
------------------
script/eval.rb scores a labelled JSONL file with whichever splam is on the
load path and prints flag rates per slice and ROC AUC, so two versions or
profiles can be compared on the same data. See the comment at its top.
Tests
-----
ruby -Ilib -Itest test/splam_test.rb # or: rake test
test/ngram_test.rb needs a Redis on localhost and is omitted without one.
TODO
- Integrate bayesian or other clever algorithm, so that scores aren't hardcoded.
- Switch to using a percentage (0.994) rather than a score (250)
- Write more plugins!
- Test against a larger Ham corpus
- Fix that nasty autoloading code in splam.rb
Copyright (c) 2008 ENTP, released under the MIT license