Skip to content
courtenayPublic

About

Simple, pluggable, easily customizable score-based spam filter plugin for Ruby-based applications

Resources

Stars

108 stars

Watchers

1 watching

Forks

Repository files navigation

Splam
=====

Splam is a simple spam scoring plugin.  It contains a set of rules that are run on a field
to help you determine the likelihood of that field being spam.  It doesn't do anything
other than give a field a score.  It's up to you to act on that score.

Check out the tests for instructions on how to use: you'll want to integrate this into
your application's workflow.

It's heavily biased towards the spam I've been seeing in the past two or three hours.
This includes lots of crap with
- bbcode [url=
- lots of links (http://)
- russian text
- links to russian or chinese websites

You can write your own plugins to Splam: simply subclass Splam::Rule. Splam is clever enough
to iterate over all Rule's subclasses and run the 'run' method on the field to be checked.
The other way to do this would be to define Rule.add_rule do ... end but I think the class
form is easier for rubyists to understand and modify.

Splam aggregates the scores from all the rules.  From the brief testing I've done, anything over
about 40 is likely to be spam.  Real spam will blow out of the scoring stratosphere with over 1,000.

Recommended serving directions:

    class Comment
      include Splam
      
      splammable :body
    end
    
    comment = Comment.new :body => "This is spam!!!1"
    comment.splam? # => false
    comment.splam_score # => 2
    comment.splam_reasons # => []

Add this to a model, check the score, and determine (based on other factors such as logged-in
user, time spent on the page, validity of request headers, length of user's membership on the 
site) whether to ban the post or not. 

We recommend showing the post to the user (spambox them in) but hide it from everyone else.

Configuration
-------------

Apps with different customers can score differently:

    Splam.configure do |c|
      c.profile = :lighthouse                       # a named preset (Splam::Config::PROFILES)
      c.weights = { :bad_words => 0.5, :href => 2 } # rule weights, overriding the preset
      c.bad_word_score = 12                         # BadWords' base score
    end

The default profile is Tender's tuning. The :lighthouse profile is the tuning
of splam 0.3.0: more word genres, a bad-word base of 15, lower link-count
thresholds, trailing-tag checks, the user-record checks, and the Fuzz and
LineLength rules (opt-in rules, which only run when a profile or a suite names
them). A suite that names its own weights keeps them.

Results per field:

    comment.splam_scores            # => { :body => 120, :title => 0 }
    comment.splam?(:body)           # that field against its suite's threshold
    comment.splam_reasons_by_field  # => { :body => [...], :title => [...] }

Your own rules
--------------

Subclass Splam::Rule and it joins the default rules (or name it in a suite).
Rules can read the request your model passes in:

    class PasteCheck < Splam::Rule
      def run
        return unless @request && @request[:time]
        add_score 300, "Submitted within 2 seconds" if @request[:time] < 2
      end
    end

    class Comment
      include Splam
      splammable :body, 250 do |s|
        s.request = lambda { |comment| { :time => comment.seconds_on_form } }
      end
    end

Mark a rule opt-in (def self.opt_in?; true; end) to keep it out of the
defaults unless a profile enables it.

Comparing versions
------------------

script/eval.rb scores a labelled JSONL file with whichever splam is on the
load path and prints flag rates per slice and ROC AUC, so two versions or
profiles can be compared on the same data. See the comment at its top.

Tests
-----

    ruby -Ilib -Itest test/splam_test.rb   # or: rake test

test/ngram_test.rb needs a Redis on localhost and is omitted without one.

TODO

- Integrate bayesian or other clever algorithm, so that scores aren't hardcoded.
- Switch to using a percentage (0.994) rather than a score (250)
- Write more plugins!
- Test against a larger Ham corpus
- Fix that nasty autoloading code in splam.rb

Copyright (c) 2008 ENTP, released under the MIT license

About

Simple, pluggable, easily customizable score-based spam filter plugin for Ruby-based applications

Resources

Stars

108 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages