Spam Control at CCRMA

From CCRMA Wiki

(Difference between revisions)
Jump to: navigation, search
(Forwarding with Spam Control)
m (Fixing a typo in the email forwarding section.)
 
(38 intermediate revisions not shown)
Line 1: Line 1:
-
Welcome to CCRMA's Spam fighter homepage.
+
Welcome to CCRMA's Spam fighter homepage.  Having a 'Spam Free' inbox is a requires vigilence on everyone's part.  In the text below, we'll describe what you can do to minimize your Spam.
-
Having a 'Spam Free' inbox is a requires vigilence on everyone's part.  In the text below, we'll describe what you can do to minimize your Spam.
+
== Spam control programs ==
-
First you need to determine which email client you will be using (e.g. Evolution, Thunderbird, WebMail, or Pine). Spam fighting is much more difficult if you use more than one email client. The descriptions below are for exclusive use of '''only''' one clientThese solutions also assume that you will '''not''' be using client side 'intelligent email filtering' (where the your email client 'learns' about Junk mail).
+
All incoming email to CCRMA is filtered through two programs. Spam is handled by [http://spamassassin.apache.org Spamassassin] and virus detection is handled through [http://www.clamav.net ClamAV]. All emails are piped automatically through these two programs '''before''' they reach your Inbox. Spamassassin uses many tests to assign a score to all emailsIf the score is high enough, the email is rejected to the sender and you will never see it.  If an email passes that first hurdle it is delivered to your mailbox through a program called '''<code>procmail</code>'''.  Your account already contains a procmail configuration file (<code>.procmailrc</code>, located in the highest level of your CCRMA home directory) that discards emails with a Spamassassin score of 5 or more.  The rules that Spamassassin uses to rate email messages are updated regularly.
-
Filtering is done through a program called '''<code>procmail</code>'''.
+
You can further tune what it does by following the instructions below.
-
== .procmailrc File ==
+
==The Default <code>.procmailrc</code>==
-
This file is key in your Spam control effort.  It is a hidden or 'dot' file, that you should create (for newer accounts it will have been created for you) in your home directory with the following contents, at a minimum:  
+
The <code>.procmailrc</code> file is located in the top level of your home directory and is a 'hidden' file (because of the dot in the file name).  You can check if it is there by typing at the terminal command line:  
-
<pre>
+
<pre>~> ls -a | grep procmail
-
# directory where mailboxes are located
+
.procmailrc
-
# this is the default used by pine
+
-
MAILDIR=$HOME
+
-
 
+
-
# pipe the message through spamassassin in cm-home
+
-
:0fw
+
-
| spamc -d 171.64.197.138
+
</pre>
</pre>
-
.procmailrc gives Sendmail [http://en.wikipedia.org/wiki/Sendmail] instructions on where to route your email once it arrives at CCRMA.  The idea is to make Sendmail route email through SpamAssassin [http://spamassassin.apache.org] before it gets to your ''Inbox.''
+
You can view the file by typing (from the top level of your home directory):
-
SpamAssassin will run each email through its filter (filter rules are updated frequently to reflect new spam 'threats').  SpamAssassin adds several lines to your email header's.  When viewing your email in your email client, you normally only see an abbreviated header which shows the 'From:', 'To:', 'Date:' and 'Subject:' lines, but you can select elect to see the full headers as well.  The lines added by SpamAssassin to your (normally hidden) email header include:
+
<pre>~> less .procmailrc</pre>
 +
 
 +
If you are a new user and still have not modified this file you should see this:
<pre>
<pre>
-
X-Spam-Level:
+
# CCRMA default procmail email filter v1.20
-
X-Spam-Checker-Version:
+
#
-
X-Spam-Status:
+
# directory where mailboxes are located, this is the default
-
X-Spam-Report:
+
# used by pine and other email clients
-
</pre>
+
MAILDIR=$HOME/mail
-
For now, let's look at '''<code>X-Spam-Level</code>''' since it is on this line, that you will create filter's in your email client. If you take a look at the header you'll see one line that looks like this:
+
### optional: run spamassassin a second time if you have a custom
 +
#            configuration in ~/.spamassassin/user_conf (by default
 +
#            all incoming email is processed by spamassassin)
 +
#:0fw
 +
#| spamc
-
<pre>
+
### score >= 5 is considered spam and discarded
-
X-Spam-Level: *****
+
:0:
 +
* ^X-Spam-Level: \*\*\*\*\*
 +
/dev/null
 +
 
 +
### optional: redirect false bounce messages to MAILDIR/bounces
 +
#            if activated this should take care of backscatter spam
 +
#:0:
 +
#* ^X-Spam-Status:.*ANY_BOUNCE_MESSAGE.*
 +
#bounces
 +
 
 +
### optional: forward your email and keep a local copy
 +
#:0 c
 +
#! <your_account>@<somewhere_else>
 +
 
 +
### optional: forward your email
 +
#:0
 +
#! <your_account>@<somewhere_else>
</pre>
</pre>
-
X-Spam-Level display's Spam Level using the asteriskHere, or example, for 'Spam Level 5' with 5 asterisks.
+
This file is key in your Spam control effortTrue, it's a bit technical, but taking a moment to understand what is happening here will not only keep unwanted emails from appearing in your Inbox, but will allow you to further configure Spamassassin if you wish, and to set email forwarding to another email account.
-
The idea then, is to establish email filter's, filtering on the asterisk, in your email client which, directing these messages into more manageable folder's or to delete the message automatically ('''wise for Spam Level 5 or above, where the risk of false positives is very low''').
+
== <code>.procmailrc</code> Breakdown (looking at each section)==
-
For new accounts we now add this to your <code>.procmailrc</code> file which automatically deletes email messages with an <code>X-Spam-Level</code> greater than 5:
+
Each section of this file serves an important purpose.  Note that any line that begins with a '<code>#</code>' is 'commented out' and will not be read by the procmail system.  If the line has '<code>###</code>', it means that these are text comments and not code, and should be left alone, and not removed.  Further, almost every line in the default <code>.procmailrc</code> file is already commented out, meaning that only two parts are operational by default:
 +
 
 +
===MAILDIR===
 +
 
 +
Sets the default mail directory in your home directory.  This is the same directory that we recommend for [[Email#IMAP_Receiving | IMAP]]:
 +
 
 +
<pre>MAILDIR=$HOME/mail</pre>
 +
 
 +
===Spam Score Threshold===
 +
 
 +
Sets your local threshold for spam messages that are to be discarded (or more accurately: sent to <code>/dev/null</code>, aka. permanently deleted). Remember, that Spamassassin has already rejected messages which are most certainly Spam ( >= score 10), but this setting allows you to delete messages of a lower score.  The default is 5 (or '<code>\*\*\*\*\*</code>' which stands for 5 stars).  You can raise or lower the threshold by editing this section of <code>.procmailrc</code>:
<pre>
<pre>
-
# score >= 5 is considered spam
+
### score >= 5 is considered spam and discarded
:0:
:0:
* ^X-Spam-Level: \*\*\*\*\*
* ^X-Spam-Level: \*\*\*\*\*
Line 51: Line 77:
</pre>
</pre>
-
A good idea is to filter any messages with Spam Level between 3-5 to an email folder so you can check it periodically.  That, too, can be done with <code>'''.procmailrc'''</code> by adding this:
+
This configuration permits <code>procmail</code> to compare portions of each email header received against this rule containing <code>\*\*\*\*\*</code>.  See a short explaination of email headers below [[Spam_Control_at_CCRMA#Email_Headers_and_Spamassassin | Email Headers and Spamassassin]].
 +
 
 +
===SpamAssassin Per User Configuration===
 +
 
 +
Configuring SpamAssassin on a per user basis requires some advanced knowledge ([http://spamassassin.apache.org/full/3.1.x/doc/Mail_SpamAssassin_Conf.html SpamAssassin configuration file], [http://wiki.apache.org/spamassassin/WritingRules?highlight=%28user_prefs%29 Writing your own rules], etc), which will be the focus of another wiki page some day.  But essentially, if you feel like you want to study the SpamAssassin rules and weight them differently, emphasizing some over others, or change any other SpamAssassin options you can do it by uncommenting:
<pre>
<pre>
-
# divert emails tagged with 3 or more stars to a directory
+
### optional: run spamassassin a second time if you have a custom
 +
#            configuration in ~/.spamassassin/user_conf (by default
 +
#            all incoming email is processed by spamassassin)
 +
:0fw
 +
| spamc
 +
</pre>
 +
 
 +
This calls SpamAssassin (<code>spamc</code>) '''again''' to reprocess your incoming messages, but this time in the context of your account so that anything changed in your <code>~/.spamassassin/user_conf</code> file will affect the outcome (where <code>~/</code> means your home directory).
 +
 
 +
===Bounces===
 +
 
 +
This section, if you can believe it, is a bit more obscure...  Occassionally, your email address will be spoofed by a spammer (meaning: used as the 'From:' address) to spam a server outside of Stanford.  When that email server rejects (bounces) the email, it goes back to you instead of to the originator of the message (the spammer).  Often this happens in waves, in that, for a while you will be stormed by these bounce messages then things go quiet.  If you find that you are having many of such emails, go ahead and uncomment this rule, and have a look in your 'bounces' IMAP folder to see if it is having any effect.  Please come and see us if things aren't working or if you are confused at all.
 +
 
 +
<pre>
 +
### optional: redirect false bounce messages to MAILDIR/bounces
 +
#            if activated this should take care of backscatter spam
:0:
:0:
-
* ^X-Spam-Level: \*\*\*
+
* ^X-Spam-Status:.*ANY_BOUNCE_MESSAGE.*
-
mail/spam3
+
bounces
</pre>
</pre>
-
Since any email with a rating of >=5 asterisks has been deleted, then any message with 3 or 4 asterisks will be routed to a directory called <code>~/mail/spam3</code> in your home directory.  This directory should also be viewable by your email client so you can check it periodically.
+
===Forwarding===
-
==Forwarding with Spam Control==
+
To forward email to another account, you'll have to edit your <code>.procmailrc</code> file (<code>~/.procmailrc</code>).  Within this file you have two choices regarding forwarding: forwarding to another address and keeping a local copy, or forwarding to another address and not keeping a local copy ('''keeping a local copy''' means your email is delivered to your CCRMA account in addition to being forwarded). If you choose to forward your email, we recommend that unless you have a good reason to do otherwise, you should choose to forward and '''not''' keep a local copy.  In order to enable forwarding, just uncomment (by removing the <code>#</code>) whichever option you like (see example below), but '''not''' both.
-
If you would like to forward your email you should add the following lines to the '''end''' of your <code>~/.procmailrc</code> file.
+
If you do forward your CCRMA email please keep your forwarding address current.  
-
===Forward Without Keeping a Local CCRMA Copy===
+
====Forward Keeping a Local CCRMA Copy====
<pre>
<pre>
-
# Forward mail from CCRMA to this account
+
### optional: forward your email and keep a local copy
-
# Place after spamassassin and email filtering
+
:0 c
-
:0
+
! <your_account>@<somewhere_else>
-
! <your-other-username>@<your-other-mail.com>
+
</pre>
</pre>
-
===Forward Keeping a Local CCRMA Copy===
+
====Forward Without Keeping a Local CCRMA Copy====
<pre>
<pre>
-
# Forward mail from CCRMA to this account, keep a copy
+
### optional: forward your email
-
# Place after spamassassin and email filtering
+
:0
-
:0 c
+
! <your_account>@<somewhere_else>
-
! <your-other-username>@<your-other-mail.com>
+
</pre>
</pre>
-
==Example .procmailrc file==
+
==Email Headers and Spamassassin==
-
So here is how the file should look in the end for a solid beginning to your spam filtering:
+
SpamAssassin will run each email through its filter (filter rules are updated frequently to reflect new spam 'threats').  SpamAssassin adds several lines to your email header's.  When viewing your email in your email client, you normally only see an abbreviated header which shows the 'From:', 'To:', 'Date:' and 'Subject:' lines, but you can select elect to see the full headers as well.  The lines added by SpamAssassin to your (normally hidden) email header include:
<pre>
<pre>
-
# directory where mailboxes are located
+
X-Spam-Level:
-
# this is the default used by pine
+
X-Spam-Checker-Version:
-
MAILDIR=$HOME
+
X-Spam-Status:
 +
X-Spam-Report:
 +
</pre>
-
# pipe the message through spamassassin in cm-home
+
For now, let's look at '''<code>X-Spam-Level</code>''' since it is on this line, that you will create filter's in your email client. If you take a look at the header you'll see one line that looks like this:
-
:0fw
+
-
| spamc -d 171.64.197.138
+
 +
<pre>
 +
X-Spam-Level: *****
 +
</pre>
 +
 +
X-Spam-Level display's Spam Level using the asterisk.  Here, or example, for 'Spam Level 5' with 5 asterisks.
 +
 +
The idea then, is to establish email filter's, filtering on the asterisk, in your email client which, directing these messages into more manageable folder's or to delete the message automatically ('''wise for Spam Level 5 or above, where the risk of false positives is very low''').
 +
 +
For new accounts we now add this to your <code>.procmailrc</code> file which automatically deletes email messages with an <code>X-Spam-Level</code> greater than 5:
 +
 +
<pre>
# score >= 5 is considered spam
# score >= 5 is considered spam
:0:
:0:
* ^X-Spam-Level: \*\*\*\*\*
* ^X-Spam-Level: \*\*\*\*\*
/dev/null
/dev/null
 +
</pre>
 +
A good idea is to filter any messages with Spam Level between 3-5 to an email folder so you can check it periodically.  That, too, can be done with <code>'''.procmailrc'''</code> by adding this:
 +
 +
<pre>
# divert emails tagged with 3 or more stars to a directory
# divert emails tagged with 3 or more stars to a directory
:0:
:0:
* ^X-Spam-Level: \*\*\*
* ^X-Spam-Level: \*\*\*
-
mail/spam3
+
spam3
-
 
+
-
# Forward mail from CCRMA to this account
+
-
# Place after spamassassin and email filtering
+
-
:0
+
-
! <your-other-username>@gmail.com
+
</pre>
</pre>
 +
 +
Since any email with a rating of >=5 asterisks has been deleted, then any message with 3 or 4 asterisks will be routed to a directory called <code>~/mail/spam3</code> in your home directory (remember we are setting MAILDIR to $HOME/mail in our default procmailrc file, so spam3 will be created inside ~/mail).  This directory should also be viewable by your email client so you can check it periodically.
 +
 +
[[Category:CCRMA User Guide]]
[[Category:CCRMA User Guide]]

Current revision as of 22:53, 1 October 2010

Welcome to CCRMA's Spam fighter homepage. Having a 'Spam Free' inbox is a requires vigilence on everyone's part. In the text below, we'll describe what you can do to minimize your Spam.

Contents

Spam control programs

All incoming email to CCRMA is filtered through two programs. Spam is handled by Spamassassin and virus detection is handled through ClamAV. All emails are piped automatically through these two programs before they reach your Inbox. Spamassassin uses many tests to assign a score to all emails. If the score is high enough, the email is rejected to the sender and you will never see it. If an email passes that first hurdle it is delivered to your mailbox through a program called procmail. Your account already contains a procmail configuration file (.procmailrc, located in the highest level of your CCRMA home directory) that discards emails with a Spamassassin score of 5 or more. The rules that Spamassassin uses to rate email messages are updated regularly.

You can further tune what it does by following the instructions below.

The Default .procmailrc

The .procmailrc file is located in the top level of your home directory and is a 'hidden' file (because of the dot in the file name). You can check if it is there by typing at the terminal command line:

~> ls -a | grep procmail
.procmailrc

You can view the file by typing (from the top level of your home directory):

~> less .procmailrc

If you are a new user and still have not modified this file you should see this:

# CCRMA default procmail email filter v1.20
#
# directory where mailboxes are located, this is the default 
# used by pine and other email clients
MAILDIR=$HOME/mail

### optional: run spamassassin a second time if you have a custom
#             configuration in ~/.spamassassin/user_conf (by default
#             all incoming email is processed by spamassassin)
#:0fw
#| spamc

### score >= 5 is considered spam and discarded
:0:
* ^X-Spam-Level: \*\*\*\*\*
/dev/null

### optional: redirect false bounce messages to MAILDIR/bounces
#             if activated this should take care of backscatter spam
#:0:
#* ^X-Spam-Status:.*ANY_BOUNCE_MESSAGE.*
#bounces

### optional: forward your email and keep a local copy
#:0 c
#! <your_account>@<somewhere_else>

### optional: forward your email
#:0
#! <your_account>@<somewhere_else>

This file is key in your Spam control effort. True, it's a bit technical, but taking a moment to understand what is happening here will not only keep unwanted emails from appearing in your Inbox, but will allow you to further configure Spamassassin if you wish, and to set email forwarding to another email account.

.procmailrc Breakdown (looking at each section)

Each section of this file serves an important purpose. Note that any line that begins with a '#' is 'commented out' and will not be read by the procmail system. If the line has '###', it means that these are text comments and not code, and should be left alone, and not removed. Further, almost every line in the default .procmailrc file is already commented out, meaning that only two parts are operational by default:

MAILDIR

Sets the default mail directory in your home directory. This is the same directory that we recommend for IMAP:

MAILDIR=$HOME/mail

Spam Score Threshold

Sets your local threshold for spam messages that are to be discarded (or more accurately: sent to /dev/null, aka. permanently deleted). Remember, that Spamassassin has already rejected messages which are most certainly Spam ( >= score 10), but this setting allows you to delete messages of a lower score. The default is 5 (or '\*\*\*\*\*' which stands for 5 stars). You can raise or lower the threshold by editing this section of .procmailrc:

### score >= 5 is considered spam and discarded
:0:
* ^X-Spam-Level: \*\*\*\*\*
/dev/null

This configuration permits procmail to compare portions of each email header received against this rule containing \*\*\*\*\*. See a short explaination of email headers below Email Headers and Spamassassin.

SpamAssassin Per User Configuration

Configuring SpamAssassin on a per user basis requires some advanced knowledge (SpamAssassin configuration file, Writing your own rules, etc), which will be the focus of another wiki page some day. But essentially, if you feel like you want to study the SpamAssassin rules and weight them differently, emphasizing some over others, or change any other SpamAssassin options you can do it by uncommenting:

### optional: run spamassassin a second time if you have a custom
#             configuration in ~/.spamassassin/user_conf (by default
#             all incoming email is processed by spamassassin)
:0fw
| spamc

This calls SpamAssassin (spamc) again to reprocess your incoming messages, but this time in the context of your account so that anything changed in your ~/.spamassassin/user_conf file will affect the outcome (where ~/ means your home directory).

Bounces

This section, if you can believe it, is a bit more obscure... Occassionally, your email address will be spoofed by a spammer (meaning: used as the 'From:' address) to spam a server outside of Stanford. When that email server rejects (bounces) the email, it goes back to you instead of to the originator of the message (the spammer). Often this happens in waves, in that, for a while you will be stormed by these bounce messages then things go quiet. If you find that you are having many of such emails, go ahead and uncomment this rule, and have a look in your 'bounces' IMAP folder to see if it is having any effect. Please come and see us if things aren't working or if you are confused at all.

### optional: redirect false bounce messages to MAILDIR/bounces
#             if activated this should take care of backscatter spam
:0:
* ^X-Spam-Status:.*ANY_BOUNCE_MESSAGE.*
bounces

Forwarding

To forward email to another account, you'll have to edit your .procmailrc file (~/.procmailrc). Within this file you have two choices regarding forwarding: forwarding to another address and keeping a local copy, or forwarding to another address and not keeping a local copy (keeping a local copy means your email is delivered to your CCRMA account in addition to being forwarded). If you choose to forward your email, we recommend that unless you have a good reason to do otherwise, you should choose to forward and not keep a local copy. In order to enable forwarding, just uncomment (by removing the #) whichever option you like (see example below), but not both.

If you do forward your CCRMA email please keep your forwarding address current.

Forward Keeping a Local CCRMA Copy

### optional: forward your email and keep a local copy
:0 c
! <your_account>@<somewhere_else>

Forward Without Keeping a Local CCRMA Copy

### optional: forward your email
:0
! <your_account>@<somewhere_else>

Email Headers and Spamassassin

SpamAssassin will run each email through its filter (filter rules are updated frequently to reflect new spam 'threats'). SpamAssassin adds several lines to your email header's. When viewing your email in your email client, you normally only see an abbreviated header which shows the 'From:', 'To:', 'Date:' and 'Subject:' lines, but you can select elect to see the full headers as well. The lines added by SpamAssassin to your (normally hidden) email header include:

X-Spam-Level:
X-Spam-Checker-Version:
X-Spam-Status:
X-Spam-Report:

For now, let's look at X-Spam-Level since it is on this line, that you will create filter's in your email client. If you take a look at the header you'll see one line that looks like this:

X-Spam-Level: *****

X-Spam-Level display's Spam Level using the asterisk. Here, or example, for 'Spam Level 5' with 5 asterisks.

The idea then, is to establish email filter's, filtering on the asterisk, in your email client which, directing these messages into more manageable folder's or to delete the message automatically (wise for Spam Level 5 or above, where the risk of false positives is very low).

For new accounts we now add this to your .procmailrc file which automatically deletes email messages with an X-Spam-Level greater than 5:

# score >= 5 is considered spam
:0:
* ^X-Spam-Level: \*\*\*\*\*
/dev/null

A good idea is to filter any messages with Spam Level between 3-5 to an email folder so you can check it periodically. That, too, can be done with .procmailrc by adding this:

# divert emails tagged with 3 or more stars to a directory
:0:
* ^X-Spam-Level: \*\*\*
spam3

Since any email with a rating of >=5 asterisks has been deleted, then any message with 3 or 4 asterisks will be routed to a directory called ~/mail/spam3 in your home directory (remember we are setting MAILDIR to $HOME/mail in our default procmailrc file, so spam3 will be created inside ~/mail). This directory should also be viewable by your email client so you can check it periodically.

Personal tools