Subversion Repositories

?revision_form?Rev ?revision_input??revision_submit??revision_endform?

Rev 1 | Rev 67 | Go to most recent revision | Details | Compare with Previous | Last modification | View Log | RSS feed

Rev Author Line No. Line
1 magnus 1
                          GREYLISTING with SA-Exim
2
                          ------------------------
3
 
4
 
5
INTRODUCTION
6
------------
7
SA-Exim allows for intelligent greylisting by combining the idea of greylisting
8
with Spam scores from SpamAssassin
9
 
10
If you don't know what greylisting is, you should probably go read up there:
11
http://projects.puremagic.com/greylisting/
12
(note that this implementation works differently than the one described there)
13
 
14
So, SA-Exim isn't just yet another greylisting implementation. By tying it
15
into SA-Exim, and especially by running SA at SMTP time, you can do the
16
following things:
17
- do not bother greylisting people who send messages detected as spam by SA
18
  (indeed, regular greylisting will accept mail from a spammer if he retries
19
  or sends it from an open relay)
20
  SA-Exim will never greylist, or whitelist a sender based on a mail clearly
21
  marked as spam by SA.
22
 
23
- do not delay mail from people who aren't spamming you (this one is the most
24
  important feature of SA-Exim greylisting, as it removes the biggest
25
  disadvantage linkes to greylisting)
26
 
27
- only greylist (and maybe later whitelist) hosts that send you mail with
28
  a certain SA score.
29
 
30
 
31
IMPLEMENTATION
32
--------------
33
So how does this all work?
34
SA comes with a patch for SA 2.x (and a module for SA 3.x) that does the
35
following things:
36
- add a greylisting rule which gets run at the very end, and where if
37
  the score is already higher than a configured value, we do not bother
38
  greylisting the host. We just return a rule failure, which doesn't
39
  change the score and lets SA-Exim reject the mail as usual
40
- if the score is lower than the "surely spam" threshold (shown as 11 in the
41
  example below), check for a file in
42
  /var/spool/sa-exim/messageids/co/nn/ect/ip/envfrom/envto
43
  - if it's there, check if it was written more than x seconds ago (1800s/30mn
44
    in the example below)
45
    - if so, change the status to whitelisted and return true so that SA applies
46
      the whitelist negative score
47
    - if not, simply increase counters, host is still greylisted
48
  - if the file is not there, create it
49
- every x time (like 4 hours or two days), remove all greylist entries that
50
  only saw one mail (i.e. still greylisted, not whitelisted yet).
51
  This is done with a find cron job
52
- every y time (like 1 week), remove whitelisted entries so that your filesystem
53
  doesn't clutter up with hosts you're not going to hear from again in a while
54
 
55
 
56
Then, you call the greylisting rule with this (in SA's local.cf)
57
# reseval is a special eval which only runs after you have the result from
58
# everything else (lets us not greylist a host that is sending spam, otherwise
59
# this rule might set a sufficiently negative score that the next spam would
60
# be allowed in)
61
# Note the 'key' -> 'value'; syntax. It's a special hack to go through SA's
62
# config parser. You need to keep that exact syntax
63
# greylistsecs: how long you greylist a tuplet because whitelisting it
64
# greylistnullfrom: set to 1 to also greylist mail with a null env from
65
# greylistfourthbyte: keep the 4 bytes of the connecting host instead of 3
66
header GREYLIST_ISWHITE reseval:greylisting("( 'dir' => '/var/spool/sa-exim/tuplets'; 'method' => 'dir'; 'greylistsecs' => '1800'; 'dontgreylistthreshold' => 11; 'connectiphdr' => 'X-SA-Exim-Connect-IP'; 'envfromhdr' => 'X-SA-Exim-Mail-From'; 'rcpttohdr' => 'X-SA-Exim-Rcpt-To'; 'greylistnullfrom' => 0; 'greylistfourthbyte' => 0 )")
67
describe GREYLIST_ISWHITE The incoming server has been whitelisted for this rece
68
ipient and sender
69
score GREYLIST_ISWHITE  -1.5
70
 
71
Note that SA greylisting depends on X-SA-Exim-Rcpt-To, so you have to ensure
72
that SAmaxrcptlistlength is set to a reasonably high value (up to 8000) instead
73
of the current default of 0 (you can remove the header in exim's system_filter
74
or a transport if you don't want it to show in user's mails, see "privacy
75
warning" in README)
76
 
77
 
78
Now, in case you aren't confused yet, you get even more knobs to play with :)
79
If a spammer resends you a spam until it gets whitelisted (or typically, it
80
gets sent to a relay that resends it to you), even if you are setup to
81
accept the spam at the point, you don't want to lower the SA score too much
82
just because the mail was resent to you several times (i.e. a rather negative
83
score for GREYLIST_ISWHITE). So, you can actually configure SA-Exim to temp
84
reject messages on a much higher score than usual, if they don't have the
85
GREYLIST_ISWHITE tag.
86
 
87
In other words, let's say you have this in sa-exim.conf:
88
SApermreject: 11.0
89
SAtempreject: 3.0
90
SAgreylistraisetempreject: 6.5
91
 
92
If a mail comes in at less than 3.0, the SA patch/module remembers the sending
93
server's connecting IP, the env from, and the rcpt to(s), and whitelists those.
94
(those will be referred to as tuplets, one for each rcpt to)
95
 
96
If the score is between 3.0 and 11.0,
97
- if at least one of the tuplets is already whitelisted, SA applies the -1.5
98
  score and yields an end score below 9.5, Now, at the same time, SAtempreject
99
  is temporarily raised by 6.5, so everything under 9.5 is accepted, which
100
  basically means that the mail goes through.
101
- if none of the tuplets are whitelisted, they get greylisted
102
- if they are greylisted, they can get upgraded to whitelisted status if the
103
  sending server has been trying for long enough (1800secs in the example given
104
  above). At this point the same thing happens as in case #1 and the mail is
105
  accepted
106
 
107
If a tuplet that is going to be whitelisted or greylisted, already is, SA
108
updates counters to let you run reports and anything else you want, like
109
deciding if or when you'd like to expire the entry
110
 
111
If by now you wonder why you would want to both decrease the SA score and
112
increase the maximum score you'll accept mails on, the reason is as follows:
113
You probably don't want to lower the SA score by 8 just because the tuplet
114
is whitelisted (not only does it mess with the SA scoring of messages that used
115
to be flagged as spam, but a spam with a score of 13.5 would then be lowered
116
to 5.5, be temprejected, and be close to the accept range).
117
Instead, giving an SA score of -1.5, a message with 13.5 becomes 12.0 and still
118
gets rejected right away. You also do not overly (and artifically) lower the
119
score of a message, just for SA-Exim's sake
120
 
121
If you so wish, you can also give the SA rule a score of -0.1, and only
122
dynamically raise the tempreject score for messages that are whitelisted.
123
 
124
 
125
SCORE SETUP
126
-----------
127
It makes little sense to have
128
SAtempreject + SAgreylistraisetempreject + SA GREYLIST_ISWHITE > SApermreject
129
as there is little point to raise SAtempreject if the message that's
130
whitelisted still gets refused by the SApermreject score
131
 
132
As to whether you want to put more points into SA GREYLIST_ISWHITE or
133
SAgreylistraisetempreject, this is your call, but as a general rule, you only
134
want to change the SA score in a way that makes sense for spam scoring,
135
as it similarly affects the score of all messages, whether SA-Exim sees them
136
in the non spam range, tempreject range, or "this is spam that I would never
137
let in" range.
138
 
139
 
140
FILE SETUP
141
----------
142
Make very sure that uid nobody can traverse /var/spool/sa-exim and
143
create tuplets writeable by nobody (or whoever you run SA as)
144
 
145
Then, setup a cron job to delete tuplets that are older than 14 days for
146
whitelisted entries, and 2 days for greylisted entries (or whatever
147
values you fancy).
148
Note that because this implementation does not systematically force the senders to resend you mail, unless they sent something that looks too much like spam,
149
you will typically see few whitelisted entries, and those will either be
150
potential spam that was actually resent to you at least 30mn after the
151
initial copy (or whatever value you setup in "header GREYLIST_ISWHITE"), or
152
people who sent you several Emails (where the second Email will just happen to
153
trigger a whitelisting).
154
 
155
 
156
FILE SETUP
157
----------
158
You should install greylistclean.cron in /etc/cron.d/ on your system to
159
call greylistclean and clean up greylisted entries and whitelisted entries
160
that haven't been used in a while.
161
You can optionally modify it to tweak the cleanup times.
162
Note that you need to tweak greylistclean.cron to match the user spamd runs
163
as if you aren't using the recommended --username=nobody
164
 
165
 
166
SA PATCH (SA 2.x)
167
-----------------
168
For all this to work, you also need to patch SA with SA-greylist.diff
169
from the source tar (or /usr/share/doc/sa-exim*/ for a precompiled package).
170
This patch never made it to the main SA 2.x branch as the developers had mostly
171
switched to 3.x where you can use plugins.
172
If you still use SA 3.x, you can go to /usr/share/perl5/Mail (or wherever
173
appropriate on your system), and run
174
patch -p0 -s < /path/to/sa-exim/SA-greylisting.diff
175
Note that while the patch works, it will not be maintained anymore since
176
it is deprecated for the SA 3.x plugin
177
 
178
 
179
SA PLUGIN (SA 3.x)
180
------------------
181
Newer versions of SpamAssassin support plugins, so there is no need to
182
patch SA anymore, you can just install the Greylisting.pm module on your
183
system and get SA to use it
184
This is how you call the module in SA 3.x (i.e. put this in your
185
/etc/spamassassin/local.cf)
186
 
187
# Note the 'key' -> 'value'; syntax. It's a special hack to go through SA's
188
# config parser. You need to keep that exact syntax
189
# greylistsecs: how long you greylist a tuplet because whitelisting it
190
# greylistnullfrom: set to 1 to also greylist mail with a null env from
191
# greylistfourthbyte: keep the 4 bytes of the connecting host instead of 3
192
loadplugin Greylisting /usr/share/perl5/Mail/SpamAssassin/Plugin/Greylisting.pm
193
header GREYLIST_ISWHITE eval:greylisting("( 'dir' => '/var/spool/sa-exim/tuplets'; 'method' => 'dir'; 'greylistsecs' => '1800'; 'dontgreylistthreshold' => 11; 'connectiphdr' => 'X-SA-Exim-Connect-IP'; 'envfromhdr' => 'X-SA-Exim-Mail-From'; 'rcpttohdr' => 'X-SA-Exim-Rcpt-To'; 'greylistnullfrom' => 1; 'greylistfourthbyte' => 0 )")
194
describe GREYLIST_ISWHITE The incoming server has been whitelisted for this recipient and sender
195
score GREYLIST_ISWHITE  -1.5
196
# Run SpamAssassin last, after all other rules.
197
# (lets us not greylist a host that is sending spam, otherwise this rule might
198
# set a sufficiently negative score that the next spam would be allowed in)
199
priority GREYLIST_ISWHITE 99999
200
 
201
 
202
SA-EXIM NEW BEHAVIOR CONCERNS
203
-----------------------------
204
What greylisting changes as far as spam accepting or rejection is concerned:
205
Once a tuplet has been whitelisted, spam from that host is more likely
206
to be accepted until the tuplet expires. In the case of a mailing list,
207
unless you run a find / rm based on the creation time and not the last
208
modified time, you will then be a bit more likely to accept spam from
209
that list.
210
If this turns out to not be acceptable in your case, there isn't a whole
211
lot you can do about this, except deleting greylist entries for the host
212
from cron before they get promoted to whitelist.
213
 
214
What you can do on top of the existing greylisting code:
215
Parse the SA-Exim logs and if you get spam from an IP, you can decide
216
to delete greylist entries in /var/spool/sa-exim/tuplets/IP or just
217
/var/spool/sa-exim/tuplets/IP/envfrom
218
This may not may not be a good thing if you receive the occasional spam
219
from a mailing list as you'll then re-delay mail for that list, but then
220
again, it will also remove whitelisting for a host that spammed you once
221
with an Email that managed to get under the SA scoring radar
222
 
223
 
224
GREYLISTING AND MXES
225
--------------------
226
Depending on your configuration, you may have realized that SA-Exim doesn't
227
play very well with secondary MXes for your domain if they don't run SA-Exim
228
too (for instance, you'd send a tempreject on spam and clog up your
229
secondary, or maybe even teergrube it if you forgot to add your MX's IP
230
in the do not teergrube list.
231
For greylisting, it's even more simple:
232
If your secondary MXes aren't running SA-Exim with greylisting, then
233
greylisting's efficiency will be greatly reduced as most spammers will send
234
their spams to your secondary MXes which will accept the mail for you,
235
even if it's sent only once, and then your MXes will resend the spam to you
236
until you accept it (rendering greylisting useless)
237
 
238
Now, if your secondaries are running greylisting too, most mail will flow
239
through with no delay whatsoever. However, in the worst case scenario, a mail
240
that isn't spam, but triggers greylisting because its score is high enough to
241
generate a tempreject, could be delayed up to twice the whitelisting time
242
if it were to go to your secondary MX first (assuming your primary is
243
unreachable or temporarily overloaded), and then be resent to your primary
244
MX, which would trigger a second greylisting delay
245
FIXME: implement a whitelist of sending IPs so that greylisting returns
246
whitelisted right away
247
 
248
 
249
SECURITY
250
--------
251
The greylisting function works around the SA parser by sending all the options
252
as a hash inside a string. In turn, greylisting evals the said string.
253
This is a security problem if you allow your users to run custom rules and it
254
gives them access to run spamassassin as a user different from their own, or
255
in a way that they otherwise wouldn't be able to.
256
Do not run greylisting if this a problem for you (in the default SA/SA-Exim
257
setup, this shouldn't be a concern since it doesn't even parse users' config
258
files)