Rev 1 | Details | Compare with Previous | Last modification | View Log | RSS feed
Rev | Author | Line No. | Line |
---|---|---|---|
1 | magnus | 1 | GREYLISTING with SA-Exim |
2 | ------------------------ |
||
3 | |||
4 | |||
5 | INTRODUCTION |
||
6 | ------------ |
||
7 | SA-Exim allows for intelligent greylisting by combining the idea of greylisting |
||
8 | with Spam scores from SpamAssassin |
||
9 | |||
10 | If you don't know what greylisting is, you should probably go read up there: |
||
11 | http://projects.puremagic.com/greylisting/ |
||
12 | (note that this implementation works differently than the one described there) |
||
13 | |||
14 | So, SA-Exim isn't just yet another greylisting implementation. By tying it |
||
15 | into SA-Exim, and especially by running SA at SMTP time, you can do the |
||
16 | following things: |
||
17 | - do not bother greylisting people who send messages detected as spam by SA |
||
18 | (indeed, regular greylisting will accept mail from a spammer if he retries |
||
19 | or sends it from an open relay) |
||
20 | SA-Exim will never greylist, or whitelist a sender based on a mail clearly |
||
21 | marked as spam by SA. |
||
22 | |||
23 | - do not delay mail from people who aren't spamming you (this one is the most |
||
24 | important feature of SA-Exim greylisting, as it removes the biggest |
||
25 | disadvantage linkes to greylisting) |
||
26 | |||
27 | - only greylist (and maybe later whitelist) hosts that send you mail with |
||
28 | a certain SA score. |
||
29 | |||
30 | |||
31 | IMPLEMENTATION |
||
32 | -------------- |
||
33 | So how does this all work? |
||
34 | SA comes with a patch for SA 2.x (and a module for SA 3.x) that does the |
||
35 | following things: |
||
36 | - add a greylisting rule which gets run at the very end, and where if |
||
37 | the score is already higher than a configured value, we do not bother |
||
38 | greylisting the host. We just return a rule failure, which doesn't |
||
39 | change the score and lets SA-Exim reject the mail as usual |
||
40 | - if the score is lower than the "surely spam" threshold (shown as 11 in the |
||
41 | example below), check for a file in |
||
42 | /var/spool/sa-exim/messageids/co/nn/ect/ip/envfrom/envto |
||
43 | - if it's there, check if it was written more than x seconds ago (1800s/30mn |
||
44 | in the example below) |
||
45 | - if so, change the status to whitelisted and return true so that SA applies |
||
46 | the whitelist negative score |
||
47 | - if not, simply increase counters, host is still greylisted |
||
48 | - if the file is not there, create it |
||
49 | - every x time (like 4 hours or two days), remove all greylist entries that |
||
50 | only saw one mail (i.e. still greylisted, not whitelisted yet). |
||
51 | This is done with a find cron job |
||
52 | - every y time (like 1 week), remove whitelisted entries so that your filesystem |
||
53 | doesn't clutter up with hosts you're not going to hear from again in a while |
||
54 | |||
55 | |||
56 | Then, you call the greylisting rule with this (in SA's local.cf) |
||
57 | # reseval is a special eval which only runs after you have the result from |
||
58 | # everything else (lets us not greylist a host that is sending spam, otherwise |
||
59 | # this rule might set a sufficiently negative score that the next spam would |
||
60 | # be allowed in) |
||
61 | # Note the 'key' -> 'value'; syntax. It's a special hack to go through SA's |
||
62 | # config parser. You need to keep that exact syntax |
||
63 | # greylistsecs: how long you greylist a tuplet because whitelisting it |
||
64 | # greylistnullfrom: set to 1 to also greylist mail with a null env from |
||
65 | # greylistfourthbyte: keep the 4 bytes of the connecting host instead of 3 |
||
66 | header GREYLIST_ISWHITE reseval:greylisting("( 'dir' => '/var/spool/sa-exim/tuplets'; 'method' => 'dir'; 'greylistsecs' => '1800'; 'dontgreylistthreshold' => 11; 'connectiphdr' => 'X-SA-Exim-Connect-IP'; 'envfromhdr' => 'X-SA-Exim-Mail-From'; 'rcpttohdr' => 'X-SA-Exim-Rcpt-To'; 'greylistnullfrom' => 0; 'greylistfourthbyte' => 0 )") |
||
67 | describe GREYLIST_ISWHITE The incoming server has been whitelisted for this rece |
||
68 | ipient and sender |
||
69 | score GREYLIST_ISWHITE -1.5 |
||
70 | |||
71 | Note that SA greylisting depends on X-SA-Exim-Rcpt-To, so you have to ensure |
||
72 | that SAmaxrcptlistlength is set to a reasonably high value (up to 8000) instead |
||
73 | of the current default of 0 (you can remove the header in exim's system_filter |
||
74 | or a transport if you don't want it to show in user's mails, see "privacy |
||
75 | warning" in README) |
||
76 | |||
77 | |||
78 | Now, in case you aren't confused yet, you get even more knobs to play with :) |
||
79 | If a spammer resends you a spam until it gets whitelisted (or typically, it |
||
80 | gets sent to a relay that resends it to you), even if you are setup to |
||
81 | accept the spam at the point, you don't want to lower the SA score too much |
||
82 | just because the mail was resent to you several times (i.e. a rather negative |
||
83 | score for GREYLIST_ISWHITE). So, you can actually configure SA-Exim to temp |
||
84 | reject messages on a much higher score than usual, if they don't have the |
||
85 | GREYLIST_ISWHITE tag. |
||
86 | |||
87 | In other words, let's say you have this in sa-exim.conf: |
||
88 | SApermreject: 11.0 |
||
89 | SAtempreject: 3.0 |
||
90 | SAgreylistraisetempreject: 6.5 |
||
91 | |||
92 | If a mail comes in at less than 3.0, the SA patch/module remembers the sending |
||
93 | server's connecting IP, the env from, and the rcpt to(s), and whitelists those. |
||
94 | (those will be referred to as tuplets, one for each rcpt to) |
||
95 | |||
96 | If the score is between 3.0 and 11.0, |
||
97 | - if at least one of the tuplets is already whitelisted, SA applies the -1.5 |
||
98 | score and yields an end score below 9.5, Now, at the same time, SAtempreject |
||
99 | is temporarily raised by 6.5, so everything under 9.5 is accepted, which |
||
100 | basically means that the mail goes through. |
||
101 | - if none of the tuplets are whitelisted, they get greylisted |
||
102 | - if they are greylisted, they can get upgraded to whitelisted status if the |
||
103 | sending server has been trying for long enough (1800secs in the example given |
||
104 | above). At this point the same thing happens as in case #1 and the mail is |
||
105 | accepted |
||
106 | |||
107 | If a tuplet that is going to be whitelisted or greylisted, already is, SA |
||
108 | updates counters to let you run reports and anything else you want, like |
||
109 | deciding if or when you'd like to expire the entry |
||
110 | |||
111 | If by now you wonder why you would want to both decrease the SA score and |
||
112 | increase the maximum score you'll accept mails on, the reason is as follows: |
||
113 | You probably don't want to lower the SA score by 8 just because the tuplet |
||
114 | is whitelisted (not only does it mess with the SA scoring of messages that used |
||
115 | to be flagged as spam, but a spam with a score of 13.5 would then be lowered |
||
116 | to 5.5, be temprejected, and be close to the accept range). |
||
117 | Instead, giving an SA score of -1.5, a message with 13.5 becomes 12.0 and still |
||
118 | gets rejected right away. You also do not overly (and artifically) lower the |
||
119 | score of a message, just for SA-Exim's sake |
||
120 | |||
121 | If you so wish, you can also give the SA rule a score of -0.1, and only |
||
122 | dynamically raise the tempreject score for messages that are whitelisted. |
||
123 | |||
124 | |||
125 | SCORE SETUP |
||
126 | ----------- |
||
127 | It makes little sense to have |
||
128 | SAtempreject + SAgreylistraisetempreject + SA GREYLIST_ISWHITE > SApermreject |
||
129 | as there is little point to raise SAtempreject if the message that's |
||
130 | whitelisted still gets refused by the SApermreject score |
||
131 | |||
132 | As to whether you want to put more points into SA GREYLIST_ISWHITE or |
||
133 | SAgreylistraisetempreject, this is your call, but as a general rule, you only |
||
134 | want to change the SA score in a way that makes sense for spam scoring, |
||
135 | as it similarly affects the score of all messages, whether SA-Exim sees them |
||
136 | in the non spam range, tempreject range, or "this is spam that I would never |
||
137 | let in" range. |
||
138 | |||
139 | |||
140 | FILE SETUP |
||
141 | ---------- |
||
142 | Make very sure that uid nobody can traverse /var/spool/sa-exim and |
||
143 | create tuplets writeable by nobody (or whoever you run SA as) |
||
144 | |||
145 | Then, setup a cron job to delete tuplets that are older than 14 days for |
||
146 | whitelisted entries, and 2 days for greylisted entries (or whatever |
||
147 | values you fancy). |
||
148 | Note that because this implementation does not systematically force the senders to resend you mail, unless they sent something that looks too much like spam, |
||
149 | you will typically see few whitelisted entries, and those will either be |
||
150 | potential spam that was actually resent to you at least 30mn after the |
||
151 | initial copy (or whatever value you setup in "header GREYLIST_ISWHITE"), or |
||
152 | people who sent you several Emails (where the second Email will just happen to |
||
153 | trigger a whitelisting). |
||
154 | |||
155 | |||
156 | FILE SETUP |
||
157 | ---------- |
||
158 | You should install greylistclean.cron in /etc/cron.d/ on your system to |
||
159 | call greylistclean and clean up greylisted entries and whitelisted entries |
||
160 | that haven't been used in a while. |
||
161 | You can optionally modify it to tweak the cleanup times. |
||
162 | Note that you need to tweak greylistclean.cron to match the user spamd runs |
||
163 | as if you aren't using the recommended --username=nobody |
||
164 | |||
165 | |||
166 | SA PATCH (SA 2.x) |
||
167 | ----------------- |
||
168 | For all this to work, you also need to patch SA with SA-greylist.diff |
||
169 | from the source tar (or /usr/share/doc/sa-exim*/ for a precompiled package). |
||
170 | This patch never made it to the main SA 2.x branch as the developers had mostly |
||
171 | switched to 3.x where you can use plugins. |
||
172 | If you still use SA 3.x, you can go to /usr/share/perl5/Mail (or wherever |
||
173 | appropriate on your system), and run |
||
174 | patch -p0 -s < /path/to/sa-exim/SA-greylisting.diff |
||
175 | Note that while the patch works, it will not be maintained anymore since |
||
176 | it is deprecated for the SA 3.x plugin |
||
177 | |||
178 | |||
179 | SA PLUGIN (SA 3.x) |
||
180 | ------------------ |
||
181 | Newer versions of SpamAssassin support plugins, so there is no need to |
||
182 | patch SA anymore, you can just install the Greylisting.pm module on your |
||
183 | system and get SA to use it |
||
184 | This is how you call the module in SA 3.x (i.e. put this in your |
||
185 | /etc/spamassassin/local.cf) |
||
186 | |||
187 | # Note the 'key' -> 'value'; syntax. It's a special hack to go through SA's |
||
188 | # config parser. You need to keep that exact syntax |
||
189 | # greylistsecs: how long you greylist a tuplet because whitelisting it |
||
190 | # greylistnullfrom: set to 1 to also greylist mail with a null env from |
||
191 | # greylistfourthbyte: keep the 4 bytes of the connecting host instead of 3 |
||
192 | loadplugin Greylisting /usr/share/perl5/Mail/SpamAssassin/Plugin/Greylisting.pm |
||
193 | header GREYLIST_ISWHITE eval:greylisting("( 'dir' => '/var/spool/sa-exim/tuplets'; 'method' => 'dir'; 'greylistsecs' => '1800'; 'dontgreylistthreshold' => 11; 'connectiphdr' => 'X-SA-Exim-Connect-IP'; 'envfromhdr' => 'X-SA-Exim-Mail-From'; 'rcpttohdr' => 'X-SA-Exim-Rcpt-To'; 'greylistnullfrom' => 1; 'greylistfourthbyte' => 0 )") |
||
194 | describe GREYLIST_ISWHITE The incoming server has been whitelisted for this recipient and sender |
||
195 | score GREYLIST_ISWHITE -1.5 |
||
196 | # Run SpamAssassin last, after all other rules. |
||
197 | # (lets us not greylist a host that is sending spam, otherwise this rule might |
||
198 | # set a sufficiently negative score that the next spam would be allowed in) |
||
199 | priority GREYLIST_ISWHITE 99999 |
||
200 | |||
201 | |||
202 | SA-EXIM NEW BEHAVIOR CONCERNS |
||
203 | ----------------------------- |
||
204 | What greylisting changes as far as spam accepting or rejection is concerned: |
||
205 | Once a tuplet has been whitelisted, spam from that host is more likely |
||
206 | to be accepted until the tuplet expires. In the case of a mailing list, |
||
207 | unless you run a find / rm based on the creation time and not the last |
||
208 | modified time, you will then be a bit more likely to accept spam from |
||
209 | that list. |
||
210 | If this turns out to not be acceptable in your case, there isn't a whole |
||
211 | lot you can do about this, except deleting greylist entries for the host |
||
212 | from cron before they get promoted to whitelist. |
||
213 | |||
214 | What you can do on top of the existing greylisting code: |
||
215 | Parse the SA-Exim logs and if you get spam from an IP, you can decide |
||
216 | to delete greylist entries in /var/spool/sa-exim/tuplets/IP or just |
||
217 | /var/spool/sa-exim/tuplets/IP/envfrom |
||
218 | This may not may not be a good thing if you receive the occasional spam |
||
219 | from a mailing list as you'll then re-delay mail for that list, but then |
||
220 | again, it will also remove whitelisting for a host that spammed you once |
||
221 | with an Email that managed to get under the SA scoring radar |
||
222 | |||
223 | |||
224 | GREYLISTING AND MXES |
||
225 | -------------------- |
||
226 | Depending on your configuration, you may have realized that SA-Exim doesn't |
||
227 | play very well with secondary MXes for your domain if they don't run SA-Exim |
||
228 | too (for instance, you'd send a tempreject on spam and clog up your |
||
229 | secondary, or maybe even teergrube it if you forgot to add your MX's IP |
||
230 | in the do not teergrube list. |
||
231 | For greylisting, it's even more simple: |
||
232 | If your secondary MXes aren't running SA-Exim with greylisting, then |
||
233 | greylisting's efficiency will be greatly reduced as most spammers will send |
||
234 | their spams to your secondary MXes which will accept the mail for you, |
||
235 | even if it's sent only once, and then your MXes will resend the spam to you |
||
236 | until you accept it (rendering greylisting useless) |
||
237 | |||
238 | Now, if your secondaries are running greylisting too, most mail will flow |
||
239 | through with no delay whatsoever. However, in the worst case scenario, a mail |
||
240 | that isn't spam, but triggers greylisting because its score is high enough to |
||
241 | generate a tempreject, could be delayed up to twice the whitelisting time |
||
242 | if it were to go to your secondary MX first (assuming your primary is |
||
243 | unreachable or temporarily overloaded), and then be resent to your primary |
||
244 | MX, which would trigger a second greylisting delay |
||
245 | FIXME: implement a whitelist of sending IPs so that greylisting returns |
||
246 | whitelisted right away |
||
247 | |||
248 | |||
249 | SECURITY |
||
250 | -------- |
||
251 | The greylisting function works around the SA parser by sending all the options |
||
252 | as a hash inside a string. In turn, greylisting evals the said string. |
||
253 | This is a security problem if you allow your users to run custom rules and it |
||
254 | gives them access to run spamassassin as a user different from their own, or |
||
255 | in a way that they otherwise wouldn't be able to. |
||
256 | Do not run greylisting if this a problem for you (in the default SA/SA-Exim |
||
257 | setup, this shouldn't be a concern since it doesn't even parse users' config |
||
258 | files) |