我们或许可以通过让贝叶斯垃圾邮件过滤器追踪邮件中的链接,看看链接另一端是什么,来提高过滤的准确率。death2spam 的 Richard Jowsey 目前在处理疑似垃圾邮件的边缘案例时就采用了这种方法,并反馈说效果很好。

We may be able to improve the accuracy of Bayesian spam filters by having them follow links to see what's waiting at the other end. Richard Jowsey of death2spam now does this in borderline cases, and reports that it works well.

但为什么只在边缘案例中这样做?又为什么只做一次呢?

Why only do it in borderline cases? And why only do it once?

正如我在《过滤器会消灭垃圾邮件吗?》一文中所提到的,追踪垃圾邮件中的所有 URL 会带来一个有趣的副作用。如果主流邮件客户端都通过这种方式来过滤垃圾邮件,垃圾邮件发送者的服务器将会承受沉重的打击。我越思考这个问题,就越觉得这是个好主意。这不仅有趣,而且很难想象还有什么能比这更精准地反击垃圾邮件发送者了。

As I mentioned in Will Filters Kill Spam?, following all the urls in a spam would have an amusing side-effect. If popular email clients did this in order to filter spam, the spammer's servers would take a serious pounding. The more I think about this, the better an idea it seems. This isn't just amusing; it would be hard to imagine a more perfectly targeted counterattack on spammers.

因此,我想给那些开发垃圾邮件过滤器的人建议一个新功能:一个“惩罚”模式。一旦开启,它就会对疑似垃圾邮件中的每个 URL 进行 $n$ 次爬取,其中 $n$ 的数值可由用户自行设置。[1]

So I'd like to suggest an additional feature to those working on spam filters: a "punish" mode which, if turned on, would spider every url in a suspected spam n times, where n could be set by the user. [1]

正如许多人指出的那样,现行电子邮件系统的问题之一在于它太被动了。你让它做什么,它就做什么。到目前为止,所有解决这个问题的建议似乎都涉及新协议。而这个方案不需要。

As many people have noted, one of the problems with the current email system is that it's too passive. It does whatever you tell it. So far all the suggestions for fixing the problem seem to involve new protocols. This one wouldn't.

如果被广泛使用,这种具有自动获取功能的垃圾邮件过滤器将让电子邮件系统产生“反弹”效应。垃圾邮件庞大的数量——至今为止一直是垃圾邮件发送者的优势——现在将反过来对付他,就像一根树枝猛地弹回,抽在他自己脸上。自动获取的过滤器会拉高垃圾邮件发送者的运营成本,并降低其销售额:他的带宽占用会冲上天际,他的服务器会因超载而瘫痪,从而导致那些真正想响应垃圾邮件的人无法访问。

If widely used, auto-retrieving spam filters would make the email system rebound. The huge volume of the spam, which has so far worked in the spammer's favor, would now work against him, like a branch snapping back in his face. Auto-retrieving spam filters would drive the spammer's costs up, and his sales down: his bandwidth usage would go through the roof, and his servers would grind to a halt under the load, which would make them unavailable to the people who would have responded to the spam.

你一小时群发一百万封邮件,你的服务器一小时就会迎来一百万次访问。

Pump out a million emails an hour, get a million hits an hour on your servers.

我们需要确保这只针对疑似垃圾邮件进行。通常来说,任何发送给数百万人的 URL 都很可能是垃圾邮件链接,因此对每封邮件中的每个 HTTP 请求进行这种操作,几乎在所有时候都能正常工作。但在少数情况下,事实并非如此:例如,从 Yahoo Mail 和 Hotmail 等免费邮件服务发送的邮件底部的 URL。

We would want to ensure that this is only done to suspected spams. As a rule, any url sent to millions of people is likely to be a spam url, so submitting every http request in every email would work fine nearly all the time. But there are a few cases where this isn't true: the urls at the bottom of mails sent from free email services like Yahoo Mail and Hotmail, for example.

为了保护这些网站并防止滥用,自动获取应该与“垃圾推广网站黑名单”结合使用。只有黑名单上的网站才会被爬取,而且网站只有在经过人工审核后才会被列入黑名单。垃圾邮件的生命周期至少有几个小时,因此应该很容易及时更新此类名单,从而干预推广新网站的垃圾邮件。[2]

To protect such sites, and to prevent abuse, auto-retrieval should be combined with blacklists of spamvertised sites. Only sites on a blacklist would get crawled, and sites would be blacklisted only after being inspected by humans. The lifetime of a spam must be several hours at least, so it should be easy to update such a list in time to interfere with a spam promoting a new site. [2]

高流量的自动获取只对高带宽连接的用户才实用,但这类用户已经足够给垃圾邮件发送者带来严重麻烦了。事实上,这个解决方案完美地映射了问题本身。垃圾邮件的问题在于,为了触达少数容易上当受骗的人,发送者会给每个人发信。那些不上当的接收者仅仅是附带伤害。但是,不上当的多数人如果不去阻止(或威胁阻止)那些上当的人做出响应,他们自己就无法免受垃圾邮件的打扰。自动获取的垃圾邮件过滤器为他们提供了一种实现这一目标的方法。

High-volume auto-retrieval would only be practical for users on high-bandwidth connections, but there are enough of those to cause spammers serious trouble. Indeed, this solution neatly mirrors the problem. The problem with spam is that in order to reach a few gullible people the spammer sends mail to everyone. The non-gullible recipients are merely collateral damage. But the non-gullible majority won't stop getting spam until they can stop (or threaten to stop) the gullible from responding to it. Auto-retrieving spam filters offer them a way to do this.

这会消灭垃圾邮件吗?倒也不至于。最大的垃圾邮件发送者大概能保护他们的服务器免受自动获取过滤器的影响。然而,对他们来说,最简单、最便宜的应对方法是在邮件中加入有效的退订链接。而这对于小角色,以及雇佣垃圾邮件发送者进行推广的“正规”网站来说,将是必不可少的。因此,如果自动获取过滤器普及开来,它们就会变成自动退订过滤器。

Would that kill spam? Not quite. The biggest spammers could probably protect their servers against auto-retrieving filters. However, the easiest and cheapest way for them to do it would be to include working unsubscribe links in their mails. And this would be a necessity for smaller fry, and for "legitimate" sites that hired spammers to promote them. So if auto-retrieving filters became widespread, they'd become auto-unsubscribing filters.

在这种情况下,垃圾邮件就会像操作系统崩溃、病毒和弹窗广告一样,成为只困扰那些懒得使用正确软件的人的瘟疫之一。

In this scenario, spam would, like OS crashes, viruses, and popups, become one of those plagues that only afflict people who don't bother to use the right software.

注释

Notes

[1] 自动获取过滤器必须支持重定向,并且在某些情况下(例如一个仅显示“点击这里”的页面)应该追踪多层链接。还要确保 HTTP 请求与主流浏览器的请求无法区分,包括请求顺序和 Referer 属性。

[1] Auto-retrieving filters will have to follow redirects, and should in some cases (e.g. a page that just says "click here") follow more than one level of links. Make sure too that the http requests are indistinguishable from those of popular Web browsers, including the order and referrer.

如果响应在 $x$ 时间内没有返回,则默认判定为较高的垃圾邮件概率。

If the response doesn't come back within x amount of time, default to some fairly high spam probability.

与其将 $n$ 设为常数,不如将其设为提及该网站的垃圾邮件数量的函数。这将针对滥用和意外情况提供进一步的保护。

Instead of making n constant, it might be a good idea to make it a function of the number of spams that have been seen mentioning the site. This would add a further level of protection against abuse and accidents.

[2] 本文的最初版本使用的是“白名单(whitelist)”而非“黑名单(blacklist)”。尽管它们的作用像黑名单,但我更倾向于称它们为白名单,因为这可能会使它们不易受到法律攻击。不过,这似乎只会让读者感到困惑。

[2] The original version of this article used the term "whitelist" instead of "blacklist". Though they were to work like blacklists, I preferred to call them whitelists because it might make them less vulnerable to legal attack. This just seems to have confused readers, though.

可能应该有多个黑名单。单点故障容易受到攻击和滥用。

There should probably be multiple blacklists. A single point of failure would be vulnerable both to attack and abuse.

感谢 Brian Burton、Bill Yerazunis、Dan Giffin、Eric Raymond 和 Richard Jowsey 阅读了本文的草稿。

Thanks to Brian Burton, Bill Yerazunis, Dan Giffin, Eric Raymond, and Richard Jowsey for reading drafts of this.