"And then you're scrambling to figure out how the spammers managed to exploit your setup this time."
...
" ... writing a MTA with the goal of minimizing configuration and being secure and resistant to attacks by default. "
As a 20+ year UNIX sysadmin and fellow owner of my own email infrastructure for 18 of those, I am surprised to read this and am not even sure what you are talking about.
Can you explain what you mean by attacks and exploits from spammers ?
Other than accidentally running as a relay or actually having a remote exploit in your server(s), what are the attacks that you have in mind ?
"Configuration that will soon be exploited in strange ways."
Again, am genuinely interested in an explanation ... other than running as a relay, and maybe handling backscatter (but that's not really config, it's just a blacklist) what are you referring to ?
I'm in the same boat - been running my own mail since the 90s. I could understand complaints about lots of weird hoops to jump through to get mail from your server accepted elsewhere - that has become harder over time[1] - but there's a very short list of things that need to be done to avoid being hijacked.
Mail server setup for the uninitiated does look a little daunting, especially if you're more accustomed to "all-in-one"-ish software. If you want to do this, I recommend starting with making sure you have a clear mental model of mail server architecture, especially where it touches other things (DNS, DKIM, spam filters, local delivery, probably IMAP, maybe LDAP, maybe databases, etc.) Without a clear idea of the dataflow and reasons for different decision-points, you're going to have a very bad time troubleshooting things.
[1] SBCGlobal.net has been rejecting me for over a decade despite there being zero spam ever having been emitted by my domain. Eh, my userbase communicates with exactly one SBC user; ATT can bite me.
I've been running my mail server for 6 years now. Mail to Microsoft's servers (live.com, etc.) keeps ending up in the spam folder, even though spf, dkim and dmarc are set up and my IP has been clean for the entire period. The bright side is that I only notice this in the rare case of "mail to all contacts", since noone is on outlook.com these days.
Yes, live.com is a problem for us too. It seems they silently blacklist ip addresses and make it very difficult to get off that blacklist. There's a thread here about it with some of us trying to figure out how to solve this problem:
But other than that, running my own mail server hasn't been much of an issue. Set up sendmail, use public blacklists for spam control, and it pretty much runs without any intervention.
Outlook.com/Microsoft blacklists entire subnets because there are spammers in the same IP range. It's ridiculously lazy practice. They should just block IP addresses that are actually sending SPAM. It's not like that's a problem technically. You can fit all IPv4 addresses into a 512MB database.
I'm thinking of solving it by blacklisting outlook.com domain, so that senders at least know that I can't respond to them. I can put a message in the error response, that will be reliably relayed to the sender by the sending system.
Google's slightly better. Recently I did an experiment and created a few gmail accounts and sent some riduculously spammy messages full of typical keywords in between those gmail accounts, and they were all successfully delivered. Always.
Then I sent e-mail from new gmail account to my email server and simply responded and it went to spam. It's ridiculous that such a simple heuristic like someone responding to a message gmail user sends doesn't get the message through spam filters, even though the system can clearly determine taht the message is legit based on many variables (References field referencing message-id of the original message (noone else than the recipient should know this), reply being from a correct source (DKIM/SPF), message having normal looking business content, etc.).
There's way too heavy a weight on sending server IP range reputation.
With gmail you can at least request that they unblock you, and they will do that. With live.com and icloud.com you have to spend inordinate amounts of time bouncing between useless support people before you get anywhere. gmail in general seems to have the best spam filter (lowest false positives and negatives).
I had a problem with ipv6 myself with gmail, but the problem was that I just didn't have ipv6 fully set up on my server, so either SPF or reverse DNS wasn't working or something like that. I think I just configured sendmail to only use ipv4 and that solved the gmail issue.
My issue was that I wasn't very familiar with ipv6, and my ISP (OVH) apparently gave my server a range of about 256 ipv6 addresses, and I didn't really know how to properly set up reverse DNS and SPF. After spending a day or two getting nowhere, I just decided to turn off ipv6 completely for the server.
To be fair I've been running my infrastructure for years and it's usually fine.
The problem I have is that it's a pain in the ass to setup correctly and when things do go wrong it's really hard to figure out what's going on.
> Can you explain what you mean by attacks and exploits from spammers?
I haven't encountered a remote exploit (yet) but yes: accidentally running an open relay, backscattering, etc. You also have to worry about the system security of the box you're running it all on but let's treat that as a separate concern.
I'd been running my setup for years and as recently as August of last year I was dealing with an outage of my email server. I must've missed something somewhere because someone had figured it out and started using my server as a relay. It took me a few days to figure it out and I'm still not sure what the problem was or how I fixed it. Though my server stopped bouncing emails and started forwarding things again so... win?
As I was fixing this issue, not the first time, it occurred to me that it was just criminally easy to not know if your system is being exploited as a relay or not. There are a bunch of different configuration files in all different formats and guides that require working through dozens of steps. It's way too easy to get something wrong.
For someone like me who works at deploying software running on hundreds of nodes it seems manageable but I don't think it's ready for my cousin who's good with computers.
That's where the idea for a secure-by-default MTA that couldn't possibly be configured to be an open relay came from. Minimize the configuration so it was hard to get wrong even at the expense of flexibility.
I dunno, maybe it's not a great idea. But it's fun to hack on in the evenings when I've got nothing better to do and hopefully I'll have a fewer text files to manage in the future.
> That's where the idea for a secure-by-default MTA that couldn't possibly be configured to be an open relay came from.
You're close to describing the motivation behind Postfix. Between the design and the documentation, Postfix is hard to screw up, from a security perspective. And it is really easy to configure, at least in contrast to What Came Before - believe me, if you think this is complex, buy a crusty sysadmin a beer sometime and mention 'sendmail.cf'.
The problem here is that almost all mail servers need to selectively relay, and I don't see how the server is going to guess appropriate policy. For instance, trusted IP ranges (mynetworks, in Postfix). I suppose you could demand authentication unconditionally but that tends to break down when not all of your senders are made of meat[1]. Maybe that's acceptable to you, but it won't be for many.
In my other comment in this thread, I recommended gaining an clear understanding the architecture if you're going to do this. That includes things like knowing what (for postfix) mynetworks does - you can get mad at software for not intuiting local policy preferences, but I've never found that to get me very far.
[1] Getting better, but I still depend on a fair bit of software and hardware that doesn't speak SMTP auth.
I've run my own mail server since probably 1998 or so. I have had a few problems here and there, but nothing major. The worst is when the power goes out, but smtp has a wonderful resend feature when the destination server isn't responding.
I know my emails aren't being farmed from the NSA (at least from my side). I know Google isn't scanning all my emails to sell me garbage. I know it won't be shut down because of some errant algorithm. I have a really short email address. I can setup family and friends easily. I can send newsletters without having to deal with someone else's policies (on my side anyway). I love it.
Backup MX is usually not necessary, especially for home setups. Anything RFC-compliant will retry sending its email to you for days after being rejected.
It's also generally not a wise idea unless you also have your anti-spam/anti-virus setup running on it as well, since things like graylisting can only be done BEFORE the MTA accepts the message.
Spammers WILL know how to search your mx records for things like mailbagging servers and try to exploit them to bypass your security, just don't do it - the sending party will retry.
You can chain SMTP like this: SMTP (backup) -> SMTP (spam filter) -> MTA.
Checking white-lists can take a while and some SMTP servers will give up if they're stuck for over a second, sometimes less (eg when your server checks the spam-lists). Some SMTP servers or DNS server (cought Google caugh) will not try secondary though, so if you can afford it - have just one powerful front server with good uptime and network, or load balance / proxy to internal farm.
My backup MX is set up exactly that way, with an almost identical configuration to my main MX, to avoid this problem. Fortunately the config hasn't changed in over five years.
Running since 1998, haven't been on any blocklists so far.
In the age of password leaks it is really common that email accounts are easy targets and get hijacked – most users are probably still using the same password for their email as they used for LinkedIn in 2012. So it might happen from time to time that an otherwise normal account suddenly sends 20 000 emails to hotmail.com unless you have somehow predicted that and configured the mail server to not allow it.
Oh yes it is, we have around 200k users and fighting with old leaked passwords is like a full time job. Very easy to abuse as well, the attackers can make low frequency requests from scattered IPs and thus are almost impoasible to prevent
You added a predicate not present in the statement you replied to, "where mailprovider like LinkedIn_dump.mailprovider". Anecdotally, in authorized enterprise password audits I am frequently forced to remind people who were part of that breach that they need to change all instances of their compromised password, not just the one for the email they used in the breach. Maltego is a thing.
Also, with regard to your claim to have data about which passwords work. You should be aware that accessing someone's account with their username:password without permission (even a breached password) is likely illegal in the United States under the CFAA.
Identity aware proxy. User has to auth over http, then mail client can connect to IAP and ultimately to mail. And or two factor. Giving the user 10% easier usage gives the sysadmins 200% harder job. Take away the 10%.
Most users that reuse passwords statistically did not rotate their passwords across all websites because most users that reuse passwords statistically do not rotate their passwords.
We aren't talking about the overall Linkedin userbase, just that specific subset that reuses passwords for secure services.
...
" ... writing a MTA with the goal of minimizing configuration and being secure and resistant to attacks by default. "
As a 20+ year UNIX sysadmin and fellow owner of my own email infrastructure for 18 of those, I am surprised to read this and am not even sure what you are talking about.
Can you explain what you mean by attacks and exploits from spammers ?
Other than accidentally running as a relay or actually having a remote exploit in your server(s), what are the attacks that you have in mind ?
"Configuration that will soon be exploited in strange ways."
Again, am genuinely interested in an explanation ... other than running as a relay, and maybe handling backscatter (but that's not really config, it's just a blacklist) what are you referring to ?