This is another aide-memoire about changes in mail on Tarragon.

Some weeks back a change in one of the website contact pages was done and the captcha code was inadvertantly omitted. There followed a period of massive junk email directed at the owner of the site, on her google gmail account. Google decided to cut off tarragon’s ip address.

Although the problem is fixed, google has not relented. And this is the same kind of issue I have had in the past with microsoft. Although I have never been a source of spam, the big mail outfits are quick to ban the ip address of any small personal smtp server, and it takes a lot of effort to convince them to release the ban. I am tired of it. Despite my quixotic desire to run my own mail server as a symbolic cry against the erosion of personal services on the internet, I am tired of fighting, and I think it is time to stop.

But I will not yield completely. I can still operate the postfix and dovecot instances, along with the ancillary amavis/spamassassin/clamav/opendkim services; I can still control my own spam filtering; I can continue to have as many different mailboxes as I need so as to preserve my private email and use other mailboxes for public things. I can set the spam filtering levels as tight as I need and quarantine the marginal crap. I can still offer authenticated smtpd submission service to all my other boxes, so they have a place to send mail.

The only thing I can’t do, any longer, is operate the smtp service, i.e. send mail out on port 25 from tarragon. Instead I have set up a relayhost in the tarragon main.cf, pointing at the Amazon SES service.

I used the Amazon SES (Simple Email Service) service in the past on my friend’s cancer website, because she had 7000 users registered, and on those very few occasions when she wanted to send email to all of them with an announcement, we knew that there were going to be lots of bounces and issues. So using Amazon’s SES coupled with the SNS (Simple Notification Service) enabled us to track bounces and try to repair or mask the users who had bad email addresses. When debugging that capability I had set up the SES and SNS in my own account as well, and played with it a little, trying to learn how it worked, so I could do it for her. Now I want to go back and use that facility for my production email sending.

The things one has to do are, first to prove ownership either of a domain or a particular email address. I have done both. Usually (for a domain) this would involve putting in cname records, but Amazon is clever enough to figure out if the domain is hosted on its own service by the requesting user, and if so, it adds the appropriate records itself. The records it adds are actually the necessary DKIM records. Instead of using the normal sort of TXT or CNAME records, they put in the DKIM records and then use the presence of those as indication of ownership.

I added wmbuck.net, thegraygeek.com and wmbuck.com as verified domains. I used some features that Amazon supplies to do test sending from these domains, from the console, and that worked.

The next thing I had to do was get Amazon to allow “production” use. When you start up with SES, they initially put you in “the sandbox”, and you can only send messages to designated email addresses (those which you have proved that you are the owner of – see above). To use this for sending “for real”, you have to fill out a form in which you explain that you are not a spammer. Now it turns out that the form you use is the same form that one uses to increase one’s sending limit. At the outset, while in the sandbox, one is limited to 200 emails per day. That would be fine for me. I filled out the form, and explained why I wanted to use the service. Just for amusement, I reproduce here what I wrote:

Production access request Service: SES Sending Limits
Please enable production access 
Use case description: I have run postfix in my EC2 instance for 15+ years, but it is a constant struggle keeping up a reputation so I don't get blocked. I send no spam, but recently a website I host was updated without a captcha, and started sending a bunch of email to the website owner, on gmail, and the server got blocked by google. The problem is fixed, but I'm tired of dealing with this. You big outfits really want all the mail to go through the big servers, so I'm giving up. I have migrated all mail other than my own off the server. I want to start sending my mail through the amazon ses servers and stop trying to fight with the big guys. I'm a 75 year old retired geek, and I want to maintain control of my private mail, for spam filtering etc., but I want to start relaying my small amount of mail through your SES server. Your metrics are useful, and the 200 emails a day limit is more than I need. I intend to stop sending directly from the EC2 instance and change my postfix to relay through SES.

Note that I specifically said that the current 200 emails per day limit was more than adequate. After 2 days I got a response, in which they said, inter alia:

Thank you for submitting your request to increase your sending limits. Your new sending quota is 50,000 messages per day. This takes effect immediately in the US West (Oregon) region. ... We understand that you requested a larger sending quota than we granted. We granted the largest increase that we could with the information that was available to us. ...

So, ok, they don’t really read what you write, and instead of a limit of 200 emails per day, I have a limit of 50,000 emails per day, and 14 emails per second. I think it will suffice.

Then I had to change postfix on tarragon.

The changes I made were as follows:

• I added a sasl_passwds file with the userid/password for the SES service, and hashed it. See note below – that turned out to be confusing
• I added a tls_policy file with the requirements for tls to the SES service, and hashed it.
• I added in main.cf:
  • relayhost = [email-smtp.us-west-2.amazonaws.com]:submission
  • smtp_sasl_auth_enable =yes
  • smtp_sasl_security_options = noanonymous
  • smtp_tls_security_level = encrypt
  • smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwds
  • smtp_sasl_tls_policy_maps = hash:/etc/postfix/tls_policy

The effect of all this is that when postfix wants to send mail it authenticates to amazon using the credentials in sasl_passwds, and then relays the mail through the submission port 587 at the amazon service.

The credentials were tricky I found. I had a number of accounts I had set up with the Amazon IAM service (Identity and Access Management, I think it stands for). This is a very powerful and therefore complex and convoluted service through which one sets up different identities with different roles and privileges. I had done this several years back, so that various things could be done at Amazon programmatically without my having to store my master credentials in various scripts. I have a separate document with a list of all the Amazon user ids, “AWS Access Keys” and “Secret Keys” (passwords).

Long story short, I tried to use one of the existing AWS identities for sending mail. And I was being rejected. It turns out that one has to have an Amazon SES SMTP username and password which are separate from and different from the AWS User Identity, and which you can establish when you create the Identity – but I could find no way to add SMTP username/password to an existing identity. I created a new User “tarragon” for this. One does this on the Amazon SES->SMTP Settings page, where you can “Create SMTP credentials”. There is a notice there: “You must have an SES SMTP user name and password to access the SMTP interface. These credentials are different from your AWS access keys and are unique to each region.” This accompanies a button to “Manage my existing SMTP credentials” which takes you into the IAM module, but (as I said) I couldn’t find a way to add SMTP credentials to an existing identity.

Back on the SMTP Settings page, in addition to the “manage existing” button, there is also a “Create SMTP credentials” button, and if one uses that to create a new user, one gets the opportunity to get SMTP credentials for the new user. This is a completely different pair of strings, like

• userid: AKIAVQUMSDL3IOTJMTJB
• password: QFNyRlcWHcjN8JvLr3J/Kn7b540Qc3SzfwqdSydbF3Ne

(I have messed these up – they are not valid). These are similar to the SMTP credentials for user tarragon, who is a member of the (newly created) user group “net.wmbuck.mailsending”, which group has the permission policies: AmazonSESFullAccess and IAMUserChangePassword.

Once I had the userid and password, and put them into the sasl_passwds file, I restarted postfix on tarragon and it began to send through Amazon SES. My highpoint of emails per day, so far during the first 7 days is a whopping 6 emails.