Date: 2024-04-10
Author: Simon Jackson
Lets explain..
I had a customer today mentioning his mails are always being reported as arriving in spam.
I talked my customer through SPF and DKIM setup... and it was all setup pretty much perfectly.
Emails were still arriving in the SPAM/JUNK-Email Folder...
I viewed the customer mail headers, and sent them to MX Toolbox's Analyze Headers tool - and this is the report: https://mxtoolbox.com/Public/Tools/EmailHeaders.aspx?huid=2efca3fe-7a98-4183-a117-9e575cb6c0bc
Before I dig into what exactly is happening. I figured i'd sketch a little diagram; I do love a good old diagram.
Find out who your DNS Provider is. GoDaddy, CloudFlare, AWS, SquareSpace.. whoever it is - find their administration console to manage DNS records.
The following record is an example, showing what a Google Mail (Gmail) hosted company would use. Find the domain name in question, find the TXT record with `v=spf1` at the start of the value. Then edit the record. Find a space before the `-all` (or ~all" whichever you have) and paste in ONLY the highlighted section of text from the table below. Be careful not to modify any other items in that record. Finally click Save.
Now we have 5x Processes. List explain what they do exactly.
Gmail needs to resolve the MX record of the recepients mail-server (or Receiving MTA). This is achieved with a simple DNS lookup.
Gmail Workspace has registered customers, each customer with their own domain-name has their own 'Authorised DKIM settings' for outbound mail delivery.
Google Workspace Administrators can set up DKIM Authentication in two parts...
A. Goto admin.google.com
Navigate to:
Apps > Google Workspace > Settings for Gmail > Authenticate Email > DKIM Authentication
Click on Generate New Record.
For reference, here is mine:
B. The above Generated DNS record needs to be copied and pasted for public DNS resolution to work. The reference picture to the right shows a private RSA key generated with an AES 256 bit algorithm ( key length = 1024-bit).
2048-bit is recommended.
If you don't wait long enough after copying the record into public DNS, then you cannot enable Authentication signing. (Start Authentication button)...
If you get this.. come back in an hour.
There is technically a TLS layer here - but it's out of scope for the discussion point of this article. I will cover that in another blog in future, and provide a link here.
3. Shortly after an SMTP connection is established, and the FROM and TO headers are exchanged.. (before the payload of the email envelope is sent)... the receiving MTA has a job to do. To filter out absolute junk, this is achieved firstly with a SPF record check.
This is a simple DNS lookup for a TXT record, of the TO field in the email headers. Domains might have more than one TXT record, so they are filtered for the record beginning with `v=spf`.
The next character defines the sender protection framework number to follow (1), which has got a published RFC standard for the actions expected under what condition.
- If the TXT record doesn't match, the connection is dropped. No rejection, no bounce, no response what-so-ever - the mail never reaches a processing queue!
- If the TXT record matches the connection - the payload is accepted and queued for processing.
4. Email Headers are full of useful information; including the DKIM-Signature from the original email.
There may be more than one signature, this depends on how many mail-providers it's relayed through!
The one that has the DKIM selector from the FROM domain - in our case `s=google`. Here is such a signature:
ISSUE IDENTIFIED
The private key starting (b=xxxxxxxx) is supposed to be an un-terminated string. I have highlighted two spaces, to clearly demonstrate that an outbound, gmail-initiated, gmail-dkim-signed private key contains spaces, which breaks the Verification of Public-Key and Private-Key pairs.
->
SOLUTION
Gmail themselves need to identify the failing component, internally within their mail-system. Likely a code block that is involved with the DKIM Private-Key creation process, and they need to correct this to a single un-terminated string.
5. The final step is extremely useful - checking for the sending domain's DMARC policy within public DNS. Honour the rules defined in that policy.
Policies include:
"None" - We are not defining a action for our policy, you do what you want with it.
"Quarantine" - We don't trust the sender; as they don't comply to either SPF or DKIM checks - we recommend you quarantine the mail (mark it as spam, and or place it on hold for review)
"Reject" - We don't trust the sender; as they don't comply to either SPF or DKIM checks - we recommend you reject mail the delivery.
There are other parameters that CAN be included within a SPF, DKIM and DKIM configurations.
It's worth referencing SPF, DKIM and DMARC published RFCs:
A friend and long-term coleague shared this beautiful little website. it really helps wrap your head around where each parameter in which SPF, DKIM and DMARK come from.