Author Topic: Sent emails disappearing or going to recipients spam  (Read 1549 times)

Sent emails disappearing or going to recipients spam
« on: 01 November, 2021, 12:51:07 pm »
This seems to be an increasing problem for me, and it seemed to ramp up when we went into the first lockdown. I send a lot of business emails with attachments (pdf's of cad drawings mostly) and even to clients I've been dealing with for years without problem are now reporting lost or spam filtered emails. It's got to the point where I have to send an email to say I'm going to send an email with attachments, and even that one might not arrive! Mails are sent from my own domain via names.co.uk which I've used for 20+ years. I've contacted them - initially I got the stock response to say it was the other party's fault and I shouldn't include words and phrases like 'buy direct' in my emails (which I don't). I then asked about the TXT entry to the current record - v=spf1 include:spf.hosts.co.uk ~all - which they said they had updated (looked the same to me). The problem persists. Can anyone with more technical nous than me help?
I've also seen some of my .mac emails go to the recipients junk as well, which is very surprising.

Feanor

  • It's mostly downhill from here.
Re: Sent emails disappearing or going to recipients spam
« Reply #1 on: 01 November, 2021, 01:28:48 pm »
Things change over 20 years!
Spf didnt exist 20 years ago, but it looks like that's been set up.

However, spf by itself is no longer good enough for many recipient mail servers.

You need to set up dkim and dmarc also.
If you have a 3rd party providing your mail service, they need to do this.
If you are runing your own mailservers then its your job, and can be one reason not to run your own if the admin burden is becoming too much.

Thats a starting point.

Re: Sent emails disappearing or going to recipients spam
« Reply #2 on: 01 November, 2021, 01:47:31 pm »
Thanks, that's good - I need to know what to ask for! I've just run an audit with postmastery - as well as dkim and dmarc flagged, this was at the top of the list (as I use BT broadband);

The email was sent from the following IP address: 213.120.69.101 with HELO name re-prd-fep-045.btinternet.com. A mail server IP should also have a hostname which can be verified by DNS lookups in both directions. This is called the 'Forward Confirmed reverse DNS' (FCrDNS) test.
There is a reverse DNS (PTR) record with value mailomta8-re.btinternet.com. but it does not match the HELO name.
There is a forward DNS (A) record with value 213.120.69.94 but it does not match the IP.

Kim

  • Timelord
    • Fediverse
Re: Sent emails disappearing or going to recipients spam
« Reply #3 on: 01 November, 2021, 01:50:21 pm »
"Damn and blast British Telecom!" said Kim, the words coming easily through force of habit.

I'm not sure getting fussy about what's sent in the HELO field is necessarily sensible (I don't think the RFC requires it to be a valid domain name, just something that identifies the server), but if recipient mail servers are doing it, it's going to have to match.

Just checked, and AAISP has the same problem.


Feanor

  • It's mostly downhill from here.
Re: Sent emails disappearing or going to recipients spam
« Reply #4 on: 01 November, 2021, 02:10:47 pm »
Requiring valid rDNS has been a thing for a long time, IIRC.

On my own hmailserver here, you can configure the helo to match the external name the machine has in DNS / rDNS.

This is necessary in my case because the machine has a different Internal domain / hostname vs its external name. The mailserver software by default picks up the local internal name, having no knowledge of my external DNS configuration.

This stuff really is arcane for an end user, and your mail provider should be on top of it.

Re: Sent emails disappearing or going to recipients spam
« Reply #5 on: 01 November, 2021, 02:19:55 pm »
Are you running your own mail server? When I ran my own I had to relay outgoing mail via my isp’s mail server as anything coming from my own IP address was very prone to being flagged as spam. The general pool of ip addresses that the isp was allocating out was viewed as highly suspect by a lot of receiving mail servers.

(I gave up running my own mail server in the end as I couldn’t keep up with continually adding and modifying spam detection rules.)

Re: Sent emails disappearing or going to recipients spam
« Reply #6 on: 01 November, 2021, 02:34:21 pm »
Are you running your own mail server? When I ran my own I had to relay outgoing mail via my isp’s mail server as anything coming from my own IP address was very prone to being flagged as spam. The general pool of ip addresses that the isp was allocating out was viewed as highly suspect by a lot of receiving mail servers.

(I gave up running my own mail server in the end as I couldn’t keep up with continually adding and modifying spam detection rules.)

No, I'm not, that would be beyond my technical abilities I think! I have my own domains and use them for mail addresses, as well as .mac and gmail emails for private, as opposed to business, emails.

Re: Sent emails disappearing or going to recipients spam
« Reply #7 on: 01 November, 2021, 04:10:40 pm »
Hmmm, names.co.uk now tell me "Unfortunately DKIM and DMARC is currently unavailable for our email services." It seems I may need to consider moving my business to another provider who does a bit more than just take my money, although that fills me with horror. Any recommendations to think about?

Re: Sent emails disappearing or going to recipients spam
« Reply #8 on: 01 November, 2021, 04:53:56 pm »
Google will deal with your email for your domain. Think you need Google workspace with Google domains or something similar.   Sounds like your current provider is sticking with their 1970s model…

If you let Google manage, the falling into a spam black hole will largely disappear. Sure Microsoft will offer something similar etc.

Sadly spam filters have gone mad in recent years whenever you include attachments and aren’t with the big boys

Re: Sent emails disappearing or going to recipients spam
« Reply #9 on: 01 November, 2021, 04:59:14 pm »
Krystal OK and good VFM. Customer help was good the last time I used it (ages ago).
Little bit of a learning curve as everything is configurable.

Re: Sent emails disappearing or going to recipients spam
« Reply #10 on: 01 November, 2021, 06:56:22 pm »
Google will deal with your email for your domain. Think you need Google workspace with Google domains or something similar.   

No, it's much easier, just under "Accounts and Import" -you set up and validate your domain email address and then rules to use (eg, always respond with the recipient email), and you set your MX to Google. There are a couple more steps to use GMail SMTP for send, but nothing too strenuous

Re: Sent emails disappearing or going to recipients spam
« Reply #11 on: 02 November, 2021, 11:11:52 am »
Hmmm, names.co.uk now tell me "Unfortunately DKIM and DMARC is currently unavailable for our email services." It seems I may need to consider moving my business to another provider who does a bit more than just take my money, although that fills me with horror. Any recommendations to think about?

Fastmail.com
I think you'll find it's a bit more complicated than that.

Re: Sent emails disappearing or going to recipients spam
« Reply #12 on: 02 November, 2021, 12:10:01 pm »
Hmmm, names.co.uk now tell me "Unfortunately DKIM and DMARC is currently unavailable for our email services." It seems I may need to consider moving my business to another provider who does a bit more than just take my money, although that fills me with horror. Any recommendations to think about?

Fastmail.com
This. It just works and the import is super easy.

Re: Sent emails disappearing or going to recipients spam
« Reply #13 on: 02 November, 2021, 03:16:23 pm »
Thanks for all your comments and suggestions - really helpful. Namesco have come back to me to say;
What is happening with your emails, is that recipient's email servers can see it is a Namesco email address and is being sent by a BT email server.
Other email servers will then believe they are spam emails as they have come from a location they should not have. We highly recommend using our Authenticated SMTP server to send.

This for a mere £21 pa. I did used to have it, it made no difference, and BT is STMP anyway, so I cancelled it.
It doesn't explain why I can send from my .mac address and a gmail address, without any issues.
It seems that without DKIM and DMARC it's no brainer to move away.
I have 2 domains, business and hobby, with web sites at TSO (who I'm not that impressed with) and it seems sensible to move everything to one place.
I should say I'm a POP3 man for all my business stuff.

Re: Sent emails disappearing or going to recipients spam
« Reply #14 on: 02 November, 2021, 06:48:11 pm »
What is happening with your emails, is that recipient's email servers can see it is a Namesco email address and is being sent by a BT email server.
Other email servers will then believe they are spam emails as they have come from a location they should not have. We highly recommend using our Authenticated SMTP server to send.

This for a mere £21 pa. I did used to have it, it made no difference, and BT is STMP anyway, so I cancelled it.
In case it's not obvious to you, the point is not about it being an SMTP server. SMTP is just the simple mail transfer protocol - implementing the basic rules for sending out mail that all servers have to follow. The focus is on the mismatch between mail that clearly is sent by a BT server, and yet claims to come from a Namesco address. That discrepancy is often a sign of spam, even though there are plenty of legitimate reasons for it to arise as well, yours being one.

This is where the Internet comes in. It's not a single entity; it's an Inter-networking of separate systems, each with their own systems managers, who get to decide what mail they will and will not accept. It's perfectly legitimate for one systems manager to reject mail with the above mismatch, and another to accept it. They are running their own systems, and are accountable for preventing virus and spam attacks. And sometimes the amount of rejected spam can overwhelm amounts of legitimate mail.

The authenticated SMTP comes in because it's foolish for an SMTP server to accept mail from just anyone. It would be asking to get used for sending spam. So, typically, BT will only relay mail sent from BT customers on BT connections, and so on and so forth. But Namesco probably don't provide Internet connections. So, they require your mail client to provide a username and password (authentication) when using their SMTP server to relay mail.

In theory, therefore, Namesco are offering a sensible solution that means that your mail will come from a Namesco SMTP server and a Namesco address, and other systems managers will happily accept it. In theory. But your experience suggests that they might not.

POP3 (Post Office protocol) is usually used for receiving mail, as opposed to sending it. Lots of stuff about email is like the real mail. Different rules for sending and receiving is one of those things. Receiving real mail, you wait eagerly by your front door for the sound of the letter flap. Sending it, you walk to the post box, and only the sender needs a stamp.

Re: Sent emails disappearing or going to recipients spam
« Reply #15 on: 02 November, 2021, 07:15:18 pm »
Friends don't let friends use POP3.

Seriously this is the 21st century IMAP is far superior.
I think you'll find it's a bit more complicated than that.

Re: Sent emails disappearing or going to recipients spam
« Reply #16 on: 02 November, 2021, 07:33:35 pm »
Readers may substitute any of the following in the above, and continue the conversation without a pause:

For POP3: upright bikes, trikes, steel or road bikes

For IMAP: recumbents, upwrongs, carbon or MTBs

Re: Sent emails disappearing or going to recipients spam
« Reply #17 on: 02 November, 2021, 07:36:52 pm »
Interesting that you are using TSO and BT and are having issues.

https://yacf.co.uk/forum/index.php?topic=121160.0
"No matter how slow you go, you're still lapping everybody on the couch."

Re: Sent emails disappearing or going to recipients spam
« Reply #18 on: 02 November, 2021, 07:47:10 pm »
Readers may substitute any of the following in the above, and continue the conversation without a pause:
For POP3: upright bikes, trikes, steel or road bikes
For IMAP: recumbents, upwrongs, carbon or MTBs
Well, I have an alloy 'all seasons' bike, a steel tourer, an alloy MTB, an alloy Hybrid shopper and an old steel MTB. No carbon. I think I lean well into the POP3 camp!
I've nothing against IMAP - I use it for my .Mac stuff, but it's never seemed 100% reliable and I have lost stuff on occasions (this may be my fault or Apple's and not a generic IMAP issue).

Re: Sent emails disappearing or going to recipients spam
« Reply #19 on: 02 November, 2021, 08:21:01 pm »
Interesting that you are using TSO and BT and are having issues.

https://yacf.co.uk/forum/index.php?topic=121160.0

I only use TSO for hosting the sites - I moved to them (when they were Vidahost) mid 2016, which was before the present mail issue. BUT, looking back through emails from them, they did some transfer to Cloudmark in March 2020, which was about when this issue started. They said I should add v=spf1 ip4:185.24.99.18 include:secureserver.net to my DNS, but I didn't as I don't use them for email and forgot about it. In May they said they were migrating to cPanel and charging more for the privilege. I'm still waiting for that.

Gattopardo

  • Lord of the sith
  • Overseaing the building of the death star
Re: Sent emails disappearing or going to recipients spam
« Reply #20 on: 03 November, 2021, 08:10:58 pm »
Have had this too recently, from companies.  How do you prove that you didn't get the email?