Help, my outgoing emails are getting marked as spam by Google recipients

[Ian H]

Reading through my gmail spam folder I am struck by the sheer variety of different reasons Google gives for rejection, and in two different colours; red for really nasty and yellow for merely spammish. Charlotte's are yellow and "similar to messages that have been detected by our spam filters".

[Charlotte]

Okay so maybe it's because my marketing emails (sent one by one and usually AFTER a phone conversation with a target person from a PR agency or whatever) are cut and paste jobs with an attached PDF, an image and some HTML?

I've made a few more changes and I'm now running my site at www.charlottebarnes.uk with a redirect from charlottebarnes.CO.uk in order to make sure that both have totally different DNS records so that emails originating from the latter domain are only associated with that server.

Maybe that'll work.  I'll give it a few hours and send some more tests to unfamiliar gmail inboxes.

[Kim]

Hmm, what happens if you send via gmail's SMTP relay?

[Afasoas]

Are you using charlottebarnes.uk for your email ?
I'm not returning an SPF record for it.

I'd expect to see something like:

Code: [Select]

$ dig some.domain SPF

; <<>> DiG 9.9.5-3ubuntu0.3-Ubuntu <<>> some.domain SPF
;; global options: +cm
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54235
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;some.domain. IN SPF

;; ANSWER SECTION:
some.domain. 3411 IN SPF "v=spf1 ip4:111.222.333.444/32 a:some.domain -all"

;; Query time: 58 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Wed Aug 05 19:13:20 BST 2015
;; MSG SIZE  rcvd: 111

I have PMed you my gmail address.

[Charlotte]

Hmm, what happens if you send via gmail's SMTP relay?

I've not tried that.  I suspect that as the outgoing server doesn't match the incoming one, I might be in as much spammy shit.

Are you using charlottebarnes.uk for your email ?
I'm not returning an SPF record for it.

No, I've moved my website onto charlottebarnes.uk temporarily to make sure it wasn't anything to do with my site host, Zenfolio.

[robgul]

It does seem that there are massive inconsistencies - the mass of crap that was getting (funeral plans, voice messages etc) has all but dried up ... but I'm finding that more and more of my outgoing messages are being trapped.

I did read the other day that spam mail volume has dropped massively in the last few months ... perhaps that's as a result of more aggressive filtering/trapping - and that impacts on genuine stuff?

Rob

[Kim]

Hmm, what happens if you send via gmail's SMTP relay?

I've not tried that.  I suspect that as the outgoing server doesn't match the incoming one, I might be in as much spammy shit.

Yes, but that might be cancelled out by being traceable to an authenticated google user...

[Afasoas]

I don't think having the same/different sending/receiving servers will make any difference in terms of spam scoring. Many larger organisations and ISPs/hosts will use different servers for receiving and sending email.

Kim, when you say using Gmail as an SMTP relay, you mean configuring the sending server to use gmail as a relayhost? I don't think Charlotte has that kind of control over the sending email server.

As I've just told Charlotte, it doesn't look like 1and1 support SPF, which isn't great. Hopefully it's just a case that their documentation is out of date.

[Kim]

Kim, when you say using Gmail as an SMTP relay, you mean configuring the sending server to use gmail as a relayhost? I don't think Charlotte has that kind of control over the sending email server.

I was thinking just re-configure the client software.  It appears that Charlotte is using Apple Mail...

[Charlotte]

As I've just told Charlotte, it doesn't look like 1and1 support SPF, which isn't great. Hopefully it's just a case that their documentation is out of date.

They do.  And I've set one.  It just isn't seeming to show up, which is frustrating.

[Afasoas]

Hi Charlotte,

The support is two fold.

Firstly, you need to be able to create an SPF record on the DNS server for your domain. The DNS server for charlottebarnes.co.uk is ns67.1and1.co.uk
Directly querying it for an SPF record doesn't return a result. So there's a problem there. The 1and1 control panel might let you create an SPF record, but the DNS server isn't getting updated.
It might be possible to create a TXT record in the control panel instead and see if that actually gets returned by the DNS server, but that is pretty futile until we we sort out the second fold.


Secondly, to support SPF, the (email) host ideally needs to provide an SPF record for all of their mail servers, which you can include in your SPF record. Or, alternatively, provide details somewhere of all the mail servers they use for sending email. You send email via auth.smtp.1and1.co.uk - but from what I've seen of the email from you, that propagates to a raft of mail servers for actually sending the email. So an SPF record which simply authorises "auth.smtp.1and1.co.uk" to send email on behalf of your domain isn't going to work.

Kim, when you say using Gmail as an SMTP relay, you mean configuring the sending server to use gmail as a relayhost? I don't think Charlotte has that kind of control over the sending email server.

I was thinking just re-configure the client software.  It appears that Charlotte is using Apple Mail...

Now I might totally wrong here, but I'm not sure you can get a gmail smtp server to send email on behalf of another domain from a mail client. I've frigged about with this in the dim and distant past - wound up with my own SMTP server which would then used gmail as a relayhost.


Charlotte,
I'm sorry this is turning into such a pain for you.
I think it's a case of raising it with 1and1's help desk and seeing what they way.

Or shifting your mail back to AAISP, perhaps having checked with them you aren't going to have the same kind of issues.

[fuaran]

Now I might totally wrong here, but I'm not sure you can get a gmail smtp server to send email on behalf of another domain from a mail client. I've frigged about with this in the dim and distant past - wound up with my own SMTP server which would then used gmail as a relayhost.

It was possible. First you have to go into the Gmail settings, then add your other address for "Send mail as". Then setup the Gmail SMTP in your mail client, using your Gmail username/password. I have used this for sending from Thunderbird via Gmail, either from my own domain or a Fastmail address.
Note the "From" address will be your own domain, but the "Sender" address will be your Gmail address. So that address could be visible for some people, and they end up replying to that instead.

But this page suggests it no longer works for new accounts (but if you have set it up previously, it should still work). http://googlesystem.blogspot.co.uk/2014/08/external-addresses-no-longer-use-gmail.html

[Afasoas]

Ahh yes .. I remember now. I wasn't happy that the recipient could easily see the email had been sent via Gmail.

Thanks fuaran.

[Charlotte]

Jesus this is annoying.  I've done everything I possibly can and all the portents are that my mail is not spam:

This message is an automatic response from isNOTspam's authentication verifier service. The service allows email senders to perform a simple check of various sender authentication mechanisms. It is provided free of charge, in the hope that it is useful to the email community. While it is not officially supported, we welcome any feedback you may have at .

Thank you for using isNOTspam.

The isNOTspam team

==========================================================
Summary of Results
==========================================================

SPF Check : neutral
Sender-ID Check : neutral
DomainKeys Check : neutral
DKIM Check : neutral
SpamAssassin Check : ham (non-spam)
==========================================================
Details:
==========================================================

HELO hostname: mout.kundenserver.de
Source IP: 212.227.126.130
mail-from: charlotte@charlottebarnes.co.uk
Anonymous To: ins-fmmx356s@isnotspam.com
---------------------------------------------------------
SPF check details:
----------------------------------------------------------

Result: neutral
ID(s) verified: smtp.mail=charlotte@charlottebarnes.co.uk
DNS record(s):


----------------------------------------------------------
Sender-ID check details:
----------------------------------------------------------

Result: neutral

ID(s) verified: smtp.mail=charlotte@charlottebarnes.co.uk
DNS record(s):


----------------------------------------------------------
DomainKeys check details:
----------------------------------------------------------

Result: neutral (message not signed)
ID(s) verified: header.From=charlotte@charlottebarnes.co.uk
Selector=
domain=
DomainKeys DNS Record=

----------------------------------------------------------
DKIM check details:
----------------------------------------------------------

Result: neutral (message not signed)
ID(s) verified: header.From=charlotte@charlottebarnes.co.uk
Selector=
domain=
DomainKeys DNS Record=

----------------------------------------------------------
SpamAssassin check details:
----------------------------------------------------------
SpamAssassin v3.3.1 (2010-03-19)

Result: ham (non-spam) (02.8points, 10.0 required)

pts rule name description
---- ---------------------- -------------------------------


* -1.0 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
* trust
* [212.227.126.130 listed in list.dnswl.org]
* 3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%
* [score: 1.0000]
* 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.
* See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
* for more information.
* [URIs: charlottebarnes.co.uk]
* 0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
* [score: 1.0000]
* 0.1 HTML_MESSAGE BODY: HTML included in message
X-Spam-Status: Yes, hits=2.8 required=-20.0 tests=BAYES_99,BAYES_999,
HTML_MESSAGE,RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=no version=3.3.1
X-Spam-Score: 2.8

To learn more about the terms used in the SpamAssassin report, please search
here: http://wiki.apache.org/spamassassin/

==========================================================
Explanation of the possible results (adapted from
draft-kucherawy-sender-auth-header-04.txt):
==========================================================

"pass"
the message passed the authentication test.

"fail"
the message failed the authentication test.

"softfail"
the message failed the authentication test, and the authentication
method has either an explicit or implicit policy which doesn't require
successful authentication of all messages from that domain.

"neutral"
the authentication method completed without errors, but was unable
to reach either a positive or a negative result about the message.

"temperror"
a temporary (recoverable) error occurred attempting to authenticate
the sender; either the process couldn't be completed locally, or
there was a temporary failure retrieving data required for the
authentication. A later retry may produce a more final result.

"permerror"
a permanent (unrecoverable) error occurred attempting to
authenticate the sender; either the process couldn't be completed
locally, or there was a permanent failure retrieving data required
for the authentication.




And yet, Google still thinks I'm spam.  The only option is to pay Google to send my mail for me.

Extortion?  Yeah, that   

[Afasoas]

* -1.0 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
* trust
* [212.227.126.130 listed in list.dnswl.org]
* 3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%

I think that needs some digging into.
And the SPF/TXT record is still not getting returned when I ask your 1&1 name server for it.

I'm blaming your hosting.

On Edit:
I visited https://www.dnswl.org/?page_id=72 and fed it the IP address that mail you sent came from:
212.227.126.130

The trust level returned is:

Code: [Select]

Low           Occasional spam occurrences, actively corrected but
              less promptly. This is the default for most categories.



So, in summary, you are using an email service that occasionally emits spam and there are the measures you've tried to put in place to show that your email can be trusted haven't been effective because 1and1 hosting don't update their name server with your requested DNS record changes.



The SPF record is a biggy. If you can get onto 1and1 again and make them fix it, you will be much better placed. Without it, anyone could send email purporting to from charlottebarnes.co.uk which is why mail servers don't trust it. (You need SPF to be verified instead of neutral. It's only neutral instead of pass/fail because there's no rule set)

[Kim]

The spammasassin Bayesian filter is a heuristic based on how similar it looks to previous messages that were unambiguously categorised as either spam or ham^[1].  A sort of "if it looks like a duck and quacks like a duck" test.  I suspect you're unlikely to beat this with a speculative business email, regardless of how you format it.  This is a feature.

However, as you can see, the bayes filter is usually given fairly low weighting in the grand scheme of things (in that example it only makes your email twice as spammy as the presence of HTML).


_{[1] Technical term, the opposite of spam.}

[drossall]

Did you try my reply-to workaround? May not meet your requirements though.

[Charlotte]

Okay, so the SPF record that 1&1 tech suport have carefully placed on file for me is *still* not showing when you interrogate it.  This is clearly wrong - I'll call them in the morning for some more hassle.

Did you try my reply-to workaround? May not meet your requirements though.

No, not yet.  Besides - wouldn't there be the same problem with charlotte@charlottebarnes.uk or anything else I own hosted at 1&1

[drossall]

Depends what tests they are applying. Deliverability shouldn't be affected by anything in Reply-to, and messages should get through if your From address is in the same domain as your mail server. So send from the @1and1.co.uk address that they have probably given you, and see whether that gets through. Then try with charlotte@charlottebarnes.uk in Reply-to.

[Charlotte]

Sadly, having just looked, they will only offer me an address at a domain I own...

[Afasoas]

I've just checked both the originating name servers* for charlottebarnes.co.uk and there's still no SPF or TXT records returned.

[perpetual dan]

http://mxtoolbox.com/SuperTool.aspx? is reporting that most things are OK with your DNS ,including at SPF record.

[Charlotte]

Thanks Dan - yes, a glimmer of light is opening up.  Tech support have escalated this issue and have been dicking with my SPF records for me.

A couple of tests this morning indicate although they might be working, I'm concerned that the DNS changes they have put in place don't conform to best practice.  Specifically, I now have two SPF records, appearing as a TXT file and a SPF file.  It is usually advised never to have more than one SPF record:

http://www.openspf.org/FAQ/Common_mistakes
https://support.google.com/a/answer/4568483?hl=en

My domain centre page now says:

TXT:     charlottebarnes.co.uk. IN TXT "v=spf1 a:mout.kundenserver.de -all" SPF
SPF:     charlottebarnes.co.uk v=spf1 a:mout.kundenserver.de -all

As a non-expert user, I am surprised to have needed to go into this level of detail to solve a problem which ought to have been pre-configured for me on opening my account with 1&1.  I cannot believe that all their hundreds of thousands of email customers can be expected to have to write their own SPF records!
 
Although I am pleased that it looks like it may have been resolved,  I am unsatisfied with the service I have received from 1&1.  When I first contacted them about this issue over a week ago, their customer service agent informed me that I ought to contact all the people that I was emailing and ask them to add me to their whitelist!  He also suggested that it was my fault and that I should change my subject headers.  I strongly suspect that they're not telling me what's really gone wrong here. What do you think?

Also, when testing with Dan's link on mxtoolbox.com, I'm interested to see that only one of the SPF records comes up, no DMARC Records exist and the DKIM looks dodgy (not that I know too much about this).

In addition to my own account, YACF runs off 1&1 servers and I'm the account holder, paying the best part of £750 a year for the service, so I'm hoping that this buys me a bit more clout.

Anyway - if anyone has any gmail (or Yahoo, Virgin or other bigco) accounts that I've yet to test with, can you please PM me with your address?  I'm running out of people to test with and creating gmail accounts is a major PITA.

[Afasoas]

To be honest I have never come across a host or an ISP that does setup SPF records for hosted email on a custom domain. Possibly because for people who choose to host their email elsewhere it would cause as many issues as it would resolve for those that don't. I suppose it wouldn't hurt for them to document it and provide better support.

If you just want to reply to just send me an email, I'll check your are getting SPF verified in your email headers next time near a PC.

[Afasoas]

I should have said with my last post, having two records (one TXT, t'other SPF) is fine. Generic TXT records were used for SPIF upon it's introduction before SPF records were widely supported and some legacy mail systems probably still use them.

I got my mail server up and running a few days ago. I have about 7 different email accounts for four different domains. I sent an email from each account to my Gmail. After about the fourth email, they all started getting spammed. I have set-up SPF records for each domain and DKIM* on the mail server. The mail server is on a dedicated IP which doesn't appear in any black lists (that I can find). So I checked each received mail message.

The first message, in it's headers, showed:

SPF pass
DKIM pass - weak key.

So did the second and third messages.
The fourth email however, had no reference to DKIM.
I checked the server and I'd misspelt the domain name that the fourth email account within the OpenDKIM configuration.

So, it seems sending a quick succession of unsolicited emails (without DKIM?) is enough to get you spammed.


*Albeit with a 768-bit key, which is considered weak by Google. Unfortunately DKIM relies on a DNS TXT record and my DNS restricts the length of it's DNS records. Hence I can't use a 1024 or 2048 bit key. If it continues to be a problem, I could set-up my own name server on the VPS - but I've still got to get some other services running on there and I'm worried about memory pressure.