Our postings seemed to cross when I replied from the phone 
It looks llike a simillar problem to what happened under K9 5.7* ?
A forum thread here on that pointed me to the K9 5.6* apk
What was done to fix that, or was it something different?
Cant you do it all as was working for K9 5.6? And which still works in K9 5.6* today?
Hostgator is an enormous market share …
What K-9 Mail 5.600 did wasn’t great either. See Always send IP instead of hostname in HELO/EHLO by Valodim · Pull Request #3798 · k9mail/k-9 · GitHub
Perhaps using something in the 172.16.0.0/12 range would elicit a less foolish response from poorly configured/managed servers.
Is this for mail dropped directly on gmail (as the configured “outgoing mail server”) or addressed to someone at gmail but dropped on your hostgator server?
Seems to similarly affect inwards Gmail as well.
I’m not seeing this with gmail.
“Inwards” for Gmail, in the sense that it’s being sent to it from K9 5.8* via Hostgator server sending it.
It is fickle sometimes in that mail can get through to some people, perhaps under Reseller accounts on Hostgator.
This is just flying a kite, one man I have mostly been able to send to under 5.8* is under our Reseller account, and 127.0.0.1 might actually make sense then, if that helps as a diagnostic?
I won’t be available to answer for a while – after midnight here now
It appears that this is due to hostgator foolishly filtering based on the EHLO on the (authenticated) submission port. Providers do need to have things in place in case of stolen credentials, but dumping mail because of:
EHLO [127.0.0.1]
isn’t anywhere in the range of “best current practices” when handling authenticated submission.
One last from me for the night …
Is this the same issue as this one refered to earlier here and on the tracker?
https://github.com/k9mail/k-9/issues/4428
Yes, that’s a discussion of this issue. I (for what it’s worth) think that cketti’s position is correct. Do what you want in terms of filtering on smtp/25, but on the submission/587 & 465 ports, which are (intended to be) authenticated, filtering on things like the EHLO is foolish and counter productive. In addition to privacy issues, the IPnumber or name that a mobile device or desktop machine presents has a low probability of resolving correctly (forward and reverse) so should (as is the standard practice) be ignored in an authenticated context. Presenting a fixed value is a reasonable approach.
That hostgator is silently dumping this mail is not standard practice and makes matters even worse. [If for some reason you don’t want to accept mail from your authenticated customer, tell them.]
Ok, thre is a Hostgator “cases-support”: issue:
Case :: 34226747 - HostGator-HostGator-Email-Send/Receive Mail Issues
… Now on their system.
It may be worth pushing that forward from your end using that reference.
I wrote this to them—
For Hostgator technical, can this be fixed please? It stops us sending mail …
A quote from the K9 Email app developer “njeyaakili” on
5.8 showing as sent, but not always arriving at destination - #18 by njeyaakiliAlso refering to this technical discussion …
Change HELO greeting · Issue #4428 · thunderbird/thunderbird-android · GitHub
"Do what you want in terms of filtering on smtp/25, but on the submission/587 & 465 ports, which are (intended to be) authenticated, filtering on things like the EHLO is foolish and counter productive. In addition to privacy issues, the IPnumber or name that a mobile device or desktop machine presents has a low probability of resolving correctly (forward and reverse) so should (as is the standard practice) be ignored in an authenticated context. Presenting a fixed value is a reasonable approach.
"That hostgator is silently dumping this mail is not standard practice and makes matters even worse. [If for some reason you don’t want to accept mail from your authenticated customer, tell them.]
Looking around the web it seemed that some other hosting services are doing the same sorts of messy things.
I doubt that we can clean them all up.
Someone here had suggested adding a permission to K9 and sending the IP-domain-etc for the current connection with the email, even though of course it is not going to resolve backwards always, but it would still be valid going forward?
And would it clear us out of this mess overall?
My message to Hostgator got a lengthy reply from a Tier Administrator which I answered in this way (I’m waiting for another response) …
Thank you for patiently explaining that.
Yes you are right, the K9 developers have wanted to provide some sort of client privacy protection as you have shared.
I know from DroidScript, that if I query my phone 127.0.0.1 is one of the valid IP addresses it could show for it.
For their part as you will have seen, the K9 developers state that a duly authenticated user should not have to worry about all this, asking why would a logged in authenticated user be filtered as potential spam?
As a User I can not understand why such mail would die unnotified? If it is effectively (from the User’s persoective) being “rejected”, wouldn’t it be helpful for it to be returned to “sender” with an appropriate message added to the top of it, rather than just going into a black hole?
I lost several mission critical email moments, from sending mail that successfully to me was nonetheless showing in the Sent folder, but never got to destination, and was effectively (from my perspective) rejected without notification by the Hostgator servers.
I could in such circumstances understand unauthorised mail just being sent to black holes, but authenticated mail?
If authenticated - but apparently viewed as spam mail (possibly from stolen credentials) were rejected and bounced back with a message would that not warn the genuine User that something was up?
And if mail attempting to be sent actually were from a genuine User, would this not warn them that their genuine mail was failing, and why, so that they could then follow up as necessary?
Rather than unknowingly to them (people like me) their mail show as sent, but not be so?
Are there any suggestions for what can be done to resolve this going forward please?
Kindest regards,
Paul
Here’s the reply I received …
Hello Paul,
Thank you for contacting HostGator. I would like to address your concerns separately.
“For their part as you will have seen, the K9 developers state that a duly authenticated user should not have to worry about all this, asking why would a logged in authenticated user be filtered as potential spam?”
The developers are only putting forth their opinion on how a mail server should operate, from their perspective as a developer (and based on how they think email should operate). Simply because a user is authenticated does not, in any way, mean they are not going to send spam. Whether or not a user is authenticated is not a viable method for determining their intent post-authentication, since spam being sent from compromised email accounts has been an issue for decades. We reject tens of thousands of spam emails everyday that are sent from authenticated email clients that have been compromised. I understand their reasoning for using a spoofed IP address during the EHLO transaction, although I believe other methods could be used in an obfuscation effort.
However, for the purpose of what is occurring in this case, the email server itself has no way of determining what you (or anyone else) will be doing after they login, so we must implement at least some spam-fighting techniques at every step of the email transaction process. Many mail providers implement IP checks during the transactional process of sending emails, that way if an IP address may be in a blacklist, the IP can be blocked from ever being able to send an email in the first place, stopping the problem before it starts.
“If authenticated - but apparently viewed as spam mail (possibly from stolen credentials) were rejected and bounced back with a message would that not warn the genuine User that something was up?”
CPanel already has this functionality built-in by default and will clearly send end users notifications if their email accounts hit certain outgoing email thresholds and still, the vast majority of users ignore these notifications entirely, despite the fact that these notifications not only tell the user something wrong is going on with their account, it includes the entire email that was sent but then blocked, as well.
We also have other measures in place that bounce verified spam back to end users, explaining why the emails were flagged as spam, however many users ignore those as well. Even when emails are bounced back from email recipients with rejections that make it very clear that spam was/is sent, we have many, many end users that completely ignore those notifications as well.
Over the last 15 years that I have worked at HostGator, I’ve seen VPS and dedicated servers where users have sent tens of millions of spam emails over the course of months – complete with literally tens thousands of bounce backs being sent back to the originating server – and users ignore those notifications entirely, and then they get mad at HostGator for allowing their emails to be blocked and/or to allow this spam to ever occur in the first place when we (HostGator) didn’t know it was occurring in the first place.
The issue is rarely with whether or not an end user received some kind of notification that their email was blocked, for any reason, the issue we frequently encounter is whether or not the end user knows what the notification is for and what to do (or not do) with that information.
“Are there any suggestions for what can be done to resolve this going forward please?”
Unfortunately, unless you decide to switch email applications away from K9, or utilize another mail provider, this issue will remain unresolved on our network, simply due to the choices the developers of K9 are making, in their fight for user privacy. While RFC5321 states that a literal address local can be substituted in place of the user’s own IP/FQDN during the EHLO process, spoofing that information to reflect 127.0.0.1 is a purposeful choice the developers are making, rather than finding other options for generating the IP/FQDN, such as creating a hash of random data as an FQDN which is based on both random data and the user’s existing hostname. Perhaps they have tried these things in the past, I fully admit that I do not know, but I do know that other mobile email clients do not have this issue
The generally used approach to dealing with stolen credentials is in the area of rate-limiting and the like. If an outbound delivered message is reported back as spam the MSP can then deal with the specific user. Most don’t do simplistic spam filtering, though I suspect do take action on messages that rate high on spam scoring.
Their approach of dumping these (and who knows what others):
1mAtIQ-002zkr-2s => /dev/null joe@dah.dah.com
is really not the way to treat a customer. If their problem is that customers can be confused by notifications they then need to work on that, not not notify the customer.
Find a MSP that actually knows what it’s doing, both technically and from a place of reasonable customer expectations.
Are there any suggestions for what can be done to resolve this going forward please?
So, what can we do about this. I just now found out that all the emails I’ve been sending via my hostgator account since the update never made it to their destinations, and that’s been over 50 emails. I don’t have time to screw around with this crap or do debug traces. Is there a solution coming soon? I love k9 mail and have been using it for ages, but if I can’t trust it to work, what choice do I have but to find another client.
Thanks,
-m
The choices have been spelled out already. Find a new email client or find a new email provider.
I just want to point out again that your provider accepted emails for delivery and then dropped them without telling you (or your email client) about it. If I were you, I would definitely switch email providers.
I think you need to find a new MSA. If Hostgator is dumping authenticated submissions - “1mAtIQ-002zkr-2s => /dev/null” - because they don’t like the IPnumber on the EHLO without bothering to tell the client (or returning it to the authenticated user), who knows what other excuses they may have for not delivering your mail. This isn’t really a mail client issue. I simply wouldn’t trust them.
Hi Mike,
Hostgator is not the only mail server effected.
There are two different things happening here at once.
K9 (version 5.6) was working fine in this, but the developers made some “improvements” in preparing their entirely new version of K9 (5.8) which have broken some things.
In this instance, one of their “improvements” is that in real-world terms, K9 can now strangely look more like a spammer’s utility to a mail server than is at all necessary in the circumstances.
The K9 developers have even been given advice by Hostgator on how they could better handle the issue.
I have had lengthy dialogues with Hostgator even in writing.
I also would like to see Hostgator and other mail services give better messaging in the circumstances, but understand that such a response method could also flood a User’s account where they had lost their authentication details and password to cyber crime—if Hostgator and other such mail services just too simply followed through on what’s been demanded of them here.
The effect on Users must be paramount here.
It is also I believe the prime duty of developers of apps like K9 5.8.
Hoping finally for a better new day in all this.
Paul
Switching providers for me is not an option. I’ve been using them for a decade and have too many hosted websites there. Unless I can find a separate outbound email provider that will work with this update I’m stuck looking for a new mail client, unfortunately.
Just think what would happen if Thunderbird had this problem. Luckily they don’t. If they did you know it would get fixed pretty damn fast.
It’s fine for a mailserver to reject e-mail it believes is spam, it’s not OK to accept it and then silently drop it.