My email is on my private servers and I have an OpenVPN tunnel for k9mail on our Android phones.
When using OpenVPN I am able to send from k9mail whether connected through cellular network or wifi. However when on cellular network I am usually unable to fetch mail - I can only reliably fetch mail when on wifi. This is all the more annoying because occasionally it does fetch when on cellular, nost not reliably.
OpenVPN is making the tunnel whether on cellular network or wifi. My outgoing server is Postfix and my imap server is Dovecot, both those and my OpenVPN endpoint are running on Gentoo Linux.
I thought I had some setting wrong on the phone, but I validated that k9mail had full access to data even when Data Saver was active.
I had a problem with the same symptoms while in Yellowknife. It turned out I was at the edge of the cell towers range. when I was in the back of the house it would not work properly. I had to be in the front of the house. It also had to be decent weather.
This also caused delays with sending/receiving texts.
I wound up just keeping my phone on the wireless LAN for email… had to just deal with the texts.
No error, the little indicator keeps spinning and spinning, and at some point I have to get back to the real world.
To put it more simply, when I’m at home and the phone is on my LAN and desk, an email will show up on Thunderbird and within a minute the phone will give a notification sound. (Is this “push”? (or poll)) Same thing when I’m on wifi. And once in a blue moon on cell. I normally don’t have to tell it to fetch, it usually just shows up.
If there is a logfile and you can point me to it I can take a look.
I’ve had it fail to work standing at the base of a cell tower.
I’ve had it work (one of three times in recent history) with 1.5 bars in my living room.
OpenVPN says it has established a connection, and I have the little key up in the status area. Outgoing mail works just fine also, as sent by k9mail to the postfix server in my basement. I’m in the cellular backwater of Vermont, so I don’t always have enough signal at home to test this and have to go somewhere. My normal test consists of two emails, one from my gmail address to my home servers and the other from k9mail to my gmail address. From what I’ve seen, this latter path works reliably.
This is the first I’ve heard of TURN, I’ll start looking it up, and if you have a reference I’d be interested in seeing it.
I’ve been gone a while, a lot of life has happened in the meantime. I have a logfile, but have to figure out a way to link it here. I was delaying with that because I wanted a matching logfile from my server, but I’ve not had luck with that.
As for further diagnostics I also installed “Blue Mail” and it has the exact same symptoms. In both cases outgoing mail is able to make it through OpenVPN to my Postfix server and go, but for incoming it appears to be a socket problem. Actually I can probably include one line here…
02-24 08:49:47.768 15041 15091 E RealImapConnection: Failed to login, closing connection for conn70170219
02-24 08:49:47.770 15041 15091 E RealImapFolder: IOException for 848a4b19-8de9-400e-a92e-f4cf22c2a5db:INBOX/MessagingController/conn70170219
02-24 08:49:47.770 15041 15091 E RealImapFolder: java.net.SocketTimeoutException: Read timed out
02-24 08:49:47.770 15041 15091 E RealImapFolder: at java.net.SocketInputStream.socketRead0(Native Method)
OK, four lines, but I think it gets the idea across. Because Blue Mail does the same thing I think this is an Android issue, but that doesn’t mean that I (really mostly my wife) don’t want it fixed.
One other data point. I mentioned having it work at home with wifi turned off. I don’t think that’s real, because this is Google Fi, and it has “opportunistic wifi”, so it probably turned wifi on behind my back and that’s why it worked, not an unusual connection over cell.
(This has been on hiatus due to real life, back on it.)
I don’t think I need TURN. When I connect, outgoing mail through my Postfix server in the basement works OK on cellular or wifi. It’s just incoming mail that works only on wifi, but not on cellular. If it were a problem that required TURN I would expect neither to work.
I’m using an OpenVPN endpoint on my server in the basement. Is there any reason to believe Wireguard would do better?
I enabled logging, results are on a recent reply on this thread. I wanted to re-do this experiment and get logs from my server, but I haven’t been able to do that yet.
Are you sure all ports are mapped through your OpenVPN connection? I interpret your log excerpts such as the socket connection times out. So the problem occurs on the socket level which can be either a miss-identification or lack of unbound socket.
If you use Wireguard, at least the default configuration is to map all ports through the connection.
Let me go through my scenario again…
At home I’m running Postfix for SMTP and Dovecot for IMAP - in addition to OpenVPN.
My remote experiment is twofold:
With gmail I attempt to send an email to my address that comes to my home server.
With k9mail I attempt to send an email to my gmail address.
When I’m connected through wifi and OpenVPN active everything works as wished.
When I’m connected through cellular network and OpenVPN, the email makes it from k9mail to gmail, but the email from gmail does not make it to k9mail.
Configuration-wise nothing was changed between wifi and cellular network experiments.
So for some reason on cellular network the IMAP doesn’t make it, but SMTP does.
I’ve installed BlueMail and it behaves exactly the same as k9mail.
I had a fly in the ointment the other night. Just for jollies I turned on OpenVPN and a bunch of email promptly came in on k9mail. I wondered what was going on, then I thought about opportunistic wifi and wondered if that’s why this seems to occasionally work. There is a setting to turn it off, and I need to experiment with that in a place where it seems to work.
Thought occurred to me. How about if I take my Linux laptop and install android-tools on it. Then go to a remote location where I have cell service, hook them together with USB, and get into adbshell from the laptop.
I’m thinking in terms of using “netstat” or “ip” commands, or better yet “tcptraceroute” which can send a TCP packet to a specific port and find if/where it fails. I’m thinking that this problem is some behavior of Android, not k9mail, because Blue Mail acts the same way. (Unless both have adopted a common behavior.) The troubling thing is that SMTP goes through while IMAP doesn’t. That suggests to me that the problem is outside of the OpenVPN tunnel.
Figured it out. It was an MTU problem. Apparently Google Fi has a shorter MTU than the conventional 1500. With OpenVPN using “tun-mtu 1408” and “mssfix 1368” and things are working well. I found the number by pinging with different payload sizes.