I have an odd problem that just started. The same issue affects both my ai ultra and vacuglide 2. I have been using mobile hotspot on my phone for these devices for a long time, but now neither of them can go online using this hotspot when my phone is connected to mobile data. However, when the phone is connected to wifi instead of mobile data, both devices are able to go online normally via the phone’s wifi hotspot. I have gone through device setup and they can connect to the hotspot no problem and other devices can use the hotspot and access the internet no problem when using mobile data. Possibly the provider is blocking a connection or maybe DNS lookup is failing.
Is there a way to extract the logs from either the ai ultra or vacuglide in order to determine what is failing when the devices attempt to go online using this wifi hotspot when the phone is connected to mobile data?
Since both the AI Ultra and VacuGlide 2 work through the same hotspot when your phone itself is connected to Wi-Fi, but stop working when the phone switches to mobile data, I don’t think the problem is with the devices connecting to your hotspot.
It sounds more like something changes in the network path when mobile data is being used.
One possibility I’d look at is IPv4/IPv6. Mobile providers often use IPv6, IPv4/IPv6 or carrier-side translation (CGNAT/NAT64). Phones and computers normally handle this without any problem, but embedded devices can sometimes have trouble with it.
If your phone allows it, I would try changing the mobile APN protocol temporarily from IPv4/IPv6 to IPv4 only and see if the AI Ultra/VacuGlide suddenly connect. I’d also try:
Temporarily disabling Private DNS/VPN/ad blocking on the phone.
Switching from 5G to 4G/LTE and testing again.
Trying the hotspot with a SIM from another mobile provider, if possible.
Trying your current SIM in another phone and using that phone as the hotspot.
The last two tests would be especially useful because they could tell you whether the problem follows the mobile provider/SIM or the phone.
Since both devices started doing exactly the same thing at the same time, I’d be more suspicious of the mobile carrier/APN/network configuration than of two separate device failures.
If you have a Windows laptop, there’s another thing you could try: connect the laptop to the same phone hotspot while the phone is using mobile data and run a few DNS/IPv4/IPv6 tests against Autoblow’s servers. That could help narrow down exactly where the connection is failing.
Thank you for the reply. However, it does not address the question of how to get logs off of the device. That would be the most helpful diagnostic tool as it would show what connection or lookup is actually failing. Next most helpful would be a list of what the AB device actually does when attempting to go online. i.e. DNS lookup for api.autoblow.com, tcp connection via https 443 to the resultant DNS resolution. These are just my guesses on what its doing. If I had these online connectivity steps, then i could connect a windows or linux device to the hotspot and test these.
The reply appears to be AI generated, based on tone and content. That said, I have completed some of the suggestions provided without impacting the issue.
VPN is off, no ad blocking and DNS privatization disabled, same result
4G same result as 5G
Switching to IPV4 only for hotspot, same result
I’m unable to test a different SIM or a different mobile network as I don’t have those resources.
Thanks again for the reply. Can anyone help with how to obtain logs for the device or what steps the AB device takes after connecting to WIFI when attempting to go online?
A bit more info on the issue: when attempting to go online with the AB device, i see the device connect to the hotspot successfully, there is 0.01MB of data transfer, then the AB device disconnects from the hotspot. This reinforces my guess that it failing a name lookup or connection.
There is no way to get any connection logs. I can help you with some online logs and see if the devices reach our server in any way. You can PM me your token info and i can look into it.
Thanks Alex. Will PM device token shortly and the timing a connection attempt was made from the same hotspot both via 5g (non-working) and local wifi (working). Do you have a sequence diagram or data flow diagram or other document of what the AB attempts to do when it tries to go online? I see it connect to the hotspot successfully and a bit of data is transferred, but then the AB device disconnects then from wifi. There is no red error at the device, just flashing white lights during the connection attempt, resulting in steady white lights when the connection attempt fails.
I will PM you the token. Here are the tests and results.
Fail - 4:28PM UTC - AB ultra connected to phone hotspot with phone connected to mobile data on T-mobile
Pass - 4:29PM UTC - AB ultra connected to phone hotspot with phone connected to t-mobile fiber wifi (a separate service than mobile)
Other tests. With phone hotspot connected to mobile data, I connected a windows machine to the hotspot. Through the hotspot I was able to resolve fun.autoblow.com and connect on port 443.
Token sent to AB yesterday and looking forward to the results, but I’m not too hopeful AB can see anything at their end. Since the device can’t go online when the hotspot uses the t-mobile cellular network, its likely they won’t the see token at all for the failed online attempts. This problem has the semblance of a name resolution, connection or possibly certificate validation error. Without logs and without guidance from AB on what names, ports and protocols the device tries to reach out to when it goes online, then I’m a bit stuck. I’m also wondering if this issue is somehow unique to my hotspot (which worked for last 2 years) or phone, or if other customers on t-mobile cellular will have the same issue.
I’ve ordered a travel router to try to attempt to work around the issue. The idea is to connect the travel router to the phone hotspot (or usb tether) in router mode, then connect the AB to the travel router’s wifi. All traffic from the AB device will come from the travel router and packets get repackaged. If it still doesn’t work, then via the router with open source firmware and root access, this may also allow some packet capture to identify failed lookups or connections that can’t be done on the phone without root.
I’m able to see the successful attempt in our logs as expected however I get no connections for the unsuccessful attempt so it does not reach our servers at all.
If possible test your connection to https://latency.autoblowapi.com/ This is the URL your device will attempt to connect to our servers. Also, please note that in the setup screen we show a successful setup when a wifi connection is established however the setup screen does not attempt to reach our servers. The device only connects to our servers when it’s in interactive mode and your router/firewall can block this connection.
I plan to add some connection debugging logs in setup mode to allow for proper diagnostics when we have situations like these.
Thank you @Alex_Autoblow_Dev . Knowing what hostname its attempting to resolve, connect on tcp 443 and setup a security association with that host. I can test those processes synthetically through the hotspot using a windows or linux device. Does it reach out to any other internet resources in order to “go-online” or just the host you listed?
I also have a travel router arriving today that i can try. That will obscure the AB device from the mobile hotspot and if needed allow for custom dns or vpn tunnel if t-mobile is blocking either the lookup or connection on their network.
Connection debugging logs would be extremely helpful and welcome in situations like these.
This reply is going to be long, sorry, there is a lot of information to share. Autoblow might not be very interested in this issue if I am the only one reporting it. My testing doesn’t show a reason for only me to be affected, so this may turn out to be a wider issue affecting more customers. Both AB ultra and vacuglide 2 affected and weren’t affected just weeks ago without known changes. I have developed a successful workaround for the issue. How much time i put into the diags and workaround is a testament to how much i enjoy the products.
Summary:
REASON FOR ONLINE FAILURE: AB device cannot go online because it cannot form a tcp connection on port 443 to the hosts behind latency.autoblowapi.com when using the phone mobile hotspot where the internet connection is t-mobile cellular 5g network. It doesn’t matter which host is selected by DNS load balancing.
name resolution works
TCP syn packet goes out and no syn-ack is returned. AB devices resend the syn packet several times and eventually gives up trying to connect.
syn packet must be dropped or filtered upstream of the AB device
WORKAROUND: Using a travel router with its internet connection as the phone hotpot with the phone using t-mobile cellular network for internet AND the AB device connecting wifi to the travel router client network AND the router tunneling all traffic from the AB device through a VPN is an effective workaround. Under these conditions the AB can connect and go online even when the phone hotspot is using tmobile cellular network. VPN tunnel is the key to making this work.
CAUSE: There is something unique about the packets generated by the AB device that result in the connection failure suggesting this could be corrected via AB firmware.
Making the same connection to latency.autoblowapi.com through the same path client->router->internet via mobile data (without VPN) with a windows device works correctly while this fails for the AB device.
Since VPN tunnel works, this drops the traffic off a different place on the internet to then reach AB servers. This suggests a t-mobile network or t-mobile to AWS peering issue or intentional filter.
What i would do next is investigate any differences in the SYN packets between devices (AB vs windows in this test) and sythetically attempt each difference until identifying what causes the flow to break.
Evidence:
The reason AB device doesn’t go online
AB connected to travel router that is connected mobile hotspot using mobile data for internet. Traffic captured on the router at the lan interface. No tunneling or firewall blocks at the router. The same result is seen when capturing at the external radio of the router.
Device gets an IP address, queries successfully for latency.autoblowapi.com and attempts to connect to the resultant IP. No reply is ever received for the first phase of the handshake. Possibilities: The syn packet is being dropped at the phone, the carrier network, peering between t-mobile and AWS, or AWS.
AB connected to travel router in turn connected mobile hotspot using mobile data. Traffic captured on the router at the lan interface. Traffic tunneled through VPN to surfshark via openvpn tcp port 1443. No point capturing traffic at the radio as the traffic is encrypted there.
AB device gets an IP, successful lookup for latency.autoblow.com, successful connection on TCP 443, TLS negotiation, etc. Device goes online normally
Forced name resolution (via router DNS server) to return the same IP as the first failed test, but this time through VPN tunnel. Works fine. This rules out individual AB servers hosting latency.autoblowapi.com
Test for non-AB client to make the failed connection to AB server
Client pc connected to same wifi network as the AB (the travel router), making a synthetic https connection to latency.autoblowapi.com. Router not tunneling traffic via VPN tunnel. Works fine, showing that connections to that host via that protocol are allowed on the t-mobile cellular network and flow through the same path used by the AB device.
