I am using Spring Boot + Spring Web in an application for WebRTC Signalling. It is working very well and everyone testing is able to use it with their browser. But I discovered one case, where one person was not able to connect to my service via websockets. This person is using a TP-Link M7350, this is a Mini-Wifi Router that has a SIM Card (Vodafone) inside. I researched and debugged and tried several things in my application to catch the error. But I found out, it has nothing to do with my app or websockets, it seems to be an issue with TomCat / Spring Boot itself. I was able to reproduce the issue with this simple spring boot web application:
https://github.com/mxk1011/springboot-mini-demo
It is a very simple web application that outputs "I am alive" when accessing /test in one single Controller.
I cloned this app on a fresh server not related with any of my apps, opening port 443 with ufw and start it with mvn -U clean spring-boot:run. From all the different networks that I tested I was able to see "I am alive" when accessing /test on the server, I used different WiFis and Hotspots from mobile phones. But when I switched to the WiFI of this TP-Link router with the Vodafone SIM, I'm getting:
ERR_CONNECTION_TIMED_OUT in my Chrome and all other browsers.
I switched the SIM Card in the router to a Telekom SIM card and restarted it. I was able to access the page and see the "I am alive". Putting the Vodafone Card into my mobile phone, I was NOT able to open the page and also got a timeout. When I turned on any VPN on my computer while connected to the router with the Vodafone card, I was able to access it correctly.
So for some reason this single Vodafone SIM Card causes that I am not able to connect to this Mini Webserver with the Github Link above.
For better debugging I changed the LOG LEVEL to TRACE. Here is what is happening when you are able to connect to the page:
https://github.com/mxk1011/springboot-mini-demo/blob/master/works.log
And this happens when I connect with the Vodafone SIM card to the server:
https://github.com/mxk1011/springboot-mini-demo/blob/master/fails.log
In my production app I am using a certificate from Let's Encrypt, for this example I am using a self signed key in a store (see github, it is the one I used for testing). Using a Let's Encrypt Certificate on the demo server does not change anything I also tried this.
I have no clue why this could fail. The person who I got this router + SIM card from is using it every day for his work and has no problems at all with it. I was also able to push the files to GitHub and open this StackOverflow Thread with it. Basically this demo app + my own is are the only pages I discovered that does not work correctly with it. On Monday I will go to a retail store and buy another Vodafone SIM card to find out if this is a general issue of Vodafone or anything specific to this SIM - I have to say at this point that I am no expert with this matters. Therefore I am opening this thread here and hoping someone has any idea how I can sort this issue, because when this happens on one device/router it can happen on many out there and I need my service to be up for all customers. Thanks for everyone in advance, I hope this information provided can help.
EDIT I know it is likely, that something between the devices, all the way over Vodafone up to the app is blocking the request and there's nothing to blame Spring web/boot. But as you can see in the logs, there is an incomming request from my IP adress from the router. If you maybe don't have a solution for my problem, maybe you have an idea what could help to find out more about it, maybe by using any tool / script on the server / client? I am open for any idea. Thank you!
EDIT 2 To prove that nothing in the middle is blocking I set up a simple nginx server with https on the same machine and opened it at port 443. With the Router + Vodafone SIM I was able to connect to it via https. So for me it really seems like the point of failure is the Spring / Java part.
EDIT 3 I tried now to access the server via curl -vvv to maybe get more information with this command. I accessed the server with it, but got this response:
* Trying 1.2.3.x...
* TCP_NODELAY set
* Connected to xxx.com (1.2.3.x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
Fair enough, I need a certificate. I added a script to obtain a Let's encrypt certificate and convert it to pkcs12 to the git above. After generating it, I copied it to the classpath and started again. I did again curl -vvv myhost.com and I was able to access to without a problem:
* Trying x.x.x.x...
* TCP_NODELAY set
* Connected to myhost.com (x.x.x.x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=myhost.com
* start date: Jun 21 10:08:36 2020 GMT
* expire date: Sep 19 10:08:36 2020 GMT
* subjectAltName: host "myhost.com" matched cert's "myhost.com"
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
> GET /test HTTP/1.1
> Host: myhost.com
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200
< Vary: Origin
< Vary: Access-Control-Request-Method
< Vary: Access-Control-Request-Headers
< Content-Type: text/plain;charset=UTF-8
< Content-Length: 11
< Date: Sun, 21 Jun 2020 11:11:27 GMT
<
* Connection #0 to host myhost.com left intact
I am alive!* Closing connection 0
This works while opening the same URL in any browser FAILS with this SIM Card. With all other networks, no problem.