Android app access local WiFi device with no Internet while phone mobile data is on

I’m working on an Android application that communicates with a Socket to a hardware device that serves a WiFi access point. This WiFi access point does not provide access to the Internet. I’m unable to connect to the socket while Mobile Data on the phone is turned on. How can I communicate with this device without requiring the user to turn off mobile data? I don’t need to communicate with the Internet over the mobile data connection at the same time as I am communicating with the device over WiFi, but I would like to be able to switch back and forth. I’ve tried binding the socket to the IP address associated with the network interface connected to the WiFi access point, but that didn’t work.