Can two IoT applications access a cellular modem at the same time so I don't have to kill ModemManager and lose my internet connection?

Viewed 676

I'm having trouble getting GPS data from my modem consistently. Normally I can get the GPS in Python with the following code:

import time
from serial import Serial

# If a "port" is given, then the port will be opened immediately.
ser = Serial(port="/dev/ttyUSB3", timeout=2, write_timeout=2)

# The following prints as "True"
print(ser.is_open)

def read_until_timeout(ser, timeout=5):
    """Turn GPS on and read until either we find the line we want, 
    or the timeout occurs
    """

    # Turn GPS on
    ser.write(b"AT+CGPS=1,1\r\n")

    # Request GPS
    ser.write(b"AT+CGPSINFO=1\r\n")

    # Start the loop
    time_start = time.time()
    while ((time.time() - time_start) < timeout):        
        print("Trying ser.readline().decode('ascii', errors='replace')...")
        try:
            # Read from the serial port
            line = ser.readline().decode('ascii', errors='replace')
        except serial.serialutil.SerialException as err:
            print(f"Problem reading data: {err}")

        if "+CGPSINFO:" in line:
            print(f"Found '+CGPSINFO:' in line: {line}")
            break

On a working gateway, that last print statement would return a line like the following, which contains the GPS latitude and longitude:

print(read_until_timeout(ser, timeout=5))
>>> '+CGPSINFO: 5100.506298,N,11404.432885,W,031120,234519.0,1080.1,0.0,204.6\r\n'

Sometimes the output from running this function is as follows, which I think just means the GPS can't get a "fix":

print(read_until_timeout(ser, timeout=5))
>>> Found '+CGPSINFO:' in line: +CGPSINFO: ,,,,,,,,

But sometimes it says it's "busy" as follows:

print(read_until_timeout(ser, timeout=5))
>>> Problem reading data: read failed: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

Is the above because more than two applications can't access the serial port at the same time?

If I try sending AT messages to the modem from Linux Bash, it says the modem is "busy"...

$ echo 'AT+CGPSINFO=1' | socat - /dev/ttyUSB3,crnl
>>> 2020/11/20 19:18:24 socat[3251] E open("/dev/ttyUSB3", 02002, 0666): Device or resource busy

So I check which devices are using the modem:

$ sudo lsof /dev/ttyUSB3
>>> COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
ModemMana  800 root   12u   CHR  188,3      0t0  449 /dev/ttyUSB3
python    1706 root    3u   CHR  188,3      0t0  449 /dev/ttyUSB3

But if I shut down ModemManager with sudo kill 800 the cellular internet connection dies, and I can't have that because these are remote IoT devices far in the wilderness (I have about 50 of them in operation around North America).

Is there a way to share the modem with ModemManager so I can access the GPS results from Python or from the shell, while on a cellular internet connection?

This question is related so I put a bounty on it too: https://superuser.com/questions/1455327/trying-to-read-a-serial-port-device-or-resource-busy

Update Nov 22, 2020:

@GitFront helpfully suggested using ModemManager directly. Great suggestion. Neat trick with the gi library too.

From Ubuntu Bash, here's what I get when I try to get the GPS from ModemManager directly:

user@user-Default-string:~$ mmcli -m 0 --location-status

/org/freedesktop/ModemManager1/Modem/0
  ----------------------------
  Location |   capabilities: '3gpp-lac-ci'
           |        enabled: '3gpp-lac-ci'
           |        signals: 'no'

Notice under location capabilities, no gps-raw, gps-nmea, agps shows up for some reason, only 3gpp-lac-ci, but I know this gateway/SIMCom 7600 modem has GPS/GNSS capabilities, and it's got a passive antenna hooked up properly.

Does anyone know why gps-raw, gps-nmea, agps are not showing up as capabilities?

The exact same gateway/modem/antenna combination in my office does have gps-raw, gps-nmea, agps capabilities, and everything works perfectly for it... The only difference is the SIM card, which should not matter. The gateway in my office reports the following capabilities, by contrast:

user@ax1000008:~$ mmcli -m 0 --location-status
  ------------------------
  Location | capabilities: 3gpp-lac-ci, gps-raw, gps-nmea, agps
           |      enabled: 3gpp-lac-ci
           |      signals: no
  ------------------------
  GPS      | refresh rate: 30 seconds

I request to enable gps-raw, gps-nmea with:

user@ax1000008:~$ sudo mmcli -m 0 --location-enable-gps-raw --location-enable-gps-nmea
successfully setup location gathering

Then I request the GPS successfully (on the gateway/modem in my office...) with:

user@ax1000008:~$ sudo mmcli -m 0 --location-get
  --------------------------
  3GPP |      operator code: 302
       |      operator name: 220
       | location area code: 2BC4
       | tracking area code: 0000
       |            cell id: 04B68931
  --------------------------
  GPS  |               nmea:
       |                     $GPGGA,183553.00,5100.511216,N,11404.429721,W,1,06,1.7,1078.0,M,-16.0,M,,*6C
       |                     $GPRMC,183552.00,A,5100.511138,N,11404.429729,W,0.0,341.3,221120,15.4,E,A*10
       |                     $GPGSA,A,2,01,03,21,22,31,32,,,,,,,2.0,1.7,0.9*3C
       |                     $GPGSV,4,1,15,01,45,295,34,03,11,291,29,10,31,109,20,11,,,36*4F
       |                     $GPGSV,4,2,15,12,08,056,19,21,53,258,35,22,33,292,32,23,02,111,15*79
       |                     $GPGSV,4,3,15,25,06,092,22,31,33,163,23,32,70,066,33,08,02,236,*7C
       |                     $GPGSV,4,4,15,17,04,344,,20,,,,24,,,*48
       |                     $GPVTG,341.3,T,325.9,M,0.0,N,0.0,K,A*2B
       |                     $PQXFI,183553.0,5100.511216,N,11404.429721,W,1078.0,9.61,5.64,0.13*4E
       |                utc: 183553.00
       |          longitude: -114.073829
       |           latitude: 51.008520
       |           altitude: 1078.000000

On the non-working gateway in the field, here's what I get when I try to enable gps-raw and gps-nmea:

user@user-Default-string:~$ sudo mmcli -m 0 --location-enable-gps-raw --location-enable-gps-nmea
error: couldn't setup location gathering: 'GDBus.Error:org.freedesktop.ModemManager1.Error.Core.Unsupported: Cannot enable unsupported location sources: 'gps-raw, gps-nmea''
2 Answers

You should be able to utilize ModemManager to get the GPS location instead of getting it directly. This way you can avoid using the same serial port from two services.

There is documentation on how to do this from the command line using the mmcli tool in the ModemManager manpage: https://www.freedesktop.org/software/ModemManager/man/1.0.0/mmcli.8.html#lbBC

There is also a libmm-glib library which you should be able to use from Python via GObject Introspection without running the command line tool.

import gi
gi.require_version('ModemManager', '1.0')
from gi.repository import ModemManager

Keep in mind that your device may not be able to provide the location while a data session is active (from the documentation):

The Location interface allows devices to provide location information to client applications. Not all devices can provide this information, or even if they do, they may not be able to provide it while a data session is active.

I think I've mostly figured it out now, without having to kill ModemManager (which risks killing the internet connection on the IoT remote gateways).

The main answer is that I'm simply not getting a GPS fix (permissions issues aside).

I wasn't doing anything wrong in my Python code, and sometimes ModemManager was using the AT port I was sending commands to (either /dev/ttyUSB2 or /dev/ttyUSB3)... Upgrading ModemManager to v1.10.0 also might work, but I'm reluctant to do that on a live gateway working in the field.

@rm5248 suggested using the dedicated NMEA port at /dev/ttyUSB1 on a SIMCom 7600 like mine, and he was right, I think.

I installed GPSD (GPSDaemon) and the various clients for it (e.g. cgps and gpsmon) using this documentation. GPSD and its clients are excellent for getting GPS info, if the GPS has a fix.

How to test the GPSD software: https://gpsd.gitlab.io/gpsd/installation.html#_how_to_test_the_software

# Start GPSD on /dev/ttyUSB1 with debug level 5, 
# -N (don't daemonize), and -n (don't wait for 
# client to connect to poll GPS)

# no daemon, for debugging
$ sudo gpsd -D 5 -N -n /dev/ttyUSB1

# daemonized to run in background
$ sudo gpsd -D 5 -n /dev/ttyUSB1

Then run one of the following two clients and watch the dedicated NMEA port on /dev/ttyUSB1, as opposed to an "AT" port at /dev/ttyUSB2 or /dev/ttyUSB3:

$ cgps
$ gpsmon /dev/ttyUSB1

On a working gateway cgps works like this: working gps on gateway with cgps

And gpsmon /dev/ttyUSB1 works like this: working gps on gateway with gpsmon

But on a non-working gateway cgps looks like this, showing NO FIX: non-working gateway, no fix

Related