I'll use GDAL as a specific example. In order to install the gdal Python package, you must have the GDAL C library installed first. You must also install a version of Python gdal to match the version of your installed C library.
$ gdal-config --version
3.2.0 <-- this is the version needed for Python gdal
I'm wondering whether there is any way to configure Poetry to install a version of gdal the matches the installed C library.