It depends on the API you are using to access the GPU. Under vulkan, the capability to query the available devices is provided in the API itself. e.g.
vkEnumeratePhysicalDevices //< use to iterate over devices
vkGetPhysicalDeviceProperties //< use to determine make/model
vkGetPhysicalDeviceFeatures //< use to determine the capabilities
Under OpenGL, this is a little harder, because the API doesn't really provide a nice robust way of iterating over connected GPU's (this is very much left to the underlying OS). Aside from calling glGetString with GL_VENDOR, and/or GL_RENDERER, after the context has been created, that's about all you have got.
With graphics API's in general, you will typically choose the device to use up front, and then forget about it. (i.e. you know which GPU is in use, because it's the one you requested)
So for example, on windows you'd use the WIn32 API call EnumDisplayDevices to iterate over a bunch of DISPLAY_DEVICE structures, which give you a breakdown of the GPU devices. If you then want to query the device capabilities, you'd need to initialise an OpenGL context (with the correct device), and make multiple calls to glGetIntegerv to query the actual capabilities.
Sadly, on linux this is a little trickier. Iterating the display devices is pretty much going to depend on the underlying API you are using to initialise the GPU (e.g. GLX, SDL, Qt, etc).
So for example, in SDL you would iterate over the available devices using SDL_GetRenderDriverInfo, before passing your preferred device index to SDL_CreateRenderer at app startup.