I've seen this in many places, but I don't know how to do it.

I've seen this in many places, but I don't know how to do it.

That project appears to be using Shields.io, which you can also use for free. It works by grabbing datas from APIs and creating images from that API, which you can then render as Markdown code.
Use the build page and figure out which build system your project is using (if any). If you don't already have a build system, GitHub Actions is free for open source projects. Once you've decided on a build system, click it on the build page, enter the required URL fields, and use Copy Markdown from the dropdown button to get your Markdown code
If you have a package published on PyPi, you can use the following Markdown badge code, replacing PACKAGE with your package's name.

