Lingering package dependencies in xcode that were removed

When started a project in xcode, I added a Firebase package dependency through Swift Package Manager. I later removed it from the project package dependencies. However, since then, under the source control navigator, those package dependencies still show up under repositories. Is there a way to remove those?