Can't Install AWS Client Through Pip Because of of (Seemingly) Circular Dependencies

user_5921 0
Viewed 758

I'm trying to install the AWS CLI tools on Linux (Mint), and running into a problem. When I run:

pip3 install aws-sam-cli

I get:

ERROR: awscli 1.18.216 has requirement botocore==1.19.56, but you'll have botocore 1.17.63 which is incompatible.

Ok, should be simple enough to fix; I'll just run:

pip3 install botocore==1.19.56

... except then I get:

ERROR: boto3 1.14.63 has requirement botocore<1.18.0,>=1.17.63, but you'll have botocore 1.19.56 which is incompatible.

Ok, great, let's install the right botocore:

pip3 install botocore==1.18.0

... and now I've broken boto3 (not to mention the awscli package):

ERROR: boto3 1.14.63 has requirement botocore<1.18.0,>=1.17.63, but you'll have botocore 1.18.0 which is incompatible.

ERROR: awscli 1.18.216 has requirement botocore==1.19.56, but you'll have botocore 1.18.0 which is incompatible.

If I try a slightly lower (but still acceptable?) botocore of 1.17.63 it fixesboto3 ... but then AWS is still broken:

pip3 install botocore==1.17.63

ERROR: awscli 1.18.216 has requirement botocore==1.19.56, but you'll have botocore 1.17.63 which is incompatible.

And of course if I try to fix botocore, we're back to breaking boto3:

pip3 install botocore==1.19.56

ERROR: boto3 1.14.63 has requirement botocore<1.18.0,>=1.17.63, but you'll have botocore 1.19.56 which is incompatible.

It would seem Amazon has a set of dependencies which are impossible to satisfy!

But I know that can't be the case, so can anyone explain what am I not understanding? I haven't had to work with pip in years, so please go easy on me.

pip aws-cli
1 Answers

The reason why you're running into these circular dependencies is because the aws-sam-cli package that you're trying to install is the latest version, while the boto3 package already installed on your system is relatively quite old.

Now coming onto why the dependencies of these packages are hard bounded, it's for ensuring that the specific version of the package does not break if there's an update on the dependencies. Hence the AWS champions have well tested and verified that boto3 1.14.63 has requirement botocore<1.18.0,>=1.17.63 , awscli 1.18.216 has requirement botocore==1.19.56 .

Though if you take a closer look on the current packages that you have installed already and the ones that you're trying to install:

  1. boto3 1.14.63 - Released on Sep 17, 2020 has dependency on botocore >= 1.17.63 - Released on Sep 17, 2020 and botocore < 1.18.0 - Released on Sep 17, 2020
  2. aws-sam-cli latest(Since you haven't specified to a version) - Released on Dec 18, 2020 has dependency on awscli 1.18.216 - Released on Jan 16, 2021 which in turn has a dependency on botocore == 1.19.56 - Released on Jan 16, 2021

As you can see, when you're trying to install a latest version of aws-sam-cli, it needs a newer version of botocore (1.19.56), while your boto3 installation is directed to not go beyond version 1.18.0 of botocore.

Hence, to ensure that the ecosystem of these packages works well, we need to have boto3 and aws-sam-cli installed to a version that both of them work with the given version of botocore, which can be done in two ways:

  1. We update the packages that are old to a newer versions, i.e. we update boto3 to version 1.16.62 - Released on Jan 29, 2021 (Latest at the point of writing) . This would need you to ensure that your existing code works well with the updated version of boto3 and there are no regressions.
pip3 install boto3==1.16.62 aws-sam-cli==1.15.0

or

  1. You direct pip to install an older version of aws-sam-cli, for e.g. 1.3.0 - Released on Sep 23, 2020 .
pip3 install aws-sam-cli==1.3.0

Note: Another piece of advice that I would give is to have your requirements bounded to a specific version so as to ensure that you're not hit by surprises in production, since there could be package updates which could break your dependencies again or make the code not work as expected. Though, this applies only when you're on a CI/CD model and pushing newer versions of your product out.

user_150966800
Related