How to efficiently use S3 remote with DVC among multiple developers with different aws configs?

Viewed 25

The DVC remote configuration allows to define a profile for the AWS CLI to use. However, some developers might have their local AWS cli configuration use different profiles whose name they find helpful.

Is there a way to override the profile used by DVC on a dvc push / dvc pull without modifying the remote configuration of an s3 repo?

1 Answers

There are a few options to achieve this.

Use local remote config option to set remote storage config parameters that are specific to a user:

$ dvc remote modify --local myremote profile myprofile

It will create a file .dvc/config.local that will be Git-ignored, and options from this files will be merged with the main config when users run DVC commands.

Alternatively, users can use the AWS_PROFILE environment variable to specify their local profile name. In this case remember to not include profile name into DVC remote config.

$ export AWS_PROFILE=myprofile 
$ dvc push
Related