Is it possible to configure AWS CodeArtifact PyPi repository to be used in AWS MWAA?

user_5002023 0
Viewed 1271

AWS MWAA (Managed Workflows for Apache Airflow) is relatively new service provided by AWS. When configuring the MWAA environment, it is possible to provide custom requirements.txt file, which is used to install additional Python packages in that environment.

In the company I work for, we use AWS CodeArtifact for custom PyPi package repository, where we upload private Python packages. We want to use some of them in Airflow DAGs. That's why I was wondering if MWAA environment can be configured somehow to use the PyPi repository from CodeArtifact?

Or is there any way how to install custom Python packages (not in public PyPi) in MWAA environment?

mwaa aws-codeartifact amazon-web-services python
1 Answers

Didn't try but should work:

# aws codeartifact login --tool pip --domain **--repository **
# awk '/index-url/ {print "-i "$3}' ~/.config/pip/pip.conf  > requirements.txt
# echo <my python package> >> requirements.txt

It can be used as normal requirements file:

# pip3 install -r requirements.txt 
Looking in indexes: https://aws:****

Please note CodeArtifact token expires maximum in 12 hours. You can create a recurring job to regenerate this file...

user_155650080
Related