I am working in AWS Sagemaker Jupyter notebook. I have installed clearml package in AWS Sagemaker in Jupyter. ClearML server was installed on AWS EC2. I need to store artifacts and models in AWS S3 bucket, so I want to specify credentials to S3 in clearml.conf file. How can I change clearml.conf file in AWS Sagemaker instance? looks like permission denied to all folders on it. Or maybe somebody can suggest a better approach.
1 Answers
Disclaimer I'm part of the ClearML (formerly Trains) team.
To set credentials (and clearml-server hosts) you can use Task.set_credentials.
To specify the S3 bucket as output for all artifacts (and debug images for that matter) you can just set it as the files_server.
For example:
from clearml import Task
Task.set_credentials(api_host='http://clearml-server:8008', web_host='http://clearml-server:8080', files_host='s3://my_bucket/folder/',
key='add_clearml_key_here', secret='add_clearml_key_secret_here')
To pass your S3 credentials, just add a cell at the top of your jupyter notebook, and set the standard AWS S3 environment variables:
import os
os.environ['AWS_ACCESS_KEY_ID'] = 's3_bucket_key_here'
os.environ['AWS_SECRET_ACCESS_KEY'] = 's3_bucket_secret_here'
# optional
os.environ['AWS_DEFAULT_REGION'] = 's3_bucket_region'
Related