Can I specify S3 bucket for sagemaker.sklearn.estimator's SKLearn?

user_1207596 0
Viewed 575

I'm following this example notebook to learn SageMaker's processing jobs API: https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker_processing/scikit_learn_data_processing_and_model_evaluation/scikit_learn_data_processing_and_model_evaluation.ipynb

I'm trying to modify their code to avoid using the default S3 bucket, namely: s3://sagemaker-<region>-<account_id>/

For their data processing step with the .run method:

from sagemaker.processing import ProcessingInput, ProcessingOutput

sklearn_processor.run(
    code="preprocessing.py",
    inputs=[ProcessingInput(source=input_data, destination="/opt/ml/processing/input")],
    outputs=[
        ProcessingOutput(output_name="train_data", source="/opt/ml/processing/train"),
        ProcessingOutput(output_name="test_data", source="/opt/ml/processing/test"),
    ],
    arguments=["--train-test-split-ratio", "0.2"],
)

I was able to modify it to use my own S3 bucket by using the destination parameter like this:

sklearn_processor.run( 
    code=output_bucket_uri + "preprocessing.py", 
    inputs=[ProcessingInput( 
        source=input_bucket_uri + "census-income.csv", 
        destination=path+"input/", 
    )], 
    outputs=[ 
        ProcessingOutput( 
            output_name="train_data", 
            source=path+"train/", 
            destination=output_bucket_uri + "train/", 
        ), 
        ProcessingOutput( 
            output_name="test_data", 
            source=path+"test/", 
            destination=output_bucket_uri + "test/", 
        ), 
    ], 
    arguments=["--train-test-split-ratio", "0.2"], 
)

But for the .fit method:

sklearn.fit({"train": preprocessed_training_data})

I have not been able to find a parameter to pass it so that the output artifacts are saved to a S3 bucket that I specify instead of the default s3 bucket s3://sagemaker-<region>-<account_id>/.

amazon-sagemaker scikit-learn machine-learning amazon-web-services amazon-s3
2 Answers

In case it helps others...

To get .fit() to output to a designated S3 bucket, I ended configuring the estimator with output_path.

Here's the example code:

from sagemaker.sklearn.estimator import SKLearn 

sklearn = SKLearn( 
    entry_point="../processor/code/train.py", 
    output_path=output_bucket_uri, 
    framework_version="0.20.0", 
    instance_type="ml.m5.xlarge", 
    role=role, 
)
sklearn.fit({"train": preprocessed_training_data})

Here's the docs relating to the base estimator class: https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html

user_12075960

For SKLearnProcessor, the ideal way to specify default bucket is by creating a sagemaker session with that bucket, and sending that as sagemaker_session parameter. Example:

from sagemaker.session import Session    
sklearn_processor = SKLearnProcessor(framework_version='0.20.0',
                                     role='<arn-role>',
                                     instance_type='ml.m5.xlarge',
                                     instance_count=1,
                                     sagemaker_session=Session(default_bucket='<s3-bucket-name>'))

I know this is not your exact question but you have added an alternative to this in your question details. So I am adding it here as a cleaner approach.

user_33392010
Related