Azure Machine Learning Designer Error: JobConfigurationMaxSizeExceeded

user_1486998 0
Viewed 358

I have an Azure Machine Learning Designer pipeline that I've run successfully many dozens of times. Suddenly, today, The pipeline is getting down to the 'Train Model' node and failing with the following error:

JobConfigurationMaxSizeExceeded: The specified job configuration exceeds the max allowed size of 32768 characters. Please reduce the size of the job's command line arguments and environment settings

How do I address this error in designer-built pipelines?

I have even gone back to previously successful runs of this pipeline and resubmitted one of these runs which also failed with the exact same error. A resubmitted run should have the exact same pipeline architecture and input data (afaik), so it seems like a problem outside my control.

Pipeline with error: enter image description here

Pipeline run overview: enter image description here Any ideas?

EDIT: I'm able to repro this with a really simple pipeline. Simply trying to exclude columns in a Select Columns node from a dataset gives me this error:

enter image description here

azure-machine-learning-studio
2 Answers

This appears to be a bug introduced by Microsoft's rollout of their new Compute Common Runtime.

If I go into any nodes failing with the JobConfigurationMaxSizeExceeded exception and manually set AZUREML_COMPUTE_USE_COMMON_RUNTIME:false in their Environment JSON field, then they work correctly.

user_14869980

Runtine variables should be set in RunConfiguration and not in Environment JSON.

user_121265620
Related