Azure Stream Analytics: ML Service function call in cloud job results in no output events

I've got a problem with an Azure Stream Analytics (ASA) job that should call an Azure ML Service function to score the provided input data. The query was developed und tested in Visual Studio (VS) 2019 with the "Azure Data Lake and Stream Analytics Tools" Extension. As input the job uses an Azure IoT-Hub and as output the VS local output for testing purposes (and later even with Blobstorage). Within this environment everything works fine, the call to the ML Service function is successfull and it returns the desired response. Using the same query, user-defined functions and aggregates like in VS in the cloud job, no output events are generated (with neither Blobstorage nor Power BI as output). In the ML Webservice it can be seen, that ASA successfully calls the function, but somehow does not return any response data. Deleting the ML function call from the query results in a successfull run of the job with output events.

For the deployment of the ML Webservice I tried the following (working for VS, no output in cloud):

• ACI (1 CPU, 1 GB RAM)

• AKS dev/test (Standard_B2s VM)

• AKS production (Standard_D3_v2 VM)

The inference script function schema:

• input: array

• output: record

Inference script input schema looks like:

@input_schema('data', NumpyParameterType(input_sample, enforce_shape=False))
@output_schema(NumpyParameterType(output_sample))  # other parameter type for record caused error in ASA
def run(data):
    response = {'score1': 0,
                'score2': 0,
                'score3': 0,
                'score4': 0,
                'score5': 0,
                'highest_score': None}

And the return value:

return [response]

The ASA job subquery with ML function call:

with raw_scores as (
select
    time, udf.HMMscore(udf.numpyfySeq(Sequence)) as score
from Sequence
)

and the UDF "numpyfySeq" like:

// creates a N x 18 size array
function numpyfySeq(Sequence) {
    'use strict';
    var transpose = m => m[0].map((x, i) => m.map(x => x[i]));
    var array = [];
    for (var feature in Sequence) {
        if (feature != "time") {
            array.push(Sequence[feature])
        }
    }
    return transpose(array);
}

"Sequence" is a subquery that aggregates the data into sequences (arrays) with an user-defined aggregate.

In VS the data comes from the IoT-Hub (cloud input selected). The "function signature" is recognized correctly in the portal as seen in the image: Function signature

I hope the provided information is sufficient and you can help me.

Edit:

The authentication for the Azure ML webservice is key-based. In ASA, when selecting to use an "Azure ML Service" function, it will automatically detect and use the keys from the deployed ML model within the subscription and ML workspace. Deployment code used (in this example for ACI, but looks nearly the same for AKS deployment):

from azureml.core.model import InferenceConfig, Model
from azureml.core.environment import Environment
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.webservice import AciWebservice


ws = Workspace.from_config()
env = Environment(name='scoring_env')
deps = CondaDependencies(conda_dependencies_file_path='./deps')
env.python.conda_dependencies = deps

inference_config = InferenceConfig(source_directory='./prediction/', 
                                   entry_script='score.py', 
                                   environment=env)
deployment_config = AciWebservice.deploy_configuration(auth_enabled=True, cpu_cores=1, 
                                                       memory_gb=1)
model = Model(ws, 'HMM')

service = Model.deploy(ws, 'hmm-scoring', models, 
                       inference_config, 
                       deployment_config, 
                       overwrite=True,)

service.wait_for_deployment(show_output=True)

with conda_dependencies:

name: project_environment
dependencies:
  # The python interpreter version.

  # Currently Azure ML only supports 3.5.2 and later.

- python=3.7.5

- pip:
  - sklearn
  - azureml-core
  - azureml-defaults
  - inference-schema[numpy-support]
  - hmmlearn
- numpy
- pip
channels:
- anaconda
- conda-forge

The code used in the score.py is just a regular score operation with the loaded models and formatting like so:

score1 = model1.score(data)
score2 = model2.score(data)
score3 = model3.score(data)
# Same scoring with model4 and model5

# scaling of the scores to a defined interval and determination of model that delivered highest score

response['score1'] = score1
response['score2'] = score2
# and so on