Is azure functions a right fit for long running tasks?

user_1463551 0
Viewed 9617

A TCP Host server is located on-premises. Now from Azure environment I need to connect with the host server. Once the connection is established, the host server will be sending messages continuously, and the received messages will be forwarded to Azure Service Bus queue.

I was given the following architecture(simplified). The architecture proposes a function app which will be used to connect with the on-prem server, and then receives messages continuously, and then place the messages on to the service bus queue.

enter image description here

Per Microsoft guidelines it says we must avoid long running functions, as it causes timeout issues.

Considering the above requirement (establishing connection and receiving messages - i.e. long running task) and MS guidelines, is Azure Function a right choice? I believe a web job or a web app would be a better fit for this purpose.

azure-webjobs azure-functions azure
4 Answers
user_13845390

A function is the wrong technology choice in this case, you could easily switch out the function for a Continuous WebJob that lives in an Azure Web App. Web Apps and Function Apps are both types of Azure App Services so it's quite a simple change.

user_89449080

as per Microsoft guidelines it says we must avoid long running functions, as it causes timeout issues.

Yes this is the recommended way when you are using Azure function that runs for less that 60 minutes. azure Function in premium plan supports guaranteed 60 minutes .However for long running scenarios you can use Durable Function which are intented to solve complex scenarios which lets you split up your jobs into smaller junks.

In your scenario i would recommend you to add a middleware component such as eventhub which could retain the data for certain amount of time and use different function to process them later asynchronously.

user_17494030

Few queries I have:

query 1 - will TCP Host server sending messages continuously to Azure Function or will Azure Function talk to 'TCP Host Server' and get the message and then add to the Service Bus Queue?

Query 2 - if Azure Function will talk to the 'TCP Host Server' and get message, will it be like makes a connection, gets a message, adds message to queue and disconnects OR will it be make one connection which is never ending and keeps getting messages and keeps adding them to the queue?

if it makes a connection to get one message or more than one message lets say every few seconds or few minutes then you can use Azure Function. In this scenario you can use a CRON job type Azure function with a schedule to achieve this and this will not count as a long running Function as it will always end in a schedule you define.

But if you want to make one long connection, then I would not recommend using Azure Function, even with Durable Function as it beats the concept of durable Function as well as it supposed to end. Either use Event Hub if you can send the message from your on-prem server to Event Hub. Event Hub would be a very scalable way to do this as you can scale it up if you have millions or trillions of messages and it is specifically built for this. You can also use Azure IoT Hub with Protocol Gateway, where your gateway can translate your TCP messages to an IoT Hub supported protocol (HTTP, AMQP or MQTT).

I know I made this confusing, if you want to keep it simple then use Function but with a scheduled CRN job like manner to avoid the long running pitfall.

user_74896770
Related