Serverless computing with Azure Functions – interaction with Event Hub

In a previous article, I described my first steps with Azure Functions – one of the implementation mechanisms for serverless computing: Serverless Computing – Function as a Service (FaaS) – with Azure Functions – first small steps with a Node/JavaScript function. Functions can be triggered in many ways – by HTTP requests, the clock (scheduled), by database modifications and by events. In this article, I will look at a Function that is triggered by an event on the Azure Event Hub. I will also show how a function (triggered by an HTTP request) can write to Event Hub.

Functions can have triggers and input bindings. The first is the cause of the function to run – which can have a payload. An input binding is a declarative definition of data that the function has (read) access to during execution. Function also can have Output bindings – for each of the channels to which they write results.

Steps

The first steps: arrange Azure account, create Event Hubs namespace – as the context in which to create individual event hubs (the latter are comparable to Kafka topics)

On the Event Hub side of the world:

• Create Event Hub
• Create Shared Access Policy
• Get Connection String URL for the shared access policy

In Azure Functions –

• At the function app level: Create Connection String for Connection String URL copied from shared access policy
• Create a function based on the template Data Processing/JavaScript/EventHub Trigger – a JavaScript function triggered by a message on the indicated Event Hub in the Event Hub namespace addressed through the connection string; save and (test) run the function (this will publish an event to the event hub)
• Optionally: create a second function, for example triggered by an HTTP Request, and have it write to an output binding to the Event Hub; in that case, an HTTP request to the second function will indirectly – through Event Hub – cause the first function to be executed

 

In Event Hub Namespace

Create Event Hub GreetingEvents. Set the name and accept all defaults. Press Create.

 

Once the Event Hub creation is complete, we can inspect the details – such as 1 Consumer Group, 2 Partitions and 1 Day message retention:

This is our current situation:

 

Now return to the overview and click on the link Connection Strings. We need the to create a connection from Azure Functions app to Event Hub Namespace using the URL for the Shared Policy we want to leverage for that connection.

Click on Connection Strings to bring up a list of Shared Policies. Click on the Shared Policy to use for accessing the Event Hub namespace from Azure Functions.

Click the copy button to copy the RootManageSharedAccessKey connection string to the clipboard.

In Azure Function App

In order for the Function to access the Event Hub [Namespace], the connection string to the Event Hub [Namespace[ needs to be configured as app setting in the function app [context in which the Function to be triggered by Event Hub is created]. Note: that is the value in the clipboard.

Scrolll down.

Create Connection String to Event Hub Namespace using the value in the clipboard

 

Save changes in function app

At this point, a link is established between the function app (context) and the Event Hub Namespace. Any function in the app can link to any event hub in the namespace.

 

Create Function to be Triggered by Event

With the connection string in place, we can create a function that is executed when an event is published on Event Hub greetingevents. That is done like this:

Type the name of the function, click on the link new and select event hub greetingevents to associate the function with:

 

 

Click on create.

The function is created – including the template code:

 

The configuration of the function is defined in the file function.json. Its contents can be inspected and edited:

The value of connection is a reference to an APP Setting that has been created when the function was created, based on the connection string to Event Hub Namespace.

Click on Save and Run. A test event is published to the Event Hub greetingevents. In the log window – we can see the function reacting to that event. So we have lift off for our function – it is triggered by an event (and therefore presumably by all events) on the Event Hub and processes these events according to the (limited) logic it currently contains.

The set up looks like this:

 

 

Publish to Event Hub from Azure Function

 

To make things a little bit more interesting we will make the Azure Function that was introduced in a previous article for handling HTTP Request “events” also produce output to the Event Hub greetingevents. This means that any HTTP request sent to function HttpTriggerJS1 leads to an event published to Event Hub greetingevents and in turn to function EventHubTrigger-GreetingEvents being triggered.

 

To add this additional output flow to the function, first open the Integration tab for the function and create a new Output Binding, of type Azure Event Hubs. Select the connection string and the target Event Hub – greetingevents. Define the name of the context parameter that provides the value to be published to the Event Hub – outputEventHubMessage:

We now need to modify the code of the function, to actually set the value of this context parameter called outputEventHubMessage:

At this point, we can test the function – and see how it sends the event

that indirectly triggers our former function.

When the HTTP Request is sent to the function HttpTriggerJS1 from Postman for example

The function returns it response and also publishes the event. We can tell, because in the logging for function EventHubTrigger-GreetingEvents we see the name sent as parameter to the HttpTriggerJS1 function.

(Note: In this receiving function, I have added the line the red to see the contents of the event message.)

 

 

Resources

Azure Function – Event Hub binding – https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs

Azure Documentation on Configuring App Settings – https://docs.microsoft.com/en-us/azure/azure-functions/functions-how-to-use-azure-function-app-settings#settings

Azure Event Hubs Overview – https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-what-is-event-hubs

Azure Functions Triggers and Binding Concepts – https://docs.microsoft.com/en-us/azure/azure-functions/functions-triggers-bindings