How to guide on running Serverless.com's Event Gateway on Kubernetes on Azure. This is derived from Kelsey Hightower's Serverless Event Gateway on Kubernetes to use Azure. Kelsey's original tutorial is based on GCP.
This tutorial assumes you have Azure Subscription to deploy into.
In this tutorial, you need azure-cli command line tool to run this workthrough. Suppose you want to operate locally and you don't yet have azure-cli installed on your local environment, you can install the azure-cli like this:
sudo pip install -U azure-cli
You can skip this installation if you're running this workthrough on Azure Cloud Shell Bash where the azure-cli is pre-installed.
- Creating a Kubernetes Cluster(AKS)
- Bootstrapping an Event Gateway Cluster
- Routing Events to Azure Functions
- Routing Events to Kubernetes Services
- Clean Up
First of all, execute the following commands which are needed in case that it's the first time to manage network & compute resources with your subscription:
$ az provider register -n Microsoft.Network
$ az provider register -n Microsoft.Compute
$ az group create --name rg-eventgateway --location eastus
[note] This tutorial assumes that you create the resource group named
rg-eventgateway
ineastus
region
The remainder of this tutorial requires access to a Kubernetes 1.9.7+
cluster
$ az aks create \
--resource-group rg-eventgateway \
--name myaks-eventgateway \
--kubernetes-version 1.9.9 \
--generate-ssh-keys
- [note1] This tutorial assumes that you create the AKS cluster named
myaks-eventgateway
under the resource group namedrg-eventgateway
- [note2] SSH key files
$HOME/.ssh/id_rsa
and$HOME/.ssh/id_rsa.pub
have been generated under ~/.ssh to allow SSH access to the VM. If using machines without permanent storage like Azure Cloud Shell without an attached file share, back up your keys to a safe location- If you already have a ssh key generated and you want to use it instead of generating new key, specify your SSH key with
--ssh-key-value
option instead of--generate-ssh-keys
in creating AKS Cluster. Please see azure CLI command reference for az aks create for more details
If you want to install kubectl CLI locally, run the following command:
$ az aks install-cli
[note] You can skip this installation if you're running this workthrough on Azure Cloud Shell Bash where the kubeclt is pre-installed.
Run the following command to configure kubectl to connect to your Kubernetes cluster, run the following command:
$ az aks get-credentials --resource-group=rg-eventgateway --name=myaks-eventgateway
Finally, check if you can connect to the cluster by running the following command:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
aks-nodepool1-97802812-0 Ready agent 14m v1.9.9
aks-nodepool1-97802812-1 Ready agent 14m v1.9.9
aks-nodepool1-97802812-2 Ready agent 14m v1.9.9
In this section you will bootstrap a two node Event Gateway cluster suitable for learning and demonstration purposes.
The Event Gateway configuration used in this tutorial is not recommended for production as it lacks any form of security or authentication.
etcd is used to store and broadcast configuration across an Event Gateway cluster. A dedicated etcd cluster should be provisioned for the Event Gateway. Create the etcd
statefulset:
$ kubectl apply -f statefulsets/etcd.yaml
statefulset "etcd" created
service "etcd" created
Verify the etcd
cluster is up and running:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
etcd-0 1/1 Running 0 20s
Create the event-gateway
deployment:
$ kubectl apply -f deployments/event-gateway.yaml
deployment "event-gateway" created
service "event-gateway" created
At this point the Event Gateway should be deployed and exposed via an external load balancer accessible to external clients. Verify the Event Gateway is up and running:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
etcd-0 1/1 Running 0 12m
event-gateway-5ff8554766-jhs6v 1/1 Running 0 4m
event-gateway-5ff8554766-jpmwp 1/1 Running 0 4m
Print the event-gateway
service details:
$ kubectl get svc event-gateway
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
event-gateway LoadBalancer 10.0.112.249 XX.XXX.XXX.XX 4000:30152/TCP,4001:30426/TCP 5m
Extract the event-gateway
external IP address and store it:
export EVENT_GATEWAY_IP=$(kubectl get svc event-gateway \
-o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $EVENT_GATEWAY_IP
In this section you will deploy the echo
function in Azure Functions used to test the event routing functionality of the Event Gateway.
Sample
echo
function that is used in this tutorial is not recommended for production as it lacks any form of security or authentication (authLevel isanonymous
).
Deploy the echo
function in Azure Function by executing scripts/deploy-function.sh
. Before executing the script, you need to add globally unique Azure Function acccount name and Azure Storage account name to FUNCTION_NAME
and STORAGE_NAME
variable respectively.
#!/bin/bash
set -x -e
cwd=`dirname "$0"`
expr "$0" : "/.*" > /dev/null || cwd=`(cd "$cwd" && pwd)`
RESOURCE_GROUP="rg-eventgateway"
REGION="eastus"
FUNCTION_NAME="myazfunc001"
STORAGE_NAME="myazfuncstore001"
## Create Storage Account for Azure Function
echo "Create Storage Account for Azure Function..."
az storage account create \
--name $STORAGE_NAME \
--resource-group $RESOURCE_GROUP \
--sku Standard_LRS \
--kind Storage
## Create Functions App (Consumption Plan)
echo "Create Functions App (Consumption Plan)..."
az functionapp create \
--name $FUNCTION_NAME \
--resource-group $RESOURCE_GROUP \
--consumption-plan-location $REGION \
--storage-account $STORAGE_NAME
## Deploying functions
echo "Deploying functions..."
cd $cwd/../functions
zip -r functions.zip .
az functionapp deployment source config-zip --name $FUNCTION_NAME \
--resource-group $RESOURCE_GROUP \
--src functions.zip
rm functions.zip
echo "echo function URL: https://$FUNCTION_NAME.azurewebsites.net/api/echo"
[note] This tutorial assumes that you create the Azure Function account named
myazfunc001
and Azure Storage account namedmyazfuncstore001
under the resource group namedrg-eventgateway
whichi is located ineastus
region. Again, both Azure Function account name and Storage account name must be globally unique.
Then, execute the script, scripts/deploy-function.sh
to deploy the echo
function in Azure Function:
$ scripts/deploy-function.sh
Get the HTTPS URL assigned to the echo
function and store it to FUNCTION_URL
enviroment variable:
export FUNCTION_URL="https://${FUNCTION_NAME}.azurewebsites.net/api/echo"
The FUNCTION_URL
environment variable will be used in the next section to register the echo
cloud function with the Event Gateway.
In this section you will register the echo
cloud function with the Event Gateway.
Register the echo
cloud function by posting the function registration object to the Event Gateway:
curl --request POST \
--url http:https://${EVENT_GATEWAY_IP}:4001/v1/spaces/default/functions \
--header 'content-type: application/json' \
--data "{
\"functionId\": \"echo\",
\"type\": \"http\",
\"provider\":{
\"url\": \"${FUNCTION_URL}\"
}
}"
At this point the echo
function has been registered with the Event Gateway, but before it can receive events a subscription must be created.
A subscription binds an event to a function. In this section you will create an HTTP event subscription that binds the echo
cloud function to HTTP events received on the POST
method and the /
path pair:
curl --request POST \
--url http:https://${EVENT_GATEWAY_IP}:4001/v1/spaces/default/subscriptions \
--header 'content-type: application/json' \
--data '{
"functionId": "echo",
"event": "http",
"method": "POST",
"path": "/"
}'
With the echo
function registered and subscribed to HTTP events you can test the configuration by emitting HTTP events to the Event Gateway.
Submit an HTTP event to the Event Gateway:
curl -i --request POST \
--url http:https://${EVENT_GATEWAY_IP}:4000/ \
--data '{"message": "Hello world!"}'
The echo
function will respond with the data submitted in the HTTP event:
HTTP/1.1 200 OK
Compute-Type: function
Date: Wed, 18 Jul 2018 14:41:36 GMT
Content-Length: 26
Content-Type: text/plain; charset=utf-8
{"message":"Hello world!"}
Notice the value of the
Compute-Type
HTTP header. It was set tofunction
by theecho
function.
In addition, check the echo
function logs in the portal Logs window. Output similar to the following is logged in executing the function:
2018-07-18T15:08:45.581 [Info] Function started (Id=8dea1402-8e84-4de1-8729-8e89fe92fb0c)
2018-07-18T15:08:45.581 [Info] JavaScript HTTP trigger function processed a request.
2018-07-18T15:08:45.581 [Info] Function completed (Success, Id=8dea1402-8e84-4de1-8729-8e89fe92fb0c, Duration=3ms)
In modern Serverless architectures events are typically routed to functions running on fully managed FaaS platforms such as Azure Functions, Google Cloud Functions, or AWS Lambda. In some situations, such as low latency requirements, it maybe preferable to route events to existing applications running on traditional infrastructure.
In this section you will deploy the echo
application using Kubernetes and configure the Event Gateway to route HTTP events to it.
Create the echo
deployment and service:
$ kubectl create -f deployments/echo.yaml
deployment "echo" created
service "echo" created
Verify the echo
deployment is up and running:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
echo-77d48cb484-728kq 1/1 Running 0 1m
etcd-0 1/1 Running 0 2m
event-gateway-5ff8554766-jhs6v 1/1 Running 0 2m
event-gateway-5ff8554766-jpmwp 1/1 Running 0 2m
Register the echo
service using an unique function ID:
curl --request POST \
--url http:https://${EVENT_GATEWAY_IP}:4001/v1/spaces/default/functions \
--header 'content-type: application/json' \
--data '{
"functionId": "echo-service",
"type": "http",
"provider":{
"url": "http:https://echo.default.svc.cluster.local"
}
}'
The provider URL follows the standard format for accessing cluster local services. In this case the
echo
deployment runs in thedefault
namespace. This configuration works because the Event Gateway is running in the same cluster as theecho
deployment.
At this point the echo
service has been registered with the Event Gateway.
In this section you will create an HTTP event subscription that binds the echo-service
to HTTP events received on the POST
method and the /echosvc
path pair:
curl --request POST \
--url http:https://${EVENT_GATEWAY_IP}:4001/v1/spaces/default/subscriptions \
--header 'content-type: application/json' \
--data '{
"functionId": "echo-service",
"event": "http",
"method": "POST",
"path": "/echosvc"
}'
If you want to subscribe this new function to HTTP events on POST method and
/
path, you need to delete the current subscription for HTTP events on thePOST
method/
path pair first, then create an HTTP event subscription for the echo-service function as there can only be one function subscribed to HTTP events for a given path and method pair.
Test the echo
service by emitting an HTTP event to the Event Gateway:
curl -i --request POST \
--url http:https://${EVENT_GATEWAY_IP}:4000/echosvc \
--data '{"message": "Hello World!"}'
HTTP/1.1 200 OK
Compute-Type: container
Date: Wed, 18 Jul 2018 18:09:43 GMT
Content-Length: 27
Content-Type: text/plain; charset=utf-8
{"message": "Hello World!"}
Notice the value of the
Compute-Type
HTTP header. It was set tocontainer
by the echo service.
Review the echo
container logs:
$ kubectl logs echo-77d48cb484-728kq
2018/07/18 17:28:26 Starting HTTP server...
2018/07/18 18:08:22 Handling HTTP event 18efc24a-5735-41a8-8d39-45fa70f02b6b ...
$ az group delete --name rg-eventgateway
[note] This tutorial assumes that you create all resources in Azure under the resource group named
rg-eventgateway
.