AWS Lambda for forwarding a WebHook to Slack

Recently I had to deal with a monitoring service that provided a WebHook to send responses out.  However it was not Slack aware and did not know how to format the data sent in the WebHook so that it would show up in Slack Correctly.

So I thought…

One way to fix this is to have some service sit in-between and capture the outgoing WebHook filter the data and send it on to Slack in a format it understands.

Something like this (I had an Outgoing WebHook in Distill.io).

This in theory would work.  Set up a Server, write a bit of code and run it on the server to accomplish this.  

1.      But that does cause some other problems to crop up…
First I need to rent a server 24x7…. A cloud server is cheap enough… but even at a few $$/ a month it’s a bit overkill for just a few messages every now and then.

2.      I need to write, maintain, debug and deploy the code.

3.      I need to set up some DNS entries for my server to route the traffic

4.      I need to set up SSL certs to encrypt my traffic

5.      If I want a HA system I need to run a couple of servers behind a load balancer in case one should go down.

6.      I need to monitor my system somehow so that when it goes down I know it!

Etc. etc. etc.

All those things may be worth it if you need to relay lots of messages.  But what if you only need to send 100, or 1,000 a month?  What if you only need to send at most 60 per hour.  Those are some pretty low numbers these days.

I think AWS lambda may be an ideal solution for this problem.

What is AWS Lambda?

AWS lambda is a tool that allows you to put a bit of code out there and let AWS handle all the HA, scaling, Monitoring… etc.

https://aws.amazon.com/lambda/ [1]

But it comes at a cost https://aws.amazon.com/lambda/pricing/ [2]

For my purposes my costs should come up to $0.00 a month… So free J

Now onto setting one up

Log into the AWS console and open up the Lambda tool.

Click on create function!

Choose Author from Scratch

Give it a name.  Chose a runtime you want to use.  I am using Node.js 6.10 for this example.

Then you need to give it permissions (a role).  In this simple case I do not need to give it access to anything special like an S3 bucket or anything so I just chose an existing role lambda_basic_execution.  See http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html [3] for more details.

Click Create Function on the lower right.

Congratulations you have a lambda function… Now what?

Setting up a restful endpoint

I can set up a restful endpoint by setting an API gateway.  (Using this tool will have some costs https://aws.amazon.com/api-gateway/pricing/ [6] But, should be minimal.

Click on API Gateway.

Now click the Configuration required.

I am going to choose the defaults here and for the security just select open for now.

Click Add

Click Save

Now I have a URL

In my case it is.

https://8zj3nwhe2m.execute-api.us-east-1.amazonaws.com/prod/webhookRelay

(don’t worry after I finishing writing this up I will be deleting this lambda function entirely)

Now let me see if I can use it!

I am just going to use a simple curl to see if I can get something to show up in the logs.

> curl https://8zj3nwhe2m.execute-api.us-east-1.amazonaws.com/prod/webhookRelay

OK a server error…

Looking around I found this page that goes over how to fix this issue http://www.awslessons.com/2017/lambda-api-gateway-internal-server-error/ [4]

The API gateway is expecting a json object with the three following… statusCode, headers, and body.

So let me update my code to the following and test it again.

exports.handler = (event, context, callback) => {         var response = {         'statusCode': 200,         'headers': { 'Content-Type': 'application/json' },         'body':  "Testing it out"     }     console.log("LOG:: " + JSON.stringify(response))         callback(null, response); };

Here is my updated code…

> curl https://8zj3nwhe2m.execute-api.us-east-1.amazonaws.com/prod/webhookRelay

Wahoo past that hurtle. 

Let me do another quick test to get some JSON response back.

exports.handler = (event, context, callback) => {         var details = {         "one": "44",         "two" : {             "details" :"Some details",             "another" :"More details"         }     }         var response = {         'statusCode': 200,         'headers': { 'Content-Type': 'application/json' },         'body': JSON.stringify(details)     }     console.log("LOG:: " + JSON.stringify(response))         callback(null, response); };

Now let me curl it again put this time pass it to a tool called jq to format it.

> curl -s https://8zj3nwhe2m.execute-api.us-east-1.amazonaws.com/prod/webhookRelay | jq .

Wahoo worked!

Logging

Let’s figure out how to do some logging of incoming data.  I really need that if I am going to debug how the webhook from Distil.io sends data.

Let me update my code to log what I receive. 

exports.handler = (event, context, callback) => {             console.log("MYLOG" + JSON.stringify(event))         var details = {         "one": "44",         "two" : {             "details" :"Some details",             "another" :"More details"         }     }         var response = {         'statusCode': 200,         'headers': { 'Content-Type': 'application/json' },         'body': JSON.stringify(details)     }     console.log("LOG:: " + JSON.stringify(response))         callback(null, response); };

Save it and run a curl test

> curl -s -X POST https://8zj3nwhe2m.execute-api.us-east-1.amazonaws.com/prod/webhookRelay | jq .

I got back the results I wanted.  Now let me go find the local logs in Lambda.

Click On monitoring

Click on Jump to Logs

Click on the time then click on relative

Choose 3 hours.  That will give us the last 3 hours of logs.

Search for the text string “MYLOG”

You may have to hit refresh

There is my text and then there is the JSON file sent to my code !

OK now that I have that working I can work on my webhook to send data to this Lambda function.

Distill.io

I am just using Distill.io in this example

I already have a page I am monitoring in Distilio as a test.  I am just going to open it up and add another action to it.

Click on Add Action and select “Call Webhook”.

From a

Click on Options

Here is what I see.  It looks like this particular outgoing webhook allows you to tweak it a bit to send the data you want to send.

I am going to leave it as is, but put the URL for my lambda function.

Now to save it and test it !

(In distill.io’s case I need to edit the web page it is monitoring to get it to send a message to lambda)

It worked.

It put the info sent into the “body” section.

OK now that I can see what I am being sent let me get the lambda code to send a simple message to a slack room.

Simple Slack message

OK I just want it to send a simple slack message no matter what data come in.  Here is my simple code that accomplishes that.

var https = require('https'); exports.handler = (event, context, callback) => {         console.log("MYLOG" + JSON.stringify(event))         var post_data = JSON.stringify({       "text": "TESTING IT OUT 111"     });       // An object of options to indicate where to post to     var post_options = {       host: 'hooks.slack.com',       port: '443',       path: '/services/YOURWEBHOOKHERE',       method: 'POST',       headers: {           'Content-Type': 'application/json',           'Content-Length': Buffer.byteLength(post_data)       }     };         // Set up the request     var post_req = https.request(post_options, function(res) {       res.setEncoding('utf8');       res.on('data', function (chunk) {           console.log('Response: ' + chunk);       });     });         // post the data     post_req.write(post_data);     post_req.end();             var details = {         "one": "44",         "two" : {             "details" :"Some details",             "another" :"More details"         }     }         var response = {         'statusCode': 200,         'headers': { 'Content-Type': 'application/json' },         'body': JSON.stringify(details)     }     console.log("LOG:: " + JSON.stringify(response))         callback(null, response); };

Of course if you wanted it to hit your slack WebHook you would need to update the path to the correct path.

Now a simple curl to test it.

> curl -s -X POST https://8zj3nwhe2m.execute-api.us-east-1.amazonaws.com/prod/webhookRelay | jq .

Wahoo it worked J

Now let me update my page and see if the Distilio webhook will do the same thing..

OK that worked!

Now to clean it up and make it more complex.

Complex Message

I am using some old notes of mine at http://www.whiteboardcoder.com/2015/03/posting-message-to-slack-via-webhooks.html [5] which show how to use the slack WebHook integration and how to make some fancier formatted messages.

Here is the code I came up with.

var https = require('https'); exports.handler = (event, context, callback) => {         console.log("MYLOG" + JSON.stringify(event))         var post_data = JSON.stringify({         "username": "Distil.io",         "icon_emoji":":distillio:",         "channel": "#distillio",         "attachments": [             {                 "color": "danger",                 "fields": [                     {                         "title": "Website Changed!",                         "value": "The website changed",                         "short": false                     }                 ]             }         ]     });       // An object of options to indicate where to post to     var post_options = {       host: 'hooks.slack.com',       port: '443',       path: path: '/services/YOURWEBHOOKHERE',       method: 'POST',       headers: {           'Content-Type': 'application/json',           'Content-Length': Buffer.byteLength(post_data)       }     };         // Set up the request     var post_req = https.request(post_options, function(res) {       res.setEncoding('utf8');       res.on('data', function (chunk) {           console.log('Response: ' + chunk);       });     });         // post the data     post_req.write(post_data);     post_req.end();             var details = {         "one": "44",         "two" : {             "details" :"Some details",             "another" :"More details"         }     }         var response = {         'statusCode': 200,         'headers': { 'Content-Type': 'application/json' },         'body': JSON.stringify(details)     }     console.log("LOG:: " + JSON.stringify(response))         callback(null, response); };

If I run a simple curl test against this I get

It now uses an emoji as a user icon, has a username of Distil.io, and highlights the message with a red line.

What it does not do yet is grab incoming data and put it into my slack message.

OK after many tries and tweaks got this one working.

var https = require('https'); exports.handler = (event, context, callback) => {         console.log("MYLOG" + JSON.stringify(event))         //Custom Fields sent in from Distilio     var body = JSON.parse(event.body)     var name = body.name     var uri = body.uri     var text = body.text.substring(0, 120) //Don't want all the data         var post_data = JSON.stringify({         "username": "Distil.io",         "icon_emoji":":distillio:",         "channel": "#distillio",         "attachments": [             {                 "color": "danger",                 "fields": [                     {                         "title": name + " Website Changed!",                         "value": "uri: " + uri                                + "\nbody: " + text,                         "short": false                     }                 ]             }         ]     });       // An object of options to indicate where to post to     var post_options = {       host: 'hooks.slack.com',       port: '443',       path: '/services/YOURWEBHOOKHERE',       method: 'POST',       headers: {           'Content-Type': 'application/json',           'Content-Length': Buffer.byteLength(post_data)       }     };         // Set up the request     var post_req = https.request(post_options, function(res) {       res.setEncoding('utf8');       res.on('data', function (chunk) {           console.log('Response: ' + chunk);       });     });         // post the data     post_req.write(post_data);     post_req.end();             var details = {         "status": "OK"     }         var response = {         'statusCode': 200,         'headers': { 'Content-Type': 'application/json' },         'body': JSON.stringify(details)     }     console.log("LOG:: " + JSON.stringify(response))         callback(null, response); };

I also put this up as a gist for easy copying.

https://gist.github.com/patmandenver/16f164d9ccd88af976d1fea21d8b92d4

References

[1]        AWS Lambda

            https://aws.amazon.com/lambda/
                Accessed 12/2017

[2]        AWS Lambda Pricing

            https://aws.amazon.com/lambda/pricing/
                Accessed 12/2017

[3]        AWS Lambda intro to permissions.

            http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html
                Accessed 12/2017

[4]        Solving AWS Lambda and API Gateway Internal Server Errors

            http://www.awslessons.com/2017/lambda-api-gateway-internal-server-error/
                Accessed 12/2017

[5]        Posting Message to Slack via Webhooks Integration

            http://www.whiteboardcoder.com/2015/03/posting-message-to-slack-via-webhooks.html
                Accessed 12/2017

[6]        Amazon API Gateway Pricing

            https://aws.amazon.com/api-gateway/pricing/
                Accessed 12/2017