This is a working memo for a simple Slack Bot using AWS Lambda and Serverless Framework.
Creating a Slack Bot is a laborious process with many steps, but with AWS Lambda, you don't have to worry about the server environment.
In addition, using the Serverless Framework for Lambda development eliminates the need for troublesome work on the AWS console, so it is recommended that it be created rather easily.
We have written a separate article on Slack Bot's general description and GUI work, so if you are completely new to Slack Bot, it would be easier to understand if you read this article first.
This article is primarily a note on how to implement the system.
Requirements for Slack Bot Creation
There are several requirements that must be met in order to create a Slack Bot, so we will focus on those requirements and how to solve them.
Prepare URL and web server
Slack Bot is, roughly speaking, a system that notifies a bot user on a Slack workspace of the contents of an action, such as sending a message, to a pre-registered URL from Slack when the user performs some action.
In other words, a Slack Bot is a mechanism in which a bot serves as a window for collecting events in Slack, and webhooks events that occur in Slack to a specified URL.
The URL must be prepared by the bot developer, but it is quite troublesome to obtain a domain, arrange a web server, etc.
With AWS Lambda, the URL is issued by Amazon and there is no need to prepare a server. Moreover, it doesn't cost money when not in use, so I decided to use Lambda to create a Slack Bot.
Implement URL confirmation
When registering a URL in Slack, Slack will send a confirmation message to the URL to check if the URL is really for that bot. Only after a correct response to the message is received will the URL be registered in Slack.
Slack sends the following messages.
{ "token": "xxxx", "challenge": "xxxx", "type": "url_verification" }
On the server side, the following processes are performed.
- Since the URL will be exposed to the outside world, first check and authenticate if the "token" is from the Bot.
- Slack will send you not only a confirmation message for the URL, but also an event from the bot. check the URL confirmation message, whose "type" will be "url_verification".
- If they are correct, the "challenge" value is responded.
The implementation is as follows
index.ts
exports.handlerHttp = async (event: any, context: any) => { try { if (event.body.token !== SLACK_VERIFICATION_TOKEN) { throw new Error('ERROR : invalid verification token'); } // url verify if (event.body.type === 'url_verification') { return { challenge: event.body.challenge }; } console.log('END : do nothing'); } catch (err) { console.error(err); } };
Note: Always return code 200
Even if the intermediate process fails, we do not catch it with catch() and do not throw, but always respond with a success response with code 200.
If Slack accesses the URL and responds with a code other than 200, Slack recognizes that the access failed and retries. This is true not only for URL confirmation messages, but also for sending events.
And if retries exceed a certain number, Slack will not send any more after that.
So, regardless of what is processed after the URL is accessed, we will always reply to Slack with code 200 so that Slack will not stop sending events.
Implement event processing
Once the URL is successfully registered, the bot will send events to the URL.
Whether the information sent is an event or not is determined by looking at whether the "type" is "event_callback".
The event is assigned an "api_app_id" that identifies which application the event is for, so check that the "api_app_id" is correct.
The implementation is as follows
exports.handlerHttp = async (event: any, context: any) => { try { if (event.body.token !== SLACK_VERIFICATION_TOKEN) { throw new Error('ERROR : invalid verification token'); } // url verify if (event.body.type === 'url_verification') { return { challenge: event.body.challenge }; } if (event.body.api_app_id !== SLACK_APP_ID) { throw new Error('ERROR : invalid app id'); } // slack event if (event.body.type === 'event_callback') { return; } console.log('END : do nothing'); } catch (err) { console.error(err); } };
Here, only return is returned and nothing is done about the event.
No matter what message arrives, as mentioned above, it responds with a code 200, so no retry occurs and the same event never arrives from Slack more than once.
Responds within 3 seconds
If Slack sends an event and receives no response within 3 seconds, Slack recognizes that the access has failed and retries. And as mentioned above, if the retries continue beyond a certain point, the event sending will stop.
Three seconds can be quickly exceeded even when writing a small process.... Therefore, when you receive an event from Slack, you need to reply to Slack first, and then take your time to do the processing you originally wanted to do after you have replied.
As a mechanism for this, we have chosen to use the Simple Queue Service (SQS) of AWS.
What is AWS Simple Queue Service (SQS)?
AWS SQS is a service that temporarily stores messages. You can store a message in SQS and retrieve it later.
SQS and Lambda can easily be used in conjunction with each other: by linking SQS and Lambda functions, Lambda periodically checks for messages in SQS, and if a message arrives, it calls the linked Lambda function with that message as an argument. If a message arrives, Lambda will call the linked Lambda function with the message as an argument.
When an event is sent from Slack, the event content is saved to SQS and the function ends by responding to Slack first. After that, Lambda periodically checks the SQS, so it finds the event it just saved and calls Lambda.
This allows the process to be passed asynchronously to the next Lambda while responding and exiting to Slack.
The implementation is as follows
exports.handlerHttp = async (event: any, context: any) => { try { if (event.body.token !== SLACK_VERIFICATION_TOKEN) { throw new Error('ERROR : invalid verification token'); } // url verify if (event.body.type === 'url_verification') { return { challenge: event.body.challenge }; } if (event.body.api_app_id !== SLACK_APP_ID) { throw new Error('ERROR : invalidate app id'); } // slack event // send sqs and response if (event.body.type === 'event_callback') { const params = { QueueUrl: AWS_SQS_URL, MessageBody: JSON.stringify(event.body) }; return await AWS_SQS.sendMessage(params).promise(); } console.log('END : do nothing'); } catch (err) { console.error(err); } } exports.handlerSqs = async (event: any, context: any) => { try { const sqs_msg = JSON.parse(event.Records[0].body); return; } catch (err) { console.error(err); } };
The event content is sent to SQS and ends with a code 200 as is.
Then the Lambda function "handlerSqs()" will be called by SQS. Here, it does nothing and returns return to terminate normally.
If the Lambda function called from SQS completes successfully, the message is deleted from SQS, so you will not receive the same message more than once.
Responding to Events
The minimum requirements for a Slack Bot have now been met.
All that remains is to write the process of your choice in the Lambda (handlerSqs()) called by SQS.
If you want to make it look like a reply from a bot, you can call the Slack API using the bot user's token.
For example, you can send a message to a bot and have the bot parrot it back to you as follows.
async function postMessage(msg: any) { const event = msg.event; if (!event.bot_id && event.text && event.channel_type === 'im') { const url = `https://slack.com/api/chat.postMessage`; await axios.request({ headers: { authorization: `Bearer ${SLACK_BOT_TOKEN}` }, url, method: 'POST', data: { channel: event.channel, text: event.text, as_user: true } }); } } exports.handlerSqs = async (event: any, context: any) => { try { const sqs_msg = JSON.parse(event.Records[0].body); return await postMessage(sqs_msg); } catch (err) { console.error(err); } }
Since I wanted to make it look like a bot app, I used Google's translation API to translate the message into English and reply when spoken to.
async function translateMessage(msg: any) { const event = msg.event; if (!event.bot_id && event.text && event.channel_type === 'im') { // translate into english const [translation] = await GCP_TRANSLATE.translate(event.text, 'en'); const url = `https://slack.com/api/chat.postMessage`; await axios.request({ headers: { authorization: `Bearer ${SLACK_BOT_TOKEN}` }, url, method: 'POST', data: { channel: event.channel, text: translation, as_user: true } }); } } exports.handlerSqs = async (event: any, context: any) => { try { const sqs_msg = JSON.parse(event.Records[0].body); return await translateMessage(sqs_msg); } catch (err) { console.error(err); } }
The implementation based on the above and more will be as follows.
The information required for authentication is set and passed to the Lambda environment variable.
index.ts
const SLACK_VERIFICATION_TOKEN = process.env.SLACK_VERIFICATION_TOKEN; const SLACK_APP_ID = process.env.SLACK_APP_ID; const SLACK_BOT_TOKEN = process.env.SLACK_BOT_TOKEN; const SLACK_USER_TOKEN = process.env.SLACK_USER_TOKEN; const GCP_PROJECT_ID = process.env.GCP_PROJECT_ID; const AWS_SQS_URL = process.env.AWS_SQS_URL; import axios from 'axios'; import * as AWS from 'aws-sdk'; import { Translate } from '@google-cloud/translate'; const AWS_SQS = new AWS.SQS(); const GCP_TRANSLATE = new Translate({ projectId: GCP_PROJECT_ID }); async function translateMessage(msg: any) { const event = msg.event; if (!event.bot_id && event.text && event.channel_type === 'im') { // translate into english const [translation] = await GCP_TRANSLATE.translate(event.text, 'en'); const url = `https://slack.com/api/chat.postMessage`; await axios.request({ headers: { authorization: `Bearer ${SLACK_BOT_TOKEN}` }, url, method: 'POST', data: { channel: event.channel, text: translation, as_user: true } }); } } exports.handlerHttp = async (event: any, context: any) => { try { if (event.body.token !== SLACK_VERIFICATION_TOKEN) { throw new Error('ERROR : invalid verification token'); } // url verify if (event.body.type === 'url_verification') { return { challenge: event.body.challenge }; } if (event.body.api_app_id !== SLACK_APP_ID) { throw new Error('ERROR : invalidate app id'); } // slack event // send sqs and response if (event.body.type === 'event_callback') { const params = { QueueUrl: AWS_SQS_URL, MessageBody: JSON.stringify(event.body) }; return await AWS_SQS.sendMessage(params).promise(); } console.log('END : do nothing'); } catch (err) { console.error(err); } } exports.handlerSqs = async (event: any, context: any) => { try { const sqs_msg = JSON.parse(event.Records[0].body); return await translateMessage(sqs_msg); } catch (err) { console.error(err); } }
Deploy Lambda with Serverless Framework
Now that we have the program, all we need to do is upload it to Lambda, configure API Gateway, configure SQS, and map it to Lambda: ....... I would like to say that this is quite tedious .......
Therefore, we decided to use the Serverless Framework.
Although the detailed usage of Serverless is omitted, the method of deploying Lambda this time is roughly as follows.
- Install Serverless Framework with
npm install -g serverless - Place the Serverless configuration file "serverless.yml" in the folder where "index.js" is located.
- Set environment variables to be set in Lambda to local environment variables
- Deploy with
serverless deploy
The configuration file is shown below.
serverless.yml
service: slack-bot-test provider: name: aws iamManagedPolicies: - 'arn:aws:iam::aws:policy/AmazonSQSFullAccess' endpointType: REGIONAL runtime: nodejs8.10 stage: dev region: ap-northeast-1 deploymentBucket: name: <アップ先S3バケット名> environment: SLACK_VERIFICATION_TOKEN: '${env:SLACK_VERIFICATION_TOKEN}' SLACK_APP_ID: '${env:SLACK_APP_ID}' SLACK_BOT_TOKEN: '${env:SLACK_BOT_TOKEN}' SLACK_USER_TOKEN: '${env:SLACK_USER_TOKEN}' GOOGLE_APPLICATION_CREDENTIALS: '${env:GOOGLE_APPLICATION_CREDENTIALS}' GCP_PROJECT_ID: '${env:GCP_PROJECT_ID}' AWS_SQS_URL: Ref: slackBotTestSqs resources: Resources: slackBotTestSqs: Type: 'AWS::SQS::Queue' Properties: QueueName: 'slackBotTestSqs' package: exclude: - package.json - package-lock.json functions: slack-bot-test-http-function: name: slack-bot-test-http-function handler: index.handlerHttp events: - http: path: slack/bot method: post integration: lambda slack-bot-test-sqs-function: name: slack-bot-test-sqs-function handler: index.handlerSqs events: - sqs: arn: Fn::GetAtt: - slackBotTestSqs - Arn batchSize: 1
The Severless configuration file is simple, so you will know what we are doing when you see it, but here is a brief supplementary explanation.
- The Lambda environment variable is set to the value of the local environment variable using "env:\~".
- Since we use SQS, we add SQS access rights to the Lambda execution policy.
SQS is created explicitly in the "resources" section because Serverless does not create it automatically.
- Note that you can use AWS CloudFormation's template format for the "resources" section. SQS is created via CloudFormation.
The SQS URL is determined when the SQS is created, so it is passed to the program through the environment variable "AWS_SQS_URL".
You can get the SQS URL by referencing the SQS CloudFormation logical name "slackBotTestSqs" in Ref.
- When reading messages from SQS, multiple messages can be read together, but the "batchSize" sets the number of reads to only 1.
- The Google/GCP section is only used for translation API use, so it can be removed.
Impressions, etc.
That's all for now, we have a template for a Slack Bot. It's still a hassle.
Since Lambda, API Gateway, and SQS are intertwined, it is difficult to try to do the same thing on the AWS console.
Using the Serverless Framework is really convenient because I don't have to touch the AWS console at all, I just have to write the configuration file... Without the Serverless Framework, I would have failed for sure....
At first, I didn't care about error handling and response time, I just made it as I went along and got stuck...
If you return an error code when a problem occurs (or leave it alone without catching an exception), the loop will be "Error->Slack retry->Error..." and finally stop.
Also, if you are careless and write all the processing there when you receive an event, the processing will not finish in time, and it will become a loop of "time over during processing->Slack retry->time over during processing...," and the same processing (response) will be done over and over again, stopping at the end. The same process (response) is performed over and over again, and finally it stops.
Also, the maximum message size that can be sent to SQS is 256 kb, so for full-scale operation, it would be better to use a system in which the body of the post is not sent to SQS but is retrieved again in the Lambda that processes the SQS message.
