It’s often useful to set certain scripts to run regularly, or in response to some event. There are some websites that help you do this such as Zapier or IFTTT which integrate with other online services. However, they cost money, and its often more useful to be able to control this yourself. You could use cron on your local machine – but then you’d have to leave it on permanently. You could run it on a server – but what happens if you need to change or remove the server or don’t want to pay the cost – all the scripts need setting up again. Enter AWS lambda.
This service allows you to run a specific piece of code (Python, Node.js java etc) in response to an AWS event, or at a scheduled time (using AWS Cloudwatch) which you can specify it using a cron expression. The code is serverless – you don’t need to do any management at all, and you are billed per 100ms, only for the time your code executes – so it is extremely cheap! The main disadvantage is that the AWS console is quite complicated at first, and there are a few pitfalls. This walkthrough will identify a couple of the ones I came across.
1. Objective
I record my timesheets in Toggl, but I would like to post a summary each week to my company slack account so my boss can check on my progress. I also want to total up the hourly years to date and send that too. So I’m interacting with these two websites via their APIs.
2. Python Code
The python code is fairly straightforward – see my github repo for details.
In addition we need a file named config.py which has variables with the various user specific details. You’ll need to get your slack and toggl. api, and toggl workspace number:
SLACKAPI=”XYZ”
TOGGL_API=”XYZ”
TOGGL_WORKSPACE=”XYZ”
YEAR_PROJECT=”XYZ” (the name of the project I want to get a yearly running total for).
The important thing to note is that the code to be executed needs to be inside a function we will need to specify, in this case called lambda_handler. Furthermore, as I discovered after getting errors in the logs, this function MUST accept 2 arguments, event and context. Even though the function does not require these, the AWS Cloudwatch ‘cronjob’ sends these to the function when it triggers.
3. Packaging the python code and dependencies
AWS Lambda provides a python environment to execute the code, however in most cases there will be some additional packages you will need to include, which has to be uploaded to AWS lambda in a specific deployment package along with your code. The easiest way to do this, is on your local machine to create a python virtual environment, and install the packages, and then zip the files up as per AWS’s instructions. In this case we need the requests and slack packages.
virtualenv env source env/bin/activate pip install requests slack zip package.zip weeklytime.py zip -r package.zip config.py cd ./env/lib/python2.7/site-packages/ zip -r ../../../../package.zip *
The github repo has a script to automate this – and we are now ready to upload it to AWS.
4. Creating AWS lambda function in the console
4.1 Sign into AWS console, and goto Services > Compute > Lambda and click ‘Create Lambda Function’
4.2 On the next screen, select the Python2.7 runtime and select the hello world python example.
4.3 In the triggers section, select CloudWatch Events – Schedule from the dropdown box. We can now fill in the Rule Name and Rule Description, and the Schedule expression. This final piece of information could be a rate (every minute, hour etc) of which there are examples in the dropdown box. I wanted my script to run every monday at 7am. Here is where you need to watch out to conform to AWS CloudWatch Schedule cron description. The standard way to specify this cron schedule on a linux crontab is (0 7 * * 1), however you also need to specify the year and one of the day-of-month or day-of-week values must be a question mark (?), hence the final statement: cron(0 7 ? * 1 *).
4.4 In the Configure function section, enter the name and description for the lambda function, and select python2.7 as the runtime. The code entry type should be upload a zip file, and you can select the package.zip file we created in step 3. We also need to specify a handler, in the format filename.function_name, in our case weeklytime.lambda_handle. The roles should be generated automatically – now click next.
4.5 Finally in the review section click create function. If for some reason the cron command is not correct, it will create the lambda function but not the trigger. In this case you’ll need to navigate to your newly created function, and click on the trigger tab, and follow the steps to create the CloudWatch – Schedule trigger as above.
5. That’s it – this code will now be executed on the schedule you specify – no server needed! I tested my function by temporarily adding a 1 minute recurring trigger, which I then deleted.
