Configure AWS Lamba programmatically with Boto3
Posted on:23 Apr 2019
Serverless computing has generated a lot of interest in the development community. AWS Lambda is the AWS version of Serverless. In this post, we will look at configuring AWS Lambda programmatically with boto3. We will configure a lambda function that connects to a Postgres DB on an EC2 instance in a private VPC using sqlalchemy. This would need packaging of the dependencies of sqlalchemy along with the lambda function.
By default, the instance where AWS lambda is running has Boto installed (AWS Python SDK). So, importing a boto3 package in python code will work without any other packaging. But importing a package like “import sqlalchemy” will lead to python error “Unable to import module sqlalchemy”. This article shows how to achieve the same.
How to Configure AWS Lambda programmatically with Boto3
Step 1: Write the AWS Lambda function
First we need to write the python code which will run as an AWS Lambda. This below code creates a python file and zips it. Note down, the python code that we wanted to execute should contain a method called lambda_handler which serves as an entry point for the lambda to start the execution.
One quick aside: SqlAlchemy complains about not being able to find the right
provider when connecting to postgres from lambda. So might have to add
postgressql + pscycopg2 as part of connection string.
Don't forget to open the port in the security group and enable incoming
connections from all in the host file of the Postgres server.
This can be a cause of a lot of heartaches.
Also, remember if you the system you package the code is windows, then you are in
some trouble as the pip will pick windows related drivers. So you
need to run this on lambda compatible OS (read as Amazon Linux or compatible OS).
Else, you will end up with a lot of wierd run time errors.
Step 2: Create the AWS Lambda Function
Now use boto3 client API create_function to map it to the above code. If you take a close look, Handler=‘lambda_function.lambda_handler’ means that it will look for the python file lambda_function.py and inside it it will look for lambda_handler function. Also, you need to pass the right arn for this API to work.
Step 3: Set the frequency of lambda
Set the rule and the frequency at which you want the lambda to run, it can be a simple expression like rate(1 minute) or rate(5 minutes) or a cron expression too. Also, set the rule target and grant permission for the lambda to execute. In this case, we are configuring the rule to run every minute.
Step 4: Additional Package Deployment
Do this step if we need other packages for our python code to run. For creating deployment folders, you can use this python script create_deployment.py andrequirements.txt. Basically, when the script file runs, it picks the packages that need to be installed from requirements.txt and does a pip -install -t and which creates a folder, pull all the packages and zips them. If you need to have a particular version you can say sseclient==0.0.11 or you can omit the version to pull the latest.
Final Step: Putting it together
Now putting everything together yields the following output in the AWS CloudWatch logs, no import module error this time. This can be modified to use whichever package you need for your development purposes.
Hope this saves a few hours trying to jump a few hoops which weren’t really apparent at the onset!