Serverless is all the rage. Ever since AWS Lambda was announced at re:Invent 2014, there's been an explosion in the serverless space. We've seen more FaaS offerings, more event sources, more managed services, and more pay-per-use pricing models.
Serverless has unlocked a whole new world for developers. But how do you get started? Because serverless is so new, it requires learning new patterns and skills.
In this post, I'll cover a few categories that I see as good entry-level projects with serverless. It's kind of like having your training wheels on -- you get used to the motions and how things work without making a big bet with your core business applications.
There's a little something for everyone. In this post, we'll cover:
For the Ops engineer: how to extend AWS functionality with Lambda and the Boto3 SDK
For the web developer: how to move existing web applications to run on AWS Lambda
For anyone that wants to be a hero: how to handle a webhook from a third-party SaaS
Let's get started!
Extending AWS with Lambda and the Boto3 SDK
If your job title is 'Infrastructure Engineer', 'Cloud Architect' or anything else that requires bookmarking the AWS documentation, you probably see little holes in AWS offerings.
Perhaps you want to schedule periodic backups of your EBS volumes or DynamoDB tables, something that wasn't possible until the recent addition of AWS Backup in early 2019.
Or maybe you want to scale up your DynamoDB throughput early in the morning before your traffic spikes, then scale it back down at night to save money.
Or you want to terminate all instances in your development account that have been running for more than 7 days.
While you wait for AWS to add these little features, you can fill the gaps yourself. One popular way to do this is to trigger AWS Lambda with a CloudWatch Event on a given schedule to run bits of arbitrary code.
This is the 'gateway drug' to AWS Lambda for a lot of operations folks. You don't need to pay for a full-time server to manage your cron jobs -- you can run it basically for free with AWS Lambda.
Example Ops Projects
Tools of the Trade for Ops projects
A Lambda deployment tool. Building your Lambda zip files and wiring up the necessary resources manually is a huge hassle. Luckily, there are a ton of tools to make it easy.
I recommend the Serverless Framework (full disclosure: I work for Serverless, Inc., and you should come work with me). It's the most widely-used serverless deployment tool available, and it has a great community. There are a wide variety of plugins available to extend the core functionality.
You can also use AWS SAM or a variety of other deployment tools.
Python. Python is the lingua franca for most ops tasks due to easy learning curve and pseudocode-like syntax. It's also a great fit for AWS Lambda as it has a strong standard library so you don't have to worry about using a bunch of third-party packages.
Boto3, the AWS SDK for Python. If you're doing ops tasks with Lambda, you'll probably interact with the AWS APIs. Boto3 is a solid tool with good documentation and lots of community examples.
Boto3 comes built into the AWS Lambda environment for convenience. However, make sure you read Tom McLaughlin's piece on whether you should bundle your own version of Boto3 in AWS Lambda.
Moving your existing web application to AWS Lambda
This project is aimed at the other side of the Wall of Confusion: web developers.
If you're a web developer, you don't want to mess with operations. You just want the dang thing to run so you can focus on building.
However, with Heroku and Elastic Beanstalk, you're not entirely out of the ops game. You still need to capacity plan to determine how many instances you want to run. Further, you're paying for these instances regardless of the amount of traffic you have on them.
If you run your web application on AWS Lambda, those concerns go away. The compute layer is infinitely scalable, and you pay only when your application is being used.
Further, with the tools below, you can convert your existing application with minimal changes.
Using existing web frameworks in Lambda isn't ideal -- you're not embracing serverless as fully as you should -- but the downsides are overstated. For web developers new to serverless, this is a great on-ramp to see the magic of serverless.
Just make sure it's an on-ramp and not a final destination 😉.
Example web application resources
Below are a few good articles for using AWS Lambda with existing web applications:
Tools of the Trade for Web Apps
Additional tools are:
serverless-http: This is a plugin for the Serverless Framework to make it simple to convert existing NodeJS applications written in Express, Koa, Hapi, and many other frameworks.
serverless-wsgi: Another Serverless Framework plugin, but aimed at the Python crowd. This lets you convert WSGI-based applications, such as Django, Flask, or Pyramid, into Lambda-compatible services.
serverless-rack: Ruby fans aren't left behind. This lets you deploy Rack-based applications like Rails, Sinatra, and Padrino to AWS Lambda.
Handling a webhook from a third-party SaaS
A third example project is to handle a webhook from a SaaS solution. This could be:
Handling a GitHub webhook that notifies you of a new Pull Request or Issue
Building a Slack slash command to perform DevOps tasks or to track employee sentiment
Catching a Stripe webhook that alerts you that a charge was denied
While this is architecturally similar to the previous example (setting up HTTP endpoints using API Gateway & Lambda), I think this use case is a better on-ramp for future serverless usage for a few reasons:
It's accessible to everyone.
SaaS is used by everyone, from application developers to sysadmins to marketing folks to the HR team. Not everyone is building a full user-facing app, but everyone uses SaaS.
It's so clearly better than the alternatives.
Without serverless, standing up an HTTP endpoint to handle a single webhook is a huge burden.
First, you need to provision a server ($$$).
Then you need to figure out how to get your code on there, which could be an ugly copy-paste or might involve a
git clonewhich means now you need to worry about your GitHub credentials.
Finally, you need to make sure it's publicly-accessible and has a DNS record configured.
All that trouble for a simple webhook?
With serverless, it's much easier. You can set up an infinitely-scalable webhook that costs basically nothing.
Big time hero potential.
You actually learn the Lambda model.
When you're converting an existing web app to Lambda, you're only going partway down the serverless path. You get the simple deployments, easy scaling, and nice pricing model, but you're still thinking in the old-world, serverfull mindset.
With a small webhook, it's easier to buy into the Lambda model. You use the default Lambda function signature of
(event, context)to handle your function rather than the safety of Express's
Further, if you need to store a little state, it's easy to reach for something like DynamoDB. The simple use case is one of the key DynamoDB design patterns I see due to its perfect fit with Lambda and key-value data model.
After setting up a few webhooks, you start to feel more comfortable. Before you know it, you're using serverless technologies for a larger use case.
Example webhook projects
GitHub webhook example to alert you in Slack when someone stars your project.
Tools of the Trade for Webhooks
In addition to AWS Lambda and the Lambda deployment tools mentioned in previous sections, you should also check out:
Serverless applications are making it easier and easier to build powerful applications. Yet ramping up with serverless can be a daunting experience, particularly for those without significant experience with cloud infrastructure.
The three categories of projects in this post are great ways to get acquainted with serverless. For each of them, they can solve a key pain point without re-platforming your entire production infrastructure.
Are there additional projects you'd like to hear about? Reach out and let me know!