Serverless is becoming mainstream, but what exactly does "Serverless" mean?

It means different things to different people, but there are always servers...

There’s been a lot of discussion recently on the topic of Serverless is becoming mainstream (or variations on that wording) and I didn’t want to miss my chance to contribute. I’ve been curious what earns a cloud-based feature the Serverless badge of honour, versus simply being Cloud-based or Cloud-native. It seems that lots of really smart people have asked this same question, and there’s no concise answer.

Over the last month, I’ve read and listened to people’s description of what Serverless means to them. My conclusion: There’s no single definition of whether something is Serverless or not. Instead, it’s a set of characteristics allowing you to focus on the business logic of your application, rather than the heavy-lifting of managing infrastructure. Any particular service falls somewhere on the Serverless spectrum, based on how well it exhibits these characteristics.

Wait? Serverless is Mainstream?

Yes, it’s definitely moving in that direction. It’s no longer viewed as an exclusive cutting-edge way of developing applications, with fresh-off-the-press videos showing how little effort is required to bootstrap and execute code, all for the low-low price of $0.00001. It still does those things, but many developers now see it as a “tool in their tool belt” to save time and money as they focus on building their application.

To illustrate the maturity path, let’s draw a comparison to Object-Oriented Programming (OOP).

When I was an undergraduate student in the 1990s, we learned Pascal and C as our two main programming languages. We also heard about Smalltalk (from 1972), with objects and messages, but never had a chance to write a program. It all seemed very abstract and I honestly didn’t understand the point of objects. It wasn’t until I wrote a bank teller simulation using the Simula language (from 1962) that it felt natural, and started to make sense.

As a graduate student (late 1990s), I was a regular at the OOPSLA conference, surrounded by academics and industry practitioners who were ahead of the game in Object Oriented Programming and Design. The exhibit halls were flooded with vendors selling their latest tool or technique. I brushed shoulders with people who are now legends, and saw the birth of Java and UML. It was all ground-breaking at the time, but not yet mainstream.

Fast forward to 2024 - The university students I interact with use TypeScript, Python, or Java, each with objects baked into the standard language. The students learn OO programming as the “normal” way of doing things, as if there was never any other way! They take these expectations into industry, and enthusiastically work in OO style. These days, the OOPSLA conference has rebranded into a much smaller format, mostly attended by academics. OOP has become mainstream - the normal way of doing things, rather than something special and buzz-worthy.

Serverless is not as far along the maturity curve as OOP, but it’s heading in the correct direction. It’s not a passing fad that’ll be gone in a few years. There are plenty of Serverless conferences generating buzz, but students are learning to deploy containers or functions-as-a-service to the cloud, without the cognitive burden of SSHing to a Linux host. They’re saving money by paying per request, rather than by instance-hour. It’s becoming a normal and desirable way to operate.

But, back to the topic - what exactly is Serverless? Or, more to the point, what are the characteristics of the services that are referred to as Serverless?

What Do the Experts Say?

The obvious place to start is by looking at what the industry or academic experts are saying. Here are some quotes:

From Wikipedia’s Serverless Computing topic…

Serverless computing is a cloud computing execution model in which the cloud provider allocates machine resources on demand, taking care of the servers on behalf of their customers. "Serverless" is a misnomer in the sense that servers are still used by cloud service providers to execute code for developers. However, developers of serverless applications are not concerned with capacity planning, configuration, management, maintenance, fault tolerance, or scaling of containers, virtual machines, or physical servers. When an app is not in use, there are no computing resources allocated to the app. Pricing is based on the actual amount of resources consumed by an application.

From the AWS Serverless page…

AWS offers technologies for running code, managing data, and integrating applications, all without managing servers. Serverless technologies feature automatic scaling, built-in high availability, and a pay-for-use billing model to increase agility and optimize costs. These technologies also eliminate infrastructure management tasks like capacity provisioning and patching, so you can focus on writing code that serves your customers.

From Pluralsight…

Serverless is one of those terms that’s used for all sorts of stuff — some legitimate and some marketing fluff. But, in essence, serverless is a design approach that lets you build and run entire applications without having to directly manage servers. 

From a UC Berkeley research paper…

Serverless cloud computing handles virtually all the system administration operations needed to make it easier for programmers to use the cloud. It provides an interface that greatly simplifies cloud programming, and represents an evolution that parallels the transition from assembly language to high-level programming languages

In summary, the goal of Serverless is to allow developers to focus on their application’s business logic, rather than on undifferentiated heavy lifting. Although each service does this differently, I’ve noticed some common characteristics:

• Avoiding the need for command-shell access (the administration is done for you).

• Avoiding the need to size/scale servers, or to be aware of them at all.

• Paying only when constructive work is being performed.

• Thinking about upgrading software

Let’s discuss in more detail.

Command Shell Access

With traditional infrastructure (cloud-based or on-premise), administrators have command-shell access to their servers, typically via SSH or RDP. This is a basic need when you’re responsible for security, upgrades, backups, or other server-side tasks. In contrast, the promise of Serverless is for cloud providers to handle this mundane work for you.

There’s clearly a benefit of having somebody else do the admin tasks, but for many developers this alleviates the need to learn how it’s done in the first place. Most application developers I’ve met prefer to focus their time and skills on developing business logic, without the distraction of skilling-up on IT responsibilities.

Which AWS services fit in this category? Clearly Amazon EC2 is not Serverless because the first thing you do is log into the command line or console. Container services, such as Amazon ECS are closer to being Serverless if you use ECS on AWS Fargate, but if you run ECS containers on EC2 you still have access to the underlying servers. Even with Fargate, it’s possible to run a command shell inside a container, similar to running docker exec on your local machine.

Perhaps the most famous Serverless offering from Amazon is AWS Lambda, providing time-limited general-purpose compute (running on Python, JavaScript, Java etc), without requiring access to the underlying servers. Instead, the developer selects a run-time environment (such as NodeJS), then uploads their code to the service. The service manages the complexity.

Looking further into the list of Serverless offerings, there are many services where command-shell access is not even possible, such as Amazon SQS, Amazon SNS, AWS Step Functions, Amazon EventBridge, and Amazon DynamoDB. All of these services have very specific purposes, but none of them provide access to general-purpose compute, nor a command shell.

Interestingly, we might question whether Amazon RDS Postgres meets this requirement, given that users are not permitted command-shell access to the underlying servers. This is true, but there are other reasons RDS doesn’t fit the Serverless model.

Awareness of Servers and Their Size

Another expectation of Serverless is that you don’t think about servers. That is, you don’t need to worry how large the servers are (RAM and CPU), how much disk space is attached, or how many servers you’ve been allocated. All of those details are managed by the cloud provider.

This is where RDS Postgres immediately fails the Serverless test, since the first thing you do is specify server size. You then worry if you have large enough servers, or whether you’ve over-provisioned and are paying too much. An ideal Serverless environment takes away these concerns, with Amazon DynamoDB going a long way in this direction (you just create a table, and don’t worry about how it’s stored/accessed).

Services such as AWS Lambda or AWS Fargate eliminate the need to worry about server sizing, as they automatically find a suitable server for your workload, allocating new servers whenever necessary to scale. However, at the function/container level, both Lambda and Fargate require you to specify the RAM and CPU needed for running your software. Tweaking these parameters can be time-consuming, given that RAM vs CPU trade-off significantly impacts software performance.

The other Serverless offerings mentioned earlier (SQS, SNS, Step Functions, EventBridge, DynamoDB) fit nicely into this model, since you never think about quantity or size of servers, nor about how much memory is allocated to your software. Of course, these services don’t provide general-purpose compute, which is how they differ from Lambda.

In terms of developer burden, a couple of edge cases are worth mentioning. First, a Lambda function can be impacted by cold-start latency where the service takes slightly longer than normal to scale the underlying server fleet on your behalf. Second, all services have limits/quotas to stop customers scaling up more than anticipated. Both these important scenarios place a burden on developers to be aware that servers are a real thing.

Elasticity and Pay-Per-Use

A significant benefit of Serverless technology, and of cloud computing in general, is paying for what you use. Capacity can be allocated and deallocated almost instantly, and you only pay while those resources are active. This is in stark contrast to the weeks (or months) required to install new equipment in your own data centre, paying the bulk of the cost before the hardware is even plugged in.

The interesting difference with Serverless is that you pay per request (or message/event, or state transition), rather than pre-allocating the resources and paying for them, even if they’re idle. For example, allocating an EC2 instance involves paying for every second the instance is dedicated for your use, even if the server isn’t doing meaningful work. In contrast, Lambda functions only execute when there’s meaningful work to be done, then immediately terminate rather than being idle. There are no charges for Lambda when there’s no work to do.

Anything in the Serverless world should therefore “scale to zero” and not charge you anything when idle. With this in mind, AWS Lambda, Step Functions, EventBridge, SNS, and SQS all fit into this model.

However, it’s debatable whether Amazon DynamoDB or AWS Fargate meet this requirement. For DynamoDB, you’ll always be charged for data storage, even in the absence of read/write requests. For Fargate, you can quickly scale up/down the number of instances, but you’ll still need to pay for idle instances that aren’t currently processing requests.

Upgrades

Although it’s not often discussed in the literature, there’s a cost of thinking about software upgrades. Serverless alleviates the need for upgrading or patching the underlying operating system, but that’s not always true for the run-time environment.

For example, with AWS Fargate, you’re still responsible for managing the Dockerfile, describing which libraries, frameworks, or applications run inside the container. If third-party software is used, there’s ongoing effort to upgrade packages to adopt new features, avoid EOLed versions, or to fix security vulnerabilities.

For AWS Lambda, there’s less effort involved, but with new versions of the runtime environment (e.g. NodeJS) being available every year, there’s ongoing effort to perform software upgrades.

On the other hand, services such as Step Functions, EventBridge, SQS and SNS maintain backward compatibility, not requiring the code or configuration to be updated on a periodic basis.

Event-Driven Architectures

One perspective I’ve seen is that Serverless is closely tied to Event-Driven Architectures (EDA). That is, your software consists of multiple collaborating services, each generating events of their own, and/or reacting to events from other services. In this model, there’s a clear benefit from only paying when there are events to process, or scaling-to-zero when the service is idle.

But, it wouldn’t be fair to say that EDA implies Serverless, or that Serverless implies EDA. In fact, if you have a predictable workload passing through your system, and you rarely scale-to-zero, it can be more efficient to use pre-allocated containers or VMs. In this approach, each service processes a high number of concurrent requests within the same container/VM, rather than starting up a new Function-as-a-Service environment for each request.

Serverless Compute

Another interesting perspective is that Serverless implies general-purpose compute, such as AWS Lambda, where you can run any type of code written in Java, JavaScript, Python, Golang, Rust, etc. In contrast, services such as DynamoDB, EventBridge, SNS or SQS don’t fit this definition of Serverless, as they have very specific modes of operation (fetching/storing data records, or propagating events).

I haven’t discovered the source of this thinking, but it’s likely because databases and message brokers were traditionally managed by dedicated teams (DBAs, or IT), rather than by application developers. From that perspective, we never thought much about servers in the first place.

And My Point Was?

After reading a lot of opinion, I learned that Serverless is not black or white… instead, it’s a colourful spectrum of characteristics making a service easier to use. By eliminating the undifferentiated heavy lifting, developers can focus more on building out their applications.

I’m not the first person to have these thoughts, and many have done a great job of documenting this topic. I encourage you to read/view:

• Mike Roberts / Thoughtworks - A detailed perspective of all the characteristics of Serverless, including the history, the pros, the cons, and forward-looking trends. A long document to read, but it’s worth it.

• Jeremy Daly - A fun presentation to watch, referring to a bunch of other experts who have equally interesting opinions.

• Ben Kehoe - Describes many of the edge cases of the Serverless definition, and includes the idea of it being a spectrum.

Regardless of what Serverless really means, and whether we’re using the word correctly, the characteristics we’ve discussed are super useful, especially if you’d rather focus on business logic. For this reason, I feel the characteristics of Serverless have a bright future, no matter what they’re called.