Ten years ago Berkeley researchers published a seminal paper about cloud computing and the future of the cloud. Earlier this week Berkeley’s RISE Lab researchers published a comprehensive technical report on serverless computing. They outline the enormous benefits and describe existing challenges for serverless computing. They also rebuild five large scale distributed applications with serverless functions and analyze performance and cost considerations.
In this post, we summarize their findings and go beyond their analysis to suggest how serverless can be used for many other applications as well.
What is serverless computing?
Berkeley sees the transition from the serverful cloud to serverless computing as analogous to the move from assembly language to higher level programming languages. Such programming languages automatically manage underlying resources like CPU registers and memory, dramatically improving developer productivity and making programming accessible to many more people. Similarly, serverless platforms manage underlying servers and networks so developers can focus on their application logic.
More specifically, they define serverless computing as having three traits:
- Decoupling of computation and storage
- Executing code without managing resource allocation
- Paying in proportions to resources used instead of resources allocated
They break down serverless services into two layers. The compute, or Function as a Service (FaaS), layer runs user code in the form of functions and auto-scales resources to meet variations in demand. The storage, or Backend as a Service (BaaS), layer stores data and should scale with compute needs for capacity and IOPS.
Berkeley also mentions Platform as a Service (PaaS) systems and container orchestration systems like Kubernetes. In their view, those sit on the spectrum between VM based serverful computing and serverless computing. However, PaaS or Kubernetes charge users by resources allocated and not by usage. While they also auto-scale resources, they do so much slower than serverless platforms, potentially resulting in significant cost overheads.
Attractiveness of serverless computing
Berkeley views serverless computing as beneficial to users (developers), customers (the ones paying the bill) and cloud providers alike. From a developer perspective, serverless lets developers focus on their application logic rather than boilerplate infrastructure. This improves productivity and democratizes scale, allowing less experienced developers to build scalable, production-grade applications.
Cloud users like serverless computing because novices can deploy functions without any understanding of the cloud infrastructure and because experts save development time and stay focused on problems unique to their application.
From a customer perspective, many applications benefit from significant cost savings due to the simple fact that they only pay for resources they actually consume. While still use case dependent today, they expect that as serverless evolves, these benefits will carry into other use cases as well.
Finally, cloud providers themselves can benefit from many optimization opportunities that arise from managing servers for the user. These include harvesting of unused capacity, statistical multiplexing of different workloads and better management of different hardware types and hardware generations.
Challenges with serverless computing
Berkeley examines five applications, which they have rebuilt using serverless functions:
- A real-time video encoding application
- A MapReduce application for data processing
- A linear algebra application for manipulating matrix data
- Machine learning training
- A database built out of functions
These five applications are data-centric in nature and already have equivalent large scale distributed alternatives, based on serverful architecture. Based on their experience with these applications, Berkeley researchers identify several challenges with existing serverless systems:
- Lack of storage that is fine-grained (can operate on small objects), fast enough and yet can automatically scale capacity and IOPS as quickly as serverless compute scales.
- Lack of fine-grained coordination mechanisms for passing and sharing data between functions.
- Networking overheads arising from code granularity. With a larger number of smaller functions, more messages are needed to coordinate. This was most noticeable in the shuffle phase of the MapReduce application.
- Unpredictable performance resulting from ‘cold starts’, or the time required by the underlying serverless platform to provision resources for function execution.
What serverless computing should become
Berkeley continue to list what they believe is required in order for serverless to mature as a computing paradigm:
Abstraction improvements automating resource management, like allocating just enough memory for each function or assigning hardware accelerators to functions without the user having to specify their requirements. Current abstraction is also missing ways for developers to describe the relationships between functions, specifically around data access patterns.
System improvements around fast, affordable and elastic storage and coordination/signaling services. A key improvement is the minimization of startup time, including resource provisioning and loading of code.
Network improvements to better manage data flow between functions.
Security improvements around isolation, fine-grained permission management and preventing leaks through analysis of network traffic patterns (think spectre/meltdown on a cloud scale).
Hardware management improvements bringing to serverless hardware accelerators like GPUs or the ability to run optimized code for specific versions of CPU.
Our goal at Binaris is to turn serverless into the standard for building cloud applications. The advantages brought on by a fully managed infrastructure and coding in fine grain functions, revolutionize the way developers build code. Serverless makes expert developers more productive and enables novice developers to build at cloud scale.
Binaris offers its own serverless service (running on top of AWS) which is optimized for fine grain functions. It offers a number of capabilities lacking from existing platforms:
- Fast invocation and no cold starts. By invoking functions in single digit millisecond latency, we enable many interactive use cases for serverless, like API serving, data streaming, gaming and more.
- 1ms billing enables IO-bound workloads, like interactive web and mobile backends. We only charge for actual CPU time used and not while your functions wait for IO or for other functions invoked synchronously.
- Invocation flow management to track and control complex invocation patterns like chaining, fan-out, fan-in and recursion, and so applications can easily recover if a single function or multiple functions fail.
- Cache management to allow functions to manage local data in a stateless environment. This reduces storage and network costs and performance penalties.
The end result allows developers to break down their applications into smaller functions, and compose functions into full applications. Each function becomes easier to code, debug, deploy and test at cloud scale. Functions also become easier to share and reuse within and between dev teams.
We full heartedly join Berkeley researchers in their assessment that
By providing a simplified programming environment, serverless computing makes the cloud much easier to use, thereby attracting more people who can and will use it.
We predict that serverless use will skyrocket.
We welcome you to join us on our road to serverless applications. Give Binaris a try.