The serverless revolution deserves more than just a serverless database
I was recently on a Future of Databases panel and asked what technological shifts spurred the new wave of serverless databases. I believe this shift has been largely sociological: a desire to make problems go away and instead focus on product. However, there’s a looming technical challenge we need to solve if the serverless revolution is to reach its goals.
This looming challenge is end-to-end data management. The serverless movement has eliminated so many problems but it has left application developers with the hardest one of all: managing distributed state.
Building products not scaffolding
The “serverless revolution” is a marked shift in how developers build their apps, driven heavily by platforms like Netlify and Vercel. Once upon a time, building a scalable, performant web application meant deploying web servers, distributing static assets to a CDN, dealing with request routing, managing release process. These issues are largely a thing of the past for developers in the serverless ecosystem. Developers build apps in a framework like Next.js, put up a PR on GitHub, test out automatic previews while their code is reviewed, merge their changes, and then their app gets automatically deployed around the world. It’s a tremendously convenient workflow that you should try out if you haven’t already. CDNs and Platforms as a Service aren't new technologies but what is new is how developers think about their applications. It wasn’t long ago that the vast majority developers wanted to own their stack. In the days of the LAMP stack, it was a rite of passage to firewall a Linux machine, configure MySQL, set up replication, etc. If a company outsourced these to a third party, that was usually considered a sign of low standards, that it wasn’t worth it to invest in a faster, cheaper, secure, and reliable self-hosted backend. (Even I was responsible for migrating Dropbox off of S3, but that’s a story for another time.)
These days using a platform for hosting your web app is a sign of high standards. It’s extremely difficult for a developer to replicate the performance, scale, and reliability of a third-party serverless backend, let alone approach the productivity gain from not having to build their infrastructure. Use of third-party platforms is also often a sign of better priorities. I can’t remember the last time I heard people passionately arguing about Red Hat vs Debian. These days most application developers just want to focus their time on building actual products that benefit actual end users.
Making problems go away
If we were to differentiate “serverless” (Netlify/Vercel) from “Infrastructure as a Service” (AWS/Google Cloud), I think the key is that serverless isn’t just about providing tools that simplify infrastructure, it’s about making problems go away entirely. EC2, S3, ECR, ELB and Route53 are all amazing technologies, but they generally require a lot of work to combine into an actual solution to a problem. AWS hasn’t made infra teams go away, they just build infra on AWS now. Modern serverless platforms make problems go away, to the point that there’s a new generation of developers who don’t even know that some of these problems existed.
Convex is the backend application platform with everything you need to build your project. Cloud functions, a database, file storage, scheduling, search, and realtime updates fit together seamlessly.
The missing link is data
Serverless platforms have improved life for application developers… up until they have to deal with managing data. Reliable hosted databases exist now, which eliminate a lot of operational concerns, but this has never been the real challenge when dealing with databases. The real challenge is a distributed systems one: keeping data on one host (the browser) consistent with another (the database). Gone is postgresql.conf; in is WebSockets, polling, API servers, SWR, caching, server components.
This challenge is perhaps best illustrated via this 2-minute demo:
The missing link in the serverless vision, the vision of truly making problems go away, is data. There are a lot of companies out there making databases more convenient to run, but not that much more convenient to use. These platforms are useful but they don’t eliminate this class of data management problems. The serverless revolution deserves a counterpart that solves the problem of data management and we believe that solution is Convex.
Convex is the backend application platform with everything you need to build your project. Cloud functions, a database, file storage, scheduling, search, and realtime updates fit together seamlessly.