Lobsters (43)In 2011, during the Google Plus roll out, Googler Steve Yegge criticizes the company's struggle to build platforms. In that post, he makes a contrast with Amazon. He describes Jeff Bezos' Big Mandate from ~2002. The key points of the mandate:
1. All teams will henceforth expose their data and functionality through service interfaces.
2. Teams must communicate with each other through these interfaces.
3. There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team's data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
The post went superviral. It helped kick the SOA revolution into high-gear. HTTP interfaces were stood up everywhere.
I was on a team that got swept up in the ecstasy. I guess a lot of us forgot that the architectures being described were for the two biggest internet companies on the planet.
Zooming out, there are two reasons one might justify an HTTP interface in front of their database:
- Security and permissions: You want tight controls over who can see what.
- Abstractions: You want to be able to change your underlying data model without affecting your consumers.
These both matter a lot when you have external consumers. And at a company like Amazon, another team - perhaps even located in another city - can feel a lot like an external consumer.
But for many teams, these goals are hardly worth the cost. The API provisioner has to build and maintain a translator from HTTP query params to SQL, one that will never come close to approximating SQL in power. And the consumer is hampered by a poor (and often poorly documented) query interface in front of the data they seek.
Does a high-growth startup have to worry about proper service abstraction? Hardly.
Retool is a company that understands this very well. The drag-and-drop interface for React components is compelling. But the ability to populate those components with SQL queries is the secret sauce. Building an internal REST API that powers your team's dashboards is maybe the least value-differentiated thing your engineers can work on.
That's not to say that as a startup scales, there isn't a point where the trade-off is worth it. It's just further along than most startups think.
SQL is a lingua franca. Reading directly with SQL queries means you eliminate a whole class of boilerplate API code, on both client and server.
For writes, on the other hand, it makes sense to go through an API. A lot of people have validation logic in their code. And writes are probably where you'll want to introduce more controls first, to prevent the team from inserting bad data or making mistakes.
Fortunately, writes are far easier to both build and use.
Hence the mantra: "Read from databases, write to APIs."