Design Microservices Architectures The Right Way

Created on: April 30, 2023Modified on: September 16, 2023

Source

This is a great video. Below are my notes on it.

Intro

Could you change this URL from https://foo.com/latest.bar.js to https://foo.com/1.5.3/bar.js?

-> Sorry, that would take weeks. We don't have the resources to do that.

The above is not what we want.

Great architecture

No so great architecture

We trade near term velocity for future paralysis.

Misconceptions

1. Microservices enable our teams to choose the best programming languages and frameworks for their tasks.

Reality: this is very expensive. Look at Google, they have 20-30 thousand engineers and they use 8 programming languages. So a good metric is 1 programming language for every 3000 engineers.

2. Code generation is evil

Reality: it's just a technique. What's important is to create a defined schema that is 100% trusted.

3. The event log must be the source of truth

Reality: events are critical parts of an interface. But it's okay for services to be the system of record for their resources. Which means we receive a REST request to create a user, then we create the user, but then we guarantee these messages are going to end up in the event stream.

4. Developers can maintain no more than 3 services each

Reality: wrong metric, use automation.

Architecture of Flow.io

API definition

Clojure
{:user {:description "Represents a single user in the system"
        :fields      [{:name "id"
                       :type :string}
                      {:name        "email"
                       :type        :string
                       :required    false
                       :annotations [:personal_data]}
                      {:name        "name"
                       :type        :name
                       :annotations [:personal_data]}
                      {:name    "status"
                       :type    :user-status
                       :default :active}]}}

{:user-form {:fields [{:name        "email"
                       :type        :string
                       :required    false
                       :annotations [:personal_data]}
                      {:name        "password"
                       :type        :string
                       :required    false
                       :annotations [:personal_data]}
                      {:name        "name"
                       :type        :name-form
                       :required    false
                       :annotations [:personal_data]}]}}

The resource is :user and to create a resource you use a form which by convention is :user-form.

Clojure
{:resources {"io.flow.common.v0.models.user" {:operations [{:method :get
                                                            :description "Returns information about a specific user."
                                                            :path "/:id"
                                                            :responses {200 {:type "io.flow.common.v0.models.user"}
                                                                        401 {:type "unit"}
                                                                        403 {:type "unit"}}}
                                                           {:method :post
                                                            :description "Create a new user. Note that new users will be created with a status of pending and will not be able to ... Flow team."
                                                            :body {:type :user-form}
                                                            :responses {201 {:type "io.flow.common.v0.models.user"}
                                                                        401 {:type "unit"}
                                                                        422 {:type "io.flow.error.v0.models.generic_error"}}}]}}}

APIs are first-class, which means they're in their own dedicated git repository. To modify an API you open the repository and make a PR. When you create the PR the CI will validate the definition. This also runs linter which validates things as:

The CI should verify if there are breaking changes:

Code generation

Use a CLI tool to generate the scaffolding: apibuilder update --app user

The goal is not to make things "possible", the goal is to make the code generation so nice that a developer will love using it.

Database Architecture

Each micro service has its own database. Others connect to it using API and Events.

Use a single CLI called dev which is what developers will use.

They have a hash_code column on the database that is used to compare if something changed or not.

The code generation is smart enough to know that the email field has an index so it generates a findByEmail method.

Continuous Delivery

Events

Producers

Consumers

Summary: Critical Decisions