"My" service is a proxy between certain modules of a large project. At first glance, you can study it in one evening and get down to more important things. But starting to work, I realized that I was mistaken. The service was written six months ago in a couple of weeks with the task of testing MVP. All this time he refused to work: he lost events and data, or rewrote them. The project was thrown from team to team, because no one wanted to do it, not even its creators. Now it became clear why they were looking for a separate programmer for it.
"My" service is an example of poor architecture and inherently incorrect design. We all understand that you cannot do this. But why not, what consequences it leads to and how to try to fix everything, I will tell you.
How bad architecture gets in the way
Typical story:
- make MVP;
- test hypotheses on it;
- , MVP;
- ...;
- PROFIT.
But this cannot be done (which we all understand).
When systems are built in a hurry, the only way to keep releasing new versions of a product is to "bloat" the staff. Initially, the developers show productivity close to 100%, but when the initially "raw" product is overgrown with features and dependencies, it takes longer and longer to figure it out.
With each new version, developer productivity drops. Nobody thinks about code cleanliness, design and architecture. As a result, the price of a line of code can increase 40 times.
These processes can be clearly seen in the graphs from Robert Martin. Despite the fact that the development staff is increasing from version to version, the dynamics of product growth is only slowing down. Costs are growing, revenue is falling, which is already leading to a reduction in staff.
Clean architecture challenge
It doesn't matter to business how the application is designed and written. It is important for business that the product behaves the way users want it and be profitable. But sometimes (not sometimes, but often) the business changes its solutions and requirements. With poor structure, it is difficult to adapt to new requirements, change products, and add new functionality.
A well-designed system is easier to match with the desired behavior. Again, Robert Martin believes that behavior is secondary and can always be corrected if the system is well designed.
Clean architecture promotes communication between the layers in the project, where the center is business logic with all its entities that deal with applied problems.
- All outer layers are adapters for communication with the outside world.
- Elements of the outside world should not penetrate the central part of the project.
Business logic doesn't care who it is: a desktop application, a web server, or a microcontroller. It shouldn't depend on the "label". She must perform specific tasks. Everything else is details, for example, databases or desktop.
With a clean architecture, we get an independent system. For example, it is independent of the database or framework version. We can replace the desktop application for the needs of the server without changing the internal component of the business logic. This is what business logic is valued for.
A clean architecture reduces the cognitive complexity of the project, the support costs, and simplifies the development and further maintenance of programmers.
How to identify "bad" architecture
There is no concept of "bad" architecture in programming. There are criteria for poor architecture: stiffness, immobility, toughness and excessive repeatability. For example, these are the criteria I used to understand that the architecture of my microservice is bad.
Rigidity . It is the inability of the system to react to even small changes. When it becomes difficult to change parts of a project without damaging the entire system, the system is rigid. For example, when one structure is used in several layers of a project at once, then its small change creates problems in the entire project at once.
The problem is cured by converting on each layer. When each layer operates only their objects, which were obtained by "converting" the external object, the layers become fully independent
Immobility... When the system was built with poor separation (or absence) into reusable modules. Fixed systems are hard to refactor.
For example, when information about databases enters the area of business logic, replacing the database with another will lead to refactoring of all business logic.
Viscosity . When the division of responsibilities between packages leads to unnecessary centralization. Interestingly, what happens the other way around, when viscosity leads to decentralization - everything is divided into too small packages. In Go, this can lead to circular imports. For example, this happens when adapter packets begin to receive extra logic.
Excessive repeatability... In Go, the phrase "Small copy is better than small dependency" is popular. But this does not lead to the fact that there are fewer dependencies - it just becomes more copies. I often see copies of code from other packages in different Go packages.
For example, Robert Martin writes in his book "Clean Architecture" that in the past Google required to reuse any strings it could, and allocate them into separate libraries. This caused changing 2-3 lines of a small service to affect all other related services. The company is still fixing problems with this approach.
Desire to refactor... This is a bonus criterion for bad architecture. But there are nuances. No matter how badly the project was written, by you or not, you should never rewrite it from scratch, this will only create additional problems. Do iterative refactoring.
How to design relatively correctly
“My” proxy service lived for six months and all this time did not fulfill its tasks. How did he live for so long?
When a business tests a product and it shows ineffectiveness, it is abandoned or destroyed. This is normal. When the MVP is tested and it turns out to be efficient, then it lives on. But usually MVP is not rewritten and it lives on "as is", overgrown with code and functionality. Therefore, "zombie products" that were created for MVPs are a common practice.
When I found out how my proxy service was not working, the team decided to rewrite it. This business was assigned to me and a colleague and allocated two weeks: there is little business logic, the service is small. This was another mistake.
The service began to be rewritten entirely. When they cut, rewrote parts of the code and uploaded them to the test environment, part of the platform crashed. It turned out that the service had a lot of undocumented business logic that no one knew about. My colleague and I failed, but this is an error in the service logic.
We decided to approach refactoring from the other side:
- roll back to the previous version;
- the code is not rewritten;
- we divide the code into parts - packages;
- each package is wrapped in a separate interface.
We didn't understand what the service was doing, because no one understood it. Therefore, "sawing" the service in parts and figuring out what each part is responsible for is the only option.
After that, it became possible to refactor each package separately. We could fix each part of the service separately and / or implement it in other parts of the project. At the same time, work on the service continues to this day.
It turned out like this.
How would we write a similar service if we had designed it “well” from the start? Let me show you with the example of a small microservice that registers and authorizes a user.
Introductory
We need: the core of the system, an entity that defines and executes business logic by manipulating external modules.
type Core struct {
userRepo UserRepo
sessionRepo SessionRepo
hashing Hasher
auth Auth
}
Next, you need two contracts that will allow you to use the repo layer. The first contract provides us with an interface. With its help, we will communicate with the database layer that stores information about users.
// UserRepo interface for user data repository.
type UserRepo interface {
// CreateUser adds to the new user in repository.
// This method is also required to create a notifying hoard.
// Errors: ErrEmailExist, ErrUsernameExist, unknown.
CreateUser(context.Context, User, TaskNotification) (UserID, error)
// UpdatePassword changes password.
// Resets all codes to reset the password.
// Errors: unknown.
UpdatePassword(context.Context, UserID, []byte) error
// UserByID returning user info by id.
// Errors: ErrNotFound, unknown.
UserByID(context.Context, UserID) (*User, error)
// UserByEmail returning user info by email.
// Errors: ErrNotFound, unknown.
UserByEmail(context.Context, string) (*User, error)
// UserByUsername returning user info by id.
// Errors: ErrNotFound, unknown.
UserByUsername(context.Context, string) (*User, error)
}
The second contract "communicates" with the layer that stores information about user sessions.
// SessionRepo interface for session data repository.
type SessionRepo interface {
// SaveSession saves the new user Session in a database.
// Errors: unknown.
SaveSession(context.Context, UserID, TokenID, Origin) error
// Session returns user Session.
// Errors: ErrNotFound, unknown.
SessionByTokenID(context.Context, TokenID) (*Session, error)
// UserByAuthToken returning user info by authToken.
// Errors: ErrNotFound, unknown.
UserByTokenID(context.Context, TokenID) (*User, error)
// DeleteSession removes user Session.
// Errors: unknown.
DeleteSession(context.Context, TokenID) error
}
Now you need an interface for working with passwords, hashing and comparing them. And also the latest interface for working with authorization tokens, which will allow them to be generated and also identified.
// Hasher module responsible for working with passwords.
type Hasher interface {
// Password returns the hashed version of the password.
// Errors: unknown.
Password(password string) ([]byte, error)
// Compare compares two passwords for matches.
Compare(hashedPassword []byte, password []byte) error
}
// Auth module is responsible for working with authorization tokens.
type Auth interface {
// Token generates an authorization auth with a specified lifetime,
// and can also use the UserID if necessary.
// Errors: unknown.
Token(expired time.Duration) (AuthToken, TokenID, error)
// Parse and validates the auth and checks that it's expired.
// Errors: ErrInvalidToken, ErrExpiredToken, unknown.
Parse(token AuthToken) (TokenID, error)
}
Let's start writing the logic itself. The main question is what do we want from the business logic of the application?
- User registration.
- Checking mail and nickname.
- Authorization.
Checks
Let's start with simple methods - checking email or nickname. Our UserRepo has no methods to check. But we will not add them, we can check whether this or that data is busy by requesting the user for this data.
// VerificationEmail for implemented UserApp.
func (a *Application) VerificationEmail(ctx context.Context, email string) error {
_, err := a.userRepo.UserByEmail(ctx, email)
switch {
case errors.Is(err, ErrNotFound):
return nil
case err == nil:
return ErrEmailExist
default:
return err
}
}
// VerificationUsername for implemented UserApp.
func (a *Application) VerificationUsername(ctx context.Context, username string) error {
_, err := a.userRepo.UserByUsername(ctx, username)
switch {
case errors.Is(err, ErrNotFound):
return nil
case err == nil:
return ErrUsernameExist
default:
return err
}
}
There are two nuances here.
Why
ErrNotFounddoes the check pass for an error ? The implementation of business logic should not depend on SQL or any other database, so it sql.ErrNoRowsshould be converted into the error that is convenient for our business logic.
We also raise the error of the business logic layer with the API layer, and the error code or something else should be resolved at the API level. Business logic should not depend on the communication protocol with the client and make decisions based on this.
Registration and authorization
// CreateUser for implemented UserApp.
func (a *Application) CreateUser(ctx context.Context, email, username, password string, origin Origin) (*User, AuthToken, error) {
passHash, err := a.password.Password(password)
if err != nil {
return nil, "", err
}
email = strings.ToLower(email)
newUser := User{
Email: email,
Name: username,
PassHash: passHash,
}
_, err = a.userRepo.CreateUser(ctx, newUser)
if err != nil {
return nil, "", err
}
return a.Login(ctx, email, password, origin)
}
// Login for implemented UserApp.
func (a *Application) Login(ctx context.Context, email, password string, origin Origin) (*User, AuthToken, error) {
email = strings.ToLower(email)
user, err := a.userRepo.UserByEmail(ctx, email)
if err != nil {
return nil, "", err
}
if err := a.password.Compare(user.PassHash, []byte(password)); err != nil {
return nil, "", err
}
token, tokenID, err := a.auth.Token(TokenExpire)
if err != nil {
return nil, "", err
}
err = a.sessionRepo.SaveSession(ctx, user.ID, tokenID, origin)
if err != nil {
return nil, "", err
}
return user, token, nil
}
It is simple, imperative code that is easy to read and maintain. You can start writing this code right away when designing. It doesn't matter to which database we add the user, which protocol we choose to communicate with clients, or how passwords are hashed. Business logic is not interested in all these layers, it is only important for it to perform the tasks of its application area.
Simple hashing layer
What does it mean? All external non-layers should not make decisions on tasks that are related to the application area. They perform a specific and simple task that our business logic requires. For example, let's take a layer for hashing passwords.
// Package hasher contains methods for hashing and comparing passwords.
package hasher
import (
"errors"
"github.com/zergslaw/boilerplate/internal/app"
"golang.org/x/crypto/bcrypt"
)
type (
// Hasher is an implements app.Hasher.
// Responsible for working passwords, hashing and compare.
Hasher struct {
cost int
}
)
// New creates and returns new app.Hasher.
func New(cost int) app.Hasher {
return &Hasher{cost: cost}
}
// Hashing need for implements app.Hasher.
func (h *Hasher) Password(password string) ([]byte, error) {
return bcrypt.GenerateFromPassword([]byte(password), h.cost)
}
// Compare need for implements app.Hasher.
func (h *Hasher) Compare(hashedPassword []byte, password []byte) error {
err := bcrypt.CompareHashAndPassword(hashedPassword, password)
switch {
case errors.Is(err, bcrypt.ErrMismatchedHashAndPassword):
return app.ErrNotValidPassword
case err != nil:
return err
}
return nil
}
This is some simple layer for performing password hashing and comparison tasks. It's all. He is thin and simple and does not know anything else. And it shouldn't.
Repo
Let's think about the storage interaction layer.
Let's declare the implementation and indicate which interfaces it should implement.
var _ app.SessionRepo = &Repo{}
var _ app.UserRepo = &Repo{}
// Repo is an implements app.UserRepo.
// Responsible for working with database.
type Repo struct {
db *sqlx.DB
}
// New creates and returns new app.UserRepo.
func New(repo *sqlx.DB) *Repo {
return &Repo{db: repo}
}
It will be possible to let the reader of the code understand which contracts are implemented by the layer, as well as take into account the tasks set for our Repo.
Let's get down to implementation. In order not to stretch the article, I will give only a part of the methods.
// CreateUser need for implements app.UserRepo.
func (repo *Repo) CreateUser(ctx context.Context, newUser app.User, task app.TaskNotification) (userID app.UserID, err error) {
const query = `INSERT INTO users (username, email, pass_hash) VALUES ($1, $2, $3) RETURNING id`
hash := pgtype.Bytea{
Bytes: newUser.PassHash,
Status: pgtype.Present,
}
err = repo.db.QueryRowxContext(ctx, query, newUser.Name, newUser.Email, hash).Scan(&userID)
if err != nil {
return 0, fmt.Errorf("create user: %w", err)
}
return userID, nil
}
// UserByUsername need for implements app.UserRepo.
func (repo *Repo) UserByUsername(ctx context.Context, username string) (user *app.User, err error) {
const query = `SELECT * FROM users WHERE username = $1`
u := &userDBFormat{}
err = repo.db.GetContext(ctx, u, query, username)
if err != nil {
return nil, err
}
return u.toAppFormat(), nil
}
The Repo layer has simple and basic methods. They do not know how to do anything other than "Save, submit, update, delete, find." The task of the layer is only to be a convenient provider of data to any database that our project needs.
API
There is still an API layer for interacting with the client.
It is required to transfer data from the client to the business logic, return the results back, and fully satisfy all the HTTP needs - convert application errors.
func (api *api) handler(w http.ResponseWriter, r *http.Request) {
params := &arg{}
err := json.NewDecoder(r.Body).Decode(params)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
origin := orifinFromReq(r)
res, err := api.app.CreateUser(
r.Context(),
params.Email,
params.Username,
params.Password,
request,
)
switch {
case errors.Is(err, app.ErrNotFound):
http.Error(w, app.ErrNotFound.Error(), http.StatusNotFound)
case errors.Is(err, app.ErrChtoto):
http.Error(w, app.ErrChtoto.Error(), http.StatusTeapot)
case err == nil:
json.NewEncoder(w).Encode(res)
default:
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
}
}
On this, his tasks end: he brought the data, got the result, converted it into a format convenient for HTTP.
What is clean architecture really needed for?
What is it all for? Why implement certain architectural solutions? Not for "cleanliness" of the code, but for testability. We need the ability to conveniently, simply and easily test our own code.
For example, code like this is bad :
func (api *api) handler(w http.ResponseWriter, r *http.Request) {
params := &arg{}
err := json.NewDecoder(r.Body).Decode(params)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
rows, err := api.db.QueryContext(r.Context(), "sql query", params.Param)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
var arrayRes []val
for rows.Next() {
value := val{}
err := rows.Scan(&value)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
arrayRes = append(arrayRes, value)
}
//
err = json.NewEncoder(w).Encode(arrayRes)
w.WriteHeader(http.StatusOK)
}
Note: forgot to point out that this code is bad. This could be misleading if you read before the update. Sorry about that.
The ability to test your code without major problems is the main benefit of a clean architecture.
We can test all business logic by abstracting from the database, server, protocol. It is only important for us to perform the applied tasks of our application. Now, following certain and simple rules, we can easily expand and change our code painlessly.
Any product has business logic. A good architecture helps, for example, to pack business logic into one package, the task of which is to operate with external modules to perform application tasks.
But clean architecture isn't always good. Sometimes it can turn into evil, bring unnecessary complexity. If you try to write perfectly right away, we will waste precious time and let the project down. You don't have to write perfect - write well based on your business goals.
, Golang Live 2020 14 17 . — 14 , — , .