While we continue our sale for the most discerning tastes, we turn your attention to another theme of our creative search: Event-Driven Architecture (EDA). Beautiful flowcharts and a story about how this innovative paradigm helps in the development of web applications await you under the cut.
This article will look at some of the challenges driving innovation in modern web development. Next, we'll dive into Event-Driven Architecture (EDA) to address these challenges by redefining server architecture.
Web applications have come a long way since the days when static HTML content was served from a server. Today, web applications have become much more complex, they use a variety of frameworks, data centers and technologies. In the past couple of years, two trends can be noted that determine the development of the IT market:
- Transferring applications to the cloud ;
- Implementation of microservice architecture.
These ideas largely determine how software is designed and built today. We can say that today we are no longer building applications, but platforms. Applications no longer take up shared computing space. Instead, they have to exchange information with each other using lightweight communication protocols such as REST APIs or remote procedure calls (RPC). This model has led to great products like Facebook, Netflix, Uber, and more.
This article will explore some of the challenges driving innovation in modern web development. Next, we'll dive into Event-Driven Architecture (EDA), which addresses these issues by redefining server architecture.
Current problems of the modern web
Any web technology must cope with the challenges that modern multi-user asynchronous applications must meet, designed to run smoothly:
Availability
Now we work not with one application, but with many - dozens or even hundreds - of related services, and each of them must solve their problems around the clock, seven days a week. How can this be achieved? Most often, the service is horizontally scaled to multiple instances that can be distributed across multiple data centers to ensure high availability. All requests for this particular service are routed and evenly distributed across all instances. Some deployment tools provide self-healing capabilities, so if one instance fails, another is created to take its place.
Scalability
Scalability is in many ways akin to availability. The essence of accessibility is to ensure that at least one instance of the service is up and running, ready to service incoming requests. Scalability, in turn, is primarily related to performance. If an application is overloaded, new instances of that application are created to accommodate the increased number of requests. But scaling applications vertically is not a trivial task, especially when it comes to stateful applications .
One source of truth
Before the advent of microservices, this was a fairly simple task. All data was located in one location, as a rule, it was one or another relational database. However, when multiple services share a database, problems such as dependencies between different teams regarding schema changes or performance issues can be created. Usually this problem was solved by allocating its own database for each service. A distributed source of truth is very helpful in maintaining a clean architecture, but in such a situation you have to deal with distributed transactions and the complexity of supporting multiple databases.
Synchronicity
In a typical request-response scenario, the client waits for the server to respond; it blocks all actions until it receives a response, or until the specified delay expires. If you take this behavior and implement it into a microservice architecture using call chains that run through the entire system, you can easily end up in so-called "microservice hell." It all starts with calling just one service, let's call it "service A". But then service A has to call service B, and the fun begins. The problem with this behavior is this: if the service itself is associated with blocked resources (for example, a thread is hanging), then the delays grow exponentially. If we allow a delay of 500 ms per service, and there are five service calls in the chain, then the first service will need a delay of 2500 ms (2.5 seconds), and the last one - 500 ms.
The challenges of the modern web
Introduction to event-driven architecture
Event-Driven Architecture (EDA) is a software architecture paradigm that facilitates the generation, discovery, consumption, and response of events.
In classic three-tier applications, the core of the system is the database. In EDA, the focus shifts to events and how they filter through the system. This shift in focus allows us to completely change the way we design applications and solve the aforementioned problems.
Before looking at exactly how this is done in EDA, let's look at what an "event" is. An event is an action that initiates either some notification or a change in the state of the application. The light turned on (notification), the thermostat turned off the heating system (notification), the user changed their address (change of state), one of your friends changed their phone number (change of state). These are all events, but it is not yet a fact that we should add them to an event-driven solution. It is assumed that only business-relevant events are added to the architecture. The event “user places an order” is important from a business point of view, but “user eats an ordered pizza or lunch” is not.
If you think about some events, then about some it is immediately clear that they are important for the business, and about some - not. Especially about those that occur in response to other events. A technique called " event assault " is used to identify events passing through the system . Participants in application development (from programmers to business logic developers and subject matter experts) are convened and jointly map all business processes, presenting them in the form of specific events. When such a map is ready, the result of the work is formulated in the form of requirements that must be met when developing applications.
An example of an application for booking described by the event assault method
Having identified the events of interest to us and having decided how to identify them, let's look at how this paradigm can solve the typical problems mentioned above.
The flow of events is unidirectional: from producer to consumer. Compare this situation with a REST call. The event producer does not in principle expect a response from the consumer, whereas in the case of a REST call, there will always be a response. No response means no need to block the execution of the code until something else happens. In this case, events become asynchronous in nature, which completely eliminates the risk of getting bogged down in delays.
Events happen as a result of an action, so there is no target system; service A cannot be said to trigger events on service B; but we can say that service B is interested in the events generated by service A. True, there may be other "interested parties" in this system, for example, services C or D.
How can we make sure that an event initiated in a certain system reaches all "Interested" services? Typically, such systems are solved using message brokers. A broker is simply an application that acts as an intermediary between the event emitter (the application that generated the event) and the event consumer. Thus, applications can be neatly detached from each other, taking care of the accessibility issue, which was discussed above in this post. If the application is not available at the moment, then, returning online, it will begin to consume events and process them, making up for all those events that have happened during the period while it was unavailable.
What about data warehouse? Can events be stored in a database, or is something else required instead of a database? Certainly, events can be stored in databases, but in this case their "event" essence is lost. Once an event has occurred, we can no longer correct it, therefore events are inherently immutable. Databases, in turn ... are changeable, after entering the data into the database, they can be changed.
Better to store events in event logs . Event logs are nothing more than centralizeddata warehouse, where each event is recorded as a sequence of unchangeable records, the so-called "log". A log can be compared to a log, where each new event is added to the end of the list. You can always recreate the most current state by replaying all the log events from the beginning to the present.
So, we have covered all issues except scalability. Event-driven services are always designed to be deployed across multiple instances. Since the state as such is stored in the event log, the service itself will be stateless, which allows for surgically accurate scaling of any service of interest.
The only exception to this principle is services designed to create materialized views.... In essence, a materialized view is a state that describes an event log at a specific point in time. This approach is used to make it easier to query data. Returning to the issue of scaling, a materialized view is simply an aggregate view of events that resembles a table; but where do we store these tables? Most often, you see such aggregations in memory, and at the same time our service automatically turns into a stateful one. A quick and easy solution is to provide a local database for every service that creates materialized views. This way the state is stored in the database and the service runs stateless.
Although event-driven architecture has been around for over 15 years, it has only recently gained serious popularity, and this is no coincidence. Most companies are going through a stage of " digital transformation " with wild demands. Because of the complexity of these requirements, engineers have to master new approaches to software design, implying, in particular, weakening the coupling of services with each other and reducing the cost of maintaining services. EDA is one possible solution to these problems, but not the only one. Also, do not expect that all problems will be solved, you just have to switch to EDA. Some features may still require robust old-fashioned REST APIs or storing information in a database. Choose the one that works best for you and design it properly!