EDA event-driven architecture and EventBridge

Today's more successful companies have realized that in order to maximize operational efficiency and customer experience, they must closely integrate business and technology initiatives. Operational events or changes in business situations are the focus of many companies today. These changes can provide practical and useful information to business leaders, and the purpose of architectural design is to gain insights from information such as customer contacts, transactions, operations, etc. The two complement each other. Traditional technologies have always had many limitations on the speed at which companies can gain insights from events, such as batch ETL (extract, transform, load) used to record, collect and process such events.

Event-driven architecture (EDA) is in the ascendant. As a serverless application concept, it has a profound impact on cloud native architecture. When we discuss a specific architecture, the first thing we consider is whether its development is technologically advanced. Here, we will start with the familiar MVC architecture and SOA architecture, and talk about the history and development trends of the message event field.

Development trends in the news event field

As early as 2018, Gartner's evaluation report listed the Event-Driven Model as one of the top 10 strategic technology trends, and event-driven architecture (EDA) will become the mainstream of microservices in the future, and made the following assertions:

By 2022, software models for event notifications will become solutions for more than 60% of new digital businesses;
By 2022, more than 50% of business organizations will participate in an ecosystem of event-driven digital business services;
George Santayana mentioned in The Life of Reason that those who fail to learn History are doomed to repeat it. Let’s learn from history and see why architecture evolved to event-driven.

There is no good or bad architecture itself. It is a set of technical decisions that determines all functional development of subsequent projects (framework, coding standards, documentation, processes...). Here we will talk about why certain frameworks are introduced and what problems this framework solves in software development.

Monolithic architecture: In a single-node service, all modules of a monolithic application are encapsulated in a single process and communicate through the same stack call. This model easily leads to unclear structures and relationships, making it difficult to change and refactor the system. It's like an opaque, sticky, fragile, and rigid Big Ball of Mud!
Layered architecture: In the classic layered architecture, layers are used in a rather cautious way. That is, a layer can only know the data of the layer below it. In subsequent practical applications, it is more common for a layer to access any layer below it. The layered architecture solves the logical separation problem of the monolithic architecture. Each layer can be replaced by an equivalent, and the layer distinction is more standardized. At the same time, a layer can be used by several different/higher-level layers. Of course, the layer also has obvious disadvantages. The layer cannot encapsulate everything. For example, a field added to the UI may also need to be added to the DB, and the extra layers will seriously damage the system performance.
MVC architecture: The reason for the emergence of MVC architecture is actually very simple. With the increasing complexity of business systems, the so-called "full-stack engineers" are no longer applicable to most scenarios. In order to reduce the complexity of front-end and back-end integration, the MVC architecture began to be promoted. Among them, Model represents business logic, View represents the view layer, such as a small component of the front-end UI, and Controller provides coordination between View and Model, such as converting a user's operation into business logic. There are many extended architectures here, such as Model-View-Presenter, Model-View-Presenter-ViewModel, Resource-Method-Representation, Action-Domain-Responder.
EBI architecture: Entity, Boundary (interface), Interactor (control). EBI architecture regards the system boundary as a complete connection, not just a view, controller or interface. EBI's entity represents the actual entity that holds data and ends related behaviors, which is very similar to Alibaba Cloud's POP API. EBI is mainly a backend concept, and it complements MVC.
Onion architecture: Onion architecture is a low-coupling, high-cohesion architecture model. All applications are built around independent object models, with the inner layer defining interfaces and the outer layer implementing interfaces. The coupling direction is cohesive toward the center, and all codes can be compiled and run independently with the infrastructure.
SOA architecture: SOA is the abbreviation of Service Orientated Architecture. It means that each function is provided by an independent service. The service defines a clear callable interface, and the orchestration and call between services complete a complete business. In fact, this architecture is also the most mature and most commonly used architecture model in the current architecture.

What is EDA Architecture?

After we have discussed all the previous architectural trends, let’s take a look back at what EDA architecture is.

EDA Event-Driven Architecture is a system architecture model whose core capability is to discover system "events" or important business moments (such as transaction nodes, site visits, etc.) and take necessary actions on the corresponding events in real time or near real time. This model replaces the traditional "request/response" model, in which the service must wait for a reply before proceeding to the next task. The process of event-driven architecture is run by events.

The above diagram actually explains the EDA architecture model very well, but it is not clear enough. So, here we compare it with the monolithic architecture to see the difference between them.

In the comparison chart above, we can actually see the difference between it and the traditional architecture. In the general traditional architecture, after the order creation operation occurs, a series of operations are actually completed through a system. The concept of event-driven is to convert all operations into the concept of "events". The downstream decides which system to call to complete what operation by capturing a certain "event".

In summary, event-driven actually encapsulates more important business moments into "events" and routes the events to downstream systems through an EventBus.

We understand the entire process of EDA architecture, but we have not yet figured out what this so-called "EventBUS" is.

The above picture is the core logic architecture of event-driven. Does it look very similar to a traditional MQ? Don't worry, I will talk about the complex part of this architecture below. After talking about EventBus, let's look back at "events". The more important part of the introduction just now is actually to convert operations into a certain type of event for distribution. So how do we define this event?

In simple terms, an event is a significant change in state, triggered when a user takes a specific action. Take the example of a 4S store selling a car:

When a customer buys a car and its state changes from For Sale to Sold is an event.
After a successful transaction, deducting the amount from the account is an event.
After clicking Book a Test Drive, one event occurs from adding the reservation information to the specified user.
Each event may trigger one or more options as a response.

Regarding events, the Cloud Native Computing Foundation (CNCF) hosted the open source CloudEvents project in 2018. The project aims to describe events in a unified and standardized format to enhance interoperability between different services, platforms, and systems. Under the definition of the project, the general event specification is as follows:

Events are mainly composed of Json bodies, which describe the events that occurred through different fields.

Practical thinking on the implementation of EDA architecture

Before we start introducing the implementation, let's take a look at a classic EDA architecture model:

This is a very classic EDA order architecture, which mainly uses EventBridge and FC function computing (if you are not familiar with FaaS, you can regard the FC node as a POD node of ECS or K8s), and drives various businesses to collaborate through events.

Therefore, the central node (EventBridge) has three important capabilities:

For Event Capturing: Ability to collect events
For Routing: The ability to distribute event routing to downstream through event content
For Event Processing (Event Filtering/Replacement): The ability to desensitize or preliminarily filter and screen events. Usually, it is difficult to achieve these three capabilities. For example, Event Capturing may require familiarity with Dell Boomi, Snaplogic, MuleSoft, Dataflow, Apache Apex, etc., Routing may require RocketMQ, RabbitMQ, ActiveMQ, Apache Kafka, and Event Processing requires understanding Apache Storm, Apache Flink. Therefore, the logical architecture mentioned above is actually very ideal. To achieve a complete EDA event-driven, these core capabilities must also be included.

In fact, we can also glimpse some information from the architecture just now. EDA architecture is actually not that simple. So what are its advantages and disadvantages? Below I will briefly list the advantages of EDA architecture in practice:

Loose coupling: The event-driven architecture is a highly loosely coupled and highly distributed architecture model. The creator (source) of the event only knows that the event has occurred, but does not know how the event is processed or how many interested parties subscribe to the event.
Asynchronous execution: EDA architecture is the most suitable execution tool in asynchronous scenarios. We can keep the required events in the queue until the status is normal and then execute them.
Scalability: Event-driven architecture can quickly divide services through routing and filtering capabilities, providing more convenient expansion and routing distribution.
Agility: Event-driven architecture can provide more agile and efficient deployment solutions by distributing events to anywhere.
Of course, the disadvantages are also obvious:
Complex architecture: The event-driven architecture is complex, with many routing nodes, complex system formation, and many functional requirements.
Difficult routing and distribution: Event routing and distribution are difficult. Flexible event routing requires strong real-time computing capabilities and places high demands on the overall distribution system.
Unable to track: Event tracking is the guarantee of the entire EDA architecture. It is often difficult to track the event processing status in the EDA architecture, which requires a lot of customized development.
Poor reliability: Event-driven systems usually have poor reliability and cannot guarantee delivery because they require multi-system integration.

How Alibaba Cloud EventBridge solves the dilemma in EDA scenarios

In response to these problems in EDA scenarios, Alibaba Cloud launched EventBridge, a serverless event bus service, whose mission is to serve as the hub of cloud events, connect cloud products and applications, applications and applications with the standardized CloudEvents 1.0 protocol, provide centralized event governance and driving capabilities, and help users easily build loosely coupled, distributed event-driven architectures; in addition, there are a large number of SaaS services in vertical fields in the cloud market outside of Alibaba Cloud. EventBridge will help customers create a complete, event-driven, efficient and controllable cloud experience with its excellent cross-product, cross-organization and cross-cloud integration and integration capabilities. It also provides targeted solutions to the EDA dilemma.

Complex architecture: It provides common Source, Buses, Rules, and Targets module management capabilities in the industry, and supports both EventBus and EventStream modes, which greatly reduces the difficulty of event-driven architecture.

Routing distribution: EventBridge is driven by event rules, supports 8 major event modes, and 4 converters to meet all the requirements of routing distribution.

Unable to track: Exclusively provide event tracking capabilities, event analysis/query capabilities, and improve the overall event chain for users.

Poor reliability: Supports DLQ/retry mechanism, which greatly guarantees event failures and delays caused by user downstream systems. At the same time, on this basis, EventBridge supports 82 Alibaba Cloud products and 847 event types.

Alibaba Cloud EventBridge More Scenarios

1. Classic EDA event-driven: The most important capability of EventBridge is to build EDA (Event-driven Architectures) event-driven architecture by connecting applications, cloud services, and Serverless services, driving the connection between applications and applications, and applications and the cloud.

2. Streaming ETL scenario: Another core capability of EventBridge is to provide basic filtering and transformation capabilities for streaming data pipelines, perform data synchronization/cross-region backup between different data warehouses, between data processing programs, and between data analysis and processing systems, and connect different systems and different services.

3. Unified event notification service: EventBridge provides a wide range of cloud product event sources and event lifecycle management tools. You can directly monitor the data generated by cloud products through the bus and report it to downstream services such as monitoring and notification.