😛 👩🏿 📚 Deferred Tasks in Micro-Service Architecture 📛 📃 💅🏿

Often in projects there is a need to perform deferred tasks, such as sending emails, pushing and other specific tasks inherent in the domain of your application. Difficulties begin when a regular crontab is no longer sufficient, when batch processing is not suitable and when each unit of a task has its own execution time or it is assigned dynamically.

To solve this problem, ~~another~~ solution was created called Trigger Hook . A schematic diagram of work is shown in Figure 1. The diagram shows what happens to tasks during their entire life cycle. A change in color means a change in the status of the task.

Figure 1 - Schematic diagram of the Trigger Hook

	a task whose launch time will not come soon
	a task whose launch time is coming soon
	task whose start time has come
	processed job
	unconfirmed job status in database
	delete command

Task life cycle:

When a task is created, it goes into the database (square block) (red and yellow).
( ), ( ->). ().
, ( ->). .
, ( ->->). , .

API

Id UUIDv4. , . id . UNIX.

task := &domain.Task{
	Id: 	    id,
	ExecTime: time,
}
triggerHook.Create(task)

triggerHook.Delete(task.Id)

for {
	result := triggerHook.Consume()
	if err != send(result.Task()) {
		result.Rollback()
	}
	result.Confirm()
}

, , . , . . .

, - . , , . . Trigger Hook - .

, , . . . Trigger Hook, , - . .

, . , . , , . . , . .

, , - .

AWS EC2 Ubuntu 20
t2.micro
1 vCPUs 2.5 GHz
1 GiB RAM

AWS RDS MySQL 8.0
db.t3.micro
2 vCPUs
1 GiB RAM
Network: 2085 Mbps

		(/)
	1 11	1396	100000
	52	1920	100000
( )	498	200668	100000
( )	2	49905	100000

Trigger Hook time-series . . .

. :

InfluxDB+Grafana

Trigger Hook -

- , , . Trigger Hook -, . , (, ) ( ) , .

, 2 . - , , RabbitMQ. - , , HTTP. ( ), . , . , - , . , Push .

. , “” - .

Figure 2 - Scheme of communication through an asynchronous channel — 2 -

3 4 .

Figure 3 - The process of creating an entity with deferred execution — 3 -

Figure 4 - Executing an entity assignment — 4 -

. , . Trigger Hook . Trigger Hook , . , , , , , , , Trigger Hook.

. , , , . , “ email ” “ YouTube”, Trigger Hook “ ”. , Trigger Hook “ ”. , , , , “ ”. 5 6 .

Figure 5 - Creating a job using an intermediate layer — 5 -

Figure 6 - Processing an event using an intermediate layer — 6 -

. - . , , . , , (, ) -. , . - - “ ”?

Figure 7 - Task manager in one m / s with Trigger Hook — 7 - / Trigger Hook

7 , , -, , . , - -. - . RabbitMq direct.

Figure 8 - Task manager as part of the client m / s — 8 - /

8 , - . - - Trigger Hook -.

, . , , , PHP + MySQL. PHP , Nginx , , MySQL PHP . MySQL , PHP , .

, . . Trigger Hook . , . , , 5 . , Trigger Hook . . API delete. , . .

, Trigger Hook . . , Trigger Hook , , Trigger Hook - , , .

9 . Trigger Hook , ( ). Trigger Hook . , - hash map , , Redis, -:

task_id:instance_host

Figure 9 - Horizontal scaling scheme — 9 -

. , . , Trigger Hook . id ( ) trigger hook ( ). .

Trigger Hook

-. Docker . Kubernetes. minikube. .

Figure 10 - Simplified diagram of the interaction of micro-services — 10 - -

Message service - ( 11), API email . .

.
.
PHP, Symfony 5.
. API Nginx. - supervisor RabbitMQ. .
8 .

Figure 11 - Message service — 11 - Message service

Message Dashboard - Message service ( 12).

Figure 12 - Demo application interface — 12 - -

Mailer . . , .

Trigger service - . GRPC , AMQP ().

Figure 13 - Trigger service — 13 - Trigger service

Monitoring - , . 14 . Grafana InfluxDB. .

Figure 14 - Technical metrics Trigger Hook — 14 - Trigger Hook

I hope you find the app and article useful! Follow my github , follow the project , put asterisks). Thank!

Deferred Tasks in Micro-Service Architecture

API

Trigger Hook -

Trigger Hook

More articles: