Yesterday I was sitting quietly, as usual I don't bother anyone. Here, from two different contacts, they almost simultaneously send a link to the well-known tweet about JSON from SQL. One of the messages looked like this:
It was already a direct challenge. I could not ignore him. So I decided to tell a story that still evokes ambivalent feelings in me. Three years later.
At that blessed time, everyone dreamed of crypto, engaged in ICOs and sculpted crypto exchanges. It was really something new. I had experience in creating systems for classical asset management (stocks, bonds, etc.). The problem was that it formed around accounting systems. I wanted to realize myself in creating an exchange. No wonder I dived into this boiling cauldron with pleasure.
This is the background. There was a lot of interesting things, but today I want to tell you about a specific case - how we created our matcher.
Matcher, this is the core of the exchange. It is in it that transactions take place. In the classic stereotype, this is a high-performance subsystem. But this is true for large exchanges. There, open applications for buying and selling are in the hundreds of thousands.
“ ” . . — , . — .
.
Java . , , . , . “” . .
. , , “” . . , 270 . RabbitMQ. ….
. . 2 . , , . , , .
. . .. , … , .. . - :
! ()
, . . … . .
, . , . , . — .
, . , 100, 5. 100 7 . , .
SQL . , . MySQL . , MySQL . .. . :
SET depth_sell.`limit` = depth_sell.`limit` - IF(
((@tofill := IF(
depth_sell.`limit` <= @limit,
depth_sell.`limit`,
@limit
))
+ (@limit := @limit - @tofill)),
@tofill,
@tofill
),
depth_sell.`executed` = @tofill
WHERE @limit > 0;
@limit := @limit - @tofill() .
IN MEMORY . .. . .
. — . . , , .
CREATE TABLE transactions
(
id INT AUTO_INCREMENT PRIMARY KEY,
moment TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
side1 INT NOT NULL,
side2 INT NOT NULL,
price BIGINT NOT NULL,
volume BIGINT NOT NULL
);
CREATE TABLE depth_buy
(
id BIGINT PRIMARY KEY NOT NULL AUTO_INCREMENT,
order_id BIGINT NOT NULL,
type INT DEFAULT '0' NOT NULL,
market INT DEFAULT '0' NOT NULL,
account INT NOT NULL,
price BIGINT DEFAULT '0' NOT NULL,
`limit` BIGINT DEFAULT '0' NOT NULL,
taker INT,
rev_price BIGINT NOT NULL,
executed BIGINT
) ENGINE = MEMORY;
CREATE TABLE depth_sell
(
id BIGINT PRIMARY KEY NOT NULL AUTO_INCREMENT,
order_id INT NOT NULL,
type INT DEFAULT '0' NOT NULL,
market INT DEFAULT '0' NOT NULL,
account INT NOT NULL,
price BIGINT DEFAULT '0' NOT NULL,
`limit` BIGINT DEFAULT '0' NOT NULL,
taker INT,
rev_price BIGINT NOT NULL,
executed BIGINT
) ENGINE = MEMORY;
CREATE PROCEDURE `make_order_v2`(IN order_id INT,
IN order_type INT,
IN order_account INT,
IN order_market INT,
IN order_limit BIGINT,
IN order_price BIGINT)
BEGIN
START TRANSACTION;
SET @limit := order_limit;
IF order_type = 21 THEN
UPDATE depth_sell
INNER JOIN (
SELECT id
FROM depth_sell
WHERE market = order_market
AND depth_sell.price <= order_price
ORDER BY depth_sell.price + id ASC
) source ON depth_sell.id = source.id
SET depth_sell.taker = order_id,
depth_sell.`limit` = depth_sell.`limit` - IF(
((@tofill := IF(
depth_sell.`limit` <= @limit,
depth_sell.`limit`,
@limit
))
+ (@limit := @limit - @tofill)),
@tofill,
@tofill
),
depth_sell.`executed` = @tofill
WHERE @limit > 0;
INSERT INTO transactions (moment, side1, side2, price, volume)
SELECT now(), depth_sell.id, order_id, depth_sell.price, depth_sell.executed
FROM depth_sell
WHERE depth_sell.`taker` = order_id;
DELETE
FROM depth_sell
WHERE market = order_market
AND depth_sell.`limit` = 0;
IF @limit > 0 THEN
INSERT INTO depth_buy (order_id, type, market, account, price, rev_price, `limit`)
VALUES (order_id, order_type, order_market, order_account, order_price, -order_price, @limit);
END IF;
ELSE
UPDATE depth_buy
INNER JOIN (
SELECT id
FROM depth_buy
WHERE market = order_market
AND depth_buy.price >= order_price
ORDER BY depth_buy.rev_price - id ASC
) source ON depth_buy.id = source.id
SET depth_buy.taker = order_id,
depth_buy.`limit` = depth_buy.`limit` - IF(
((@tofill := IF(
depth_buy.`limit` <= @limit,
depth_buy.`limit`,
@limit
))
+ (@limit := @limit - @tofill)),
@tofill,
@tofill
),
depth_buy.`executed` = @tofill
WHERE @limit > 0;
INSERT INTO transactions (moment, side1, side2, price, volume)
SELECT now(), depth_buy.id, order_id, depth_buy.price, depth_buy.executed
FROM depth_buy
WHERE depth_buy.`taker` = order_id;
DELETE
FROM depth_buy
WHERE market = order_market
AND depth_buy.`limit` = 0;
IF @limit > 0 THEN
INSERT INTO depth_sell (order_id, type, market, account, price, rev_price, `limit`)
VALUES (order_id, order_type, order_market, order_account, order_price, -order_price, @limit);
END IF;
END IF;
COMMIT;
END;CREATE PROCEDURE do_load_matcher_v2(IN market INT)
BEGIN
DECLARE count INT DEFAULT 100000;
WHILE count > 0 DO
call make_order_v2(
count,
IF(count % 2 = 0, 21, 22),
1,
market,
FLOOR(1 + (RAND() * 1000)),
FLOOR(1 + (RAND() * 1000))
);
SET count = count - 1;
END WHILE;
END;, .. . :
- depth_buy — ;
- depth_sell — ;
- transaction — = .
make_order_v2 :
- order_id — .
- order_type — . 21 — , 22 — .
- order_account — ( ).
- order_market — . .
- order_limit — . . , 100. .. 1.15btc 115.
- order_price — . .
, . — . , . .
, .
, . . 100 . 95 . 1.46 . 570 . .
, . . , . . .. .
, ? :
- . .
- , , . .
:
- . , . .
- . . .
Of course, I do not share the optimism of the author of the original post that everything can be done through a DBMS, but experience shows that everything can be done through a DBMS. If you want to. Therefore, you need to be careful about your desires. Otherwise, you will have to experience mental pain for years.
All good!
Original tweet