Node.js + MongoDB: Transaction Performance

“Sometimes we pay the most for what we get for free.” - A. Einstein


MongoDB 4+ recently introduced support for multi-document transactions.



And since our project just migrated to version 4.2, questions naturally arose:



  • What will happen to the performance?
  • How much will the operation slow down?
  • Are we willing to sacrifice speed for (at least some) accuracy?


When studying the documentation and the Internet, questions only increased:



  • Will all operations be slowed down by transactions?
  • How much will the combination of operations slow down?


Let's try to find out.



In order to claim at least some scanty share of the truth, you have to work a little.



For ease of perception, I will divide the implementation into 3 steps:



  1. Tool selection
  2. Description of combinations of operations and obtaining results
  3. Analysis of results


Now about each step separately.



Choice of tools:



  1. A test MongoDB (replica with a minimum number of mongod processes) and a driver for it are required : mongodb-memory-server , mongodb .
  2. For ease of measuring time, I chose the "microseconds" module
  3. : ttest, stdlib.


Description of combinations of operations and obtaining results: We



implement each (of the main) separate operations insertOne, updateOne, deleteOne, findOne, insertMany * updateMany * deleteMany * find * and their combinations insertOne + updateOne + deleteOne, insertOne + updateOne + deleteOne + findOne, insertMany * + updateMany * + deleteMany * insertMany * + updateMany * + deleteMany * + find * with, and without using transactions.



Measure the execution time of each operation.



For example - insertMany + updateMany + deleteMany with, and without a transaction











Each operation / measurement will be repeated 300 times (for the analysis we will use 100 results "in the middle", that is, from the 101st to the 200th) ** - let's call this "microiteration" (by iterations of individual operations or combinations).



Now, constantly changing the sequence, we will carry out 100 "macroiterations" (1 "macroiteration" = the total number of "microiterations" * 300) *

* the number of 300 is chosen absolutely empirically

** for more complete information about the implementation, I invite you to visit the github repository (link below in the text)



Analysis of the results:



As a result of all iterations, we received 20,000 measurements for each operation and combination of operations (10,000 using a transaction, 10,000 - without) in the form of arrays.







Next, we need to carry out some calculations.



Trim results that clearly fall outside the sample







Compute mean







Compute standard deviation







Determine the existence of a statistically significant difference between samples using ttest (confirmation or refutation of the null hypothesis)







Using simple graphs, we visualize the results. For example, let's take the combination insertMany + updateMany + deleteMany and separately insertOne (all other results will be presented in text format in the "Conclusions" section). As a result, the generated html files contain a graph whose name corresponds to the name of an operation or a combination of an operation (non-transactional iterations are indicated in turquoise, and transactional in orange). "Is statistically significant" (true / false) tells whether there was any statistically significant difference at all. Everything else is absolute and relative values ​​in microseconds and percentages, respectively.











Conclusions:



  1. : insertMany + updateMany + deleteMany ( )
  2. ( 7%): updateMany, find, insertOne + updateOne + deleteOne + findOne, insertMany + updateMany + deleteMany + find
  3. , (91%): updateOne, deleteMany, findOne
  4. ( 197% 792%): insertOne, insertMany, deleteOne, insertOne + updateOne + deleteOne


For more information and to check the results by running the scripts yourself, visit github .



Thanks for reading.



Feel free to comment, hopefully we have a good discussion.



Alternatively, you can run it all yourself and get your own results. It will be cool to compare them



Useful links:

medium.com/cashpositive/the-hitchhikers-guide-to-mongodb-transactions-with-mongoose-5bf8a6e22033

blog.yugabyte.com/are-mongodb-acid-transactions-ready-for-high- performance-applications

medium.com/@Alibaba_Cloud/multi-document-transactions-on-mongodb-4-0-eebd662ac237

www.mongodb.com/blog/post/mongodb-multi-document-acid-transactions-general-availability

docs.mongodb.com/manual/core/write-operations-atomicity

www.dbta.com/Columns/MongoDB-Matters/Limitations-in-MongoDB-Transactions-127057.aspx

dzone.com/articles/multi-document-transactions-on-mongodb-40

www.dbta.com/Columns/MongoDB-Matters/MongoDB-Transactions-In-Depth-125890.aspx

www.codementor.io/@christkv/mongodb-transactions-vs-two-phase-commit-u6blq7465

docs.mongodb.com/manual/core/read-isolation-consistency-recency

mathworld.wolfram.com/Outlier.html

support.minitab.com/en-us/minitab-express/1/help-and-how-to/basic-statistics/inference/how-to/two-samples/2-sample-t/interpret-the-results/key-results



All Articles