“Sometimes we pay the most for what we get for free.” - A. Einstein
MongoDB 4+ recently introduced support for multi-document transactions.
And since our project just migrated to version 4.2, questions naturally arose:
- What will happen to the performance?
- How much will the operation slow down?
- Are we willing to sacrifice speed for (at least some) accuracy?
When studying the documentation and the Internet, questions only increased:
- Will all operations be slowed down by transactions?
- How much will the combination of operations slow down?
Let's try to find out.
In order to claim at least some scanty share of the truth, you have to work a little.
For ease of perception, I will divide the implementation into 3 steps:
- Tool selection
- Description of combinations of operations and obtaining results
- Analysis of results
Now about each step separately.
Choice of tools:
- A test MongoDB (replica with a minimum number of mongod processes) and a driver for it are required : mongodb-memory-server , mongodb .
- For ease of measuring time, I chose the "microseconds" module
- : ttest, stdlib.
Description of combinations of operations and obtaining results: We
implement each (of the main) separate operations insertOne, updateOne, deleteOne, findOne, insertMany * updateMany * deleteMany * find * and their combinations insertOne + updateOne + deleteOne, insertOne + updateOne + deleteOne + findOne, insertMany * + updateMany * + deleteMany * insertMany * + updateMany * + deleteMany * + find * with, and without using transactions.
Measure the execution time of each operation.
For example - insertMany + updateMany + deleteMany with, and without a transaction
Each operation / measurement will be repeated 300 times (for the analysis we will use 100 results "in the middle", that is, from the 101st to the 200th) ** - let's call this "microiteration" (by iterations of individual operations or combinations).
Now, constantly changing the sequence, we will carry out 100 "macroiterations" (1 "macroiteration" = the total number of "microiterations" * 300) *
* the number of 300 is chosen absolutely empirically
** for more complete information about the implementation, I invite you to visit the github repository (link below in the text)
Analysis of the results:
As a result of all iterations, we received 20,000 measurements for each operation and combination of operations (10,000 using a transaction, 10,000 - without) in the form of arrays.
Next, we need to carry out some calculations.
Trim results that clearly fall outside the sample
Compute mean
Compute standard deviation
Determine the existence of a statistically significant difference between samples using ttest (confirmation or refutation of the null hypothesis)
Using simple graphs, we visualize the results. For example, let's take the combination insertMany + updateMany + deleteMany and separately insertOne (all other results will be presented in text format in the "Conclusions" section). As a result, the generated html files contain a graph whose name corresponds to the name of an operation or a combination of an operation (non-transactional iterations are indicated in turquoise, and transactional in orange). "Is statistically significant" (true / false) tells whether there was any statistically significant difference at all. Everything else is absolute and relative values in microseconds and percentages, respectively.
Conclusions:
- : insertMany + updateMany + deleteMany ( )
- ( 7%): updateMany, find, insertOne + updateOne + deleteOne + findOne, insertMany + updateMany + deleteMany + find
- , (91%): updateOne, deleteMany, findOne
- ( 197% 792%): insertOne, insertMany, deleteOne, insertOne + updateOne + deleteOne
For more information and to check the results by running the scripts yourself, visit github .
Thanks for reading.
Feel free to comment, hopefully we have a good discussion.
Alternatively, you can run it all yourself and get your own results. It will be cool to compare them
Useful links:
medium.com/cashpositive/the-hitchhikers-guide-to-mongodb-transactions-with-mongoose-5bf8a6e22033
blog.yugabyte.com/are-mongodb-acid-transactions-ready-for-high- performance-applications
medium.com/@Alibaba_Cloud/multi-document-transactions-on-mongodb-4-0-eebd662ac237
www.mongodb.com/blog/post/mongodb-multi-document-acid-transactions-general-availability
docs.mongodb.com/manual/core/write-operations-atomicity
www.dbta.com/Columns/MongoDB-Matters/Limitations-in-MongoDB-Transactions-127057.aspx
dzone.com/articles/multi-document-transactions-on-mongodb-40
www.dbta.com/Columns/MongoDB-Matters/MongoDB-Transactions-In-Depth-125890.aspx
www.codementor.io/@christkv/mongodb-transactions-vs-two-phase-commit-u6blq7465
docs.mongodb.com/manual/core/read-isolation-consistency-recency
mathworld.wolfram.com/Outlier.html
support.minitab.com/en-us/minitab-express/1/help-and-how-to/basic-statistics/inference/how-to/two-samples/2-sample-t/interpret-the-results/key-results