This article is not an answer to many questions about databases (DB) and database management systems (DBMS). As an author, I express my own opinion about trends, trying to rely on impartial indicators, statistics, etc., but citing my own experience as an example. I am not an engaged representative of any company and I express my point of view based on my experience of more than 25 years of working with various DBMS, including one that I created with my own hands. There are not so many even experienced programmers and architects who know all the terms, technologies, what pitfalls and where the movement is heading. The topic is truly huge, so even the upper level of information cannot be disclosed within the framework of one article. Unless someone comes across their favorite DBMS or its incredible plus worth mentioning,then I ask you to indicate in the comments and thereby supplement the overall picture, which will help others to understand and understand better the subject area. Go!
Open Source DBMS vs Commercial DBMS
To begin with, I will give a graph from the site, db-engines.com, in my opinion, it is good at tracking database trends. It was this graph that added the desire to write an article on the current state of affairs. When we say the phrase "database", in fact, we often mean a specific database management system (DBMS), so if the text contains a database instead of a DBMS, it is due to such a habit.
Open Source DBMS - Open source database management systems have caught up with commercial closed source DBMSs. Open Source 49.98% versus 50.02% for Commercial. The end of 2020 is the moment when it will be possible to say that open source is no less popular. As you can see, this situation did not appear suddenly. The calculation on the graph is not in a numerical ratio, but in points, which are gained by certain systems.
. Microsoft Access , COBOL , . , IBM DB2 . 10 – 75% . 10 .
Open Source , , Oracle, MS SQL, IBM DB2 , . . enterprise licenses ( ) . , 4 16 - 64 .
MS SQL - 439 936$
Oracle - 1 368 000$
, , , , , . .
2019 2020 AMD , EPYC, , , . AMD 64 - . ? . . AMD , Oracle, Microsoft . Microsoft SQL Server Linux, . IBM DB2 .
( « ») , , Open Source . .
, . , : (Microsoft), (Oracle), . 10 Forbes , (Amazon RDS), (Google Bigtable).
open source . , , , . , , , , , , . , .
MySQL open source 1 .$, Sun Microsystem, Oracle. MySQL MariaDB. , . MariaDB 12 .
, , , , , , . open source , , Facebook c MySQL. open source , . 10 SQLite - . -, .
: , 70-80 , Oracle, , , , .. .
, , . () ( ), . , – , , . open source. ClickHouse, Tarantool, PostgreSQL .., . . , , , « ».
open source . , Oracle . « » Open Source. , , PostgreSQL, MySQL, SQLite, MongoDB .
, open source commercial, online , , .. online vs. offline.
online offline
2 , - , - . .
, , , ( ) ( ), Windows . WISA (Windows, IIS, MS SQL, ASP.Net). WISA, , StackOverflow. , LAMP (Linux Apache MySQL PHP). , , Windows . .
, . , , .. ( ) , ERP, CRM SAP, Microsoft Dynamics NAV AX, 1C . , , Oracle, MS SQL, IBM DB2.
10 , 2 Microsoft Alphabet (Google). . . , .. 2020 .
. , , , . 113 . 20. open source , . - () , open source. , , .
- online . , .
, . RDBMS – . ¼ , .
, MS SQL(Oracle) MongoDB, Redis, MySQL, ClickHouse, ElasticSearch ..
:
Relational: , . 2- , . . 2 – (Key). , , SQL (Structured Query Language), . . 2- .
Document stores:
, . . , . MongoDB – JSON (Java Script Object Notation). BSON (Binary JSON), , .
( )
.
Key-Value : NoSQL - , - . , , . – Redis. , . , . , , . , 1 IP . IP, +1. , (). TTL X .
Search Engines: – . c ID (, , ..) , , , . . , , , , , , , . ElasticSearch , ELK (. ) ElasticSearch+Logstash+Kibana , , .
Wide column stores: Key-Value . , . .
– Cassandra, HBase.
Graph : , , . () (). .
(), - () . , — , . 18 , . - Neo4j.
Columnstore
db-engines , , . , . , , . , . ClickHouse, Vertica. ClickHouse.
ClickHouse . , .
Multi-model databases
, . , , . «multi-model». multi-model NoSQL . - , .
SQL vs NoSQL
NoSQL 10 2009 « ». , + SQL NoSQL, , . . , , . , , , , , , , .. . , , , , , ..
, , 10 .
, 0- 1 Gb , . 100-200 .
SQL 10 100 . , , , , , . , . , .
, NoSQL , , SQL, . , NoSQL - , , .
NoSQL . .
A SQL – Structured Query Language – ? 1970- , , . , (features - ), SQL, , , SQL. SQL , . , NoSQL , SQL. , MongoDB Studio3T, SQL MongoDB, MongoDB SQL . ClickHouse Tarantool ( ) SQL . NoSQL SQL, , join-, , NULL ..
Cloud DBs vs DBs
Gartner 5 2 .
BPaaS IaaS. . AWS , Microsoft , Alibaba , .
(DBaaS) , , .
:
, , , . , CPU, RAM, SSD(HDD) , .
, . . , , , , , , ..
, , . , , . , , , , Mb. AWS Azure . NASA, , . .
. , , . . , , . Gb, , .
, - , . , . .
, , . , Alibaba , 12 - , .
, , IP . .
. , . .
., SberCloud, . , , . , Microsoft, Google, .
: , , , . c .
, , , . . .
, , .
OLTP vs OLAP
Online Transaction Processing (OLTP): , , , .. . , . OLTP – . , OLTP.
, Online Analytical Processing (OLAP). OLAP , . , , , . OLAP , , . Data Scientist. , . SQL, Python, R, , ..
OLAP OLTP, . OLAP , . OLAP , , SIMD (single instruction, multiple data) , 1 , .
, . , , . , , NVidia Tesla.
SSD vs HDD vs Storage vs Tape vs Other
.
2020 , SSD HDD. , . , , , . . SSD, HDD . HDD , . 1000 , HDD. OLTP HDD. SSD c 6000 IOPS ( ), NVME , , .
HDD OLAP , , . 1 – . HDD SSD 1 .
SSD HDD . . SSD HDD, , , SLC MLC, , TLC QLC, . . SSD 1Tb 1000$ - . , , , . SSD HDD, . SSD .
- SSD – 2.5 , PCI-X , U.2– M.2, . SSD – NVME.
Storage – () - , . , . – , 10 . . – , . . , . AllFlash – HDD SSD.
EMC DELL . Huawei . , .
Intel Optane (3D Xpoint) – , , , SSD. - . SSD+NVME /. Optane SSD, RAID .
RAID – , . . , RAID . OLTP RAID10.
Tape – . , 2020 . , . , . , , . , .
– , . , , .., . . . .
: Horizontal scaling vs Vertical Scaling.
, 10-15 . 2000-x Oracle, Microsoft, IBM , , Web API, , 2000-x , ( ) . CPU, , , .. . , .
, , , . , , . , , .
, . : , , . . , . , . , . . , hash , . ().
, 8 , , 4 , 2 , . , 2 8 10 . : , .. 2-3 , . .
2 2 , , , , , .
: , , , . , .
- High Availability. Master-master, master-slave.
High Availability– . . . , , . : , , , ..
, , , RAID . , , , , , .
online , . . – , , .
. , , -. Slave- , . Master-Slave.
, . , , . , , master , . Master-Master. master slave .
, .
, , , . , , . master , , slave master, . , , . , , . , .
. . , .
Online maintenance - online alter
24/7/365 – , 24 , 7 365 . (maintenance). , , , , . , , , , , . , , , .
. , ? . , , , . , , . . , .
, . , , , , . , , , , , . – , . , .
, , % . .
: – SQL, , , , , , product- ..
, , , DataGrip JetBrains ( Kotlin, ReSharper, GoLand ..) . , .
, . , - , . , , , . . , . , , , .
, . , , , , , SQL . , - . , . , Flashback Data Archive, Temporal history, Change Tracking, Data Audit ..
, , . , , , , , , , , .
-
, . , . , , , , , . , .
- - , . -. , (, -) - , , , .. , . , , , , , , .. - , -.
, - , .. .
, , ClickHouse . - , .
JSON
NuGet- Micrsoft Visual Studio C# - JSON (JavaScript Object Notation). , , Microsoft, XML. JSON , , JSON .
In Memory
RAM . , . , . . -, .
- In-Memory , , , Tarantool.
, . , .. OLAP . , , , , CPU. , .
(temporary)
, , . , . , , , . , , .
MapReduce
Google . Map – Reduce – . Apache Hadoop Spark – , HDFS . Hive, SQL. .
, . , . , .. Hive , .
. ? , , . (spatial index) R-tree .
Graph data
. .
. . - , , , . , - .
, . , . , . , , , , , .. . , , . - . , . . (), - .
GPU, NPU (Neural Processing Unit), Google TPU (Tensor Processing Unit)
- . , GPU NPU , , GPU NPU , ( ) . . DPU (Data Processing Unit) – , . .
Community
, , . , – . . StackOverflow.com , . . KnowledgeBase.
Tag |
Count |
MySQL |
598,350 |
SQL Server |
285,092 |
Mongodb |
129,907 |
Oracle |
122,385 |
Postgresql |
117,427 |
sqlite |
82,596 |
ms-access |
46,177 |
elasticsearch |
44,482 |
redis |
18,290 |
db2 |
10,485 |
… |
|
clickhouse |
530 |
tarantool |
103 |
, , , , . % - (holy war - ). , . . Oracle IBM DB2 – .
: . OpenSource Commercial , . Online . . SQL. -. , . , . , .