I once worked as a data engineer at a startup. He grew rapidly and at some point decided to buy one large company. There were more than a hundred employees there - it turned out, almost all from India. While our developers were tinkering with exotic code that could not be deciphered at once, the Indian monitoring department caught my attention.
To monitor the network and servers, you can use a variety of software. The main thing is to be sure: if something goes wrong, an alert will work. You will see a notification and have time to fix everything. They had their own idea of monitoring. Imagine several rooms with tables, chairs and huge plasmas on the walls, almost two dozen people are sitting inside. On each monitor, multi-colored graphics are displayed and pieces of paper with phone numbers are glued next to it.
The job of these twenty people was to sit and look at the charts, and if the line suddenly turned out to be above the sticker pasted next to it, call the phone number that was written there and sound the alarm. Here they sat and watched without stopping.
The purchase of the company was inexpensive, but maintaining such an infrastructure cost exorbitant money. The Indians used the expensive Vertica, where, in addition to paying for the hardware, they also had to unbuckle for a license. We decided to try moving to ClickHouse. It is practically a free analogue of Vertica. Both products work on a similar principle: columnar DBMS with sharding, with data partitioning.
And that was another adventure.
ClickHouse killer feature - of course, saving money
, . ClickHouse . ClickHouse, , (, PostgreSQL) .
ClickHouse , : DoubleDelta ! ClickHouse , MergeTree.
, , HDFS , ClickHouse 700 , . , «» SQL- - ClickHouse, , , .
— . , , ClickHouse «» . : TSBS , , — . — http- , Rest .
ClickHouse , , , , . , . — , , . .
, , , ?
, , , .
. -, , . , , , , . SDK . PHP JSON , MySQL. , — .
( ), . - Hadoop, HDFS . . : JSON-, Hive. .
Hive, , . JSON-. HDFS Hadoop - . . , , . ! , , — . . , , , Spark , . , , , , RDD .
, - - . HDFS , — .
, . , , , , Java, - . Scala, , !
ClickHouse. , C++. , , , . Scala, TCP, Native , .
— HDFS. Kafka, , Kafka ClickHouse. HDFS , : HDFS .
- ClickHouse. , , .
ClickHouse — ,
, ClickHouse production-ready , , , , DWH .
, , « » - Hive MySQL, ClickHouse – - , - JOIN - ..
, , , . , . , – , .
, . , , .
, UUID, :
```
SELECT * FROM db PREWHERE uuid != '00000000-0000-0000-0000-000000000000'
```
segfault.
. -! , GitHub, Issues, .
, ClickHouse
Hive , . Tabix. , ClickHouse . . Redash - ClickHouse.
Redshift. - ClickHouse Redshift (, , SQL ). — , Redshift, ClickHouse. , Redshift — 30 . ClickHouse Redash.
, ClickHouse Tableau 70 !
Tableau — . , ClickHouse . PostgreSQL. - , PostgreSQL . ClickHouse PostgreSQL Tableau! . PostgreSQL. .
. Tableau — , , SQL , . . 70 .
Tableau . , ClickHouse . . , Tableau - ClickHouse. !
Redash Tableau — . , ClickHouse — . , - , Tableau.
ClickHouse — . , . .
, , — . , , ClickHouse.
, - , . — Vertica Redshift .