How can we simplify the development and maintenance of the data warehouse?

1. Adapted Anchor modeling methodology

The data warehouse core architecture should follow the adapted (non-original) Anchor modeling methodology described below (but not the Data Vault ).





Table type





Examples of table name (description in brackets)





What types of tables can it be linked to?





Required field type





Field name examples





Entities (Anchor, Entity type). Denoted by a square





TR_Transaction (semiconductor for debit or credit), AC_Account (synthetic account)





Relationships, Entity Attribute





Entity surrogate key





TR_ID, AC_ID





Attribute of entities (Attribute). Denoted by a circle





TR_TDT_TransactionDate (date of the transaction)





Entities





Entity surrogate key (is the primary key for the lifetime of the record)





TR_ID





The start date and time of the entry





TR_TDT_FROM





Record expiration date and time (not inclusive)





TR_TDT_BEFORE





Entity Attribute





TR_TDT





Connections (Tie, Relationship). Denoted by a rhombus





TR_AC_DC_Transaction_Account_DrCr (ledger account in semiconductor)





Entities





Surrogate keys for each related entity (together with some relationship attributes that indicate the type of relationship, form a composite key during the lifetime of the record)





TR_ID, AC_ID





The start date and time of the entry





TR_AC_DC_FROM





Record expiration date and time (not inclusive)





TR_AC_DC_BEFORE









DC (/)





Example data schema

-- --. – . , , , .. . , «» – .





Anchor modeling , . Anchor modeling .





NULL, , ( , , , , , ). / , (, '0001-01-01', '-infinity', '9999-12-31', 'infinity').





, (. ). Crockford's base32.





(knot) Anchor modeling. .





( , ) - Online Modeler Online Modeler (test version), SQL- . SQL- ( SQL-) Anchor modeling Python Microsoft Excel.





2. ULID

ULID. , -.





( ) ULID ( UUID), :





  • ttttttttttrrrrrrrrrrrrrrxx (: 01F5B023PBG3C48TSBDQQ3V9TR)





  • ttttttttttsssrrrrrrrrrrrxx (: 01F5B023PB00448TSBDQQ3V5TR)









t – (Timestamp) (10 48 ), UNIX-time (UTC)





s – 0 32768, , (Sequence) (3 15 )





r – (Randomness) (14/11 65/55 )





x – (Entity type) (2 10 )





Crockford's base32.





ULID’ :





  1. ULID’





  2. ULID’





  3. ( )





  4. ULID’ ( – , – )





  5. ULID’ (, , UUID ), (HDD SSD)





  6. ( 5 ) ULID’ (HDD SSD) (, ULID’)





- , . , .





3.

SCD2 (): « » ( _FROM) « ( )» ( _BEFORE).





, , (valid time) , , (transaction time). «» (, ).





4.

, ( ), (transaction time), . , , , .





: TR_TIMESTAMP, TR_TDT_TIMESTAMP, TR_AC_DC_TIMESTAMP.





5.

( , , , ..), – . , - ..





(, ) , , .





6.

.





, . , – .





, , . , :





  • /,





  • ,





  • ,





  • ,





  • ,





  • .





7.

true false, , array, hstore jsonb.





8.

( ) , . , , .





, , IT-.





. , , «». , .





, array, hstore jsonb.





9.

.





, Anchor modeling. , .





10. SQL-

SQL- , Anchor modeling, . SQL- SQL- , . SQL- . SQL- ( Python Microsoft Excel).





SQL-, . (. ).





Anchor modeling . SQL-, .





11.

, SQL ORM . , , . :





  • – , ,





  • , .





.








All Articles