Clever Geek Handbook

Correlation between time series: what could be simpler?

Preliminary remarks





UPD: Unfortunately, the design of this article took place with great difficulty, so at first it was even posted in the cloud , and only a truncated version without hyperlinks, pictures, formulas and most spoilers got here, but with a detailed discussion of the features of the new WYSIWYG editor. Now, thanks to the site moderators, most of the bugs have been fixed. But deleting an entire section from the article "retroactively" is probably wrong: now it is not 1984 . So I'll leave this spoiler in place anyway:





A tale of why I did not handle the design

, , . . , , , , habrastorage. .





, , . , , - , - . , - . , - Notepad .





, , : wysiwyg ( ), , ? , - ? ! , wysiwyg - ... ? ?! " "? ! . , ...





, , . . ? "Heading N". , ! , ... (. )! . "/" "/" ( , , - , , - !) , , . (. ).





, - . "...", , "". , . - , IT IT?! , - ?!





, . , habrastorage. . , ... , ? (. )! .





, . , . - . . , , , . , :





"". "" , habrastorage, ( ) . - . , . ... habrastorage: " ".





, - .





, , . , . , MS DOS - "" "", "Ctrl+" "Ctrl+", , Shift. , 2021, Google Chrome ( 87.0.4280.141-64), . , , ( - ). , , , . , - , , - , , , . , IT, . , .





, , . . - - , , . , - ?





, . , . , ( ?) . , , . , , ?





P.S. - , , neo@habr.team, . , - ( ). . ..





UPD: ! : — . , , . , .





Thanks to the moderators, most of the design bugs were fixed! And all the above comments to the original version of the editor can now be considered as my bug report ;-)





I hope that this couple of extra paragraphs at the beginning of the article will not offend or strain anyone ... And now - to the point:





The link between sea pirates and global temperature seems clear. Interestingly, for copyright piracy (not shown in the figure), the correlation with warming is much stronger, only the sign of the correlation will be the opposite.
The link between sea pirates and global temperature seems clear. Interestingly, for copyright piracy (not shown in the figure), the correlation with warming is much stronger, only the sign of the correlation will be the opposite.

Introduction

() , . , , . , – , () . , , , . , . , "" . , , . " " , , . , "", " ". , , , " " . , ( ) – . – ... , ?





:

  • 1.





  • 2. –





  • 3.





  • 4.









: , , ?





1 –

, , 2-.





, . , , , . , – . , , .





, – . , - , , . , . – , , – . , .





, X Y, Rxy . Rxy , Rxy. , , ( ), , ( ) Rxy, X Y . , X Y , 95%- z95 :





z_ {95} = 2 / \ sqrt {N}

N – ( , ). , Rxy X Y , 100 , X Y 5% (.. 20- ) Rxy 0.2.









z_ {99} = 3 / \ sqrt {N} = 0.3

1% . , Rxy=0.4 ( , , z), , , , . , X Y ( " "), , , . , X Y .





- , , : ? , , . , , .





( ) , ( ) X Y, . ( ) Rxy , , , . . , X Y , ( ) . Rxy=0.4 N=100 .





, z , , X Y . – "" .   , ( 0 180° ) . ( , , ). , L , ,





, , - , , L, , L . – . : "" , , . ?





, . , , , , z Rxy. , X Y. , " ", , .





-, ? , , ... , (, , ) , . , . , , 99%- , , 97%-. , , . , 10000 Rxy=0.25 ( 3/sqrt(N) z99=0.03), X Y . z !





, , , . . - , . , , , X Y (Rxy=0.25 10000) X Y.





, ?





2. –

, , .





: 10000 "" ( , , ..) |Rxy| > 0.1 . , , 0.01.





: , , , Rxy , 0.1. – . , . , "" . – , , . , ? ...





. , , ( , 5 , ):





The water level in a well in Kamchatka (above) and the motor activity of Colombian cockroaches (below). Motor activity was measured automatically, within the framework of an experiment for predicting earthquakes by a biological method
() (). ,

, Rxy= –0.35 20 000:





By the way, a very revealing picture! On the one hand, the correlation field clearly demonstrates that the relationship between variables is rather vague. On the other hand, the formal 99% significance level, equal in this case to 0.02, is exceeded by two orders of magnitude. That is, the criteria clearly say that this correlation is not accidental. It seems to me alone that these two statements are somehow not quite ideally consistent with each other?
, ! , , . - 99%- , 0.02, . , , . , - ?

Rxy= +0.16. , 99%- . - ?!





. , 1975-1985 , 20 ( ). , , 2800 (10.7 ) 1955-1965 . , . ! Rxy , , . 99%- . -





, NN 1-6 +0.04, +0.12, -0.01, +0.28, +0.45 +0.17  3800 , z99=0.05. 15 99%- 9 ...





, . , – , . ! , . - ( 30 ) – -. , - ( +100 ), ( ). , 1899 ( ), , . , ( ), ! , , Rxy, .





: " " , – . , N, z. , [-z, +z], .





, ? "" ? , ""   ? ( , - ), ?!





, , , . , . ( , , : , , ). – , . , , / , . , , , Rxy. , – .





– , – . , , , , , (, , ..). , , " " .









, , ( ), . , . (, ). , . , , , ...





, !





. , !





3.

, , . , . , "" , , , .





, – . - , . , . , . , , , , . – , .





. , , . , t . t t+dt . , , , . , . , . . , .





, , , ? , ?! ( ;-)





, . - . , , , ! , , - . ! , X Y, X – Y. , , , . .





, . , .. , . .





, , – . ( , ), . . , , . , , ?





.





, (, , ) . ( , ), . , . , " " – ( ). ,





, - , ..





, W T b. , b - 0.5 2.0. ( b=2), . ( t+1) t . , . , .





Examples of model series with a power-law spectrum. Power parameter b is labeled on the graphs
. b

b, . , b, . , , . , .





, , , - Rxy. , .





: , , , . , , - , ,





, . , " " , |Rxy| >> z, Rxy = 0. , , . , () , . ,





:

1) X Y

2) - X Y

3) X Y –

(.. (Xi, Yi), )





Rxy , , z, z - N, 2.









, , ( , ), . , , Rxy.





4.

, . - . X Y .





  1. , , , X Y. , .





  2. X Y. – Rxy.





  3. Rxy , – , 1, 2 3 – .





  4. Rxy . , Rxy , , . , 1, 2 3 – .





  5. , , X Y, . , 2 3, 1. , : !





, "" . z, , z , Rxy . , " " . , .





, ,





, , . S(t) (t=0,1,... 1000000). : P(0)=0, P(t+1)=P(t)+S(t). P(t). ( ). , . , z Rxy . . , , ?





, b=[0.5, 2.0]. , , – -, . , , . , . , , ±1. , . , , . , – (= ) . z Rxy N=2. z ?





, - .





, - . , .. - "", , , (, , ..); , ... , , ...





  1. – ! , - . - . , (= ) ( ) . , .





  2. , , ! , . , ( ). ( X Y), . , Rxy X Y, t. t, , . Rxy t, , .





  3. , . , . , - (, ) , , , . , !





  4. , , , , , , . , , " ", . , - . , , , .





  5. , , ? , , , ..? ? . . , , .





P.S. .. .





:

, , ? 





-, . - , , . "" , , - : ! - , ( ) ( ). , , , -. ,





- , ( ). . , . , , - “ ”, -. , . , , . . , . , , , (- - , ), - ..





It is safe to say that for the overwhelming majority of signals received during long-term monitoring, the CLT conditions are not met. First, there is no guarantee that the behavior of a controlled quantity depends on many small and independent causal factors - on the contrary, they are usually correlated with each other, and the contribution of some prevails. But it is even more important that practically all natural processes are nonstationary, which immediately takes them beyond the scope of phenomena to which the CLT can be applied. However, this is already a separate issue, which is discussed in the third part of the article .







More articles:


All Articles