Photo by Brooke Cagle on Unsplash
You are reading the third article in the Designer and Metrics series. In the first, we thought about whether a product designer needed metrics, and in the second, we looked at what metrics there are. If you haven't read these articles yet, I recommend starting with them.
This article will discuss how to answer the question whether we have improved the product or not.
Days after release
: ? ?
, , . — .
, : , , — , , , .
«» «»
— R (), — , — Z ().
, , R ≠ Z.
, , , . — .
, , , , R.
, , — , 300 ( ), .
Z, . , , Z ≠ R.
?
. , R.
. , 300 , 5 , , 1,7%.
, R. :
- — 1.7%
- , — 300
- ( ) — — 146 .
- , . 90, 95 99%
, R 90, 95 99% ( , ).
, , R ( ) 0% 3,59% .
, , 0 5 268 274 . ( — 2,6 ).
, .
- «» «»
, . : «» «» , .
, , - .
, , , - . .
, 2 , 5 000 . CTR 2% ( ), 3%. , ?
, :
:
- — 2%
- — 5 000
- 95%
, R 95% [ 1,61% – 2,39% ]
( 5 000 ) [ 2,53% – 3,47% ]
, , , .
- , , .
- .
- If the confidence intervals "before" and "after" do not intersect, it means that the designer's work has definitely given a positive result.
What's next
This was the third and final article in the Designer and Metrics series.
In the next article, I plan to tell you why the user shouldn't be asked what he likes and what he doesn't. Subscribe not to miss!