Introduction
Both here and elsewhere on the web there are tons of articles promoting automated testing in general and unit tests in particular. The articles describe the benefits of testing, using it to eliminate fragile code, increase quality, migrate from old systems to new ones, refactoring. And, at the same time, there is almost no mention of their shortcomings anywhere, and there are no "silver bullets" in engineering!
In fact, there are "silver bullets", but they were invented by the first engineers, and they are perceived by us as boring platitudes: "wash your hands before eating", "wipe your feet", "structure the code", "do not write without indentation", "localize state "etc. Nevertheless, tests are not a "silver bullet", but one of the most effective and widely used tools, which means that it has drawbacks.
In this post I will try to structure and write down exactly the shortcomings of tests, mainly unit tests. I will try not to write about the merits, because there are already so many materials about this, just stretch out your hand. Of course, somewhere I will inevitably forget something important, and somewhere I will exaggerate too much. Therefore, please consider this article more like an invitation to talk than something finished. From my point of view, the topic is quite ripe, and therefore I would very much like to discuss it in detail.
Why functional programming? So we test almost exclusively functions.
Farewell to illusions or 33 platitudes
In general, it is not a secret that even 100% test coverage does not guarantee correct program behavior. For example, let's take a look at the code:
def f( a, b):
x = 0
if a:
x += 2
else:
x += 0
if b:
x += 2
else:
x += 0
return x
#
assert f(True, False) == 2
assert f(False, True) == 2
We went through all the branches, everything is fine, but have we proved that the function f always returns a two?
, , 100% , , , , , . - - . - , "" .
property-base testing: QuickCheck Haskell, GAST Clean, Kotlintest, QCheck Ocaml, Hypothesis Python' . . , : , .
, -, , . Geant4 "" (), " " ( ) ( 5 , ).
, , - - . , 20 000 , , — 50 000 . , .
, property-based testing — , , . QuickCheck John Hughes — Building on developers' intuitions (...) | Lambda Days 19. , ...
, — : , , . , .
, , : ?
propertyDoubleEq :: Double -> bool
propertyDoubleEq x = (x == x)
, - - , , , .
2 + 2 = 5? ?
, , , . Jef239.
, - , , . - , , , copy-paste:
string monthName(unsigned int n) {
static vector<string> months = {"", "", ... };
return months[n % months.size()];
}
void testMonthNames() {
assert( monthName(0) == "");
...
}
, , "" . , , : , .
, - " ". , , , . , .
, , , . , , - !
, — , .
- —
, — , . , — , - , :
- . , - , "" , , .
- . , . , , , , .
- , , (. 1). , , . , !
, , , , . . , , , . , , .
- . , . , . -, , , .
— , .
, , - — , , .
- —
, , . , , .
- , , , . , , , , - . :
. , , — , .. . write-check-correct loop — , .
, Ocaml — , , - "" — () . -, , .
. - , - .
CI - - CI , - , . , , , PR .
, - -. , .
- —
, -, - , . , , , , . , , , , . - WindowMaker, Quake I, Heroes 2 , , . , TeX, , .
, -. , . , — " , ".
, - , , . , , - . This does not add business value, right?
, — , - - , , : . — , , Python 2 Python 3 Boost, . , !
: , , - . .
, "" — , , .
, . , . , , mutation testing, - .
, , - , , — , , , . , , . , , , - .
, - — , , . " – , , ".
" ", , , IDE , . , , - , . , , , , , . , , - , .
, , . , , . !