👋🏽 📓 🔽 Choosing an assertion library for a Kotlin project 🤝 🙆🏾 🌛

In one of the old projects, asserts from JUnit, kotlin.test and AssertJ were piled up. This was not his only problem: it was generally written as a letter from Uncle Fyodor, and there was no time to stop and bring it to a single form. And now this time has come.

The article will contain a mini-research about which asserts are better according to subjective criteria. At first I wanted to do something simple: throw in a set of tests in order to quickly rivet options with copy-paste. Then he highlighted the general test data, automated some of the checks, and how everything went ... As a result, we got a small Rosetta stone and this article can be useful to you in order to choose a library of asserts that suits your realities.

I will make a reservation right away that the article will not compare testing frameworks, testing approaches, and some tricky approaches to data validation. We will talk about simple asserts.

If you are too lazy to read boring reasoning, the history of my ordeals and other details, then you can go directly to the comparison results .

A bit of background

Scala, — ScalaTest. , - , - .

Kotlin, - , , Kotest ( , ).

, , — . :

Kotlin IntelliJ Idea. Scala- — . , ScalaTest , . IntelliJ <Click to see difference>, . , , IntelliJ Idea — Kotlin- - , ?
. 1 != 2 , , "expected" "actual". " 100 , , ", , "… , , , ". , ? , — , , - .
. assertEquals(expected, actual) — , . , — /, " , , " , contains, includes. — , .
. - assertThat("Friendship").contains("end").
. - JUnit4, , ExpectedException @Rule.
() .
.
. — , . , : , , generic-, . -: assertThat(generic<Boolean>(input)).isEqualTo(true). <Boolean> . .
, . . ? — , , . - , equals. - .

, , , , , . , — , . , -, QA-.

, 5 , , — .

ScalaTest — .
JUnit 5 — Java-.
kotlin.test — multiplatform . — JUnit, .
AssertJ — . FestAssert, - .
Kotest — KotlinTest, kotlin.test. , ScalaTest. 1-22 — scala.
Truth — . , AssertJ.
Hamrest — . valid4j.
Strikt — AssertJ .
Kluent — , JUnit ( — kotlin.test), Kotest. — , .
Atrium — , AssertJ, . — ( maven/gradle).
Expekt — Chai.js. : — 4 .
AssertK — AssertJ, AssertK ( ).
HamKrest — Hamrest, HamKrest ( Hamcrest ).

— , -.

80 , , , . - , .

, , 1 — 1 . , , - , " , " "".

, - , JUnit , , , , , . ScalaTest, : , , — . : ? , :). / , , .

: listOf(1,2,3)? — -, — . , , : , . , .

type erasure. Reified inline-, .

assertThrows<T>{...}

, reified :

assertThrows(expectedClass){...}

. , kotlin.test Asserter: , . , ?:)

GitHub. ScalaTest , .

: 0 — , 0.5 — , 1 — . — 9 .

-, , . . , . , - , " " , " " . .


Kotest	±	±	+	+	+	+	+	-	6.0
Kluent	±	±	+	+	+	+	+	-	6.0
AssertJ	±	+	±	+	±	+	+	±	6.0
Truth	±	+	+	+	-	+	+	-	5.5
Strikt	±	±	±	+	+	+	+	-	5.5
ScalaTest	±	±	±	+	+	+	+	-	5.5
HamKrest	±	-	±	+	+	±	+	-	5.5
AssertK	±	±	±	+	±	+	+	-	5.0
Atrium	±	±	±	+	+	±	+	-	5.0
Hamrest	±	±	±	+	-	±	+	-	5.0
JUnit	+	+	-	±	+	-	±	-	4.5
kotlin.test	+	±	-	-	+	-	-	-	3.5
Expekt	±	±	-	+	-	±	+	-	3.5

Kotest

To connect only asserts, you need to dig deeper. In my opinion, it's hard to understand this straight away.
, reified, : .
: . .
: <Click to see difference> .
: .
: , .

Kluent

"hello".shouldBeEqualTo("hello"), "hello" `should be equal to` "hello". DSL .

invoking { block() } shouldThrow expectedClass.kotlin withMessage expectedMessage

, , . Expected Iterable to contain none of "[1, 3]" — , Iterable .
: <Click to see difference> .
: .

AssertJ

— … , containsExactly, — hasSameElementsAs, — .usingRecursiveComparison().isEqualTo.
: <Click to see difference> .
: - , . , , .
: .usingRecursiveComparison(), . : , , . , , ,
```
assertThat(actual)
    .usingRecursiveComparison()
    .isNotEqualTo(unexpected)
```
: .
: . , DSL.

Truth

, .
: , , assertThrows JUnit5. , JUnit , ?
: , , : containsAtLeastElementsIn. , assertThat(actual).isEqualTo(expected).
: <Click to see difference> .
: .
: , .

" ":

expected: 1
but was : 2
at asserts.truth.TruthAsserts.simpleAssert(TruthAsserts.kt:10)
at common.FailedAssertsTestBase.simple assert should have descriptive message(FailedAssertsTestBase.kt:20)
at [[Reflective call: 4 frames collapsed (https://goo.gl/aH3UyP)]].(:0)
at [[Testing framework: 27 frames collapsed (https://goo.gl/aH3UyP)]].(:0)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
at [[Testing framework: 9 frames collapsed (https://goo.gl/aH3UyP)]].(:0)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
at [[Testing framework: 9 frames collapsed (https://goo.gl/aH3UyP)]].(:0)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
at [[Testing framework: 17 frames collapsed (https://goo.gl/aH3UyP)]].(:0)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
...

Strikt

, reified, : .
: expectThat(haystack).not().contains(needle), expectThat(collection).doesNotContain(items).
: contentEquals. : expectThat(actual).not().contentEquals(unexpected). , , Array<T> Strikt - . — containsExactly, — containsExactlyInAnyOrder.
: . , . :
```
val actual: Array<String> = arrayOf("1")
val expected: Array<String> = arrayOf("2")
expectThat(actual).contentEquals(expected)
```
, contentEquals. , contentEquals :
```
infix fun <T> Assertion.Builder<Array<out T>>.contentEquals(other: Array<out T>)
```
-
```
val actual: Array<out String> = arrayOf("1")
val expected: Array<String> = arrayOf("2")
expectThat(actual).contentEquals(expected)
```
: <Click to see difference>.
: , .
: .

ScalaTest

: .
: , . .
: DSL contains, contains include, theSameElementsAs.
: , .
: , .

HamKrest

, , . — , .
, Hamcrest, - : -.

— :

assertThat( {
    block()
}, throws(has(RuntimeException::message, equalTo(expectedMessage))))

: . - 3,5 . : assertThat(collection, allOf(items.map { hasElement(it) })).
.
: .
: <Click to see difference>.

— - :

expected: a value that not contains 1 or contains 3
but contains 1 or contains 3

AssertK

— AssertJ. ( , -).
: AssertJ assertThat(collection).containsAll(items), AssertK , containsAll vararg. , containsAll(1,2,3), . , , — . — . , containsOnly containsExactly.
: <Click to see difference>.
: - , , .
: .usingRecursiveComparison() .
— ( ), :
```
expected to contain exactly:<[3, 4, 5]> but was:<[1, 2, 3]>
at index:0 unexpected:<1>
at index:1 unexpected:<2>
at index:1 expected:<4>
at index:2 expected:<5>
```
?
: .

Atrium

: fluent infix. assertThat(x).isEqualTo(y) x shouldBe y, , expect(x).toBe(y) expect(x) toBe y. , , "". - o: expect(x) contains o atLeast 1 butAtMost 2 value "hello". , , . infix- ( - ), Atrium fluent-.
: : notToBe, containsNot. . , : contains vararg, containsElementsOf , . , contains(1,2,3), . expect(collection).containsNot.elementsOf(items).
, toList.
, reified, : .
: .
: <Click to see difference>.

: ( , ), :

expected that subject: [4, 2, 1]        (java.util.Arrays.ArrayList <938196491>)
◆ does not contain:
⚬ an entry which is: 1        (kotlin.Int <211381230>)
  ✘  number of such entries: 1
       is: 0        (kotlin.Int <1934798916>)
    has at least one element: true
       is: true

Hamcrest

: (
```
assertThat(actual, `is`(not(unexpected)))
```
```
assertThat(actual, not(unexpected))
```
containsString vs contains vs hasItem vs hasItems. , : hasItems vararg, Set<T> T . , hasItems(1,2,3), .
```
assertThat(collection, allOf(items.map { hasItem(it) }))
```
:
```
assertThat(collection, not(anyOf(items.map { hasItem(it) })))
```
hasItems, ± "", , .
: .
: <Click to see difference>.
: .
: .

JUnit

: - assertEquals(expected, actual), : assertArrayEquals, assertIterableEquals ..
: , JUnit - , .
: assertLinesMatch(listOf(".*$needle.*"), listOf(haystack)) , .
: assertLinesMatch, , assertIterableEquals.
: , assertIterableEquals Map Set , .
: .

kotlin.test

. JUnit, . , .
, JUnit, :
.
assertIterableEquals, .
: JUnit' assertEquals, kotlin.test , .
: .

Expekt

expect(x).equal(y) x.should.equal(y), . , .
: contains(item) should.have.elements(items) should.contain.elements(items). containsAll. , : should.have.elements vararg. , should.have.elements(1,2,3), . any: .should.not.contain.any.elements.
: , .
.
.
: .
: .
: <Click to see difference>.

Kotest, Kluent AssertJ. , Kotlin , AssertJ ( ). , .

— , , AssertJ. , , .

Choosing an assertion library for a Kotlin project

A bit of background

More articles: