Testing
In one sentence
Section titled “In one sentence”The test framework is built into the language and Cargo: a function with #[test] is a test, cargo test runs all of them, and unit tests usually live in the same file as the code.
The Java you know
Section titled “The Java you know”In Java, tests are in a separate source tree (src/test/java), use a framework (JUnit), and test the public or package-private API. In Rust, unit tests are a module at the bottom of the file they test:
Java
class DiscountsTest {
@Test void tenPercentOff() { assertEquals(1_800, Discounts.applyDiscount(new long[] {1_000, 1_000}, 10)); }
@Test void rejectsLargeDiscounts() { assertThrows(IllegalArgumentException.class, () -> Discounts.applyDiscount(new long[] {1_000}, 60)); }
@Test @Disabled("slow: checks about 100 million cases") void everyPercentAndTotal() { // ... }}Rust
#[cfg(test)] // compiled only for `cargo test`mod tests { use super::*; // the tests can see private items of the parent module
#[test] fn ten_percent_off() { assert_eq!(apply_discount(&[1_000, 1_000], 10), Ok(1_800)); }
#[test] fn rejects_large_discounts() { assert_eq!( apply_discount(&[1_000], 60), Err(DiscountError::TooLarge(60)) ); }
#[test] fn empty_cart_is_an_error() { let result = apply_discount(&[], 10); assert!(result.is_err(), "expected an error, got {result:?}"); // a custom message }
#[test] #[should_panic(expected = "at least 1")] // passes only if the code panics with this text fn split_by_zero_panics() { split_bill(1_000, 0); }
#[test] fn returns_result() -> Result<(), DiscountError> { let total = apply_discount(&[500], 0)?; // `?` fails the test with the error assert_eq!(total, 500); Ok(()) }
#[test] #[ignore = "slow: checks about 100 million cases"] // run with `cargo test -- --ignored` fn every_percent_and_total() { for percent in 0..=50 { for total in (0..10_000_000).step_by(5) { assert!(apply_discount(&[total], percent).unwrap() <= total); } } }}The code under test:
/// Apply a percentage discount to a cart total.////// ```/// use concepts::testing::apply_discount;////// assert_eq!(apply_discount(&[1_000, 1_000], 10), Ok(1_800));/// ```pub fn apply_discount(prices: &[u64], percent: u64) -> Result<u64, DiscountError> { if prices.is_empty() { return Err(DiscountError::Empty); } if percent > 50 { return Err(DiscountError::TooLarge(percent)); } let total: u64 = prices.iter().sum(); Ok(total - total * percent / 100)}
/// The share of each person, when `people` split a bill.////// # Panics////// If `people` is 0.pub fn split_bill(total_cents: u64, people: u64) -> u64 { assert!(people > 0, "people must be at least 1"); total_cents.div_ceil(people)}The Rust you will see
Section titled “The Rust you will see”#[cfg(test)] // compiled only for `cargo test`mod tests { use super::*; // the tests can see private items of the parent module
#[test] fn ten_percent_off() { assert_eq!(apply_discount(&[1_000, 1_000], 10), Ok(1_800)); }
#[test] fn rejects_large_discounts() { assert_eq!( apply_discount(&[1_000], 60), Err(DiscountError::TooLarge(60)) ); }
#[test] fn empty_cart_is_an_error() { let result = apply_discount(&[], 10); assert!(result.is_err(), "expected an error, got {result:?}"); // a custom message }
#[test] #[should_panic(expected = "at least 1")] // passes only if the code panics with this text fn split_by_zero_panics() { split_bill(1_000, 0); }
#[test] fn returns_result() -> Result<(), DiscountError> { let total = apply_discount(&[500], 0)?; // `?` fails the test with the error assert_eq!(total, 500); Ok(()) }
#[test] #[ignore = "slow: checks about 100 million cases"] // run with `cargo test -- --ignored` fn every_percent_and_total() { for percent in 0..=50 { for total in (0..10_000_000).step_by(5) { assert!(apply_discount(&[total], percent).unwrap() <= total); } } }}- Line 1:
#[cfg(test)]: the module is compiled only bycargo test. It is not in the release build. - Line 3: The test module is a child of the code module, so it can call private functions too. Java tests usually cannot.
- Line 7:
assert_eq!prints both values when they differ. Both must implementDebugandPartialEq. - Line 21:
assert!with a format string: the message appears only when the assertion fails. - Line 25:
#[should_panic]isassertThrowsfor panics.expectedchecks part of the panic message. - Line 31: A test can return
Result. AnErrfails the test, so?works inside it. - Line 38:
#[ignore]is JUnit@Disabled. The reason is part of the attribute.
| Rust | JUnit 5 |
|---|---|
#[test] |
@Test |
assert_eq!(left, right) |
assertEquals(expected, actual). Rust has no fixed order: a failure prints both values as left and right |
assert!(cond, "msg") |
assertTrue(cond, "msg") |
#[should_panic(expected = "...")] |
assertThrows(...) for a panic |
a test that returns Result |
assertDoesNotThrow |
#[ignore = "reason"] |
@Disabled("reason") |
cargo test name |
run tests whose name contains name |
Most errors are values, not panics. To test an error, compare the Result with assert_eq! or matches!, as in rejects_large_discounts. #[should_panic] is only for code that really panics.
How to read it
Section titled “How to read it”Three kinds of tests
Section titled “Three kinds of tests”| Kind | Where | Can see | Java |
|---|---|---|---|
| Unit tests | #[cfg(test)] mod tests in the same file |
private items | JUnit tests in the same package |
| Integration tests | files in tests/ next to src/ |
only the public API | tests of a JAR from outside |
| Doc tests | code blocks in /// comments |
only the public API | (none) |
An integration test is a separate crate that uses the library or runs the binary:
use std::process::Command;
#[test]fn prints_the_total() { let output = Command::new(env!("CARGO_BIN_EXE_cargo-demo")) .output() .expect("the binary runs"); let stdout = String::from_utf8(output.stdout).expect("output is UTF-8"); assert!(stdout.contains("total with VAT: 2162"), "got: {stdout}");}A doc test is an example in the documentation, like the one on apply_discount above. cargo test compiles and runs it, so the documentation cannot go out of date. This site uses doc tests with compile_fail to prove that some examples do not compile.
Useful crates
Section titled “Useful crates”The built-in tools cover most needs. You will also see these crates (check each one on crates.io before you accept it in a review):
| Crate | What it does | Java |
|---|---|---|
rstest |
parameterized tests and fixtures | @ParameterizedTest |
proptest, quickcheck |
property-based tests with random inputs | jqwik |
insta |
snapshot tests | ApprovalTests |
mockall |
mock objects generated from traits | Mockito |
What cargo test runs
Section titled “What cargo test runs”cargo test builds and runs all three kinds. The tests of one crate run in parallel threads by default, so tests must not depend on each other. Useful options:
cargo test discountruns the tests whose names contain “discount”.cargo test -- --ignoredruns only the ignored tests.cargo test -- --nocaptureshowsprintln!output, which is hidden by default for tests that pass.cargo test --workspaceruns the tests of every package in a workspace.
What good looks like
Section titled “What good looks like”- Names that say the expected behavior:
rejects_large_discounts, nottest_discount_2. - Tests for errors and edge cases, not only the happy path: empty input, the boundary values, the largest values.
assert_eq!with exact values, notassert!(result > 0).- Fakes instead of mocks, and real values instead of fakes where possible.
- No
sleep: join the thread, or wait on a channel. #[ignore]with a reason, and the ignored tests run somewhere, for example in a nightly CI job.
Red flags in AI output
Section titled “Red flags in AI output”- Only happy-path tests. An agent that writes the code and its tests often tests what the code does, not what it should do.
- Weak assertions:
assert!(result.is_ok()),assert!(total > 0). They pass for most wrong answers. #[ignore]without a reason, or a test that was ignored to make CI pass.- Mocks for everything, with expectations on exact call order. See S30: Mocks for everything; tests with
sleep. thread::sleepin tests to wait for background work. See S30: Mocks for everything; tests withsleep.unwrap()everywhere in tests is fine. In tests, a panic is a failure with a message. Do not ask to remove it.
- S30Severity: LowMocks for everything; tests with
sleepTests that are brittle, slow, or test the mocks instead of the code.
Better vs worse
Section titled “Better vs worse”The task: test a shipping price function with a weight limit.
WorseOne test with one input and an assertion that only checks that the price is positive.
pub fn shipping_cents(weight_grams: u32) -> u32 {if weight_grams <= 1_000 {500} else {500 + (weight_grams - 1_000) / 10}}#[cfg(test)]mod tests {use super::*;#[test]fn test_shipping() {let result = shipping_cents(500);assert!(result > 0);}}Run in Playground (opens the Rust Playground in a new tab) BetterThree named tests: the base price with its boundary, the price per weight, and the largest input.
pub fn shipping_cents(weight_grams: u32) -> u32 {if weight_grams <= 1_000 {500} else {500 + (weight_grams - 1_000) / 10}}#[cfg(test)]mod tests {use super::*;#[test]fn light_parcels_pay_the_base_price() {assert_eq!(shipping_cents(0), 500);assert_eq!(shipping_cents(1_000), 500); // the boundary is included}#[test]fn heavy_parcels_pay_one_cent_per_10_grams() {assert_eq!(shipping_cents(1_010), 501);assert_eq!(shipping_cents(3_000), 700);}#[test]fn the_largest_weight_does_not_overflow() {assert_eq!(shipping_cents(u32::MAX), 500 + (u32::MAX - 1_000) / 10);}}Run in Playground (opens the Rust Playground in a new tab)
The “worse” test passes for almost any wrong implementation: a price of 1 cent, a formula with the wrong rate, or a limit at 999 grams. It is the kind of test that an agent writes to make a coverage number go up.
The “better” tests pin exact values, test both sides of the 1 000 gram boundary, and check that the largest u32 does not overflow. Each test name says the rule that it checks, so a failure explains itself.
Check yourself
Section titled “Check yourself”Why can a Rust unit test call a private function?
Show the answer
The test module is a child module of the code (
mod testsin the same file), and child modules can see the private items of their parents.What does
#[cfg(test)]on the test module do?Show the answer
It compiles the module only for
cargo test. The tests and their helpers are not in normal builds.An agent wrote
#[should_panic]on a test for a function that returnsResult. Is that right?Show the answer
Usually not. A function that returns
Resultreports errors asErrvalues. The test should check theErrwithassert_eq!ormatches!.should_panicpasses for any panic, including an unrelated bug.Where do integration tests go, and what can they use?
Show the answer
In the
tests/folder next tosrc/. Each file is a separate crate that can use only the public API of the library, or run the binary.