Skip to content

Testing

The test framework is built into the language and Cargo: a function with #[test] is a test, cargo test runs all of them, and unit tests usually live in the same file as the code.

In Java, tests are in a separate source tree (src/test/java), use a framework (JUnit), and test the public or package-private API. In Rust, unit tests are a module at the bottom of the file they test:

Java

class DiscountsTest {
@Test
void tenPercentOff() {
assertEquals(1_800, Discounts.applyDiscount(new long[] {1_000, 1_000}, 10));
}
@Test
void rejectsLargeDiscounts() {
assertThrows(IllegalArgumentException.class,
() -> Discounts.applyDiscount(new long[] {1_000}, 60));
}
@Test
@Disabled("slow: checks about 100 million cases")
void everyPercentAndTotal() {
// ...
}
}

Rust

#[cfg(test)] // compiled only for `cargo test`
mod tests {
use super::*; // the tests can see private items of the parent module
#[test]
fn ten_percent_off() {
assert_eq!(apply_discount(&[1_000, 1_000], 10), Ok(1_800));
}
#[test]
fn rejects_large_discounts() {
assert_eq!(
apply_discount(&[1_000], 60),
Err(DiscountError::TooLarge(60))
);
}
#[test]
fn empty_cart_is_an_error() {
let result = apply_discount(&[], 10);
assert!(result.is_err(), "expected an error, got {result:?}"); // a custom message
}
#[test]
#[should_panic(expected = "at least 1")] // passes only if the code panics with this text
fn split_by_zero_panics() {
split_bill(1_000, 0);
}
#[test]
fn returns_result() -> Result<(), DiscountError> {
let total = apply_discount(&[500], 0)?; // `?` fails the test with the error
assert_eq!(total, 500);
Ok(())
}
#[test]
#[ignore = "slow: checks about 100 million cases"] // run with `cargo test -- --ignored`
fn every_percent_and_total() {
for percent in 0..=50 {
for total in (0..10_000_000).step_by(5) {
assert!(apply_discount(&[total], percent).unwrap() <= total);
}
}
}
}

The code under test:

/// Apply a percentage discount to a cart total.
///
/// ```
/// use concepts::testing::apply_discount;
///
/// assert_eq!(apply_discount(&[1_000, 1_000], 10), Ok(1_800));
/// ```
pub fn apply_discount(prices: &[u64], percent: u64) -> Result<u64, DiscountError> {
if prices.is_empty() {
return Err(DiscountError::Empty);
}
if percent > 50 {
return Err(DiscountError::TooLarge(percent));
}
let total: u64 = prices.iter().sum();
Ok(total - total * percent / 100)
}
/// The share of each person, when `people` split a bill.
///
/// # Panics
///
/// If `people` is 0.
pub fn split_bill(total_cents: u64, people: u64) -> u64 {
assert!(people > 0, "people must be at least 1");
total_cents.div_ceil(people)
}
#[cfg(test)] // compiled only for `cargo test`
mod tests {
use super::*; // the tests can see private items of the parent module
#[test]
fn ten_percent_off() {
assert_eq!(apply_discount(&[1_000, 1_000], 10), Ok(1_800));
}
#[test]
fn rejects_large_discounts() {
assert_eq!(
apply_discount(&[1_000], 60),
Err(DiscountError::TooLarge(60))
);
}
#[test]
fn empty_cart_is_an_error() {
let result = apply_discount(&[], 10);
assert!(result.is_err(), "expected an error, got {result:?}"); // a custom message
}
#[test]
#[should_panic(expected = "at least 1")] // passes only if the code panics with this text
fn split_by_zero_panics() {
split_bill(1_000, 0);
}
#[test]
fn returns_result() -> Result<(), DiscountError> {
let total = apply_discount(&[500], 0)?; // `?` fails the test with the error
assert_eq!(total, 500);
Ok(())
}
#[test]
#[ignore = "slow: checks about 100 million cases"] // run with `cargo test -- --ignored`
fn every_percent_and_total() {
for percent in 0..=50 {
for total in (0..10_000_000).step_by(5) {
assert!(apply_discount(&[total], percent).unwrap() <= total);
}
}
}
}
  1. Line 1: #[cfg(test)]: the module is compiled only by cargo test. It is not in the release build.
  2. Line 3: The test module is a child of the code module, so it can call private functions too. Java tests usually cannot.
  3. Line 7: assert_eq! prints both values when they differ. Both must implement Debug and PartialEq.
  4. Line 21: assert! with a format string: the message appears only when the assertion fails.
  5. Line 25: #[should_panic] is assertThrows for panics. expected checks part of the panic message.
  6. Line 31: A test can return Result. An Err fails the test, so ? works inside it.
  7. Line 38: #[ignore] is JUnit @Disabled. The reason is part of the attribute.
Rust JUnit 5
#[test] @Test
assert_eq!(left, right) assertEquals(expected, actual). Rust has no fixed order: a failure prints both values as left and right
assert!(cond, "msg") assertTrue(cond, "msg")
#[should_panic(expected = "...")] assertThrows(...) for a panic
a test that returns Result assertDoesNotThrow
#[ignore = "reason"] @Disabled("reason")
cargo test name run tests whose name contains name

Most errors are values, not panics. To test an error, compare the Result with assert_eq! or matches!, as in rejects_large_discounts. #[should_panic] is only for code that really panics.

Kind Where Can see Java
Unit tests #[cfg(test)] mod tests in the same file private items JUnit tests in the same package
Integration tests files in tests/ next to src/ only the public API tests of a JAR from outside
Doc tests code blocks in /// comments only the public API (none)

An integration test is a separate crate that uses the library or runs the binary:

use std::process::Command;
#[test]
fn prints_the_total() {
let output = Command::new(env!("CARGO_BIN_EXE_cargo-demo"))
.output()
.expect("the binary runs");
let stdout = String::from_utf8(output.stdout).expect("output is UTF-8");
assert!(stdout.contains("total with VAT: 2162"), "got: {stdout}");
}

A doc test is an example in the documentation, like the one on apply_discount above. cargo test compiles and runs it, so the documentation cannot go out of date. This site uses doc tests with compile_fail to prove that some examples do not compile.

The built-in tools cover most needs. You will also see these crates (check each one on crates.io before you accept it in a review):

Crate What it does Java
rstest parameterized tests and fixtures @ParameterizedTest
proptest, quickcheck property-based tests with random inputs jqwik
insta snapshot tests ApprovalTests
mockall mock objects generated from traits Mockito

cargo test builds and runs all three kinds. The tests of one crate run in parallel threads by default, so tests must not depend on each other. Useful options:

  • cargo test discount runs the tests whose names contain “discount”.
  • cargo test -- --ignored runs only the ignored tests.
  • cargo test -- --nocapture shows println! output, which is hidden by default for tests that pass.
  • cargo test --workspace runs the tests of every package in a workspace.
  • Names that say the expected behavior: rejects_large_discounts, not test_discount_2.
  • Tests for errors and edge cases, not only the happy path: empty input, the boundary values, the largest values.
  • assert_eq! with exact values, not assert!(result > 0).
  • Fakes instead of mocks, and real values instead of fakes where possible.
  • No sleep: join the thread, or wait on a channel.
  • #[ignore] with a reason, and the ignored tests run somewhere, for example in a nightly CI job.
  • Only happy-path tests. An agent that writes the code and its tests often tests what the code does, not what it should do.
  • Weak assertions: assert!(result.is_ok()), assert!(total > 0). They pass for most wrong answers.
  • #[ignore] without a reason, or a test that was ignored to make CI pass.
  • Mocks for everything, with expectations on exact call order. See S30: Mocks for everything; tests with sleep.
  • thread::sleep in tests to wait for background work. See S30: Mocks for everything; tests with sleep.
  • unwrap() everywhere in tests is fine. In tests, a panic is a failure with a message. Do not ask to remove it.

The task: test a shipping price function with a weight limit.

  1. WorseOne test with one input and an assertion that only checks that the price is positive.

    pub fn shipping_cents(weight_grams: u32) -> u32 {
    if weight_grams <= 1_000 {
    500
    } else {
    500 + (weight_grams - 1_000) / 10
    }
    }
    #[cfg(test)]
    mod tests {
    use super::*;
    #[test]
    fn test_shipping() {
    let result = shipping_cents(500);
    assert!(result > 0);
    }
    }
    Run in Playground (opens the Rust Playground in a new tab)
  2. BetterThree named tests: the base price with its boundary, the price per weight, and the largest input.

    pub fn shipping_cents(weight_grams: u32) -> u32 {
    if weight_grams <= 1_000 {
    500
    } else {
    500 + (weight_grams - 1_000) / 10
    }
    }
    #[cfg(test)]
    mod tests {
    use super::*;
    #[test]
    fn light_parcels_pay_the_base_price() {
    assert_eq!(shipping_cents(0), 500);
    assert_eq!(shipping_cents(1_000), 500); // the boundary is included
    }
    #[test]
    fn heavy_parcels_pay_one_cent_per_10_grams() {
    assert_eq!(shipping_cents(1_010), 501);
    assert_eq!(shipping_cents(3_000), 700);
    }
    #[test]
    fn the_largest_weight_does_not_overflow() {
    assert_eq!(shipping_cents(u32::MAX), 500 + (u32::MAX - 1_000) / 10);
    }
    }
    Run in Playground (opens the Rust Playground in a new tab)

The “worse” test passes for almost any wrong implementation: a price of 1 cent, a formula with the wrong rate, or a limit at 999 grams. It is the kind of test that an agent writes to make a coverage number go up.

The “better” tests pin exact values, test both sides of the 1 000 gram boundary, and check that the largest u32 does not overflow. Each test name says the rule that it checks, so a failure explains itself.

  1. Why can a Rust unit test call a private function?

    Show the answer

    The test module is a child module of the code (mod tests in the same file), and child modules can see the private items of their parents.

  2. What does #[cfg(test)] on the test module do?

    Show the answer

    It compiles the module only for cargo test. The tests and their helpers are not in normal builds.

  3. An agent wrote #[should_panic] on a test for a function that returns Result. Is that right?

    Show the answer

    Usually not. A function that returns Result reports errors as Err values. The test should check the Err with assert_eq! or matches!. should_panic passes for any panic, including an unrelated bug.

  4. Where do integration tests go, and what can they use?

    Show the answer

    In the tests/ folder next to src/. Each file is a separate crate that can use only the public API of the library, or run the binary.