Why Code Coverage Shouldn't Be the Main Goal of Unit Testing

There’s one number that almost always comes up in software quality discussions: code coverage. Engineering teams set targets of 80%, 90%, even 100%. CI pipelines are configured to reject pull requests that lower that percentage. Coverage reports are displayed on dashboards as proof that “we’re testing properly.” At a glance, this looks like mature engineering practice. But there’s a fundamental problem behind this approach — a problem that isn’t visible until you start reading the tests one by one.

Code coverage measures whether a line of code is executed, not whether the code is correct. The difference between these two phrases is a very deep chasm, and understanding that chasm will change how you write and evaluate unit tests forever.

What Code Coverage Actually Measures

Before talking about the problem, it’s important to understand exactly what code coverage measures and — far more importantly — what it doesn’t measure.

Code coverage reports the proportion of code executed while the test suite runs. There are several types of coverage commonly used:

What you need to note: not one of all these coverage types measures whether the result of code execution is correct. Coverage only answers: “was this code executed?” It doesn’t answer: “did this code produce the correct output?”

This isn’t a weakness of a particular measurement tool. It’s a fundamental limitation of the coverage concept itself.

# Function to be tested
def calculate_discount(price: float, member: bool) -> float:
    if member:
        return price * 0.9
    return price


# The following test achieves 100% branch coverage
def test_calculate_discount():
    calculate_discount(100_000, True)   # executes the member=True branch
    calculate_discount(100_000, False)  # executes the member=False branch

The test above achieves 100% branch coverage. But notice — there isn’t a single assert. We never verify that the 10% discount is calculated correctly. If there’s a bug in the formula — for example price * 0.99 or price - 0.9 — this test won’t catch it. 100% coverage, zero confidence.

The assertion-less coverage trap is one of the most dangerous anti-patterns in unit testing, precisely because it doesn’t look dangerous. The coverage report looks green, CI passes, but the system isn’t truly validated.

Goodhart’s Law and Why Coverage Breaks When Made a Target

There’s a principle formulated by the economist Charles Goodhart:

“When a measure becomes a target, it ceases to be a good measure.”

This principle was originally used to critique monetary policy, but it applies perfectly to code coverage in software engineering.

When coverage is still an observation — something you monitor while writing meaningful tests — it provides useful information. You can see “oh, there’s an important function here that’s never been tested at all.” That’s valuable information.

But when coverage becomes a target — something that must be achieved before you can merge into the main branch — team behavior changes. And the change isn’t in a better direction.

flowchart TD
    A[Coverage Made a Target] --> B[Pressure to Raise the Number]
    B --> C{Fastest Way?}
    C --> D[Write tests without meaningful assertions]
    C --> E[Test trivial code that poses no risk]
    C --> F[Avoid complex scenarios that are hard to test]
    D --> G[Coverage rises ✓]
    E --> G
    F --> H[Important edge cases untested]
    G --> I[False Confidence]
    H --> I
    I --> J[Bugs slip into production]

This is a real cycle that happens in many teams. As soon as there’s a coverage KPI or gate, incentives shift from “write tests that catch bugs” to “write tests that raise the number.” These two goals look the same, but their results are very different.

Five Anti-Patterns That Appear When Coverage Is Made a Goal

1. Tests Without Meaningful Assertions

This is the most direct anti-pattern. Tests run to execute code, but their assertions are too weak or don’t exist at all.

# ANTI-PATTERN: the test runs, coverage rises, but nothing is verified
def test_process_payment_anti_pattern():
    payment = PaymentService()
    result = payment.process(order_id="ORD-001", amount=500_000)
    assert result is not None  # this is almost always True; useless


# CORRECT: verify specific and meaningful behavior
def test_process_payment_success():
    payment = PaymentService(gateway=MockGateway(success=True))
    result = payment.process(order_id="ORD-001", amount=500_000)

    assert result.status == PaymentStatus.SUCCESS
    assert result.transaction_id is not None
    assert result.amount_charged == 500_000
    assert result.timestamp is not None


def test_process_payment_fails_insufficient_funds():
    payment = PaymentService(gateway=MockGateway(success=False, error_code="INSUFFICIENT_FUNDS"))
    
    with pytest.raises(InsufficientFundsError) as exc_info:
        payment.process(order_id="ORD-001", amount=500_000)
    
    assert "ORD-001" in str(exc_info.value)

2. Testing Trivial Code to Raise the Number

Instead of spending time on difficult but important tests, developers are tempted to test simple getters, setters, and constructors that can hardly ever be wrong.

// ANTI-PATTERN: testing getters/setters that have no logic
@Test
void testGetName() {
    User user = new User();
    user.setName("Alice");
    assertEquals("Alice", user.getName()); // this almost never fails
}

@Test
void testGetEmail() {
    User user = new User();
    user.setEmail("[email protected]");
    assertEquals("[email protected]", user.getEmail());
}

// CORRECT: focus on logic that can actually be wrong
@Test
void testInvalidEmailFormatValidation() {
    assertThrows(InvalidEmailException.class, () -> {
        new User().setEmail("not-an-email");
    });
}

@Test
void testDisallowedEmailDomain() {
    assertThrows(DisallowedDomainException.class, () -> {
        new User().setEmail("[email protected]");
    });
}

@Test
void testEmailNormalizedToLowercase() {
    User user = new User();
    user.setEmail("[email protected]");
    assertEquals("[email protected]", user.getEmail());
}

3. Only Testing the Happy Path

When teams chase coverage the fastest way, they tend to execute only the success path. One test per function is enough to raise line coverage significantly. Edge cases and error handling — which are often the actual source of production bugs — get ignored.

// ANTI-PATTERN: only testing the success scenario
func TestSendEmail(t *testing.T) {
    service := NewEmailService(MockSMTP{})
    err := service.Send("[email protected]", "Hello", "Message body")
    if err != nil {
        t.Fatal(err)
    }
    // SendEmail function coverage: 60%+ from just this one test
}

// CORRECT: test all scenarios that can happen in production
func TestSendEmail_Success(t *testing.T) {
    mock := &MockSMTP{ShouldSucceed: true}
    service := NewEmailService(mock)
    err := service.Send("[email protected]", "Hello", "Message body")
    assert.NoError(t, err)
    assert.Equal(t, 1, mock.SendCallCount)
}

func TestSendEmail_EmptyAddress(t *testing.T) {
    service := NewEmailService(&MockSMTP{})
    err := service.Send("", "Hello", "Message body")
    assert.ErrorIs(t, err, ErrEmptyAddress)
}

func TestSendEmail_SMTPTimeout(t *testing.T) {
    mock := &MockSMTP{ReturnError: ErrTimeout}
    service := NewEmailService(mock)
    err := service.Send("[email protected]", "Hello", "Message body")
    assert.ErrorIs(t, err, ErrSMTPTimeout)
}

func TestSendEmail_RetryAfterFailure(t *testing.T) {
    mock := &MockSMTP{FailFirstN: 2, ShouldSucceed: true}
    service := NewEmailService(mock)
    err := service.Send("[email protected]", "Hello", "Message body")
    assert.NoError(t, err)
    assert.Equal(t, 3, mock.SendCallCount) // 2 failures + 1 success
}

4. Tests Too Tied to the Implementation

To make sure every line is executed, developers sometimes write tests that check internal implementation details — not external behavior. Such tests are very fragile and break on refactors, even refactors that don’t change behavior at all.

# ANTI-PATTERN: the test checks internal implementation details
def test_final_price_calculation():
    service = PriceService()
    
    # This test assumes a specific internal implementation
    with patch.object(service, '_get_exchange_rate') as mock_rate:
        with patch.object(service, '_calculate_tax') as mock_tax:
            mock_rate.return_value = 15_000
            mock_tax.return_value = 110_000
            
            result = service.calculate_final_price(product_id="P001")
            
            # Verify internal methods were called
            mock_rate.assert_called_once()
            mock_tax.assert_called_once_with(100_000, rate=15_000)


# CORRECT: the test verifies a real external contract
def test_final_price_calculation_with_tax():
    # Mock external dependencies (not internal ones)
    mock_product_repo = MockProductRepository(price_usd=10)
    mock_rate_api = MockRateAPI(idr_rate=15_000)
    
    service = PriceService(product_repo=mock_product_repo, rate_api=mock_rate_api)
    result = service.calculate_final_price(product_id="P001")
    
    # Verify the output, not how it was achieved
    assert result.base_price == 150_000  # 10 USD * 15,000
    assert result.tax == 16_500          # 11% of 150,000
    assert result.total == 166_500       # base + tax

5. Avoiding Hard Code with Exclusion Flags

When there’s a genuinely hard-to-test part of the code — for example error handling for rare conditions, or complex integration — developers sometimes take a shortcut: marking that code with an exclusion flag so it isn’t counted in coverage.

# ANTI-PATTERN: escaping hard tests with an exclude flag
def handle_database_connection_error(error):  # pragma: no cover
    # complex logic for recovering from DB failures
    logger.critical(f"DB down: {error}")
    notify_ops_team(error)
    trigger_circuit_breaker()
    # ... 50 lines of recovery logic

# This might be the MOST IMPORTANT code to test,
# because it runs when the system is in a critical state

Exclusion flags do have legitimate use cases — for example code that genuinely can’t be unit tested (platform-specific code, debug utilities). But when they’re used to escape hard tests in order to maintain the coverage number, that’s a real form of metric manipulation.

False Confidence: The Invisible Danger

Of all the problems caused by coverage obsession, the most dangerous is false confidence — a sense of security with no basis.

Imagine a payment system with 95% coverage. The team feels confident. Every PR is verified not to lower this number. The dashboard is always green. Then one day a bug appears in production: tax calculation for bundle products produces a negative number under certain conditions.

It turns out the existing tests do execute the tax calculation function — its coverage is 100%. But not a single test covers the combination of bundle products with discounts and taxes at once. The code lines run, but the critical scenario was never validated.

sequenceDiagram
    participant Dev as Developer
    participant CI as CI Pipeline
    participant Prod as Production

    Dev->>CI: Push code (coverage 95%)
    CI-->>Dev: ✓ Coverage OK, merge allowed
    Dev->>Prod: Deploy
    Note over Prod: Bug: negative tax calculation for bundle products
    Prod-->>Dev: 🔥 User complaints
    Dev->>Dev: Investigation: coverage is high, but this scenario was never tested

This isn’t a hypothetical scenario. It’s a common occurrence. High coverage sends the signal that “we’ve tested the codebase well,” when in reality we’ve only ensured every line was executed at some point — not that every important scenario was ever validated.

100% coverage doesn’t guarantee zero bugs. It only guarantees that every line of code was executed by the test suite at some point. Two very different things. A system can have perfect coverage and still have critical bugs in unimagined edge cases.

The Hidden Cost: Wasted Engineer Time

There’s another aspect often missing from the discussion: the economic cost of chasing coverage as a target.

Engineer time is a very limited resource. Every hour spent writing getter/setter tests to raise coverage from 78% to 80% is an hour not spent on:

• Testing complex business edge cases
• Writing tests for high-risk new features
• Fixing existing tests with weak assertions
• Doing exploratory testing to find unexpected bugs

This is a real opportunity cost. And ironically, by chasing the coverage number, teams often miss the highest-risk parts — because those parts are hard to test and need more time than just adding trivial tests.

flowchart LR
    subgraph Time["Time Allocation (Limited)"]
        direction TB
        W1[Write tests for getters/setters]
        W2[Write tests without meaningful assertions]
        W3[Test important business edge cases]
        W4[Test complex error handling]
    end
    
    subgraph Target["If Coverage Is the Target"]
        T1["✓ Priority (raise the number)"]
        T2["✓ Priority (raise the number)"]
        T3["✗ De-prioritized (hard, takes time)"]
        T4["✗ De-prioritized (hard, takes time)"]
    end
    
    subgraph Ideal["If Coverage Is a Guide"]
        I1["✗ Skip (low risk)"]
        I2["✗ Skip (meaningless)"]
        I3["✓ Top priority"]
        I4["✓ Top priority"]
    end
    
    W1 --- T1
    W2 --- T2
    W3 --- T3
    W4 --- T4
    
    W1 --- I1
    W2 --- I2
    W3 --- I3
    W4 --- I4

Meaningful Tests: What Should Actually Be Measured

If not coverage, what should be the quality indicator for unit tests? This question has no single answer, but there are several more meaningful dimensions:

Mutation Score

Mutation testing is a technique where a tool automatically creates “broken” versions of your code — for example changing > to >=, removing a condition, or replacing a return value. Then it checks whether your test suite catches those “mutants.”

# Original code
def is_adult(age: int) -> bool:
    return age >= 18

# Mutations the tool creates:
# Mutant 1: return age > 18   (boundary condition changed)
# Mutant 2: return age <= 18  (logic inverted)
# Mutant 3: return True        (condition removed)
# Mutant 4: return age >= 17  (threshold value changed)

# A meaningful test must KILL all these mutants
def test_is_adult():
    assert is_adult(18) == True   # kills Mutant 1 and 4
    assert is_adult(17) == False  # kills Mutant 2, 3, and 4
    assert is_adult(0) == False   # kills Mutant 3
    assert is_adult(100) == True  # sanity check

The mutation score (percentage of mutants “killed” by the tests) is far more meaningful than ordinary coverage, because it measures the tests’ ability to detect wrong changes — not just their ability to execute code.

Behaviour Coverage, Not Line Coverage

A healthier way of thinking is focusing on behaviour coverage: for every behavior the system should exhibit, is there a test verifying it?

Function: validate_password(password: str) -> ValidationResult

Behaviors that should be tested:
  ✓ Password shorter than 8 characters → FAIL
  ✓ Password without an uppercase letter → FAIL  
  ✓ Password without a digit → FAIL
  ✓ Password without a special character → FAIL
  ✓ Password meeting all requirements → SUCCESS
  ✓ Password with unicode/emoji characters → (what behavior is expected?)
  ✓ Password that is an empty string → FAIL
  ✓ Very long password (>1000 characters) → (is there a limit?)

When you map expected behaviors like this, you can see which tests are still missing without even glancing at a coverage report.

Confidence During Refactoring

One of the best indicators of test suite quality is: how confident is the team when refactoring? If every small refactor makes many tests fail — even when external behavior doesn’t change — that’s a sign the tests are too tied to the implementation. If the team fears refactoring because “something could break without being noticed” — that’s a sign the tests don’t really capture important behaviors.

A good test suite provides a net that catches real bugs without exploding when you do legitimate refactors.

How to Use Coverage Healthily

Coverage isn’t the enemy. It’s a useful tool when used correctly. Here’s how to position it healthily:

Coverage as Detection, Not a Target

Use coverage reports to find blind spots — areas of code that have never been executed at all. Those areas are candidates for adding tests, but with the right question: “does this area have important business logic that needs validation?” not “how do I execute this line as fast as possible?”

flowchart TD
    A[Run the test suite] --> B[Look at the coverage report]
    B --> C{Any areas completely uncovered?}
    C -- No --> D[Continue, coverage is just a guide]
    C -- Yes --> E[Evaluate that area]
    E --> F{Is there important logic here?}
    F -- Yes --> G[Write meaningful tests to validate behavior]
    F -- No --> H[Note it, but no need to prioritize]
    G --> A

Avoid Rigid Coverage Gates

A coverage gate (CI that fails when coverage drops) can be useful, but with an important caveat: it prevents declines in coverage, not meaningless tests. Someone can still add tests without sufficient assertions and still pass the gate.

If you use a coverage gate, make sure it’s accompanied by a culture and review process that checks test quality, not just the number.

Set Priority Areas Based on Risk

Not all code has the same risk. Core business logic — price calculations, authentication, transaction handling — is far more critical than simple utility helpers. Allocate testing energy based on risk, not on how easy the code is to test.

High Priority Areas (must be deeply tested):
  ✓ Financial calculations (price, tax, discount, interest)
  ✓ Authentication and authorization logic
  ✓ User input validation
  ✓ State machines (order flow, payment status)
  ✓ Error handling for critical conditions

Medium Priority Areas:
  ✓ Data transformation / mapping
  ✓ Integration with external services
  ✓ Caching logic

Low Priority Areas (no exhaustive tests needed):
  ✗ Getters and setters without logic
  ✗ Configuration and constants
  ✗ Debug utilities
  ✗ Generated scaffolding code

Make Coverage a Discussion Topic, Not a Gatekeeper

Coverage is most effective when used in retrospectives or code reviews as discussion material: “coverage in the payment module dropped from 85% to 72% after this PR — did any important area get missed?” That question is far more productive than automatically rejecting a PR because the number dropped.

Coverage and TDD: A Often Misunderstood Relationship

Test-Driven Development (TDD) is often associated with coverage, even though the two have different philosophies.

In TDD, you write tests before writing code. The process is:

1. Write a failing test for one behavior
2. Write the minimum code to make the test pass
3. Refactor
4. Repeat

As a result, coverage becomes a natural side effect of TDD — not something chased separately. Code written through TDD almost certainly has high coverage, but more importantly, every line of existing code has a meaningful test behind it because the code was written to make those tests pass.

sequenceDiagram
    participant Dev as Developer
    participant Test as Test
    participant Code as Code

    Note over Dev: TDD Flow
    Dev->>Test: Write a test for behavior "X"
    Test-->>Dev: ✗ Fails (code doesn't exist yet)
    Dev->>Code: Write minimal code so X passes
    Code-->>Test: ✓ Passes
    Dev->>Code: Refactor safely
    Note over Test,Code: Coverage is a side effect, not the goal

This is fundamentally different from “write code first, then chase 80% coverage.” In the second approach, coverage is a separately chased goal, and the result is often tests that trail the code — not tests that define the behavioral contract.

Case Study: High Coverage, Bug in Production

Here’s a scenario that illustrates the problem concretely.

A team is building a promo feature. There’s a function to calculate the total price after a promo is applied:

// Production code
fun calculatePriceAfterPromo(
    originalPrice: Long,
    discountPercent: Int,
    maxDiscount: Long
): Long {
    val discount = originalPrice * discountPercent / 100
    return if (discount > maxDiscount) {
        originalPrice - maxDiscount
    } else {
        originalPrice - discount
    }
}

The team writes tests:

// ANTI-PATTERN: the existing tests only cover the happy path
@Test
fun testCalculatePriceAfterPromo() {
    // Test 1: discount doesn't exceed the limit
    assertEquals(90_000L, calculatePriceAfterPromo(100_000L, 10, 50_000L))
    
    // Test 2: discount exceeds the limit
    assertEquals(50_000L, calculatePriceAfterPromo(100_000L, 70, 50_000L))
}
// Coverage: 100% - all branches covered

100% coverage. All branches covered. CI passes. The code is deployed.

Then a report comes in: a user got a negative price. It turns out there’s an edge case never tested:

// Bug found by a user: 100% discount with a small maxDiscount
calculatePriceAfterPromo(100_000L, 100, 10_000L)
// discount = 100_000 * 100 / 100 = 100_000
// discount > maxDiscount (100_000 > 10_000) → true
// return originalPrice - maxDiscount = 100_000 - 10_000 = 90_000
// Should be free (price = 0), but the user is still charged!

// Another edge case: very large originalPrice → integer overflow
calculatePriceAfterPromo(Long.MAX_VALUE, 10, 1_000_000L)
// originalPrice * discountPercent could overflow!

Meaningful tests should have included these scenarios:

// CORRECT: tests covering business edge cases
@Test
fun testCalculatePriceAfterPromo_FullDiscount() {
    // 100% discount, no maxDiscount
    assertEquals(0L, calculatePriceAfterPromo(100_000L, 100, Long.MAX_VALUE))
}

@Test
fun testCalculatePriceAfterPromo_PriceNeverNegative() {
    // Make sure the price is never negative
    val result = calculatePriceAfterPromo(100_000L, 100, 10_000L)
    assertTrue(result >= 0, "Price must not be negative, but got: $result")
}

@Test
fun testCalculatePriceAfterPromo_ZeroDiscount() {
    assertEquals(100_000L, calculatePriceAfterPromo(100_000L, 0, 50_000L))
}

@Test
fun testCalculatePriceAfterPromo_ZeroPrice() {
    assertEquals(0L, calculatePriceAfterPromo(0L, 50, 50_000L))
}

These scenarios wouldn’t raise coverage much because the same branches are already covered. But they’re the tests that genuinely protect the system from real bugs.

Summary

• Coverage measures execution, not correctness — a line of code being executed doesn’t mean the code produces the correct output.
• Goodhart’s Law applies perfectly here — once coverage becomes a target, it stops being a good measure because team behavior shifts to manipulating the number, not improving quality.
• Five main anti-patterns appear when coverage is made a goal: tests without meaningful assertions, testing trivial code, only happy paths, being too tied to the implementation, and avoiding hard code with exclusion flags.
• False confidence is the biggest danger — high coverage provides an unfounded sense of security, which is actually more dangerous than having no tests at all because the team stops being vigilant.
• Engineer time is a limited resource — chasing the coverage number sacrifices time that should be allocated to tests that genuinely protect the system.
• Coverage should be a guide, not a goal — use coverage reports to find blind spots and start discussions, not as an automatic gatekeeper determining quality.
• Behaviour coverage is more meaningful — map all the behaviors the system should exhibit, then make sure every behavior is validated by a test.
• Good tests are measured by confidence — how confident the team feels during refactoring is a far more honest quality indicator for a test suite than a coverage percentage.