docs(adr): ADR-0019: Testing Strategy

ADR-0019: Testing Strategy

Status

Accepted

Context

mdbook-htmx is a complex plugin that transforms MDBook output into HTMX-powered documentation. A comprehensive testing strategy is required to ensure correctness, prevent regressions, and maintain quality as the codebase evolves.

Testing challenges include:

• Rust-based renderer with multiple modules
• Tera template rendering
• HTMX-specific behavior
• CSS theming and styling
• Integration with MDBook's build system
• Various deployment targets (static, Cloudflare Workers, Kubernetes)

Decision Drivers

1. Confidence - High test coverage for critical paths
2. Speed - Fast feedback during development
3. Maintenance - Tests should be easy to update
4. Realism - Tests should reflect actual usage
5. Documentation - Tests serve as living documentation

Decision

Implement a multi-layer testing strategy with unit, integration, snapshot, and end-to-end tests.

Test Pyramid

         ╱╲
        ╱  ╲
       ╱ E2E╲         Few, slow, high-confidence
      ╱──────╲
     ╱Snapshot╲       Template output verification
    ╱──────────╲
   ╱ Integration╲     Module interactions
  ╱──────────────╲
 ╱    Unit Tests  ╲   Fast, isolated, many
╱──────────────────╲

Layer 1: Unit Tests

Test individual functions and modules in isolation.

// tests/unit/frontmatter_test.rs
use mdbook_htmx::frontmatter::{Frontmatter, parse_frontmatter};

#[test]
fn parse_valid_frontmatter() {
    let content = r#"---
title: Test Page
scope: internal
---
# Content"#;

    let (frontmatter, body) = parse_frontmatter(content).unwrap();

    assert_eq!(frontmatter.title, Some("Test Page".to_string()));
    assert_eq!(frontmatter.scope, Some("internal".to_string()));
    assert_eq!(body.trim(), "# Content");
}

#[test]
fn parse_empty_frontmatter() {
    let content = "# No Frontmatter";
    let (frontmatter, body) = parse_frontmatter(content).unwrap();

    assert!(frontmatter.is_empty());
    assert_eq!(body, content);
}

#[test]
fn parse_invalid_yaml_returns_error() {
    let content = r#"---
title: [unclosed
---"#;

    let result = parse_frontmatter(content);
    assert!(result.is_err());
}

// tests/unit/manifest_test.rs
use mdbook_htmx::manifest::{Manifest, ManifestEntry};

#[test]
fn manifest_version_validation() {
    let manifest = Manifest::new("1.0.0");
    assert!(manifest.is_compatible_with("1.0.0"));
    assert!(manifest.is_compatible_with("1.0.1"));
    assert!(!manifest.is_compatible_with("2.0.0"));
}

#[test]
fn manifest_entry_ordering() {
    let mut manifest = Manifest::new("1.0.0");
    manifest.add_entry(ManifestEntry::new("/guide/intro.html", 2));
    manifest.add_entry(ManifestEntry::new("/guide/getting-started.html", 1));

    let entries: Vec<_> = manifest.entries().collect();
    assert_eq!(entries[0].path, "/guide/getting-started.html");
    assert_eq!(entries[1].path, "/guide/intro.html");
}

Layer 2: Integration Tests

Test module interactions and the rendering pipeline.

// tests/integration/renderer_test.rs
use mdbook_htmx::HtmxRenderer;
use mdbook::MDBook;
use tempfile::tempdir;

#[test]
fn render_simple_book() {
    let book_dir = tempdir().unwrap();
    setup_test_book(book_dir.path(), TestBookConfig::simple());

    let mdbook = MDBook::load(book_dir.path()).unwrap();
    let renderer = HtmxRenderer::default();

    let result = renderer.render(&mdbook.root, &mdbook.book, &mdbook.config);

    assert!(result.is_ok());
    assert!(book_dir.path().join("book/index.html").exists());
}

#[test]
fn render_with_frontmatter_scope() {
    let book_dir = tempdir().unwrap();
    setup_test_book(book_dir.path(), TestBookConfig::with_scopes());

    let mdbook = MDBook::load(book_dir.path()).unwrap();
    let renderer = HtmxRenderer::default();

    renderer.render(&mdbook.root, &mdbook.book, &mdbook.config).unwrap();

    // Verify scope is preserved in manifest
    let manifest: Manifest = read_manifest(book_dir.path().join("book/manifest.json"));
    let internal_page = manifest.get("/internal/secrets.html").unwrap();
    assert_eq!(internal_page.scope, Some("internal".to_string()));
}

#[test]
fn render_generates_search_index() {
    let book_dir = tempdir().unwrap();
    setup_test_book(book_dir.path(), TestBookConfig::with_search());

    let mdbook = MDBook::load(book_dir.path()).unwrap();
    let renderer = HtmxRenderer::default();

    renderer.render(&mdbook.root, &mdbook.book, &mdbook.config).unwrap();

    let search_index = book_dir.path().join("book/search-index.json");
    assert!(search_index.exists());

    let index: SearchIndex = serde_json::from_reader(
        std::fs::File::open(&search_index).unwrap()
    ).unwrap();

    assert!(index.pages.len() > 0);
}

Layer 3: Snapshot Tests

Verify template output matches expected results using insta.

// tests/snapshot/template_test.rs
use insta::assert_snapshot;
use mdbook_htmx::templates::render_page;

#[test]
fn snapshot_page_layout() {
    let context = PageContext {
        title: "Getting Started".to_string(),
        content: "<p>Welcome to the guide.</p>".to_string(),
        breadcrumbs: vec![
            Breadcrumb::new("/", "Home"),
            Breadcrumb::new("/guide/", "Guide"),
        ],
        ..Default::default()
    };

    let html = render_page(&context).unwrap();
    assert_snapshot!("page_layout", html);
}

#[test]
fn snapshot_navigation() {
    let nav = NavigationContext {
        chapters: vec![
            Chapter::new("Introduction", "/intro.html", false),
            Chapter::new("Getting Started", "/getting-started.html", true),
            Chapter::new("Advanced", "/advanced.html", false),
        ],
    };

    let html = render_navigation(&nav).unwrap();
    assert_snapshot!("navigation", html);
}

#[test]
fn snapshot_search_results() {
    let results = SearchResults {
        query: "authentication".to_string(),
        hits: vec![
            SearchHit::new("/auth/login.html", "Login", "How to implement <mark>authentication</mark>..."),
            SearchHit::new("/auth/oauth.html", "OAuth", "OAuth <mark>authentication</mark> flow..."),
        ],
        total: 2,
    };

    let html = render_search_results(&results).unwrap();
    assert_snapshot!("search_results", html);
}

#[test]
fn snapshot_error_pages() {
    let contexts = vec![
        ("401", ErrorContext::unauthorized()),
        ("403", ErrorContext::forbidden()),
        ("404", ErrorContext::not_found("/missing")),
        ("500", ErrorContext::internal_error("DB connection failed")),
    ];

    for (name, ctx) in contexts {
        let html = render_error(&ctx).unwrap();
        assert_snapshot!(format!("error_{}", name), html);
    }
}

Snapshot directory structure:

tests/snapshots/
├── template_test__page_layout.snap
├── template_test__navigation.snap
├── template_test__search_results.snap
├── template_test__error_401.snap
├── template_test__error_403.snap
├── template_test__error_404.snap
└── template_test__error_500.snap

Layer 4: End-to-End Tests

Test complete workflows including build and serve.

// tests/e2e/build_test.rs
use assert_cmd::Command;
use predicates::prelude::*;

#[test]
fn build_example_book() {
    let mut cmd = Command::cargo_bin("mdbook").unwrap();

    cmd.arg("build")
       .arg("tests/fixtures/example-book")
       .assert()
       .success()
       .stdout(predicate::str::contains("Rendering with htmx backend"));
}

#[test]
fn build_fails_with_invalid_frontmatter() {
    let mut cmd = Command::cargo_bin("mdbook").unwrap();

    cmd.arg("build")
       .arg("tests/fixtures/invalid-frontmatter")
       .assert()
       .failure()
       .stderr(predicate::str::contains("Invalid frontmatter"));
}

#[test]
fn watch_mode_rebuilds_on_change() {
    let book_dir = setup_temp_book();

    // Start watch in background
    let mut child = Command::cargo_bin("mdbook")
        .unwrap()
        .arg("watch")
        .arg(&book_dir)
        .spawn()
        .unwrap();

    // Wait for initial build
    wait_for_file(book_dir.join("book/index.html"), Duration::from_secs(10));

    // Modify a file
    std::fs::write(book_dir.join("src/chapter.md"), "# Updated").unwrap();

    // Wait for rebuild
    sleep(Duration::from_millis(500));

    let content = std::fs::read_to_string(book_dir.join("book/chapter.html")).unwrap();
    assert!(content.contains("Updated"));

    child.kill().unwrap();
}

Layer 5: HTMX Behavior Tests

Test HTMX interactions using headless browser (playwright-rust or similar).

// tests/e2e/htmx_test.rs
use playwright::Playwright;

#[tokio::test]
async fn navigation_uses_htmx_swap() {
    let playwright = Playwright::initialize().await.unwrap();
    let browser = playwright.chromium().launch(Default::default()).await.unwrap();
    let page = browser.new_page(Default::default()).await.unwrap();

    // Navigate to docs
    page.goto("http://localhost:8080/").await.unwrap();

    // Click a navigation link
    page.click("a[href='/guide/getting-started.html']").await.unwrap();

    // Verify HTMX swapped content (not full page load)
    let requests = page.evaluate("window.htmxRequestCount").await.unwrap();
    assert!(requests.as_u64().unwrap() > 0);

    // Verify URL updated
    assert_eq!(page.url().await.unwrap(), "http://localhost:8080/guide/getting-started.html");

    // Verify content updated
    let heading = page.inner_text("h1").await.unwrap();
    assert_eq!(heading, "Getting Started");
}

#[tokio::test]
async fn search_works_with_debounce() {
    let page = setup_page().await;
    page.goto("http://localhost:8080/").await.unwrap();

    // Type in search
    page.fill("#search-input", "auth").await.unwrap();

    // Wait for debounce
    page.wait_for_selector(".search-results", Default::default()).await.unwrap();

    // Verify results appear
    let results = page.query_selector_all(".search-result").await.unwrap();
    assert!(results.len() > 0);
}

#[tokio::test]
async fn theme_toggle_persists() {
    let page = setup_page().await;
    page.goto("http://localhost:8080/").await.unwrap();

    // Toggle to dark theme
    page.click("#theme-toggle").await.unwrap();

    // Verify localStorage
    let theme = page.evaluate("localStorage.getItem('theme')").await.unwrap();
    assert_eq!(theme.as_str().unwrap(), "dark");

    // Reload and verify persistence
    page.reload(Default::default()).await.unwrap();

    let body_class = page.get_attribute("body", "class").await.unwrap();
    assert!(body_class.unwrap().contains("dark"));
}

Test Fixtures

Book Fixtures

Create reusable test book configurations:

// tests/fixtures/mod.rs
pub enum TestBookConfig {
    Simple,
    WithScopes,
    WithSearch,
    WithAuth,
    Large,
    Multilingual,
}

impl TestBookConfig {
    pub fn setup(&self, dir: &Path) -> Result<()> {
        match self {
            TestBookConfig::Simple => setup_simple_book(dir),
            TestBookConfig::WithScopes => setup_scoped_book(dir),
            // ...
        }
    }
}

fn setup_simple_book(dir: &Path) -> Result<()> {
    fs::write(dir.join("book.toml"), r#"
[book]
title = "Test Book"
authors = ["Test Author"]

[output.htmx]
    "#)?;

    fs::create_dir_all(dir.join("src"))?;
    fs::write(dir.join("src/SUMMARY.md"), r#"
# Summary

- [Introduction](./intro.md)
- [Chapter 1](./chapter1.md)
    "#)?;

    fs::write(dir.join("src/intro.md"), "# Introduction\n\nWelcome!")?;
    fs::write(dir.join("src/chapter1.md"), "# Chapter 1\n\nContent here.")?;

    Ok(())
}

Snapshot Fixtures

Store expected outputs:

tests/fixtures/
├── books/
│   ├── simple/
│   ├── with-scopes/
│   └── with-search/
└── expected/
    ├── simple/
    │   ├── index.html
    │   ├── manifest.json
    │   └── search-index.json
    └── with-scopes/
        └── manifest.json

Configuration

Cargo.toml

[dev-dependencies]
# Unit testing
proptest = "1.4"
rstest = "0.18"

# Snapshot testing
insta = { version = "1.34", features = ["yaml", "redactions"] }

# Integration testing
tempfile = "3.9"
assert_cmd = "2.0"
predicates = "3.0"

# E2E testing
playwright = { git = "https://github.com/nicktrav/playwright-rust" }
tokio = { version = "1", features = ["full", "test-util"] }

# Test utilities
pretty_assertions = "1.4"
test-case = "3.3"

CI Configuration

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  unit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo test --lib

  integration:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo test --test integration_*

  snapshot:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo insta test --accept
      - run: git diff --exit-code

  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - name: Install Playwright
        run: npx playwright install chromium
      - name: Build and serve
        run: |
          cargo build --release
          ./target/release/mdbook-htmx serve tests/fixtures/example-book &
          sleep 5
      - run: cargo test --test e2e_*

  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: llvm-tools-preview
      - uses: taiki-e/install-action@cargo-llvm-cov
      - run: cargo llvm-cov --all-features --lcov --output-path lcov.info
      - uses: codecov/codecov-action@v3
        with:
          files: lcov.info

Coverage Requirements

Area	Minimum Coverage
Core renderer	80%
Frontmatter parsing	90%
Template rendering	70% (snapshot tests)
Error handling	85%
Overall	75%

Consequences

Positive

• High confidence in correctness
• Fast feedback during development
• Regression prevention
• Living documentation

Negative

• Initial test setup takes time
• Snapshot tests need maintenance
• E2E tests are slower
• Headless browser adds complexity

Mitigation

• Use test fixtures for quick setup
• Automate snapshot updates in CI
• Run E2E tests only on PR/main
• Cache browser binaries in CI

Alternatives Considered

Manual Testing Only

Rely on manual QA for validation.

Rejected because:

• Slow and error-prone
• No regression prevention
• Doesn't scale

Only Unit Tests

Skip integration and E2E tests.

Rejected because:

• Misses interaction bugs
• Can't verify HTMX behavior
• Templates need rendering tests

Property-Based Testing Only

Use proptest/quickcheck for everything.

Rejected because:

• Harder to reason about failures
• Not suitable for all scenarios
• Complementary, not replacement

References

• Rust Testing Book
• insta Snapshot Testing
• assert_cmd
• Playwright for Rust
• Test Pyramid (Martin Fowler)