docs: add comprehensive project wiki for v1.0

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-06-04 14:07:55 -06:00
parent a758e8e0bc
commit 0166fb6511
32 changed files with 1483 additions and 0 deletions
@@ -0,0 +1,50 @@
+{
+  "generatedAt": "2026-06-04T13:55:00Z",
+  "commitHash": "753928239e5a86414f6a6912b908da4cf0960231",
+  "branch": "master",
+  "pageCount": 30,
+  "topLevelSections": [
+    "overview",
+    "by-the-numbers",
+    "lore",
+    "how-to-contribute",
+    "applications",
+    "systems",
+    "features",
+    "security",
+    "deployment",
+    "reference"
+  ],
+  "pageOrder": [
+    "overview/index.md",
+    "overview/architecture.md",
+    "overview/getting-started.md",
+    "overview/glossary.md",
+    "by-the-numbers.md",
+    "lore.md",
+    "how-to-contribute/index.md",
+    "how-to-contribute/development-workflow.md",
+    "how-to-contribute/testing.md",
+    "how-to-contribute/debugging.md",
+    "how-to-contribute/patterns-and-conventions.md",
+    "how-to-contribute/tooling.md",
+    "applications/index.md",
+    "applications/rustunnel-cli.md",
+    "systems/index.md",
+    "systems/tunnel-engine.md",
+    "systems/tls-stack.md",
+    "systems/framing-protocol.md",
+    "systems/socks5-proxy.md",
+    "systems/credential-generation.md",
+    "features/index.md",
+    "features/connection-keys.md",
+    "features/stream-multiplexing.md",
+    "features/reconnect-and-shutdown.md",
+    "security.md",
+    "deployment.md",
+    "reference/index.md",
+    "reference/configuration.md",
+    "reference/data-models.md",
+    "reference/dependencies.md"
+  ]
+}
@@ -0,0 +1,3 @@
+# Applications
+
+rustunnel ships as a single binary application with five CLI commands. There are no separate services, daemons, or web frontends.
@@ -0,0 +1,57 @@
+# rustunnel CLI
+
+The rustunnel binary exposes five subcommands defined in `src/cli.rs` and dispatched in `src/main.rs`.
+
+## Commands
+
+### `listen`
+
+Start the HTTPS tunnel listener. Binds an HTTPS server that accepts mTLS connections from connectors.
+
+Key arguments:
+- `--listen` — bind address (default `0.0.0.0:4180`)
+- `--advertise` — public address embedded in auto-generated connection keys
+- `--connection-key` — reusable key generated by `keygen` or a previous `listen` run
+- `--socks` — optional server-side SOCKS5 proxy address for connector-side network access
+- `--cert`, `--key`, `--ca-cert` — TLS material paths (required unless using a connection key)
+- `--auth-token`, `--auth-token-file` — application auth token
+- `--insecure-skip-tls-verify` — skip client certificate verification (lab only)
+
+When run without explicit certificate paths and without a connection key, `listen` auto-generates a new connection key and prints it.
+
+### `connect`
+
+Connect to the listener and expose a local SOCKS5 proxy.
+
+Key arguments:
+- `CONNECTION_KEY` — positional connection key (optional)
+- `--target` — listener address (default from connection key)
+- `--connection-key` — connection key via flag or env `RUSTUNNEL_KEY`
+- `--socks` — local SOCKS5 proxy address (default `127.0.0.1:1180`)
+- `--cert`, `--key`, `--ca-cert` — TLS material paths (required unless using a connection key)
+- `--insecure-skip-tls-verify` — skip server certificate verification (lab only)
+
+### `generate`
+
+Generate local certificate and auth material into an output directory.
+
+Creates: `ca.pem`, `ca.key`, `server.crt`, `server.key`, `client.crt`, `client.key`, `token.txt`, `config.json`.
+
+### `keygen`
+
+Generate a single reusable connection key string. This bundles all certificate material, the auth token, and the target address into a base64-encoded JSON blob prefixed with `rtun1.`.
+
+### `version`
+
+Print version, edition, platform, and architecture.
+
+## Entry point
+
+`src/main.rs` installs the ring crypto provider, initializes tracing with an env-filter defaulting to `info`, parses the CLI, and dispatches to the appropriate runner function.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/cli.rs` | clap `Parser` and `Subcommand` definitions |
+| `src/main.rs` | Entry point, command dispatch, host:port parsing helpers |
@@ -0,0 +1,52 @@
+# By the numbers
+
+Data collected on 2026-06-04.
+
+## Size
+
+| Metric | Value |
+| ------ | ----- |
+| Total source files | 12 |
+| Total lines of code | ~7,535 |
+| Largest file | `src/tunnel.rs` (3,526 lines) |
+| Second largest | `src/socks5.rs` (789 lines) |
+| Third largest | `src/framing.rs` (776 lines) |
+| Test files | 0 standalone test files (all tests are inline `#[cfg(test)]` modules) |
+| Languages | Rust only |
+
+```mermaid
+xychart-beta
+    title "Lines of code by module"
+    x-axis [tunnel, socks5, framing, tls, main, generate, redact, cli, config, connkey, errors, signal]
+    y-axis "Lines" 0 --> 3600
+    bar [3526, 789, 776, 609, 601, 440, 242, 237, 118, 93, 82, 22]
+```
+
+## Activity
+
+- **Commits:** 16 total (as of HEAD)
+- **Active development period:** June 3–4, 2026 (~2 days)
+- **Most changed file:** `src/tunnel.rs` — underwent major rewrites from per-request HTTPS to persistent multiplexed tunnel, then to binary framing, then to reconnect/shutdown handling.
+- **Second most changed:** `.github/workflows/ci.yml` — iterated on cross-platform E2E cleanup and smoke tests.
+
+## Bot-attributed commits
+
+- **Bot commits:** 7 of 16 (44%)
+- **Bot types:** `factory-droid[bot]`
+- Note: this is a lower bound. Inline AI tools like Copilot leave no trace in git history.
+
+## Complexity
+
+| File | Lines | Description |
+| ---- | ----- | ----------- |
+| `src/tunnel.rs` | 3,526 | Core tunnel engine, listener, connector, SOCKS5 proxy, reconnect loop, stream mux, ~30 inline tests |
+| `src/socks5.rs` | 789 | SOCKS5 protocol parser, greeting, auth, CONNECT, target resolution |
+| `src/framing.rs` | 776 | Binary frame serialization, deserialization, reader/writer, target encoding |
+| `src/tls.rs` | 609 | rustls config builders, certificate loading, identity validation, fingerprinting |
+| `src/main.rs` | 601 | CLI entry point, command dispatch, connection key integration, argument parsing helpers |
+
+## Dependency count
+
+- **Runtime dependencies:** 26 crates
+- **Dev dependencies:** 1 crate (`tempfile`)
+- **Largest dependency families:** Tokio ecosystem (`tokio`, `tokio-util`, `tokio-rustls`), rustls ecosystem (`rustls`, `rustls-pki-types`, `rustls-pemfile`), Hyper ecosystem (`hyper`, `hyper-util`, `http-body-util`, `tower-service`)
@@ -0,0 +1,54 @@
+# Deployment
+
+rustunnel is deployed as a single static binary. There are no containers, orchestrators, or external services required.
+
+## Release builds
+
+CI builds release artifacts on Linux, macOS, and Windows:
+
+```sh
+cargo build --release
+```
+
+The resulting binary is uploaded as a GitHub Actions artifact with a 7-day retention.
+
+## Cross-platform CI
+
+The `.github/workflows/ci.yml` runs on `ubuntu-latest`, `windows-latest`, and `macos-latest`:
+
+1. `cargo fmt` — format check
+2. `cargo clippy` — lint check
+3. `cargo test` — unit and inline tests
+4. `cargo build` — debug build
+5. `cargo build --release` — release build with artifact upload
+6. CLI smoke tests — `--help`, `version`, invalid command/port handling
+7. Minimal E2E — generate, listen, connect, SOCKS5, curl
+
+## E2E test flow
+
+The E2E job uses `cargo run --release --` for OS-neutral invocation and performs:
+
+1. Generate credentials to a temp directory
+2. Start a Python HTTP server as the target
+3. Start `rustunnel listen` in the background
+4. Start `rustunnel connect` in the background
+5. Wait for services to start
+6. curl through the SOCKS5 proxy and assert HTTP 200 + expected body content
+7. Cleanup via `trap cleanup EXIT` with PID-based and port-based killing
+
+## No production deployment guidance
+
+rustunnel is explicitly a lab and development tool. It does not include:
+
+- Container images or Dockerfiles
+- Kubernetes manifests
+- Systemd service files
+- Log rotation or monitoring infrastructure
+
+For production tunneling, consider established tools like WireGuard, OpenVPN, or SSH.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `.github/workflows/ci.yml` | CI matrix: fmt, clippy, test, build, smoke, E2E |
@@ -0,0 +1,58 @@
+# Connection keys
+
+A connection key bundles all credential material and the target address into a single copy-pasteable string.
+
+## Purpose
+
+Simplify distribution of tunnel credentials between machines. Instead of transferring eight separate files, a user can generate one key and paste it into the `listen` and `connect` commands.
+
+## Format
+
+Connection keys start with the prefix `rtun1.` followed by base64url-encoded (no padding) JSON:
+
+```json
+{
+  "version": 1,
+  "target": "198.51.100.10:4180",
+  "ca_cert_pem": "-----BEGIN CERTIFICATE-----...",
+  "server_cert_pem": "-----BEGIN CERTIFICATE-----...",
+  "server_key_pem": "-----BEGIN PRIVATE KEY-----...",
+  "client_cert_pem": "-----BEGIN CERTIFICATE-----...",
+  "client_key_pem": "-----BEGIN PRIVATE KEY-----...",
+  "auth_token": "a1b2c3..."
+}
+```
+
+## Key abstractions
+
+| Type | File | Description |
+| ---- | ---- | ----------- |
+| `ConnectionKey` | `src/connkey.rs` | Struct with all fields, version check, validation |
+| `ConnectionKey::encode` | `src/connkey.rs` | Serialize to JSON, base64url-encode, prepend prefix |
+| `ConnectionKey::decode` | `src/connkey.rs` | Strip prefix, base64url-decode, deserialize, validate |
+| `looks_like_connection_key` | `src/connkey.rs` | Quick check if a string starts with `rtun1.` |
+
+## Validation
+
+`decode` validates:
+
+- Prefix must be `rtun1.`
+- Base64 decoding must succeed
+- JSON deserialization must succeed
+- Version must be exactly `1`
+- All string fields must be non-empty after trimming
+
+## Integration
+
+`src/main.rs` uses `ConnectionKey::decode` when the `--connection-key` flag or positional argument is provided. The decoded material is passed to `ServerTlsMaterial::Pem` or `ClientTlsMaterial::Pem` variants, which bypass file loading and use the embedded PEM strings directly.
+
+## Entry points for modification
+
+- To change the key format or add versioning: modify `src/connkey.rs`.
+- To add compression or encryption: consider extending the encode/decode pipeline in `ConnectionKey`.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/connkey.rs` | Connection key struct, encoding, decoding, validation |
@@ -0,0 +1,7 @@
+# Features
+
+rustunnel's user-visible and developer-visible capabilities:
+
+- [Connection keys](connection-keys.md) — single-string credential bundles
+- [Stream multiplexing](stream-multiplexing.md) — multiple SOCKS5 streams over one tunnel
+- [Reconnect and shutdown](reconnect-and-shutdown.md) — operational resilience
@@ -0,0 +1,57 @@
+# Reconnect and shutdown
+
+The connector is designed to recover from tunnel interruptions without dropping the SOCKS5 proxy.
+
+## Purpose
+
+Lab networks and development environments are unstable. The connector should tolerate listener restarts, network hiccups, and temporary failures without requiring manual intervention.
+
+## Reconnect behavior
+
+`run_reconnect_loop` in `src/tunnel.rs` continuously attempts to maintain a tunnel:
+
+1. Connect to the target address via TCP.
+2. Perform TLS handshake and server certificate verification.
+3. Send the HTTP auth request and wait for 200 OK.
+4. On success, create a `StreamMux`, publish it via the `watch::channel`, and run the framing loop.
+5. When the tunnel drops (framing task ends), log the disconnection and retry.
+
+### Backoff
+
+- Initial delay: 1 second
+- Exponential doubling up to a maximum of 30 seconds
+- Auth failures are terminal (the loop exits rather than retrying with a bad token)
+
+### SOCKS5 proxy during reconnect
+
+The SOCKS5 proxy stays bound. If a client connects while the tunnel is down, the proxy sees `mux_receiver` is `None` and rejects the connection with a warning log. Once the reconnect loop succeeds, new SOCKS5 requests work immediately.
+
+## Shutdown
+
+Both listener and connector handle graceful shutdown:
+
+- Unix: SIGINT or SIGTERM triggers shutdown via `tokio::signal::unix`.
+- Windows: Ctrl+C or Ctrl+Break triggers shutdown via `tokio::signal::ctrl_c`.
+
+### Listener shutdown
+
+1. The shutdown signal breaks the `tokio::select!` in `run_listener`.
+2. The TCP listener is dropped, releasing the bind port.
+3. The SOCKS5 proxy task (if any) is aborted.
+4. Active framing sessions end when their streams close.
+
+### Connector shutdown
+
+1. The shutdown signal breaks the `tokio::select!` in `run_socks5_proxy`.
+2. The SOCKS5 listener is dropped.
+3. The reconnect loop handle is aborted.
+4. Existing SOCKS5 connections receive EOF when their streams close.
+
+E2E tests `e2e_graceful_shutdown_listener` and `e2e_graceful_shutdown_connector` verify that ports are released within 3 seconds of abort.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/tunnel.rs` | `run_reconnect_loop`, `run_socks5_proxy`, `run_listener`, shutdown handling |
+| `src/signal.rs` | Cross-platform shutdown signal future |
@@ -0,0 +1,47 @@
+# Stream multiplexing
+
+rustunnel carries multiple SOCKS5 connections over a single persistent TLS tunnel using stream IDs.
+
+## Purpose
+
+Avoid the overhead of establishing a new HTTPS connection for every SOCKS5 request. A single mTLS+auth session stays open, and multiple logical streams are interleaved via the binary framing protocol.
+
+## How it works
+
+### Connector side
+
+1. A local SOCKS5 client connects to the connector's SOCKS5 proxy.
+2. The connector's `StreamMux::open_stream` allocates the next odd stream ID.
+3. It sends a CONNECT frame with the target address encoded in the payload.
+4. It waits for a CONNECT_REPLY frame (status 0x00 = OK).
+5. On success, it returns a `mpsc::Receiver<Bytes>` for tunnel-to-SOCKS5 data.
+6. The connector spawns a task to read from the SOCKS5 client and send DATA frames to the tunnel.
+7. Another path reads from the `data_rx` channel and writes raw bytes back to the SOCKS5 client.
+
+### Listener side
+
+1. The listener receives a CONNECT frame with a stream ID and target address.
+2. It resolves the target address and opens a TCP connection.
+3. It spawns a task to read from the target and send DATA frames back to the tunnel.
+4. It sends a CONNECT_REPLY frame to acknowledge the stream.
+5. Incoming DATA frames for that stream ID are written to the target connection.
+6. CLOSE frames trigger shutdown of the target connection.
+
+### Stream isolation
+
+Each stream has its own ID. Data from one stream never mixes with another. The E2E test `e2e_concurrent_streams_isolation` verifies this by sending unique payloads through two concurrent streams and asserting each echo response matches its original payload.
+
+## Key abstractions
+
+| Type | File | Description |
+| ---- | ---- | ----------- |
+| `StreamMux` | `src/tunnel.rs` | Connector-side multiplexer |
+| `ServerStreamEntry` | `src/tunnel.rs` | Listener-side per-stream target write half |
+| `ConnectorStreamEntry` | `src/tunnel.rs` | Connector-side per-stream data channel and reply channel |
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/tunnel.rs` | StreamMux, open_stream, dispatch_frame, framing loops |
+| `src/framing.rs` | Frame types, CONNECT/DATA/CLOSE/ERROR encoding |
@@ -0,0 +1,53 @@
+# Debugging
+
+## Logs
+
+rustunnel uses `tracing` with an `EnvFilter` defaulting to `info`. Set `RUST_LOG=debug` or `RUST_LOG=trace` for more detail.
+
+```sh
+RUST_LOG=debug ./target/release/rustunnel listen --listen 127.0.0.1:4180 ...
+```
+
+All logged secrets are redacted. The `Redacted` wrapper shows `[REDACTED(len=N)]` instead of the raw value. If you see raw secrets in logs, that is a bug.
+
+## Common errors
+
+### TLS handshake failures
+
+If the connector fails with a TLS error, check:
+
+- Do the listener and connector share the same CA certificate?
+- Does the server certificate's SAN match the target hostname? Use `generate` with the correct `--advertise` address, or pass `--insecure-skip-tls-verify` (lab only).
+- Are the certificate files readable and valid PEM?
+
+rustunnel prints TLS hint messages for common failures:
+
+- `BadSignature` — the listener certificate is not signed by the CA trusted by the connector.
+- `NotValidForName` — the listener certificate does not match the target host.
+
+### Auth failures
+
+- Verify the token file is not empty and has no trailing whitespace.
+- The listener and connector must use the same token value.
+- Auth failures are terminal for the connector's reconnect loop (it will not retry with a bad token).
+
+### Port conflicts
+
+If binding fails with "address already in use":
+
+- Check for orphaned processes from previous test runs.
+- The E2E tests use random free ports; manual testing may need explicit port management.
+
+### SOCKS5 connection refused
+
+If curl fails through the SOCKS5 proxy:
+
+- Is the connector running and the tunnel established?
+- Is the SOCKS5 proxy port correct?
+- Is the target address reachable from the listener side (for default mode) or connector side (when using `--socks` on the listener)?
+
+## Tracing
+
+The connector logs effective config at startup, including the target, SOCKS5 address, and redacted auth token. The listener logs its bind address, TLS fingerprints, and auth token (redacted).
+
+State transitions are logged at `info` level: "Tunnel session established", "Tunnel disconnected", "Reconnecting in Ns".
@@ -0,0 +1,25 @@
+# Development workflow
+
+## Branching
+
+Use feature branches off `master`. The project is small enough that long-lived branches are unnecessary.
+
+## Coding
+
+- Follow existing formatting (enforced by `cargo fmt`).
+- Use `anyhow` for application-level errors and `thiserror` for library-style error enums.
+- Prefer `tracing` over `println!` for diagnostic output. Secrets must be wrapped with `Redacted` before logging.
+- Use `Arc<String>` for shared strings across async boundaries (e.g., auth tokens in `ListenerConfig` and `ConnectorConfig`).
+- Keep synchronous protocol parsing in dedicated modules (`socks5.rs`, `framing.rs`) and async I/O in `tunnel.rs`.
+
+## Testing
+
+- Inline tests live in `#[cfg(test)]` modules at the bottom of each source file.
+- Use `tempfile::tempdir()` for generating test credentials. Clean up temp directories with `std::fs::remove_dir_all`.
+- Use `std::net::TcpListener::bind("127.0.0.1:0")` to get free ports for E2E tests.
+- E2E tests spawn tokio tasks for listener, connector, and target servers, then abort them in cleanup.
+
+## Committing
+
+- Follow conventional commit style: `feat:`, `fix:`, `docs:`, `chore:`.
+- Keep commits focused. The git history shows the evolution from HTTP-per-stream to multiplexed tunnel; preserve that narrative clarity.
@@ -0,0 +1,29 @@
+# How to contribute
+
+rustunnel is a small, focused project. Contributions should align with the project's purpose: a lab and development tunneling tool with strong security defaults.
+
+## Work pickup
+
+- Check the CI status before starting work.
+- Open issues or PRs against the `master` branch.
+- Keep changes focused. The codebase is small enough that large refactors should be discussed first.
+
+## PR process
+
+1. Run the full test suite locally: `cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test --all-targets`
+2. Ensure E2E tests pass if your change touches the tunnel or SOCKS5 path.
+3. Write tests for new behavior. The project uses inline `#[cfg(test)]` modules.
+4. Update relevant wiki pages if your change adds or removes features.
+
+## Review expectations
+
+- Security-sensitive changes (TLS, auth, framing) get extra scrutiny.
+- All CI jobs must pass: fmt, clippy, test, build, smoke, and E2E.
+- The project runs CI on Linux, macOS, and Windows. Avoid platform-specific code unless necessary (the signal handling module is the only current exception).
+
+## Definition of done
+
+- Code is formatted with `rustfmt`.
+- Clippy warnings are resolved.
+- Tests cover the new behavior, including edge cases.
+- No `TODO` or `FIXME` comments are introduced (the project currently has zero).
@@ -0,0 +1,55 @@
+# Patterns and conventions
+
+## Error handling
+
+The project uses a layered error strategy:
+
+- `thiserror` enums for domain-specific errors (`TlsError`, `AuthError`, `TunnelError`, `HostError`, `Socks5Error`, `FrameError`). These live in `src/errors.rs` and `src/socks5.rs` / `src/framing.rs`.
+- `anyhow` for application-level propagation. CLI commands return `anyhow::Result` and `main` exits with `process::exit(exit_code)`.
+- All errors are designed to be actionable and fail-closed. There are no silent degradation paths to insecure modes.
+
+## Secret handling
+
+- The `Redacted` wrapper in `src/redact.rs` must be used before logging any secret.
+- `Redacted::inner()` and `into_inner()` are marked `#[allow(dead_code)]` to discourage use, but available for file writes.
+- `is_sensitive()` detects PEM private key blocks. `is_auth_token()` detects long alphanumeric strings.
+- `is_connection_key()` detects strings starting with `rtun1.`.
+
+## Async patterns
+
+- Tokio's multi-threaded runtime is used on both listener and connector sides.
+- `tokio::spawn` is used for per-connection and per-stream tasks.
+- `tokio::select!` with `biased;` prioritizes shutdown signals over new connections.
+- `watch::channel` is used for publishing the active `StreamMux` to the SOCKS5 proxy (supports reconnect).
+- `mpsc::channel` and `oneshot::channel` are used for stream data and CONNECT_REPLY coordination.
+
+## Constant-time comparison
+
+Auth token comparison in `src/tunnel.rs` uses a manual constant-time XOR loop to prevent timing attacks:
+
+```rust
+fn constant_time_compare(a: &str, b: &str) -> bool {
+    let a_bytes = a.as_bytes();
+    let b_bytes = b.as_bytes();
+    if a_bytes.len() != b_bytes.len() {
+        return false;
+    }
+    let mut result: u8 = 0;
+    for (x, y) in a_bytes.iter().zip(b_bytes.iter()) {
+        result |= x ^ y;
+    }
+    result == 0
+}
+```
+
+## Frame reader partial read discipline
+
+`FrameReader` in `src/framing.rs` never consumes bytes from the buffer until a complete frame (header + payload) is available. This prevents corruption when TCP segments arrive in arbitrary chunks.
+
+## TLS material as enum
+
+Both `ServerTlsMaterial` and `ClientTlsMaterial` are enums with `Paths` and `Pem` variants. This lets the listener and connector accept either file paths (from CLI args) or embedded PEM strings (from connection keys).
+
+## Insecure mode
+
+`--insecure-skip-tls-verify` exists for DPI/intercepted environments only. When enabled, the listener skips client certificate verification and the connector skips server certificate verification. This is logged at `warn` level.
@@ -0,0 +1,75 @@
+# Testing
+
+## Test organization
+
+All tests are inline `#[cfg(test)]` modules at the bottom of each source file. There are no standalone test files or integration test directories.
+
+## Running tests
+
+```sh
+cargo test --all-targets
+```
+
+This runs unit tests and doc tests across all modules.
+
+## Test patterns
+
+### Credential generation tests
+
+Tests in `src/generate.rs` verify that `generate()` creates all expected files, that certificates contain valid PEM blocks, that private keys are detected as sensitive by the redaction module, and that config JSON is valid.
+
+### Framing tests
+
+Tests in `src/framing.rs` cover:
+
+- Frame roundtrips for all frame types (CONNECT, CONNECT_REPLY, DATA, CLOSE, ERROR)
+- Partial read handling — buffers must not be consumed until a complete frame is available
+- Multiple frames in one buffer
+- Target address encoding/decoding for IPv4 and domain names
+
+### SOCKS5 tests
+
+Tests in `src/socks5.rs` cover:
+
+- Greeting parsing (no-auth, username/password, invalid versions)
+- Auth request parsing (valid and empty credentials)
+- CONNECT request parsing (IPv4, domain, unsupported commands)
+- Reply building
+- Target resolution (IPv4 direct, domain DNS lookup)
+- Malformed input handling
+
+### TLS tests
+
+Tests in `src/tls.rs` cover:
+
+- Loading certificates and keys from files and PEM strings
+- Building valid server and client configs
+- Identity validation against SANs and CN
+- Certificate fingerprinting
+- Expiration checking
+- Insecure config variants
+
+### Tunnel tests
+
+Tests in `src/tunnel.rs` are the most extensive. They include:
+
+- **Auth tests:** valid mTLS + auth, invalid token, missing client cert
+- **E2E tests:** basic forwarding, server-side SOCKS5 reaching connector-side targets, concurrent stream isolation, target failure handling, listener-side target origin verification
+- **Operational tests:** reconnect restoring traffic after tunnel drop, graceful shutdown of listener and connector, shutdown during active streams
+- **Log tests:** verify effective config is logged with redacted tokens, verify auth failures don't leak token values in error messages
+
+## E2E test structure
+
+Typical E2E tests follow this pattern:
+
+1. Generate test credentials with `generate_test_creds()`.
+2. Pick a free port with `get_free_port()`.
+3. Spawn the listener as a tokio task.
+4. Spawn the connector (and optionally a target server) as tokio tasks.
+5. Sleep briefly to let services start.
+6. Connect via SOCKS5, send data, verify response.
+7. Abort all tasks in cleanup.
+
+## CI test matrix
+
+Tests run on `ubuntu-latest`, `windows-latest`, and `macos-latest`.
@@ -0,0 +1,40 @@
+# Tooling
+
+## Build system
+
+Cargo is the sole build tool. No custom build scripts or Makefiles are used.
+
+- `cargo build` — debug build
+- `cargo build --release` — optimized release build
+- `cargo check` — fast type-checking
+
+## Linting and formatting
+
+- `cargo fmt --check` — enforces rustfmt formatting
+- `cargo clippy --all-targets -- -D warnings` — runs clippy on all targets with warnings as errors
+
+## CI/CD
+
+GitHub Actions runs the full matrix on every push and PR:
+
+| Job | Platforms | Purpose |
+| --- | --------- | ------- |
+| `fmt` | Ubuntu | Format check |
+| `clippy` | Linux, macOS, Windows | Lint check |
+| `test` | Linux, macOS, Windows | Unit and inline tests |
+| `build` | Linux, macOS, Windows | Debug build |
+| `release-build` | Linux, macOS, Windows | Release build + artifact upload |
+| `smoke` | Linux, macOS, Windows | CLI smoke tests (--help, version, invalid command/port) |
+| `e2e-minimal` | Linux, macOS, Windows | Full end-to-end flow: generate, listen, connect, SOCKS5, curl |
+
+The E2E job uses `cargo run --release --` for OS-neutral binary invocation and includes cross-platform cleanup helpers (`kill_process`, `kill_by_port`) invoked via `trap cleanup EXIT`.
+
+## Dependencies
+
+Dependencies are managed in `Cargo.toml`. Notable choices:
+
+- `tokio` with `full` features for async runtime
+- `rustls` with `ring` for TLS (no OpenSSL)
+- `rcgen` for certificate generation
+- `clap` with `derive` for CLI parsing
+- `tracing` and `tracing-subscriber` for structured logging
@@ -0,0 +1,53 @@
+# Lore
+
+## Project origins
+
+rustunnel was created in June 2026 as a lab and development tunneling tool. The goal was a cross-platform, self-contained Rust binary that could tunnel traffic over HTTPS with mutual TLS and an application auth token, without relying on external infrastructure.
+
+## Eras
+
+### Foundation (June 3, 2026)
+
+The first commit (`eb4ef2b`) laid out the project skeleton: CLI surface using clap, a `generate` command for self-signed certificates, secret redaction, and a cross-platform CI matrix. A follow-up commit (`70411e5`) addressed initial scrutiny findings.
+
+### Secure tunnel implementation (June 3, 2026)
+
+The core tunnel logic landed in `393100d`, implementing HTTPS mTLS with an auth handshake. Shortly after, `76bda0b` addressed secure-tunnel scrutiny findings. Dependency updates and CLI refinements followed in `fc776fc`.
+
+### SOCKS5 and multiplexing (June 3, 2026)
+
+Three major commits transformed the architecture:
+
+- `6466753` — Implemented a SOCKS5 proxy with stream multiplexing over HTTPS. This replaced the earlier design where each SOCKS5 request opened a new HTTPS connection.
+- `f871ba7` — Replaced per-SOCKS-request HTTPS connections with a single persistent multiplexed tunnel session.
+- `d6a0a40` — Replaced HTTP-shaped forwarding with a true persistent multiplexed tunnel using a custom binary framing protocol.
+
+### Framing hardening (June 3, 2026)
+
+- `b67248e` — Fixed frame header corruption on partial reads. The framing reader was refined to never consume bytes from the buffer until a complete frame (header + payload) is available.
+
+### Operational features (June 4, 2026)
+
+- `10b8b24` — Added operational reconnect and shutdown handling. The connector now has a background reconnect loop with exponential backoff, and both sides handle graceful shutdown on SIGINT/SIGTERM.
+- `32a29c8` — Fixed flaky generate tests and formatted `tunnel.rs`.
+- `b0210a6` — Improved cross-platform CI E2E cleanup with `trap` and portable port killing.
+- `241d7d1` — Unmasked SOCKS5 E2E curl failures in CI and added response content assertions.
+- `cc1eb55` — Added the project README.
+- `a758e8e` — Added connection key encoding (`connkey.rs`) and the full tunnel implementation integrating keys into `listen` and `connect`.
+
+## Longest-standing code
+
+Given the project's short lifespan (2 days), all code is roughly the same age. The modules that have survived the most rewrites are:
+
+- `src/cli.rs` — CLI structure has remained stable since the first commit.
+- `src/redact.rs` — Secret redaction has not changed since the foundation commit.
+- `src/errors.rs` — Error types have been extended but the core structure is original.
+
+## Deprecated features
+
+None so far. The project is new enough that no features have been removed.
+
+## Major rewrites
+
+- **From HTTP-per-stream to persistent multiplexed tunnel (June 3, 2026)** — The original SOCKS5 implementation opened a new HTTPS connection for every stream. This was replaced by a single persistent tunnel with a binary framing protocol, dramatically reducing connection overhead.
+- **From HTTP forwarding to binary framing (June 3, 2026)** — After multiplexing was introduced, the data plane switched from raw HTTP forwarding to a lightweight binary protocol with CONNECT/DATA/CLOSE/ERROR frames.
@@ -0,0 +1,86 @@
+# Architecture
+
+rustunnel is organized as a single Rust binary crate with a modular source tree. The architecture separates CLI parsing, credential generation, TLS configuration, SOCKS5 handling, binary framing, and the core tunnel engine into distinct modules.
+
+## Component diagram
+
+```mermaid
+graph TD
+    A[CLI `src/cli.rs`] --> B[Main dispatch `src/main.rs`]
+    B --> C[Credential generation `src/generate.rs`]
+    B --> D[Tunnel engine `src/tunnel.rs`]
+    B --> E[Connection keys `src/connkey.rs`]
+    D --> F[TLS stack `src/tls.rs`]
+    D --> G[SOCKS5 proxy `src/socks5.rs`]
+    D --> H[Framing protocol `src/framing.rs`]
+    D --> I[Signal handling `src/signal.rs`]
+    F --> J[Rustls + ring]
+    G --> K[Tokio TCP]
+    H --> L[Bytes + tokio-util]
+    C --> M[rcgen]
+```
+
+## Data flow
+
+A typical tunnel session flows like this:
+
+1. **Listener** binds a TCP socket and wraps it with `tokio_rustls::TlsAcceptor`.
+2. Connector initiates a TCP connection and upgrades it with `tokio_rustls::TlsConnector`.
+3. Both sides perform mTLS handshakes: the listener requires a client certificate signed by the shared CA; the connector validates the server certificate against the same CA.
+4. The connector sends an HTTP POST to `/tunnel` with an `X-Rustunnel-Token` header.
+5. The listener validates the token with a constant-time comparison (`src/tunnel.rs`).
+6. On success, both sides switch from HTTP/1.1 to a binary framing protocol (`src/framing.rs`).
+7. The connector's local SOCKS5 proxy accepts client connections and opens multiplexed streams over the single persistent tunnel.
+8. The listener decodes each stream's target address, opens a local TCP connection to that target, and forwards data bidirectionally.
+
+```mermaid
+sequenceDiagram
+    participant Client as Local app
+    participant Socks as SOCKS5 proxy
+    participant Conn as Connector
+    participant TLS as mTLS tunnel
+    participant List as Listener
+    participant Target as Target server
+
+    Client->>Socks: SOCKS5 CONNECT
+    Socks->>Conn: open stream
+    Conn->>TLS: TLS handshake
+    TLS->>List: accept + verify client cert
+    Conn->>List: POST /tunnel + auth token
+    List->>Conn: 200 OK
+    Conn->>List: binary framing CONNECT frame
+    List->>Target: TCP connect
+    Target->>List: response
+    List->>Conn: DATA frame
+    Conn->>Socks: raw bytes
+    Socks->>Client: response
+```
+
+## Module responsibilities
+
+| Module | Responsibility |
+| ------ | -------------- |
+| `src/cli.rs` | clap-based CLI definitions for all subcommands and arguments |
+| `src/main.rs` | Entry point, crypto provider installation, tracing init, command dispatch |
+| `src/tunnel.rs` | Core tunnel engine: listener accept loop, connector reconnect loop, SOCKS5 proxy, stream mux, HTTP auth handshake, framing loop |
+| `src/tls.rs` | rustls server and client config builders, certificate loading, identity validation, insecure verifier |
+| `src/socks5.rs` | SOCKS5 greeting, auth, CONNECT request parsing, target resolution |
+| `src/framing.rs` | Binary frame serialization/deserialization, `FrameReader`, `FrameWriter` |
+| `src/generate.rs` | Self-signed CA, server cert, client cert, and auth token generation with rcgen |
+| `src/connkey.rs` | `ConnectionKey` struct: encode/decode all material as a base64 string |
+| `src/redact.rs` | `Redacted` wrapper and helpers to prevent secrets from leaking into logs |
+| `src/errors.rs` | `thiserror`-based error enums for TLS, auth, tunnel, and host operations |
+| `src/config.rs` | JSON configuration file format for generated credential sets |
+| `src/signal.rs` | Cross-platform shutdown signal handler (SIGINT/SIGTERM on Unix, Ctrl+C on Windows) |
+
+## Threading model
+
+The project uses Tokio's multi-threaded runtime. Each side (listener and connector) spawns tasks for:
+
+- Accepting new TCP connections
+- TLS handshakes
+- Per-tunnel framing loops
+- Per-stream target forwarding
+- SOCKS5 client handling
+
+The connector's SOCKS5 proxy stays bound permanently while a background reconnect loop maintains the tunnel. When the tunnel drops, existing streams receive EOF and new streams wait for the reconnect loop to re-establish the session.
@@ -0,0 +1,101 @@
+# Getting started
+
+## Prerequisites
+
+- Rust 1.85+ (the project uses Edition 2024)
+- A POSIX shell or PowerShell for running commands
+- curl (for E2E testing)
+
+## Build
+
+```sh
+cd rustunnel
+cargo build --release
+```
+
+The release binary is produced at `target/release/rustunnel`.
+
+## Run tests
+
+```sh
+cargo fmt --check
+cargo clippy --all-targets -- -D warnings
+cargo check
+cargo test --all-targets
+```
+
+The test suite includes unit tests for framing, SOCKS5 parsing, TLS config building, credential generation, connection key roundtrips, and full end-to-end tunnel flows.
+
+## Generate credentials
+
+```sh
+./target/release/rustunnel generate --out ./creds
+```
+
+This creates:
+
+- `creds/ca.pem` — CA certificate
+- `creds/ca.key` — CA private key
+- `creds/server.crt` — server certificate
+- `creds/server.key` — server private key
+- `creds/client.crt` — client certificate
+- `creds/client.key` — client private key
+- `creds/token.txt` — auth token
+- `creds/config.json` — configuration file with paths and defaults
+
+## Run a tunnel
+
+### Start the listener
+
+```sh
+./target/release/rustunnel listen \
+  --listen 127.0.0.1:4180 \
+  --cert ./creds/server.crt \
+  --key ./creds/server.key \
+  --ca-cert ./creds/ca.pem \
+  --auth-token-file ./creds/token.txt
+```
+
+### Start the connector
+
+```sh
+./target/release/rustunnel connect \
+  --target 127.0.0.1:4180 \
+  --socks 127.0.0.1:1180 \
+  --cert ./creds/client.crt \
+  --key ./creds/client.key \
+  --ca-cert ./creds/ca.pem \
+  --auth-token-file ./creds/token.txt
+```
+
+### Use the SOCKS5 proxy
+
+```sh
+curl --proxy socks5h://127.0.0.1:1180 http://127.0.0.1:4181/
+```
+
+## Using a connection key
+
+Instead of passing individual certificate paths, you can generate a single connection key:
+
+```sh
+./target/release/rustunnel keygen --target 127.0.0.1:4180
+```
+
+Then start the listener with the key (it auto-generates credentials):
+
+```sh
+./target/release/rustunnel listen --connection-key <key>
+```
+
+And connect with the same key:
+
+```sh
+./target/release/rustunnel connect <key>
+```
+
+## Defaults
+
+- Listener bind address: `0.0.0.0:4180`
+- Connector target: required (or taken from connection key)
+- Connector SOCKS5 proxy: `127.0.0.1:1180`
@@ -0,0 +1,19 @@
+# Glossary
+
+**Connection key** — a base64-encoded JSON blob (prefix `rtun1.`) that bundles the CA certificate, server certificate, server key, client certificate, client key, target address, auth token, and version. Generated by `rustunnel keygen` and consumed by `rustunnel listen` and `rustunnel connect`.
+
+**Connector** — the `rustunnel connect` side. Establishes an outbound HTTPS+mTLS connection to the listener, authenticates, then exposes a local SOCKS5 proxy for local applications.
+
+**Framing protocol** — the binary protocol used after the HTTP auth handshake completes. Frame types: CONNECT, CONNECT_REPLY, DATA, CLOSE, ERROR. All multi-byte integers are big-endian. Max payload per frame is 4 MiB.
+
+**Listener** — the `rustunnel listen` side. Binds an HTTPS server that accepts mTLS connections, validates the auth token, then upgrades the connection to the binary framing protocol.
+
+**mTLS** — mutual TLS. Both the listener and connector present certificates and verify the peer's certificate against a shared CA. rustunnel uses rustls with ring as the crypto provider.
+
+**Security gate** — one of the validation steps that must pass before tunnel forwarding begins: TLS handshake, client certificate verification (listener), server certificate verification (connector), application auth token validation.
+
+**SOCKS5 proxy** — the local proxy exposed by the connector. Implements RFC 1928 CONNECT with optional username/password auth (RFC 1929). Supports IPv4 and domain name targets.
+
+**Stream ID** — a monotonically increasing odd integer (1, 3, 5, ...) assigned by the connector for each SOCKS5 connection. Even IDs are reserved for future server-initiated streams.
+
+**StreamMux** — the connector-side structure that manages multiple concurrent streams over a single tunnel connection. Maps stream IDs to per-stream data channels.
@@ -0,0 +1,20 @@
+# rustunnel
+
+`rustunnel` is a cross-platform Rust CLI tool for lab and development tunneling over HTTPS with mutual TLS (mTLS) and an application-level auth token. It exposes a local SOCKS5 proxy on the connector side and forwards traffic only after HTTPS, certificate, and auth checks succeed.
+
+The tool is designed for local labs, development environments, and controlled testing where you own or are authorized to operate both endpoints. It is not a production-grade tunnel or a mechanism for accessing systems without permission.
+
+## What it does
+
+- **Listener** (`rustunnel listen`) — binds an HTTPS server that accepts mTLS connections from connectors, validates an auth token, then upgrades the connection to a persistent binary-framed tunnel.
+- **Connector** (`rustunnel connect`) — connects to the listener over HTTPS with mTLS, authenticates, then exposes a local SOCKS5 proxy that forwards traffic through the tunnel.
+- **Credential generation** (`rustunnel generate`) — creates a self-signed CA, server certificate, client certificate, and auth token for a local session.
+- **Connection keys** (`rustunnel keygen`) — bundles all credential material into a single base64-encoded string that can be copy-pasted between machines.
+
+## Quick links
+
+- [Architecture](../overview/architecture.md)
+- [Getting started](../overview/getting-started.md)
+- [Tunnel engine](../systems/tunnel-engine.md)
+- [TLS stack](../systems/tls-stack.md)
+- [Security posture](../security.md)
@@ -0,0 +1,82 @@
+# Configuration
+
+## CLI arguments
+
+### `listen`
+
+| Argument | Default | Description |
+| -------- | ------- | ----------- |
+| `--listen` | `0.0.0.0:4180` | HTTPS listener bind address |
+| `--advertise` | derived from `--listen` | Public address in auto-generated connection keys |
+| `--connection-key` | none | Reusable key (env: `RUSTUNNEL_KEY`) |
+| `--socks` | none | Server-side SOCKS5 proxy address |
+| `--cert` | "" | Server certificate path |
+| `--key` | "" | Server private key path |
+| `--ca-cert` | "" | CA certificate path |
+| `--auth-token` | "" | Auth token value |
+| `--auth-token-file` | none | Auth token file path |
+| `--insecure-skip-tls-verify` | false | Skip client cert verification |
+
+### `connect`
+
+| Argument | Default | Description |
+| -------- | ------- | ----------- |
+| `CONNECTION_KEY` | none | Positional connection key |
+| `--target` | from key | Listener address |
+| `--connection-key` | none | Key via flag (env: `RUSTUNNEL_KEY`) |
+| `--socks` | `127.0.0.1:1180` | Local SOCKS5 proxy address |
+| `--cert` | "" | Client certificate path |
+| `--key` | "" | Client private key path |
+| `--ca-cert` | "" | CA certificate path |
+| `--auth-token` | "" | Auth token value |
+| `--auth-token-file` | none | Auth token file path |
+| `--insecure-skip-tls-verify` | false | Skip server cert verification |
+
+### `generate`
+
+| Argument | Default | Description |
+| -------- | ------- | ----------- |
+| `--out` | `.` | Output directory |
+| `--ca-name` | `rustunnel-ca` | CA common name |
+| `--server-name` | `rustunnel-server` | Server common name |
+| `--client-name` | `rustunnel-client` | Client common name |
+
+### `keygen`
+
+| Argument | Default | Description |
+| -------- | ------- | ----------- |
+| `--target` | `127.0.0.1:4180` | Listener address embedded in key |
+| `--ca-name` | `rustunnel-ca` | CA common name |
+| `--server-name` | `rustunnel-server` | Server common name |
+| `--client-name` | `rustunnel-client` | Client common name |
+
+## Environment variables
+
+- `RUST_LOG` — tracing filter (e.g., `debug`, `trace`)
+- `RUSTUNNEL_KEY` — connection key for `listen` and `connect`
+
+## Config file format
+
+`config.json` (generated by `rustunnel generate`):
+
+```json
+{
+  "listen_address": "0.0.0.0",
+  "listen_port": 4180,
+  "socks_address": "127.0.0.1",
+  "socks_port": 1180,
+  "ca_cert_path": "ca.pem",
+  "server_cert_path": "server.crt",
+  "server_key_path": "server.key",
+  "client_cert_path": "client.crt",
+  "client_key_path": "client.key",
+  "auth_token_path": "token.txt"
+}
+```
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/cli.rs` | clap argument definitions and defaults |
+| `src/config.rs` | `Config` struct for JSON serialization |
@@ -0,0 +1,93 @@
+# Data models
+
+## ConnectionKey
+
+```rust
+pub struct ConnectionKey {
+    pub version: u8,          // must be 1
+    pub target: String,       // listener address
+    pub ca_cert_pem: String,
+    pub server_cert_pem: String,
+    pub server_key_pem: String,
+    pub client_cert_pem: String,
+    pub client_key_pem: String,
+    pub auth_token: String,
+}
+```
+
+Encoded as: `rtun1.` + base64url(JSON) — no padding.
+
+## Config
+
+```rust
+pub struct Config {
+    pub listen_address: String,
+    pub listen_port: u16,
+    pub socks_address: String,
+    pub socks_port: u16,
+    pub ca_cert_path: String,
+    pub server_cert_path: String,
+    pub server_key_path: String,
+    pub client_cert_path: String,
+    pub client_key_path: String,
+    pub auth_token_path: String,
+}
+```
+
+## Frame
+
+```rust
+pub struct Frame {
+    pub frame_type: u8,   // 0x01 CONNECT, 0x02 CONNECT_REPLY, 0x03 DATA, 0x04 CLOSE, 0x05 ERROR
+    pub stream_id: u64,   // odd for client-initiated
+    pub flags: u8,        // reserved (0x00)
+    pub payload: Bytes,
+}
+```
+
+Wire format: 1 byte type + 8 bytes stream_id (big-endian) + 4 bytes len (big-endian) + 1 byte flags + len bytes payload.
+
+Max payload: 4 MiB.
+
+## Socks5Target
+
+```rust
+pub enum Socks5Target {
+    Ipv4(std::net::Ipv4Addr, u16),
+    Domain { domain: String, port: u16 },
+}
+```
+
+## GeneratedMaterial
+
+```rust
+pub struct GeneratedMaterial {
+    pub ca_cert_pem: String,
+    pub ca_key_pem: String,
+    pub server_cert_pem: String,
+    pub server_key_pem: String,
+    pub client_cert_pem: String,
+    pub client_key_pem: String,
+    pub auth_token: String,
+}
+```
+
+## Error enums
+
+- `TlsError` — missing/invalid certificates, identity mismatch, expiration, verification failure
+- `AuthError` — missing or invalid token
+- `TunnelError` — TLS error, auth error, bind error, connection refused, port in use, protocol error
+- `HostError` — DNS resolution failure, invalid hostname, no addresses
+- `Socks5Error` — invalid version, unsupported auth/command/addr type, parse errors
+- `FrameError` — frame too short, payload overflow, I/O error, protocol error
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/connkey.rs` | `ConnectionKey` |
+| `src/config.rs` | `Config` |
+| `src/framing.rs` | `Frame`, `FrameReader`, `FrameWriter` |
+| `src/socks5.rs` | `Socks5Target`, `Socks5State`, `Socks5Error` |
+| `src/generate.rs` | `GeneratedMaterial` |
+| `src/errors.rs` | `TlsError`, `AuthError`, `TunnelError`, `HostError` |
@@ -0,0 +1,45 @@
+# Dependencies
+
+## Runtime dependencies
+
+| Crate | Version | Purpose |
+| ----- | ------- | ------- |
+| `clap` | 4 | CLI parsing with derive macros |
+| `tokio` | 1 | Async runtime (full features) |
+| `tokio-util` | 0.7 | I/O utilities |
+| `tracing` | 0.1 | Structured logging |
+| `tracing-subscriber` | 0.3 | Log formatting with env-filter |
+| `anyhow` | 1 | Application error handling |
+| `thiserror` | 1 | Library error enums |
+| `serde` | 1 | Serialization (derive) |
+| `serde_json` | 1 | JSON encoding/decoding |
+| `rcgen` | 0.14 | Certificate generation |
+| `x509-parser` | 0.16 | Certificate parsing for identity validation |
+| `time` | 0.3 | Date/time formatting for certificate validity |
+| `rand` | 0.8 | Random token generation |
+| `hex` | 0.4 | Hex encoding for auth tokens |
+| `rustls` | 0.23 | TLS implementation (ring provider) |
+| `rustls-pki-types` | 1 | rustls type definitions |
+| `rustls-pemfile` | 2 | PEM parsing for certificates and keys |
+| `tokio-rustls` | 0.26 | Tokio integration for rustls |
+| `hyper` | 1 | HTTP types (used for auth handshake) |
+| `hyper-util` | 0.1 | HTTP utilities |
+| `http-body-util` | 0.1 | HTTP body utilities |
+| `tower-service` | 0.3 | Service trait (Hyper ecosystem) |
+| `base64` | 0.22 | Connection key encoding |
+| `sha2` | 0.10 | Certificate fingerprinting |
+| `bytes` | 1 | Byte buffers for framing |
+| `futures` | 0.3 | Future utilities |
+| `url` | 2 | URL parsing |
+
+## Dev dependencies
+
+| Crate | Version | Purpose |
+| ----- | ------- | ------- |
+| `tempfile` | 3 | Temporary directories for tests |
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `Cargo.toml` | Dependency declarations and features |
@@ -0,0 +1,7 @@
+# Reference
+
+Configuration, data models, and external dependencies.
+
+- [Configuration](configuration.md)
+- [Data models](data-models.md)
+- [Dependencies](dependencies.md)
@@ -0,0 +1,59 @@
+# Security
+
+rustunnel's security posture is built on multiple layers: TLS transport, mutual authentication, application-level auth tokens, secret redaction, and fail-closed error handling.
+
+## Security gates
+
+Before any traffic flows, four gates must pass:
+
+1. **TLS handshake** — both sides negotiate a TLS 1.2+ connection via rustls with ring.
+2. **Server certificate verification** — the connector verifies the listener's certificate against the shared CA.
+3. **Client certificate verification** — the listener verifies the connector's certificate against the same CA (skipped in insecure mode).
+4. **Application auth token** — the connector sends an `X-Rustunnel-Token` header; the listener validates it with constant-time comparison.
+
+The E2E test `auth_required_in_addition_to_mtls` verifies that valid mTLS alone is not sufficient; the correct auth token is also required.
+
+## mTLS
+
+- The listener requires client certificates signed by the shared CA. It uses `WebPkiClientVerifier`.
+- The connector presents its client certificate and validates the server certificate against the CA.
+- Certificate identities are validated against SANs (Subject Alternative Names) and CN. The server certificate includes SANs for `localhost`, `rustunnel`, and `127.0.0.1`, plus the bind IP if applicable.
+
+## Auth token
+
+- Generated as 32 random bytes encoded as 64 hex characters.
+- Compared with a constant-time XOR loop to prevent timing attacks.
+- Never logged in raw form. All log output uses the `Redacted` wrapper.
+- Auth failures are terminal: the connector's reconnect loop exits on auth failure rather than retrying.
+
+## Secret redaction
+
+- `Redacted` in `src/redact.rs` wraps strings and displays `[REDACTED(len=N)]`.
+- `redact_pem()` shows only the PEM block type and line count.
+- `redact_config_json()` scrubs sensitive keys from JSON.
+- `is_sensitive()` detects PEM private key blocks.
+- `is_auth_token()` detects long alphanumeric strings.
+- `is_connection_key()` detects strings starting with `rtun1.`.
+
+The E2E test `ops_auth_failure_is_actionable_and_redacted` verifies that error messages do not contain raw token values.
+
+## Insecure mode
+
+`--insecure-skip-tls-verify` disables certificate verification on either side. This is intended for DPI/intercepted environments only. When enabled:
+
+- The listener uses `with_no_client_auth()` instead of `WebPkiClientVerifier`.
+- The connector uses a custom `InsecureServerCertVerifier` that accepts any certificate.
+- A `warn` level log is emitted.
+
+## Fail-closed design
+
+All error paths reject the connection rather than falling back to an insecure mode. There are no bypasses around TLS, certificate verification, or auth.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/tls.rs` | TLS config, certificate loading, identity validation |
+| `src/tunnel.rs` | Auth handshake, constant-time comparison, security gate tests |
+| `src/redact.rs` | Secret redaction wrappers and helpers |
+| `src/errors.rs` | Error enums for TLS, auth, tunnel |
@@ -0,0 +1,45 @@
+# Credential generation
+
+The credential generation module in `src/generate.rs` creates all the certificate and key material needed for a rustunnel session.
+
+## Purpose
+
+Generate a self-signed CA, server certificate, client certificate, and a random auth token. Write them to disk in a specified output directory.
+
+## Key abstractions
+
+| Type/Function | File | Description |
+| ------------- | ---- | ----------- |
+| `GeneratedMaterial` | `src/generate.rs` | Struct holding all generated PEM strings and the auth token |
+| `generate` | `src/generate.rs` | Generate material and write 8 files to an output directory |
+| `generate_material` | `src/generate.rs` | Generate material without writing to disk |
+| `generate_ca` | `src/generate.rs` | Create a self-signed CA with rcgen |
+| `generate_server_cert` | `src/generate.rs` | Create a server cert signed by the CA |
+| `generate_client_cert` | `src/generate.rs` | Create a client cert signed by the CA |
+| `generate_auth_token` | `src/generate.rs` | Generate a 32-byte random hex token |
+
+## Certificate details
+
+- **CA** — CN from `--ca-name`, org "rustunnel", serial 1, constrained CA basic constraints
+- **Server cert** — CN from `--server-name`, org "rustunnel", serial 2, SANs: `localhost`, `rustunnel`, `127.0.0.1`, and optionally the bind IP if not `0.0.0.0` or `::`
+- **Client cert** — CN from `--client-name`, org "rustunnel", serial 3
+- **Auth token** — 64 hex characters (32 random bytes)
+
+## Output files
+
+| File | Content |
+| ---- | ------- |
+| `ca.pem` | CA certificate (PEM) |
+| `ca.key` | CA private key (PEM) |
+| `server.crt` | Server certificate (PEM) |
+| `server.key` | Server private key (PEM) |
+| `client.crt` | Client certificate (PEM) |
+| `client.key` | Client private key (PEM) |
+| `token.txt` | Auth token (plain text) |
+| `config.json` | Paths and default addresses (JSON) |
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/generate.rs` | Certificate generation, token generation, file writing |
@@ -0,0 +1,50 @@
+# Framing protocol
+
+The framing protocol in `src/framing.rs` defines the binary message format used after the HTTP auth handshake upgrades the connection to a persistent tunnel.
+
+## Purpose
+
+Carry multiple bidirectional streams over a single TLS connection with minimal overhead.
+
+## Frame format
+
+All multi-byte integers are big-endian.
+
+```
+--------+----------+----------+-------+-----------+
+| Type   | StreamID | Len      | Flags | Payload.. |
+| 1 byte | 8 bytes  | 4 bytes  | 1 byte| variable  |
+--------+----------+----------+-------+-----------+
+```
+
+- **Type** — 0x01 CONNECT, 0x02 CONNECT_REPLY, 0x03 DATA, 0x04 CLOSE, 0x05 ERROR
+- **StreamID** — odd IDs for client-initiated streams, even reserved for server-initiated
+- **Len** — payload length, max 4 MiB
+- **Flags** — reserved for future use (currently always 0x00)
+
+## Key abstractions
+
+| Type | File | Description |
+| ---- | ---- | ----------- |
+| `Frame` | `src/framing.rs` | Struct holding frame_type, stream_id, flags, payload |
+| `FrameReader` | `src/framing.rs` | Async reader with buffering that handles partial TCP reads |
+| `FrameWriter` | `src/framing.rs` | Async writer that serializes and flushes frames |
+| `encode_target` | `src/framing.rs` | Encode a `Socks5Target` into CONNECT frame payload |
+| `decode_target` | `src/framing.rs` | Decode a `Socks5Target` from CONNECT frame payload |
+
+## Partial read discipline
+
+`FrameReader::read_frame` loops until a complete frame is available:
+
+1. Attempt to deserialize from the buffer.
+2. If the frame is incomplete (`FrameTooShort`), read more bytes from the underlying TCP stream into the buffer.
+3. If the connection closes with a partial frame in the buffer, return an error.
+4. Only consume bytes from the buffer once the full frame is present.
+
+This prevents header corruption when TCP segments arrive in arbitrary sizes.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/framing.rs` | Frame format, serialization, async reader/writer, target encoding |
@@ -0,0 +1,9 @@
+# Systems
+
+rustunnel's internal architecture is divided into five systems:
+
+- [Tunnel engine](tunnel-engine.md) — listener accept loop, connector reconnect loop, stream multiplexing, SOCKS5 proxy integration
+- [TLS stack](tls-stack.md) — rustls configuration, certificate loading, identity validation
+- [Framing protocol](framing-protocol.md) — binary frame format, serialization, partial read handling
+- [SOCKS5 proxy](socks5-proxy.md) — RFC 1928 handshake, CONNECT support, target resolution
+- [Credential generation](credential-generation.md) — self-signed CA and certificate generation with rcgen
@@ -0,0 +1,37 @@
+# SOCKS5 proxy
+
+The SOCKS5 module in `src/socks5.rs` implements RFC 1928 (SOCKS5 protocol) and RFC 1929 (username/password authentication).
+
+## Purpose
+
+Handle the SOCKS5 handshake on the connector side, parse CONNECT requests, and resolve target addresses so that local applications can forward traffic through the tunnel.
+
+## Supported features
+
+- CONNECT command only (no BIND or UDP)
+- No-auth and username/password authentication methods
+- IPv4 and domain name targets (no IPv6)
+- Target DNS resolution on the connector side
+
+## Key abstractions
+
+| Type | File | Description |
+| ---- | ---- | ----------- |
+| `Socks5Target` | `src/socks5.rs` | Enum: `Ipv4(addr, port)` or `Domain { domain, port }` |
+| `Socks5State` | `src/socks5.rs` | State machine: ExpectGreeting → ExpectAuth → ExpectRequest → RequestReceived → Established → Closed |
+| `Socks5Error` | `src/socks5.rs` | Error enum for malformed input, unsupported methods, and parse failures |
+| `parse_greeting` | `src/socks5.rs` | Parse client greeting and choose auth method |
+| `parse_auth_request` | `src/socks5.rs` | Parse username/password auth request |
+| `parse_connect_request` | `src/socks5.rs` | Parse CONNECT request and extract target |
+| `resolve_target` | `src/socks5.rs` | Async DNS resolution of a `Socks5Target` to a `SocketAddr` |
+
+## Integration
+
+The connector's `run_socks5_proxy` in `src/tunnel.rs` calls `socks5_handshake` (an async function in `src/tunnel.rs` that uses `src/socks5.rs` constants and helpers) for each incoming SOCKS5 connection. After handshake, it opens a stream via `StreamMux::open_stream` and forwards data bidirectionally.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/socks5.rs` | SOCKS5 protocol constants, parsing, target resolution |
+| `src/tunnel.rs` | Async SOCKS5 handshake and proxy integration |
@@ -0,0 +1,45 @@
+# TLS stack
+
+The TLS stack in `src/tls.rs` configures rustls for both server (listener) and client (connector) roles with mutual TLS.
+
+## Purpose
+
+Provide safe, fail-closed TLS configuration builders that load certificates and keys, validate certificate identities, and support both secure and insecure (lab-only) modes.
+
+## Key abstractions
+
+| Type/Function | File | Description |
+| ------------- | ---- | ----------- |
+| `build_server_config` | `src/tls.rs` | Build a rustls `ServerConfig` with client certificate verification |
+| `build_client_config` | `src/tls.rs` | Build a rustls `ClientConfig` with server certificate verification |
+| `build_server_config_insecure` | `src/tls.rs` | Build a server config that skips client cert verification |
+| `build_client_config_insecure` | `src/tls.rs` | Build a client config that skips server cert verification |
+| `load_certs` | `src/tls.rs` | Load PEM-encoded certificates from a file |
+| `load_key` | `src/tls.rs` | Load a PEM-encoded private key from a file |
+| `validate_cert_identity` | `src/tls.rs` | Check SANs and CN against an expected hostname |
+| `check_cert_not_expired` | `src/tls.rs` | Verify certificate validity period |
+| `cert_fingerprint_sha256` | `src/tls.rs` | Compute SHA-256 fingerprint of a certificate |
+| `InsecureServerCertVerifier` | `src/tls.rs` | Dangerous verifier that accepts any certificate |
+
+## How it works
+
+Server config building:
+
+1. Load the server certificate chain and private key.
+2. Load the CA certificate into a `RootCertStore`.
+3. Build a `WebPkiClientVerifier` from the root store.
+4. Use `ServerConfig::builder().with_client_cert_verifier(...).with_single_cert(...)`.
+
+Client config building:
+
+1. Load the client certificate chain and private key.
+2. Load the CA certificate into a `RootCertStore`.
+3. Use `ClientConfig::builder().with_root_certificates(...).with_client_auth_cert(...)`.
+
+Identity validation uses `x509-parser` to extract Subject Alternative Names and the subject CN, matching against the expected hostname or IP address.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/tls.rs` | TLS configuration, certificate loading, identity validation |
@@ -0,0 +1,69 @@
+# Tunnel engine
+
+The tunnel engine in `src/tunnel.rs` is the largest module (~3,500 lines). It implements the listener, connector, SOCKS5 proxy, stream multiplexer, reconnect loop, and HTTP auth handshake.
+
+## Purpose
+
+The tunnel engine establishes a persistent, authenticated, mTLS-encrypted tunnel between a listener and a connector, then multiplexes multiple SOCKS5 streams over that single connection.
+
+## Key abstractions
+
+| Type | File | Description |
+| ---- | ---- | ----------- |
+| `ListenerConfig` | `src/tunnel.rs` | Bind address, optional SOCKS5 address, TLS material, auth token, insecure flag |
+| `ConnectorConfig` | `src/tunnel.rs` | Target host/port, TLS material, auth token, insecure flag |
+| `ServerTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for server-side TLS |
+| `ClientTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for client-side TLS |
+| `StreamMux` | `src/tunnel.rs` | Connector-side multiplexer: manages stream IDs, data channels, and frame dispatch |
+| `ServerStreamEntry` | `src/tunnel.rs` | Listener-side target write half for an active stream |
+| `ConnectorStreamEntry` | `src/tunnel.rs` | Connector-side data channel and reply channel for an active stream |
+
+## How it works
+
+### Listener side
+
+1. `run_listener` binds a TCP socket, builds the TLS acceptor from `ServerTlsMaterial`, and starts accepting connections.
+2. `handle_listener_connection` performs the TLS accept, verifies the client certificate (unless insecure), reads the HTTP POST to `/tunnel`, and validates the `X-Rustunnel-Token` header with `constant_time_compare`.
+3. On auth success, it calls `handle_framing_server` to switch to binary framing.
+4. `handle_framing_server` reads frames from the tunnel. A CONNECT frame triggers a TCP connection to the target address (opened from the listener side). DATA frames are forwarded to the target. CLOSE frames shut down the target connection.
+5. If the listener was started with `--socks`, a server-side SOCKS5 proxy runs via `run_socks5_proxy`, receiving the active `StreamMux` through a `watch::channel`.
+
+### Connector side
+
+1. `run_connector_with_socks` builds the client TLS config and starts two concurrent pieces: a reconnect loop and a permanently-bound SOCKS5 proxy.
+2. `run_reconnect_loop` repeatedly attempts to connect to the listener with exponential backoff (1s to 30s). Each attempt re-runs all security gates: TLS handshake, server cert verification, and auth token exchange.
+3. On success, the loop creates a `StreamMux`, publishes it via the `watch::channel`, and waits for the framing task to finish (indicating disconnection).
+4. When the tunnel drops, the loop logs the disconnection and retries.
+5. `run_socks5_proxy` accepts SOCKS5 connections. For each connection, it checks if the `StreamMux` is available, opens a stream via `mux.open_stream()`, and performs bidirectional forwarding.
+
+### Stream multiplexing
+
+`StreamMux::open_stream` allocates the next odd stream ID, registers a data channel and reply channel, sends a CONNECT frame, and waits for a CONNECT_REPLY. The caller receives a `mpsc::Receiver<Bytes>` for tunnel-to-SOCKS5 data.
+
+`StreamMux::dispatch_frame` routes inbound CONNECT_REPLY, DATA, CLOSE, and ERROR frames to the correct stream by ID.
+
+```mermaid
+graph TD
+    A[SOCKS5 client] -->|CONNECT| B[SOCKS5 proxy]
+    B -->|open_stream| C[StreamMux]
+    C -->|CONNECT frame| D[Framing writer]
+    D -->|TLS| E[Listener framing]
+    E -->|decode target| F[TCP connect to target]
+    F -->|DATA| E
+    E -->|DATA frame| D
+    D -->|dispatch| C
+    C -->|data_rx| B
+    B -->|raw bytes| A
+```
+
+## Entry points for modification
+
+- To change auth behavior: modify `handle_listener_connection` (listener) and `authenticate_tunnel` (connector) in `src/tunnel.rs`.
+- To change reconnect strategy: modify `run_reconnect_loop` in `src/tunnel.rs`.
+- To add listener-side SOCKS5 behavior: the server-side SOCKS5 proxy is currently minimal; expand `run_socks5_proxy` in `src/tunnel.rs`.
+
+## Key source files
+
+| File | Purpose |
+| ---- | ------- |
+| `src/tunnel.rs` | Full tunnel engine: listener, connector, SOCKS5 proxy, reconnect, framing loops, stream mux |