diff --git a/.gitignore b/.gitignore index 56621f6..b6f4e57 100644 --- a/.gitignore +++ b/.gitignore @@ -2,5 +2,6 @@ /release **/*.pem **/*.json +!.wiki-meta.json .env *.log diff --git a/droid-wiki/.wiki-meta.json b/droid-wiki/.wiki-meta.json new file mode 100644 index 0000000..2a3f81c --- /dev/null +++ b/droid-wiki/.wiki-meta.json @@ -0,0 +1,50 @@ +{ + "generatedAt": "2026-06-04T13:55:00Z", + "commitHash": "753928239e5a86414f6a6912b908da4cf0960231", + "branch": "master", + "pageCount": 30, + "topLevelSections": [ + "overview", + "by-the-numbers", + "lore", + "how-to-contribute", + "applications", + "systems", + "features", + "security", + "deployment", + "reference" + ], + "pageOrder": [ + "overview/index.md", + "overview/architecture.md", + "overview/getting-started.md", + "overview/glossary.md", + "by-the-numbers.md", + "lore.md", + "how-to-contribute/index.md", + "how-to-contribute/development-workflow.md", + "how-to-contribute/testing.md", + "how-to-contribute/debugging.md", + "how-to-contribute/patterns-and-conventions.md", + "how-to-contribute/tooling.md", + "applications/index.md", + "applications/rustunnel-cli.md", + "systems/index.md", + "systems/tunnel-engine.md", + "systems/tls-stack.md", + "systems/framing-protocol.md", + "systems/socks5-proxy.md", + "systems/credential-generation.md", + "features/index.md", + "features/connection-keys.md", + "features/stream-multiplexing.md", + "features/reconnect-and-shutdown.md", + "security.md", + "deployment.md", + "reference/index.md", + "reference/configuration.md", + "reference/data-models.md", + "reference/dependencies.md" + ] +} diff --git a/droid-wiki/applications/index.md b/droid-wiki/applications/index.md new file mode 100644 index 0000000..a1982ae --- /dev/null +++ b/droid-wiki/applications/index.md @@ -0,0 +1,3 @@ +# Applications + +rustunnel ships as a single binary application with five CLI commands. There are no separate services, daemons, or web frontends. diff --git a/droid-wiki/applications/rustunnel-cli.md b/droid-wiki/applications/rustunnel-cli.md new file mode 100644 index 0000000..72264ab --- /dev/null +++ b/droid-wiki/applications/rustunnel-cli.md @@ -0,0 +1,57 @@ +# rustunnel CLI + +The rustunnel binary exposes five subcommands defined in `src/cli.rs` and dispatched in `src/main.rs`. + +## Commands + +### `listen` + +Start the HTTPS tunnel listener. Binds an HTTPS server that accepts mTLS connections from connectors. + +Key arguments: +- `--listen` — bind address (default `0.0.0.0:4180`) +- `--advertise` — public address embedded in auto-generated connection keys +- `--connection-key` — reusable key generated by `keygen` or a previous `listen` run +- `--socks` — optional server-side SOCKS5 proxy address for connector-side network access +- `--cert`, `--key`, `--ca-cert` — TLS material paths (required unless using a connection key) +- `--auth-token`, `--auth-token-file` — application auth token +- `--insecure-skip-tls-verify` — skip client certificate verification (lab only) + +When run without explicit certificate paths and without a connection key, `listen` auto-generates a new connection key and prints it. + +### `connect` + +Connect to the listener and expose a local SOCKS5 proxy. + +Key arguments: +- `CONNECTION_KEY` — positional connection key (optional) +- `--target` — listener address (default from connection key) +- `--connection-key` — connection key via flag or env `RUSTUNNEL_KEY` +- `--socks` — local SOCKS5 proxy address (default `127.0.0.1:1180`) +- `--cert`, `--key`, `--ca-cert` — TLS material paths (required unless using a connection key) +- `--insecure-skip-tls-verify` — skip server certificate verification (lab only) + +### `generate` + +Generate local certificate and auth material into an output directory. + +Creates: `ca.pem`, `ca.key`, `server.crt`, `server.key`, `client.crt`, `client.key`, `token.txt`, `config.json`. + +### `keygen` + +Generate a single reusable connection key string. This bundles all certificate material, the auth token, and the target address into a base64-encoded JSON blob prefixed with `rtun1.`. + +### `version` + +Print version, edition, platform, and architecture. + +## Entry point + +`src/main.rs` installs the ring crypto provider, initializes tracing with an env-filter defaulting to `info`, parses the CLI, and dispatches to the appropriate runner function. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/cli.rs` | clap `Parser` and `Subcommand` definitions | +| `src/main.rs` | Entry point, command dispatch, host:port parsing helpers | diff --git a/droid-wiki/by-the-numbers.md b/droid-wiki/by-the-numbers.md new file mode 100644 index 0000000..068763b --- /dev/null +++ b/droid-wiki/by-the-numbers.md @@ -0,0 +1,52 @@ +# By the numbers + +Data collected on 2026-06-04. + +## Size + +| Metric | Value | +| ------ | ----- | +| Total source files | 12 | +| Total lines of code | ~7,535 | +| Largest file | `src/tunnel.rs` (3,526 lines) | +| Second largest | `src/socks5.rs` (789 lines) | +| Third largest | `src/framing.rs` (776 lines) | +| Test files | 0 standalone test files (all tests are inline `#[cfg(test)]` modules) | +| Languages | Rust only | + +```mermaid +xychart-beta + title "Lines of code by module" + x-axis [tunnel, socks5, framing, tls, main, generate, redact, cli, config, connkey, errors, signal] + y-axis "Lines" 0 --> 3600 + bar [3526, 789, 776, 609, 601, 440, 242, 237, 118, 93, 82, 22] +``` + +## Activity + +- **Commits:** 16 total (as of HEAD) +- **Active development period:** June 3–4, 2026 (~2 days) +- **Most changed file:** `src/tunnel.rs` — underwent major rewrites from per-request HTTPS to persistent multiplexed tunnel, then to binary framing, then to reconnect/shutdown handling. +- **Second most changed:** `.github/workflows/ci.yml` — iterated on cross-platform E2E cleanup and smoke tests. + +## Bot-attributed commits + +- **Bot commits:** 7 of 16 (44%) +- **Bot types:** `factory-droid[bot]` +- Note: this is a lower bound. Inline AI tools like Copilot leave no trace in git history. + +## Complexity + +| File | Lines | Description | +| ---- | ----- | ----------- | +| `src/tunnel.rs` | 3,526 | Core tunnel engine, listener, connector, SOCKS5 proxy, reconnect loop, stream mux, ~30 inline tests | +| `src/socks5.rs` | 789 | SOCKS5 protocol parser, greeting, auth, CONNECT, target resolution | +| `src/framing.rs` | 776 | Binary frame serialization, deserialization, reader/writer, target encoding | +| `src/tls.rs` | 609 | rustls config builders, certificate loading, identity validation, fingerprinting | +| `src/main.rs` | 601 | CLI entry point, command dispatch, connection key integration, argument parsing helpers | + +## Dependency count + +- **Runtime dependencies:** 26 crates +- **Dev dependencies:** 1 crate (`tempfile`) +- **Largest dependency families:** Tokio ecosystem (`tokio`, `tokio-util`, `tokio-rustls`), rustls ecosystem (`rustls`, `rustls-pki-types`, `rustls-pemfile`), Hyper ecosystem (`hyper`, `hyper-util`, `http-body-util`, `tower-service`) diff --git a/droid-wiki/deployment.md b/droid-wiki/deployment.md new file mode 100644 index 0000000..a0f4748 --- /dev/null +++ b/droid-wiki/deployment.md @@ -0,0 +1,54 @@ +# Deployment + +rustunnel is deployed as a single static binary. There are no containers, orchestrators, or external services required. + +## Release builds + +CI builds release artifacts on Linux, macOS, and Windows: + +```sh +cargo build --release +``` + +The resulting binary is uploaded as a GitHub Actions artifact with a 7-day retention. + +## Cross-platform CI + +The `.github/workflows/ci.yml` runs on `ubuntu-latest`, `windows-latest`, and `macos-latest`: + +1. `cargo fmt` — format check +2. `cargo clippy` — lint check +3. `cargo test` — unit and inline tests +4. `cargo build` — debug build +5. `cargo build --release` — release build with artifact upload +6. CLI smoke tests — `--help`, `version`, invalid command/port handling +7. Minimal E2E — generate, listen, connect, SOCKS5, curl + +## E2E test flow + +The E2E job uses `cargo run --release --` for OS-neutral invocation and performs: + +1. Generate credentials to a temp directory +2. Start a Python HTTP server as the target +3. Start `rustunnel listen` in the background +4. Start `rustunnel connect` in the background +5. Wait for services to start +6. curl through the SOCKS5 proxy and assert HTTP 200 + expected body content +7. Cleanup via `trap cleanup EXIT` with PID-based and port-based killing + +## No production deployment guidance + +rustunnel is explicitly a lab and development tool. It does not include: + +- Container images or Dockerfiles +- Kubernetes manifests +- Systemd service files +- Log rotation or monitoring infrastructure + +For production tunneling, consider established tools like WireGuard, OpenVPN, or SSH. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `.github/workflows/ci.yml` | CI matrix: fmt, clippy, test, build, smoke, E2E | diff --git a/droid-wiki/features/connection-keys.md b/droid-wiki/features/connection-keys.md new file mode 100644 index 0000000..c50e29e --- /dev/null +++ b/droid-wiki/features/connection-keys.md @@ -0,0 +1,58 @@ +# Connection keys + +A connection key bundles all credential material and the target address into a single copy-pasteable string. + +## Purpose + +Simplify distribution of tunnel credentials between machines. Instead of transferring eight separate files, a user can generate one key and paste it into the `listen` and `connect` commands. + +## Format + +Connection keys start with the prefix `rtun1.` followed by base64url-encoded (no padding) JSON: + +```json +{ + "version": 1, + "target": "198.51.100.10:4180", + "ca_cert_pem": "-----BEGIN CERTIFICATE-----...", + "server_cert_pem": "-----BEGIN CERTIFICATE-----...", + "server_key_pem": "-----BEGIN PRIVATE KEY-----...", + "client_cert_pem": "-----BEGIN CERTIFICATE-----...", + "client_key_pem": "-----BEGIN PRIVATE KEY-----...", + "auth_token": "a1b2c3..." +} +``` + +## Key abstractions + +| Type | File | Description | +| ---- | ---- | ----------- | +| `ConnectionKey` | `src/connkey.rs` | Struct with all fields, version check, validation | +| `ConnectionKey::encode` | `src/connkey.rs` | Serialize to JSON, base64url-encode, prepend prefix | +| `ConnectionKey::decode` | `src/connkey.rs` | Strip prefix, base64url-decode, deserialize, validate | +| `looks_like_connection_key` | `src/connkey.rs` | Quick check if a string starts with `rtun1.` | + +## Validation + +`decode` validates: + +- Prefix must be `rtun1.` +- Base64 decoding must succeed +- JSON deserialization must succeed +- Version must be exactly `1` +- All string fields must be non-empty after trimming + +## Integration + +`src/main.rs` uses `ConnectionKey::decode` when the `--connection-key` flag or positional argument is provided. The decoded material is passed to `ServerTlsMaterial::Pem` or `ClientTlsMaterial::Pem` variants, which bypass file loading and use the embedded PEM strings directly. + +## Entry points for modification + +- To change the key format or add versioning: modify `src/connkey.rs`. +- To add compression or encryption: consider extending the encode/decode pipeline in `ConnectionKey`. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/connkey.rs` | Connection key struct, encoding, decoding, validation | diff --git a/droid-wiki/features/index.md b/droid-wiki/features/index.md new file mode 100644 index 0000000..6b45307 --- /dev/null +++ b/droid-wiki/features/index.md @@ -0,0 +1,7 @@ +# Features + +rustunnel's user-visible and developer-visible capabilities: + +- [Connection keys](connection-keys.md) — single-string credential bundles +- [Stream multiplexing](stream-multiplexing.md) — multiple SOCKS5 streams over one tunnel +- [Reconnect and shutdown](reconnect-and-shutdown.md) — operational resilience diff --git a/droid-wiki/features/reconnect-and-shutdown.md b/droid-wiki/features/reconnect-and-shutdown.md new file mode 100644 index 0000000..78cb67f --- /dev/null +++ b/droid-wiki/features/reconnect-and-shutdown.md @@ -0,0 +1,57 @@ +# Reconnect and shutdown + +The connector is designed to recover from tunnel interruptions without dropping the SOCKS5 proxy. + +## Purpose + +Lab networks and development environments are unstable. The connector should tolerate listener restarts, network hiccups, and temporary failures without requiring manual intervention. + +## Reconnect behavior + +`run_reconnect_loop` in `src/tunnel.rs` continuously attempts to maintain a tunnel: + +1. Connect to the target address via TCP. +2. Perform TLS handshake and server certificate verification. +3. Send the HTTP auth request and wait for 200 OK. +4. On success, create a `StreamMux`, publish it via the `watch::channel`, and run the framing loop. +5. When the tunnel drops (framing task ends), log the disconnection and retry. + +### Backoff + +- Initial delay: 1 second +- Exponential doubling up to a maximum of 30 seconds +- Auth failures are terminal (the loop exits rather than retrying with a bad token) + +### SOCKS5 proxy during reconnect + +The SOCKS5 proxy stays bound. If a client connects while the tunnel is down, the proxy sees `mux_receiver` is `None` and rejects the connection with a warning log. Once the reconnect loop succeeds, new SOCKS5 requests work immediately. + +## Shutdown + +Both listener and connector handle graceful shutdown: + +- Unix: SIGINT or SIGTERM triggers shutdown via `tokio::signal::unix`. +- Windows: Ctrl+C or Ctrl+Break triggers shutdown via `tokio::signal::ctrl_c`. + +### Listener shutdown + +1. The shutdown signal breaks the `tokio::select!` in `run_listener`. +2. The TCP listener is dropped, releasing the bind port. +3. The SOCKS5 proxy task (if any) is aborted. +4. Active framing sessions end when their streams close. + +### Connector shutdown + +1. The shutdown signal breaks the `tokio::select!` in `run_socks5_proxy`. +2. The SOCKS5 listener is dropped. +3. The reconnect loop handle is aborted. +4. Existing SOCKS5 connections receive EOF when their streams close. + +E2E tests `e2e_graceful_shutdown_listener` and `e2e_graceful_shutdown_connector` verify that ports are released within 3 seconds of abort. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/tunnel.rs` | `run_reconnect_loop`, `run_socks5_proxy`, `run_listener`, shutdown handling | +| `src/signal.rs` | Cross-platform shutdown signal future | diff --git a/droid-wiki/features/stream-multiplexing.md b/droid-wiki/features/stream-multiplexing.md new file mode 100644 index 0000000..bf79884 --- /dev/null +++ b/droid-wiki/features/stream-multiplexing.md @@ -0,0 +1,47 @@ +# Stream multiplexing + +rustunnel carries multiple SOCKS5 connections over a single persistent TLS tunnel using stream IDs. + +## Purpose + +Avoid the overhead of establishing a new HTTPS connection for every SOCKS5 request. A single mTLS+auth session stays open, and multiple logical streams are interleaved via the binary framing protocol. + +## How it works + +### Connector side + +1. A local SOCKS5 client connects to the connector's SOCKS5 proxy. +2. The connector's `StreamMux::open_stream` allocates the next odd stream ID. +3. It sends a CONNECT frame with the target address encoded in the payload. +4. It waits for a CONNECT_REPLY frame (status 0x00 = OK). +5. On success, it returns a `mpsc::Receiver` for tunnel-to-SOCKS5 data. +6. The connector spawns a task to read from the SOCKS5 client and send DATA frames to the tunnel. +7. Another path reads from the `data_rx` channel and writes raw bytes back to the SOCKS5 client. + +### Listener side + +1. The listener receives a CONNECT frame with a stream ID and target address. +2. It resolves the target address and opens a TCP connection. +3. It spawns a task to read from the target and send DATA frames back to the tunnel. +4. It sends a CONNECT_REPLY frame to acknowledge the stream. +5. Incoming DATA frames for that stream ID are written to the target connection. +6. CLOSE frames trigger shutdown of the target connection. + +### Stream isolation + +Each stream has its own ID. Data from one stream never mixes with another. The E2E test `e2e_concurrent_streams_isolation` verifies this by sending unique payloads through two concurrent streams and asserting each echo response matches its original payload. + +## Key abstractions + +| Type | File | Description | +| ---- | ---- | ----------- | +| `StreamMux` | `src/tunnel.rs` | Connector-side multiplexer | +| `ServerStreamEntry` | `src/tunnel.rs` | Listener-side per-stream target write half | +| `ConnectorStreamEntry` | `src/tunnel.rs` | Connector-side per-stream data channel and reply channel | + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/tunnel.rs` | StreamMux, open_stream, dispatch_frame, framing loops | +| `src/framing.rs` | Frame types, CONNECT/DATA/CLOSE/ERROR encoding | diff --git a/droid-wiki/how-to-contribute/debugging.md b/droid-wiki/how-to-contribute/debugging.md new file mode 100644 index 0000000..a7975aa --- /dev/null +++ b/droid-wiki/how-to-contribute/debugging.md @@ -0,0 +1,53 @@ +# Debugging + +## Logs + +rustunnel uses `tracing` with an `EnvFilter` defaulting to `info`. Set `RUST_LOG=debug` or `RUST_LOG=trace` for more detail. + +```sh +RUST_LOG=debug ./target/release/rustunnel listen --listen 127.0.0.1:4180 ... +``` + +All logged secrets are redacted. The `Redacted` wrapper shows `[REDACTED(len=N)]` instead of the raw value. If you see raw secrets in logs, that is a bug. + +## Common errors + +### TLS handshake failures + +If the connector fails with a TLS error, check: + +- Do the listener and connector share the same CA certificate? +- Does the server certificate's SAN match the target hostname? Use `generate` with the correct `--advertise` address, or pass `--insecure-skip-tls-verify` (lab only). +- Are the certificate files readable and valid PEM? + +rustunnel prints TLS hint messages for common failures: + +- `BadSignature` — the listener certificate is not signed by the CA trusted by the connector. +- `NotValidForName` — the listener certificate does not match the target host. + +### Auth failures + +- Verify the token file is not empty and has no trailing whitespace. +- The listener and connector must use the same token value. +- Auth failures are terminal for the connector's reconnect loop (it will not retry with a bad token). + +### Port conflicts + +If binding fails with "address already in use": + +- Check for orphaned processes from previous test runs. +- The E2E tests use random free ports; manual testing may need explicit port management. + +### SOCKS5 connection refused + +If curl fails through the SOCKS5 proxy: + +- Is the connector running and the tunnel established? +- Is the SOCKS5 proxy port correct? +- Is the target address reachable from the listener side (for default mode) or connector side (when using `--socks` on the listener)? + +## Tracing + +The connector logs effective config at startup, including the target, SOCKS5 address, and redacted auth token. The listener logs its bind address, TLS fingerprints, and auth token (redacted). + +State transitions are logged at `info` level: "Tunnel session established", "Tunnel disconnected", "Reconnecting in Ns". diff --git a/droid-wiki/how-to-contribute/development-workflow.md b/droid-wiki/how-to-contribute/development-workflow.md new file mode 100644 index 0000000..d38976d --- /dev/null +++ b/droid-wiki/how-to-contribute/development-workflow.md @@ -0,0 +1,25 @@ +# Development workflow + +## Branching + +Use feature branches off `master`. The project is small enough that long-lived branches are unnecessary. + +## Coding + +- Follow existing formatting (enforced by `cargo fmt`). +- Use `anyhow` for application-level errors and `thiserror` for library-style error enums. +- Prefer `tracing` over `println!` for diagnostic output. Secrets must be wrapped with `Redacted` before logging. +- Use `Arc` for shared strings across async boundaries (e.g., auth tokens in `ListenerConfig` and `ConnectorConfig`). +- Keep synchronous protocol parsing in dedicated modules (`socks5.rs`, `framing.rs`) and async I/O in `tunnel.rs`. + +## Testing + +- Inline tests live in `#[cfg(test)]` modules at the bottom of each source file. +- Use `tempfile::tempdir()` for generating test credentials. Clean up temp directories with `std::fs::remove_dir_all`. +- Use `std::net::TcpListener::bind("127.0.0.1:0")` to get free ports for E2E tests. +- E2E tests spawn tokio tasks for listener, connector, and target servers, then abort them in cleanup. + +## Committing + +- Follow conventional commit style: `feat:`, `fix:`, `docs:`, `chore:`. +- Keep commits focused. The git history shows the evolution from HTTP-per-stream to multiplexed tunnel; preserve that narrative clarity. diff --git a/droid-wiki/how-to-contribute/index.md b/droid-wiki/how-to-contribute/index.md new file mode 100644 index 0000000..1f4b362 --- /dev/null +++ b/droid-wiki/how-to-contribute/index.md @@ -0,0 +1,29 @@ +# How to contribute + +rustunnel is a small, focused project. Contributions should align with the project's purpose: a lab and development tunneling tool with strong security defaults. + +## Work pickup + +- Check the CI status before starting work. +- Open issues or PRs against the `master` branch. +- Keep changes focused. The codebase is small enough that large refactors should be discussed first. + +## PR process + +1. Run the full test suite locally: `cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test --all-targets` +2. Ensure E2E tests pass if your change touches the tunnel or SOCKS5 path. +3. Write tests for new behavior. The project uses inline `#[cfg(test)]` modules. +4. Update relevant wiki pages if your change adds or removes features. + +## Review expectations + +- Security-sensitive changes (TLS, auth, framing) get extra scrutiny. +- All CI jobs must pass: fmt, clippy, test, build, smoke, and E2E. +- The project runs CI on Linux, macOS, and Windows. Avoid platform-specific code unless necessary (the signal handling module is the only current exception). + +## Definition of done + +- Code is formatted with `rustfmt`. +- Clippy warnings are resolved. +- Tests cover the new behavior, including edge cases. +- No `TODO` or `FIXME` comments are introduced (the project currently has zero). diff --git a/droid-wiki/how-to-contribute/patterns-and-conventions.md b/droid-wiki/how-to-contribute/patterns-and-conventions.md new file mode 100644 index 0000000..5bd3b8a --- /dev/null +++ b/droid-wiki/how-to-contribute/patterns-and-conventions.md @@ -0,0 +1,55 @@ +# Patterns and conventions + +## Error handling + +The project uses a layered error strategy: + +- `thiserror` enums for domain-specific errors (`TlsError`, `AuthError`, `TunnelError`, `HostError`, `Socks5Error`, `FrameError`). These live in `src/errors.rs` and `src/socks5.rs` / `src/framing.rs`. +- `anyhow` for application-level propagation. CLI commands return `anyhow::Result` and `main` exits with `process::exit(exit_code)`. +- All errors are designed to be actionable and fail-closed. There are no silent degradation paths to insecure modes. + +## Secret handling + +- The `Redacted` wrapper in `src/redact.rs` must be used before logging any secret. +- `Redacted::inner()` and `into_inner()` are marked `#[allow(dead_code)]` to discourage use, but available for file writes. +- `is_sensitive()` detects PEM private key blocks. `is_auth_token()` detects long alphanumeric strings. +- `is_connection_key()` detects strings starting with `rtun1.`. + +## Async patterns + +- Tokio's multi-threaded runtime is used on both listener and connector sides. +- `tokio::spawn` is used for per-connection and per-stream tasks. +- `tokio::select!` with `biased;` prioritizes shutdown signals over new connections. +- `watch::channel` is used for publishing the active `StreamMux` to the SOCKS5 proxy (supports reconnect). +- `mpsc::channel` and `oneshot::channel` are used for stream data and CONNECT_REPLY coordination. + +## Constant-time comparison + +Auth token comparison in `src/tunnel.rs` uses a manual constant-time XOR loop to prevent timing attacks: + +```rust +fn constant_time_compare(a: &str, b: &str) -> bool { + let a_bytes = a.as_bytes(); + let b_bytes = b.as_bytes(); + if a_bytes.len() != b_bytes.len() { + return false; + } + let mut result: u8 = 0; + for (x, y) in a_bytes.iter().zip(b_bytes.iter()) { + result |= x ^ y; + } + result == 0 +} +``` + +## Frame reader partial read discipline + +`FrameReader` in `src/framing.rs` never consumes bytes from the buffer until a complete frame (header + payload) is available. This prevents corruption when TCP segments arrive in arbitrary chunks. + +## TLS material as enum + +Both `ServerTlsMaterial` and `ClientTlsMaterial` are enums with `Paths` and `Pem` variants. This lets the listener and connector accept either file paths (from CLI args) or embedded PEM strings (from connection keys). + +## Insecure mode + +`--insecure-skip-tls-verify` exists for DPI/intercepted environments only. When enabled, the listener skips client certificate verification and the connector skips server certificate verification. This is logged at `warn` level. diff --git a/droid-wiki/how-to-contribute/testing.md b/droid-wiki/how-to-contribute/testing.md new file mode 100644 index 0000000..a169e14 --- /dev/null +++ b/droid-wiki/how-to-contribute/testing.md @@ -0,0 +1,75 @@ +# Testing + +## Test organization + +All tests are inline `#[cfg(test)]` modules at the bottom of each source file. There are no standalone test files or integration test directories. + +## Running tests + +```sh +cargo test --all-targets +``` + +This runs unit tests and doc tests across all modules. + +## Test patterns + +### Credential generation tests + +Tests in `src/generate.rs` verify that `generate()` creates all expected files, that certificates contain valid PEM blocks, that private keys are detected as sensitive by the redaction module, and that config JSON is valid. + +### Framing tests + +Tests in `src/framing.rs` cover: + +- Frame roundtrips for all frame types (CONNECT, CONNECT_REPLY, DATA, CLOSE, ERROR) +- Partial read handling — buffers must not be consumed until a complete frame is available +- Multiple frames in one buffer +- Target address encoding/decoding for IPv4 and domain names + +### SOCKS5 tests + +Tests in `src/socks5.rs` cover: + +- Greeting parsing (no-auth, username/password, invalid versions) +- Auth request parsing (valid and empty credentials) +- CONNECT request parsing (IPv4, domain, unsupported commands) +- Reply building +- Target resolution (IPv4 direct, domain DNS lookup) +- Malformed input handling + +### TLS tests + +Tests in `src/tls.rs` cover: + +- Loading certificates and keys from files and PEM strings +- Building valid server and client configs +- Identity validation against SANs and CN +- Certificate fingerprinting +- Expiration checking +- Insecure config variants + +### Tunnel tests + +Tests in `src/tunnel.rs` are the most extensive. They include: + +- **Auth tests:** valid mTLS + auth, invalid token, missing client cert +- **E2E tests:** basic forwarding, server-side SOCKS5 reaching connector-side targets, concurrent stream isolation, target failure handling, listener-side target origin verification +- **Operational tests:** reconnect restoring traffic after tunnel drop, graceful shutdown of listener and connector, shutdown during active streams +- **Log tests:** verify effective config is logged with redacted tokens, verify auth failures don't leak token values in error messages + +## E2E test structure + +Typical E2E tests follow this pattern: + +1. Generate test credentials with `generate_test_creds()`. +2. Pick a free port with `get_free_port()`. +3. Spawn the listener as a tokio task. +4. Spawn the connector (and optionally a target server) as tokio tasks. +5. Sleep briefly to let services start. +6. Connect via SOCKS5, send data, verify response. +7. Abort all tasks in cleanup. + +## CI test matrix + +Tests run on `ubuntu-latest`, `windows-latest`, and `macos-latest`. diff --git a/droid-wiki/how-to-contribute/tooling.md b/droid-wiki/how-to-contribute/tooling.md new file mode 100644 index 0000000..122a15a --- /dev/null +++ b/droid-wiki/how-to-contribute/tooling.md @@ -0,0 +1,40 @@ +# Tooling + +## Build system + +Cargo is the sole build tool. No custom build scripts or Makefiles are used. + +- `cargo build` — debug build +- `cargo build --release` — optimized release build +- `cargo check` — fast type-checking + +## Linting and formatting + +- `cargo fmt --check` — enforces rustfmt formatting +- `cargo clippy --all-targets -- -D warnings` — runs clippy on all targets with warnings as errors + +## CI/CD + +GitHub Actions runs the full matrix on every push and PR: + +| Job | Platforms | Purpose | +| --- | --------- | ------- | +| `fmt` | Ubuntu | Format check | +| `clippy` | Linux, macOS, Windows | Lint check | +| `test` | Linux, macOS, Windows | Unit and inline tests | +| `build` | Linux, macOS, Windows | Debug build | +| `release-build` | Linux, macOS, Windows | Release build + artifact upload | +| `smoke` | Linux, macOS, Windows | CLI smoke tests (--help, version, invalid command/port) | +| `e2e-minimal` | Linux, macOS, Windows | Full end-to-end flow: generate, listen, connect, SOCKS5, curl | + +The E2E job uses `cargo run --release --` for OS-neutral binary invocation and includes cross-platform cleanup helpers (`kill_process`, `kill_by_port`) invoked via `trap cleanup EXIT`. + +## Dependencies + +Dependencies are managed in `Cargo.toml`. Notable choices: + +- `tokio` with `full` features for async runtime +- `rustls` with `ring` for TLS (no OpenSSL) +- `rcgen` for certificate generation +- `clap` with `derive` for CLI parsing +- `tracing` and `tracing-subscriber` for structured logging diff --git a/droid-wiki/lore.md b/droid-wiki/lore.md new file mode 100644 index 0000000..634a20a --- /dev/null +++ b/droid-wiki/lore.md @@ -0,0 +1,53 @@ +# Lore + +## Project origins + +rustunnel was created in June 2026 as a lab and development tunneling tool. The goal was a cross-platform, self-contained Rust binary that could tunnel traffic over HTTPS with mutual TLS and an application auth token, without relying on external infrastructure. + +## Eras + +### Foundation (June 3, 2026) + +The first commit (`eb4ef2b`) laid out the project skeleton: CLI surface using clap, a `generate` command for self-signed certificates, secret redaction, and a cross-platform CI matrix. A follow-up commit (`70411e5`) addressed initial scrutiny findings. + +### Secure tunnel implementation (June 3, 2026) + +The core tunnel logic landed in `393100d`, implementing HTTPS mTLS with an auth handshake. Shortly after, `76bda0b` addressed secure-tunnel scrutiny findings. Dependency updates and CLI refinements followed in `fc776fc`. + +### SOCKS5 and multiplexing (June 3, 2026) + +Three major commits transformed the architecture: + +- `6466753` — Implemented a SOCKS5 proxy with stream multiplexing over HTTPS. This replaced the earlier design where each SOCKS5 request opened a new HTTPS connection. +- `f871ba7` — Replaced per-SOCKS-request HTTPS connections with a single persistent multiplexed tunnel session. +- `d6a0a40` — Replaced HTTP-shaped forwarding with a true persistent multiplexed tunnel using a custom binary framing protocol. + +### Framing hardening (June 3, 2026) + +- `b67248e` — Fixed frame header corruption on partial reads. The framing reader was refined to never consume bytes from the buffer until a complete frame (header + payload) is available. + +### Operational features (June 4, 2026) + +- `10b8b24` — Added operational reconnect and shutdown handling. The connector now has a background reconnect loop with exponential backoff, and both sides handle graceful shutdown on SIGINT/SIGTERM. +- `32a29c8` — Fixed flaky generate tests and formatted `tunnel.rs`. +- `b0210a6` — Improved cross-platform CI E2E cleanup with `trap` and portable port killing. +- `241d7d1` — Unmasked SOCKS5 E2E curl failures in CI and added response content assertions. +- `cc1eb55` — Added the project README. +- `a758e8e` — Added connection key encoding (`connkey.rs`) and the full tunnel implementation integrating keys into `listen` and `connect`. + +## Longest-standing code + +Given the project's short lifespan (2 days), all code is roughly the same age. The modules that have survived the most rewrites are: + +- `src/cli.rs` — CLI structure has remained stable since the first commit. +- `src/redact.rs` — Secret redaction has not changed since the foundation commit. +- `src/errors.rs` — Error types have been extended but the core structure is original. + +## Deprecated features + +None so far. The project is new enough that no features have been removed. + +## Major rewrites + +- **From HTTP-per-stream to persistent multiplexed tunnel (June 3, 2026)** — The original SOCKS5 implementation opened a new HTTPS connection for every stream. This was replaced by a single persistent tunnel with a binary framing protocol, dramatically reducing connection overhead. +- **From HTTP forwarding to binary framing (June 3, 2026)** — After multiplexing was introduced, the data plane switched from raw HTTP forwarding to a lightweight binary protocol with CONNECT/DATA/CLOSE/ERROR frames. diff --git a/droid-wiki/overview/architecture.md b/droid-wiki/overview/architecture.md new file mode 100644 index 0000000..04cba3f --- /dev/null +++ b/droid-wiki/overview/architecture.md @@ -0,0 +1,86 @@ +# Architecture + +rustunnel is organized as a single Rust binary crate with a modular source tree. The architecture separates CLI parsing, credential generation, TLS configuration, SOCKS5 handling, binary framing, and the core tunnel engine into distinct modules. + +## Component diagram + +```mermaid +graph TD + A[CLI `src/cli.rs`] --> B[Main dispatch `src/main.rs`] + B --> C[Credential generation `src/generate.rs`] + B --> D[Tunnel engine `src/tunnel.rs`] + B --> E[Connection keys `src/connkey.rs`] + D --> F[TLS stack `src/tls.rs`] + D --> G[SOCKS5 proxy `src/socks5.rs`] + D --> H[Framing protocol `src/framing.rs`] + D --> I[Signal handling `src/signal.rs`] + F --> J[Rustls + ring] + G --> K[Tokio TCP] + H --> L[Bytes + tokio-util] + C --> M[rcgen] +``` + +## Data flow + +A typical tunnel session flows like this: + +1. **Listener** binds a TCP socket and wraps it with `tokio_rustls::TlsAcceptor`. +2. Connector initiates a TCP connection and upgrades it with `tokio_rustls::TlsConnector`. +3. Both sides perform mTLS handshakes: the listener requires a client certificate signed by the shared CA; the connector validates the server certificate against the same CA. +4. The connector sends an HTTP POST to `/tunnel` with an `X-Rustunnel-Token` header. +5. The listener validates the token with a constant-time comparison (`src/tunnel.rs`). +6. On success, both sides switch from HTTP/1.1 to a binary framing protocol (`src/framing.rs`). +7. The connector's local SOCKS5 proxy accepts client connections and opens multiplexed streams over the single persistent tunnel. +8. The listener decodes each stream's target address, opens a local TCP connection to that target, and forwards data bidirectionally. + +```mermaid +sequenceDiagram + participant Client as Local app + participant Socks as SOCKS5 proxy + participant Conn as Connector + participant TLS as mTLS tunnel + participant List as Listener + participant Target as Target server + + Client->>Socks: SOCKS5 CONNECT + Socks->>Conn: open stream + Conn->>TLS: TLS handshake + TLS->>List: accept + verify client cert + Conn->>List: POST /tunnel + auth token + List->>Conn: 200 OK + Conn->>List: binary framing CONNECT frame + List->>Target: TCP connect + Target->>List: response + List->>Conn: DATA frame + Conn->>Socks: raw bytes + Socks->>Client: response +``` + +## Module responsibilities + +| Module | Responsibility | +| ------ | -------------- | +| `src/cli.rs` | clap-based CLI definitions for all subcommands and arguments | +| `src/main.rs` | Entry point, crypto provider installation, tracing init, command dispatch | +| `src/tunnel.rs` | Core tunnel engine: listener accept loop, connector reconnect loop, SOCKS5 proxy, stream mux, HTTP auth handshake, framing loop | +| `src/tls.rs` | rustls server and client config builders, certificate loading, identity validation, insecure verifier | +| `src/socks5.rs` | SOCKS5 greeting, auth, CONNECT request parsing, target resolution | +| `src/framing.rs` | Binary frame serialization/deserialization, `FrameReader`, `FrameWriter` | +| `src/generate.rs` | Self-signed CA, server cert, client cert, and auth token generation with rcgen | +| `src/connkey.rs` | `ConnectionKey` struct: encode/decode all material as a base64 string | +| `src/redact.rs` | `Redacted` wrapper and helpers to prevent secrets from leaking into logs | +| `src/errors.rs` | `thiserror`-based error enums for TLS, auth, tunnel, and host operations | +| `src/config.rs` | JSON configuration file format for generated credential sets | +| `src/signal.rs` | Cross-platform shutdown signal handler (SIGINT/SIGTERM on Unix, Ctrl+C on Windows) | + +## Threading model + +The project uses Tokio's multi-threaded runtime. Each side (listener and connector) spawns tasks for: + +- Accepting new TCP connections +- TLS handshakes +- Per-tunnel framing loops +- Per-stream target forwarding +- SOCKS5 client handling + +The connector's SOCKS5 proxy stays bound permanently while a background reconnect loop maintains the tunnel. When the tunnel drops, existing streams receive EOF and new streams wait for the reconnect loop to re-establish the session. diff --git a/droid-wiki/overview/getting-started.md b/droid-wiki/overview/getting-started.md new file mode 100644 index 0000000..4d12bba --- /dev/null +++ b/droid-wiki/overview/getting-started.md @@ -0,0 +1,101 @@ +# Getting started + +## Prerequisites + +- Rust 1.85+ (the project uses Edition 2024) +- A POSIX shell or PowerShell for running commands +- curl (for E2E testing) + +## Build + +```sh +cd rustunnel +cargo build --release +``` + +The release binary is produced at `target/release/rustunnel`. + +## Run tests + +```sh +cargo fmt --check +cargo clippy --all-targets -- -D warnings +cargo check +cargo test --all-targets +``` + +The test suite includes unit tests for framing, SOCKS5 parsing, TLS config building, credential generation, connection key roundtrips, and full end-to-end tunnel flows. + +## Generate credentials + +```sh +./target/release/rustunnel generate --out ./creds +``` + +This creates: + +- `creds/ca.pem` — CA certificate +- `creds/ca.key` — CA private key +- `creds/server.crt` — server certificate +- `creds/server.key` — server private key +- `creds/client.crt` — client certificate +- `creds/client.key` — client private key +- `creds/token.txt` — auth token +- `creds/config.json` — configuration file with paths and defaults + +## Run a tunnel + +### Start the listener + +```sh +./target/release/rustunnel listen \ + --listen 127.0.0.1:4180 \ + --cert ./creds/server.crt \ + --key ./creds/server.key \ + --ca-cert ./creds/ca.pem \ + --auth-token-file ./creds/token.txt +``` + +### Start the connector + +```sh +./target/release/rustunnel connect \ + --target 127.0.0.1:4180 \ + --socks 127.0.0.1:1180 \ + --cert ./creds/client.crt \ + --key ./creds/client.key \ + --ca-cert ./creds/ca.pem \ + --auth-token-file ./creds/token.txt +``` + +### Use the SOCKS5 proxy + +```sh +curl --proxy socks5h://127.0.0.1:1180 http://127.0.0.1:4181/ +``` + +## Using a connection key + +Instead of passing individual certificate paths, you can generate a single connection key: + +```sh +./target/release/rustunnel keygen --target 127.0.0.1:4180 +``` + +Then start the listener with the key (it auto-generates credentials): + +```sh +./target/release/rustunnel listen --connection-key +``` + +And connect with the same key: + +```sh +./target/release/rustunnel connect +``` + +## Defaults + +- Listener bind address: `0.0.0.0:4180` +- Connector target: required (or taken from connection key) +- Connector SOCKS5 proxy: `127.0.0.1:1180` diff --git a/droid-wiki/overview/glossary.md b/droid-wiki/overview/glossary.md new file mode 100644 index 0000000..38ac0c5 --- /dev/null +++ b/droid-wiki/overview/glossary.md @@ -0,0 +1,19 @@ +# Glossary + +**Connection key** — a base64-encoded JSON blob (prefix `rtun1.`) that bundles the CA certificate, server certificate, server key, client certificate, client key, target address, auth token, and version. Generated by `rustunnel keygen` and consumed by `rustunnel listen` and `rustunnel connect`. + +**Connector** — the `rustunnel connect` side. Establishes an outbound HTTPS+mTLS connection to the listener, authenticates, then exposes a local SOCKS5 proxy for local applications. + +**Framing protocol** — the binary protocol used after the HTTP auth handshake completes. Frame types: CONNECT, CONNECT_REPLY, DATA, CLOSE, ERROR. All multi-byte integers are big-endian. Max payload per frame is 4 MiB. + +**Listener** — the `rustunnel listen` side. Binds an HTTPS server that accepts mTLS connections, validates the auth token, then upgrades the connection to the binary framing protocol. + +**mTLS** — mutual TLS. Both the listener and connector present certificates and verify the peer's certificate against a shared CA. rustunnel uses rustls with ring as the crypto provider. + +**Security gate** — one of the validation steps that must pass before tunnel forwarding begins: TLS handshake, client certificate verification (listener), server certificate verification (connector), application auth token validation. + +**SOCKS5 proxy** — the local proxy exposed by the connector. Implements RFC 1928 CONNECT with optional username/password auth (RFC 1929). Supports IPv4 and domain name targets. + +**Stream ID** — a monotonically increasing odd integer (1, 3, 5, ...) assigned by the connector for each SOCKS5 connection. Even IDs are reserved for future server-initiated streams. + +**StreamMux** — the connector-side structure that manages multiple concurrent streams over a single tunnel connection. Maps stream IDs to per-stream data channels. diff --git a/droid-wiki/overview/index.md b/droid-wiki/overview/index.md new file mode 100644 index 0000000..a79f618 --- /dev/null +++ b/droid-wiki/overview/index.md @@ -0,0 +1,20 @@ +# rustunnel + +`rustunnel` is a cross-platform Rust CLI tool for lab and development tunneling over HTTPS with mutual TLS (mTLS) and an application-level auth token. It exposes a local SOCKS5 proxy on the connector side and forwards traffic only after HTTPS, certificate, and auth checks succeed. + +The tool is designed for local labs, development environments, and controlled testing where you own or are authorized to operate both endpoints. It is not a production-grade tunnel or a mechanism for accessing systems without permission. + +## What it does + +- **Listener** (`rustunnel listen`) — binds an HTTPS server that accepts mTLS connections from connectors, validates an auth token, then upgrades the connection to a persistent binary-framed tunnel. +- **Connector** (`rustunnel connect`) — connects to the listener over HTTPS with mTLS, authenticates, then exposes a local SOCKS5 proxy that forwards traffic through the tunnel. +- **Credential generation** (`rustunnel generate`) — creates a self-signed CA, server certificate, client certificate, and auth token for a local session. +- **Connection keys** (`rustunnel keygen`) — bundles all credential material into a single base64-encoded string that can be copy-pasted between machines. + +## Quick links + +- [Architecture](../overview/architecture.md) +- [Getting started](../overview/getting-started.md) +- [Tunnel engine](../systems/tunnel-engine.md) +- [TLS stack](../systems/tls-stack.md) +- [Security posture](../security.md) diff --git a/droid-wiki/reference/configuration.md b/droid-wiki/reference/configuration.md new file mode 100644 index 0000000..f71fd75 --- /dev/null +++ b/droid-wiki/reference/configuration.md @@ -0,0 +1,82 @@ +# Configuration + +## CLI arguments + +### `listen` + +| Argument | Default | Description | +| -------- | ------- | ----------- | +| `--listen` | `0.0.0.0:4180` | HTTPS listener bind address | +| `--advertise` | derived from `--listen` | Public address in auto-generated connection keys | +| `--connection-key` | none | Reusable key (env: `RUSTUNNEL_KEY`) | +| `--socks` | none | Server-side SOCKS5 proxy address | +| `--cert` | "" | Server certificate path | +| `--key` | "" | Server private key path | +| `--ca-cert` | "" | CA certificate path | +| `--auth-token` | "" | Auth token value | +| `--auth-token-file` | none | Auth token file path | +| `--insecure-skip-tls-verify` | false | Skip client cert verification | + +### `connect` + +| Argument | Default | Description | +| -------- | ------- | ----------- | +| `CONNECTION_KEY` | none | Positional connection key | +| `--target` | from key | Listener address | +| `--connection-key` | none | Key via flag (env: `RUSTUNNEL_KEY`) | +| `--socks` | `127.0.0.1:1180` | Local SOCKS5 proxy address | +| `--cert` | "" | Client certificate path | +| `--key` | "" | Client private key path | +| `--ca-cert` | "" | CA certificate path | +| `--auth-token` | "" | Auth token value | +| `--auth-token-file` | none | Auth token file path | +| `--insecure-skip-tls-verify` | false | Skip server cert verification | + +### `generate` + +| Argument | Default | Description | +| -------- | ------- | ----------- | +| `--out` | `.` | Output directory | +| `--ca-name` | `rustunnel-ca` | CA common name | +| `--server-name` | `rustunnel-server` | Server common name | +| `--client-name` | `rustunnel-client` | Client common name | + +### `keygen` + +| Argument | Default | Description | +| -------- | ------- | ----------- | +| `--target` | `127.0.0.1:4180` | Listener address embedded in key | +| `--ca-name` | `rustunnel-ca` | CA common name | +| `--server-name` | `rustunnel-server` | Server common name | +| `--client-name` | `rustunnel-client` | Client common name | + +## Environment variables + +- `RUST_LOG` — tracing filter (e.g., `debug`, `trace`) +- `RUSTUNNEL_KEY` — connection key for `listen` and `connect` + +## Config file format + +`config.json` (generated by `rustunnel generate`): + +```json +{ + "listen_address": "0.0.0.0", + "listen_port": 4180, + "socks_address": "127.0.0.1", + "socks_port": 1180, + "ca_cert_path": "ca.pem", + "server_cert_path": "server.crt", + "server_key_path": "server.key", + "client_cert_path": "client.crt", + "client_key_path": "client.key", + "auth_token_path": "token.txt" +} +``` + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/cli.rs` | clap argument definitions and defaults | +| `src/config.rs` | `Config` struct for JSON serialization | diff --git a/droid-wiki/reference/data-models.md b/droid-wiki/reference/data-models.md new file mode 100644 index 0000000..4dc4613 --- /dev/null +++ b/droid-wiki/reference/data-models.md @@ -0,0 +1,93 @@ +# Data models + +## ConnectionKey + +```rust +pub struct ConnectionKey { + pub version: u8, // must be 1 + pub target: String, // listener address + pub ca_cert_pem: String, + pub server_cert_pem: String, + pub server_key_pem: String, + pub client_cert_pem: String, + pub client_key_pem: String, + pub auth_token: String, +} +``` + +Encoded as: `rtun1.` + base64url(JSON) — no padding. + +## Config + +```rust +pub struct Config { + pub listen_address: String, + pub listen_port: u16, + pub socks_address: String, + pub socks_port: u16, + pub ca_cert_path: String, + pub server_cert_path: String, + pub server_key_path: String, + pub client_cert_path: String, + pub client_key_path: String, + pub auth_token_path: String, +} +``` + +## Frame + +```rust +pub struct Frame { + pub frame_type: u8, // 0x01 CONNECT, 0x02 CONNECT_REPLY, 0x03 DATA, 0x04 CLOSE, 0x05 ERROR + pub stream_id: u64, // odd for client-initiated + pub flags: u8, // reserved (0x00) + pub payload: Bytes, +} +``` + +Wire format: 1 byte type + 8 bytes stream_id (big-endian) + 4 bytes len (big-endian) + 1 byte flags + len bytes payload. + +Max payload: 4 MiB. + +## Socks5Target + +```rust +pub enum Socks5Target { + Ipv4(std::net::Ipv4Addr, u16), + Domain { domain: String, port: u16 }, +} +``` + +## GeneratedMaterial + +```rust +pub struct GeneratedMaterial { + pub ca_cert_pem: String, + pub ca_key_pem: String, + pub server_cert_pem: String, + pub server_key_pem: String, + pub client_cert_pem: String, + pub client_key_pem: String, + pub auth_token: String, +} +``` + +## Error enums + +- `TlsError` — missing/invalid certificates, identity mismatch, expiration, verification failure +- `AuthError` — missing or invalid token +- `TunnelError` — TLS error, auth error, bind error, connection refused, port in use, protocol error +- `HostError` — DNS resolution failure, invalid hostname, no addresses +- `Socks5Error` — invalid version, unsupported auth/command/addr type, parse errors +- `FrameError` — frame too short, payload overflow, I/O error, protocol error + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/connkey.rs` | `ConnectionKey` | +| `src/config.rs` | `Config` | +| `src/framing.rs` | `Frame`, `FrameReader`, `FrameWriter` | +| `src/socks5.rs` | `Socks5Target`, `Socks5State`, `Socks5Error` | +| `src/generate.rs` | `GeneratedMaterial` | +| `src/errors.rs` | `TlsError`, `AuthError`, `TunnelError`, `HostError` | diff --git a/droid-wiki/reference/dependencies.md b/droid-wiki/reference/dependencies.md new file mode 100644 index 0000000..a9cf859 --- /dev/null +++ b/droid-wiki/reference/dependencies.md @@ -0,0 +1,45 @@ +# Dependencies + +## Runtime dependencies + +| Crate | Version | Purpose | +| ----- | ------- | ------- | +| `clap` | 4 | CLI parsing with derive macros | +| `tokio` | 1 | Async runtime (full features) | +| `tokio-util` | 0.7 | I/O utilities | +| `tracing` | 0.1 | Structured logging | +| `tracing-subscriber` | 0.3 | Log formatting with env-filter | +| `anyhow` | 1 | Application error handling | +| `thiserror` | 1 | Library error enums | +| `serde` | 1 | Serialization (derive) | +| `serde_json` | 1 | JSON encoding/decoding | +| `rcgen` | 0.14 | Certificate generation | +| `x509-parser` | 0.16 | Certificate parsing for identity validation | +| `time` | 0.3 | Date/time formatting for certificate validity | +| `rand` | 0.8 | Random token generation | +| `hex` | 0.4 | Hex encoding for auth tokens | +| `rustls` | 0.23 | TLS implementation (ring provider) | +| `rustls-pki-types` | 1 | rustls type definitions | +| `rustls-pemfile` | 2 | PEM parsing for certificates and keys | +| `tokio-rustls` | 0.26 | Tokio integration for rustls | +| `hyper` | 1 | HTTP types (used for auth handshake) | +| `hyper-util` | 0.1 | HTTP utilities | +| `http-body-util` | 0.1 | HTTP body utilities | +| `tower-service` | 0.3 | Service trait (Hyper ecosystem) | +| `base64` | 0.22 | Connection key encoding | +| `sha2` | 0.10 | Certificate fingerprinting | +| `bytes` | 1 | Byte buffers for framing | +| `futures` | 0.3 | Future utilities | +| `url` | 2 | URL parsing | + +## Dev dependencies + +| Crate | Version | Purpose | +| ----- | ------- | ------- | +| `tempfile` | 3 | Temporary directories for tests | + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `Cargo.toml` | Dependency declarations and features | diff --git a/droid-wiki/reference/index.md b/droid-wiki/reference/index.md new file mode 100644 index 0000000..86d92cb --- /dev/null +++ b/droid-wiki/reference/index.md @@ -0,0 +1,7 @@ +# Reference + +Configuration, data models, and external dependencies. + +- [Configuration](configuration.md) +- [Data models](data-models.md) +- [Dependencies](dependencies.md) diff --git a/droid-wiki/security.md b/droid-wiki/security.md new file mode 100644 index 0000000..ffa02cb --- /dev/null +++ b/droid-wiki/security.md @@ -0,0 +1,59 @@ +# Security + +rustunnel's security posture is built on multiple layers: TLS transport, mutual authentication, application-level auth tokens, secret redaction, and fail-closed error handling. + +## Security gates + +Before any traffic flows, four gates must pass: + +1. **TLS handshake** — both sides negotiate a TLS 1.2+ connection via rustls with ring. +2. **Server certificate verification** — the connector verifies the listener's certificate against the shared CA. +3. **Client certificate verification** — the listener verifies the connector's certificate against the same CA (skipped in insecure mode). +4. **Application auth token** — the connector sends an `X-Rustunnel-Token` header; the listener validates it with constant-time comparison. + +The E2E test `auth_required_in_addition_to_mtls` verifies that valid mTLS alone is not sufficient; the correct auth token is also required. + +## mTLS + +- The listener requires client certificates signed by the shared CA. It uses `WebPkiClientVerifier`. +- The connector presents its client certificate and validates the server certificate against the CA. +- Certificate identities are validated against SANs (Subject Alternative Names) and CN. The server certificate includes SANs for `localhost`, `rustunnel`, and `127.0.0.1`, plus the bind IP if applicable. + +## Auth token + +- Generated as 32 random bytes encoded as 64 hex characters. +- Compared with a constant-time XOR loop to prevent timing attacks. +- Never logged in raw form. All log output uses the `Redacted` wrapper. +- Auth failures are terminal: the connector's reconnect loop exits on auth failure rather than retrying. + +## Secret redaction + +- `Redacted` in `src/redact.rs` wraps strings and displays `[REDACTED(len=N)]`. +- `redact_pem()` shows only the PEM block type and line count. +- `redact_config_json()` scrubs sensitive keys from JSON. +- `is_sensitive()` detects PEM private key blocks. +- `is_auth_token()` detects long alphanumeric strings. +- `is_connection_key()` detects strings starting with `rtun1.`. + +The E2E test `ops_auth_failure_is_actionable_and_redacted` verifies that error messages do not contain raw token values. + +## Insecure mode + +`--insecure-skip-tls-verify` disables certificate verification on either side. This is intended for DPI/intercepted environments only. When enabled: + +- The listener uses `with_no_client_auth()` instead of `WebPkiClientVerifier`. +- The connector uses a custom `InsecureServerCertVerifier` that accepts any certificate. +- A `warn` level log is emitted. + +## Fail-closed design + +All error paths reject the connection rather than falling back to an insecure mode. There are no bypasses around TLS, certificate verification, or auth. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/tls.rs` | TLS config, certificate loading, identity validation | +| `src/tunnel.rs` | Auth handshake, constant-time comparison, security gate tests | +| `src/redact.rs` | Secret redaction wrappers and helpers | +| `src/errors.rs` | Error enums for TLS, auth, tunnel | diff --git a/droid-wiki/systems/credential-generation.md b/droid-wiki/systems/credential-generation.md new file mode 100644 index 0000000..7b6a6b0 --- /dev/null +++ b/droid-wiki/systems/credential-generation.md @@ -0,0 +1,45 @@ +# Credential generation + +The credential generation module in `src/generate.rs` creates all the certificate and key material needed for a rustunnel session. + +## Purpose + +Generate a self-signed CA, server certificate, client certificate, and a random auth token. Write them to disk in a specified output directory. + +## Key abstractions + +| Type/Function | File | Description | +| ------------- | ---- | ----------- | +| `GeneratedMaterial` | `src/generate.rs` | Struct holding all generated PEM strings and the auth token | +| `generate` | `src/generate.rs` | Generate material and write 8 files to an output directory | +| `generate_material` | `src/generate.rs` | Generate material without writing to disk | +| `generate_ca` | `src/generate.rs` | Create a self-signed CA with rcgen | +| `generate_server_cert` | `src/generate.rs` | Create a server cert signed by the CA | +| `generate_client_cert` | `src/generate.rs` | Create a client cert signed by the CA | +| `generate_auth_token` | `src/generate.rs` | Generate a 32-byte random hex token | + +## Certificate details + +- **CA** — CN from `--ca-name`, org "rustunnel", serial 1, constrained CA basic constraints +- **Server cert** — CN from `--server-name`, org "rustunnel", serial 2, SANs: `localhost`, `rustunnel`, `127.0.0.1`, and optionally the bind IP if not `0.0.0.0` or `::` +- **Client cert** — CN from `--client-name`, org "rustunnel", serial 3 +- **Auth token** — 64 hex characters (32 random bytes) + +## Output files + +| File | Content | +| ---- | ------- | +| `ca.pem` | CA certificate (PEM) | +| `ca.key` | CA private key (PEM) | +| `server.crt` | Server certificate (PEM) | +| `server.key` | Server private key (PEM) | +| `client.crt` | Client certificate (PEM) | +| `client.key` | Client private key (PEM) | +| `token.txt` | Auth token (plain text) | +| `config.json` | Paths and default addresses (JSON) | + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/generate.rs` | Certificate generation, token generation, file writing | diff --git a/droid-wiki/systems/framing-protocol.md b/droid-wiki/systems/framing-protocol.md new file mode 100644 index 0000000..fea40af --- /dev/null +++ b/droid-wiki/systems/framing-protocol.md @@ -0,0 +1,50 @@ +# Framing protocol + +The framing protocol in `src/framing.rs` defines the binary message format used after the HTTP auth handshake upgrades the connection to a persistent tunnel. + +## Purpose + +Carry multiple bidirectional streams over a single TLS connection with minimal overhead. + +## Frame format + +All multi-byte integers are big-endian. + +``` ++--------+----------+----------+-------+-----------+ +| Type | StreamID | Len | Flags | Payload.. | +| 1 byte | 8 bytes | 4 bytes | 1 byte| variable | ++--------+----------+----------+-------+-----------+ +``` + +- **Type** — 0x01 CONNECT, 0x02 CONNECT_REPLY, 0x03 DATA, 0x04 CLOSE, 0x05 ERROR +- **StreamID** — odd IDs for client-initiated streams, even reserved for server-initiated +- **Len** — payload length, max 4 MiB +- **Flags** — reserved for future use (currently always 0x00) + +## Key abstractions + +| Type | File | Description | +| ---- | ---- | ----------- | +| `Frame` | `src/framing.rs` | Struct holding frame_type, stream_id, flags, payload | +| `FrameReader` | `src/framing.rs` | Async reader with buffering that handles partial TCP reads | +| `FrameWriter` | `src/framing.rs` | Async writer that serializes and flushes frames | +| `encode_target` | `src/framing.rs` | Encode a `Socks5Target` into CONNECT frame payload | +| `decode_target` | `src/framing.rs` | Decode a `Socks5Target` from CONNECT frame payload | + +## Partial read discipline + +`FrameReader::read_frame` loops until a complete frame is available: + +1. Attempt to deserialize from the buffer. +2. If the frame is incomplete (`FrameTooShort`), read more bytes from the underlying TCP stream into the buffer. +3. If the connection closes with a partial frame in the buffer, return an error. +4. Only consume bytes from the buffer once the full frame is present. + +This prevents header corruption when TCP segments arrive in arbitrary sizes. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/framing.rs` | Frame format, serialization, async reader/writer, target encoding | diff --git a/droid-wiki/systems/index.md b/droid-wiki/systems/index.md new file mode 100644 index 0000000..e46034a --- /dev/null +++ b/droid-wiki/systems/index.md @@ -0,0 +1,9 @@ +# Systems + +rustunnel's internal architecture is divided into five systems: + +- [Tunnel engine](tunnel-engine.md) — listener accept loop, connector reconnect loop, stream multiplexing, SOCKS5 proxy integration +- [TLS stack](tls-stack.md) — rustls configuration, certificate loading, identity validation +- [Framing protocol](framing-protocol.md) — binary frame format, serialization, partial read handling +- [SOCKS5 proxy](socks5-proxy.md) — RFC 1928 handshake, CONNECT support, target resolution +- [Credential generation](credential-generation.md) — self-signed CA and certificate generation with rcgen diff --git a/droid-wiki/systems/socks5-proxy.md b/droid-wiki/systems/socks5-proxy.md new file mode 100644 index 0000000..2bf8bd2 --- /dev/null +++ b/droid-wiki/systems/socks5-proxy.md @@ -0,0 +1,37 @@ +# SOCKS5 proxy + +The SOCKS5 module in `src/socks5.rs` implements RFC 1928 (SOCKS5 protocol) and RFC 1929 (username/password authentication). + +## Purpose + +Handle the SOCKS5 handshake on the connector side, parse CONNECT requests, and resolve target addresses so that local applications can forward traffic through the tunnel. + +## Supported features + +- CONNECT command only (no BIND or UDP) +- No-auth and username/password authentication methods +- IPv4 and domain name targets (no IPv6) +- Target DNS resolution on the connector side + +## Key abstractions + +| Type | File | Description | +| ---- | ---- | ----------- | +| `Socks5Target` | `src/socks5.rs` | Enum: `Ipv4(addr, port)` or `Domain { domain, port }` | +| `Socks5State` | `src/socks5.rs` | State machine: ExpectGreeting → ExpectAuth → ExpectRequest → RequestReceived → Established → Closed | +| `Socks5Error` | `src/socks5.rs` | Error enum for malformed input, unsupported methods, and parse failures | +| `parse_greeting` | `src/socks5.rs` | Parse client greeting and choose auth method | +| `parse_auth_request` | `src/socks5.rs` | Parse username/password auth request | +| `parse_connect_request` | `src/socks5.rs` | Parse CONNECT request and extract target | +| `resolve_target` | `src/socks5.rs` | Async DNS resolution of a `Socks5Target` to a `SocketAddr` | + +## Integration + +The connector's `run_socks5_proxy` in `src/tunnel.rs` calls `socks5_handshake` (an async function in `src/tunnel.rs` that uses `src/socks5.rs` constants and helpers) for each incoming SOCKS5 connection. After handshake, it opens a stream via `StreamMux::open_stream` and forwards data bidirectionally. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/socks5.rs` | SOCKS5 protocol constants, parsing, target resolution | +| `src/tunnel.rs` | Async SOCKS5 handshake and proxy integration | diff --git a/droid-wiki/systems/tls-stack.md b/droid-wiki/systems/tls-stack.md new file mode 100644 index 0000000..62cd445 --- /dev/null +++ b/droid-wiki/systems/tls-stack.md @@ -0,0 +1,45 @@ +# TLS stack + +The TLS stack in `src/tls.rs` configures rustls for both server (listener) and client (connector) roles with mutual TLS. + +## Purpose + +Provide safe, fail-closed TLS configuration builders that load certificates and keys, validate certificate identities, and support both secure and insecure (lab-only) modes. + +## Key abstractions + +| Type/Function | File | Description | +| ------------- | ---- | ----------- | +| `build_server_config` | `src/tls.rs` | Build a rustls `ServerConfig` with client certificate verification | +| `build_client_config` | `src/tls.rs` | Build a rustls `ClientConfig` with server certificate verification | +| `build_server_config_insecure` | `src/tls.rs` | Build a server config that skips client cert verification | +| `build_client_config_insecure` | `src/tls.rs` | Build a client config that skips server cert verification | +| `load_certs` | `src/tls.rs` | Load PEM-encoded certificates from a file | +| `load_key` | `src/tls.rs` | Load a PEM-encoded private key from a file | +| `validate_cert_identity` | `src/tls.rs` | Check SANs and CN against an expected hostname | +| `check_cert_not_expired` | `src/tls.rs` | Verify certificate validity period | +| `cert_fingerprint_sha256` | `src/tls.rs` | Compute SHA-256 fingerprint of a certificate | +| `InsecureServerCertVerifier` | `src/tls.rs` | Dangerous verifier that accepts any certificate | + +## How it works + +Server config building: + +1. Load the server certificate chain and private key. +2. Load the CA certificate into a `RootCertStore`. +3. Build a `WebPkiClientVerifier` from the root store. +4. Use `ServerConfig::builder().with_client_cert_verifier(...).with_single_cert(...)`. + +Client config building: + +1. Load the client certificate chain and private key. +2. Load the CA certificate into a `RootCertStore`. +3. Use `ClientConfig::builder().with_root_certificates(...).with_client_auth_cert(...)`. + +Identity validation uses `x509-parser` to extract Subject Alternative Names and the subject CN, matching against the expected hostname or IP address. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/tls.rs` | TLS configuration, certificate loading, identity validation | diff --git a/droid-wiki/systems/tunnel-engine.md b/droid-wiki/systems/tunnel-engine.md new file mode 100644 index 0000000..95cf7cc --- /dev/null +++ b/droid-wiki/systems/tunnel-engine.md @@ -0,0 +1,69 @@ +# Tunnel engine + +The tunnel engine in `src/tunnel.rs` is the largest module (~3,500 lines). It implements the listener, connector, SOCKS5 proxy, stream multiplexer, reconnect loop, and HTTP auth handshake. + +## Purpose + +The tunnel engine establishes a persistent, authenticated, mTLS-encrypted tunnel between a listener and a connector, then multiplexes multiple SOCKS5 streams over that single connection. + +## Key abstractions + +| Type | File | Description | +| ---- | ---- | ----------- | +| `ListenerConfig` | `src/tunnel.rs` | Bind address, optional SOCKS5 address, TLS material, auth token, insecure flag | +| `ConnectorConfig` | `src/tunnel.rs` | Target host/port, TLS material, auth token, insecure flag | +| `ServerTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for server-side TLS | +| `ClientTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for client-side TLS | +| `StreamMux` | `src/tunnel.rs` | Connector-side multiplexer: manages stream IDs, data channels, and frame dispatch | +| `ServerStreamEntry` | `src/tunnel.rs` | Listener-side target write half for an active stream | +| `ConnectorStreamEntry` | `src/tunnel.rs` | Connector-side data channel and reply channel for an active stream | + +## How it works + +### Listener side + +1. `run_listener` binds a TCP socket, builds the TLS acceptor from `ServerTlsMaterial`, and starts accepting connections. +2. `handle_listener_connection` performs the TLS accept, verifies the client certificate (unless insecure), reads the HTTP POST to `/tunnel`, and validates the `X-Rustunnel-Token` header with `constant_time_compare`. +3. On auth success, it calls `handle_framing_server` to switch to binary framing. +4. `handle_framing_server` reads frames from the tunnel. A CONNECT frame triggers a TCP connection to the target address (opened from the listener side). DATA frames are forwarded to the target. CLOSE frames shut down the target connection. +5. If the listener was started with `--socks`, a server-side SOCKS5 proxy runs via `run_socks5_proxy`, receiving the active `StreamMux` through a `watch::channel`. + +### Connector side + +1. `run_connector_with_socks` builds the client TLS config and starts two concurrent pieces: a reconnect loop and a permanently-bound SOCKS5 proxy. +2. `run_reconnect_loop` repeatedly attempts to connect to the listener with exponential backoff (1s to 30s). Each attempt re-runs all security gates: TLS handshake, server cert verification, and auth token exchange. +3. On success, the loop creates a `StreamMux`, publishes it via the `watch::channel`, and waits for the framing task to finish (indicating disconnection). +4. When the tunnel drops, the loop logs the disconnection and retries. +5. `run_socks5_proxy` accepts SOCKS5 connections. For each connection, it checks if the `StreamMux` is available, opens a stream via `mux.open_stream()`, and performs bidirectional forwarding. + +### Stream multiplexing + +`StreamMux::open_stream` allocates the next odd stream ID, registers a data channel and reply channel, sends a CONNECT frame, and waits for a CONNECT_REPLY. The caller receives a `mpsc::Receiver` for tunnel-to-SOCKS5 data. + +`StreamMux::dispatch_frame` routes inbound CONNECT_REPLY, DATA, CLOSE, and ERROR frames to the correct stream by ID. + +```mermaid +graph TD + A[SOCKS5 client] -->|CONNECT| B[SOCKS5 proxy] + B -->|open_stream| C[StreamMux] + C -->|CONNECT frame| D[Framing writer] + D -->|TLS| E[Listener framing] + E -->|decode target| F[TCP connect to target] + F -->|DATA| E + E -->|DATA frame| D + D -->|dispatch| C + C -->|data_rx| B + B -->|raw bytes| A +``` + +## Entry points for modification + +- To change auth behavior: modify `handle_listener_connection` (listener) and `authenticate_tunnel` (connector) in `src/tunnel.rs`. +- To change reconnect strategy: modify `run_reconnect_loop` in `src/tunnel.rs`. +- To add listener-side SOCKS5 behavior: the server-side SOCKS5 proxy is currently minimal; expand `run_socks5_proxy` in `src/tunnel.rs`. + +## Key source files + +| File | Purpose | +| ---- | ------- | +| `src/tunnel.rs` | Full tunnel engine: listener, connector, SOCKS5 proxy, reconnect, framing loops, stream mux |