docs: add comprehensive project wiki for v1.0
CI / cargo fmt (push) Has been cancelled
CI / cargo clippy (macos-latest) (push) Has been cancelled
CI / cargo clippy (ubuntu-latest) (push) Has been cancelled
CI / cargo clippy (windows-latest) (push) Has been cancelled
CI / cargo test (macos-latest) (push) Has been cancelled
CI / cargo test (ubuntu-latest) (push) Has been cancelled
CI / cargo test (windows-latest) (push) Has been cancelled
CI / cargo build (macos-latest) (push) Has been cancelled
CI / cargo build (ubuntu-latest) (push) Has been cancelled
CI / cargo build (windows-latest) (push) Has been cancelled
CI / cargo build --release (macos-latest) (push) Has been cancelled
CI / cargo build --release (ubuntu-latest) (push) Has been cancelled
CI / cargo build --release (windows-latest) (push) Has been cancelled
CI / CLI smoke (macos-latest) (push) Has been cancelled
CI / CLI smoke (ubuntu-latest) (push) Has been cancelled
CI / CLI smoke (windows-latest) (push) Has been cancelled
CI / Minimal E2E (macos-latest) (push) Has been cancelled
CI / Minimal E2E (ubuntu-latest) (push) Has been cancelled
CI / Minimal E2E (windows-latest) (push) Has been cancelled
CI / cargo fmt (push) Has been cancelled
CI / cargo clippy (macos-latest) (push) Has been cancelled
CI / cargo clippy (ubuntu-latest) (push) Has been cancelled
CI / cargo clippy (windows-latest) (push) Has been cancelled
CI / cargo test (macos-latest) (push) Has been cancelled
CI / cargo test (ubuntu-latest) (push) Has been cancelled
CI / cargo test (windows-latest) (push) Has been cancelled
CI / cargo build (macos-latest) (push) Has been cancelled
CI / cargo build (ubuntu-latest) (push) Has been cancelled
CI / cargo build (windows-latest) (push) Has been cancelled
CI / cargo build --release (macos-latest) (push) Has been cancelled
CI / cargo build --release (ubuntu-latest) (push) Has been cancelled
CI / cargo build --release (windows-latest) (push) Has been cancelled
CI / CLI smoke (macos-latest) (push) Has been cancelled
CI / CLI smoke (ubuntu-latest) (push) Has been cancelled
CI / CLI smoke (windows-latest) (push) Has been cancelled
CI / Minimal E2E (macos-latest) (push) Has been cancelled
CI / Minimal E2E (ubuntu-latest) (push) Has been cancelled
CI / Minimal E2E (windows-latest) (push) Has been cancelled
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent
a758e8e0bc
commit
0166fb6511
@@ -2,5 +2,6 @@
|
|||||||
/release
|
/release
|
||||||
**/*.pem
|
**/*.pem
|
||||||
**/*.json
|
**/*.json
|
||||||
|
!.wiki-meta.json
|
||||||
.env
|
.env
|
||||||
*.log
|
*.log
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
{
|
||||||
|
"generatedAt": "2026-06-04T13:55:00Z",
|
||||||
|
"commitHash": "753928239e5a86414f6a6912b908da4cf0960231",
|
||||||
|
"branch": "master",
|
||||||
|
"pageCount": 30,
|
||||||
|
"topLevelSections": [
|
||||||
|
"overview",
|
||||||
|
"by-the-numbers",
|
||||||
|
"lore",
|
||||||
|
"how-to-contribute",
|
||||||
|
"applications",
|
||||||
|
"systems",
|
||||||
|
"features",
|
||||||
|
"security",
|
||||||
|
"deployment",
|
||||||
|
"reference"
|
||||||
|
],
|
||||||
|
"pageOrder": [
|
||||||
|
"overview/index.md",
|
||||||
|
"overview/architecture.md",
|
||||||
|
"overview/getting-started.md",
|
||||||
|
"overview/glossary.md",
|
||||||
|
"by-the-numbers.md",
|
||||||
|
"lore.md",
|
||||||
|
"how-to-contribute/index.md",
|
||||||
|
"how-to-contribute/development-workflow.md",
|
||||||
|
"how-to-contribute/testing.md",
|
||||||
|
"how-to-contribute/debugging.md",
|
||||||
|
"how-to-contribute/patterns-and-conventions.md",
|
||||||
|
"how-to-contribute/tooling.md",
|
||||||
|
"applications/index.md",
|
||||||
|
"applications/rustunnel-cli.md",
|
||||||
|
"systems/index.md",
|
||||||
|
"systems/tunnel-engine.md",
|
||||||
|
"systems/tls-stack.md",
|
||||||
|
"systems/framing-protocol.md",
|
||||||
|
"systems/socks5-proxy.md",
|
||||||
|
"systems/credential-generation.md",
|
||||||
|
"features/index.md",
|
||||||
|
"features/connection-keys.md",
|
||||||
|
"features/stream-multiplexing.md",
|
||||||
|
"features/reconnect-and-shutdown.md",
|
||||||
|
"security.md",
|
||||||
|
"deployment.md",
|
||||||
|
"reference/index.md",
|
||||||
|
"reference/configuration.md",
|
||||||
|
"reference/data-models.md",
|
||||||
|
"reference/dependencies.md"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# Applications
|
||||||
|
|
||||||
|
rustunnel ships as a single binary application with five CLI commands. There are no separate services, daemons, or web frontends.
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# rustunnel CLI
|
||||||
|
|
||||||
|
The rustunnel binary exposes five subcommands defined in `src/cli.rs` and dispatched in `src/main.rs`.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
### `listen`
|
||||||
|
|
||||||
|
Start the HTTPS tunnel listener. Binds an HTTPS server that accepts mTLS connections from connectors.
|
||||||
|
|
||||||
|
Key arguments:
|
||||||
|
- `--listen` — bind address (default `0.0.0.0:4180`)
|
||||||
|
- `--advertise` — public address embedded in auto-generated connection keys
|
||||||
|
- `--connection-key` — reusable key generated by `keygen` or a previous `listen` run
|
||||||
|
- `--socks` — optional server-side SOCKS5 proxy address for connector-side network access
|
||||||
|
- `--cert`, `--key`, `--ca-cert` — TLS material paths (required unless using a connection key)
|
||||||
|
- `--auth-token`, `--auth-token-file` — application auth token
|
||||||
|
- `--insecure-skip-tls-verify` — skip client certificate verification (lab only)
|
||||||
|
|
||||||
|
When run without explicit certificate paths and without a connection key, `listen` auto-generates a new connection key and prints it.
|
||||||
|
|
||||||
|
### `connect`
|
||||||
|
|
||||||
|
Connect to the listener and expose a local SOCKS5 proxy.
|
||||||
|
|
||||||
|
Key arguments:
|
||||||
|
- `CONNECTION_KEY` — positional connection key (optional)
|
||||||
|
- `--target` — listener address (default from connection key)
|
||||||
|
- `--connection-key` — connection key via flag or env `RUSTUNNEL_KEY`
|
||||||
|
- `--socks` — local SOCKS5 proxy address (default `127.0.0.1:1180`)
|
||||||
|
- `--cert`, `--key`, `--ca-cert` — TLS material paths (required unless using a connection key)
|
||||||
|
- `--insecure-skip-tls-verify` — skip server certificate verification (lab only)
|
||||||
|
|
||||||
|
### `generate`
|
||||||
|
|
||||||
|
Generate local certificate and auth material into an output directory.
|
||||||
|
|
||||||
|
Creates: `ca.pem`, `ca.key`, `server.crt`, `server.key`, `client.crt`, `client.key`, `token.txt`, `config.json`.
|
||||||
|
|
||||||
|
### `keygen`
|
||||||
|
|
||||||
|
Generate a single reusable connection key string. This bundles all certificate material, the auth token, and the target address into a base64-encoded JSON blob prefixed with `rtun1.`.
|
||||||
|
|
||||||
|
### `version`
|
||||||
|
|
||||||
|
Print version, edition, platform, and architecture.
|
||||||
|
|
||||||
|
## Entry point
|
||||||
|
|
||||||
|
`src/main.rs` installs the ring crypto provider, initializes tracing with an env-filter defaulting to `info`, parses the CLI, and dispatches to the appropriate runner function.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/cli.rs` | clap `Parser` and `Subcommand` definitions |
|
||||||
|
| `src/main.rs` | Entry point, command dispatch, host:port parsing helpers |
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# By the numbers
|
||||||
|
|
||||||
|
Data collected on 2026-06-04.
|
||||||
|
|
||||||
|
## Size
|
||||||
|
|
||||||
|
| Metric | Value |
|
||||||
|
| ------ | ----- |
|
||||||
|
| Total source files | 12 |
|
||||||
|
| Total lines of code | ~7,535 |
|
||||||
|
| Largest file | `src/tunnel.rs` (3,526 lines) |
|
||||||
|
| Second largest | `src/socks5.rs` (789 lines) |
|
||||||
|
| Third largest | `src/framing.rs` (776 lines) |
|
||||||
|
| Test files | 0 standalone test files (all tests are inline `#[cfg(test)]` modules) |
|
||||||
|
| Languages | Rust only |
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
xychart-beta
|
||||||
|
title "Lines of code by module"
|
||||||
|
x-axis [tunnel, socks5, framing, tls, main, generate, redact, cli, config, connkey, errors, signal]
|
||||||
|
y-axis "Lines" 0 --> 3600
|
||||||
|
bar [3526, 789, 776, 609, 601, 440, 242, 237, 118, 93, 82, 22]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Activity
|
||||||
|
|
||||||
|
- **Commits:** 16 total (as of HEAD)
|
||||||
|
- **Active development period:** June 3–4, 2026 (~2 days)
|
||||||
|
- **Most changed file:** `src/tunnel.rs` — underwent major rewrites from per-request HTTPS to persistent multiplexed tunnel, then to binary framing, then to reconnect/shutdown handling.
|
||||||
|
- **Second most changed:** `.github/workflows/ci.yml` — iterated on cross-platform E2E cleanup and smoke tests.
|
||||||
|
|
||||||
|
## Bot-attributed commits
|
||||||
|
|
||||||
|
- **Bot commits:** 7 of 16 (44%)
|
||||||
|
- **Bot types:** `factory-droid[bot]`
|
||||||
|
- Note: this is a lower bound. Inline AI tools like Copilot leave no trace in git history.
|
||||||
|
|
||||||
|
## Complexity
|
||||||
|
|
||||||
|
| File | Lines | Description |
|
||||||
|
| ---- | ----- | ----------- |
|
||||||
|
| `src/tunnel.rs` | 3,526 | Core tunnel engine, listener, connector, SOCKS5 proxy, reconnect loop, stream mux, ~30 inline tests |
|
||||||
|
| `src/socks5.rs` | 789 | SOCKS5 protocol parser, greeting, auth, CONNECT, target resolution |
|
||||||
|
| `src/framing.rs` | 776 | Binary frame serialization, deserialization, reader/writer, target encoding |
|
||||||
|
| `src/tls.rs` | 609 | rustls config builders, certificate loading, identity validation, fingerprinting |
|
||||||
|
| `src/main.rs` | 601 | CLI entry point, command dispatch, connection key integration, argument parsing helpers |
|
||||||
|
|
||||||
|
## Dependency count
|
||||||
|
|
||||||
|
- **Runtime dependencies:** 26 crates
|
||||||
|
- **Dev dependencies:** 1 crate (`tempfile`)
|
||||||
|
- **Largest dependency families:** Tokio ecosystem (`tokio`, `tokio-util`, `tokio-rustls`), rustls ecosystem (`rustls`, `rustls-pki-types`, `rustls-pemfile`), Hyper ecosystem (`hyper`, `hyper-util`, `http-body-util`, `tower-service`)
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
# Deployment
|
||||||
|
|
||||||
|
rustunnel is deployed as a single static binary. There are no containers, orchestrators, or external services required.
|
||||||
|
|
||||||
|
## Release builds
|
||||||
|
|
||||||
|
CI builds release artifacts on Linux, macOS, and Windows:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo build --release
|
||||||
|
```
|
||||||
|
|
||||||
|
The resulting binary is uploaded as a GitHub Actions artifact with a 7-day retention.
|
||||||
|
|
||||||
|
## Cross-platform CI
|
||||||
|
|
||||||
|
The `.github/workflows/ci.yml` runs on `ubuntu-latest`, `windows-latest`, and `macos-latest`:
|
||||||
|
|
||||||
|
1. `cargo fmt` — format check
|
||||||
|
2. `cargo clippy` — lint check
|
||||||
|
3. `cargo test` — unit and inline tests
|
||||||
|
4. `cargo build` — debug build
|
||||||
|
5. `cargo build --release` — release build with artifact upload
|
||||||
|
6. CLI smoke tests — `--help`, `version`, invalid command/port handling
|
||||||
|
7. Minimal E2E — generate, listen, connect, SOCKS5, curl
|
||||||
|
|
||||||
|
## E2E test flow
|
||||||
|
|
||||||
|
The E2E job uses `cargo run --release --` for OS-neutral invocation and performs:
|
||||||
|
|
||||||
|
1. Generate credentials to a temp directory
|
||||||
|
2. Start a Python HTTP server as the target
|
||||||
|
3. Start `rustunnel listen` in the background
|
||||||
|
4. Start `rustunnel connect` in the background
|
||||||
|
5. Wait for services to start
|
||||||
|
6. curl through the SOCKS5 proxy and assert HTTP 200 + expected body content
|
||||||
|
7. Cleanup via `trap cleanup EXIT` with PID-based and port-based killing
|
||||||
|
|
||||||
|
## No production deployment guidance
|
||||||
|
|
||||||
|
rustunnel is explicitly a lab and development tool. It does not include:
|
||||||
|
|
||||||
|
- Container images or Dockerfiles
|
||||||
|
- Kubernetes manifests
|
||||||
|
- Systemd service files
|
||||||
|
- Log rotation or monitoring infrastructure
|
||||||
|
|
||||||
|
For production tunneling, consider established tools like WireGuard, OpenVPN, or SSH.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `.github/workflows/ci.yml` | CI matrix: fmt, clippy, test, build, smoke, E2E |
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
# Connection keys
|
||||||
|
|
||||||
|
A connection key bundles all credential material and the target address into a single copy-pasteable string.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Simplify distribution of tunnel credentials between machines. Instead of transferring eight separate files, a user can generate one key and paste it into the `listen` and `connect` commands.
|
||||||
|
|
||||||
|
## Format
|
||||||
|
|
||||||
|
Connection keys start with the prefix `rtun1.` followed by base64url-encoded (no padding) JSON:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"target": "198.51.100.10:4180",
|
||||||
|
"ca_cert_pem": "-----BEGIN CERTIFICATE-----...",
|
||||||
|
"server_cert_pem": "-----BEGIN CERTIFICATE-----...",
|
||||||
|
"server_key_pem": "-----BEGIN PRIVATE KEY-----...",
|
||||||
|
"client_cert_pem": "-----BEGIN CERTIFICATE-----...",
|
||||||
|
"client_key_pem": "-----BEGIN PRIVATE KEY-----...",
|
||||||
|
"auth_token": "a1b2c3..."
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key abstractions
|
||||||
|
|
||||||
|
| Type | File | Description |
|
||||||
|
| ---- | ---- | ----------- |
|
||||||
|
| `ConnectionKey` | `src/connkey.rs` | Struct with all fields, version check, validation |
|
||||||
|
| `ConnectionKey::encode` | `src/connkey.rs` | Serialize to JSON, base64url-encode, prepend prefix |
|
||||||
|
| `ConnectionKey::decode` | `src/connkey.rs` | Strip prefix, base64url-decode, deserialize, validate |
|
||||||
|
| `looks_like_connection_key` | `src/connkey.rs` | Quick check if a string starts with `rtun1.` |
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
`decode` validates:
|
||||||
|
|
||||||
|
- Prefix must be `rtun1.`
|
||||||
|
- Base64 decoding must succeed
|
||||||
|
- JSON deserialization must succeed
|
||||||
|
- Version must be exactly `1`
|
||||||
|
- All string fields must be non-empty after trimming
|
||||||
|
|
||||||
|
## Integration
|
||||||
|
|
||||||
|
`src/main.rs` uses `ConnectionKey::decode` when the `--connection-key` flag or positional argument is provided. The decoded material is passed to `ServerTlsMaterial::Pem` or `ClientTlsMaterial::Pem` variants, which bypass file loading and use the embedded PEM strings directly.
|
||||||
|
|
||||||
|
## Entry points for modification
|
||||||
|
|
||||||
|
- To change the key format or add versioning: modify `src/connkey.rs`.
|
||||||
|
- To add compression or encryption: consider extending the encode/decode pipeline in `ConnectionKey`.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/connkey.rs` | Connection key struct, encoding, decoding, validation |
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
# Features
|
||||||
|
|
||||||
|
rustunnel's user-visible and developer-visible capabilities:
|
||||||
|
|
||||||
|
- [Connection keys](connection-keys.md) — single-string credential bundles
|
||||||
|
- [Stream multiplexing](stream-multiplexing.md) — multiple SOCKS5 streams over one tunnel
|
||||||
|
- [Reconnect and shutdown](reconnect-and-shutdown.md) — operational resilience
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# Reconnect and shutdown
|
||||||
|
|
||||||
|
The connector is designed to recover from tunnel interruptions without dropping the SOCKS5 proxy.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Lab networks and development environments are unstable. The connector should tolerate listener restarts, network hiccups, and temporary failures without requiring manual intervention.
|
||||||
|
|
||||||
|
## Reconnect behavior
|
||||||
|
|
||||||
|
`run_reconnect_loop` in `src/tunnel.rs` continuously attempts to maintain a tunnel:
|
||||||
|
|
||||||
|
1. Connect to the target address via TCP.
|
||||||
|
2. Perform TLS handshake and server certificate verification.
|
||||||
|
3. Send the HTTP auth request and wait for 200 OK.
|
||||||
|
4. On success, create a `StreamMux`, publish it via the `watch::channel`, and run the framing loop.
|
||||||
|
5. When the tunnel drops (framing task ends), log the disconnection and retry.
|
||||||
|
|
||||||
|
### Backoff
|
||||||
|
|
||||||
|
- Initial delay: 1 second
|
||||||
|
- Exponential doubling up to a maximum of 30 seconds
|
||||||
|
- Auth failures are terminal (the loop exits rather than retrying with a bad token)
|
||||||
|
|
||||||
|
### SOCKS5 proxy during reconnect
|
||||||
|
|
||||||
|
The SOCKS5 proxy stays bound. If a client connects while the tunnel is down, the proxy sees `mux_receiver` is `None` and rejects the connection with a warning log. Once the reconnect loop succeeds, new SOCKS5 requests work immediately.
|
||||||
|
|
||||||
|
## Shutdown
|
||||||
|
|
||||||
|
Both listener and connector handle graceful shutdown:
|
||||||
|
|
||||||
|
- Unix: SIGINT or SIGTERM triggers shutdown via `tokio::signal::unix`.
|
||||||
|
- Windows: Ctrl+C or Ctrl+Break triggers shutdown via `tokio::signal::ctrl_c`.
|
||||||
|
|
||||||
|
### Listener shutdown
|
||||||
|
|
||||||
|
1. The shutdown signal breaks the `tokio::select!` in `run_listener`.
|
||||||
|
2. The TCP listener is dropped, releasing the bind port.
|
||||||
|
3. The SOCKS5 proxy task (if any) is aborted.
|
||||||
|
4. Active framing sessions end when their streams close.
|
||||||
|
|
||||||
|
### Connector shutdown
|
||||||
|
|
||||||
|
1. The shutdown signal breaks the `tokio::select!` in `run_socks5_proxy`.
|
||||||
|
2. The SOCKS5 listener is dropped.
|
||||||
|
3. The reconnect loop handle is aborted.
|
||||||
|
4. Existing SOCKS5 connections receive EOF when their streams close.
|
||||||
|
|
||||||
|
E2E tests `e2e_graceful_shutdown_listener` and `e2e_graceful_shutdown_connector` verify that ports are released within 3 seconds of abort.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/tunnel.rs` | `run_reconnect_loop`, `run_socks5_proxy`, `run_listener`, shutdown handling |
|
||||||
|
| `src/signal.rs` | Cross-platform shutdown signal future |
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# Stream multiplexing
|
||||||
|
|
||||||
|
rustunnel carries multiple SOCKS5 connections over a single persistent TLS tunnel using stream IDs.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Avoid the overhead of establishing a new HTTPS connection for every SOCKS5 request. A single mTLS+auth session stays open, and multiple logical streams are interleaved via the binary framing protocol.
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
### Connector side
|
||||||
|
|
||||||
|
1. A local SOCKS5 client connects to the connector's SOCKS5 proxy.
|
||||||
|
2. The connector's `StreamMux::open_stream` allocates the next odd stream ID.
|
||||||
|
3. It sends a CONNECT frame with the target address encoded in the payload.
|
||||||
|
4. It waits for a CONNECT_REPLY frame (status 0x00 = OK).
|
||||||
|
5. On success, it returns a `mpsc::Receiver<Bytes>` for tunnel-to-SOCKS5 data.
|
||||||
|
6. The connector spawns a task to read from the SOCKS5 client and send DATA frames to the tunnel.
|
||||||
|
7. Another path reads from the `data_rx` channel and writes raw bytes back to the SOCKS5 client.
|
||||||
|
|
||||||
|
### Listener side
|
||||||
|
|
||||||
|
1. The listener receives a CONNECT frame with a stream ID and target address.
|
||||||
|
2. It resolves the target address and opens a TCP connection.
|
||||||
|
3. It spawns a task to read from the target and send DATA frames back to the tunnel.
|
||||||
|
4. It sends a CONNECT_REPLY frame to acknowledge the stream.
|
||||||
|
5. Incoming DATA frames for that stream ID are written to the target connection.
|
||||||
|
6. CLOSE frames trigger shutdown of the target connection.
|
||||||
|
|
||||||
|
### Stream isolation
|
||||||
|
|
||||||
|
Each stream has its own ID. Data from one stream never mixes with another. The E2E test `e2e_concurrent_streams_isolation` verifies this by sending unique payloads through two concurrent streams and asserting each echo response matches its original payload.
|
||||||
|
|
||||||
|
## Key abstractions
|
||||||
|
|
||||||
|
| Type | File | Description |
|
||||||
|
| ---- | ---- | ----------- |
|
||||||
|
| `StreamMux` | `src/tunnel.rs` | Connector-side multiplexer |
|
||||||
|
| `ServerStreamEntry` | `src/tunnel.rs` | Listener-side per-stream target write half |
|
||||||
|
| `ConnectorStreamEntry` | `src/tunnel.rs` | Connector-side per-stream data channel and reply channel |
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/tunnel.rs` | StreamMux, open_stream, dispatch_frame, framing loops |
|
||||||
|
| `src/framing.rs` | Frame types, CONNECT/DATA/CLOSE/ERROR encoding |
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# Debugging
|
||||||
|
|
||||||
|
## Logs
|
||||||
|
|
||||||
|
rustunnel uses `tracing` with an `EnvFilter` defaulting to `info`. Set `RUST_LOG=debug` or `RUST_LOG=trace` for more detail.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
RUST_LOG=debug ./target/release/rustunnel listen --listen 127.0.0.1:4180 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
All logged secrets are redacted. The `Redacted` wrapper shows `[REDACTED(len=N)]` instead of the raw value. If you see raw secrets in logs, that is a bug.
|
||||||
|
|
||||||
|
## Common errors
|
||||||
|
|
||||||
|
### TLS handshake failures
|
||||||
|
|
||||||
|
If the connector fails with a TLS error, check:
|
||||||
|
|
||||||
|
- Do the listener and connector share the same CA certificate?
|
||||||
|
- Does the server certificate's SAN match the target hostname? Use `generate` with the correct `--advertise` address, or pass `--insecure-skip-tls-verify` (lab only).
|
||||||
|
- Are the certificate files readable and valid PEM?
|
||||||
|
|
||||||
|
rustunnel prints TLS hint messages for common failures:
|
||||||
|
|
||||||
|
- `BadSignature` — the listener certificate is not signed by the CA trusted by the connector.
|
||||||
|
- `NotValidForName` — the listener certificate does not match the target host.
|
||||||
|
|
||||||
|
### Auth failures
|
||||||
|
|
||||||
|
- Verify the token file is not empty and has no trailing whitespace.
|
||||||
|
- The listener and connector must use the same token value.
|
||||||
|
- Auth failures are terminal for the connector's reconnect loop (it will not retry with a bad token).
|
||||||
|
|
||||||
|
### Port conflicts
|
||||||
|
|
||||||
|
If binding fails with "address already in use":
|
||||||
|
|
||||||
|
- Check for orphaned processes from previous test runs.
|
||||||
|
- The E2E tests use random free ports; manual testing may need explicit port management.
|
||||||
|
|
||||||
|
### SOCKS5 connection refused
|
||||||
|
|
||||||
|
If curl fails through the SOCKS5 proxy:
|
||||||
|
|
||||||
|
- Is the connector running and the tunnel established?
|
||||||
|
- Is the SOCKS5 proxy port correct?
|
||||||
|
- Is the target address reachable from the listener side (for default mode) or connector side (when using `--socks` on the listener)?
|
||||||
|
|
||||||
|
## Tracing
|
||||||
|
|
||||||
|
The connector logs effective config at startup, including the target, SOCKS5 address, and redacted auth token. The listener logs its bind address, TLS fingerprints, and auth token (redacted).
|
||||||
|
|
||||||
|
State transitions are logged at `info` level: "Tunnel session established", "Tunnel disconnected", "Reconnecting in Ns".
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# Development workflow
|
||||||
|
|
||||||
|
## Branching
|
||||||
|
|
||||||
|
Use feature branches off `master`. The project is small enough that long-lived branches are unnecessary.
|
||||||
|
|
||||||
|
## Coding
|
||||||
|
|
||||||
|
- Follow existing formatting (enforced by `cargo fmt`).
|
||||||
|
- Use `anyhow` for application-level errors and `thiserror` for library-style error enums.
|
||||||
|
- Prefer `tracing` over `println!` for diagnostic output. Secrets must be wrapped with `Redacted` before logging.
|
||||||
|
- Use `Arc<String>` for shared strings across async boundaries (e.g., auth tokens in `ListenerConfig` and `ConnectorConfig`).
|
||||||
|
- Keep synchronous protocol parsing in dedicated modules (`socks5.rs`, `framing.rs`) and async I/O in `tunnel.rs`.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- Inline tests live in `#[cfg(test)]` modules at the bottom of each source file.
|
||||||
|
- Use `tempfile::tempdir()` for generating test credentials. Clean up temp directories with `std::fs::remove_dir_all`.
|
||||||
|
- Use `std::net::TcpListener::bind("127.0.0.1:0")` to get free ports for E2E tests.
|
||||||
|
- E2E tests spawn tokio tasks for listener, connector, and target servers, then abort them in cleanup.
|
||||||
|
|
||||||
|
## Committing
|
||||||
|
|
||||||
|
- Follow conventional commit style: `feat:`, `fix:`, `docs:`, `chore:`.
|
||||||
|
- Keep commits focused. The git history shows the evolution from HTTP-per-stream to multiplexed tunnel; preserve that narrative clarity.
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# How to contribute
|
||||||
|
|
||||||
|
rustunnel is a small, focused project. Contributions should align with the project's purpose: a lab and development tunneling tool with strong security defaults.
|
||||||
|
|
||||||
|
## Work pickup
|
||||||
|
|
||||||
|
- Check the CI status before starting work.
|
||||||
|
- Open issues or PRs against the `master` branch.
|
||||||
|
- Keep changes focused. The codebase is small enough that large refactors should be discussed first.
|
||||||
|
|
||||||
|
## PR process
|
||||||
|
|
||||||
|
1. Run the full test suite locally: `cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test --all-targets`
|
||||||
|
2. Ensure E2E tests pass if your change touches the tunnel or SOCKS5 path.
|
||||||
|
3. Write tests for new behavior. The project uses inline `#[cfg(test)]` modules.
|
||||||
|
4. Update relevant wiki pages if your change adds or removes features.
|
||||||
|
|
||||||
|
## Review expectations
|
||||||
|
|
||||||
|
- Security-sensitive changes (TLS, auth, framing) get extra scrutiny.
|
||||||
|
- All CI jobs must pass: fmt, clippy, test, build, smoke, and E2E.
|
||||||
|
- The project runs CI on Linux, macOS, and Windows. Avoid platform-specific code unless necessary (the signal handling module is the only current exception).
|
||||||
|
|
||||||
|
## Definition of done
|
||||||
|
|
||||||
|
- Code is formatted with `rustfmt`.
|
||||||
|
- Clippy warnings are resolved.
|
||||||
|
- Tests cover the new behavior, including edge cases.
|
||||||
|
- No `TODO` or `FIXME` comments are introduced (the project currently has zero).
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Patterns and conventions
|
||||||
|
|
||||||
|
## Error handling
|
||||||
|
|
||||||
|
The project uses a layered error strategy:
|
||||||
|
|
||||||
|
- `thiserror` enums for domain-specific errors (`TlsError`, `AuthError`, `TunnelError`, `HostError`, `Socks5Error`, `FrameError`). These live in `src/errors.rs` and `src/socks5.rs` / `src/framing.rs`.
|
||||||
|
- `anyhow` for application-level propagation. CLI commands return `anyhow::Result` and `main` exits with `process::exit(exit_code)`.
|
||||||
|
- All errors are designed to be actionable and fail-closed. There are no silent degradation paths to insecure modes.
|
||||||
|
|
||||||
|
## Secret handling
|
||||||
|
|
||||||
|
- The `Redacted` wrapper in `src/redact.rs` must be used before logging any secret.
|
||||||
|
- `Redacted::inner()` and `into_inner()` are marked `#[allow(dead_code)]` to discourage use, but available for file writes.
|
||||||
|
- `is_sensitive()` detects PEM private key blocks. `is_auth_token()` detects long alphanumeric strings.
|
||||||
|
- `is_connection_key()` detects strings starting with `rtun1.`.
|
||||||
|
|
||||||
|
## Async patterns
|
||||||
|
|
||||||
|
- Tokio's multi-threaded runtime is used on both listener and connector sides.
|
||||||
|
- `tokio::spawn` is used for per-connection and per-stream tasks.
|
||||||
|
- `tokio::select!` with `biased;` prioritizes shutdown signals over new connections.
|
||||||
|
- `watch::channel` is used for publishing the active `StreamMux` to the SOCKS5 proxy (supports reconnect).
|
||||||
|
- `mpsc::channel` and `oneshot::channel` are used for stream data and CONNECT_REPLY coordination.
|
||||||
|
|
||||||
|
## Constant-time comparison
|
||||||
|
|
||||||
|
Auth token comparison in `src/tunnel.rs` uses a manual constant-time XOR loop to prevent timing attacks:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
fn constant_time_compare(a: &str, b: &str) -> bool {
|
||||||
|
let a_bytes = a.as_bytes();
|
||||||
|
let b_bytes = b.as_bytes();
|
||||||
|
if a_bytes.len() != b_bytes.len() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
let mut result: u8 = 0;
|
||||||
|
for (x, y) in a_bytes.iter().zip(b_bytes.iter()) {
|
||||||
|
result |= x ^ y;
|
||||||
|
}
|
||||||
|
result == 0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Frame reader partial read discipline
|
||||||
|
|
||||||
|
`FrameReader` in `src/framing.rs` never consumes bytes from the buffer until a complete frame (header + payload) is available. This prevents corruption when TCP segments arrive in arbitrary chunks.
|
||||||
|
|
||||||
|
## TLS material as enum
|
||||||
|
|
||||||
|
Both `ServerTlsMaterial` and `ClientTlsMaterial` are enums with `Paths` and `Pem` variants. This lets the listener and connector accept either file paths (from CLI args) or embedded PEM strings (from connection keys).
|
||||||
|
|
||||||
|
## Insecure mode
|
||||||
|
|
||||||
|
`--insecure-skip-tls-verify` exists for DPI/intercepted environments only. When enabled, the listener skips client certificate verification and the connector skips server certificate verification. This is logged at `warn` level.
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
# Testing
|
||||||
|
|
||||||
|
## Test organization
|
||||||
|
|
||||||
|
All tests are inline `#[cfg(test)]` modules at the bottom of each source file. There are no standalone test files or integration test directories.
|
||||||
|
|
||||||
|
## Running tests
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test --all-targets
|
||||||
|
```
|
||||||
|
|
||||||
|
This runs unit tests and doc tests across all modules.
|
||||||
|
|
||||||
|
## Test patterns
|
||||||
|
|
||||||
|
### Credential generation tests
|
||||||
|
|
||||||
|
Tests in `src/generate.rs` verify that `generate()` creates all expected files, that certificates contain valid PEM blocks, that private keys are detected as sensitive by the redaction module, and that config JSON is valid.
|
||||||
|
|
||||||
|
### Framing tests
|
||||||
|
|
||||||
|
Tests in `src/framing.rs` cover:
|
||||||
|
|
||||||
|
- Frame roundtrips for all frame types (CONNECT, CONNECT_REPLY, DATA, CLOSE, ERROR)
|
||||||
|
- Partial read handling — buffers must not be consumed until a complete frame is available
|
||||||
|
- Multiple frames in one buffer
|
||||||
|
- Target address encoding/decoding for IPv4 and domain names
|
||||||
|
|
||||||
|
### SOCKS5 tests
|
||||||
|
|
||||||
|
Tests in `src/socks5.rs` cover:
|
||||||
|
|
||||||
|
- Greeting parsing (no-auth, username/password, invalid versions)
|
||||||
|
- Auth request parsing (valid and empty credentials)
|
||||||
|
- CONNECT request parsing (IPv4, domain, unsupported commands)
|
||||||
|
- Reply building
|
||||||
|
- Target resolution (IPv4 direct, domain DNS lookup)
|
||||||
|
- Malformed input handling
|
||||||
|
|
||||||
|
### TLS tests
|
||||||
|
|
||||||
|
Tests in `src/tls.rs` cover:
|
||||||
|
|
||||||
|
- Loading certificates and keys from files and PEM strings
|
||||||
|
- Building valid server and client configs
|
||||||
|
- Identity validation against SANs and CN
|
||||||
|
- Certificate fingerprinting
|
||||||
|
- Expiration checking
|
||||||
|
- Insecure config variants
|
||||||
|
|
||||||
|
### Tunnel tests
|
||||||
|
|
||||||
|
Tests in `src/tunnel.rs` are the most extensive. They include:
|
||||||
|
|
||||||
|
- **Auth tests:** valid mTLS + auth, invalid token, missing client cert
|
||||||
|
- **E2E tests:** basic forwarding, server-side SOCKS5 reaching connector-side targets, concurrent stream isolation, target failure handling, listener-side target origin verification
|
||||||
|
- **Operational tests:** reconnect restoring traffic after tunnel drop, graceful shutdown of listener and connector, shutdown during active streams
|
||||||
|
- **Log tests:** verify effective config is logged with redacted tokens, verify auth failures don't leak token values in error messages
|
||||||
|
|
||||||
|
## E2E test structure
|
||||||
|
|
||||||
|
Typical E2E tests follow this pattern:
|
||||||
|
|
||||||
|
1. Generate test credentials with `generate_test_creds()`.
|
||||||
|
2. Pick a free port with `get_free_port()`.
|
||||||
|
3. Spawn the listener as a tokio task.
|
||||||
|
4. Spawn the connector (and optionally a target server) as tokio tasks.
|
||||||
|
5. Sleep briefly to let services start.
|
||||||
|
6. Connect via SOCKS5, send data, verify response.
|
||||||
|
7. Abort all tasks in cleanup.
|
||||||
|
|
||||||
|
## CI test matrix
|
||||||
|
|
||||||
|
Tests run on `ubuntu-latest`, `windows-latest`, and `macos-latest`.
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
# Tooling
|
||||||
|
|
||||||
|
## Build system
|
||||||
|
|
||||||
|
Cargo is the sole build tool. No custom build scripts or Makefiles are used.
|
||||||
|
|
||||||
|
- `cargo build` — debug build
|
||||||
|
- `cargo build --release` — optimized release build
|
||||||
|
- `cargo check` — fast type-checking
|
||||||
|
|
||||||
|
## Linting and formatting
|
||||||
|
|
||||||
|
- `cargo fmt --check` — enforces rustfmt formatting
|
||||||
|
- `cargo clippy --all-targets -- -D warnings` — runs clippy on all targets with warnings as errors
|
||||||
|
|
||||||
|
## CI/CD
|
||||||
|
|
||||||
|
GitHub Actions runs the full matrix on every push and PR:
|
||||||
|
|
||||||
|
| Job | Platforms | Purpose |
|
||||||
|
| --- | --------- | ------- |
|
||||||
|
| `fmt` | Ubuntu | Format check |
|
||||||
|
| `clippy` | Linux, macOS, Windows | Lint check |
|
||||||
|
| `test` | Linux, macOS, Windows | Unit and inline tests |
|
||||||
|
| `build` | Linux, macOS, Windows | Debug build |
|
||||||
|
| `release-build` | Linux, macOS, Windows | Release build + artifact upload |
|
||||||
|
| `smoke` | Linux, macOS, Windows | CLI smoke tests (--help, version, invalid command/port) |
|
||||||
|
| `e2e-minimal` | Linux, macOS, Windows | Full end-to-end flow: generate, listen, connect, SOCKS5, curl |
|
||||||
|
|
||||||
|
The E2E job uses `cargo run --release --` for OS-neutral binary invocation and includes cross-platform cleanup helpers (`kill_process`, `kill_by_port`) invoked via `trap cleanup EXIT`.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
Dependencies are managed in `Cargo.toml`. Notable choices:
|
||||||
|
|
||||||
|
- `tokio` with `full` features for async runtime
|
||||||
|
- `rustls` with `ring` for TLS (no OpenSSL)
|
||||||
|
- `rcgen` for certificate generation
|
||||||
|
- `clap` with `derive` for CLI parsing
|
||||||
|
- `tracing` and `tracing-subscriber` for structured logging
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# Lore
|
||||||
|
|
||||||
|
## Project origins
|
||||||
|
|
||||||
|
rustunnel was created in June 2026 as a lab and development tunneling tool. The goal was a cross-platform, self-contained Rust binary that could tunnel traffic over HTTPS with mutual TLS and an application auth token, without relying on external infrastructure.
|
||||||
|
|
||||||
|
## Eras
|
||||||
|
|
||||||
|
### Foundation (June 3, 2026)
|
||||||
|
|
||||||
|
The first commit (`eb4ef2b`) laid out the project skeleton: CLI surface using clap, a `generate` command for self-signed certificates, secret redaction, and a cross-platform CI matrix. A follow-up commit (`70411e5`) addressed initial scrutiny findings.
|
||||||
|
|
||||||
|
### Secure tunnel implementation (June 3, 2026)
|
||||||
|
|
||||||
|
The core tunnel logic landed in `393100d`, implementing HTTPS mTLS with an auth handshake. Shortly after, `76bda0b` addressed secure-tunnel scrutiny findings. Dependency updates and CLI refinements followed in `fc776fc`.
|
||||||
|
|
||||||
|
### SOCKS5 and multiplexing (June 3, 2026)
|
||||||
|
|
||||||
|
Three major commits transformed the architecture:
|
||||||
|
|
||||||
|
- `6466753` — Implemented a SOCKS5 proxy with stream multiplexing over HTTPS. This replaced the earlier design where each SOCKS5 request opened a new HTTPS connection.
|
||||||
|
- `f871ba7` — Replaced per-SOCKS-request HTTPS connections with a single persistent multiplexed tunnel session.
|
||||||
|
- `d6a0a40` — Replaced HTTP-shaped forwarding with a true persistent multiplexed tunnel using a custom binary framing protocol.
|
||||||
|
|
||||||
|
### Framing hardening (June 3, 2026)
|
||||||
|
|
||||||
|
- `b67248e` — Fixed frame header corruption on partial reads. The framing reader was refined to never consume bytes from the buffer until a complete frame (header + payload) is available.
|
||||||
|
|
||||||
|
### Operational features (June 4, 2026)
|
||||||
|
|
||||||
|
- `10b8b24` — Added operational reconnect and shutdown handling. The connector now has a background reconnect loop with exponential backoff, and both sides handle graceful shutdown on SIGINT/SIGTERM.
|
||||||
|
- `32a29c8` — Fixed flaky generate tests and formatted `tunnel.rs`.
|
||||||
|
- `b0210a6` — Improved cross-platform CI E2E cleanup with `trap` and portable port killing.
|
||||||
|
- `241d7d1` — Unmasked SOCKS5 E2E curl failures in CI and added response content assertions.
|
||||||
|
- `cc1eb55` — Added the project README.
|
||||||
|
- `a758e8e` — Added connection key encoding (`connkey.rs`) and the full tunnel implementation integrating keys into `listen` and `connect`.
|
||||||
|
|
||||||
|
## Longest-standing code
|
||||||
|
|
||||||
|
Given the project's short lifespan (2 days), all code is roughly the same age. The modules that have survived the most rewrites are:
|
||||||
|
|
||||||
|
- `src/cli.rs` — CLI structure has remained stable since the first commit.
|
||||||
|
- `src/redact.rs` — Secret redaction has not changed since the foundation commit.
|
||||||
|
- `src/errors.rs` — Error types have been extended but the core structure is original.
|
||||||
|
|
||||||
|
## Deprecated features
|
||||||
|
|
||||||
|
None so far. The project is new enough that no features have been removed.
|
||||||
|
|
||||||
|
## Major rewrites
|
||||||
|
|
||||||
|
- **From HTTP-per-stream to persistent multiplexed tunnel (June 3, 2026)** — The original SOCKS5 implementation opened a new HTTPS connection for every stream. This was replaced by a single persistent tunnel with a binary framing protocol, dramatically reducing connection overhead.
|
||||||
|
- **From HTTP forwarding to binary framing (June 3, 2026)** — After multiplexing was introduced, the data plane switched from raw HTTP forwarding to a lightweight binary protocol with CONNECT/DATA/CLOSE/ERROR frames.
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
rustunnel is organized as a single Rust binary crate with a modular source tree. The architecture separates CLI parsing, credential generation, TLS configuration, SOCKS5 handling, binary framing, and the core tunnel engine into distinct modules.
|
||||||
|
|
||||||
|
## Component diagram
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
A[CLI `src/cli.rs`] --> B[Main dispatch `src/main.rs`]
|
||||||
|
B --> C[Credential generation `src/generate.rs`]
|
||||||
|
B --> D[Tunnel engine `src/tunnel.rs`]
|
||||||
|
B --> E[Connection keys `src/connkey.rs`]
|
||||||
|
D --> F[TLS stack `src/tls.rs`]
|
||||||
|
D --> G[SOCKS5 proxy `src/socks5.rs`]
|
||||||
|
D --> H[Framing protocol `src/framing.rs`]
|
||||||
|
D --> I[Signal handling `src/signal.rs`]
|
||||||
|
F --> J[Rustls + ring]
|
||||||
|
G --> K[Tokio TCP]
|
||||||
|
H --> L[Bytes + tokio-util]
|
||||||
|
C --> M[rcgen]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Data flow
|
||||||
|
|
||||||
|
A typical tunnel session flows like this:
|
||||||
|
|
||||||
|
1. **Listener** binds a TCP socket and wraps it with `tokio_rustls::TlsAcceptor`.
|
||||||
|
2. Connector initiates a TCP connection and upgrades it with `tokio_rustls::TlsConnector`.
|
||||||
|
3. Both sides perform mTLS handshakes: the listener requires a client certificate signed by the shared CA; the connector validates the server certificate against the same CA.
|
||||||
|
4. The connector sends an HTTP POST to `/tunnel` with an `X-Rustunnel-Token` header.
|
||||||
|
5. The listener validates the token with a constant-time comparison (`src/tunnel.rs`).
|
||||||
|
6. On success, both sides switch from HTTP/1.1 to a binary framing protocol (`src/framing.rs`).
|
||||||
|
7. The connector's local SOCKS5 proxy accepts client connections and opens multiplexed streams over the single persistent tunnel.
|
||||||
|
8. The listener decodes each stream's target address, opens a local TCP connection to that target, and forwards data bidirectionally.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Client as Local app
|
||||||
|
participant Socks as SOCKS5 proxy
|
||||||
|
participant Conn as Connector
|
||||||
|
participant TLS as mTLS tunnel
|
||||||
|
participant List as Listener
|
||||||
|
participant Target as Target server
|
||||||
|
|
||||||
|
Client->>Socks: SOCKS5 CONNECT
|
||||||
|
Socks->>Conn: open stream
|
||||||
|
Conn->>TLS: TLS handshake
|
||||||
|
TLS->>List: accept + verify client cert
|
||||||
|
Conn->>List: POST /tunnel + auth token
|
||||||
|
List->>Conn: 200 OK
|
||||||
|
Conn->>List: binary framing CONNECT frame
|
||||||
|
List->>Target: TCP connect
|
||||||
|
Target->>List: response
|
||||||
|
List->>Conn: DATA frame
|
||||||
|
Conn->>Socks: raw bytes
|
||||||
|
Socks->>Client: response
|
||||||
|
```
|
||||||
|
|
||||||
|
## Module responsibilities
|
||||||
|
|
||||||
|
| Module | Responsibility |
|
||||||
|
| ------ | -------------- |
|
||||||
|
| `src/cli.rs` | clap-based CLI definitions for all subcommands and arguments |
|
||||||
|
| `src/main.rs` | Entry point, crypto provider installation, tracing init, command dispatch |
|
||||||
|
| `src/tunnel.rs` | Core tunnel engine: listener accept loop, connector reconnect loop, SOCKS5 proxy, stream mux, HTTP auth handshake, framing loop |
|
||||||
|
| `src/tls.rs` | rustls server and client config builders, certificate loading, identity validation, insecure verifier |
|
||||||
|
| `src/socks5.rs` | SOCKS5 greeting, auth, CONNECT request parsing, target resolution |
|
||||||
|
| `src/framing.rs` | Binary frame serialization/deserialization, `FrameReader`, `FrameWriter` |
|
||||||
|
| `src/generate.rs` | Self-signed CA, server cert, client cert, and auth token generation with rcgen |
|
||||||
|
| `src/connkey.rs` | `ConnectionKey` struct: encode/decode all material as a base64 string |
|
||||||
|
| `src/redact.rs` | `Redacted` wrapper and helpers to prevent secrets from leaking into logs |
|
||||||
|
| `src/errors.rs` | `thiserror`-based error enums for TLS, auth, tunnel, and host operations |
|
||||||
|
| `src/config.rs` | JSON configuration file format for generated credential sets |
|
||||||
|
| `src/signal.rs` | Cross-platform shutdown signal handler (SIGINT/SIGTERM on Unix, Ctrl+C on Windows) |
|
||||||
|
|
||||||
|
## Threading model
|
||||||
|
|
||||||
|
The project uses Tokio's multi-threaded runtime. Each side (listener and connector) spawns tasks for:
|
||||||
|
|
||||||
|
- Accepting new TCP connections
|
||||||
|
- TLS handshakes
|
||||||
|
- Per-tunnel framing loops
|
||||||
|
- Per-stream target forwarding
|
||||||
|
- SOCKS5 client handling
|
||||||
|
|
||||||
|
The connector's SOCKS5 proxy stays bound permanently while a background reconnect loop maintains the tunnel. When the tunnel drops, existing streams receive EOF and new streams wait for the reconnect loop to re-establish the session.
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Getting started
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Rust 1.85+ (the project uses Edition 2024)
|
||||||
|
- A POSIX shell or PowerShell for running commands
|
||||||
|
- curl (for E2E testing)
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd rustunnel
|
||||||
|
cargo build --release
|
||||||
|
```
|
||||||
|
|
||||||
|
The release binary is produced at `target/release/rustunnel`.
|
||||||
|
|
||||||
|
## Run tests
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo fmt --check
|
||||||
|
cargo clippy --all-targets -- -D warnings
|
||||||
|
cargo check
|
||||||
|
cargo test --all-targets
|
||||||
|
```
|
||||||
|
|
||||||
|
The test suite includes unit tests for framing, SOCKS5 parsing, TLS config building, credential generation, connection key roundtrips, and full end-to-end tunnel flows.
|
||||||
|
|
||||||
|
## Generate credentials
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./target/release/rustunnel generate --out ./creds
|
||||||
|
```
|
||||||
|
|
||||||
|
This creates:
|
||||||
|
|
||||||
|
- `creds/ca.pem` — CA certificate
|
||||||
|
- `creds/ca.key` — CA private key
|
||||||
|
- `creds/server.crt` — server certificate
|
||||||
|
- `creds/server.key` — server private key
|
||||||
|
- `creds/client.crt` — client certificate
|
||||||
|
- `creds/client.key` — client private key
|
||||||
|
- `creds/token.txt` — auth token
|
||||||
|
- `creds/config.json` — configuration file with paths and defaults
|
||||||
|
|
||||||
|
## Run a tunnel
|
||||||
|
|
||||||
|
### Start the listener
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./target/release/rustunnel listen \
|
||||||
|
--listen 127.0.0.1:4180 \
|
||||||
|
--cert ./creds/server.crt \
|
||||||
|
--key ./creds/server.key \
|
||||||
|
--ca-cert ./creds/ca.pem \
|
||||||
|
--auth-token-file ./creds/token.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
### Start the connector
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./target/release/rustunnel connect \
|
||||||
|
--target 127.0.0.1:4180 \
|
||||||
|
--socks 127.0.0.1:1180 \
|
||||||
|
--cert ./creds/client.crt \
|
||||||
|
--key ./creds/client.key \
|
||||||
|
--ca-cert ./creds/ca.pem \
|
||||||
|
--auth-token-file ./creds/token.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
### Use the SOCKS5 proxy
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl --proxy socks5h://127.0.0.1:1180 http://127.0.0.1:4181/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Using a connection key
|
||||||
|
|
||||||
|
Instead of passing individual certificate paths, you can generate a single connection key:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./target/release/rustunnel keygen --target 127.0.0.1:4180
|
||||||
|
```
|
||||||
|
|
||||||
|
Then start the listener with the key (it auto-generates credentials):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./target/release/rustunnel listen --connection-key <key>
|
||||||
|
```
|
||||||
|
|
||||||
|
And connect with the same key:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./target/release/rustunnel connect <key>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Defaults
|
||||||
|
|
||||||
|
- Listener bind address: `0.0.0.0:4180`
|
||||||
|
- Connector target: required (or taken from connection key)
|
||||||
|
- Connector SOCKS5 proxy: `127.0.0.1:1180`
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# Glossary
|
||||||
|
|
||||||
|
**Connection key** — a base64-encoded JSON blob (prefix `rtun1.`) that bundles the CA certificate, server certificate, server key, client certificate, client key, target address, auth token, and version. Generated by `rustunnel keygen` and consumed by `rustunnel listen` and `rustunnel connect`.
|
||||||
|
|
||||||
|
**Connector** — the `rustunnel connect` side. Establishes an outbound HTTPS+mTLS connection to the listener, authenticates, then exposes a local SOCKS5 proxy for local applications.
|
||||||
|
|
||||||
|
**Framing protocol** — the binary protocol used after the HTTP auth handshake completes. Frame types: CONNECT, CONNECT_REPLY, DATA, CLOSE, ERROR. All multi-byte integers are big-endian. Max payload per frame is 4 MiB.
|
||||||
|
|
||||||
|
**Listener** — the `rustunnel listen` side. Binds an HTTPS server that accepts mTLS connections, validates the auth token, then upgrades the connection to the binary framing protocol.
|
||||||
|
|
||||||
|
**mTLS** — mutual TLS. Both the listener and connector present certificates and verify the peer's certificate against a shared CA. rustunnel uses rustls with ring as the crypto provider.
|
||||||
|
|
||||||
|
**Security gate** — one of the validation steps that must pass before tunnel forwarding begins: TLS handshake, client certificate verification (listener), server certificate verification (connector), application auth token validation.
|
||||||
|
|
||||||
|
**SOCKS5 proxy** — the local proxy exposed by the connector. Implements RFC 1928 CONNECT with optional username/password auth (RFC 1929). Supports IPv4 and domain name targets.
|
||||||
|
|
||||||
|
**Stream ID** — a monotonically increasing odd integer (1, 3, 5, ...) assigned by the connector for each SOCKS5 connection. Even IDs are reserved for future server-initiated streams.
|
||||||
|
|
||||||
|
**StreamMux** — the connector-side structure that manages multiple concurrent streams over a single tunnel connection. Maps stream IDs to per-stream data channels.
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
# rustunnel
|
||||||
|
|
||||||
|
`rustunnel` is a cross-platform Rust CLI tool for lab and development tunneling over HTTPS with mutual TLS (mTLS) and an application-level auth token. It exposes a local SOCKS5 proxy on the connector side and forwards traffic only after HTTPS, certificate, and auth checks succeed.
|
||||||
|
|
||||||
|
The tool is designed for local labs, development environments, and controlled testing where you own or are authorized to operate both endpoints. It is not a production-grade tunnel or a mechanism for accessing systems without permission.
|
||||||
|
|
||||||
|
## What it does
|
||||||
|
|
||||||
|
- **Listener** (`rustunnel listen`) — binds an HTTPS server that accepts mTLS connections from connectors, validates an auth token, then upgrades the connection to a persistent binary-framed tunnel.
|
||||||
|
- **Connector** (`rustunnel connect`) — connects to the listener over HTTPS with mTLS, authenticates, then exposes a local SOCKS5 proxy that forwards traffic through the tunnel.
|
||||||
|
- **Credential generation** (`rustunnel generate`) — creates a self-signed CA, server certificate, client certificate, and auth token for a local session.
|
||||||
|
- **Connection keys** (`rustunnel keygen`) — bundles all credential material into a single base64-encoded string that can be copy-pasted between machines.
|
||||||
|
|
||||||
|
## Quick links
|
||||||
|
|
||||||
|
- [Architecture](../overview/architecture.md)
|
||||||
|
- [Getting started](../overview/getting-started.md)
|
||||||
|
- [Tunnel engine](../systems/tunnel-engine.md)
|
||||||
|
- [TLS stack](../systems/tls-stack.md)
|
||||||
|
- [Security posture](../security.md)
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
# Configuration
|
||||||
|
|
||||||
|
## CLI arguments
|
||||||
|
|
||||||
|
### `listen`
|
||||||
|
|
||||||
|
| Argument | Default | Description |
|
||||||
|
| -------- | ------- | ----------- |
|
||||||
|
| `--listen` | `0.0.0.0:4180` | HTTPS listener bind address |
|
||||||
|
| `--advertise` | derived from `--listen` | Public address in auto-generated connection keys |
|
||||||
|
| `--connection-key` | none | Reusable key (env: `RUSTUNNEL_KEY`) |
|
||||||
|
| `--socks` | none | Server-side SOCKS5 proxy address |
|
||||||
|
| `--cert` | "" | Server certificate path |
|
||||||
|
| `--key` | "" | Server private key path |
|
||||||
|
| `--ca-cert` | "" | CA certificate path |
|
||||||
|
| `--auth-token` | "" | Auth token value |
|
||||||
|
| `--auth-token-file` | none | Auth token file path |
|
||||||
|
| `--insecure-skip-tls-verify` | false | Skip client cert verification |
|
||||||
|
|
||||||
|
### `connect`
|
||||||
|
|
||||||
|
| Argument | Default | Description |
|
||||||
|
| -------- | ------- | ----------- |
|
||||||
|
| `CONNECTION_KEY` | none | Positional connection key |
|
||||||
|
| `--target` | from key | Listener address |
|
||||||
|
| `--connection-key` | none | Key via flag (env: `RUSTUNNEL_KEY`) |
|
||||||
|
| `--socks` | `127.0.0.1:1180` | Local SOCKS5 proxy address |
|
||||||
|
| `--cert` | "" | Client certificate path |
|
||||||
|
| `--key` | "" | Client private key path |
|
||||||
|
| `--ca-cert` | "" | CA certificate path |
|
||||||
|
| `--auth-token` | "" | Auth token value |
|
||||||
|
| `--auth-token-file` | none | Auth token file path |
|
||||||
|
| `--insecure-skip-tls-verify` | false | Skip server cert verification |
|
||||||
|
|
||||||
|
### `generate`
|
||||||
|
|
||||||
|
| Argument | Default | Description |
|
||||||
|
| -------- | ------- | ----------- |
|
||||||
|
| `--out` | `.` | Output directory |
|
||||||
|
| `--ca-name` | `rustunnel-ca` | CA common name |
|
||||||
|
| `--server-name` | `rustunnel-server` | Server common name |
|
||||||
|
| `--client-name` | `rustunnel-client` | Client common name |
|
||||||
|
|
||||||
|
### `keygen`
|
||||||
|
|
||||||
|
| Argument | Default | Description |
|
||||||
|
| -------- | ------- | ----------- |
|
||||||
|
| `--target` | `127.0.0.1:4180` | Listener address embedded in key |
|
||||||
|
| `--ca-name` | `rustunnel-ca` | CA common name |
|
||||||
|
| `--server-name` | `rustunnel-server` | Server common name |
|
||||||
|
| `--client-name` | `rustunnel-client` | Client common name |
|
||||||
|
|
||||||
|
## Environment variables
|
||||||
|
|
||||||
|
- `RUST_LOG` — tracing filter (e.g., `debug`, `trace`)
|
||||||
|
- `RUSTUNNEL_KEY` — connection key for `listen` and `connect`
|
||||||
|
|
||||||
|
## Config file format
|
||||||
|
|
||||||
|
`config.json` (generated by `rustunnel generate`):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"listen_address": "0.0.0.0",
|
||||||
|
"listen_port": 4180,
|
||||||
|
"socks_address": "127.0.0.1",
|
||||||
|
"socks_port": 1180,
|
||||||
|
"ca_cert_path": "ca.pem",
|
||||||
|
"server_cert_path": "server.crt",
|
||||||
|
"server_key_path": "server.key",
|
||||||
|
"client_cert_path": "client.crt",
|
||||||
|
"client_key_path": "client.key",
|
||||||
|
"auth_token_path": "token.txt"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/cli.rs` | clap argument definitions and defaults |
|
||||||
|
| `src/config.rs` | `Config` struct for JSON serialization |
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
# Data models
|
||||||
|
|
||||||
|
## ConnectionKey
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub struct ConnectionKey {
|
||||||
|
pub version: u8, // must be 1
|
||||||
|
pub target: String, // listener address
|
||||||
|
pub ca_cert_pem: String,
|
||||||
|
pub server_cert_pem: String,
|
||||||
|
pub server_key_pem: String,
|
||||||
|
pub client_cert_pem: String,
|
||||||
|
pub client_key_pem: String,
|
||||||
|
pub auth_token: String,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Encoded as: `rtun1.` + base64url(JSON) — no padding.
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub struct Config {
|
||||||
|
pub listen_address: String,
|
||||||
|
pub listen_port: u16,
|
||||||
|
pub socks_address: String,
|
||||||
|
pub socks_port: u16,
|
||||||
|
pub ca_cert_path: String,
|
||||||
|
pub server_cert_path: String,
|
||||||
|
pub server_key_path: String,
|
||||||
|
pub client_cert_path: String,
|
||||||
|
pub client_key_path: String,
|
||||||
|
pub auth_token_path: String,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Frame
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub struct Frame {
|
||||||
|
pub frame_type: u8, // 0x01 CONNECT, 0x02 CONNECT_REPLY, 0x03 DATA, 0x04 CLOSE, 0x05 ERROR
|
||||||
|
pub stream_id: u64, // odd for client-initiated
|
||||||
|
pub flags: u8, // reserved (0x00)
|
||||||
|
pub payload: Bytes,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Wire format: 1 byte type + 8 bytes stream_id (big-endian) + 4 bytes len (big-endian) + 1 byte flags + len bytes payload.
|
||||||
|
|
||||||
|
Max payload: 4 MiB.
|
||||||
|
|
||||||
|
## Socks5Target
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub enum Socks5Target {
|
||||||
|
Ipv4(std::net::Ipv4Addr, u16),
|
||||||
|
Domain { domain: String, port: u16 },
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## GeneratedMaterial
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub struct GeneratedMaterial {
|
||||||
|
pub ca_cert_pem: String,
|
||||||
|
pub ca_key_pem: String,
|
||||||
|
pub server_cert_pem: String,
|
||||||
|
pub server_key_pem: String,
|
||||||
|
pub client_cert_pem: String,
|
||||||
|
pub client_key_pem: String,
|
||||||
|
pub auth_token: String,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Error enums
|
||||||
|
|
||||||
|
- `TlsError` — missing/invalid certificates, identity mismatch, expiration, verification failure
|
||||||
|
- `AuthError` — missing or invalid token
|
||||||
|
- `TunnelError` — TLS error, auth error, bind error, connection refused, port in use, protocol error
|
||||||
|
- `HostError` — DNS resolution failure, invalid hostname, no addresses
|
||||||
|
- `Socks5Error` — invalid version, unsupported auth/command/addr type, parse errors
|
||||||
|
- `FrameError` — frame too short, payload overflow, I/O error, protocol error
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/connkey.rs` | `ConnectionKey` |
|
||||||
|
| `src/config.rs` | `Config` |
|
||||||
|
| `src/framing.rs` | `Frame`, `FrameReader`, `FrameWriter` |
|
||||||
|
| `src/socks5.rs` | `Socks5Target`, `Socks5State`, `Socks5Error` |
|
||||||
|
| `src/generate.rs` | `GeneratedMaterial` |
|
||||||
|
| `src/errors.rs` | `TlsError`, `AuthError`, `TunnelError`, `HostError` |
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# Dependencies
|
||||||
|
|
||||||
|
## Runtime dependencies
|
||||||
|
|
||||||
|
| Crate | Version | Purpose |
|
||||||
|
| ----- | ------- | ------- |
|
||||||
|
| `clap` | 4 | CLI parsing with derive macros |
|
||||||
|
| `tokio` | 1 | Async runtime (full features) |
|
||||||
|
| `tokio-util` | 0.7 | I/O utilities |
|
||||||
|
| `tracing` | 0.1 | Structured logging |
|
||||||
|
| `tracing-subscriber` | 0.3 | Log formatting with env-filter |
|
||||||
|
| `anyhow` | 1 | Application error handling |
|
||||||
|
| `thiserror` | 1 | Library error enums |
|
||||||
|
| `serde` | 1 | Serialization (derive) |
|
||||||
|
| `serde_json` | 1 | JSON encoding/decoding |
|
||||||
|
| `rcgen` | 0.14 | Certificate generation |
|
||||||
|
| `x509-parser` | 0.16 | Certificate parsing for identity validation |
|
||||||
|
| `time` | 0.3 | Date/time formatting for certificate validity |
|
||||||
|
| `rand` | 0.8 | Random token generation |
|
||||||
|
| `hex` | 0.4 | Hex encoding for auth tokens |
|
||||||
|
| `rustls` | 0.23 | TLS implementation (ring provider) |
|
||||||
|
| `rustls-pki-types` | 1 | rustls type definitions |
|
||||||
|
| `rustls-pemfile` | 2 | PEM parsing for certificates and keys |
|
||||||
|
| `tokio-rustls` | 0.26 | Tokio integration for rustls |
|
||||||
|
| `hyper` | 1 | HTTP types (used for auth handshake) |
|
||||||
|
| `hyper-util` | 0.1 | HTTP utilities |
|
||||||
|
| `http-body-util` | 0.1 | HTTP body utilities |
|
||||||
|
| `tower-service` | 0.3 | Service trait (Hyper ecosystem) |
|
||||||
|
| `base64` | 0.22 | Connection key encoding |
|
||||||
|
| `sha2` | 0.10 | Certificate fingerprinting |
|
||||||
|
| `bytes` | 1 | Byte buffers for framing |
|
||||||
|
| `futures` | 0.3 | Future utilities |
|
||||||
|
| `url` | 2 | URL parsing |
|
||||||
|
|
||||||
|
## Dev dependencies
|
||||||
|
|
||||||
|
| Crate | Version | Purpose |
|
||||||
|
| ----- | ------- | ------- |
|
||||||
|
| `tempfile` | 3 | Temporary directories for tests |
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `Cargo.toml` | Dependency declarations and features |
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
# Reference
|
||||||
|
|
||||||
|
Configuration, data models, and external dependencies.
|
||||||
|
|
||||||
|
- [Configuration](configuration.md)
|
||||||
|
- [Data models](data-models.md)
|
||||||
|
- [Dependencies](dependencies.md)
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# Security
|
||||||
|
|
||||||
|
rustunnel's security posture is built on multiple layers: TLS transport, mutual authentication, application-level auth tokens, secret redaction, and fail-closed error handling.
|
||||||
|
|
||||||
|
## Security gates
|
||||||
|
|
||||||
|
Before any traffic flows, four gates must pass:
|
||||||
|
|
||||||
|
1. **TLS handshake** — both sides negotiate a TLS 1.2+ connection via rustls with ring.
|
||||||
|
2. **Server certificate verification** — the connector verifies the listener's certificate against the shared CA.
|
||||||
|
3. **Client certificate verification** — the listener verifies the connector's certificate against the same CA (skipped in insecure mode).
|
||||||
|
4. **Application auth token** — the connector sends an `X-Rustunnel-Token` header; the listener validates it with constant-time comparison.
|
||||||
|
|
||||||
|
The E2E test `auth_required_in_addition_to_mtls` verifies that valid mTLS alone is not sufficient; the correct auth token is also required.
|
||||||
|
|
||||||
|
## mTLS
|
||||||
|
|
||||||
|
- The listener requires client certificates signed by the shared CA. It uses `WebPkiClientVerifier`.
|
||||||
|
- The connector presents its client certificate and validates the server certificate against the CA.
|
||||||
|
- Certificate identities are validated against SANs (Subject Alternative Names) and CN. The server certificate includes SANs for `localhost`, `rustunnel`, and `127.0.0.1`, plus the bind IP if applicable.
|
||||||
|
|
||||||
|
## Auth token
|
||||||
|
|
||||||
|
- Generated as 32 random bytes encoded as 64 hex characters.
|
||||||
|
- Compared with a constant-time XOR loop to prevent timing attacks.
|
||||||
|
- Never logged in raw form. All log output uses the `Redacted` wrapper.
|
||||||
|
- Auth failures are terminal: the connector's reconnect loop exits on auth failure rather than retrying.
|
||||||
|
|
||||||
|
## Secret redaction
|
||||||
|
|
||||||
|
- `Redacted` in `src/redact.rs` wraps strings and displays `[REDACTED(len=N)]`.
|
||||||
|
- `redact_pem()` shows only the PEM block type and line count.
|
||||||
|
- `redact_config_json()` scrubs sensitive keys from JSON.
|
||||||
|
- `is_sensitive()` detects PEM private key blocks.
|
||||||
|
- `is_auth_token()` detects long alphanumeric strings.
|
||||||
|
- `is_connection_key()` detects strings starting with `rtun1.`.
|
||||||
|
|
||||||
|
The E2E test `ops_auth_failure_is_actionable_and_redacted` verifies that error messages do not contain raw token values.
|
||||||
|
|
||||||
|
## Insecure mode
|
||||||
|
|
||||||
|
`--insecure-skip-tls-verify` disables certificate verification on either side. This is intended for DPI/intercepted environments only. When enabled:
|
||||||
|
|
||||||
|
- The listener uses `with_no_client_auth()` instead of `WebPkiClientVerifier`.
|
||||||
|
- The connector uses a custom `InsecureServerCertVerifier` that accepts any certificate.
|
||||||
|
- A `warn` level log is emitted.
|
||||||
|
|
||||||
|
## Fail-closed design
|
||||||
|
|
||||||
|
All error paths reject the connection rather than falling back to an insecure mode. There are no bypasses around TLS, certificate verification, or auth.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/tls.rs` | TLS config, certificate loading, identity validation |
|
||||||
|
| `src/tunnel.rs` | Auth handshake, constant-time comparison, security gate tests |
|
||||||
|
| `src/redact.rs` | Secret redaction wrappers and helpers |
|
||||||
|
| `src/errors.rs` | Error enums for TLS, auth, tunnel |
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# Credential generation
|
||||||
|
|
||||||
|
The credential generation module in `src/generate.rs` creates all the certificate and key material needed for a rustunnel session.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Generate a self-signed CA, server certificate, client certificate, and a random auth token. Write them to disk in a specified output directory.
|
||||||
|
|
||||||
|
## Key abstractions
|
||||||
|
|
||||||
|
| Type/Function | File | Description |
|
||||||
|
| ------------- | ---- | ----------- |
|
||||||
|
| `GeneratedMaterial` | `src/generate.rs` | Struct holding all generated PEM strings and the auth token |
|
||||||
|
| `generate` | `src/generate.rs` | Generate material and write 8 files to an output directory |
|
||||||
|
| `generate_material` | `src/generate.rs` | Generate material without writing to disk |
|
||||||
|
| `generate_ca` | `src/generate.rs` | Create a self-signed CA with rcgen |
|
||||||
|
| `generate_server_cert` | `src/generate.rs` | Create a server cert signed by the CA |
|
||||||
|
| `generate_client_cert` | `src/generate.rs` | Create a client cert signed by the CA |
|
||||||
|
| `generate_auth_token` | `src/generate.rs` | Generate a 32-byte random hex token |
|
||||||
|
|
||||||
|
## Certificate details
|
||||||
|
|
||||||
|
- **CA** — CN from `--ca-name`, org "rustunnel", serial 1, constrained CA basic constraints
|
||||||
|
- **Server cert** — CN from `--server-name`, org "rustunnel", serial 2, SANs: `localhost`, `rustunnel`, `127.0.0.1`, and optionally the bind IP if not `0.0.0.0` or `::`
|
||||||
|
- **Client cert** — CN from `--client-name`, org "rustunnel", serial 3
|
||||||
|
- **Auth token** — 64 hex characters (32 random bytes)
|
||||||
|
|
||||||
|
## Output files
|
||||||
|
|
||||||
|
| File | Content |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `ca.pem` | CA certificate (PEM) |
|
||||||
|
| `ca.key` | CA private key (PEM) |
|
||||||
|
| `server.crt` | Server certificate (PEM) |
|
||||||
|
| `server.key` | Server private key (PEM) |
|
||||||
|
| `client.crt` | Client certificate (PEM) |
|
||||||
|
| `client.key` | Client private key (PEM) |
|
||||||
|
| `token.txt` | Auth token (plain text) |
|
||||||
|
| `config.json` | Paths and default addresses (JSON) |
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/generate.rs` | Certificate generation, token generation, file writing |
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# Framing protocol
|
||||||
|
|
||||||
|
The framing protocol in `src/framing.rs` defines the binary message format used after the HTTP auth handshake upgrades the connection to a persistent tunnel.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Carry multiple bidirectional streams over a single TLS connection with minimal overhead.
|
||||||
|
|
||||||
|
## Frame format
|
||||||
|
|
||||||
|
All multi-byte integers are big-endian.
|
||||||
|
|
||||||
|
```
|
||||||
|
+--------+----------+----------+-------+-----------+
|
||||||
|
| Type | StreamID | Len | Flags | Payload.. |
|
||||||
|
| 1 byte | 8 bytes | 4 bytes | 1 byte| variable |
|
||||||
|
+--------+----------+----------+-------+-----------+
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Type** — 0x01 CONNECT, 0x02 CONNECT_REPLY, 0x03 DATA, 0x04 CLOSE, 0x05 ERROR
|
||||||
|
- **StreamID** — odd IDs for client-initiated streams, even reserved for server-initiated
|
||||||
|
- **Len** — payload length, max 4 MiB
|
||||||
|
- **Flags** — reserved for future use (currently always 0x00)
|
||||||
|
|
||||||
|
## Key abstractions
|
||||||
|
|
||||||
|
| Type | File | Description |
|
||||||
|
| ---- | ---- | ----------- |
|
||||||
|
| `Frame` | `src/framing.rs` | Struct holding frame_type, stream_id, flags, payload |
|
||||||
|
| `FrameReader` | `src/framing.rs` | Async reader with buffering that handles partial TCP reads |
|
||||||
|
| `FrameWriter` | `src/framing.rs` | Async writer that serializes and flushes frames |
|
||||||
|
| `encode_target` | `src/framing.rs` | Encode a `Socks5Target` into CONNECT frame payload |
|
||||||
|
| `decode_target` | `src/framing.rs` | Decode a `Socks5Target` from CONNECT frame payload |
|
||||||
|
|
||||||
|
## Partial read discipline
|
||||||
|
|
||||||
|
`FrameReader::read_frame` loops until a complete frame is available:
|
||||||
|
|
||||||
|
1. Attempt to deserialize from the buffer.
|
||||||
|
2. If the frame is incomplete (`FrameTooShort`), read more bytes from the underlying TCP stream into the buffer.
|
||||||
|
3. If the connection closes with a partial frame in the buffer, return an error.
|
||||||
|
4. Only consume bytes from the buffer once the full frame is present.
|
||||||
|
|
||||||
|
This prevents header corruption when TCP segments arrive in arbitrary sizes.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/framing.rs` | Frame format, serialization, async reader/writer, target encoding |
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# Systems
|
||||||
|
|
||||||
|
rustunnel's internal architecture is divided into five systems:
|
||||||
|
|
||||||
|
- [Tunnel engine](tunnel-engine.md) — listener accept loop, connector reconnect loop, stream multiplexing, SOCKS5 proxy integration
|
||||||
|
- [TLS stack](tls-stack.md) — rustls configuration, certificate loading, identity validation
|
||||||
|
- [Framing protocol](framing-protocol.md) — binary frame format, serialization, partial read handling
|
||||||
|
- [SOCKS5 proxy](socks5-proxy.md) — RFC 1928 handshake, CONNECT support, target resolution
|
||||||
|
- [Credential generation](credential-generation.md) — self-signed CA and certificate generation with rcgen
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# SOCKS5 proxy
|
||||||
|
|
||||||
|
The SOCKS5 module in `src/socks5.rs` implements RFC 1928 (SOCKS5 protocol) and RFC 1929 (username/password authentication).
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Handle the SOCKS5 handshake on the connector side, parse CONNECT requests, and resolve target addresses so that local applications can forward traffic through the tunnel.
|
||||||
|
|
||||||
|
## Supported features
|
||||||
|
|
||||||
|
- CONNECT command only (no BIND or UDP)
|
||||||
|
- No-auth and username/password authentication methods
|
||||||
|
- IPv4 and domain name targets (no IPv6)
|
||||||
|
- Target DNS resolution on the connector side
|
||||||
|
|
||||||
|
## Key abstractions
|
||||||
|
|
||||||
|
| Type | File | Description |
|
||||||
|
| ---- | ---- | ----------- |
|
||||||
|
| `Socks5Target` | `src/socks5.rs` | Enum: `Ipv4(addr, port)` or `Domain { domain, port }` |
|
||||||
|
| `Socks5State` | `src/socks5.rs` | State machine: ExpectGreeting → ExpectAuth → ExpectRequest → RequestReceived → Established → Closed |
|
||||||
|
| `Socks5Error` | `src/socks5.rs` | Error enum for malformed input, unsupported methods, and parse failures |
|
||||||
|
| `parse_greeting` | `src/socks5.rs` | Parse client greeting and choose auth method |
|
||||||
|
| `parse_auth_request` | `src/socks5.rs` | Parse username/password auth request |
|
||||||
|
| `parse_connect_request` | `src/socks5.rs` | Parse CONNECT request and extract target |
|
||||||
|
| `resolve_target` | `src/socks5.rs` | Async DNS resolution of a `Socks5Target` to a `SocketAddr` |
|
||||||
|
|
||||||
|
## Integration
|
||||||
|
|
||||||
|
The connector's `run_socks5_proxy` in `src/tunnel.rs` calls `socks5_handshake` (an async function in `src/tunnel.rs` that uses `src/socks5.rs` constants and helpers) for each incoming SOCKS5 connection. After handshake, it opens a stream via `StreamMux::open_stream` and forwards data bidirectionally.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/socks5.rs` | SOCKS5 protocol constants, parsing, target resolution |
|
||||||
|
| `src/tunnel.rs` | Async SOCKS5 handshake and proxy integration |
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# TLS stack
|
||||||
|
|
||||||
|
The TLS stack in `src/tls.rs` configures rustls for both server (listener) and client (connector) roles with mutual TLS.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Provide safe, fail-closed TLS configuration builders that load certificates and keys, validate certificate identities, and support both secure and insecure (lab-only) modes.
|
||||||
|
|
||||||
|
## Key abstractions
|
||||||
|
|
||||||
|
| Type/Function | File | Description |
|
||||||
|
| ------------- | ---- | ----------- |
|
||||||
|
| `build_server_config` | `src/tls.rs` | Build a rustls `ServerConfig` with client certificate verification |
|
||||||
|
| `build_client_config` | `src/tls.rs` | Build a rustls `ClientConfig` with server certificate verification |
|
||||||
|
| `build_server_config_insecure` | `src/tls.rs` | Build a server config that skips client cert verification |
|
||||||
|
| `build_client_config_insecure` | `src/tls.rs` | Build a client config that skips server cert verification |
|
||||||
|
| `load_certs` | `src/tls.rs` | Load PEM-encoded certificates from a file |
|
||||||
|
| `load_key` | `src/tls.rs` | Load a PEM-encoded private key from a file |
|
||||||
|
| `validate_cert_identity` | `src/tls.rs` | Check SANs and CN against an expected hostname |
|
||||||
|
| `check_cert_not_expired` | `src/tls.rs` | Verify certificate validity period |
|
||||||
|
| `cert_fingerprint_sha256` | `src/tls.rs` | Compute SHA-256 fingerprint of a certificate |
|
||||||
|
| `InsecureServerCertVerifier` | `src/tls.rs` | Dangerous verifier that accepts any certificate |
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
Server config building:
|
||||||
|
|
||||||
|
1. Load the server certificate chain and private key.
|
||||||
|
2. Load the CA certificate into a `RootCertStore`.
|
||||||
|
3. Build a `WebPkiClientVerifier` from the root store.
|
||||||
|
4. Use `ServerConfig::builder().with_client_cert_verifier(...).with_single_cert(...)`.
|
||||||
|
|
||||||
|
Client config building:
|
||||||
|
|
||||||
|
1. Load the client certificate chain and private key.
|
||||||
|
2. Load the CA certificate into a `RootCertStore`.
|
||||||
|
3. Use `ClientConfig::builder().with_root_certificates(...).with_client_auth_cert(...)`.
|
||||||
|
|
||||||
|
Identity validation uses `x509-parser` to extract Subject Alternative Names and the subject CN, matching against the expected hostname or IP address.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/tls.rs` | TLS configuration, certificate loading, identity validation |
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
# Tunnel engine
|
||||||
|
|
||||||
|
The tunnel engine in `src/tunnel.rs` is the largest module (~3,500 lines). It implements the listener, connector, SOCKS5 proxy, stream multiplexer, reconnect loop, and HTTP auth handshake.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The tunnel engine establishes a persistent, authenticated, mTLS-encrypted tunnel between a listener and a connector, then multiplexes multiple SOCKS5 streams over that single connection.
|
||||||
|
|
||||||
|
## Key abstractions
|
||||||
|
|
||||||
|
| Type | File | Description |
|
||||||
|
| ---- | ---- | ----------- |
|
||||||
|
| `ListenerConfig` | `src/tunnel.rs` | Bind address, optional SOCKS5 address, TLS material, auth token, insecure flag |
|
||||||
|
| `ConnectorConfig` | `src/tunnel.rs` | Target host/port, TLS material, auth token, insecure flag |
|
||||||
|
| `ServerTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for server-side TLS |
|
||||||
|
| `ClientTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for client-side TLS |
|
||||||
|
| `StreamMux` | `src/tunnel.rs` | Connector-side multiplexer: manages stream IDs, data channels, and frame dispatch |
|
||||||
|
| `ServerStreamEntry` | `src/tunnel.rs` | Listener-side target write half for an active stream |
|
||||||
|
| `ConnectorStreamEntry` | `src/tunnel.rs` | Connector-side data channel and reply channel for an active stream |
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
### Listener side
|
||||||
|
|
||||||
|
1. `run_listener` binds a TCP socket, builds the TLS acceptor from `ServerTlsMaterial`, and starts accepting connections.
|
||||||
|
2. `handle_listener_connection` performs the TLS accept, verifies the client certificate (unless insecure), reads the HTTP POST to `/tunnel`, and validates the `X-Rustunnel-Token` header with `constant_time_compare`.
|
||||||
|
3. On auth success, it calls `handle_framing_server` to switch to binary framing.
|
||||||
|
4. `handle_framing_server` reads frames from the tunnel. A CONNECT frame triggers a TCP connection to the target address (opened from the listener side). DATA frames are forwarded to the target. CLOSE frames shut down the target connection.
|
||||||
|
5. If the listener was started with `--socks`, a server-side SOCKS5 proxy runs via `run_socks5_proxy`, receiving the active `StreamMux` through a `watch::channel`.
|
||||||
|
|
||||||
|
### Connector side
|
||||||
|
|
||||||
|
1. `run_connector_with_socks` builds the client TLS config and starts two concurrent pieces: a reconnect loop and a permanently-bound SOCKS5 proxy.
|
||||||
|
2. `run_reconnect_loop` repeatedly attempts to connect to the listener with exponential backoff (1s to 30s). Each attempt re-runs all security gates: TLS handshake, server cert verification, and auth token exchange.
|
||||||
|
3. On success, the loop creates a `StreamMux`, publishes it via the `watch::channel`, and waits for the framing task to finish (indicating disconnection).
|
||||||
|
4. When the tunnel drops, the loop logs the disconnection and retries.
|
||||||
|
5. `run_socks5_proxy` accepts SOCKS5 connections. For each connection, it checks if the `StreamMux` is available, opens a stream via `mux.open_stream()`, and performs bidirectional forwarding.
|
||||||
|
|
||||||
|
### Stream multiplexing
|
||||||
|
|
||||||
|
`StreamMux::open_stream` allocates the next odd stream ID, registers a data channel and reply channel, sends a CONNECT frame, and waits for a CONNECT_REPLY. The caller receives a `mpsc::Receiver<Bytes>` for tunnel-to-SOCKS5 data.
|
||||||
|
|
||||||
|
`StreamMux::dispatch_frame` routes inbound CONNECT_REPLY, DATA, CLOSE, and ERROR frames to the correct stream by ID.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
A[SOCKS5 client] -->|CONNECT| B[SOCKS5 proxy]
|
||||||
|
B -->|open_stream| C[StreamMux]
|
||||||
|
C -->|CONNECT frame| D[Framing writer]
|
||||||
|
D -->|TLS| E[Listener framing]
|
||||||
|
E -->|decode target| F[TCP connect to target]
|
||||||
|
F -->|DATA| E
|
||||||
|
E -->|DATA frame| D
|
||||||
|
D -->|dispatch| C
|
||||||
|
C -->|data_rx| B
|
||||||
|
B -->|raw bytes| A
|
||||||
|
```
|
||||||
|
|
||||||
|
## Entry points for modification
|
||||||
|
|
||||||
|
- To change auth behavior: modify `handle_listener_connection` (listener) and `authenticate_tunnel` (connector) in `src/tunnel.rs`.
|
||||||
|
- To change reconnect strategy: modify `run_reconnect_loop` in `src/tunnel.rs`.
|
||||||
|
- To add listener-side SOCKS5 behavior: the server-side SOCKS5 proxy is currently minimal; expand `run_socks5_proxy` in `src/tunnel.rs`.
|
||||||
|
|
||||||
|
## Key source files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| ---- | ------- |
|
||||||
|
| `src/tunnel.rs` | Full tunnel engine: listener, connector, SOCKS5 proxy, reconnect, framing loops, stream mux |
|
||||||
Reference in New Issue
Block a user