docs: add comprehensive project wiki for v1.0
CI / cargo fmt (push) Has been cancelled
CI / cargo clippy (macos-latest) (push) Has been cancelled
CI / cargo clippy (ubuntu-latest) (push) Has been cancelled
CI / cargo clippy (windows-latest) (push) Has been cancelled
CI / cargo test (macos-latest) (push) Has been cancelled
CI / cargo test (ubuntu-latest) (push) Has been cancelled
CI / cargo test (windows-latest) (push) Has been cancelled
CI / cargo build (macos-latest) (push) Has been cancelled
CI / cargo build (ubuntu-latest) (push) Has been cancelled
CI / cargo build (windows-latest) (push) Has been cancelled
CI / cargo build --release (macos-latest) (push) Has been cancelled
CI / cargo build --release (ubuntu-latest) (push) Has been cancelled
CI / cargo build --release (windows-latest) (push) Has been cancelled
CI / CLI smoke (macos-latest) (push) Has been cancelled
CI / CLI smoke (ubuntu-latest) (push) Has been cancelled
CI / CLI smoke (windows-latest) (push) Has been cancelled
CI / Minimal E2E (macos-latest) (push) Has been cancelled
CI / Minimal E2E (ubuntu-latest) (push) Has been cancelled
CI / Minimal E2E (windows-latest) (push) Has been cancelled
CI / cargo fmt (push) Has been cancelled
CI / cargo clippy (macos-latest) (push) Has been cancelled
CI / cargo clippy (ubuntu-latest) (push) Has been cancelled
CI / cargo clippy (windows-latest) (push) Has been cancelled
CI / cargo test (macos-latest) (push) Has been cancelled
CI / cargo test (ubuntu-latest) (push) Has been cancelled
CI / cargo test (windows-latest) (push) Has been cancelled
CI / cargo build (macos-latest) (push) Has been cancelled
CI / cargo build (ubuntu-latest) (push) Has been cancelled
CI / cargo build (windows-latest) (push) Has been cancelled
CI / cargo build --release (macos-latest) (push) Has been cancelled
CI / cargo build --release (ubuntu-latest) (push) Has been cancelled
CI / cargo build --release (windows-latest) (push) Has been cancelled
CI / CLI smoke (macos-latest) (push) Has been cancelled
CI / CLI smoke (ubuntu-latest) (push) Has been cancelled
CI / CLI smoke (windows-latest) (push) Has been cancelled
CI / Minimal E2E (macos-latest) (push) Has been cancelled
CI / Minimal E2E (ubuntu-latest) (push) Has been cancelled
CI / Minimal E2E (windows-latest) (push) Has been cancelled
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent
a758e8e0bc
commit
0166fb6511
@@ -0,0 +1,45 @@
|
||||
# Credential generation
|
||||
|
||||
The credential generation module in `src/generate.rs` creates all the certificate and key material needed for a rustunnel session.
|
||||
|
||||
## Purpose
|
||||
|
||||
Generate a self-signed CA, server certificate, client certificate, and a random auth token. Write them to disk in a specified output directory.
|
||||
|
||||
## Key abstractions
|
||||
|
||||
| Type/Function | File | Description |
|
||||
| ------------- | ---- | ----------- |
|
||||
| `GeneratedMaterial` | `src/generate.rs` | Struct holding all generated PEM strings and the auth token |
|
||||
| `generate` | `src/generate.rs` | Generate material and write 8 files to an output directory |
|
||||
| `generate_material` | `src/generate.rs` | Generate material without writing to disk |
|
||||
| `generate_ca` | `src/generate.rs` | Create a self-signed CA with rcgen |
|
||||
| `generate_server_cert` | `src/generate.rs` | Create a server cert signed by the CA |
|
||||
| `generate_client_cert` | `src/generate.rs` | Create a client cert signed by the CA |
|
||||
| `generate_auth_token` | `src/generate.rs` | Generate a 32-byte random hex token |
|
||||
|
||||
## Certificate details
|
||||
|
||||
- **CA** — CN from `--ca-name`, org "rustunnel", serial 1, constrained CA basic constraints
|
||||
- **Server cert** — CN from `--server-name`, org "rustunnel", serial 2, SANs: `localhost`, `rustunnel`, `127.0.0.1`, and optionally the bind IP if not `0.0.0.0` or `::`
|
||||
- **Client cert** — CN from `--client-name`, org "rustunnel", serial 3
|
||||
- **Auth token** — 64 hex characters (32 random bytes)
|
||||
|
||||
## Output files
|
||||
|
||||
| File | Content |
|
||||
| ---- | ------- |
|
||||
| `ca.pem` | CA certificate (PEM) |
|
||||
| `ca.key` | CA private key (PEM) |
|
||||
| `server.crt` | Server certificate (PEM) |
|
||||
| `server.key` | Server private key (PEM) |
|
||||
| `client.crt` | Client certificate (PEM) |
|
||||
| `client.key` | Client private key (PEM) |
|
||||
| `token.txt` | Auth token (plain text) |
|
||||
| `config.json` | Paths and default addresses (JSON) |
|
||||
|
||||
## Key source files
|
||||
|
||||
| File | Purpose |
|
||||
| ---- | ------- |
|
||||
| `src/generate.rs` | Certificate generation, token generation, file writing |
|
||||
@@ -0,0 +1,50 @@
|
||||
# Framing protocol
|
||||
|
||||
The framing protocol in `src/framing.rs` defines the binary message format used after the HTTP auth handshake upgrades the connection to a persistent tunnel.
|
||||
|
||||
## Purpose
|
||||
|
||||
Carry multiple bidirectional streams over a single TLS connection with minimal overhead.
|
||||
|
||||
## Frame format
|
||||
|
||||
All multi-byte integers are big-endian.
|
||||
|
||||
```
|
||||
+--------+----------+----------+-------+-----------+
|
||||
| Type | StreamID | Len | Flags | Payload.. |
|
||||
| 1 byte | 8 bytes | 4 bytes | 1 byte| variable |
|
||||
+--------+----------+----------+-------+-----------+
|
||||
```
|
||||
|
||||
- **Type** — 0x01 CONNECT, 0x02 CONNECT_REPLY, 0x03 DATA, 0x04 CLOSE, 0x05 ERROR
|
||||
- **StreamID** — odd IDs for client-initiated streams, even reserved for server-initiated
|
||||
- **Len** — payload length, max 4 MiB
|
||||
- **Flags** — reserved for future use (currently always 0x00)
|
||||
|
||||
## Key abstractions
|
||||
|
||||
| Type | File | Description |
|
||||
| ---- | ---- | ----------- |
|
||||
| `Frame` | `src/framing.rs` | Struct holding frame_type, stream_id, flags, payload |
|
||||
| `FrameReader` | `src/framing.rs` | Async reader with buffering that handles partial TCP reads |
|
||||
| `FrameWriter` | `src/framing.rs` | Async writer that serializes and flushes frames |
|
||||
| `encode_target` | `src/framing.rs` | Encode a `Socks5Target` into CONNECT frame payload |
|
||||
| `decode_target` | `src/framing.rs` | Decode a `Socks5Target` from CONNECT frame payload |
|
||||
|
||||
## Partial read discipline
|
||||
|
||||
`FrameReader::read_frame` loops until a complete frame is available:
|
||||
|
||||
1. Attempt to deserialize from the buffer.
|
||||
2. If the frame is incomplete (`FrameTooShort`), read more bytes from the underlying TCP stream into the buffer.
|
||||
3. If the connection closes with a partial frame in the buffer, return an error.
|
||||
4. Only consume bytes from the buffer once the full frame is present.
|
||||
|
||||
This prevents header corruption when TCP segments arrive in arbitrary sizes.
|
||||
|
||||
## Key source files
|
||||
|
||||
| File | Purpose |
|
||||
| ---- | ------- |
|
||||
| `src/framing.rs` | Frame format, serialization, async reader/writer, target encoding |
|
||||
@@ -0,0 +1,9 @@
|
||||
# Systems
|
||||
|
||||
rustunnel's internal architecture is divided into five systems:
|
||||
|
||||
- [Tunnel engine](tunnel-engine.md) — listener accept loop, connector reconnect loop, stream multiplexing, SOCKS5 proxy integration
|
||||
- [TLS stack](tls-stack.md) — rustls configuration, certificate loading, identity validation
|
||||
- [Framing protocol](framing-protocol.md) — binary frame format, serialization, partial read handling
|
||||
- [SOCKS5 proxy](socks5-proxy.md) — RFC 1928 handshake, CONNECT support, target resolution
|
||||
- [Credential generation](credential-generation.md) — self-signed CA and certificate generation with rcgen
|
||||
@@ -0,0 +1,37 @@
|
||||
# SOCKS5 proxy
|
||||
|
||||
The SOCKS5 module in `src/socks5.rs` implements RFC 1928 (SOCKS5 protocol) and RFC 1929 (username/password authentication).
|
||||
|
||||
## Purpose
|
||||
|
||||
Handle the SOCKS5 handshake on the connector side, parse CONNECT requests, and resolve target addresses so that local applications can forward traffic through the tunnel.
|
||||
|
||||
## Supported features
|
||||
|
||||
- CONNECT command only (no BIND or UDP)
|
||||
- No-auth and username/password authentication methods
|
||||
- IPv4 and domain name targets (no IPv6)
|
||||
- Target DNS resolution on the connector side
|
||||
|
||||
## Key abstractions
|
||||
|
||||
| Type | File | Description |
|
||||
| ---- | ---- | ----------- |
|
||||
| `Socks5Target` | `src/socks5.rs` | Enum: `Ipv4(addr, port)` or `Domain { domain, port }` |
|
||||
| `Socks5State` | `src/socks5.rs` | State machine: ExpectGreeting → ExpectAuth → ExpectRequest → RequestReceived → Established → Closed |
|
||||
| `Socks5Error` | `src/socks5.rs` | Error enum for malformed input, unsupported methods, and parse failures |
|
||||
| `parse_greeting` | `src/socks5.rs` | Parse client greeting and choose auth method |
|
||||
| `parse_auth_request` | `src/socks5.rs` | Parse username/password auth request |
|
||||
| `parse_connect_request` | `src/socks5.rs` | Parse CONNECT request and extract target |
|
||||
| `resolve_target` | `src/socks5.rs` | Async DNS resolution of a `Socks5Target` to a `SocketAddr` |
|
||||
|
||||
## Integration
|
||||
|
||||
The connector's `run_socks5_proxy` in `src/tunnel.rs` calls `socks5_handshake` (an async function in `src/tunnel.rs` that uses `src/socks5.rs` constants and helpers) for each incoming SOCKS5 connection. After handshake, it opens a stream via `StreamMux::open_stream` and forwards data bidirectionally.
|
||||
|
||||
## Key source files
|
||||
|
||||
| File | Purpose |
|
||||
| ---- | ------- |
|
||||
| `src/socks5.rs` | SOCKS5 protocol constants, parsing, target resolution |
|
||||
| `src/tunnel.rs` | Async SOCKS5 handshake and proxy integration |
|
||||
@@ -0,0 +1,45 @@
|
||||
# TLS stack
|
||||
|
||||
The TLS stack in `src/tls.rs` configures rustls for both server (listener) and client (connector) roles with mutual TLS.
|
||||
|
||||
## Purpose
|
||||
|
||||
Provide safe, fail-closed TLS configuration builders that load certificates and keys, validate certificate identities, and support both secure and insecure (lab-only) modes.
|
||||
|
||||
## Key abstractions
|
||||
|
||||
| Type/Function | File | Description |
|
||||
| ------------- | ---- | ----------- |
|
||||
| `build_server_config` | `src/tls.rs` | Build a rustls `ServerConfig` with client certificate verification |
|
||||
| `build_client_config` | `src/tls.rs` | Build a rustls `ClientConfig` with server certificate verification |
|
||||
| `build_server_config_insecure` | `src/tls.rs` | Build a server config that skips client cert verification |
|
||||
| `build_client_config_insecure` | `src/tls.rs` | Build a client config that skips server cert verification |
|
||||
| `load_certs` | `src/tls.rs` | Load PEM-encoded certificates from a file |
|
||||
| `load_key` | `src/tls.rs` | Load a PEM-encoded private key from a file |
|
||||
| `validate_cert_identity` | `src/tls.rs` | Check SANs and CN against an expected hostname |
|
||||
| `check_cert_not_expired` | `src/tls.rs` | Verify certificate validity period |
|
||||
| `cert_fingerprint_sha256` | `src/tls.rs` | Compute SHA-256 fingerprint of a certificate |
|
||||
| `InsecureServerCertVerifier` | `src/tls.rs` | Dangerous verifier that accepts any certificate |
|
||||
|
||||
## How it works
|
||||
|
||||
Server config building:
|
||||
|
||||
1. Load the server certificate chain and private key.
|
||||
2. Load the CA certificate into a `RootCertStore`.
|
||||
3. Build a `WebPkiClientVerifier` from the root store.
|
||||
4. Use `ServerConfig::builder().with_client_cert_verifier(...).with_single_cert(...)`.
|
||||
|
||||
Client config building:
|
||||
|
||||
1. Load the client certificate chain and private key.
|
||||
2. Load the CA certificate into a `RootCertStore`.
|
||||
3. Use `ClientConfig::builder().with_root_certificates(...).with_client_auth_cert(...)`.
|
||||
|
||||
Identity validation uses `x509-parser` to extract Subject Alternative Names and the subject CN, matching against the expected hostname or IP address.
|
||||
|
||||
## Key source files
|
||||
|
||||
| File | Purpose |
|
||||
| ---- | ------- |
|
||||
| `src/tls.rs` | TLS configuration, certificate loading, identity validation |
|
||||
@@ -0,0 +1,69 @@
|
||||
# Tunnel engine
|
||||
|
||||
The tunnel engine in `src/tunnel.rs` is the largest module (~3,500 lines). It implements the listener, connector, SOCKS5 proxy, stream multiplexer, reconnect loop, and HTTP auth handshake.
|
||||
|
||||
## Purpose
|
||||
|
||||
The tunnel engine establishes a persistent, authenticated, mTLS-encrypted tunnel between a listener and a connector, then multiplexes multiple SOCKS5 streams over that single connection.
|
||||
|
||||
## Key abstractions
|
||||
|
||||
| Type | File | Description |
|
||||
| ---- | ---- | ----------- |
|
||||
| `ListenerConfig` | `src/tunnel.rs` | Bind address, optional SOCKS5 address, TLS material, auth token, insecure flag |
|
||||
| `ConnectorConfig` | `src/tunnel.rs` | Target host/port, TLS material, auth token, insecure flag |
|
||||
| `ServerTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for server-side TLS |
|
||||
| `ClientTlsMaterial` | `src/tunnel.rs` | Enum: file paths or embedded PEM strings for client-side TLS |
|
||||
| `StreamMux` | `src/tunnel.rs` | Connector-side multiplexer: manages stream IDs, data channels, and frame dispatch |
|
||||
| `ServerStreamEntry` | `src/tunnel.rs` | Listener-side target write half for an active stream |
|
||||
| `ConnectorStreamEntry` | `src/tunnel.rs` | Connector-side data channel and reply channel for an active stream |
|
||||
|
||||
## How it works
|
||||
|
||||
### Listener side
|
||||
|
||||
1. `run_listener` binds a TCP socket, builds the TLS acceptor from `ServerTlsMaterial`, and starts accepting connections.
|
||||
2. `handle_listener_connection` performs the TLS accept, verifies the client certificate (unless insecure), reads the HTTP POST to `/tunnel`, and validates the `X-Rustunnel-Token` header with `constant_time_compare`.
|
||||
3. On auth success, it calls `handle_framing_server` to switch to binary framing.
|
||||
4. `handle_framing_server` reads frames from the tunnel. A CONNECT frame triggers a TCP connection to the target address (opened from the listener side). DATA frames are forwarded to the target. CLOSE frames shut down the target connection.
|
||||
5. If the listener was started with `--socks`, a server-side SOCKS5 proxy runs via `run_socks5_proxy`, receiving the active `StreamMux` through a `watch::channel`.
|
||||
|
||||
### Connector side
|
||||
|
||||
1. `run_connector_with_socks` builds the client TLS config and starts two concurrent pieces: a reconnect loop and a permanently-bound SOCKS5 proxy.
|
||||
2. `run_reconnect_loop` repeatedly attempts to connect to the listener with exponential backoff (1s to 30s). Each attempt re-runs all security gates: TLS handshake, server cert verification, and auth token exchange.
|
||||
3. On success, the loop creates a `StreamMux`, publishes it via the `watch::channel`, and waits for the framing task to finish (indicating disconnection).
|
||||
4. When the tunnel drops, the loop logs the disconnection and retries.
|
||||
5. `run_socks5_proxy` accepts SOCKS5 connections. For each connection, it checks if the `StreamMux` is available, opens a stream via `mux.open_stream()`, and performs bidirectional forwarding.
|
||||
|
||||
### Stream multiplexing
|
||||
|
||||
`StreamMux::open_stream` allocates the next odd stream ID, registers a data channel and reply channel, sends a CONNECT frame, and waits for a CONNECT_REPLY. The caller receives a `mpsc::Receiver<Bytes>` for tunnel-to-SOCKS5 data.
|
||||
|
||||
`StreamMux::dispatch_frame` routes inbound CONNECT_REPLY, DATA, CLOSE, and ERROR frames to the correct stream by ID.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[SOCKS5 client] -->|CONNECT| B[SOCKS5 proxy]
|
||||
B -->|open_stream| C[StreamMux]
|
||||
C -->|CONNECT frame| D[Framing writer]
|
||||
D -->|TLS| E[Listener framing]
|
||||
E -->|decode target| F[TCP connect to target]
|
||||
F -->|DATA| E
|
||||
E -->|DATA frame| D
|
||||
D -->|dispatch| C
|
||||
C -->|data_rx| B
|
||||
B -->|raw bytes| A
|
||||
```
|
||||
|
||||
## Entry points for modification
|
||||
|
||||
- To change auth behavior: modify `handle_listener_connection` (listener) and `authenticate_tunnel` (connector) in `src/tunnel.rs`.
|
||||
- To change reconnect strategy: modify `run_reconnect_loop` in `src/tunnel.rs`.
|
||||
- To add listener-side SOCKS5 behavior: the server-side SOCKS5 proxy is currently minimal; expand `run_socks5_proxy` in `src/tunnel.rs`.
|
||||
|
||||
## Key source files
|
||||
|
||||
| File | Purpose |
|
||||
| ---- | ------- |
|
||||
| `src/tunnel.rs` | Full tunnel engine: listener, connector, SOCKS5 proxy, reconnect, framing loops, stream mux |
|
||||
Reference in New Issue
Block a user