docs: add comprehensive project wiki for v1.0
CI / cargo fmt (push) Has been cancelled
CI / cargo clippy (macos-latest) (push) Has been cancelled
CI / cargo clippy (ubuntu-latest) (push) Has been cancelled
CI / cargo clippy (windows-latest) (push) Has been cancelled
CI / cargo test (macos-latest) (push) Has been cancelled
CI / cargo test (ubuntu-latest) (push) Has been cancelled
CI / cargo test (windows-latest) (push) Has been cancelled
CI / cargo build (macos-latest) (push) Has been cancelled
CI / cargo build (ubuntu-latest) (push) Has been cancelled
CI / cargo build (windows-latest) (push) Has been cancelled
CI / cargo build --release (macos-latest) (push) Has been cancelled
CI / cargo build --release (ubuntu-latest) (push) Has been cancelled
CI / cargo build --release (windows-latest) (push) Has been cancelled
CI / CLI smoke (macos-latest) (push) Has been cancelled
CI / CLI smoke (ubuntu-latest) (push) Has been cancelled
CI / CLI smoke (windows-latest) (push) Has been cancelled
CI / Minimal E2E (macos-latest) (push) Has been cancelled
CI / Minimal E2E (ubuntu-latest) (push) Has been cancelled
CI / Minimal E2E (windows-latest) (push) Has been cancelled
CI / cargo fmt (push) Has been cancelled
CI / cargo clippy (macos-latest) (push) Has been cancelled
CI / cargo clippy (ubuntu-latest) (push) Has been cancelled
CI / cargo clippy (windows-latest) (push) Has been cancelled
CI / cargo test (macos-latest) (push) Has been cancelled
CI / cargo test (ubuntu-latest) (push) Has been cancelled
CI / cargo test (windows-latest) (push) Has been cancelled
CI / cargo build (macos-latest) (push) Has been cancelled
CI / cargo build (ubuntu-latest) (push) Has been cancelled
CI / cargo build (windows-latest) (push) Has been cancelled
CI / cargo build --release (macos-latest) (push) Has been cancelled
CI / cargo build --release (ubuntu-latest) (push) Has been cancelled
CI / cargo build --release (windows-latest) (push) Has been cancelled
CI / CLI smoke (macos-latest) (push) Has been cancelled
CI / CLI smoke (ubuntu-latest) (push) Has been cancelled
CI / CLI smoke (windows-latest) (push) Has been cancelled
CI / Minimal E2E (macos-latest) (push) Has been cancelled
CI / Minimal E2E (ubuntu-latest) (push) Has been cancelled
CI / Minimal E2E (windows-latest) (push) Has been cancelled
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent
a758e8e0bc
commit
0166fb6511
@@ -0,0 +1,53 @@
|
||||
# Debugging
|
||||
|
||||
## Logs
|
||||
|
||||
rustunnel uses `tracing` with an `EnvFilter` defaulting to `info`. Set `RUST_LOG=debug` or `RUST_LOG=trace` for more detail.
|
||||
|
||||
```sh
|
||||
RUST_LOG=debug ./target/release/rustunnel listen --listen 127.0.0.1:4180 ...
|
||||
```
|
||||
|
||||
All logged secrets are redacted. The `Redacted` wrapper shows `[REDACTED(len=N)]` instead of the raw value. If you see raw secrets in logs, that is a bug.
|
||||
|
||||
## Common errors
|
||||
|
||||
### TLS handshake failures
|
||||
|
||||
If the connector fails with a TLS error, check:
|
||||
|
||||
- Do the listener and connector share the same CA certificate?
|
||||
- Does the server certificate's SAN match the target hostname? Use `generate` with the correct `--advertise` address, or pass `--insecure-skip-tls-verify` (lab only).
|
||||
- Are the certificate files readable and valid PEM?
|
||||
|
||||
rustunnel prints TLS hint messages for common failures:
|
||||
|
||||
- `BadSignature` — the listener certificate is not signed by the CA trusted by the connector.
|
||||
- `NotValidForName` — the listener certificate does not match the target host.
|
||||
|
||||
### Auth failures
|
||||
|
||||
- Verify the token file is not empty and has no trailing whitespace.
|
||||
- The listener and connector must use the same token value.
|
||||
- Auth failures are terminal for the connector's reconnect loop (it will not retry with a bad token).
|
||||
|
||||
### Port conflicts
|
||||
|
||||
If binding fails with "address already in use":
|
||||
|
||||
- Check for orphaned processes from previous test runs.
|
||||
- The E2E tests use random free ports; manual testing may need explicit port management.
|
||||
|
||||
### SOCKS5 connection refused
|
||||
|
||||
If curl fails through the SOCKS5 proxy:
|
||||
|
||||
- Is the connector running and the tunnel established?
|
||||
- Is the SOCKS5 proxy port correct?
|
||||
- Is the target address reachable from the listener side (for default mode) or connector side (when using `--socks` on the listener)?
|
||||
|
||||
## Tracing
|
||||
|
||||
The connector logs effective config at startup, including the target, SOCKS5 address, and redacted auth token. The listener logs its bind address, TLS fingerprints, and auth token (redacted).
|
||||
|
||||
State transitions are logged at `info` level: "Tunnel session established", "Tunnel disconnected", "Reconnecting in Ns".
|
||||
@@ -0,0 +1,25 @@
|
||||
# Development workflow
|
||||
|
||||
## Branching
|
||||
|
||||
Use feature branches off `master`. The project is small enough that long-lived branches are unnecessary.
|
||||
|
||||
## Coding
|
||||
|
||||
- Follow existing formatting (enforced by `cargo fmt`).
|
||||
- Use `anyhow` for application-level errors and `thiserror` for library-style error enums.
|
||||
- Prefer `tracing` over `println!` for diagnostic output. Secrets must be wrapped with `Redacted` before logging.
|
||||
- Use `Arc<String>` for shared strings across async boundaries (e.g., auth tokens in `ListenerConfig` and `ConnectorConfig`).
|
||||
- Keep synchronous protocol parsing in dedicated modules (`socks5.rs`, `framing.rs`) and async I/O in `tunnel.rs`.
|
||||
|
||||
## Testing
|
||||
|
||||
- Inline tests live in `#[cfg(test)]` modules at the bottom of each source file.
|
||||
- Use `tempfile::tempdir()` for generating test credentials. Clean up temp directories with `std::fs::remove_dir_all`.
|
||||
- Use `std::net::TcpListener::bind("127.0.0.1:0")` to get free ports for E2E tests.
|
||||
- E2E tests spawn tokio tasks for listener, connector, and target servers, then abort them in cleanup.
|
||||
|
||||
## Committing
|
||||
|
||||
- Follow conventional commit style: `feat:`, `fix:`, `docs:`, `chore:`.
|
||||
- Keep commits focused. The git history shows the evolution from HTTP-per-stream to multiplexed tunnel; preserve that narrative clarity.
|
||||
@@ -0,0 +1,29 @@
|
||||
# How to contribute
|
||||
|
||||
rustunnel is a small, focused project. Contributions should align with the project's purpose: a lab and development tunneling tool with strong security defaults.
|
||||
|
||||
## Work pickup
|
||||
|
||||
- Check the CI status before starting work.
|
||||
- Open issues or PRs against the `master` branch.
|
||||
- Keep changes focused. The codebase is small enough that large refactors should be discussed first.
|
||||
|
||||
## PR process
|
||||
|
||||
1. Run the full test suite locally: `cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test --all-targets`
|
||||
2. Ensure E2E tests pass if your change touches the tunnel or SOCKS5 path.
|
||||
3. Write tests for new behavior. The project uses inline `#[cfg(test)]` modules.
|
||||
4. Update relevant wiki pages if your change adds or removes features.
|
||||
|
||||
## Review expectations
|
||||
|
||||
- Security-sensitive changes (TLS, auth, framing) get extra scrutiny.
|
||||
- All CI jobs must pass: fmt, clippy, test, build, smoke, and E2E.
|
||||
- The project runs CI on Linux, macOS, and Windows. Avoid platform-specific code unless necessary (the signal handling module is the only current exception).
|
||||
|
||||
## Definition of done
|
||||
|
||||
- Code is formatted with `rustfmt`.
|
||||
- Clippy warnings are resolved.
|
||||
- Tests cover the new behavior, including edge cases.
|
||||
- No `TODO` or `FIXME` comments are introduced (the project currently has zero).
|
||||
@@ -0,0 +1,55 @@
|
||||
# Patterns and conventions
|
||||
|
||||
## Error handling
|
||||
|
||||
The project uses a layered error strategy:
|
||||
|
||||
- `thiserror` enums for domain-specific errors (`TlsError`, `AuthError`, `TunnelError`, `HostError`, `Socks5Error`, `FrameError`). These live in `src/errors.rs` and `src/socks5.rs` / `src/framing.rs`.
|
||||
- `anyhow` for application-level propagation. CLI commands return `anyhow::Result` and `main` exits with `process::exit(exit_code)`.
|
||||
- All errors are designed to be actionable and fail-closed. There are no silent degradation paths to insecure modes.
|
||||
|
||||
## Secret handling
|
||||
|
||||
- The `Redacted` wrapper in `src/redact.rs` must be used before logging any secret.
|
||||
- `Redacted::inner()` and `into_inner()` are marked `#[allow(dead_code)]` to discourage use, but available for file writes.
|
||||
- `is_sensitive()` detects PEM private key blocks. `is_auth_token()` detects long alphanumeric strings.
|
||||
- `is_connection_key()` detects strings starting with `rtun1.`.
|
||||
|
||||
## Async patterns
|
||||
|
||||
- Tokio's multi-threaded runtime is used on both listener and connector sides.
|
||||
- `tokio::spawn` is used for per-connection and per-stream tasks.
|
||||
- `tokio::select!` with `biased;` prioritizes shutdown signals over new connections.
|
||||
- `watch::channel` is used for publishing the active `StreamMux` to the SOCKS5 proxy (supports reconnect).
|
||||
- `mpsc::channel` and `oneshot::channel` are used for stream data and CONNECT_REPLY coordination.
|
||||
|
||||
## Constant-time comparison
|
||||
|
||||
Auth token comparison in `src/tunnel.rs` uses a manual constant-time XOR loop to prevent timing attacks:
|
||||
|
||||
```rust
|
||||
fn constant_time_compare(a: &str, b: &str) -> bool {
|
||||
let a_bytes = a.as_bytes();
|
||||
let b_bytes = b.as_bytes();
|
||||
if a_bytes.len() != b_bytes.len() {
|
||||
return false;
|
||||
}
|
||||
let mut result: u8 = 0;
|
||||
for (x, y) in a_bytes.iter().zip(b_bytes.iter()) {
|
||||
result |= x ^ y;
|
||||
}
|
||||
result == 0
|
||||
}
|
||||
```
|
||||
|
||||
## Frame reader partial read discipline
|
||||
|
||||
`FrameReader` in `src/framing.rs` never consumes bytes from the buffer until a complete frame (header + payload) is available. This prevents corruption when TCP segments arrive in arbitrary chunks.
|
||||
|
||||
## TLS material as enum
|
||||
|
||||
Both `ServerTlsMaterial` and `ClientTlsMaterial` are enums with `Paths` and `Pem` variants. This lets the listener and connector accept either file paths (from CLI args) or embedded PEM strings (from connection keys).
|
||||
|
||||
## Insecure mode
|
||||
|
||||
`--insecure-skip-tls-verify` exists for DPI/intercepted environments only. When enabled, the listener skips client certificate verification and the connector skips server certificate verification. This is logged at `warn` level.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Testing
|
||||
|
||||
## Test organization
|
||||
|
||||
All tests are inline `#[cfg(test)]` modules at the bottom of each source file. There are no standalone test files or integration test directories.
|
||||
|
||||
## Running tests
|
||||
|
||||
```sh
|
||||
cargo test --all-targets
|
||||
```
|
||||
|
||||
This runs unit tests and doc tests across all modules.
|
||||
|
||||
## Test patterns
|
||||
|
||||
### Credential generation tests
|
||||
|
||||
Tests in `src/generate.rs` verify that `generate()` creates all expected files, that certificates contain valid PEM blocks, that private keys are detected as sensitive by the redaction module, and that config JSON is valid.
|
||||
|
||||
### Framing tests
|
||||
|
||||
Tests in `src/framing.rs` cover:
|
||||
|
||||
- Frame roundtrips for all frame types (CONNECT, CONNECT_REPLY, DATA, CLOSE, ERROR)
|
||||
- Partial read handling — buffers must not be consumed until a complete frame is available
|
||||
- Multiple frames in one buffer
|
||||
- Target address encoding/decoding for IPv4 and domain names
|
||||
|
||||
### SOCKS5 tests
|
||||
|
||||
Tests in `src/socks5.rs` cover:
|
||||
|
||||
- Greeting parsing (no-auth, username/password, invalid versions)
|
||||
- Auth request parsing (valid and empty credentials)
|
||||
- CONNECT request parsing (IPv4, domain, unsupported commands)
|
||||
- Reply building
|
||||
- Target resolution (IPv4 direct, domain DNS lookup)
|
||||
- Malformed input handling
|
||||
|
||||
### TLS tests
|
||||
|
||||
Tests in `src/tls.rs` cover:
|
||||
|
||||
- Loading certificates and keys from files and PEM strings
|
||||
- Building valid server and client configs
|
||||
- Identity validation against SANs and CN
|
||||
- Certificate fingerprinting
|
||||
- Expiration checking
|
||||
- Insecure config variants
|
||||
|
||||
### Tunnel tests
|
||||
|
||||
Tests in `src/tunnel.rs` are the most extensive. They include:
|
||||
|
||||
- **Auth tests:** valid mTLS + auth, invalid token, missing client cert
|
||||
- **E2E tests:** basic forwarding, server-side SOCKS5 reaching connector-side targets, concurrent stream isolation, target failure handling, listener-side target origin verification
|
||||
- **Operational tests:** reconnect restoring traffic after tunnel drop, graceful shutdown of listener and connector, shutdown during active streams
|
||||
- **Log tests:** verify effective config is logged with redacted tokens, verify auth failures don't leak token values in error messages
|
||||
|
||||
## E2E test structure
|
||||
|
||||
Typical E2E tests follow this pattern:
|
||||
|
||||
1. Generate test credentials with `generate_test_creds()`.
|
||||
2. Pick a free port with `get_free_port()`.
|
||||
3. Spawn the listener as a tokio task.
|
||||
4. Spawn the connector (and optionally a target server) as tokio tasks.
|
||||
5. Sleep briefly to let services start.
|
||||
6. Connect via SOCKS5, send data, verify response.
|
||||
7. Abort all tasks in cleanup.
|
||||
|
||||
## CI test matrix
|
||||
|
||||
Tests run on `ubuntu-latest`, `windows-latest`, and `macos-latest`.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Tooling
|
||||
|
||||
## Build system
|
||||
|
||||
Cargo is the sole build tool. No custom build scripts or Makefiles are used.
|
||||
|
||||
- `cargo build` — debug build
|
||||
- `cargo build --release` — optimized release build
|
||||
- `cargo check` — fast type-checking
|
||||
|
||||
## Linting and formatting
|
||||
|
||||
- `cargo fmt --check` — enforces rustfmt formatting
|
||||
- `cargo clippy --all-targets -- -D warnings` — runs clippy on all targets with warnings as errors
|
||||
|
||||
## CI/CD
|
||||
|
||||
GitHub Actions runs the full matrix on every push and PR:
|
||||
|
||||
| Job | Platforms | Purpose |
|
||||
| --- | --------- | ------- |
|
||||
| `fmt` | Ubuntu | Format check |
|
||||
| `clippy` | Linux, macOS, Windows | Lint check |
|
||||
| `test` | Linux, macOS, Windows | Unit and inline tests |
|
||||
| `build` | Linux, macOS, Windows | Debug build |
|
||||
| `release-build` | Linux, macOS, Windows | Release build + artifact upload |
|
||||
| `smoke` | Linux, macOS, Windows | CLI smoke tests (--help, version, invalid command/port) |
|
||||
| `e2e-minimal` | Linux, macOS, Windows | Full end-to-end flow: generate, listen, connect, SOCKS5, curl |
|
||||
|
||||
The E2E job uses `cargo run --release --` for OS-neutral binary invocation and includes cross-platform cleanup helpers (`kill_process`, `kill_by_port`) invoked via `trap cleanup EXIT`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Dependencies are managed in `Cargo.toml`. Notable choices:
|
||||
|
||||
- `tokio` with `full` features for async runtime
|
||||
- `rustls` with `ring` for TLS (no OpenSSL)
|
||||
- `rcgen` for certificate generation
|
||||
- `clap` with `derive` for CLI parsing
|
||||
- `tracing` and `tracing-subscriber` for structured logging
|
||||
Reference in New Issue
Block a user