refactor: streaming, security hardening, and MCP removal
Major overhaul of server architecture and security posture: - Streaming: Unified all I/O through PIPESIZE (8192-byte) buffers. POST bodies stream via MpscReader through the save pipeline. GET content streams from disk via decompression to client. Removed save_item_with_reader, get_item_content_info, ChannelReader. 413 responses keep partial items (nonfatal by design). - Security: XSS protection in all HTML pages via html_escape crate. Security headers middleware (nosniff, frame deny, referrer policy). CORS tightened to explicit headers. Input validation for tags (256 chars), metadata (128/4096), pagination (10k cap). Config file reads use from_utf8_lossy. Generic error messages in HTML. Diff endpoint has 10 MB per-item cap. max_body_size config option. - Panics eliminated: Path unwraps → proper error propagation. Mutex unwraps → map_err (registries) / expect with message (local). - MCP removed: Deleted all MCP code, rmcp dependency, mcp feature. - Docs: Updated README, DESIGN, AGENTS to reflect all changes.
This commit is contained in:
20
AGENTS.md
20
AGENTS.md
@@ -29,11 +29,27 @@ TERM=dumb cargo build --features server # With server feature
|
||||
- Filter plugins must implement `filter()`, `clone_box()`, and `options()`
|
||||
- Meta plugins extend `BaseMetaPlugin` for boilerplate reduction
|
||||
- Enum string representations: `#[strum(serialize_all = "snake_case")]`
|
||||
- Lint rules: `deny(clippy::all)`, `deny(unsafe_code)` (except `libc::umask` in main.rs)
|
||||
- Feature flags: `default = ["magic", "lz4", "gzip"]`; optional: `server`, `mcp`, `swagger`
|
||||
- Lint rules: `deny(clippy::all)`, `deny(unsafe_code)` (except `libc::umask` in main.rs, `unsafe impl Send` in `src/meta_plugin/magic_file.rs` for `SendCookie`)
|
||||
- Feature flags: `default = ["magic", "lz4", "gzip"]`; optional: `server`, `swagger`
|
||||
|
||||
## Testing
|
||||
|
||||
- Tests in `src/tests/` mirroring `src/` structure; shared helpers in `src/tests/common/test_helpers.rs`
|
||||
- Key helpers: `create_temp_dir()`, `create_temp_db()`, `test_compression_engine()`
|
||||
- Test naming: `test_<feature>_<scenario>`
|
||||
|
||||
## Streaming Constraint
|
||||
|
||||
**At no point should the whole file be in memory at once.** All I/O must use fixed-size buffers:
|
||||
|
||||
- `PIPESIZE` = 8192 bytes (`src/common/mod.rs:10`)
|
||||
- Server POST body streams through `save_item_raw_streaming` via `MpscReader`
|
||||
- Server GET content streams via streaming reader (not `read_to_end`)
|
||||
- When `max_body_size` is exceeded, return `413` but keep the partial item (nonfatal by design)
|
||||
- Filter/meta plugins use `PIPESIZE`-sized buffers
|
||||
|
||||
## HTML Rendering
|
||||
|
||||
- Use `html_escape` crate for all user-controlled data in HTML pages
|
||||
- `esc()` for text content, `esc_attr()` for HTML attributes
|
||||
- Security headers middleware: `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: strict-origin-when-cross-origin`
|
||||
|
||||
Reference in New Issue
Block a user