asp/keep

Files

Andrew Phillips 17be6abaab refactor: streaming, security hardening, and MCP removal

Major overhaul of server architecture and security posture:

- Streaming: Unified all I/O through PIPESIZE (8192-byte) buffers.
  POST bodies stream via MpscReader through the save pipeline. GET
  content streams from disk via decompression to client. Removed
  save_item_with_reader, get_item_content_info, ChannelReader.
  413 responses keep partial items (nonfatal by design).

- Security: XSS protection in all HTML pages via html_escape crate.
  Security headers middleware (nosniff, frame deny, referrer policy).
  CORS tightened to explicit headers. Input validation for tags
  (256 chars), metadata (128/4096), pagination (10k cap). Config
  file reads use from_utf8_lossy. Generic error messages in HTML.
  Diff endpoint has 10 MB per-item cap. max_body_size config option.

- Panics eliminated: Path unwraps → proper error propagation.
  Mutex unwraps → map_err (registries) / expect with message (local).

- MCP removed: Deleted all MCP code, rmcp dependency, mcp feature.

- Docs: Updated README, DESIGN, AGENTS to reflect all changes.

2026-03-14 00:03:42 -03:00

2.6 KiB

Raw Blame History

Agent Configuration

IMPORTANT: xxx | keep | zzz must be as performant as possible in all situations.

Build/Test Commands

IMPORTANT: Do not run the application, start the web server, or the trunk server. IMPORTANT: Cargo commands cannot be run in parallel. Prefix all commands with TERM=dumb.

TERM=dumb cargo check              # Fast compile check
TERM=dumb cargo build              # Build project
TERM=dumb cargo test               # Run all tests
TERM=dumb cargo test test_name     # Run specific test by name substring
TERM=dumb cargo test -- --nocapture  # Verbose test output
TERM=dumb cargo fmt --check        # Check formatting
TERM=dumb cargo fmt                # Apply formatting
TERM=dumb cargo clippy -- -D warnings  # Lint (warnings are errors)
TERM=dumb cargo build --release    # Release build
TERM=dumb cargo build --features server  # With server feature

Code Conventions

anyhow::Result for error handling; thiserror for custom error types (src/services/error.rs)
Plugin traits: CompressionEngine, FilterPlugin, MetaPlugin
Dynamic trait objects use clone_box() for Clone on Box<dyn Trait>
Plugin registration uses ctor constructors at module load time
Filter plugins must implement filter(), clone_box(), and options()
Meta plugins extend BaseMetaPlugin for boilerplate reduction
Enum string representations: #[strum(serialize_all = "snake_case")]
Lint rules: deny(clippy::all), deny(unsafe_code) (except libc::umask in main.rs, unsafe impl Send in src/meta_plugin/magic_file.rs for SendCookie)
Feature flags: default = ["magic", "lz4", "gzip"]; optional: server, swagger

Testing

Tests in src/tests/ mirroring src/ structure; shared helpers in src/tests/common/test_helpers.rs
Key helpers: create_temp_dir(), create_temp_db(), test_compression_engine()
Test naming: test_<feature>_<scenario>

Streaming Constraint

At no point should the whole file be in memory at once. All I/O must use fixed-size buffers:

PIPESIZE = 8192 bytes (src/common/mod.rs:10)
Server POST body streams through save_item_raw_streaming via MpscReader
Server GET content streams via streaming reader (not read_to_end)
When max_body_size is exceeded, return 413 but keep the partial item (nonfatal by design)
Filter/meta plugins use PIPESIZE-sized buffers

HTML Rendering

Use html_escape crate for all user-controlled data in HTML pages
esc() for text content, esc_attr() for HTML attributes
Security headers middleware: X-Content-Type-Options: nosniff, X-Frame-Options: DENY, Referrer-Policy: strict-origin-when-cross-origin

2.6 KiB Raw Blame History