- Add streaming tar-based export (--export produces .keep.tar)
- Add streaming tar import (--import reads .keep.tar archives)
- Add server endpoints GET /api/export and POST /api/import
- Rename CompressionType::None to CompressionType::Raw with "none" as alias
- Add DB migration to update existing "none" compression values to "raw"
- Fix export endpoint to propagate errors to client instead of swallowing
- Fix import endpoint to return 413 on max_body_size instead of truncating
Export streams items as tar archives without loading entire files into memory.
Import extracts items with new IDs, preserving original order. Both work
locally and via client/server mode.
Co-Authored-By: opencode <noreply@opencode.ai>
- Add infer crate as meta plugin for MIME type detection
- Add tree_magic_mini crate as alternative meta plugin for MIME type detection
- Add zstd, infer, tree_magic_mini to default features
- Fix static build script to use musl target instead of glibc+crt-static
- Remove hardcoded shell list from --generate-completion help text
- Fix update() in both new plugins to emit MIME metadata when buffer fills
Export/import:
- Add --export and --import modes for both local and client paths
- Use strfmt crate for --export-filename-format templates ({id}, {tags}, {ts}, {compression})
- Import preserves original timestamps via server ?ts= param
- --import-data-file for file-based import; stdin fallback streams with PIPESIZE buffers
Service unification:
- Merge SyncDataService unique methods into ItemService (delete_item now returns Result<Item>)
- Delete AsyncDataService, AsyncItemService, DataService trait (dead code / async-blocking anti-pattern)
- All server handlers use spawn_blocking + ItemService directly
- Extract shared types (ExportMeta, ImportMeta) and helpers (resolve_item_id(s), check_binary_tty)
Binary detection fix:
- Replace broken metadata.get("map") + is_binary(&[]) with actual content sampling
- Both as_meta and allow_binary paths read PIPESIZE sample before deciding
- Never load entire item into memory for binary check
Other fixes:
- Fix lock consistency: all handlers use blocking_lock() in spawn_blocking (no mixed lock().await)
- Use ISO 8601 format for {ts} in export filenames
- Fix resolve_item_ids returning only 1 item for tag lookups
- Fix client get.rs triple-buffering and export.rs whole-file buffering
- Add KeepClient::get_item_content_stream() for streaming reads
- Pass all clippy --features server lints (Path vs PathBuf, &mut conn, etc.)
Replace 4 unsafe sites with safe wrappers:
- libc::pipe2 → nix::unistd::pipe2 (safe OwnedFd return)
- File::from_raw_fd → File::from(OwnedFd) (safe ownership transfer)
- unsafe impl Send for SendCookie → thread_local! lazy Cookie
(each thread gets its own independent Cookie, no Send needed)
- pre_exec + libc::fcntl → command-fds crate fd_mappings()
(handles CLOEXEC clearing safely, also fixes potential fd leak
on spawn failure via OwnedFd RAII)
Only libc::umask remains as a single unavoidable unsafe site
(no safe Rust wrapper exists for the umask syscall).
Also updates AGENTS.md to remove stale SendCookie exception.
- Add clap_complete dependency for bash/zsh/fish/elvish/powershell
- Add --generate-completion <shell> flag that prints completion script to stdout
- profile.bash sources completions via command keep --generate-completion bash
- @ and @@ aliases get completions via wrapper functions that delegate to _keep
- README updated with Shell Completion section
Major overhaul of server architecture and security posture:
- Streaming: Unified all I/O through PIPESIZE (8192-byte) buffers.
POST bodies stream via MpscReader through the save pipeline. GET
content streams from disk via decompression to client. Removed
save_item_with_reader, get_item_content_info, ChannelReader.
413 responses keep partial items (nonfatal by design).
- Security: XSS protection in all HTML pages via html_escape crate.
Security headers middleware (nosniff, frame deny, referrer policy).
CORS tightened to explicit headers. Input validation for tags
(256 chars), metadata (128/4096), pagination (10k cap). Config
file reads use from_utf8_lossy. Generic error messages in HTML.
Diff endpoint has 10 MB per-item cap. max_body_size config option.
- Panics eliminated: Path unwraps → proper error propagation.
Mutex unwraps → map_err (registries) / expect with message (local).
- MCP removed: Deleted all MCP code, rmcp dependency, mcp feature.
- Docs: Updated README, DESIGN, AGENTS to reflect all changes.
Add tiktoken-based token counting via new 'tokens' feature flag.
New components:
- Shared tokenizer module wrapping tiktoken CoreBPE (cl100k_base, o200k_base)
- TokensMetaPlugin: streaming token counter, tokenizes each chunk independently
- head_tokens(N): stream first N tokens, split at exact boundary when mid-chunk
- skip_tokens(N): skip first N tokens, stream the rest
- tail_tokens(N): bounded ring buffer (~16KB), outputs last N tokens at finalize
All filters are fully streaming — no full-stream buffering.
Meta plugin accuracy: exact for normal text, ±1-2 tokens if long whitespace
sequence spans a chunk boundary.
Also: add 'client' and 'tokens' to default features, add curl to Dockerfile builder stage.
Add server-side JWT authentication with permission-based access control
(read/write/delete claims). Password authentication now uses HTTP Basic
auth only (replacing Bearer). Add configurable username for both server
and client (--server-username/--client-username, defaults to "keep").
JWT secret supports file-based loading via --server-jwt-secret-file for
Docker secrets. OPTIONS preflight requests bypass auth. HEAD mapped to
read permission.
Co-Authored-By: opencode <noreply@opencode.ai>
Security:
- Use constant-time password comparison (subtle crate) to prevent timing attacks
- Replace permissive CORS with configurable origin-restricted CORS
- Add TLS warning when password auth is used without HTTPS
Bug fixes:
- Convert MetaPlugin panics to anyhow::Result (get_meta_plugin, outputs_mut, options_mut)
- Replace item.id.unwrap() with proper error handling across 15 call sites
- Fix panic on unknown column type in list mode
- Fix conflicting PIPESIZE constant (was 8192 vs 65536, now unified to 8192)
- Add 256MB filter chain buffer limit to prevent OOM
- Gracefully skip unregistered plugins instead of panicking
Dead code removal:
- Delete unused filter parser files (filter_parser.rs, filter.pest, parser/ module)
- ~260 lines of dead PEG parser code removed
Code consolidation:
- Add is_content_binary_from_metadata() helper (was duplicated in 4 places)
- Simplify save_item_raw() to delegate to save_item_raw_streaming() (~90 lines removed)
Incomplete features:
- Populate filter_plugins in status output from global registry
- Add FallbackMagicFileMetaPlugin (was referenced but never implemented)
- Document init_plugins() as intentional no-op
Infrastructure:
- Add Dockerfile (static musl binary on scratch, 4.8MB)
- Add .dockerignore
- Add cors_origin to ServerConfig and config.rs
Add optional TLS support for the server using axum-server with the
tls-rustls feature. When --server-cert and --server-key are provided
(and tls feature is enabled), the server binds with TLS instead of
plain HTTP.
Changes:
- Add axum-server dependency with optional tls-rustls feature
- New 'tls' feature flag (independent of 'server')
- --server-cert/--server-key CLI args gated behind tls feature
- ServerConfig extended with cert_file/key_file fields
- Conditional TLS/HTTP binding in server mod.rs
- Fix PathBuf::to_str().unwrap() panic risk -> to_string_lossy()
- Update README.md and DESIGN.md with TLS documentation
Add client mode enabling the keep CLI to connect to a remote keep
server over HTTP. Local plugins (compression, meta, filters) run on
the client; the server stores/retrieves binary blobs.
Architecture:
- Client save uses 3-thread streaming pipeline: reader thread (stdin
→ tee/stdout → hash → compress), OS pipe, streamer thread (pipe →
chunked HTTP POST). Memory usage is O(PIPESIZE) regardless of data
size.
- Server accepts compress=false, meta=false, decompress=false query
params for granular control of server-side processing.
- Streaming body handling on server via async channel → sync reader
bridge (ChannelReader).
Key additions:
- src/client.rs: KeepClient with post_stream() for chunked upload
- src/modes/client/: save, get, list, info, delete, diff, status
- --client-url / KEEP_CLIENT_URL configuration
- --client-password / KEEP_CLIENT_PASSWORD for auth
- os_pipe dependency for zero-copy pipe streaming
Co-Authored-By: andrew/openrouter/hunter-alpha <noreply@opencode.ai>
