Commit Graph

45 Commits

Author SHA1 Message Date
12de215527 feat: feature-gate CLI args by server/client features
- CLI now shows only relevant options: --server and --server-* args
  hidden when built without 'server' feature; --client-* args hidden
  without 'client' feature. Run --help only displays applicable options.
- Removed verbose 'conflicts_with_all' from all mode args — clap's
  implicit group("mode") already enforces mutual exclusivity.
- 'server' feature now includes TLS/HTTPS by default (axum-server);
  'tls' feature removed. rustls already available via client/ureq.
- Gated KeepModes::Server, server mode detection, and server-password
  validation in main.rs.
- Gated server arg reads in config.rs.
- Removed redundant #[cfg(feature = "tls")] guards from server/mod.rs.
- Gated resolve_item_id/resolve_item_ids helpers in common.rs.
- All 4 feature combinations (server+client, server-only, client-only,
  neither) compile and pass tests.
2026-03-21 16:26:27 -03:00
b3edfe7de6 chore: code review cleanup — fixes, deps, docs
Fixed:
- CLI help typo: "metatdata" -> "metadata"
- Filter buffer OOM: check size before loading into memory

Changed:
- #[inline] on HTML escape helpers for hot path performance
- Replaced once_cell and lazy_static with std::sync::LazyLock
- Removed unused once_cell and lazy_static crate dependencies

Refactored:
- Added module-level doc to services/ module

Documentation:
- README.md: zstd is native not external, "none" -> "raw"
- DESIGN.md: current schema and meta plugins section
- CHANGELOG.md: Unreleased section populated
2026-03-21 11:44:37 -03:00
52e9787edb refactor: deduplicate filter plugins, extract helpers across codebase
Bug fixes:
- client: add error field to ApiResponse to avoid swallowing server errors
- args/config: fix list_format default mismatch (5 vs 7 columns)
- client: url-encode size param in set_item_size

Dedup - filter plugins:
- Extract count_option() and pattern_option() helpers, replace 7 identical options()
- Add #[derive(Clone)] to all filter structs; remove verbose clone_box() impls
- Simplify FilterChain clone() and impl Clone for Box<dyn FilterPlugin>
- Add filter_clone_box! macro for future use
- Fix doctest example missing clone_box

Dedup - server API:
- Extract spawn_body_reader() with LimitBehavior enum for body streaming
- Extract check_binary_content() helper
- Extract stream_with_offset_and_length() helper
- Extract generate_status() helper in status.rs
- Extract append_query_params() helper in client.rs

Dedup - other:
- Extract yaml_value_to_string() in meta_plugin/mod.rs
- Extract item_from_row() in db.rs
- Delete unused DisplayListItem struct

Misc:
- Remove duplicate doc comment in compression_service.rs
2026-03-20 15:54:33 -03:00
49793a0f94 feat: add streaming tar export/import and rename "none" to "raw"
- Add streaming tar-based export (--export produces .keep.tar)
- Add streaming tar import (--import reads .keep.tar archives)
- Add server endpoints GET /api/export and POST /api/import
- Rename CompressionType::None to CompressionType::Raw with "none" as alias
- Add DB migration to update existing "none" compression values to "raw"
- Fix export endpoint to propagate errors to client instead of swallowing
- Fix import endpoint to return 413 on max_body_size instead of truncating

Export streams items as tar archives without loading entire files into memory.
Import extracts items with new IDs, preserving original order. Both work
locally and via client/server mode.

Co-Authored-By: opencode <noreply@opencode.ai>
2026-03-17 21:24:39 -03:00
cb56a398fa feat: add --ids-only flag to --list mode for scripting
Outputs one ID per line with no header. Errors if used with any mode
other than --list. Works with both local and client (remote) list.
2026-03-17 15:04:10 -03:00
a8759c4b83 feat: add infer and tree_magic_mini meta plugins, make zstd internal by default
- Add infer crate as meta plugin for MIME type detection
- Add tree_magic_mini crate as alternative meta plugin for MIME type detection
- Add zstd, infer, tree_magic_mini to default features
- Fix static build script to use musl target instead of glibc+crt-static
- Remove hardcoded shell list from --generate-completion help text
- Fix update() in both new plugins to emit MIME metadata when buffer fills
2026-03-17 14:46:51 -03:00
35ee71c3cf feat: add export/import modes, unify service layer, fix binary detection
Export/import:
- Add --export and --import modes for both local and client paths
- Use strfmt crate for --export-filename-format templates ({id}, {tags}, {ts}, {compression})
- Import preserves original timestamps via server ?ts= param
- --import-data-file for file-based import; stdin fallback streams with PIPESIZE buffers

Service unification:
- Merge SyncDataService unique methods into ItemService (delete_item now returns Result<Item>)
- Delete AsyncDataService, AsyncItemService, DataService trait (dead code / async-blocking anti-pattern)
- All server handlers use spawn_blocking + ItemService directly
- Extract shared types (ExportMeta, ImportMeta) and helpers (resolve_item_id(s), check_binary_tty)

Binary detection fix:
- Replace broken metadata.get("map") + is_binary(&[]) with actual content sampling
- Both as_meta and allow_binary paths read PIPESIZE sample before deciding
- Never load entire item into memory for binary check

Other fixes:
- Fix lock consistency: all handlers use blocking_lock() in spawn_blocking (no mixed lock().await)
- Use ISO 8601 format for {ts} in export filenames
- Fix resolve_item_ids returning only 1 item for tag lookups
- Fix client get.rs triple-buffering and export.rs whole-file buffering
- Add KeepClient::get_item_content_stream() for streaming reads
- Pass all clippy --features server lints (Path vs PathBuf, &mut conn, etc.)
2026-03-16 08:43:26 -03:00
0658d8378f fix: group all server options under Server Options help heading
The --server-password, --server-password-hash, --server-username,
--server-jwt-secret, --server-jwt-secret-file, and --server-max-body-size
options were appearing in the generic Options section instead of the
Server Options section.
2026-03-14 18:56:32 -03:00
b3ca673b52 feat: add --update mode, --meta/--meta-plugin flags, streaming diff
- Add --update mode to modify tags and metadata for existing items by ID
- Add --meta key=value flag to set metadata during save/update
- Add --meta key (bare) to delete metadata keys or filter by existence
- Add --meta-plugin/-M name:{json} flag for plugin options via CLI
- Env meta plugin now uses options from --meta-plugin instead of only env vars
- Stream decompressed content to diff via /dev/fd pipes (no temp files)
- Wire --list-format CLI arg to settings (was parsed but ignored)
- Allow --info to accept tags (was restricted to numeric IDs only)
- Change DB meta filtering to HashMap<String, Option<String>> for exact match + key existence
- Fix fcntl error checking in diff pre_exec
- Fix README inaccuracies (delete by tag, nonexistent --digest flag, meta plugin key names)
2026-03-14 15:02:16 -03:00
1a8ed56b68 feat: add --generate-completion for shell tab completion
- Add clap_complete dependency for bash/zsh/fish/elvish/powershell
- Add --generate-completion <shell> flag that prints completion script to stdout
- profile.bash sources completions via command keep --generate-completion bash
- @ and @@ aliases get completions via wrapper functions that delegate to _keep
- README updated with Shell Completion section
2026-03-14 11:02:38 -03:00
17be6abaab refactor: streaming, security hardening, and MCP removal
Major overhaul of server architecture and security posture:

- Streaming: Unified all I/O through PIPESIZE (8192-byte) buffers.
  POST bodies stream via MpscReader through the save pipeline. GET
  content streams from disk via decompression to client. Removed
  save_item_with_reader, get_item_content_info, ChannelReader.
  413 responses keep partial items (nonfatal by design).

- Security: XSS protection in all HTML pages via html_escape crate.
  Security headers middleware (nosniff, frame deny, referrer policy).
  CORS tightened to explicit headers. Input validation for tags
  (256 chars), metadata (128/4096), pagination (10k cap). Config
  file reads use from_utf8_lossy. Generic error messages in HTML.
  Diff endpoint has 10 MB per-item cap. max_body_size config option.

- Panics eliminated: Path unwraps → proper error propagation.
  Mutex unwraps → map_err (registries) / expect with message (local).

- MCP removed: Deleted all MCP code, rmcp dependency, mcp feature.

- Docs: Updated README, DESIGN, AGENTS to reflect all changes.
2026-03-14 00:03:42 -03:00
e672ec751e feat: add JWT auth, configurable username, switch password auth to Basic
Add server-side JWT authentication with permission-based access control
(read/write/delete claims). Password authentication now uses HTTP Basic
auth only (replacing Bearer). Add configurable username for both server
and client (--server-username/--client-username, defaults to "keep").

JWT secret supports file-based loading via --server-jwt-secret-file for
Docker secrets. OPTIONS preflight requests bypass auth. HEAD mapped to
read permission.

Co-Authored-By: opencode <noreply@opencode.ai>
2026-03-13 13:56:35 -03:00
bee980605f feat: add HTTPS/TLS server support via rustls
Add optional TLS support for the server using axum-server with the
tls-rustls feature. When --server-cert and --server-key are provided
(and tls feature is enabled), the server binds with TLS instead of
plain HTTP.

Changes:
- Add axum-server dependency with optional tls-rustls feature
- New 'tls' feature flag (independent of 'server')
- --server-cert/--server-key CLI args gated behind tls feature
- ServerConfig extended with cert_file/key_file fields
- Conditional TLS/HTTP binding in server mod.rs
- Fix PathBuf::to_str().unwrap() panic risk -> to_string_lossy()
- Update README.md and DESIGN.md with TLS documentation
2026-03-12 22:18:42 -03:00
c5529bedbf feat: add client mode with streaming support
Add client mode enabling the keep CLI to connect to a remote keep
server over HTTP. Local plugins (compression, meta, filters) run on
the client; the server stores/retrieves binary blobs.

Architecture:
- Client save uses 3-thread streaming pipeline: reader thread (stdin
  → tee/stdout → hash → compress), OS pipe, streamer thread (pipe →
  chunked HTTP POST). Memory usage is O(PIPESIZE) regardless of data
  size.
- Server accepts compress=false, meta=false, decompress=false query
  params for granular control of server-side processing.
- Streaming body handling on server via async channel → sync reader
  bridge (ChannelReader).

Key additions:
- src/client.rs: KeepClient with post_stream() for chunked upload
- src/modes/client/: save, get, list, info, delete, diff, status
- --client-url / KEEP_CLIENT_URL configuration
- --client-password / KEEP_CLIENT_PASSWORD for auth
- os_pipe dependency for zero-copy pipe streaming

Co-Authored-By: andrew/openrouter/hunter-alpha <noreply@opencode.ai>
2026-03-12 18:01:36 -03:00
Andrew Phillips
fdeb5f7951 Ugh 2026-02-19 13:57:39 -04:00
Andrew Phillips
22cd07284b fix: Resolve compilation errors for multiple definitions and parser issues
Co-authored-by: aider (openai/andrew/openrouter/sonoma-sky-alpha) <aider@aider.chat>
2025-09-10 16:12:11 -03:00
Andrew Phillips
58b5c8187b docs: Add Rustdoc for modules, functions, and structs
Co-authored-by: aider (openai/andrew/openrouter/sonoma-sky-alpha) <aider@aider.chat>
2025-09-10 12:28:47 -03:00
Andrew Phillips
e14c85a5af feat: Split out --status-plugins to show only plugin information
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-09-03 08:45:37 -03:00
Andrew Phillips
5afe2f6bc8 feat: Add --filters option to --get and parse filters early
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-09-03 07:42:28 -03:00
Andrew Phillips
4472f3db94 feat: add argument validation for delete mode
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-08-29 14:28:57 -03:00
Andrew Phillips
692a403a7e fix: make id mandatory for delete and optional for get/info
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-08-29 14:24:19 -03:00
Andrew Phillips
8c40b4de28 feat: add required validation for ids_or_tags with info and delete flags
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-08-29 14:19:15 -03:00
Andrew Phillips
6b632ff244 fix: make ids_or_tags optional for non-required modes
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-08-29 14:18:50 -03:00
Andrew Phillips
7b518eb2e9 fix: make verbose field public in OptionsArgs
Co-authored-by: aider (openai/andrew/openrouter/google/gemini-2.5-pro) <aider@aider.chat>
2025-08-29 11:41:31 -03:00
Andrew Phillips
bc78075b1a feat: add --generate-config mode to output default config
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-08-27 22:05:09 -03:00
Andrew Phillips
2e4cacaaba feat: add derive_more for NumberOrString and ProgramWriter
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
2025-08-27 21:50:04 -03:00
Andrew Phillips
71b29d1def chore: remove unused anyhow error import and server password fields
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-25 18:28:23 -03:00
Andrew Phillips
ad12b552a0 fix: remove unused KeyValue struct and related fields
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-25 18:27:49 -03:00
Andrew Phillips
f6454d94f3 refactor: remove unused code and fields
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-25 18:26:01 -03:00
Andrew Phillips
bc2ebaa314 fix: remove references to removed 'update' mode in argument definitions
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-18 20:16:47 -03:00
Andrew Phillips
3c6df7334a refactor: remove update mode and related references
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-18 20:15:39 -03:00
Andrew Phillips
37b0d0e3b0 chore: remove --digest argument and $KEEP_DIGEST environment variable
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-18 09:50:23 -03:00
Andrew Phillips
1c6064fdb7 refactor: remove unused struct members and methods
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:32:11 -03:00
Andrew Phillips
a00952a377 fix: remove unused delete_item handler and add accessors for unused fields
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:29:32 -03:00
Andrew Phillips
dfd855f380 fix: make unused fields public and remove pub(crate) visibility
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:25:04 -03:00
Andrew Phillips
5e111e002a style: make verbose fields private
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:23:53 -03:00
Andrew Phillips
3de5947d42 refactor: make unused fields and functions private
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:23:20 -03:00
Andrew Phillips
c3f4e03f33 refactor: remove unused fields and functions
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:22:59 -03:00
Andrew Phillips
ad1a7e44bc fix: add missing FromStr trait import
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:22:25 -03:00
Andrew Phillips
969d30924b fix: remove unused imports and make unused fields private
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-16 12:21:57 -03:00
Andrew Phillips
2764f16c20 feat: add support for salted password hash authentication
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-15 21:03:12 -03:00
Andrew Phillips
29ce003fa8 feat: implement server configuration with address and port options
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-15 17:55:13 -03:00
Andrew Phillips
067cba703b feat: add config system with --config argument and priority-based configuration
Co-authored-by: aider (openai/andrew/openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-08-15 16:31:57 -03:00
Andrew Phillips
2dfaed38b8 fix: implement FromStr for NumberOrString and KeyValue to fix clap parsing errors
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-14 17:03:55 -03:00
Andrew Phillips
6af1ac30df fix: resolve import issues for Args and ProgramWriter in tests
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-14 13:04:22 -03:00