Commit Graph

22 Commits

Author SHA1 Message Date
00be72f3d0 refactor: rename size to uncompressed_size, add compressed_size and closed columns
Schema changes:
- Rename items.size to items.uncompressed_size for clarity
- Add compressed_size (INTEGER NULL) - tracks compressed file size on disk
- Add closed (BOOLEAN NOT NULL DEFAULT 1) - tracks whether item is fully written
- Existing items default to closed=true via migration

Lifecycle:
- Items created with closed=false, set to true on successful save/import
- Compressed size captured via fs::metadata() after compression writer closes
- Truncated uploads (413) get compressed_size set, closed=true, uncompressed_size=None
- Update command now backfills both uncompressed_size and compressed_size

Also includes bug fixes and dedup from prior review:
- Fix stream_raw_content_response using uncompressed_size for raw byte Content-Length
- ApiResponse::ok()/empty() constructors, TryFrom<ItemWithMeta> for ItemInfo
- tag_names() method on ItemWithMeta, meta_filter() on Settings
- Fix .unwrap() panics in compression engine Read/Write impls
- Fix TOCTOU race in stream_raw_content_response (now uses compressed_size)
- Fix swallowed write errors in meta plugins (digest, magic_file, exec)
- Fix term::stderr().unwrap() panic in item_service
- Deduplicate ItemService::new() calls across 20 API handlers
- ImportMeta supports #[serde(alias = "size")] for backward compat

All 75 tests, 67 doc tests pass. Clippy clean.
2026-03-18 10:58:26 -03:00
a8759c4b83 feat: add infer and tree_magic_mini meta plugins, make zstd internal by default
- Add infer crate as meta plugin for MIME type detection
- Add tree_magic_mini crate as alternative meta plugin for MIME type detection
- Add zstd, infer, tree_magic_mini to default features
- Fix static build script to use musl target instead of glibc+crt-static
- Remove hardcoded shell list from --generate-completion help text
- Fix update() in both new plugins to emit MIME metadata when buffer fills
2026-03-17 14:46:51 -03:00
a90c19efc1 feat: add native zstd compression plugin and deduplicate shared compression/meta utilities
- Add zstd crate (v0.13) with native Rust compression engine (level 3)
- Gate behind 'zstd' feature flag, fall back to program-based when disabled
- Extract CompressionService::decompressing_reader/compressing_writer with zstd support
- Extract MetaService::with_collector() to eliminate Arc<Mutex<Vec>> boilerplate
- Extract read_with_bounds() helper for skip+read pattern
- Add input validation for mutually exclusive --id and --tags flags
- Add zstd round-trip tests
2026-03-16 20:03:30 -03:00
e672ec751e feat: add JWT auth, configurable username, switch password auth to Basic
Add server-side JWT authentication with permission-based access control
(read/write/delete claims). Password authentication now uses HTTP Basic
auth only (replacing Bearer). Add configurable username for both server
and client (--server-username/--client-username, defaults to "keep").

JWT secret supports file-based loading via --server-jwt-secret-file for
Docker secrets. OPTIONS preflight requests bypass auth. HEAD mapped to
read permission.

Co-Authored-By: opencode <noreply@opencode.ai>
2026-03-13 13:56:35 -03:00
d5d58bc52c feat: add lz4 command fallback, remove unused magic.rs
- Add program-based lz4 command fallback when lz4 feature is disabled
- Feature-gate lz4.rs and lz4 tests to compile without lz4_flex
- Delete legacy magic.rs (unused, no feature gating, superseded by magic_file.rs)
2026-03-13 08:51:10 -03:00
9b7cbd5244 fix: resolve doctest failures, database bugs, and remove dead code
- Fix all 96 doctest failures across 20 files by adding hidden imports and
  proper test setup (68 pass, 33 intentionally ignored)
- Fix set_item_tags: wrap in transaction and replace item.id.unwrap() with
  proper error handling
- Fix get_items_matching: replace N+1 per-item meta queries with batch
  get_meta_for_items() call
- Fix get_item_matching: apply meta filtering instead of ignoring the parameter
- Remove duplicate doc comment in store_meta
- Remove dead code files: plugin.rs, plugins.rs, binary_detection.rs
  (never declared as modules)
- Apply cargo fmt formatting fixes
- Add keep.db to .gitignore
2026-03-12 11:58:44 -03:00
8a8a6e1c4b fix: correct critical bugs and improve pipe streaming performance
Critical bug fixes:
- save_item now returns real Item from database, not a hardcoded fake
- AsyncDataService::save() reuses self.sync_service instead of creating redundant instance
- GenerateStatus trait signature mismatch fixed (CLI/API decoupling)

Performance improvements (pipe path untouched):
- CompressionEngine::open() returns Box<dyn Read + Send> enabling true streaming
- mode_get eliminates triple full-file read (was sampling then re-reading entire file)
- FilteringReader adds fast-path bypass when no filters, pre-allocates temp buffer
- text.rs meta plugin processes &[u8] slice directly, eliminates data.to_vec() clone

API correctness:
- Tag parse errors now return 400 instead of being silently discarded
- compute_diff uses similar crate (LCS-based) instead of naive positional comparison

Cleanup:
- Modernize string formatting (format!({x})) across codebase
- Remove redundant DB query in get mode
- Derive Debug/ToSchema on public types
- Delete placeholder test files with no real assertions
- Extract parse_comma_tags utility function
2026-03-11 20:45:05 -03:00
Andrew Phillips
fdeb5f7951 Ugh 2026-02-19 13:57:39 -04:00
Andrew Phillips
dca1d6c6a4 fix: resolve compilation errors by adding Sync+Send bounds and fixing syntax
Co-authored-by: aider (openai/andrew/openrouter/sonoma-sky-alpha) <aider@aider.chat>
2025-09-11 11:57:14 -03:00
Andrew Phillips
b151998144 fix: Implement clone_box for CompressionEngineProgram
Co-authored-by: aider (openai/andrew/openrouter/sonoma-sky-alpha) <aider@aider.chat>
2025-09-11 11:32:05 -03:00
Andrew Phillips
6e4b690bd8 feat: use humansize crate and which crate for program lookup
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-12 16:23:11 -03:00
Andrew Phillips
d0eecc94f2 perf: Cache program lookups to reduce filesystem operations
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-12 16:18:07 -03:00
Andrew Phillips
d4370563c3 fix: correct mutable reference handling and remove unused variables
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-10 00:25:22 -03:00
Andrew Phillips
a3eb9e7056 fix: address critical memory safety, error handling, concurrency and security issues
This commit fixes several critical issues across the codebase:
1. Memory safety & resource leaks: Added proper cleanup for compression engine processes using RAII patterns
2. Error handling: Replaced unsafe unwrap() calls with proper error propagation using ok_or_else()?
3. Concurrency issues: Improved diff mode thread safety with proper error handling and RAII guards
4. Security concerns: Added validation for item IDs to prevent path traversal vulnerabilities
5. Database design: Wrapped database operations in transactions for atomicity in save/update modes

Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
2025-08-09 23:33:06 -03:00
Andrew Phillips
3008f3fec0 fix: resolve compilation errors and warnings in keep crate
Co-authored-by: aider (openai/andrew.openrouter.qwen.qwen3-coder) <aider@aider.chat>
2025-07-29 15:37:43 -03:00
Andrew Phillips
52618586d1 chore: remove unused dependencies and dead code
Co-authored-by: aider (openai/andrew.openrouter.qwen.qwen3-coder) <aider@aider.chat>
2025-07-29 14:53:07 -03:00
Andrew Phillips
bbdfe19836 feat: add compression and digest support with database schema updates 2025-05-14 09:45:51 -03:00
Andrew Phillips
9ba3a31e95 fix: remove duplicate digest argument and adjust imports 2025-05-12 22:11:56 -03:00
Andrew Phillips (aider)
621985ccf0 fix: add missing imports for sha2, Command, and Stdio 2025-05-12 18:27:23 -03:00
Andrew Phillips
a24612ccb8 refactor: remove unused std::process import 2025-05-12 18:27:03 -03:00
Andrew Phillips (aider)
71a35a77a0 fix: correct compression module imports 2025-05-12 17:16:52 -03:00
Andrew Phillips
e3159473d0 chore: Rename compression directory to compression_engine 2025-05-12 17:08:00 -03:00