asp/keep
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,533 Commits 1 Branch 0 Tags
b166477202c59a302e9e8bf37a4891bd9dbe2d44
Commit Graph

53 Commits

Author SHA1 Message Date
Andrew Phillips
b166477202 fix: harden security, eliminate panics, remove dead code, add Dockerfile 
Security:
- Use constant-time password comparison (subtle crate) to prevent timing attacks
- Replace permissive CORS with configurable origin-restricted CORS
- Add TLS warning when password auth is used without HTTPS

Bug fixes:
- Convert MetaPlugin panics to anyhow::Result (get_meta_plugin, outputs_mut, options_mut)
- Replace item.id.unwrap() with proper error handling across 15 call sites
- Fix panic on unknown column type in list mode
- Fix conflicting PIPESIZE constant (was 8192 vs 65536, now unified to 8192)
- Add 256MB filter chain buffer limit to prevent OOM
- Gracefully skip unregistered plugins instead of panicking

Dead code removal:
- Delete unused filter parser files (filter_parser.rs, filter.pest, parser/ module)
- ~260 lines of dead PEG parser code removed

Code consolidation:
- Add is_content_binary_from_metadata() helper (was duplicated in 4 places)
- Simplify save_item_raw() to delegate to save_item_raw_streaming() (~90 lines removed)

Incomplete features:
- Populate filter_plugins in status output from global registry
- Add FallbackMagicFileMetaPlugin (was referenced but never implemented)
- Document init_plugins() as intentional no-op

Infrastructure:
- Add Dockerfile (static musl binary on scratch, 4.8MB)
- Add .dockerignore
- Add cors_origin to ServerConfig and config.rs
 2026-03-13 07:57:36 -03:00
Andrew Phillips
9b7cbd5244 fix: resolve doctest failures, database bugs, and remove dead code 
- Fix all 96 doctest failures across 20 files by adding hidden imports and
  proper test setup (68 pass, 33 intentionally ignored)
- Fix set_item_tags: wrap in transaction and replace item.id.unwrap() with
  proper error handling
- Fix get_items_matching: replace N+1 per-item meta queries with batch
  get_meta_for_items() call
- Fix get_item_matching: apply meta filtering instead of ignoring the parameter
- Remove duplicate doc comment in store_meta
- Remove dead code files: plugin.rs, plugins.rs, binary_detection.rs
  (never declared as modules)
- Apply cargo fmt formatting fixes
- Add keep.db to .gitignore
 2026-03-12 11:58:44 -03:00
Andrew Phillips
8a8a6e1c4b fix: correct critical bugs and improve pipe streaming performance 
Critical bug fixes:
- save_item now returns real Item from database, not a hardcoded fake
- AsyncDataService::save() reuses self.sync_service instead of creating redundant instance
- GenerateStatus trait signature mismatch fixed (CLI/API decoupling)

Performance improvements (pipe path untouched):
- CompressionEngine::open() returns Box<dyn Read + Send> enabling true streaming
- mode_get eliminates triple full-file read (was sampling then re-reading entire file)
- FilteringReader adds fast-path bypass when no filters, pre-allocates temp buffer
- text.rs meta plugin processes &[u8] slice directly, eliminates data.to_vec() clone

API correctness:
- Tag parse errors now return 400 instead of being silently discarded
- compute_diff uses similar crate (LCS-based) instead of naive positional comparison

Cleanup:
- Modernize string formatting (format!({x})) across codebase
- Remove redundant DB query in get mode
- Derive Debug/ToSchema on public types
- Delete placeholder test files with no real assertions
- Extract parse_comma_tags utility function
 2026-03-11 20:45:05 -03:00
Andrew Phillips
fdeb5f7951 Ugh 2026-02-19 13:57:39 -04:00
Andrew Phillips
a72395fe83 refactor: simplify filter plugin interface to use &mut dyn Read/Write 2025-09-15 17:42:35 -03:00
Andrew Phillips
059bde09e4 refactor: simplify filter plugin signatures by removing boxed parameters 2025-09-12 10:36:09 -03:00
Andrew Phillips
9ad3c1d9c8 docs: Add rustdoc comments for text metadata plugin internals 
Co-authored-by: aider (openai/andrew/openrouter/sonoma-sky-alpha) <aider@aider.chat>
 2025-09-10 13:57:51 -03:00
Andrew Phillips
fed3722ef9 fix: Resolve compilation errors by refactoring imports and type annotations 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-09-03 09:39:22 -03:00
Andrew Phillips
15496345d9 feat: Implement registry for meta plugins 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-09-03 09:33:39 -03:00
Andrew Phillips
fe19ba0c5c fix: remove filter_service and fix function arguments 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-28 21:08:41 -03:00
Andrew Phillips
8fcccf68e3 fix: remove unused filter_plugin import and unused ringbuf import 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-28 21:05:23 -03:00
Andrew Phillips
4c8466bb21 refactor: reduce code duplication in filter and item services 
Co-authored-by: aider (openai/andrew/openrouter/mistralai/mistral-medium-3.1) <aider@aider.chat>
 2025-08-28 20:51:39 -03:00
Andrew Phillips
5542f5592a refactor: remove unused line range options from text meta plugin 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-28 20:39:40 -03:00
Andrew Phillips
35e2368dea refactor: move filtering logic to filter plugins 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-28 20:38:57 -03:00
Andrew Phillips
66696af67e refactor: remove old filter implementations and use filter plugins 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-28 20:34:38 -03:00
Andrew Phillips
6c29c9c4c5 feat: add head/tail/line range processing for text meta plugin 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-28 20:06:30 -03:00
Andrew Phillips
40e4fcc74a refactor: remove binary plugin and its references 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-28 13:10:52 -03:00
Andrew Phillips
a820078214 fix: add missing MetaPluginType imports 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 21:19:59 -03:00
Andrew Phillips
7b1820cb63 refactor: remove redundant meta_name field and simplify default outputs 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 21:12:06 -03:00
Andrew Phillips
79fdf05d84 refactor: replace meta_name with MetaPluginType from strum 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 21:04:52 -03:00
Andrew Phillips
b573c0dbe7 fix: prevent emitting disabled metadata outputs as null 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 20:50:02 -03:00
Andrew Phillips
3e7f491b74 fix: handle disabled text outputs properly 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 18:32:37 -03:00
Andrew Phillips
97fb35b5f0 fix: update process_metadata_outputs to handle serde_yaml::Value 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 17:47:48 -03:00
Andrew Phillips
cb685d4329 feat: set unused text outputs to null based on method options 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 17:45:33 -03:00
Andrew Phillips
2d1174266d feat: add default boolean options for text plugin statistics 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 17:31:13 -03:00
Andrew Phillips
19848fd379 feat: add default options to meta plugins status table 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 17:27:11 -03:00
Andrew Phillips
fdcccc844e refactor: remove new_simple methods and replace with new(None, None) 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 13:59:37 -03:00
Andrew Phillips
57088471a1 chore: remove debug logging from text plugin 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 13:44:52 -03:00
Andrew Phillips
d527caa7bd fix: change default value for track_line_median_len to false 2025-08-27 13:44:50 -03:00
Andrew Phillips
a3ff74c8e9 refactor: extract helper methods to reduce code duplication in TextPlugin 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 12:21:52 -03:00
Andrew Phillips
b7bf9b20de refactor: remove redundant configure_options and default_options methods 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 10:43:30 -03:00
Andrew Phillips
e8f2c00416 feat: add line length statistics tracking flags 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 10:35:14 -03:00
Andrew Phillips
1ea4fc2180 feat: round text_line_mean_len to nearest integer 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 10:18:04 -03:00
Andrew Phillips
9088b76067 fix: add debug logging and set default values for text plugin options 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 10:09:43 -03:00
Andrew Phillips
e10605bb7e feat: add debug logging for line length statistics tracking 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 10:05:21 -03:00
Andrew Phillips
9a052bddd7 feat: add text line length statistics and options 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-27 09:24:31 -03:00
Andrew Phillips
dd217d6c6a fix: prevent text plugin from finalizing in update method 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-26 20:57:17 -03:00
Andrew Phillips
a92076bbec fix: prevent premature finalization when text buffer reaches max size 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-26 20:55:26 -03:00
Andrew Phillips
f2161c32f8 feat: add debug logs for text plugin initialization and finalization 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-26 20:54:38 -03:00
Andrew Phillips
6a1ac7284e feat: add debug logs for text metadata processing 
Co-authored-by: aider (openai/andrew/openrouter/deepseek/deepseek-chat-v3.1) <aider@aider.chat>
 2025-08-26 20:52:52 -03:00
Andrew Phillips
6becdb4fbd fix: resolve mutable borrow conflicts in text meta plugin 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:49:57 -03:00
Andrew Phillips
adc16bd761 refactor: move helper methods out of trait implementation 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:49:09 -03:00
Andrew Phillips
549a671cf9 feat: add binary detection and word/line count helper methods 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:47:01 -03:00
Andrew Phillips
43088ad6a0 fix: move helper methods to impl block 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:46:46 -03:00
Andrew Phillips
e9b9532160 refactor: extract binary detection and word line count logic into helper methods 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:44:01 -03:00
Andrew Phillips
0ad8f3ccfa refactor: change buffer to Option<Vec<u8>> and drop it after binary detection 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:40:23 -03:00
Andrew Phillips
e2bef42a55 feat: fix text plugin word and line count tracking 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:37:21 -03:00
Andrew Phillips
a620db8cfe feat: add word and line count output in finalize method 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:22:42 -03:00
Andrew Phillips
cf4254750d feat: move word and line count output to finalize method 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:20:14 -03:00
Andrew Phillips
0b57da071a feat: implement accurate word counting across block boundaries 
Co-authored-by: aider (openai/andrew/openrouter/qwen/qwen3-coder) <aider@aider.chat>
 2025-08-26 19:12:27 -03:00