keep/DESIGN.md

PROJECT RULES - KEEP FIRST

Standard Rules

1. ALWAYS keep DESIGN.md updated with any architectural or design changes
2. ALWAYS keep project rules first in this document
3. ALWAYS use git commands to remove or move files (git rm, git mv, etc.)
4. Follow Rust naming conventions and idioms
5. Use anyhow for error handling throughout the codebase
6. Maintain comprehensive logging with the log crate
7. Write unit tests for critical functionality
8. Document public APIs with rustdoc comments
9. Keep modules focused on single responsibilities
10. Prefer composition over inheritance
11. Handle errors gracefully and provide meaningful error messages
12. Ensure code is safe and avoids unsafe blocks where possible

Code - Modules

Main Module

• main.rs - Entry point, CLI argument parsing, mode dispatching
• Interacts with all mode modules based on user input
• Handles database connection setup and data directory management

Mode Modules

• modes/save.rs - Save new items with tags/metadata
• modes/get.rs - Retrieve items by ID/tags
• modes/list.rs - List items with filtering and formatting
• modes/delete.rs - Delete items by ID
• modes/update.rs - Update item tags/metadata
• modes/info.rs - Show detailed item information
• modes/diff.rs - Compare two items
• modes/status.rs - Show system status and capabilities
• modes/server.rs - REST HTTP server mode with OpenAPI documentation
• modes/common.rs - Shared utilities for all modes

Database Module

• db.rs - SQLite database operations
• Handles items, tags, and metadata storage
• Provides query functions for all modes
• Manages database migrations

Compression Engine Module

• compression_engine.rs - Trait and type definitions
• compression_engine/gzip.rs - GZip implementation
• compression_engine/lz4.rs - LZ4 implementation
• compression_engine/none.rs - No compression implementation
• compression_engine/program.rs - External program wrapper

Digest Functionality

• Digest functionality is now integrated into meta plugins
• SHA-256 and other digest algorithms are implemented as meta plugins
• External digest programs are supported through meta plugin program wrapper

Meta Plugin Module

• meta_plugin.rs - Trait and type definitions
• meta_plugin/program.rs - External program wrapper
• meta_plugin/digest.rs - Internal digest implementations

Plugins Module

• plugins.rs - Shared plugin utilities
• Contains ProgramWriter for external process communication

REST API Endpoints

Item Operations

• GET /api/item/latest - Return the latest item as JSON, with metadata and content. Optional params: tags[] (find latest with matching tags)
• GET /api/item/latest/meta - Return the latest item metadata as JSON. Optional params: tags[]
• GET /api/item/latest/content - Return the raw content of the latest item using the mime type from meta, or unknown binary. Optional params: tags[]
• GET /api/item/<#> - Return the item as JSON, with metadata and content
• GET /api/item/<#>/meta - Return the item metadata as JSON
• GET /api/item/<#>/content - Return the raw content of the item using the mime type from meta, or unknown binary
• GET /api/item/ - Get a list of items as JSON. Optional params: order=newest, start=0, count=100, tags[]
• POST /api/item/ - Add a new item. Optional params: tags[] (defaults to ['none'] if empty), meta[]
• DELETE /api/item/<#> - Delete an item