asp/keep
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: rewrite README, add LICENSE, remove outdated files

Browse Source 
- Rewrite README.md with comprehensive documentation covering all
  features: compression engines, meta plugins, filter plugins, server
  mode, MCP integration, and configuration
- Add MIT LICENSE file
- Delete README.org (consolidated into README.md)
- Delete empty PLAN.md
- Update AGENTS.md with current build/test commands and conventions

Co-Authored-By: andrew/openrouter/hunter-alpha <noreply@opencode.ai>
This commit is contained in:
Andrew Phillips
2026-03-12 18:01:23 -03:00
parent 79930f4b01
commit d2581358e9
5 changed files with 487 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

475
README.md
View File

@@ -1,16 +1,473 @@
# Keep - Temporary File Management with Compression and Metadata
# Keep
Keep is a command-line tool for managing temporary files with automatic compression, metadata generation, and querying capabilities. It supports various compression algorithms and metadata plugins for rich item inspection.
A command-line utility for managing temporary files with automatic compression, metadata generation, and querying. Instead of redirecting output to temporary files, pipe it into `keep` for organized storage and retrieval.
```sh
# Instead of this:
curl -s https://api.example.com/data > /tmp/api-data.json
cat /tmp/api-data.json
# Do this:
curl -s https://api.example.com/data | keep api-data
keep --get api-data
```
## Features
- **Store and Retrieve**: Save content with automatic compression and retrieve by ID or tags.
- **Compression Support**: Built-in support for LZ4, GZip, and more via external programs (BZip2, XZ, ZStd).
- **Metadata Plugins**: Automatic extraction of file type, digests, hostname, user info, and custom metadata.
- **Filtering**: Apply filters (head, tail, grep, etc.) when retrieving content.
- **Querying**: List, search, and diff items with flexible formatting.
- **REST API Server**: Optional HTTP server for programmatic access.
- **Modular Design**: Extensible via plugins for compression, metadata, and filtering.
- **Store and Retrieve** - Save content with tags, retrieve by ID or tag
- **Automatic Compression** - LZ4, GZip, BZip2, XZ, ZStd support
- **Metadata Plugins** - Auto-extract file type, digests, hostname, user info, and more
- **Filters** - Apply transformations (head, tail, grep, strip ANSI) on retrieval
- **Querying** - List, search, diff items with flexible formatting
- **REST API Server** - Optional HTTP server for programmatic access
- **MCP Support** - Model Context Protocol integration for AI assistants
- **Modular Design** - Extensible plugin system for compression, metadata, and filtering
## Installation
### From Source
Requires Rust and Cargo.
```sh
cargo build --release
cargo install --path .
```
### Static Binary (Linux)
```sh
./build-static.bash
# Binary at bin/keep
```
### Using Cargo
```sh
cargo install --path .
```
## Quick Start
```sh
# Save content with tags
echo "Hello, world!" | keep --save greeting
# Retrieve by tag
keep --get greeting
# List all items
keep --list
# Get item details
keep --info greeting
# Delete by tag
keep --delete greeting
```
## Usage
### Save Mode
Save stdin content with tags and metadata.
```sh
# Basic save (auto-assigned ID)
echo "data" | keep --save
# Save with tags
echo "data" | keep --save my-tag
# Save with multiple tags and metadata
cat report.pdf | keep --save report --meta project=alpha --meta env=prod
# Specify compression and digest
echo "data" | keep --save my-tag --compression gzip --digest sha256
```
### Get Mode
Retrieve items by ID or tags. Default mode when IDs are provided.
```sh
# Get by ID
keep --get 1
keep 1
# Get by tag
keep --get my-tag
keep my-tag
# Get with filters
keep --get 1 --filters "head_lines(10)"
# Get with metadata filter
keep --get --meta project=alpha
# Force binary output to TTY
keep --get 1 --force
```
### List Mode
List stored items with filtering and formatting.
```sh
# List all items
keep --list
# List by tag
keep --list my-tag
# List with metadata filter
keep --list --meta env=prod
# Custom column format
keep --list --list-format "id,time,size,tags"
# JSON output
keep --list --output-format json
# Human-readable sizes
keep --list --human-readable
```
### Info Mode
Show detailed information about an item.
```sh
keep --info 1
keep --info my-tag
keep --info --meta key=value
```
### Update Mode
Update an item's tags and metadata.
```sh
# Replace tags
keep --update 1 new-tag
# Update metadata
keep --update 1 --meta key=newvalue
# Remove metadata (no value)
keep --update 1 --meta key
```
### Delete Mode
Delete items by ID.
```sh
keep --delete 1
keep --delete 1 2 3
```
### Diff Mode
Show differences between two items.
```sh
keep --diff 1 2
```
### Status Mode
Show system status and supported features.
```sh
keep --status
keep --status-plugins
keep --status --verbose
```
## Filters
Apply transformations to item content during retrieval. Filters are chained with `|`.
```sh
# First 10 lines
keep --get 1 --filters "head_lines(10)"
# Skip first 5 lines, then grep
keep --get 1 --filters "skip_lines(5)|grep(pattern=error)"
# Strip ANSI escape codes
keep --get 1 --filters "strip_ansi"
# Last 100 bytes
keep --get 1 --filters "tail_bytes(100)"
```
### Available Filters
| Filter | Description | Parameters |
|--------|-------------|------------|
| `head_bytes(n)` | First n bytes | `count` |
| `head_lines(n)` | First n lines | `count` |
| `tail_bytes(n)` | Last n bytes | `count` |
| `tail_lines(n)` | Last n lines | `count` |
| `skip_bytes(n)` | Skip first n bytes | `count` |
| `skip_lines(n)` | Skip first n lines | `count` |
| `grep(pattern)` | Filter matching lines | `pattern` (regex) |
| `strip_ansi` | Remove ANSI escape codes | none |
Set the `KEEP_FILTERS` environment variable to apply filters by default.
## Configuration
### Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `KEEP_DIR` | Storage directory | `~/.keep` |
| `KEEP_CONFIG` | Config file path | `~/.config/keep/config.yml` |
| `KEEP_COMPRESSION` | Compression algorithm | `lz4` |
| `KEEP_META_PLUGINS` | Meta plugins to use | `env` |
| `KEEP_FILTERS` | Default filter chain | none |
| `KEEP_LIST_FORMAT` | List column format | built-in defaults |
| `KEEP_SERVER_ADDRESS` | Server bind address | `127.0.0.1` |
| `KEEP_SERVER_PORT` | Server port | `21080` |
| `KEEP_SERVER_PASSWORD` | Server password | none |
| `KEEP_SERVER_PASSWORD_HASH` | Server password hash | none |
Any config setting can be overridden with `KEEP__<SETTING>` environment variables (double underscore separator).
Capture shell commands automatically by setting `KEEP_META_command` in your shell profile (see Shell Integration).
### Configuration File
Default location: `~/.config/keep/config.yml`
Generate a default configuration:
```sh
keep --generate-config > ~/.config/keep/config.yml
```
```yaml
# Storage directory
dir: ~/.keep
# List view columns
list_format:
- name: id
label: "Item"
align: right
- name: time
label: "Time"
align: right
- name: size
label: "Size"
align: right
- name: tags
label: "Tags"
align: left
# Table styling
table_config:
style: utf8_full
content_arrangement: dynamic
# Default compression
compression_plugin:
name: gzip
# Default meta plugins
meta_plugins:
- name: env
- name: digest
options:
algorithm: sha256
# Server settings
server:
address: "127.0.0.1"
port: 21080
password: "secret"
human_readable: true
quiet: false
force: false
```
## Server Mode
Start an HTTP REST API server for programmatic access.
```sh
# Start server on default address (127.0.0.1:21080)
keep --server
# Custom address and port
keep --server --server-address 0.0.0.0 --server-port 8080
# With authentication
keep --server --server-password mypassword
```
### API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/status` | System status |
| `GET` | `/api/plugins/status` | Plugins status |
| `GET` | `/api/item/` | List items (supports `tags`, `order`, `start`, `count` params) |
| `POST` | `/api/item/` | Create item (body: raw content, query: `tags`, `metadata`) |
| `GET` | `/api/item/latest/content` | Latest item content |
| `GET` | `/api/item/latest/meta` | Latest item metadata |
| `GET` | `/api/item/{id}` | Item info by ID |
| `GET` | `/api/item/{id}/content` | Item content by ID |
| `GET` | `/api/item/{id}/meta` | Item metadata by ID |
| `GET` | `/api/item/{id}/info` | Item info by ID |
| `DELETE` | `/api/item/{id}` | Delete item by ID |
| `GET` | `/api/diff` | Diff two items (`id_a`, `id_b` params) |
### Authentication
```sh
# Bearer token
curl -H "Authorization: Bearer mypassword" http://localhost:21080/api/status
# Basic auth
curl -u keep:mypassword http://localhost:21080/api/status
```
When no password is configured, authentication is disabled.
### Swagger UI
Build with the `swagger` feature to enable OpenAPI documentation:
```sh
cargo build --features server,swagger
```
Swagger UI available at `/swagger`, OpenAPI spec at `/openapi.json`.
## MCP (Model Context Protocol)
AI assistant integration via the Model Context Protocol. Enable with the `mcp` feature.
```sh
cargo build --features server,mcp
```
MCP endpoint available at `/mcp/sse` when the server is running.
### Available Tools
| Tool | Description | Parameters |
|------|-------------|------------|
| `save_item` | Save new content | `content`, `tags[]`, `metadata{}` |
| `get_item` | Get item by ID | `id` |
| `get_latest_item` | Get latest item | `tags[]` |
| `list_items` | List items | `tags[]`, `limit`, `offset` |
| `search_items` | Search items | `tags[]`, `metadata{}` |
## Feature Flags
| Feature | Default | Description |
|---------|---------|-------------|
| `magic` | Yes | File type detection via libmagic |
| `lz4` | Yes | LZ4 compression (internal) |
| `gzip` | Yes | GZip compression (internal) |
| `server` | No | HTTP REST API server |
| `mcp` | No | Model Context Protocol support |
| `swagger` | No | Swagger UI for API docs |
| `bzip2` | No | BZip2 compression (external program) |
| `xz` | No | XZ compression (external program) |
| `zstd` | No | ZStd compression (external program) |
```sh
# Build with server and swagger
cargo build --features server,swagger
# Build with all features
cargo build --features server,mcp,swagger,magic
```
## Shell Integration
Source `profile.bash` to enable shell integration:
```sh
source /path/to/keep/profile.bash
```
This provides:
- **`keep` function** - Captures the current command in metadata automatically
- **`@` alias** - Shorthand for `keep --save`
- **`@@` alias** - Shorthand for `keep --get`
```sh
# Save with automatic command capture
curl -s api.example.com | @ api-response
# Quick retrieve
@@ api-response
```
## Compression
Items are compressed automatically on save. Default: LZ4.
| Algorithm | Type | Speed | Ratio |
|-----------|------|-------|-------|
| `lz4` | Internal | Fastest | Lower |
| `gzip` | Internal | Fast | Good |
| `bzip2` | External | Slow | Better |
| `xz` | External | Slowest | Best |
| `zstd` | External | Fast | Good |
| `none` | Internal | N/A | N/A |
```sh
# Specify compression per item
echo "data" | keep --save my-tag --compression zstd
# Set default via environment
export KEEP_COMPRESSION=gzip
```
External compression programs (`bzip2`, `xz`, `zstd`) must be installed on the system.
## Meta Plugins
Metadata is automatically extracted when saving items.
| Plugin | Key | Description |
|--------|-----|-------------|
| `env` | `*` | Capture `KEEP_META_*` environment variables |
| `magic_file` | `file_type` | File type detection (requires `magic` feature) |
| `text` | `text_line_count`, `text_word_count` | Line and word counts |
| `user` | `uid`, `user`, `gid`, `group` | Current user info |
| `shell` | `shell` | Current shell path |
| `shell_pid` | `shell_pid` | Shell process ID |
| `keep_pid` | `keep_pid` | Keep process ID |
| `digest` | `digest_sha256`, `digest_md5` | Content digests |
| `read_time` | `read_time` | Time to read content |
| `read_rate` | `read_rate` | Data read rate |
| `hostname` | `hostname`, `hostname_short` | System hostname |
| `exec` | Custom | Run external commands for metadata |
| `cwd` | `cwd` | Current working directory |
```sh
# Use specific plugins
echo "data" | keep --save tag --meta-plugins "digest,text,user"
# Capture custom metadata via environment
KEEP_META_project=alpha echo "data" | keep --save tag
```
## License
MIT License - see [LICENSE](LICENSE) for details.
## Contact
Andrew Phillips - andrew@gt0.ca