asp/keep
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: rewrite README, add LICENSE, remove outdated files

Browse Source 
- Rewrite README.md with comprehensive documentation covering all
  features: compression engines, meta plugins, filter plugins, server
  mode, MCP integration, and configuration
- Add MIT LICENSE file
- Delete README.org (consolidated into README.md)
- Delete empty PLAN.md
- Update AGENTS.md with current build/test commands and conventions

Co-Authored-By: andrew/openrouter/hunter-alpha <noreply@opencode.ai>
This commit is contained in:
Andrew Phillips
2026-03-12 18:01:23 -03:00
parent 79930f4b01
commit d2581358e9
5 changed files with 487 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
AGENTS.md
View File

@@ -37,16 +37,3 @@ TERM=dumb cargo build --features server # With server feature
- Tests in `src/tests/` mirroring `src/` structure; shared helpers in `src/tests/common/test_helpers.rs`
- Key helpers: `create_temp_dir()`, `create_temp_db()`, `test_compression_engine()`
- Test naming: `test_<feature>_<scenario>`
## Procedures
### Fix build problems
1. `TERM=dumb cargo check`
2. Read affected files, fix errors, preserve functionality, don't downgrade versions
3. Prefer `write_file` for full file rewrites; repeat from step 1
### Fix formatting
1. `TERM=dumb cargo fmt`
2. Continue with fix build problems procedure

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Andrew Phillips
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

0
PLAN.md
View File

475
README.md
View File

@@ -1,16 +1,473 @@
# Keep - Temporary File Management with Compression and Metadata
# Keep
Keep is a command-line tool for managing temporary files with automatic compression, metadata generation, and querying capabilities. It supports various compression algorithms and metadata plugins for rich item inspection.
A command-line utility for managing temporary files with automatic compression, metadata generation, and querying. Instead of redirecting output to temporary files, pipe it into `keep` for organized storage and retrieval.
```sh
# Instead of this:
curl -s https://api.example.com/data > /tmp/api-data.json
cat /tmp/api-data.json
# Do this:
curl -s https://api.example.com/data | keep api-data
keep --get api-data
```
## Features
- **Store and Retrieve**: Save content with automatic compression and retrieve by ID or tags.
- **Compression Support**: Built-in support for LZ4, GZip, and more via external programs (BZip2, XZ, ZStd).
- **Metadata Plugins**: Automatic extraction of file type, digests, hostname, user info, and custom metadata.
- **Filtering**: Apply filters (head, tail, grep, etc.) when retrieving content.
- **Querying**: List, search, and diff items with flexible formatting.
- **REST API Server**: Optional HTTP server for programmatic access.
- **Modular Design**: Extensible via plugins for compression, metadata, and filtering.
- **Store and Retrieve** - Save content with tags, retrieve by ID or tag
- **Automatic Compression** - LZ4, GZip, BZip2, XZ, ZStd support
- **Metadata Plugins** - Auto-extract file type, digests, hostname, user info, and more
- **Filters** - Apply transformations (head, tail, grep, strip ANSI) on retrieval
- **Querying** - List, search, diff items with flexible formatting
- **REST API Server** - Optional HTTP server for programmatic access
- **MCP Support** - Model Context Protocol integration for AI assistants
- **Modular Design** - Extensible plugin system for compression, metadata, and filtering
## Installation
### From Source
Requires Rust and Cargo.
```sh
cargo build --release
cargo install --path .
```
### Static Binary (Linux)
```sh
./build-static.bash
# Binary at bin/keep
```
### Using Cargo
```sh
cargo install --path .
```
## Quick Start
```sh
# Save content with tags
echo "Hello, world!" | keep --save greeting
# Retrieve by tag
keep --get greeting
# List all items
keep --list
# Get item details
keep --info greeting
# Delete by tag
keep --delete greeting
```
## Usage
### Save Mode
Save stdin content with tags and metadata.
```sh
# Basic save (auto-assigned ID)
echo "data" | keep --save
# Save with tags
echo "data" | keep --save my-tag
# Save with multiple tags and metadata
cat report.pdf | keep --save report --meta project=alpha --meta env=prod
# Specify compression and digest
echo "data" | keep --save my-tag --compression gzip --digest sha256
```
### Get Mode
Retrieve items by ID or tags. Default mode when IDs are provided.
```sh
# Get by ID
keep --get 1
keep 1
# Get by tag
keep --get my-tag
keep my-tag
# Get with filters
keep --get 1 --filters "head_lines(10)"
# Get with metadata filter
keep --get --meta project=alpha
# Force binary output to TTY
keep --get 1 --force
```
### List Mode
List stored items with filtering and formatting.
```sh
# List all items
keep --list
# List by tag
keep --list my-tag
# List with metadata filter
keep --list --meta env=prod
# Custom column format
keep --list --list-format "id,time,size,tags"
# JSON output
keep --list --output-format json
# Human-readable sizes
keep --list --human-readable
```
### Info Mode
Show detailed information about an item.
```sh
keep --info 1
keep --info my-tag
keep --info --meta key=value
```
### Update Mode
Update an item's tags and metadata.
```sh
# Replace tags
keep --update 1 new-tag
# Update metadata
keep --update 1 --meta key=newvalue
# Remove metadata (no value)
keep --update 1 --meta key
```
### Delete Mode
Delete items by ID.
```sh
keep --delete 1
keep --delete 1 2 3
```
### Diff Mode
Show differences between two items.
```sh
keep --diff 1 2
```
### Status Mode
Show system status and supported features.
```sh
keep --status
keep --status-plugins
keep --status --verbose
```
## Filters
Apply transformations to item content during retrieval. Filters are chained with `|`.
```sh
# First 10 lines
keep --get 1 --filters "head_lines(10)"
# Skip first 5 lines, then grep
keep --get 1 --filters "skip_lines(5)|grep(pattern=error)"
# Strip ANSI escape codes
keep --get 1 --filters "strip_ansi"
# Last 100 bytes
keep --get 1 --filters "tail_bytes(100)"
```
### Available Filters
| Filter | Description | Parameters |
|--------|-------------|------------|
| `head_bytes(n)` | First n bytes | `count` |
| `head_lines(n)` | First n lines | `count` |
| `tail_bytes(n)` | Last n bytes | `count` |
| `tail_lines(n)` | Last n lines | `count` |
| `skip_bytes(n)` | Skip first n bytes | `count` |
| `skip_lines(n)` | Skip first n lines | `count` |
| `grep(pattern)` | Filter matching lines | `pattern` (regex) |
| `strip_ansi` | Remove ANSI escape codes | none |
Set the `KEEP_FILTERS` environment variable to apply filters by default.
## Configuration
### Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `KEEP_DIR` | Storage directory | `~/.keep` |
| `KEEP_CONFIG` | Config file path | `~/.config/keep/config.yml` |
| `KEEP_COMPRESSION` | Compression algorithm | `lz4` |
| `KEEP_META_PLUGINS` | Meta plugins to use | `env` |
| `KEEP_FILTERS` | Default filter chain | none |
| `KEEP_LIST_FORMAT` | List column format | built-in defaults |
| `KEEP_SERVER_ADDRESS` | Server bind address | `127.0.0.1` |
| `KEEP_SERVER_PORT` | Server port | `21080` |
| `KEEP_SERVER_PASSWORD` | Server password | none |
| `KEEP_SERVER_PASSWORD_HASH` | Server password hash | none |
Any config setting can be overridden with `KEEP__<SETTING>` environment variables (double underscore separator).
Capture shell commands automatically by setting `KEEP_META_command` in your shell profile (see Shell Integration).
### Configuration File
Default location: `~/.config/keep/config.yml`
Generate a default configuration:
```sh
keep --generate-config > ~/.config/keep/config.yml
```
```yaml
# Storage directory
dir: ~/.keep
# List view columns
list_format:
- name: id
label: "Item"
align: right
- name: time
label: "Time"
align: right
- name: size
label: "Size"
align: right
- name: tags
label: "Tags"
align: left
# Table styling
table_config:
style: utf8_full
content_arrangement: dynamic
# Default compression
compression_plugin:
name: gzip
# Default meta plugins
meta_plugins:
- name: env
- name: digest
options:
algorithm: sha256
# Server settings
server:
address: "127.0.0.1"
port: 21080
password: "secret"
human_readable: true
quiet: false
force: false
```
## Server Mode
Start an HTTP REST API server for programmatic access.
```sh
# Start server on default address (127.0.0.1:21080)
keep --server
# Custom address and port
keep --server --server-address 0.0.0.0 --server-port 8080
# With authentication
keep --server --server-password mypassword
```
### API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/status` | System status |
| `GET` | `/api/plugins/status` | Plugins status |
| `GET` | `/api/item/` | List items (supports `tags`, `order`, `start`, `count` params) |
| `POST` | `/api/item/` | Create item (body: raw content, query: `tags`, `metadata`) |
| `GET` | `/api/item/latest/content` | Latest item content |
| `GET` | `/api/item/latest/meta` | Latest item metadata |
| `GET` | `/api/item/{id}` | Item info by ID |
| `GET` | `/api/item/{id}/content` | Item content by ID |
| `GET` | `/api/item/{id}/meta` | Item metadata by ID |
| `GET` | `/api/item/{id}/info` | Item info by ID |
| `DELETE` | `/api/item/{id}` | Delete item by ID |
| `GET` | `/api/diff` | Diff two items (`id_a`, `id_b` params) |
### Authentication
```sh
# Bearer token
curl -H "Authorization: Bearer mypassword" http://localhost:21080/api/status
# Basic auth
curl -u keep:mypassword http://localhost:21080/api/status
```
When no password is configured, authentication is disabled.
### Swagger UI
Build with the `swagger` feature to enable OpenAPI documentation:
```sh
cargo build --features server,swagger
```
Swagger UI available at `/swagger`, OpenAPI spec at `/openapi.json`.
## MCP (Model Context Protocol)
AI assistant integration via the Model Context Protocol. Enable with the `mcp` feature.
```sh
cargo build --features server,mcp
```
MCP endpoint available at `/mcp/sse` when the server is running.
### Available Tools
| Tool | Description | Parameters |
|------|-------------|------------|
| `save_item` | Save new content | `content`, `tags[]`, `metadata{}` |
| `get_item` | Get item by ID | `id` |
| `get_latest_item` | Get latest item | `tags[]` |
| `list_items` | List items | `tags[]`, `limit`, `offset` |
| `search_items` | Search items | `tags[]`, `metadata{}` |
## Feature Flags
| Feature | Default | Description |
|---------|---------|-------------|
| `magic` | Yes | File type detection via libmagic |
| `lz4` | Yes | LZ4 compression (internal) |
| `gzip` | Yes | GZip compression (internal) |
| `server` | No | HTTP REST API server |
| `mcp` | No | Model Context Protocol support |
| `swagger` | No | Swagger UI for API docs |
| `bzip2` | No | BZip2 compression (external program) |
| `xz` | No | XZ compression (external program) |
| `zstd` | No | ZStd compression (external program) |
```sh
# Build with server and swagger
cargo build --features server,swagger
# Build with all features
cargo build --features server,mcp,swagger,magic
```
## Shell Integration
Source `profile.bash` to enable shell integration:
```sh
source /path/to/keep/profile.bash
```
This provides:
- **`keep` function** - Captures the current command in metadata automatically
- **`@` alias** - Shorthand for `keep --save`
- **`@@` alias** - Shorthand for `keep --get`
```sh
# Save with automatic command capture
curl -s api.example.com | @ api-response
# Quick retrieve
@@ api-response
```
## Compression
Items are compressed automatically on save. Default: LZ4.
| Algorithm | Type | Speed | Ratio |
|-----------|------|-------|-------|
| `lz4` | Internal | Fastest | Lower |
| `gzip` | Internal | Fast | Good |
| `bzip2` | External | Slow | Better |
| `xz` | External | Slowest | Best |
| `zstd` | External | Fast | Good |
| `none` | Internal | N/A | N/A |
```sh
# Specify compression per item
echo "data" | keep --save my-tag --compression zstd
# Set default via environment
export KEEP_COMPRESSION=gzip
```
External compression programs (`bzip2`, `xz`, `zstd`) must be installed on the system.
## Meta Plugins
Metadata is automatically extracted when saving items.
| Plugin | Key | Description |
|--------|-----|-------------|
| `env` | `*` | Capture `KEEP_META_*` environment variables |
| `magic_file` | `file_type` | File type detection (requires `magic` feature) |
| `text` | `text_line_count`, `text_word_count` | Line and word counts |
| `user` | `uid`, `user`, `gid`, `group` | Current user info |
| `shell` | `shell` | Current shell path |
| `shell_pid` | `shell_pid` | Shell process ID |
| `keep_pid` | `keep_pid` | Keep process ID |
| `digest` | `digest_sha256`, `digest_md5` | Content digests |
| `read_time` | `read_time` | Time to read content |
| `read_rate` | `read_rate` | Data read rate |
| `hostname` | `hostname`, `hostname_short` | System hostname |
| `exec` | Custom | Run external commands for metadata |
| `cwd` | `cwd` | Current working directory |
```sh
# Use specific plugins
echo "data" | keep --save tag --meta-plugins "digest,text,user"
# Capture custom metadata via environment
KEEP_META_project=alpha echo "data" | keep --save tag
```
## License
MIT License - see [LICENSE](LICENSE) for details.
## Contact
Andrew Phillips - andrew@gt0.ca

141
README.org
View File

@@ -1,141 +0,0 @@
#+TITLE: Keep
#+AUTHOR: Andrew Phillips
* Introduction
Keep is a command-line utility designed to manage temporary files created on the command line. Instead of redirecting output to a temporary file (e.g., =command > ~/whatever.tmp=), you can use =keep= to handle the temporary files for you (e.g., =command | keep=).
* Installation
To install Keep, you need to have Rust and Cargo installed on your system. You can then build and install Keep using the following commands:
#+BEGIN_SRC sh
cargo build --release
cargo install --path .
#+END_SRC
* Usage
Keep provides several subcommands to manage temporary files. Below are some examples of how to use Keep.
** Saving an Item
To save an item with tags and metadata, you can use the =--save= option:
#+BEGIN_SRC sh
echo "Hello, world!" | keep --save example --meta key=value
#+END_SRC
** Getting an Item
To retrieve an item by its ID or by matching tags and metadata, you can use the =--get= option:
#+BEGIN_SRC sh
keep --get 1
keep --get example
keep --get --meta key=value
keep 1
keep example
#+END_SRC
** Listing Items
To list all items or filter them by tags and metadata, you can use the =--list= option:
#+BEGIN_SRC sh
keep --list
keep --list example
keep --list --meta key=value
#+END_SRC
** Updating an Item
To update an item's tags and metadata, you can use the =--update= option:
#+BEGIN_SRC sh
keep --update 1 newtag --meta key=newvalue
#+END_SRC
** Deleting an Item
To delete an item by its ID or by matching tags, you can use the =--delete= option:
#+BEGIN_SRC sh
keep --delete 1
keep --delete example
#+END_SRC
** Showing Status
To show the status of directories and supported compression algorithms, you can use the =--status= option:
#+BEGIN_SRC sh
keep --status
#+END_SRC
** Diffing Items
To show a diff between two items by ID, you can use the =--diff= option:
#+BEGIN_SRC sh
keep --diff 1 2
#+END_SRC
** Getting Information About an Item
To get detailed information about an item by its ID or by matching tags and metadata, you can use the =--info= option:
#+BEGIN_SRC sh
keep --info 1
keep --info example
keep --info --meta key=value
#+END_SRC
* Configuration
Keep can be configured using environment variables and command-line options. The following environment variables are supported:
- =KEEP_DIR=: Specify the directory to use for storage.
- =KEEP_LIST_FORMAT=: A comma-separated list of columns to display with =--list=.
- =KEEP_DIGEST=: Digest algorithm to use when saving items.
- =KEEP_COMPRESSION=: Compression algorithm to use when saving items.
* Examples
Here are some examples of how to use Keep with different options:
** Saving an Item with Compression and Digest
#+BEGIN_SRC sh
echo "Hello, world!" | keep --save example --meta key=value --compression gzip --digest sha256
#+END_SRC
** Getting an Item with Human-Readable Sizes
#+BEGIN_SRC sh
keep --get 1 --human-readable
#+END_SRC
** Listing Items with Custom Format
#+BEGIN_SRC sh
keep --list --list-format "id,time,size,tags,meta:hostname"
#+END_SRC
** Updating an Item with New Tags and Metadata
#+BEGIN_SRC sh
keep --update 1 newtag --meta key=newvalue
#+END_SRC
** Deleting an Item by Tag
#+BEGIN_SRC sh
keep --delete example
#+END_SRC
** Showing Status with Verbose Output
#+BEGIN_SRC sh
keep --status --verbose
#+END_SRC
** Diffing Items with IDs
#+BEGIN_SRC sh
keep --diff 1 2
#+END_SRC
** Getting Information About an Item with Metadata
#+BEGIN_SRC sh
keep --info 1
#+END_SRC
* License
Keep is licensed under the MIT License. See the LICENSE file for more details.
* Contributing
Contributions are welcome! Please open an issue or submit a pull request on the GitHub repository.
* Contact
For more information, please contact Andrew Phillips at andrew@gt0.ca.