Andrew Phillips c5529bedbf feat: add client mode with streaming support
Add client mode enabling the keep CLI to connect to a remote keep
server over HTTP. Local plugins (compression, meta, filters) run on
the client; the server stores/retrieves binary blobs.

Architecture:
- Client save uses 3-thread streaming pipeline: reader thread (stdin
  → tee/stdout → hash → compress), OS pipe, streamer thread (pipe →
  chunked HTTP POST). Memory usage is O(PIPESIZE) regardless of data
  size.
- Server accepts compress=false, meta=false, decompress=false query
  params for granular control of server-side processing.
- Streaming body handling on server via async channel → sync reader
  bridge (ChannelReader).

Key additions:
- src/client.rs: KeepClient with post_stream() for chunked upload
- src/modes/client/: save, get, list, info, delete, diff, status
- --client-url / KEEP_CLIENT_URL configuration
- --client-password / KEEP_CLIENT_PASSWORD for auth
- os_pipe dependency for zero-copy pipe streaming

Co-Authored-By: andrew/openrouter/hunter-alpha <noreply@opencode.ai>
2026-03-12 18:01:36 -03:00
2024-02-26 13:39:34 -04:00
Ugh
2026-02-19 13:57:39 -04:00

Keep

A command-line utility for managing temporary files with automatic compression, metadata generation, and querying. Instead of redirecting output to temporary files, pipe it into keep for organized storage and retrieval.

# Instead of this:
curl -s https://api.example.com/data > /tmp/api-data.json
cat /tmp/api-data.json

# Do this:
curl -s https://api.example.com/data | keep api-data
keep --get api-data

Features

  • Store and Retrieve - Save content with tags, retrieve by ID or tag
  • Automatic Compression - LZ4, GZip, BZip2, XZ, ZStd support
  • Metadata Plugins - Auto-extract file type, digests, hostname, user info, and more
  • Filters - Apply transformations (head, tail, grep, strip ANSI) on retrieval
  • Querying - List, search, diff items with flexible formatting
  • REST API Server - Optional HTTP server for programmatic access
  • MCP Support - Model Context Protocol integration for AI assistants
  • Modular Design - Extensible plugin system for compression, metadata, and filtering

Installation

From Source

Requires Rust and Cargo.

cargo build --release
cargo install --path .

Static Binary (Linux)

./build-static.bash
# Binary at bin/keep

Using Cargo

cargo install --path .

Quick Start

# Save content with tags
echo "Hello, world!" | keep --save greeting

# Retrieve by tag
keep --get greeting

# List all items
keep --list

# Get item details
keep --info greeting

# Delete by tag
keep --delete greeting

Usage

Save Mode

Save stdin content with tags and metadata.

# Basic save (auto-assigned ID)
echo "data" | keep --save

# Save with tags
echo "data" | keep --save my-tag

# Save with multiple tags and metadata
cat report.pdf | keep --save report --meta project=alpha --meta env=prod

# Specify compression and digest
echo "data" | keep --save my-tag --compression gzip --digest sha256

Get Mode

Retrieve items by ID or tags. Default mode when IDs are provided.

# Get by ID
keep --get 1
keep 1

# Get by tag
keep --get my-tag
keep my-tag

# Get with filters
keep --get 1 --filters "head_lines(10)"

# Get with metadata filter
keep --get --meta project=alpha

# Force binary output to TTY
keep --get 1 --force

List Mode

List stored items with filtering and formatting.

# List all items
keep --list

# List by tag
keep --list my-tag

# List with metadata filter
keep --list --meta env=prod

# Custom column format
keep --list --list-format "id,time,size,tags"

# JSON output
keep --list --output-format json

# Human-readable sizes
keep --list --human-readable

Info Mode

Show detailed information about an item.

keep --info 1
keep --info my-tag
keep --info --meta key=value

Update Mode

Update an item's tags and metadata.

# Replace tags
keep --update 1 new-tag

# Update metadata
keep --update 1 --meta key=newvalue

# Remove metadata (no value)
keep --update 1 --meta key

Delete Mode

Delete items by ID.

keep --delete 1
keep --delete 1 2 3

Diff Mode

Show differences between two items.

keep --diff 1 2

Status Mode

Show system status and supported features.

keep --status
keep --status-plugins
keep --status --verbose

Filters

Apply transformations to item content during retrieval. Filters are chained with |.

# First 10 lines
keep --get 1 --filters "head_lines(10)"

# Skip first 5 lines, then grep
keep --get 1 --filters "skip_lines(5)|grep(pattern=error)"

# Strip ANSI escape codes
keep --get 1 --filters "strip_ansi"

# Last 100 bytes
keep --get 1 --filters "tail_bytes(100)"

Available Filters

Filter Description Parameters
head_bytes(n) First n bytes count
head_lines(n) First n lines count
tail_bytes(n) Last n bytes count
tail_lines(n) Last n lines count
skip_bytes(n) Skip first n bytes count
skip_lines(n) Skip first n lines count
grep(pattern) Filter matching lines pattern (regex)
strip_ansi Remove ANSI escape codes none

Set the KEEP_FILTERS environment variable to apply filters by default.

Configuration

Environment Variables

Variable Description Default
KEEP_DIR Storage directory ~/.keep
KEEP_CONFIG Config file path ~/.config/keep/config.yml
KEEP_COMPRESSION Compression algorithm lz4
KEEP_META_PLUGINS Meta plugins to use env
KEEP_FILTERS Default filter chain none
KEEP_LIST_FORMAT List column format built-in defaults
KEEP_SERVER_ADDRESS Server bind address 127.0.0.1
KEEP_SERVER_PORT Server port 21080
KEEP_SERVER_PASSWORD Server password none
KEEP_SERVER_PASSWORD_HASH Server password hash none

Any config setting can be overridden with KEEP__<SETTING> environment variables (double underscore separator).

Capture shell commands automatically by setting KEEP_META_command in your shell profile (see Shell Integration).

Configuration File

Default location: ~/.config/keep/config.yml

Generate a default configuration:

keep --generate-config > ~/.config/keep/config.yml
# Storage directory
dir: ~/.keep

# List view columns
list_format:
  - name: id
    label: "Item"
    align: right
  - name: time
    label: "Time"
    align: right
  - name: size
    label: "Size"
    align: right
  - name: tags
    label: "Tags"
    align: left

# Table styling
table_config:
  style: utf8_full
  content_arrangement: dynamic

# Default compression
compression_plugin:
  name: gzip

# Default meta plugins
meta_plugins:
  - name: env
  - name: digest
    options:
      algorithm: sha256

# Server settings
server:
  address: "127.0.0.1"
  port: 21080
  password: "secret"

human_readable: true
quiet: false
force: false

Server Mode

Start an HTTP REST API server for programmatic access.

# Start server on default address (127.0.0.1:21080)
keep --server

# Custom address and port
keep --server --server-address 0.0.0.0 --server-port 8080

# With authentication
keep --server --server-password mypassword

API Endpoints

Method Path Description
GET /api/status System status
GET /api/plugins/status Plugins status
GET /api/item/ List items (supports tags, order, start, count params)
POST /api/item/ Create item (body: raw content, query: tags, metadata)
GET /api/item/latest/content Latest item content
GET /api/item/latest/meta Latest item metadata
GET /api/item/{id} Item info by ID
GET /api/item/{id}/content Item content by ID
GET /api/item/{id}/meta Item metadata by ID
GET /api/item/{id}/info Item info by ID
DELETE /api/item/{id} Delete item by ID
GET /api/diff Diff two items (id_a, id_b params)

Authentication

# Bearer token
curl -H "Authorization: Bearer mypassword" http://localhost:21080/api/status

# Basic auth
curl -u keep:mypassword http://localhost:21080/api/status

When no password is configured, authentication is disabled.

Swagger UI

Build with the swagger feature to enable OpenAPI documentation:

cargo build --features server,swagger

Swagger UI available at /swagger, OpenAPI spec at /openapi.json.

MCP (Model Context Protocol)

AI assistant integration via the Model Context Protocol. Enable with the mcp feature.

cargo build --features server,mcp

MCP endpoint available at /mcp/sse when the server is running.

Available Tools

Tool Description Parameters
save_item Save new content content, tags[], metadata{}
get_item Get item by ID id
get_latest_item Get latest item tags[]
list_items List items tags[], limit, offset
search_items Search items tags[], metadata{}

Feature Flags

Feature Default Description
magic Yes File type detection via libmagic
lz4 Yes LZ4 compression (internal)
gzip Yes GZip compression (internal)
server No HTTP REST API server
mcp No Model Context Protocol support
swagger No Swagger UI for API docs
bzip2 No BZip2 compression (external program)
xz No XZ compression (external program)
zstd No ZStd compression (external program)
# Build with server and swagger
cargo build --features server,swagger

# Build with all features
cargo build --features server,mcp,swagger,magic

Shell Integration

Source profile.bash to enable shell integration:

source /path/to/keep/profile.bash

This provides:

  • keep function - Captures the current command in metadata automatically
  • @ alias - Shorthand for keep --save
  • @@ alias - Shorthand for keep --get
# Save with automatic command capture
curl -s api.example.com | @ api-response

# Quick retrieve
@@ api-response

Compression

Items are compressed automatically on save. Default: LZ4.

Algorithm Type Speed Ratio
lz4 Internal Fastest Lower
gzip Internal Fast Good
bzip2 External Slow Better
xz External Slowest Best
zstd External Fast Good
none Internal N/A N/A
# Specify compression per item
echo "data" | keep --save my-tag --compression zstd

# Set default via environment
export KEEP_COMPRESSION=gzip

External compression programs (bzip2, xz, zstd) must be installed on the system.

Meta Plugins

Metadata is automatically extracted when saving items.

Plugin Key Description
env * Capture KEEP_META_* environment variables
magic_file file_type File type detection (requires magic feature)
text text_line_count, text_word_count Line and word counts
user uid, user, gid, group Current user info
shell shell Current shell path
shell_pid shell_pid Shell process ID
keep_pid keep_pid Keep process ID
digest digest_sha256, digest_md5 Content digests
read_time read_time Time to read content
read_rate read_rate Data read rate
hostname hostname, hostname_short System hostname
exec Custom Run external commands for metadata
cwd cwd Current working directory
# Use specific plugins
echo "data" | keep --save tag --meta-plugins "digest,text,user"

# Capture custom metadata via environment
KEEP_META_project=alpha echo "data" | keep --save tag

License

MIT License - see LICENSE for details.

Contact

Andrew Phillips - andrew@gt0.ca

Description
No description provided
Readme MIT 13 MiB
Languages
Rust 99.5%
Shell 0.3%
Dockerfile 0.2%