# Keep

A command-line utility for managing temporary files with automatic compression, metadata generation, and querying. Instead of redirecting output to temporary files, pipe it into `keep` for organized storage and retrieval.

```sh
# Instead of this:
curl -s https://api.example.com/data > /tmp/api-data.json
cat /tmp/api-data.json

# Do this:
curl -s https://api.example.com/data | keep api-data
keep --get api-data
```

## Features

- **Store and Retrieve** - Save content with tags, retrieve by ID or tag
- **Automatic Compression** - LZ4, GZip, BZip2, XZ, ZStd support
- **Metadata Plugins** - Auto-extract file type, digests, hostname, user info, and more
- **Filters** - Apply transformations (head, tail, grep, strip ANSI) on retrieval
- **Querying** - List, search, diff items with flexible formatting
- **REST API Server** - Optional HTTP server for programmatic access
- **MCP Support** - Model Context Protocol integration for AI assistants
- **Modular Design** - Extensible plugin system for compression, metadata, and filtering

## Installation

### From Source

Requires Rust and Cargo.

```sh
cargo build --release
cargo install --path .
```

### Static Binary (Linux)

```sh
./build-static.bash
# Binary at bin/keep
```

### Using Cargo

```sh
cargo install --path .
```

## Quick Start

```sh
# Save content with tags
echo "Hello, world!" | keep --save greeting

# Retrieve by tag
keep --get greeting

# List all items
keep --list

# Get item details
keep --info greeting

# Delete by tag
keep --delete greeting
```

## Usage

### Save Mode

Save stdin content with tags and metadata.

```sh
# Basic save (auto-assigned ID)
echo "data" | keep --save

# Save with tags
echo "data" | keep --save my-tag

# Save with multiple tags and metadata
cat report.pdf | keep --save report --meta project=alpha --meta env=prod

# Specify compression and digest
echo "data" | keep --save my-tag --compression gzip --digest sha256
```

### Get Mode

Retrieve items by ID or tags. Default mode when IDs are provided.

```sh
# Get by ID
keep --get 1
keep 1

# Get by tag
keep --get my-tag
keep my-tag

# Get with filters
keep --get 1 --filters "head_lines(10)"

# Get with metadata filter
keep --get --meta project=alpha

# Force binary output to TTY
keep --get 1 --force
```

### List Mode

List stored items with filtering and formatting.

```sh
# List all items
keep --list

# List by tag
keep --list my-tag

# List with metadata filter
keep --list --meta env=prod

# Custom column format
keep --list --list-format "id,time,size,tags"

# JSON output
keep --list --output-format json

# Human-readable sizes
keep --list --human-readable
```

### Info Mode

Show detailed information about an item.

```sh
keep --info 1
keep --info my-tag
keep --info --meta key=value
```

### Update Mode

Update an item's tags and metadata.

```sh
# Replace tags
keep --update 1 new-tag

# Update metadata
keep --update 1 --meta key=newvalue

# Remove metadata (no value)
keep --update 1 --meta key
```

### Delete Mode

Delete items by ID.

```sh
keep --delete 1
keep --delete 1 2 3
```

### Diff Mode

Show differences between two items.

```sh
keep --diff 1 2
```

### Status Mode

Show system status and supported features.

```sh
keep --status
keep --status-plugins
keep --status --verbose
```

## Filters

Apply transformations to item content during retrieval. Filters are chained with `|`.

```sh
# First 10 lines
keep --get 1 --filters "head_lines(10)"

# Skip first 5 lines, then grep
keep --get 1 --filters "skip_lines(5)|grep(pattern=error)"

# Strip ANSI escape codes
keep --get 1 --filters "strip_ansi"

# Last 100 bytes
keep --get 1 --filters "tail_bytes(100)"
```

### Available Filters

| Filter | Description | Parameters |
|--------|-------------|------------|
| `head_bytes(n)` | First n bytes | `count` |
| `head_lines(n)` | First n lines | `count` |
| `tail_bytes(n)` | Last n bytes | `count` |
| `tail_lines(n)` | Last n lines | `count` |
| `skip_bytes(n)` | Skip first n bytes | `count` |
| `skip_lines(n)` | Skip first n lines | `count` |
| `grep(pattern)` | Filter matching lines | `pattern` (regex) |
| `strip_ansi` | Remove ANSI escape codes | none |

Set the `KEEP_FILTERS` environment variable to apply filters by default.

## Configuration

### Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `KEEP_DIR` | Storage directory | `~/.keep` |
| `KEEP_CONFIG` | Config file path | `~/.config/keep/config.yml` |
| `KEEP_COMPRESSION` | Compression algorithm | `lz4` |
| `KEEP_META_PLUGINS` | Meta plugins to use | `env` |
| `KEEP_FILTERS` | Default filter chain | none |
| `KEEP_LIST_FORMAT` | List column format | built-in defaults |
| `KEEP_SERVER_ADDRESS` | Server bind address | `127.0.0.1` |
| `KEEP_SERVER_PORT` | Server port | `21080` |
| `KEEP_SERVER_PASSWORD` | Server password | none |
| `KEEP_SERVER_PASSWORD_HASH` | Server password hash | none |

Any config setting can be overridden with `KEEP__<SETTING>` environment variables (double underscore separator).

Capture shell commands automatically by setting `KEEP_META_command` in your shell profile (see Shell Integration).

### Configuration File

Default location: `~/.config/keep/config.yml`

Generate a default configuration:

```sh
keep --generate-config > ~/.config/keep/config.yml
```

```yaml
# Storage directory
dir: ~/.keep

# List view columns
list_format:
  - name: id
    label: "Item"
    align: right
  - name: time
    label: "Time"
    align: right
  - name: size
    label: "Size"
    align: right
  - name: tags
    label: "Tags"
    align: left

# Table styling
table_config:
  style: utf8_full
  content_arrangement: dynamic

# Default compression
compression_plugin:
  name: gzip

# Default meta plugins
meta_plugins:
  - name: env
  - name: digest
    options:
      algorithm: sha256

# Server settings
server:
  address: "127.0.0.1"
  port: 21080
  password: "secret"

human_readable: true
quiet: false
force: false
```

## Server Mode

Start an HTTP REST API server for programmatic access.

```sh
# Start server on default address (127.0.0.1:21080)
keep --server

# Custom address and port
keep --server --server-address 0.0.0.0 --server-port 8080

# With authentication
keep --server --server-password mypassword
```

### API Endpoints

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/status` | System status |
| `GET` | `/api/plugins/status` | Plugins status |
| `GET` | `/api/item/` | List items (supports `tags`, `order`, `start`, `count` params) |
| `POST` | `/api/item/` | Create item (body: raw content, query: `tags`, `metadata`) |
| `GET` | `/api/item/latest/content` | Latest item content |
| `GET` | `/api/item/latest/meta` | Latest item metadata |
| `GET` | `/api/item/{id}` | Item info by ID |
| `GET` | `/api/item/{id}/content` | Item content by ID |
| `GET` | `/api/item/{id}/meta` | Item metadata by ID |
| `GET` | `/api/item/{id}/info` | Item info by ID |
| `DELETE` | `/api/item/{id}` | Delete item by ID |
| `GET` | `/api/diff` | Diff two items (`id_a`, `id_b` params) |

### Authentication

```sh
# Bearer token
curl -H "Authorization: Bearer mypassword" http://localhost:21080/api/status

# Basic auth
curl -u keep:mypassword http://localhost:21080/api/status
```

When no password is configured, authentication is disabled.

### Swagger UI

Build with the `swagger` feature to enable OpenAPI documentation:

```sh
cargo build --features server,swagger
```

Swagger UI available at `/swagger`, OpenAPI spec at `/openapi.json`.

## MCP (Model Context Protocol)

AI assistant integration via the Model Context Protocol. Enable with the `mcp` feature.

```sh
cargo build --features server,mcp
```

MCP endpoint available at `/mcp/sse` when the server is running.

### Available Tools

| Tool | Description | Parameters |
|------|-------------|------------|
| `save_item` | Save new content | `content`, `tags[]`, `metadata{}` |
| `get_item` | Get item by ID | `id` |
| `get_latest_item` | Get latest item | `tags[]` |
| `list_items` | List items | `tags[]`, `limit`, `offset` |
| `search_items` | Search items | `tags[]`, `metadata{}` |

## Feature Flags

| Feature | Default | Description |
|---------|---------|-------------|
| `magic` | Yes | File type detection via libmagic |
| `lz4` | Yes | LZ4 compression (internal) |
| `gzip` | Yes | GZip compression (internal) |
| `server` | No | HTTP REST API server |
| `mcp` | No | Model Context Protocol support |
| `swagger` | No | Swagger UI for API docs |
| `bzip2` | No | BZip2 compression (external program) |
| `xz` | No | XZ compression (external program) |
| `zstd` | No | ZStd compression (external program) |

```sh
# Build with server and swagger
cargo build --features server,swagger

# Build with all features
cargo build --features server,mcp,swagger,magic
```

## Shell Integration

Source `profile.bash` to enable shell integration:

```sh
source /path/to/keep/profile.bash
```

This provides:

- **`keep` function** - Captures the current command in metadata automatically
- **`@` alias** - Shorthand for `keep --save`
- **`@@` alias** - Shorthand for `keep --get`

```sh
# Save with automatic command capture
curl -s api.example.com | @ api-response

# Quick retrieve
@@ api-response
```

## Compression

Items are compressed automatically on save. Default: LZ4.

| Algorithm | Type | Speed | Ratio |
|-----------|------|-------|-------|
| `lz4` | Internal | Fastest | Lower |
| `gzip` | Internal | Fast | Good |
| `bzip2` | External | Slow | Better |
| `xz` | External | Slowest | Best |
| `zstd` | External | Fast | Good |
| `none` | Internal | N/A | N/A |

```sh
# Specify compression per item
echo "data" | keep --save my-tag --compression zstd

# Set default via environment
export KEEP_COMPRESSION=gzip
```

External compression programs (`bzip2`, `xz`, `zstd`) must be installed on the system.

## Meta Plugins

Metadata is automatically extracted when saving items.

| Plugin | Key | Description |
|--------|-----|-------------|
| `env` | `*` | Capture `KEEP_META_*` environment variables |
| `magic_file` | `file_type` | File type detection (requires `magic` feature) |
| `text` | `text_line_count`, `text_word_count` | Line and word counts |
| `user` | `uid`, `user`, `gid`, `group` | Current user info |
| `shell` | `shell` | Current shell path |
| `shell_pid` | `shell_pid` | Shell process ID |
| `keep_pid` | `keep_pid` | Keep process ID |
| `digest` | `digest_sha256`, `digest_md5` | Content digests |
| `read_time` | `read_time` | Time to read content |
| `read_rate` | `read_rate` | Data read rate |
| `hostname` | `hostname`, `hostname_short` | System hostname |
| `exec` | Custom | Run external commands for metadata |
| `cwd` | `cwd` | Current working directory |

```sh
# Use specific plugins
echo "data" | keep --save tag --meta-plugins "digest,text,user"

# Capture custom metadata via environment
KEEP_META_project=alpha echo "data" | keep --save tag
```

## License

MIT License - see [LICENSE](LICENSE) for details.

## Contact

Andrew Phillips - andrew@gt0.ca