joshbeard/jendex

By joshbeard

•Updated 11 months ago

JSON-based API for managing arbitrary data.

Image

928

Overview Tags

joshbeard/jendex repository overview

⁠Jendex

Lightweight REST API server for file-based data storage with automatic metadata management and querying.

Jendex exposes REST endpoints backed by filesystem or S3 storage. Drop JSON/YAML/CSV files into local directories or S3 buckets, or manage data through the REST API. Metadata (ID, timestamps) is derived from file properties or S3 object metadata. No schema or database required.

⁠Features

REST API backed by filesystem or S3 storage
Schema-free: JSON, YAML, or CSV files
Automatic metadata from file properties
Response format control via ?raw=true query parameter
Query filtering via URL parameters
Bearer token authentication (API) and basic auth (UI)
Optional web UI

⁠Installation

⁠Pre-built Packages

Binaries and packages for macOS, Linux (.deb, .rpm, .apk), and FreeBSD are available from GitHub releases⁠.

⁠Homebrew (macOS)

brew tap joshbeard/jendex
brew install jendex
jendex --config config.yaml

⁠Install Script

curl -sfL https://raw.githubusercontent.com/joshbeard/jendex/master/install.sh | sh -

⁠From Source

go mod download
go build -o jendex ./cmd/server
./jendex --config config.yaml

⁠Docker

docker run -p 8080:8080 -v $(pwd)/config.yaml:/app/config.yaml jendex:latest

⁠Configuration

Configuration is via YAML file. See config.yaml⁠ for options including authentication, storage backends, API routing, and web UI settings.

⁠API Reference

All endpoints except /health require authentication.

Method	Endpoint	Description
`GET`	`/health`	Health check
`GET`	`/api/collections`	List all collections
`GET`	`/api/collection/:collection`	Get all items in collection
`GET`	`/api/collection/:collection?key=val`	Query items with filters
`GET`	`/api/collection/:collection/:id`	Get item by ID
`POST`	`/api/collection/:collection`	Create item
`PUT`	`/api/collection/:collection/:id`	Update item
`DELETE`	`/api/collection/:collection/:id`	Delete item

Query Parameters:

?raw=true - Return data without metadata wrapper (works with GET collection and item endpoints)
?key=value - Filter items by data fields (can combine multiple filters)

⁠Web UI

Optional web interface for CRUD operations on collections and items. Enable in config.yaml:

ui:
  enabled: true
  basic_auth:
    users:
      - username: admin
        password: changeme
        permissions: ["read", "write"]
        collections: ["**"]

UI uses basic authentication (separate from API bearer tokens) with the same permission scoping model as the API.

⁠Authentication

API: Bearer token authentication
UI: Basic authentication
Permissions: Both support scoped access by collection and operation (read/write)

Configure in config.yaml.

⁠Quick Start

# Create collection directory
mkdir -p ./data/services

# Drop in a JSON file
cat > ./data/services/postgres.json << 'EOF'
{
  "name": "PostgreSQL",
  "version": "15.3",
  "port": 5432
}
EOF

# Start server
jendex --config config.yaml

# Query via API (metadata added in response)
curl "http://localhost:8080/api/collection/services" -H "Authorization: Bearer your-key"
# [{"id":"postgres","data":{"name":"PostgreSQL",...},"created_at":"...","updated_at":"..."}]

# Get raw data
curl "http://localhost:8080/api/collection/services/postgres?raw=true" -H "Authorization: Bearer your-key"
# {"name":"PostgreSQL","version":"15.3","port":5432}

# Create via API
curl -X POST http://localhost:8080/api/collection/services \
  -H "Authorization: Bearer your-key" \
  -d '{"name":"Redis","version":"7.2","port":6379}'

⁠Data Storage

⁠Storage Backends

Filesystem: Local directories and files S3: AWS S3 (or compatible) buckets and objects

Both backends support JSON, YAML, and CSV formats.

⁠Architecture

Collections are directories (filesystem) or S3 prefixes. Each item is a single file or S3 object.

Files/objects contain raw data only - no metadata wrapper. Your data structure is the storage structure:

{
  "name": "PostgreSQL",
  "version": "15.3",
  "port": 5432
}

Metadata is derived at query time:

id: Filename (filesystem) or object key (S3), without extension
created_at, updated_at: File modification time (filesystem) or S3 object metadata

⁠Storage Structure

Filesystem example:

data/
├── services/
│   ├── postgres.json
│   ├── redis.yaml
│   └── nginx.json
└── devices/
    ├── server-01.json
    └── server-02.json

S3 example:

s3://my-bucket/prefix/
├── services/postgres.json
├── services/redis.yaml
├── services/nginx.json
├── devices/server-01.json
└── devices/server-02.json

⁠Response Formats

GET requests return structured metadata by default, or raw data with ?raw=true:

# Default: Wrapped with metadata
curl "http://localhost:8080/api/collection/services/postgres"
# {"id":"postgres","data":{"name":"PostgreSQL",...},"created_at":"...","updated_at":"..."}

# Raw: Data only
curl "http://localhost:8080/api/collection/services/postgres?raw=true"
# {"name":"PostgreSQL","version":"15.3","port":5432}

API writes (POST/PUT) accept raw data and store it as-is. Metadata is never stored in files/objects - it's always derived at query time.

⁠Usage Patterns

⁠Direct File Storage

# Create collection directory
mkdir -p ./data/services

# Drop in raw JSON file
cat > ./data/services/postgres.json << 'EOF'
{
  "name": "PostgreSQL",
  "version": "15.3",
  "port": 5432
}
EOF

# Or YAML
cat > ./data/services/redis.yaml << 'EOF'
name: Redis
version: 7.2
port: 6379
EOF

# Access via API immediately
curl "http://localhost:8080/api/collection/services" \
  -H "Authorization: Bearer $TOKEN"

⁠API-Based Management

# Create item
curl -X POST http://localhost:8080/api/collection/services \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"name":"nginx","version":"1.24","port":80}'

# Update (overwrites file)
curl -X PUT http://localhost:8080/api/collection/services/nginx \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"name":"nginx","version":"1.25","port":80}'

# Get raw data
curl "http://localhost:8080/api/collection/services/nginx?raw=true" \
  -H "Authorization: Bearer $TOKEN"

Both approaches work together - files you create manually and items created via API are accessible the same way.

⁠CI/CD Integration

script:
  - curl -X POST ${JENDEX_API}/api/collection/services \
      -H "Authorization: Bearer ${JENDEX_API_KEY}" \
      -H "Content-Type: application/json" \
      -d @catalog.json

⁠Similar Projects

json-server⁠ - Mock REST API from JSON files
JSONBin.io⁠ - Hosted JSON storage
lowdb⁠ - JSON database library for Node.js

Jendex focuses on production use with authentication, S3 support, and transparent file-based storage where metadata is derived rather than stored.

⁠Contributing

See DEVELOPMENT.md⁠.

⁠License

MIT⁠

Tag summary

Recent tags

Content type

Image

Digest

sha256:1406c6b60…

Size

13.9 MB

Last updated

11 months ago

docker pull joshbeard/jendex