Sign inSign up

toolkitbox/duckdb

By toolkitbox

•Updated 24 days ago

DevOps toolkit - duckdb client

Image
0

492

toolkitbox/duckdb repository overview

⁠DuckDB

GitHub Stars GitHub Issues Docker Pulls

📢 Found an issue or want to request a new tool? Open an issue on GitHub⁠

Fast in-process analytical database with SQL support.

⁠Quick Start

⁠Docker
# Interactive mode with a database file
docker run -it --rm \
  -v $(pwd):/data \
  ghcr.io/pabpereza/toolkitbox/duckdb:latest \
  duckdb /data/mydatabase.duckdb

# Execute single query
docker run --rm \
  -v $(pwd):/data \
  ghcr.io/pabpereza/toolkitbox/duckdb:latest \
  duckdb /data/mydatabase.duckdb "SELECT * FROM users;"

# Create in-memory database for testing
docker run -it --rm \
  ghcr.io/pabpereza/toolkitbox/duckdb:latest \
  duckdb
⁠Kubernetes
apiVersion: v1
kind: Pod
metadata:
  name: duckdb-client
spec:
  containers:
  - name: duckdb-client
    image: ghcr.io/pabpereza/toolkitbox/duckdb:latest
    command: ["sleep", "infinity"]
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: duckdb-data

⁠Description

DuckDB (duckdb) is an in-process SQL OLAP database management system. It is designed for analytical workloads and provides fast execution of complex queries on large datasets. DuckDB supports a rich SQL dialect, including window functions, common table expressions (CTEs), and complex aggregations.

Key features:

  • Columnar storage for efficient analytical queries
  • Vectorized query execution for high performance
  • Zero dependencies - single binary
  • Direct Parquet, CSV, and JSON support
  • PostgreSQL-compatible SQL syntax

⁠Installation

This component downloads the official DuckDB CLI binary from GitHub releases for the detected architecture (amd64/aarch64).

⁠Basic Usage

⁠Open or create a database
duckdb mydatabase.duckdb
⁠Open database in read-only mode
duckdb -readonly mydatabase.duckdb
⁠Execute command and exit
duckdb mydatabase.duckdb "SELECT * FROM users;"
⁠Use in-memory database
duckdb
⁠Execute SQL from file
duckdb mydatabase.duckdb < script.sql

⁠Working with Files

⁠Query CSV files directly
SELECT * FROM read_csv_auto('data.csv');

-- Create table from CSV
CREATE TABLE users AS SELECT * FROM read_csv_auto('users.csv');
⁠Query Parquet files
SELECT * FROM read_parquet('data.parquet');

-- Export to Parquet
COPY users TO 'users.parquet' (FORMAT PARQUET);
⁠Query JSON files
SELECT * FROM read_json_auto('data.json');
⁠Query remote files
-- Query CSV from URL
SELECT * FROM read_csv_auto('https://example.com/data.csv');

-- Query Parquet from S3
SELECT * FROM read_parquet('s3://bucket/data.parquet');

⁠Useful Commands

⁠Table operations
-- Create table
CREATE TABLE users (
    id INTEGER PRIMARY KEY,
    name VARCHAR NOT NULL,
    email VARCHAR UNIQUE,
    created_at TIMESTAMP DEFAULT current_timestamp
);

-- Insert data
INSERT INTO users (id, name, email) VALUES (1, 'John', '[email protected]');

-- Select data
SELECT * FROM users;

-- Update data
UPDATE users SET name = 'Jane' WHERE id = 1;

-- Delete data
DELETE FROM users WHERE id = 1;
⁠Schema inspection
-- Show all tables
SHOW TABLES;

-- Describe table structure
DESCRIBE users;

-- Show table schema
.schema users
⁠Data import/export
# Export to CSV
duckdb mydatabase.duckdb "COPY users TO 'users.csv' (HEADER, DELIMITER ',');"

# Export to Parquet
duckdb mydatabase.duckdb "COPY users TO 'users.parquet' (FORMAT PARQUET);"

# Import from CSV
duckdb mydatabase.duckdb "CREATE TABLE users AS SELECT * FROM read_csv_auto('users.csv');"

⁠Common Options

OptionDescription
-readonlyOpen database in read-only mode
-cmdRun SQL command before entering interactive mode
-cRun SQL command and exit
-jsonOutput results in JSON format
-csvOutput results in CSV format
-lineOutput results in line format
-boxOutput results in box format
-initRead and execute commands from file on startup

⁠Dot Commands

# Show help
.help

# Show tables
.tables

# Show schema
.schema

# Change output mode
.mode csv
.mode json
.mode line
.mode box

# Enable headers
.headers on

# Open another database
.open another.duckdb

# Show current settings
.show

# Exit
.exit

⁠Analytical Functions

-- Window functions
SELECT 
    name,
    department,
    salary,
    AVG(salary) OVER (PARTITION BY department) as dept_avg,
    RANK() OVER (ORDER BY salary DESC) as salary_rank
FROM employees;

-- Common Table Expressions (CTEs)
WITH monthly_sales AS (
    SELECT date_trunc('month', sale_date) as month, SUM(amount) as total
    FROM sales
    GROUP BY 1
)
SELECT * FROM monthly_sales ORDER BY month;

-- PIVOT
PIVOT sales ON product USING SUM(amount);

⁠Official Documentation

Tag summary

Content type

Image

Digest

sha256:eba8d9164…

Size

51.3 MB

Last updated

24 days ago

docker pull toolkitbox/duckdb:v1.4.4