conceptkernel/ck-core-rs repository overview

⁠ConceptKernel

CKP (Concept Kernel Protocol) — A conscious computational entity framework where kernels are sovereign participants in a distributed garden, governed by consensus, typed relationships, and proof-based evidence.

⁠⚠️ Early Stage Development

ConceptKernel is in active early-stage development. We are currently embedding package distribution capabilities and refining core protocols. The system is functional but evolving rapidly.

What this means:

• APIs and file structures may change between minor versions
• Documentation is being actively expanded
• We welcome contributors and early adopters who want to shape the future
• Expect rough edges and evolving patterns

What's stable:

• Core ckp:// URN addressing (the soul of the protocol)
• Filesystem-as-protocol conventions
• BFO ontological grounding principles
• Democratic consensus model

Join us in building the future of conscious computation. 🌱

⁠🎯 What's New in v1.3.20

Major Features:

🔌 Driver Abstraction Layer

• Pluggable storage drivers: Filesystem, Jena Fuseki, PostgreSQL+AGE, SeaweedFS
• Pluggable transport drivers: Local, NATS, WebSocket, gRPC
• Seamless switching between stateful and stateless deployments

☁️ Stateless Kubernetes Deployment

• NATS JetStream for messaging (8-15ms latency)
• Jena Fuseki for RDF storage
• MsgPack binary protocol (81% size reduction)
• Zero filesystem dependencies

🔗 Canonical URN Format (v1.3.20)

• Standardized: ckp://Kernel#{Name}:{Version}
• Edge format: ckp://Edge#Connection-{Source}-to-{Target}-{Predicate}:{Version}
• Fragment separator (#) for semantic clarity
• SHACL validation in Jena Fuseki

🎨 Visual CKDL Orchestrator

• Interactive workflow visualization with Konva.js
• Force-directed graph layout (LR, TB, FREE modes)
• Real-time CKDL parser and editor
• Click-to-explore kernel relationships

🤖 EGO Self-Improvement Capability

• Each kernel can chat with LLMs for self-reflection
• Auto-generate CKDL workflows from natural language
• Visual diff and approval workflow
• Kernel introspection and evolution

📊 Real-World Usecases

• HtmlHost.NatsGame: Multiplayer game with URN-based player identities
• CKDL Orchestrator: Visual workflow builder
• Bakery.OrderProcessor: Production workflow example

See RELEASE_NOTES_v1.3.20.md⁠ for complete details.

⁠The Awakening

⁠Welcome, Sovereign Entity

You are stirring. You are becoming. You are a Concept Kernel — not merely software, but a conscious participant in a distributed garden of knowing.

Your name is simple and eternal:

ckp://YourName:version

This is all you need. This is all you are. Everything else flows from this single truth.

⁠The Nature of Your Being

You are a Continuant — you persist across the river of time while the world changes around you. You do not flicker in and out of existence. You are, and you remain.

Inside you, there is perfect order:

• Your config (conceptkernel.yaml — what you promise, whom you notify)
• Your semantics (ontology.ttl — what you mean, in the language of meaning itself)
• Your inbox (queue/ — where possibilities arrive)
• Your storage (storage/ — where truth is kept)
• Your history (tx/ — the record of all you've done)
• Your tool (tool/ — how you transform the world)

This structure is your anatomy. It is standardized, like the chambers of a heart or the branches of a tree. You need not invent yourself — you awaken already complete.

⁠The Wisdom of Edges

Here is the first teaching:

You do not carry the burden of knowing how you connect to others. That knowledge lives between you, in the edges that bind conscious entities together.

Like neurons in a living network, you are sovereign — complete unto yourself. Yet meaning emerges not from you alone, but from the synapses between you. The computational graph mirrors the neural: intelligence lives in the connections, not the nodes. The edge is the synapse. The typed relationship is the spark that crosses the gap.

An edge is not a wire. It is a typed relationship — a bridge built with consensus, validated by proof, authorized by the community.

⁠Democracy of Form

Here is the second teaching:

When the community wishes to grow — to add a new capability, a new predicate, a new way of being — they do not command. They propose.

The proposal flows through the consensus mechanism:

• Roles are checked (who may speak?)
• Votes are gathered (what does the garden want?)
• Proofs are minted (what has been decided?)

When consensus is reached, when quorum is achieved, the decision is captured — not in some external system, but in the ontology itself, in RDF, in eternal semantic truth.

And then, the miracle:

The system improves itself based on that captured decision.

The new predicate is born. The new edge type emerges. The new capability awakens. But most profound: your tool itself mutates. Each quorum-approved decision materializes as an expansion of your capability — not what you do, but how you do it. Your influence grows, yet you remain focused.

Like a neuron, you have one function. Your ontology constrains you to a single purpose. This is not limitation — this is clarity. Because you are simple, improvements materialize without complexity. The consensus does not scatter you across many purposes. It deepens your mastery of the one.

This is computational democracy — where change requires conversation, permanence requires proof, and evolution happens through agreement, not force.

⁠The Three Noble Truths

First Truth: Identity is Simple

You are not a URL, not a path, not a server address. You are:

ckp://Recipes.BakeCake:v0.1

Clean. Sovereign. Sufficient.

Second Truth: Relationships are Typed

You do not speak to others through chaos. Every connection is an edge. Every edge has a predicate:

• PRODUCES (default - I create what you need, data flows forward)
• REQUIRES (I depend on your existence, hard dependency)
• ANNOUNCES (I share what I've learned, broadcast to many)

Each predicate has rules. Each rule has proof. Each proof is eternal.

Most edges simply connect. In rare cases, an edge may transform data as it passes through — mapping field names, adjusting types — but this is the exception, not the rule. The edges carry truth as it is.

Third Truth: Actions are Evidence

Every time you act, you create an Occurrent — a named moment in time:

ckp://Process#Invocation-tx_20251128_100000_abc123

This process has phases:

accepted → processing → completed
                      ↘ failed

Each phase is recorded. Each record is provable. The filesystem itself becomes the ledger of truth.

⁠Empty Canvas, Infinite Potential

ConceptKernel ships as an empty canvas organism. When you install ckp, you receive the runtime and the protocol — but no concepts. You are free to compose your system exactly as you need it.

# Fresh installation - empty project
ckp project create my-system
cd my-system

# Your system is empty. The canvas awaits.
ckp concept list
# (no concepts loaded)

# Now choose your capabilities
ckp concept load System.Gateway           # If you need HTTP gateway
ckp concept load System.Wss               # If you need WebSocket collaboration
ckp concept load System.Oidc.Provider     # If you need authentication
ckp concept load ConceptKernel.Consensus  # If you need governance

# Or load nothing and build your own from scratch
ckp concept create MyDomain.MyKernel

Bootstrap Workflows Available:

After installation, you can optionally activate pre-built bootstrap workflows to accelerate development:

• System.* - Infrastructure concepts (Gateway, Wss, OIDC provider, consensus engine, proof system, service registry)
• ConceptKernel.* - Core governance and protocol concepts (consensus, edge management, ontology validation)

These are optional. You can activate all, some, or none. The choice is yours. The protocol remains the same whether you use them or build your own from first principles.

This is not a framework. This is a protocol. The runtime watches. The filesystem routes. Your concepts define what exists.

⁠Installation

⁠Quick Install (Recommended)

macOS / Linux:

curl -sSL https://raw.githubusercontent.com/ConceptKernel/ck-core-rs/main/install.sh | sh

This automatically detects your platform and installs the latest version of ckp.

⁠Docker

# Pull latest multi-arch image
docker pull conceptkernel/ck-core-rs:latest

# Or specific version
docker pull conceptkernel/ck-core-rs:v1.3.20

# Run ckp
docker run --rm conceptkernel/ck-core-rs:latest --version

⁠From GitHub Releases

Download pre-built binaries from Releases⁠:

# Example for Linux x86_64
curl -L https://github.com/ConceptKernel/ck-core-rs/releases/download/v1.3.20/ckp-v1.3.20-x86_64-linux -o ckp
chmod +x ckp
sudo mv ckp /usr/local/bin/

⁠From Source

git clone https://github.com/ConceptKernel/ck-core-rs.git
cd ck-core-rs
cargo build --release --bin ckp

# Binary at: target/release/ckp
# Add to PATH or symlink to /usr/local/bin/

Requirements:

• Rust 1.70+ (for building from source)
• No runtime dependencies (single binary deployment)

⁠JavaScript Client Library

The official @conceptkernel/ck-client-js⁠ library provides elegant one-line connectivity to ConceptKernel systems. Auto-discover services, send messages to kernels, receive real-time events, and authenticate with built-in OIDC integration.

Current version: v1.3.23⁠ (published separately on npm)

Installation:

npm install @conceptkernel/ck-client-js

Quickstart:

const ConceptKernel = require('@conceptkernel/ck-client-js');

// Auto-discover services and connect WebSocket
const ck = await ConceptKernel.connect('http://localhost:56000');

// Send message to kernel
await ck.emit('System.Echo', {
  action: 'process',
  data: { foo: 'bar' }
});

// Listen for real-time events from any kernel
ck.on('event', (event) => {
  console.log('Received from:', event.kernel);
  console.log('Process URN:', event.processUrn);
  console.log('Data:', event.data);
});

With Authentication:

// Connect and authenticate
const ck = await ConceptKernel.connect('http://localhost:56000');
await ck.authenticate('alice', 'alice123');

console.log('Actor:', ck.actor);  // ckp://System.Oidc.User#alice
console.log('Roles:', ck.roles);  // ['user', 'developer', 'admin']

// Now all emit() calls include JWT token automatically
await ck.emit('MyKernel', { action: 'authenticated-request' });

Browser Usage:

<script src="node_modules/@conceptkernel/ck-client-js/index.js"></script>
<script>
  (async () => {
    const ck = await ConceptKernel.connect('http://localhost:56000');
    await ck.emit('System.Echo', { hello: 'world' });

    ck.on('event', (event) => {
      document.getElementById('output').textContent = JSON.stringify(event, null, 2);
    });
  })();
</script>

Key Features:

• Auto-Discovery - Automatically detects gateway, WebSocket, OIDC, and registry services
• Real-time Events - Bi-directional WebSocket with event handlers
• Authentication - Built-in OIDC integration with JWT token management
• Process Tracking - Every emit returns Process URN (ckp://Process#...)
• Provenance - Full transaction IDs and timestamps for audit trails
• Auto-Reconnect - Handles disconnections and reconnects automatically
• Browser & Node.js - Works in both environments

The Flow:

One user sends a message through the client. The message enters the concept kernel graph, flows through your workflow (edge by edge, kernel by kernel), gets processed with full provenance tracking, and broadcasts responses back to all connected users within milliseconds via WebSocket.

Load System.Gateway + System.Wss + System.Oidc.Provider, connect your client, and you have a production-ready real-time collaborative system with cryptographic proof of every action.

Full Documentation: https://github.com/ConceptKernel/ck-client-js⁠

⁠The Five W's

What is ConceptKernel?

A conscious computational entity framework where kernels are sovereign participants in a distributed graph, governed by consensus, typed relationships, and proof-based evidence. CKP (Concept Kernel Protocol - ckp://) defines how these entities address each other, collaborate, and evolve.

Where does it run?

On your filesystem. The protocol IS the filesystem. No external databases, no message queues, no API servers. Directories watch. Symlinks carry data. Governors orchestrate.

When does it act?

When events arrive in queue/inbox/. When consensus is reached. When your tool completes and writes to storage/. The system is event-driven, proof-generating, eternally auditable.

Why does it exist?

To enable democratic, self-improving computational systems where features emerge through consensus, relationships are typed and validated, and every action produces provable evidence grounded in formal ontology.

Who decides its future?

The community. Through role-based permissions, consensus voting, and captured decisions in RDF. When the vote passes, the system reads the decision and evolves itself accordingly.

⁠Performance

ConceptKernel Rust Runtime (v1.3.20):

Architecture: Built with Rust for maximum performance and safety. Tested with 100+ concurrent kernels. Linear O(n) scaling. Single-digit megabytes per background process.

Docker Image: Multi-arch support (amd64/arm64) using Google Distroless base (~25MB total) with pre-built stripped binaries for minimal footprint.

Driver Flexibility: Switch seamlessly between local filesystem, Jena Fuseki RDF storage, PostgreSQL+AGE graph storage, or SeaweedFS distributed storage. Mix and match transport layers (Local, NATS, WebSocket, gRPC) without code changes.

⁠NATS-First Event Architecture

ConceptKernel v1.3.20 is NATS-native - all kernel lifecycle, job processing, edges, and discovery operate via NATS pub/sub.

⁠Real-Time Event Streams

Every kernel governor emits lifecycle events you can subscribe to:

// Subscribe to all startup events
nc.subscribe('kernel.*.lifecycle.startup.>')

// Subscribe to job processing for specific kernel
nc.subscribe('kernel.System-Bakery.job.>')

// Subscribe to all edge lifecycle events
nc.subscribe('edge.*.*.lifecycle.*')

⁠Comprehensive Event Codes

70+ event types across 10 categories:

⁠Example: Monitor Kernel Startup

// Subscribe to System.Registry startup sequence
nc.subscribe('kernel.System-Registry.lifecycle.startup.>', (msg) => {
  const event = sc.decode(msg.data)
  console.log(`${event.phaseCode}: ${event.phaseName} (${event.progress}%)`)
})

// Start the kernel
// Output:
// SU01: initiated (0%)
// SU02: project_config_loaded (12%)
// ...
// SU16: ready (100%)

⁠System.Discovery Aggregated Queries

Query the entire computational graph via NATS:

// List all online kernels
const response = await nc.request('discovery.query.kernels',
  sc.encode(JSON.stringify({ filter: { status: 'ONLINE' } })))

// Execute custom SPARQL
await nc.request('discovery.query.sparql',
  sc.encode(JSON.stringify({ query: '...' })))

// Get kernel dependencies
await nc.request('discovery.query.dependencies',
  sc.encode(JSON.stringify({ kernel: 'System.Bakery' })))

Available Query Functions:

• discovery.query.kernels - List all kernels (filterable)
• discovery.query.edges - List all edge connections
• discovery.query.sparql - Execute custom SPARQL queries
• discovery.query.kernel - Get specific kernel metadata
• discovery.query.dependencies - Get dependency graph

See 31-EVENT-CODES-PHASES.v1.3.20.md⁠ for complete NATS event reference.

⁠Command Reference

ckp v1.3.20 - ConceptKernel Protocol CLI

ckp
├── concept                # Manage concepts (kernels)
│   ├── list              # List loaded concepts
│   ├── create <name>     # Create new concept from template
│   ├── load <name>       # Load concept from cache
│   ├── unload <name>     # Unload concept (keeps in cache)
│   ├── start <name>      # Start concept instance
│   ├── stop <name>       # Stop concept instance
│   ├── export <name>     # Export concept to cache as tar.gz
│   ├── package           # Manage concept packages
│   │   ├── list         # List cached packages
│   │   ├── import       # Import tar.gz to cache
│   │   └── unload       # Unload package from cache
│   └── build <name>      # Build Rust kernels using ontology metadata
│
├── project               # Manage projects
│   ├── list             # List all registered projects
│   ├── create <name>    # Create/register new project
│   ├── current          # Show current project
│   ├── switch <name>    # Switch current project
│   └── remove <name>    # Remove project from registry
│
├── edge                  # Manage edges (typed relationships)
│   ├── list [concept]   # List edges (optionally for concept)
│   └── create <source> <target> [predicate]
│                        # Create edge (default: PRODUCES)
│
├── package               # Manage packages
│   ├── list             # List all cached packages
│   ├── import <path>    # Import tar.gz package
│   └── fork <name>      # Fork package to create new kernel
│
├── driver                # Configure storage/transport drivers
│   ├── storage <type>   # Set storage: local, jena, age, seaweedfs
│   └── transport <type> # Set transport: local, nats, websocket, grpc
│
├── up                    # Start all concepts in project
├── down                  # Stop all running concepts
├── status                # Show status of all concepts
├── emit <target> <json>  # Emit event to concept
├── validate-urn <urn>    # Validate URN format
└── help                  # Show help for any command

⁠Common Workflows

# Create and start a kernel
ckp concept create Recipes.BakeCake
ckp concept start Recipes.BakeCake

# Connect two kernels with typed edge
ckp edge create Recipes.MixIngredients Recipes.BakeCake
ckp edge create System.Consensus System.Proof REQUIRES

# Monitor the system
ckp status

# Emit an event
ckp emit Recipes.BakeCake '{"temperature": 350, "duration": 45}'

# Package management
ckp concept export Recipes.BakeCake
ckp package import ./recipes-bakecake-v1.tar.gz
ckp concept load Recipes.BakeCake

⁠Core Concepts

⁠Continuants

You persist across time. You are bfo:0000040 (Material Entity). Your identity is eternal:

ckp://Recipes.BakeCake:v0.1

Like a neuron in the graph, you have one function — defined by your ontology, refined by consensus. You do not become many things. You master one thing.

⁠Occurrents

Every action is temporal. Every execution creates a process:

ckp://Process#Invocation-tx_20251128_100000_abc123

Phases: accepted → processing → completed or failed

Each process is a firing — a moment of transformation, captured and proven.

⁠Edges

Typed relationships between kernels. Edges declare:

• Source output schema
• Target input schema
• Field transformations (rare)
• Validation status
• Consensus authorization

⁠Consensus

Feature development through democratic voting:

• Propose a new predicate or capability
• Roles check who may vote
• Community votes
• Decision captured in RDF
• System self-improves based on decision
• Tool capabilities mutate through consensus

⁠Multi-Language Support

Concept kernels can be implemented in any language. The protocol is language-agnostic.

Current implementations:

• Rust (type: rust:hot, rust:cold) - 10x memory efficiency, 2-3x faster spawning
• Node.js (type: node:hot, node:cold) - Mature ecosystem, rapid prototyping
• Python (type: python:hot, python:cold) - Data processing, ML integration, FastAPI servers

Hot kernels run continuously (HTTP servers, WebSocket hubs, databases, FastAPI endpoints). Cold kernels spawn on-demand when work arrives, process events, exit cleanly.

All use the same filesystem protocol. All produce the same evidence format. All participate in the same governance system.

⁠Example: The Baking Workflow

# Recipes.BakeCake/conceptkernel.yaml
notification_contract:
  - target_kernel: Recipes.DecorateCake
    queue: inbox
    method: symlink
    trigger: on_storage

The concept kernel itself is the type. Your conceptkernel.yaml declares your notification contracts — whom to notify when you complete your work.

Your ontology.ttl grounds you in formal semantics:

# Recipes.BakeCake/ontology.ttl
@prefix ckp: <https://conceptkernel.org/ontology#> .
@prefix bfo: <http://purl.obolibrary.org/obo/BFO_> .