transferchain/transferchain-sdk repository overview

⁠TransferChain SDK

TransferChain SDK Secure Simple Storage Service

Image: transferchain/transferchain-sdk:2.0.0

A local HTTP API in front of the TransferChain Go SDK: account, storage and log collect endpoints, a local bbolt database (<dataPath>/sdk.db) and a syncher that follows the blockchain for the account's addresses.

Architecture note: the API has no authentication and answers CORS with Access-Control-Allow-Origin: *. Run it on a local or trusted network next to the application that uses it. Protecting the endpoint (network isolation, reverse proxy, firewall) is the operator's responsibility.

⁠Quick start

Docker run:

docker run --name transferchain-sdk \
  -v ./env/sdk.yaml:/etc/transferchain/sdk.yaml \
  -v ./data:/var/data/transferchain \
  -v ./log:/var/log/transferchain \
  -e LOG_PATH=/var/log/transferchain \
  -e PORT=8080 \
  -e WITHOUT_LISTEN=false \
  -p "8080:8080" \
  transferchain/transferchain-sdk:2.0.0

Docker compose:

name: transferchain-sdk

services:
  sdk-local:
    image: transferchain/transferchain-sdk:2.0.0
    environment:
      PORT: 8080
      WITHOUT_LISTEN: 'false'
      LOG_PATH: /var/log/transferchain
    volumes:
      - ./env/sdk.yaml:/etc/transferchain/sdk.yaml
      - ./data:/var/data/transferchain
      - ./log:/var/log/transferchain
    restart: always
    ports:
      - '8080:8080'

The image only starts in prod mode (default). Keep collect/ on the persistent data volume: entries not uploaded yet live there.

⁠Configuration

Example sdk.yaml:

version: v1
behaviour: "standard" # or log-collect

# TransferChain authentication and authorization information.
account:
    api:
        token:
        secret:
        url: https://connect.transferchain.io
        proxy: # this option is optional
          type: "http" # http|socks5|socks5h
          host:
          port:
    identifier: 0
    mnemonics: # written by init/restore; not returned in responses
    ns:
    uuid:

# TransferChain file operation's (storage upload, download, delete) Server URL.
fileoperation:
    host: https://doperation.transferchain.io
    fingerprint:
    insecure: false
    downloadurl:
    proxy: # this option is optional
      type: "http" # http|socks5|socks5h
      host:
      port:

limits:
    bodysize: 0 # in bytes, the default is 0, so it's 5368709120(5gb)
    timeout: 0 # in seconds, the default is 0, so it's 1800(30minutes)

# Logger configurations - required in log-collect mode
logger:
  sizeThreshold: 1048576 # in bytes (min 1048576); no longer used
  prefix: "sdk-" # no longer used
  wal: # all optional, defaults shown
    flushBytes: 1048576
    flushInterval: 200ms
    checkpointBytes: 67108864
    checkpointInterval: 5m
    maxBytes: 1073741824

# TransferChain Blockchain information.
readnode:
    address: https://read-node-01.transferchain.io
    chain:
        name: medusa # constant
        version: v2 # constant
    wsaddress: wss://read-node-01.transferchain.io/ws
    fingerprint:
    insecure: false
    proxy: # this option is optional
      type: "http" # http|socks5|socks5h
      host:
      port:

# APIs verbose mode. if open showed all log events.
log:
  verbose: false
  syslog: false
  beauty: false
  format: "json" # or 'text'

listen: true

# Upload "files" absolute paths and download "path" must be under these
# directories (otherwise 403). Default: <dataPath>/files
allowedpaths:
    - /var/data/transferchain/files

# If true and an account exists, forced init and forced restore return 403.
disableforceinit: false

⁠Usage

Every JSON answer has success, message and, when present, data, total_count, errors and detail. Replace placeholders such as <txID> and <word1>; never put real mnemonics into commands.

⁠Account

Init: creates the cloud account for the first time, or re-creates it with force. On success the mnemonics are written to the config file; they are not returned. Without force over an existing account: 400; with disableforceinit: true, force over an existing account: 403.

curl -X POST http://localhost:8080/v1/account/init \
  -H "Content-Type: application/json" \
  -d '{"force": false}'

{
  "success": true,
  "message": "Successfully created new cloud account and new mnemonics wrote on config"
}

Restore: without force over an existing account 400, forced restore with disableforceinit: true 403, invalid mnemonics 400; none of them delete anything. A valid forced restore deletes the previous account's local records first. The mnemonics are not returned.

curl -X POST http://localhost:8080/v1/account/restore \
  -H "Content-Type: application/json" \
  -d '{"mnemonics": ["<word1>", "<word2>", "<...>"], "force": false}'

Info (403 when no account): address, ns, user_id, user_uuid, wallet_id, wallet_uuid, email.

curl http://localhost:8080/v1/account/info

Status: buildstamp, database, syncher, account, synced_last_block, usage counters.

curl http://localhost:8080/v1/status

⁠Storage

Upload with a JSON body (application/json or no Content-Type). Entries in files that are absolute paths are read from disk and must be under allowedpaths (otherwise 403); other entries are uploaded as content. Optional fields: recipients, delete_after, url_flag, notes (at most 500 characters), download_limit (at most 300).

curl -X POST http://localhost:8080/v1/storage/upload \
  -H "Content-Type: application/json" \
  -d '{"files": ["raw file data", "/var/data/transferchain/files/file1.zip"]}'

Upload raw content of one file (application/octet-stream); the file name is the name query parameter, the other fields are optional query parameters. Any other Content-Type gets 415.

curl -X POST "http://localhost:8080/v1/storage/upload?name=report.pdf" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @report.pdf

Upload response (201):

{
  "data": {
    "transactions": [
      {
        "id": "<txID>",
        "version": 2,
        "type": "storage",
        "sender_addr": "<your random address>",
        "recipient_addr": "<your random address>",
        "data": "<base encoded, encrypted data>",
        "sign": "<base encoded, sha512 hashed sender signed data>",
        "fee": 0
      }
    ],
    "storages": [
      {
        "UUID": "<string>",
        "FileName": "<uuid_v4>",
        "Size": 97,
        "Slots": [
          {
            "base_uuid": "<string>",
            "uuid": "<string>",
            "storage_service": "<provider title>",
            "storage_code": "<provider code>",
            "size": 0,
            "size_rl": 0,
            "user_id": 0,
            "chunk_size": 0
          }
        ],
        "KeyAES": "<base64 encoded random bytes>",
        "KeyHMAC": "<base64 encoded random bytes>",
        "UploadDate": "RFC3339 TIMESTAMP",
        "Policy": {
          "ID": "<string>",
          "Identifier": "<string>",
          "Owner": "<your random address>",
          "SourceIdentifier": "<your random address>",
          "TargetIdentifier": "<your random address>",
          "OPCode": "storage",
          "Policy": "owner",
          "EventType": "create",
          "CreatedAt": "RFC3339 TIMESTAMP"
        }
      }
    ],
    "mapped_storages": {
      "<txID>": {
        "UUID": "<string>",
        "FileName": "<uuid_v4>",
        "Size": 97,
        "Slots": [
          {
            "base_uuid": "<string>",
            "uuid": "<string>",
            "storage_service": "<provider title>",
            "storage_code": "<provider code>",
            "size": 0,
            "size_rl": 0,
            "user_id": 0,
            "chunk_size": 0
          }
        ],
        "KeyAES": "<base64 encoded random bytes>",
        "KeyHMAC": "<base64 encoded random bytes>",
        "UploadDate": "RFC3339 TIMESTAMP",
        "Policy": {
          "ID": "<string>",
          "Identifier": "<string>",
          "Owner": "<your random address>",
          "SourceIdentifier": "<your random address>",
          "TargetIdentifier": "<your random address>",
          "OPCode": "storage",
          "Policy": "owner",
          "EventType": "create",
          "CreatedAt": "RFC3339 TIMESTAMP"
        }
      }
    }
  }
}

List (limit: tx count per page, default 40; last_id: last tx id of the previous page):

curl "http://localhost:8080/v1/storage/?limit=40&last_id=<txID>"

{
  "data": {
    "transactions": [
      {
        "id": "<txID>",
        "version": 2,
        "type": "storage",
        "sender_addr": "<your random address>",
        "recipient_addr": "<your random address>",
        "data": "<base encoded, encrypted data>",
        "sign": "<base encoded, sha512 hashed sender signed data>",
        "fee": 0
      }
    ]
  }
}

Get one record (from the local database, otherwise from the chain; a chain record is accepted only if sent from the account's own addresses, else 403; unknown 404):

curl http://localhost:8080/v1/storage/<txID>

Download: with path the file is written to that directory (must be under allowedpaths, else 403) and the answer is JSON; without path the answer is the file content.

curl -X POST http://localhost:8080/v1/storage/download/<txID> \
  -H "Content-Type: application/json" \
  -d '{"path": "/var/data/transferchain/files"}'

curl -X POST http://localhost:8080/v1/storage/download/<txID> -o report.pdf

Delete (also DELETE /v1/storage/deletion):

curl -X DELETE http://localhost:8080/v1/storage/ \
  -H "Content-Type: application/json" \
  -d '{"tx_id": ["<txID_1>", "<txID_2>"]}'

⁠Log collect

Only with behaviour: log-collect. The body format is selected by Content-Type (parameters such as ; charset=utf-8 are ignored):

For both formats an empty body gets 400, a body larger than 10 MB gets 413, and 201 means the entry is durable.

curl -X POST "http://localhost:8080/v1/log/collect" \
  -H "Content-Type: application/json" \
  -d '{"data": "one log line"}'

curl -X POST "http://localhost:8080/v1/log/collect" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @app.log

Uploaded log segments: GET /v1/log/, GET /v1/log/<txID> and POST /v1/log/download/<txID>, with the same paging and path rules as storage.

⁠Log collect write-ahead log (WAL)

/v1/log/collect entries are first written to a write-ahead log, <dataPath>/collect/wal.log, and then uploaded in segments to TransferChain storage:

1. A single writer appends the entry; as soon as no other entry is queued (or flushBytes is reached) it writes a flush marker and fsyncs. Entries that arrive during an fsync share the next one (group commit).
2. Only then do the waiting requests get 201.
3. When checkpointBytes is reached, every checkpointInterval and after account init, a checkpoint uploads the unuploaded entries as one segment named seg-<account>-<seq>.log (one entry per line, as before) and writes the local records.
4. A checkpoint marker is written and the file is trimmed.

Guarantees

• 201 means durable: the entry is written and fsynced before the answer.
• No loss on crash: on startup the WAL is replayed up to the last flush marker; anything after it was never acknowledged.
• Torn write safety: every entry has a CRC32C checksum; a partial or damaged tail is cut off.
• Upload failures keep the data: a failed upload leaves the WAL as it is and is retried with backoff.
• Stable segment boundaries: an INTENT record stores the segment range before the upload, so a restart (even with a changed checkpointBytes) uploads exactly the same range.
• Legacy files: log files left by older versions in <dataPath>/collect are uploaded once at startup; each file is deleted only after its local record is written.
• Throughput: group commit (one fsync for many requests); on a local ext4 disk with 4 KiB entries and the default settings about 110 MB/s (upload time not included).

Guards

• Request body larger than 10 MB: 413 (also the largest WAL entry).
• WAL disk budget (maxBytes) full or no disk space: 507; entries already accepted are kept. Other write errors: 500.
• One checkpoint reads and uploads at most checkpointBytes.
• A panic during a segment upload is caught; the segment is kept and retried.
• A warning is logged when data waits longer than checkpointInterval.
• An unreadable WAL file is moved aside as wal.log.corrupt-<time>, a file with an older format version as wal.log.v1-<time>; neither is deleted.
• Server shutdown flushes pending entries.
• A segment is not uploaded twice (log_segments status). If the process stops after the upload but before the local record, the segment is uploaded again and a warning is logged (it may appear twice).

⁠Security and limits

• No authentication, CORS *: run on a trusted network.
• Connection buffers: 16 KiB per connection (also the maximum header size); the body limit (limits.bodysize, default 5 GiB) is separate. Log collect accepts at most 10 MB.
• Gzip limits: compressed Addresses transactions from the chain are opened with at most 2 MiB input and 16 MiB output.
• allowedpaths: upload file paths and download targets outside these directories get 403.
• Generic error messages: details (including panics and stack traces) go only to the log.
• File modes: config file and database 0600; created data, log and collect directories 0750.
• Syncher sender check: account-scoped transactions are accepted only when signed and sent from the account's own addresses; others are skipped with a warning.
• Mnemonics are never returned by the API.

⁠Status codes

⁠2.0.0 upgrade notes / breaking changes

• Container user: UID 1001 (group 0); chown -R 1001:0 the mounted volumes and config (see Quick start).
• allowedpaths: upload file paths and download targets must be under these directories (default <dataPath>/files); otherwise 403.
• Upload Content-Type: application/json (or no Content-Type) keeps the previous JSON body; application/octet-stream sends the raw content of one file with the file name in the name query parameter; any other Content-Type gets 415.
• disableforceinit: when true and an account already exists, forced /v1/account/init and forced /v1/account/restore get 403.
• Mnemonics: init, restore and info responses no longer return the mnemonics. They are kept in the config file (account.mnemonics), which is written with 0600 permissions.
• Log collect: entries go through the WAL; a 201 answer waits for the fsync (at most flushInterval, ~200 ms) and means the entry is durable; 507 when the WAL budget or disk is full; uploaded log files are named seg-<account>-<seq>.log; legacy log files are migrated; logger.sizeThreshold and logger.prefix are no longer used; raw application/octet-stream bodies are accepted. Keep <dataPath>/collect on a persistent volume.