truebitprotocol/truebit-eth repository overview

⁠What is Truebit?

⁠Documentation Update in Progress

For the latest documentation, please visit our documentation site: https://docs.truebit.io/v1docs⁠

This README will be updated soon. Thank you for your patience.

Truebit⁠ is a blockchain enhancement which enables smart contracts to securely perform complex computations in standard programming languages at reduced gas costs. As described in the whitepaper⁠ and this graphical, developer-oriented overview⁠, Task Givers can issue computational tasks while Solvers and Verifiers receive remuneration for correctly solving them. You may wish to familiarize yourself with the installation instructions user guide⁠ before proceeding.

Feel free to browse the legacy wiki⁠, contribute to this repo's wiki, or check out these classic development blog posts:

• Developing with Truebit: An Overview⁠
• Using the Truebit Filesystem⁠
• Truebit Toolchain & Transmute⁠
• Writing a Truebit Task in Rust⁠
• JIT for Truebit⁠

In addition, Truebit's Reddit⁠ channel features links to some excellent introductions and mainstream media articles about Truebit. If you'd like to speak with developers working on this project, come say hello on Truebit's Gitter⁠ and Discord⁠ channels.

⁠Quickstart guide: computational playground

This tutorial demonstrates how to get started with Truebit using Ethereum mainnet or hardhat (testnet) to solve, verify, issue, and build tasks. Please refer to this guide for step by step installation instructions.

⁠GETTING STARTED

Follow the getting started⁠ guide to learn about:
-Install Truebit on Ethereum
-Truebit on Ethereum Structure
-Start Docker container
-Create a Wallet
-Synchronize the Ethereum Node
-Start a Truebit on Ethereum Terminal for Each Role
-Start Truebit on Ethereum
-Purchase TRU tokens
-Check Balances
-Purchase Solver License
-Start Solve and Verify
-IPFS connection for distributed file sharing

⁠Sample tasks via smart contracts

In general, Dapps will issue tasks from smart contracts rather than the Truebit OS command line. This allows Truebit to call back to the smart contract with a Truebit-verified solution. Typically the Task Owner smart contract fixes the task function code during deployment while the Task Submitter puts forth the function inputs at runtime. To demonstrate this method, we deploy and issue some tasks that are preinstalled in your container. One can deploy each of the samples onto the blockchain as follows.

cd /truebit-eth/wasm-ports/samples
sh deploy.sh

To run a sample task, cd into that directory and run node send.js as explained below. You may wish to edit ../deploy.js or send.js by replacing the '0' in accounts[0] with the index of your desired Geth account.

⁠Scrypt (C++)

cd /truebit-eth/wasm-ports/samples/scrypt
node send.js <text>

Computes scrypt. The string is extended to 80 bytes. See the source code here⁠. Originally by @chriseth.

⁠Bilinear pairing (C++)

cd /truebit-eth/wasm-ports/samples/pairing
node send.js <text>

For <text>, enter a string with more than 32 characters. This example uses the libff library to compute bilinear pairings for a bn128 curve. It reads two 32 byte data pieces a and b which are used like private keys to get a*O and b*O. Then a bilinear pairing is computed. The result has several components, and one of them is posted as output. (To be clear, the code just shows that libff can be used to implement bilinear pairings with Truebit). See the source code here⁠.

⁠Chess (C++)

cd /truebit-eth/wasm-ports/samples/chess
node send.js <text>

This example checks moves in a game of chess. Players could use a state channel to play a chess match, and if there is a disagreement, then the game sequence can be posted to Truebit. This method will always work for state channels because both parties have the data available. See the source code here⁠. The source code doesn't implement all the rules of chess, and is not much tested.

⁠Validate WASM file (Rust)

cd /truebit-eth/wasm-ports/samples/wasm
node send.js <wasm file>

Uses parity-wasm to read and write a WASM file. See the source code here⁠.

⁠Size of video packets in a file (C)

cd /truebit-eth/wasm-ports/samples/ffmpeg
node send.js input.ts

See the source code here⁠.

⁠Building your own tasks

We now explore the Truebit Toolchain. Each of the samples below produces a task code file called task.wasm, and each such file is produced by running a script called compile.sh. You can use the compile.sh files as templates for generating your own tasks. Inspect sample source codes and their respective compile scripts here⁠. Each compile script first compiles C, C++, or Rust source code, along with included library dependencies, to a pair of WebAssembly JavaScript runtime files using WASI-SDK⁠. Truebit's Wasm module wrapper⁠ then converts these files into a WebAssemebly format executable in Truebit. Here are is a helpful, legacy tutorial⁠ for creating and deploying Truebit tasks as well as a demo video⁠ illustrating this process.

⁠Compiling from C/C++

You should now be able to re-compile the sample tasks yourself in C++ (chess, scrypt, pairing), and C (ffmpeg) below.

cd /truebit-eth/wasm-ports/samples/chess
sh compile.sh
cd /truebit-eth/wasm-ports/samples/scrypt
sh compile.sh
cd /truebit-eth/wasm-ports/samples/pairing
sh compile.sh
cd /truebit-eth/wasm-ports/samples/ffmpeg
sh compile.sh

Note that sh compile.sh will rebuild both the build and dist directories.

⁠Compiling from Rust

For Rust tasks, take a look @georgeroman's walk-through⁠. You can use this guide to compile the /truebit-eth/wasm-ports/samples/wasm task via the steps below. First, set up the Rust compiler.

ipfs init

Then you can recompile the Rust sample task as follows.

mv /truebit-eth/wasm-ports/samples/wasm /
cp /truebit-eth/rust-tool/build.sh /wasm
sed -i "s|rust_project_name=REPLACE_ME|rust_project_name=wasm_sample|" /wasm/build.sh
cd /wasm
rm -r build dist target task.wasm
sh compile.sh

⁠Runtime

Once you have compiled your task to Truebit-flavored WebAssmebly, try using the files contract.sol, and send.js, and ../deploy.js as templates for issuing your own tasks directly from smart contracts. The API reference below⁠ will help you navigate through these templates.

When building and executing your own tasks, you may have to adjust some of the interpreter execution parameters (within range 5 to 30 exclusive), including:

memory-size: depth of the Merkle tree for memory

table-size: depth of Merkle tree for the call table

globals-size: depth of Merkle tree for the globals table

stack-size: depth of Merkle tree for the stack

call-stack-size: depth of Merkle tree for the call stack

Try adjusting memory-size first. Greater parameters make the task more likely to execute. See this file⁠ for a complete list of interpreter options.

⁠Native installation

You may wish to experiment with this tutorial on your native command line rather than running it inside the Docker container. To set up natively, first install git⁠ and clone the Truebit repo.

git clone https://github.com/TruebitProtocol/truebit-eth

⁠Running samples natively

A Node.js installation⁠ is a prerequisite for running the smart contract samples. Once you have confirmed⁠ that your Node.js installation includes npm, install Truebit's node packages from the Truebit repository's top-level directory:

cd truebit-eth
npm i

You can then deploy the sample task according to the instructions above⁠. Truebit toolchain task compilations should be done inside the Docker container as native setup is relatively complex⁠.

⁠Running Truebit OS natively

You will need Clef, Geth, IPFS, and Truebit OS.

⁠Installation

To get started, install both Geth⁠ & IPFS⁠ natively (not in the Docker container). Be sure to install a version of Geth that includes Clef. Then download a pre-built Truebit OS executable here:

https://downloads.truebit.io⁠

Choose from Linux, MacOS, or Windows flavors, and paste your chosen executable into the top level of the truebit-eth directory. This downloads page contains the latest Truebit OS version, and you should keep your local copy updated to avoid client errors. Truebit OS displays your local version at startup, and you can compare this against the latest one listed in the badge at the top of this README file. For consistency with the Docker container, the steps below⁠ assume that you rename your executable file to truebit-os, however you may choose any name for your downloaded file.

You may need to give your system permission to execute the truebit-os download. In MacOS, for example, this can be done via the following commands.

mv truebit-macos truebit-os
chmod 755 truebit-os

If running ./truebit-os in MacOS returns a pop-up error like: “truebit-os” cannot be opened because the developer cannot be verified, go to System Preferences -> Security & Privacy and where it says: "truebit-os" was blocked from use because it is not from an identified developer, choose "Allow Anyway."

Next, build the Truebit WebAssmebly interpreter from source as described below.

⁠macOS interpreter install

In macOS, once Brew⁠ is installed, one can install the interpreter as follows, starting from the truebit-eth directory:

brew install libffi ocaml ocamlbuild opam pkg-config
opam init -y
eval $(opam config env)
opam install cryptokit ctypes ctypes-foreign yojson -y
cd ocaml-offchain/interpreter
make

If you previously installed an older version of OCaml, you may need to run ocaml update, ocaml upgrade, and/or brew upgrade. You can also get a fresh install by removing ~/.opam.

⁠Linux interpreter install

From the truebit-eth directory, use the following Ubuntu install (or modify it to suit your Linux flavor).

apt-get update
apt-get install -y libffi-dev libzarith-ocaml-dev m4 opam pkg-config zlib1g-dev
opam init -y
eval `opam config env`
opam update
opam install cryptokit ctypes ctypes-foreign yojson -y
cd ocaml-offchain/interpreter
make

⁠Windows interpreter install

Follow the patterns above for Linux and macOS. You may wish to use a Linux emulator.

⁠Starting up

We now walk through the steps to start IPFS, Clef, Geth, and then Truebit-OS.

By default, MacOS stores Clef files in ~/Library/Signer/ and Geth files in ~/Library/Ethereum/. These locations differ from the location in the linux-based Docker container, which are ~/.clef and ~/.geth, /mainnet.sh startup templates. For testing purposes use local fork⁠ This means that in MacOS you'll probably find Clef's IPC socket at:

~/Library/Signer/clef.ipc

Geth's IPC socket at one of these:

~/Library/Ethereum/geth.ipc

and the keystore files at one of these:

~/Library/Ethereum/keystore

The --chainid for mainnet is still 1.

For Ethereum mainnet, it might look like this:

ipfs daemon &
clef --advanced --rules ~/Library/Signer/ruleset.js --chainid 1
geth console --syncmode light --signer ~/Library/Signer/clef.ipc
./truebit-os -p ~/Library/Ethereum/geth.ipc

For Linux, try one of the following.

ipfs daemon &
clef --advanced --rules ~/.clef/ruleset.js --chainid 1GörliGörli
geth console --syncmode light --signer ~/.clef/clef.ipc
./truebit-os -p ~/.ethereum/geth.ipc

For Windows, follow the templates above.

⁠Contract API reference

The following reference highlights some key Solidity⁠ functions that you may wish to use in your own smart contracts or interact with via web3.js⁠. For testing purposes use local fork⁠. An analogous file for Ethereum mainnet appears in the same directory. The tru token contract follows the standard ERC-20 interface⁠.

Below is a simple "hello world" JavaScript example which prints task data from Truebit's fileSystem and incentiveLayer.

const fs = require('fs')
const Web3 = require('web3')
const net = require('net')
const web3 = new Web3('/root/.ethereum/mainnet/geth.ipc', net)

// Get contract artifacts
let artifacts = JSON.parse(fs.readFileSync('/truebit-eth/wasm-client/mainnet.json'))
let FileSystem = new web3.eth.Contract(artifacts.fileSystem.abi, artifacts.fileSystem.address)
let IncentiveLayer = new web3.eth.Contract(artifacts.incentiveLayer.abi, artifacts.incentiveLayer.address)

// Sample contract interaction
let example
(async () => {
  let taskID = '0x50ee4af6810cf64ea04c1f9818101a200f3b8c00c4f6a5e85a14b327a1f8d03f'
  let taskInfo = await IncentiveLayer.methods.taskParameters(taskID).call()
  console.log(taskInfo)
  let bundleID = taskInfo.bundleId
  let codeFileID = await FileSystem.methods.getCodeFileId(bundleID).call()
  console.log(await FileSystem.methods.vmParameters(codeFileID).call())
  process.exit(1)
})()

To execute the example above or the auxiliary functions⁠ in the next section, first install⁠ Node.js and npm, check⁠ your installations, and install the prerequisite packages:

npm i truebit-util web3

Each code template can then be pasted into a .js file and run using Node.js, e.g. node example.js. The following script provides an additional template for interacting with Truebit's smart contract API.

truebit-eth/wasm-ports/samples/deploy.js

For a template interaction with Solidity.

truebit-eth/wasm-ports/samples/scrypt/contract.sol
truebit-eth/wasm-ports/samples/scrypt/send.js

⁠Preparing task inputs

We present three JavaScript functions and one Truebit OS method to assist in preparing code and data for use in Truebit. Their uses will become clear in the creating files⁠ section below. In order to run the following code templates, your working directory must include a data file called example.txt containing the input you wish to process.

⁠getRoot

When writing CONTRACT and IPFS files, one must tell Truebit the Merkle root⁠ of the data. Such a root for a file example.txt may be computed using the following web3.js⁠ template.

const fs = require('fs')
const merkleRoot = require('truebit-util').merkleRoot.web3
const Web3 = require('web3')
const web3 = new Web3('http://localhost:8545')

function getRoot(filePath) {
  let fileBuf = fs.readFileSync(filePath)
  return merkleRoot(web3, fileBuf)
}

let root = getRoot("./example.txt")
console.log(root)

⁠getSize

Truebit also needs to know the size of the file being created. Size can be computed as follows in web3.js⁠:

const fs = require('fs')

function getSize(filePath) {
  let fileBuf = fs.readFileSync(filePath)
  return fileBuf.byteLength
}

let size = getSize("./example.txt")
console.log(size)

⁠uploadOnchain

CONTRACT files should be created with web3.js⁠ using the template function uploadOnchain below. This function returns the contract address for the new CONTRACT file prefixed with a string needed for retrieval. You must be connected to an Ethereum-compatible backend, (e.g. geth) in order to run this function.

const fs = require('fs')
const Web3 = require('web3')
const net = require('net');
const web3 = new Web3('/root/.ethereum/mainnet/geth.ipc', net)

async function uploadOnchain(filePath) {

  let fileBuf = fs.readFileSync(filePath)
  let sz = fileBuf.length.toString(16)
  if (sz.length == 1) sz = "000" + sz
  else if (sz.length == 2) sz = "00" + sz
  else if (sz.length == 3) sz = "0" + sz

  let init_code = "61" + sz + "600061" + sz + "600e600039f3"
  let hex_data = Buffer.from(fileBuf).toString("hex")

  let contract = new web3.eth.Contract([])
  let accounts = await web3.eth.getAccounts()
  deployedContract = await contract.deploy({ data: '0x' + init_code + hex_data }).send({ from: accounts[0], gas: 1000000 })
  return deployedContract.options.address
}

let contractAddress = uploadOnchain("./example.txt")
contractAddress.then(address => console.log(address))

⁠Obtaining codeRoot

Truebit requires a codeRoot input when registering a .wasm or .wast program file to Truebit's file system. The codeRoot for a task program file can be obtained inside Truebit OS using the task initial command and read off from the vm.code entry. In order to use the template command below, create a task .json. Be sure to run task initial with the same virtual machine parameters that you plan to use when you later issue the task.

truebit-os:> task -f scrypt.json initial
[03-04 19:05:12] info: TASK GIVER: Created local directory: /truebit-eth/tmp.giver_kquvus9u7680
Executing: ./../wasm-client/ocaml-offchain/interpreter/wasm -m -disable-float -input -memory-size 20 -stack-size 20 -table-size 20 -globals-size 8 -call-stack-size 10 -file output.data -file input.data -wasm task.wasm
{
  vm: {
    code: '0xef34a351f42869ed0a1c4f5ba39f4be2377415082083f18592d858bc4361629b',
    stack: '0xb4c11951957c6f8f642c4af61cd6b24640fec6dc7fc607ee8206a99e92410d30',
    memory: '0xb4c11951957c6f8f642c4af61cd6b24640fec6dc7fc607ee8206a99e92410d30',
    input_size: '0x7c047c6a0b7e3f293efb6009eca04353577f88f7a88afd4b6d53c11d724c442f',
    input_name: '0xd3e24e0303f49b3dd3032fa2523603b320c2b2b0eea3693532c6401d315e8a32',
    input_data: '0xb9646f6bfbecf908827f695715a6774c4eb652c305f9da846d12157eaed6b428',
    call_stack: '0xb4c11951957c6f8f642c4af61cd6b24640fec6dc7fc607ee8206a99e92410d30',
    globals: '0xb4c11951957c6f8f642c4af61cd6b24640fec6dc7fc607ee8206a99e92410d30',
    calltable: '0x7bf9aa8e0ce11d87877e8b7a304e8e7105531771dbff77d1b00366ecb1549624',
    calltypes: '0xb4c11951957c6f8f642c4af61cd6b24640fec6dc7fc607ee8206a99e92410d30',
    pc: 0,
    stack_ptr: 0,
    call_ptr: 0,
    memsize: 0
  },
  hash: '0x60d2429c93b96f2ccc81b06525dfb5538fe18ef096dee42834562fcfedadfb91'
}

In this example, the codeRoot of task.wasm is 0xc8ada82e770779e03b2058b5e0b9809c0c2dbbdc6532ebf626d1f03b61e0a28d.

⁠fileSystem API

Recall that Truebit reads and writes three [file types], 0: BYTES, 1: CONTRACT, and 2: IPFS. Truebit stores BYTES file contents as bytes32 arrays.

⁠Creating files

We enumerate methods for creating Truebit files.

function createFileFromBytes(string calldata name, uint nonce, bytes calldata data) external returns (bytes32);

createFileFromBytes returns a fileID for a BYTES file called name with content data. Here nonce is a random, non-negative integer that uniquely identifies the newly created file. This method converts dataand stores it as a bytes32 array.

EXAMPLE:

bytes32 fileID = createFileFromBytes("input.data", 12345, "hello world!");

function createFileFromArray(string calldata name, uint nonce, bytes32[] calldata arr, uint fileSize) external returns (bytes32);

createFileFromArray returns a fileID for a BYTES file called name whose contents consist of a concatenation of array arr. nonce can be any random, non-negative integer that uniquely identifies the new file. fileSize should be the total number of bytes in the concatenation of elements in arr, excluding any "empty" bytes in the final element. This operation adds arr to contract storage.

EXAMPLE:

bytes32[] memory empty = new bytes32[](0);

filesystem.createFileFromArray("output.data", 67890, empty, 0);

function addContractFile(string calldata name, uint size, address contractAddress, bytes32 root, uint nonce) external returns (bytes32);

addContractFile returns a fileID for a CONTRACT file called name using existing data stored at address contractAddress. The data stored at contractAddress must conform to the uploadOnchain⁠ format above, and root must conform to getRoot⁠. The size parameter can be obtained using getSize⁠. nonce can be any random, non-negative integer that uniquely identifies the new file.

function addIpfsFile(string calldata name, uint size, string calldata IPFShash, bytes32 root, uint nonce) external returns (bytes32);

addIpfsFile returns a fileID for an IPFS file called name using existing data stored at IPFS address IPFShash. Root must conform to getRoot⁠, and the size parameter can be obtained using getSize⁠. nonce can be any random, non-negative integer that uniquely identifies the new file.

function setCodeRoot(uint nonce, bytes32 codeRoot, uint8 codeType, uint8 stack, uint8 memory, uint8 globals, uint8 table, uint8 call) external;

setCodeRoot designates the fileID associated with nonce as an executable WebAssembly program file. It also sets the virtual machine parameters to be used when executing the program file as part of a Truebit task. setCodeRoot must be called from the same address that originally generated the fileID.

⁠For more information, click here⁠