EthScope paper link: https://arxiv.org/abs/2005.08278.
This is the trial version of the EthScope system. The formal version will be released here.
This image is built on elasticsearch:7.4.2, and includes a few files:
elastic-schema: Create the database schema discussed in the paper, which will be automatically executed when the container is started for the first time.
data-import: Import the sample data into the database, which will be automatically executed when the container is started for the first time.
replay: Replay Engine with the instrumentation framework.
querys: Examples of using the data aggregator.
tracer-scripts: Examples of using the instrumentation framework.
docker run -d --name es-trial --volume elasticsearch-data:/usr/share/elasticsearch/data swaywu/ethscope-trial:<tag>
If you want to get the progress of importing sample data, you need to remove -d option.
docker exec -it es-trial /bin/bash
We usually detect malicious behaviours using EthScope in two steps.
Filter out suspicous transactions according to transaction features including normal transaction, internal transaction and transaction log;
Verify malicious behaviours by hooking into EVM when replaying transactions.
For example, we detect reentrancy attacks in the sample data, which contains 50,000 blocks ranging from 1,700,000 to 1,750,000.
First, we label a normal transaction as suspicious, when it meets the following two characteristics:
Part of internal transactions triggered by it form a loop that contains at least one reentry point;
There is at least one internal transaction involving with the Ether or ERC20 token transfer.
Second, we replay suspicious transactions to verify attacks at runtime. During this process, an analysis script will first construct a set of variables that could influence jump targets of the opcode JUMPI or values of transferred Ether. For each variable in this set, we define the callback function for the opcode SSTORE to monitor whether the variable is updated after the reentry point. If so, we will label that as an attack.
py-scripts/example.py implements above two steps.
$ python /usr/share/elasticsearch/py-scripts/example.py
$ replay Transactions --ipsfile ./batch-replay-sample/cluster-hosts.json --progress --tracer ./batch-replay-sample/4byte-tracer.js --result ./re ./batch-replay-sample/transaction-test-set.json
$ replay -h
NAME:
replay - replay and trace arbitrary transactions occurred in the Ethereum main net
USAGE:
replay [global options] command [command options] [arguments...]
COMMANDS:
OneTransaction replay and trace an arbitrary transaction
Transactions replay and trace a batch of arbitrary transactions
OneCustomizedBlock run a customized block
help, h show a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help (default: false)
$ replay OneTransaction -h
NAME:
replay OneTransaction - replay and trace an arbitrary transaction
USAGE:
replay OneTransaction [command options] [arguments...]
DESCRIPTION:
The OneTransaction command replays and traces a transaction specified using the transaction hash. The user-provided script will be executed during this process.
OPTIONS:
--tracer value the file path of the analysis script, which will be executed during the trace.
--result value the file path of the tracing result
--ipsfile value the file path of a list of ips combined with your elastic cluster, the default list contains ips from 192.168.1.3 to 192.168.1.14.
--help, -h show help (default: false)
$ replay Transactions -h
NAME:
replay Transactions - replay and trace a batch of arbitrary transactions
USAGE:
replay Transactions [command options] [arguments...]
DESCRIPTION:
The Transactions command replay and trace a batch of transactions specified by the file path,
which stores a list of transaction hashes. Use the trace flag to specify the analysis script that will be executed during this process.
OPTIONS:
--tracer value the file path of the analysis script
--result value the file path of the trace result
--singlecpu singlecpu turn off go-routine (default: false)
--parameters value the file path of parameters analysis script will use
--progress turn on to show the progress bar (default: false)
--printcost turn on to show the cost distribution (default: false)
--ipsfile value the file path of a list of ips combined with your elastic cluster, the default list contains ips from 192.168.1.3 to 192.168.1.14
--help, -h show help (default: false)
$ replay OneCustomizedBlock -h
NAME:
replay OneCustomizedBlock – replay a customized block
USAGE:
replay OneCustomizedBlock [command options] [arguments...]
OPTIONS:
--tracer value the file path of the analysis script
--resultfolder value the folder storing the trace result
--help, -h show help (default: false)
transactionStart : At the very beginning of each external transaction.
transactionEnd: At the end of each external transaction. The value returned by that will be stored in the result file (identified by --result)
callStart: At the very beginning of each internal transaction (right after the context switch).
callEnd: At the end of each internal transaction (right before the context switch).
{op}: Before the execution of {op}(before changing stack, memory ...).
after{Op}: After the execution of {Op}(after changing stack, memory ...).
Global Functions
toHex
toWord
toAddress
isPrecompiled
Basic APIs
log.op.getN(): n of PUSHn, DUPn, SWAPn, LOGn
log.op.toNumber()
log.op.toString()
log.stack.length()
log.stack.peek(n): n = 0, peek the top of stack
log.memory.slice(start, end)
log.memory.getUint(offset): Returns the 32bytes at the specified address interpreted as uint
log.contract.getSelfAddress()
log.contract.getCodeAddress()
log.contract.getCaller()
log.contract.getValue(): Value of internal transaction
log.contract.getInput(): Input data of internal transaction
log.getBlockNumber()
log.getTxnIndex()
log.getTxnHash()
log.getPc(): Current program counter
log.getGas(): Current remain gas
log.getDepth(): EVM depth
log.getReturnData(): Return data of last internal transaction
State APIs
db.getBalance(addr)
db.getNonce(addr)
db.getCode(addr)
db.getState(addr)
db.exists(addr): Returns a boolean value to indicate whether the account specified by addr exists.
Taint engine APIs
log.taint.labelStack(n, tag): n = 0, label tag to the top of stack
log.taint.clearStack(n)
log.taint.labelMemory(offset, size, tag)
log.taint.clearMemory(offset, size)
log.taint.labeInput(offset, size, tag)
log.taint.clearInput(offset, size)
log.taint.labelReturnData(offset, size, tag)
log.taint.clearReturnData(offset, size)
log.taint.labelStorage(addr, slot, tag)
log.taint.clearStorage(addr, slot)
log.taint.peekStack(n): Returns a string list
log.taint.peekMemory(offset): peek taint in memory[offset:offset+32]
log.taint.peekMemorySlice(offset, size)
log.taint.peekInput(offset)
log.taint.peekInputSlice(offset, size)
log.taint.peekReturnData(offset)
log.taint.peekReturnDataSlice(offset, size)
log.taint.peekStorage(slot)
taints.length: Returns the length of taints
taints[n]: Returns taints[n]
Parameter APIs
log.params.length()
log.params.get(offset)
Now, only support a list of int or string for each transaction
CFG Hijacking API
cfg.hijack(isJump): booleanIt only works at instrumentation point beforeJumpi
Input Json Format:
Chain: "Main", "Ropsten", "Rinkeby" and "Goerli" (defaults to "Main")
Number: block number, just setting this one will not affect other block information, like GasLimit (defaults to "7000000")
Difficulty: block info (defaults to "32")
GasLimit: block info (defaults to "8000029")
GasUsed: block info (defaults to "5953354")
Miner: block info (defaults to "0x36")
Timestamp: block info (defaults to "1546466952000")
Transactions: a list of transactions (Required)
GasLimit: transaction info (defaults to "1048575")
GasPrice: transaction info (defaults to "0")
Value: amount of transferred Ether (defaults to "0")
From: address of sender (Required)
To: address of receiver (Required)
Hash: hash of transaction (Required)
Input: input data of transaction (Required)
Nonce: current nonce of sender account (Required)
Accounts: a list of related accounts (Required)
Address: address of account (Required)
Balance: current balance of account (defaults to "0")
Nonce: current nonce of account (defaults to "0")
Storage: current storage of account (defaults to empty)
Code: code of account (defaults to nil)
All the above options are passed in string.
Basic usage:
{
ids : {},
store: function(id, size) {
var key = "" + toHex(id) + "-" + size;
this.ids[key] = this.ids[key] + 1 || 1;
},
main: function(ct, log) {
if (!ct) {
return;
}
var inSz = log.stack.peek(ct+1).valueOf();
if (inSz >= 4) {
var inOff = log.stack.peek(ct).valueOf();
this.store(log.memory.slice(inOff, inOff+4), inSz-4);
}
},
call: function(log, db) {
if (isPrecompiled(toAddress(log.stack.peek(1).toString(16)))) {
return;
}
this.main(3, log);
},
callcode: function(log, db) {
if (isPrecompiled(toAddress(log.stack.peek(1).toString(16)))) {
return;
}
this.main(3, log);
},
delegatecall: function(log, db) {
if (isPrecompiled(toAddress(log.stack.peek(1).toString(16)))) {
return;
}
this.main(2, log);
},
staticcall: function(log, db) {
if (isPrecompiled(toAddress(log.stack.peek(1).toString(16)))) {
return;
}
this.main(2, log);
},
transactionEnd: function(log, db) {
// console.log( this.ids ); print to screen
return this.ids;
},
}
$ replay OneTransaction --tracer JsScriptExample.js --result ./result.txt 0x374914d79f2b21555a78710e6bed70c54349a60411fc2136a38d0e8f26a898b9
$ cat result.txt
{"0x88c2a0bf-32":1,"0x0d9f5aed-96":1}
$ replay Transactions --tracer JsScriptExample.js --result ./result.txt ./test.json
$ cat result.txt
{"0x084d72f4-32":1}
$ cat test.json
[
// also accept 0x
"dc39ca7548a382ef08897331f827cf2090ca907ba237aa770f4a67f783efcbd4"
]
$ replay OneTransaction --tracer default --result ./trace.json 0x374914d79f2b21555a78710e6bed70c54349a60411fc2136a38d0e8f26a898b9
$ head trace.json
{
"gas": 157007,
"failed": true,
"returnValue": "",
"structLogs": [
{
"CodeAddr": "0xc5f60Fa4613493931b605b6dA1e9febbdeB61E16",
"ContextAddr": "0xc5f60Fa4613493931b605b6dA1e9febbdeB61E16",
"create": false,
"pc": 0,
......
Usage of peeking storage:
{
sstore: function(log, db) {
contract = log.contract.getSelfAddress()
key = log.stack.peek(0).toString(16)
nowValue = log.stack.peek(1).toString(16)
preValue = db.getState(contract, key)
console.log("SSTORE")
console.log(toHex(contract), "0x"+key)
console.log(toHex(preValue), "0x"+nowValue)
console.log()
},
sload: function(log, db) {
contract = toHex(log.contract.getSelfAddress())
key = log.stack.peek(0).toString(16)
value = db.getState(contract, key)
console.log("SLOAD")
console.log(contract, "0x"+key)
console.log(toHex(value))
console.log()
}
}
$ replay OneTransaction --tracer getState.js 0x1ef2c5e597db05e34eafe816cb86458e67072082a1744d33c6576d5b5386b5e4
SLOAD
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0x8
0x0000000000000000000000000000000000000000000000000000000000000001
SLOAD
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0x5
0x0000000000000000000000009d9bcdd249e439aaab545f59a33812e39a8e3072
SLOAD
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0x4
0x000000000000000000000000686e5ac50d9236a9b7406791256e47feddb26aba
SLOAD
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0x5
0x0000000000000000000000009d9bcdd249e439aaab545f59a33812e39a8e3072
SLOAD
0x9d9bcdd249e439aaab545f59a33812e39a8e3072 0x8
0x00000000000000000000000068c4b7d05fae45bcb6192bb93e246c77e98360e1
SLOAD
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0x24abbdd189b09f09fcd0b0ae80e2a57f91bf535083885b57bd8bd57e45e38408
0x00000000000000000000000000000000000000000000014542ba12a337c00000
SSTORE
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0x24abbdd189b09f09fcd0b0ae80e2a57f91bf535083885b57bd8bd57e45e38408
0x00000000000000000000000000000000000000000000014542ba12a337c00000 0x0
SLOAD
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0xc12dad6a1dd7786f6b720742b444f55fa5b513facf97a8f9507460e718e17fc8
0x0000000000000000000000000000000000000000000000000000000000000000
SSTORE
0xa3d58c4e56fedcae3a7c43a725aee9a71f0ece4e 0xc12dad6a1dd7786f6b720742b444f55fa5b513facf97a8f9507460e718e17fc8
0x0000000000000000000000000000000000000000000000000000000000000000 0x14542ba12a337c00000
Usage of taint engine:
{
succaller: [],
transactionEnd: function(log, db) {
if (this.succaller.length > 0) {
return toHex(log.getTxnHash())
}
},
afterCaller: function(log, db) {
caller = log.stack.peek(0).toString(16)
log.taint.labelStack(0, caller)
},
sstore: function(log, db) {
value = log.stack.peek(1).toString(16)
valueTaint = log.taint.peekStack(1)
for (i = 0; i < valueTaint.length(); i++) {
if (valueTaint.peek(i) == value) {
this.succaller.push(value)
}
}
},
}
$ replay OneTransaction --tracer taint.js --result ./result.txt 0xa6c2bf7e0e7f7d45d8fea2ed59e63b63774ec3fe0b837b6089c35a7a4fb5a590
$ cat result.txt
"0xa6c2bf7e0e7f7d45d8fea2ed59e63b63774ec3fe0b837b6089c35a7a4fb5a590"
Usage of running customized block:
block1.json
{
"Chain": "Ropsten",
"Number": "7998484",
"Transactions": [
{
"Hash": "0xd4edf163e0dae131ba64ea251d0442b208f279a9068d37dc035811900fdbbfcc",
"From": "0x79f7f5116f58a91f0d88520ef4b8034b5019525b",
"To": "",
"Input": "0x......",
"Nonce": "218"
},
{
"Hash": "0xc6d9003e7048dd89e917da6c29985b73d0b36702cc4c658df2630d7433f6adef",
"From": "0x79f7f5116f58a91f0d88520ef4b8034b5019525b",
"To": "",
"Input": "0x......",
"Nonce": "219"
},
{
"Hash": "0xa03dea679eba875e2f4060eb5eed424a2f0403a2d16d33e1dbeb5e3c2196ad9b",
"From": "0x79f7f5116f58a91f0d88520ef4b8034b5019525b",
"To": "",
"Input": "0x......",
"Nonce": "220",
"Value": "1000000000000000000"
},
{
"Hash": "0x191588f7eac99c0b2b73d1f47c147104c7b3dd098060c195a6097bad0a714d4b",
"From": "0x79f7f5116f58a91f0d88520ef4b8034b5019525b",
"To": "0x63c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe",
"Input": "0x......",
"Nonce": "221"
},
{
"Hash": "0xfc2e1cf17d00221746640411be388d30c1d6e680c3fdff3848d1b73963ed7d2d",
"From": "0x79f7f5116f58a91f0d88520ef4b8034b5019525b",
"To": "0x63c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe",
"Input": "0x893d20e8",
"Nonce": "222"
}
],
"Accounts": [
{
"Address": "0x79f7f5116f58a91f0d88520ef4b8034b5019525b",
"Balance": "2000000000000000000",
"Nonce": "218"
}
]
}
taintargs.js
{
transactionStart: function(log, db) {
console.log("\nTransaction:", toHex(log.getTxnHash()))
},
callStart: function(log, db) {
inputs = toHex(log.contract.getInput()).substring(2)
console.log("Callee: ", toHex(log.contract.getSelfAddress()) ,"Inputs:", inputs)
n = inputs.length
for (i = 8; i < n; i += 64) {
log.taint.labelInput(i/2, 32, inputs.substr(i, 64))
}
},
sstore: function(log, db) {
key = log.stack.peek(0).toString(16)
value = log.stack.peek(1).toString(16)
taints = log.taint.peekStack(1)
taintsLen = taints.length
if (taintsLen > 0) {
for (i = 0; i < taintsLen; i++) {
t = taints[i]
console.log("Key: ", key, "Value: ", value, "Arg:", t)
}
} else {
console.log("Key: ", key, "Value: ", value)
}
}
}
$ replay OneCustomizedBlock --tracer ./tracer-scripts/taintargs.js ./customized-blocks/block1.json
Transaction: 0xd4edf163e0dae131ba64ea251d0442b208f279a9068d37dc035811900fdbbfcc
Callee: 0x324d074e63e1cf9bbdc3ee6bce3e7bac8a187287 Inputs:
Key: 0 Value: 0
Key: 1 Value: 168109d0340
Key: 6 Value: 79f7f5116f58a91f0d88520ef4b8034b5019525b
Transaction: 0xc6d9003e7048dd89e917da6c29985b73d0b36702cc4c658df2630d7433f6adef
Callee: 0xad66660b2527f512716f431b571f8ae991b7ef8b Inputs:
Key: 2 Value: 79f7f5116f58a91f0d88520ef4b8034b5019525b
Transaction: 0xa03dea679eba875e2f4060eb5eed424a2f0403a2d16d33e1dbeb5e3c2196ad9b
Callee: 0x63c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe Inputs:
Key: 2 Value: 79f7f5116f58a91f0d88520ef4b8034b5019525b
Transaction: 0x191588f7eac99c0b2b73d1f47c147104c7b3dd098060c195a6097bad0a714d4b
Callee: 0x63c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe Inputs: e80cd3ab000000000000000000000000324d074e63e1cf9bbdc3ee6bce3e7bac8a1872870000000000000000000000000000000000000000000000000000000000000001
Key: 0 Value: 324d074e63e1cf9bbdc3ee6bce3e7bac8a187287 Arg: 000000000000000000000000324d074e63e1cf9bbdc3ee6bce3e7bac8a187287
Key: 0 Value: 1324d074e63e1cf9bbdc3ee6bce3e7bac8a187287 Arg: 0000000000000000000000000000000000000000000000000000000000000001
Key: 0 Value: 1324d074e63e1cf9bbdc3ee6bce3e7bac8a187287 Arg: 000000000000000000000000324d074e63e1cf9bbdc3ee6bce3e7bac8a187287
Key: 1 Value: 7a0c15
Callee: 0x324d074e63e1cf9bbdc3ee6bce3e7bac8a187287 Inputs: 2c0e00540000000000000000000000000000000000000000000000000000000000000001
Key: 8 Value: 63c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe
Key: 8 Value: 163c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe Arg: 0000000000000000000000000000000000000000000000000000000000000001
Key: 7 Value: 7a0c15
Transaction: 0xfc2e1cf17d00221746640411be388d30c1d6e680c3fdff3848d1b73963ed7d2d
Callee: 0x63c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe Inputs: 893d20e8
Callee: 0x324d074e63e1cf9bbdc3ee6bce3e7bac8a187287 Inputs: 0339f30018846565161cf4f242490aef5d9b1062187eafe284946d517614fa63a548bc2b00000000000000000000000063c1311f0dc39687f0dec9742f9ef1f2f8b6ccbe
: after any word in a string, like str = "add:", especially the word is the name of an instrumentation point, the correct way is str = "add" + ":".Content type
Image
Digest
Size
636.3 MB
Last updated
over 6 years ago
docker pull swaywu/ethscope-trial