A microservice for uploading files to IPFS. To be used with Ocean Uploader Backend.
Clone the repository and navigate to the project directory:
git clone https://github.com/oceanprotocol/uploader_ipfs.git
cd uploader_ipfs
Make sure that you have a compatible version of node.js installed (as specified in .nvmrc). We recommend running:
nvm use
Install the necessary dependencies:
npm install
Start the server using:
npm start
Description: Gets a quote in order to store some files
Path: POST /getQuote
Arguments:
{
"type": "ipfs",
"files": [{ "length": 2343545 }, { "length": 2343545 }],
"duration": 4353545453,
"payment": {
"chainId": 1,
"tokenAddress": "0xWETH_on_ETHERUEM"
},
"userAddress": "0x456"
}
Where:
Returns:
{
"tokenAmount": 500,
"approveAddress": "0x123",
"chainId": 1,
"tokenAddress": "0xWETH_on_MAINNET",
"quoteId": "xxxx"
}
Where:
Description: Upload some files
Path: POST /upload
Input:
{
"quoteId": "23",
"nonce": 12345.12345,
"signature": "0x2222",
"files": ["ipfs://xxxx", "ipfs://yyyy"]
}
Microservice will upload files to ipfs and it will take the payment
Returns: 200 OK if all the pre-checks pass. Upload occurs asynchronously.
Call getStatus to monitor status.
Description: Gets status for a job
Path: POST /getStatus?quoteId=xxx
Returns:
{
"status": 0
}
Where:
| Status | Status Description |
|---|---|
| 0 | No such quote |
| 1-99 | Waiting for files to be uploaded by the user |
| 100-199 | Processing payment |
| 200-299 | Processing payment failure modes |
| 300-399 | Uploading files to storage |
| 400 | Upload done |
| 401-499 | Upload failure modes |
Description: Gets DDO files object for a job
Path: POST /getLink?quoteId=xxx&nonce=1&signature=0xXXXXX
Input:
{
"quoteId": "23",
"nonce": 12345.12345,
"signature": "0x2222"
}
Where:
Returns:
[
{
"type": "ipfs",
"cid": "xxxx"
}
]
Description: Gets history quotes for a certain user
Path: GET /getHistory?userAddress=xxx&nonce=1&signature=0xXXXXX
Input:
{
"userAddress": "0x1234",
"nonce": 12345.12345,
"signature": "0x2222"
}
Where:
Returns:
[
{
"type": "ipfs",
"quoteId": "23",
"userAddress": "0x111",
"status": 400,
"chainId": 80001,
"tokenAddress": "0x222",
"tokenAmount": "999999999",
"approveAddress": "0x1234",
"cid": "xxxx"
}
]
Every 10 minutes (configurable), ipfs microservice should register itself to
Uploader Backend, using the register endpoint. DBS_URI will be defined as env.
POST DBS_URI/register
{
"type": "ipfs",
"description": "File storage on ipfs",
"url": "http://microservice.url",
"payment": [
{
"chainId": 1,
"acceptedTokens": [
{ "OCEAN": "0xWETH_on_ETHEREUM" },
{ "DAI": "0xOCEAN_ON_ETHEREUM" }
]
},
{
"chainId": 137,
"acceptedTokens": [
{ "OCEAN": "0xWMATIC_on_POLYGON" },
{ "DAI": "0xOCEAN_ON_POLYGON" }
]
}
]
}
export PORT=8081
export PRIVATE_KEY="0000000000000000000000000000000000000000000000000000000000000000"
export SQLITE_DB_PATH=/path/to/db/file
export REGISTRATION_INTERVAL=30000 # default: 30000 ms
export DBS_URI="https://localhost" # "DEBUG" to skip registration
export SELF_URI="https://localhost"
export IPFS_GATEWAY="https://cloudflare-ipfs.com/ipfs/" # should have trailing slash
export IPFS_GATEWAY="https://IPFS.net/" # should have trailing slash
export MAX_UPLOAD_SIZE=1099511627776 # in bytes, 0 means unlimited
npm start
Testing requires two accounts:
PRIVATE_KEY is the server account TEST_PRIVATE_KEY is the client account
export TEST_PRIVATE_KEY="0000000000000000000000000000000000000000000000000000000000000000"
export ENABLE_EXPENSIVE_TESTS="false"
npm test
This repo uses Github Actions to run the tests.
The tests use public testnets: Goerli and Mumbai
Warning Don't run multiple instances of the tests at the same time. It will cause race conditions with token approvals.
The large upload test is run periodically (quarterly) via Github Actions. It requires over 4 WMATIC in the client account (TEST_PRIVATE_KEY).
curl -d '{ "type":"ipfs", "userAddress": "0x0000000000000000000000000000000000000000", "files": [{"length": 1048576}, {"length": 256}], "payment": {"chainId": 137, "tokenAddress": "0x0000000000000000000000000000000000000000"} }' -X POST -H 'Content-Type: application/json' http://localhost:8081/getQuote
curl -d '{ "quoteId":"60f7d48ccd08653b2ef2edfe4bbe4620", "signature": "0x0000000000000000000000000000000000000000", "files": ["https://example.com/", "ipfs://xxx"], "nonce": 0 }' -X POST -H 'Content-Type: application/json' http://localhost:8081/upload
curl -d '{ "type":"ipfs", "userAddress": "0x0000000000000000000000000000000000000000", "files": [{"length": 1256}, {"length": 5969}], "payment": {"chainId": 80001, "tokenAddress": "0x0000000000000000000000000000000000001010"} }' -X POST -H 'Content-Type: application/json' http://localhost:8081/getQuote
curl -d '{ "quoteId":"40acc6937e1bd98631f47e7cbda72920", "signature": "0x0000000000000000000000000000000000000000", "files": ["https://example.com/", "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png"], "nonce": 0 }' -X POST -H 'Content-Type: application/json' http://localhost:8081/upload
curl 'http://localhost:8081/getStatus?quoteId=40acc6937e1bd98631f47e7cbda72920'
curl 'http://localhost:8081/getLink?quoteId=40acc6937e1bd98631f47e7cbda72920&signature=0x0000000000000000000000000000000000000000&nonce=0'
Please open issues on github if you need support of have any questions.
Stay tuned for more integrations and services. Follow the issues on github to see the latest development plans.
This project is fully open-source, backed by the OCEAN community and is open for contributions.
Released under the Apache 2.0 License.
Content type
Image
Digest
sha256:df717b33f…
Size
410.2 MB
Last updated
almost 3 years ago
docker pull oceanprotocol/uploader_ipfs:next