neverminedio/burner
Proxy for connecting tokenized web services nevermined.io
This project allows the access to private HTTP web services for users holding a valid Nevermined access token. This access token is generated by Nevermined platform when users demonstrate they are NFT holders. The proxy is a web2 component that protect access to web2 web services using subscription NFTs as access control mechanism.
More details in this blog post: https://medium.com/nevermined-io/bringing-online-subscriptions-into-web3-with-nfts-5fc2e9570122
The proxy implementation combines with Nevermined providing the final piece to answer the following questions:
You can find the technical details at the architecture documentation of the proxy, or in the user flows documentation.
The proxy uses the following environment variables:
SERVER_HOST
- The host used by the Oauth Server. By default and the recommended configuration is to use 127.0.0.1
so only the proxy process (NGINX) can connect to the local OAuth introspection server.SERVER_PORT
- The port used by the OAuth server. By default is 4000
. This port in normal configurations will be internal so won't be exposed and only will be accesible by the proxy process.JWT_SECRET_PHRASE
- Shared secret between a Node instance and the Proxy. This secret phrase will be used to encrypt JWT messages by the Node and decrypt by the Proxy.The OAuth Server accepts the following environment variables:
SERVER_HOST
- The host or address the OAuth server will be listen. By default 127.0.0.1
but if you need to configure from out of the server/pod it can be 0.0.0.0
SERVER_PORT
- Port the OAuth server will be listening. It is 4000
by defaultJWT_SECRET_PHRASE
- Shared secret between a Node instance and the Proxy. This secret phrase will be used to encrypt JWT messages by the Node and decrypt by the ProxyBACKEND_API_URI
- The URL of the Nevermined Backend APIPROXY_AUTH_TOKEN
- The token to be used to authenticate the requests going to the Nevermined backend coming from the Proxy (Query Protocol)The NGINX container accepts the following environment variables:
INTROSPECTION_URL
- The url to call to perform the OAuth introspection. It is the URL to the OAuth server. By default http://127.0.0.1:4000/introspect
The project has 3 Docker containers:
nevermined-io/proxy:latest
- It bundles the NGINX and OAuth server in the same imageneverminedio/proxy:nginx-latest
- NGINXneverminedio/proxy:oauth-latest
- The OAuth serverYou can build it locally too and run it:
docker build . -t nginx-proxy
docker run -v $(pwd)/conf/certs:/ssl/certs -p 443:443 -p 3128:3128 -e "INTROSPECTION_URL=http://127.0.0.1:4000" nginx-proxy
It requires a NGNIX proxy with auth_request
and njs
modules
apt install nginx-module-njs
Configure the NGINX conf files from the conf/nginx
folder. And configure the SSL certificates. You have some info about how to do it in the Dockerfile
.
# Install dependencies and compile
yarn
yarn build
# Start the web service
yarn run start:web-service
# Start the Oauth introspection server
yarn run start:oauth-server
# Start NGINX
sudo service nginx restart
# NGINX logs are available here
sudo tail -f /var/log/nginx/*.log
# Go to the SDK-JS and run
yarn run integration:external
# If we want to see this working using curl, copyt the Access Token from the previous test and export as `NVM_TOKEN` env var
export NVM_TOKEN="eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..hI4CMYSs0tqFYdof4hFiUw.2jK41Lgpua6XKZtFvqwjQS3gJDbSs0DVDMNnSW55NVOKVqQqBA1RE2InYpUY3aVJsqdsQ1nT5KpNH-MfwCyk85paoMSTiuHOW1t0bN8dB7PwyMkM5Ubf-8bg3q3rIEpDT7QtQ2M7YbP1t3HL8jhJZDStJ_2AYnumUvCVmKDtPUe_FmVdPcW66ta-d3YWKXkwKN1Ajrdnlsav58f-u6wE-qck_UtzqMpOI1ePmK3I-FBTYtSnpyUZrQu3XOXV2TR23kKaUtclhSdtHSMQHug__5Oe2Ibo3QI0AauThAHD6q98BL3iZn9fH2aCsUP2uFifRc0kC2PrWCz1F1upmaKWg2oJ9Yh9YADA95mcOjH_KSM.7tYdcSQ-AS9zdROBRUOh1g"
# Make a HTTP request to the webservice using NGINX as proxy
curl -vvv -k --proxy-insecure -X GET http://127.0.0.1:3000 --proxy http://127.0.0.1:3128 -H "Authorization: Bearer $NVM_TOKEN"
Hello World!
# Making a request directly to the endpoint should fail
curl -X GET http://127.0.0.1:3000
Unauthorized!!!!
Assuming we have NGINX already running from the previous demo:
export OPENAI_API_KEY="YOUR OPENAI API KEY"
export SERVICE_ENDPOINT="https://api.openai.com/v1/completions"
## We need this 2 if we want to use the SDK test
export AUTHORIZATION_TOKEN=$OPENAI_API_KEY
export PROXY_URL="https://127.0.0.1:443"
export SERVICE_ENDPOINT="https://api.openai.com/v1/completions"
export REQUEST_DATA='{"model": "text-davinci-003", "prompt": "Say this is a test", "temperature": 0, "max_tokens": 7}'
We go to the sdk-js and run the integration test
cd sdk-js
yarn run integration:external
...
200 - {"id":"cmpl-6kbaUQIZFe6iamppJLEHtx5j7r0xS","object":"text_completion","created":1676565594,"model":"text-davinci-003","choices":[{"text":"\n\nThis is indeed a test","index":0,"logprobs":null,"finish_reason":"length"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}
...
We can also see this working with curl, copy the "Access Token" from the previous test and run:
export NVM_TOKEN="eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..1LYqALYanLBQPmsMvPP0ug.For0wiUUMVUAxB6vvhhSjCjSucfb8dgf5pz3v-YJPxiDZ5QMr7oB5ShSUh9OErlDShmqumd-rWRcqfXuns7R6FvDMC457jTZe6P2YyFZ_rsU3TLBiv6cyF7Br3B-wshZIaG_MiKoCZqZQJXtIbZhIx4TXdtJc7yKLSRkMP_-kSMROLKlrKuwWuLow6_5G-aOyqJkU0CdZJ-iEY42eh4L0YYALZ3LZlDII-Wv45pPm6Yki3DcgCYfpZ7zSEHJpoSXm3wCB4FJ7enKPxQ02ViRMwwJldvQzrPO2XMbGAmg7OVxMN2iI6PaenUQSO76toX04cgsEnEimKUOifY0Gl_MBBFr4R0AAoCdCW7Jxxq0VBsy1H8qVRb29rR2Ql2IRmOX.1ZN258dHRBHZewKNDhwFbQ"
curl -k -v -H "Content-Type: application/json" -H "Authorization: Bearer $NVM_TOKEN" -d "$REQUEST_DATA" -H "Host: api.openai.com" https://127.0.0.1:443/v1/completions
...
{"id":"cmpl-6kc2PI82Y2OzdvOoqNDH00m6DrXDn","object":"text_completion","created":1676567325,"model":"text-davinci-003","choices":[{"text":"\n\nThis is indeed a test","index":0,"logprobs":null,"finish_reason":"length"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}
...
The credits burner is a process that runs in the background and it is responsible to burn the credits of the users that are using the proxy. It is a NodeJS process that uses the Nevermined SDK to interact with the Nevermined contracts.
The burner uses the following environment variables:
PG_HOST
- The host of the PostgreSQL database where the upstream logs generated by the proxy are storedPG_PORT
- The port of the PostgreSQL databasePG_USER
- The user of the PostgreSQL databasePG_PASSWORD
- The password of the PostgreSQL databasePG_DB
- The database nameARTIFACTS
- The path where the Nevermined contracts artifacts can be found. If not given it will be the artifacts
folder in the root path of the project.WEB3_PROVIDER_URL
- The URL of the Ethereum nodeMARKETPLACE_API_URL
- The URL of the Nevermined Marketplace APIBACKEND_API_URI
- The URL of the Nevermined Backend APIBACKEND_AUTH_TOKEN
- The token to authenticate with the Nevermined Backend API (requires to have Admin permissions)NVM_NODE_URL
- The URL of the Nevermined NodeNODE_ADDRESS
- The Ethereum address of the Nevermined NodeNETWORK_ID
- The Ethereum network idNETWORK_NAME
- The Ethereum network nameCONTRACTS_VERSION
- The version of the Nevermined contractsCONTRACTS_TAG
- The tag of the Nevermined contractsVERBOSE
- If true
the process runs in verbose modeMAX_RETRIES
- Number of retries the process will do try to burn the creditsSLEEP_DURATION
- Number of milliseconds the process will sleep between executions (default: 5000 ms)Related with the account used to interact with the Smart Contracts:
SEED_PHRASE
- The mnemonic of the account to be used to interact with the Smart Contracts
OR:KEYFILE_PATH
- The path to the account keyfileKEYFILE_PASSWORD
- The password of the account keyfiledocker pull neverminedio/burner