Sign inSign up

distributivenetwork/dcp-worker

By distributivenetwork

•Updated about 1 month ago

Docker based worker that does slices from the public group on the distributive compute network.

Image
Machine learning & AI
Developer tools
Data science
1

7.4K

distributivenetwork/dcp-worker repository overview

Running the docker worker can be as simple as issuing the following docker command:

docker run distributivenetwork/dcp-worker:latest

However, you can pass the following options to it to configure how it behaves to suit your preferences and system.

--------8<--------

  -h, --help                         Display this help and exit
      --dcp-identity=string          Identity key, in hexadecimal
      --dcp-identity=json            Identity keystore, in json format
      --dcp-identity=filename        Name of file containing identity keystore
      --earnings-account=string      Bank account into which to deposit earnings
      --earnings-account=keystore    Keystore, in json format, for bank account
      --earnings-account=filename    Name of file containing bank account keystore
  -c, --cores=[#cpu][,#gpu]          Override detected number of CPU and GPU
                                     cores available to the evaluator.
  -u, --utilization=[cpu][,gpu]      default proportion of CPU,GPU to utilize: eg use
                                     -u 0.8,0.75 for 80% CPU and 75% GPU utilization
  -m, --max-sandboxes=integer        Override the default Maximum number of sandboxes
  -H, --hostname                     Override evaluator hostname
  -p, --port                         Override evaluator port number
  -o, --output=mode                  Set the console mode: text, dashboard, or none
  -l, --logger=type                  Add an additional logger: syslog, logfile (all
                                     platforms), event-log (win32)
      --logfile=filename             Change filename, implies --logger=logfile
      --overwrite-logfile            Overwrite previous log files instead of appending
      --syslog=url                   Change syslog url, implies --logger=syslog. Syslog
                                     facility (eg. local7) is the URL pathname.
      --syslog=facility              Change syslog facility, implies --logger=syslog.
      --min-level=[logger,]level     Set the lowest level of log sent to a given logger
      --min-level=[console,]level    Set the lowest level of log sent to the console
  -j, --job                          Restrict worker to a specific set of jobs. Use more
                                     than once for multiple jobs.
  -g, --join=                        Join a compute group; the format is
                                     "joinKey,joinSecret" or "joinKey,eh1-joinHash".
                                     Use more than once to join multiple groups.
  -a, --allow-origin=origin          Grant all jobs network access to specified origin
                                     Use more than once for multiple origins.
      --zero-trust                   Do not load configuration data from scheduler
      --no-global                    Do not fetch work from the Global Compute Group
      --show=                        Show configuration data and exit. Valid arguments:
                                      - worker-id: the worker id
                                      - identity: the address of the active DCP identity
                                      - earnings-account: where funds will be deposited
                                      - owner: the address of the registered owner
                                      - allowed-origins: origins in origin manager
                                      - compute-groups: credentials for task requests
                                      - config: entire worker.config
                                      - <dot-path>: a property of worker.config
      --kvin | --json                Use KVIN or JSON for show output
      --startup-delay=num            Delay for num seconds during startup
      --startup-delay-rnd=num        Delay for a random amount of time up to num seconds
      --disable-httpd                Disable web interface (undefined)
      --register=key                 Use the given key to register this worker with your
                                     Distributive portal account and exit
      --unmanaged                    Do not report statistics to or read configuration
                                     from scheduler; implies --no-claim-earnings
      --no-claim-earnings            Do not transfer unclaimed earnings to the earnings
                                     account when the worker starts
      --claim-earnings=              Transfer unclaimed earnings to the specified
                                     account and exit
  -f, --pidfile=filename             Specify the location of the pid file
      --no-pidfile                   Do not write the pid file
  -v, --verbose                      Enable verbose output. Use more than once to
                                     increase verbosity.

-------->8--------

The earnings account is the DCP Bank Account into which compute credits earned by the worker are deposited. To specify an earnings account, provide the public address associated with that account. You can create a DCP Bank Account on the DCP Portal (https://dcp.cloud⁠), then copy its public address and paste it into the command you use to launch the worker. As the worker completes work and earns compute credits, you can monitor your bank balance in the portal. Any of the arguments listed above can be passed on the docker command line and will be read by the worker accordingly.

An example command:

docker run -it distributivenetwork/dcp-worker:latest --earnings-account=0x079dac0612c710ab4e975dab7171c7e4bef78c5a

When the worker starts up in a new docker container, it autogenerates its identity keystore which uniquely identifies the worker to the DCP Scheduler. Autogenerating the identity keystore is a simpler solution to the problem of having to configure a unique identity manually and then trying to copy it in the right place inside the docker's filesystem. If you are working with ephemeral dockers than this will happen each time you restart the docker.

If you are not using ephemeral dockers than the container will remember the options you pass to it including things like payment address so you can merely restart the container to put it back to work.

Here are the common options to keep in mind.

--earnings-account=<bankAccountAddress>

Obtain an account at https://dcp.cloud⁠ and copy the account address into the command option.

-c <number>

This sets the number of sandboxes to run. The default is the number of cores detected minus 1, however, if you have hyperthreading (which is most modern CPUs) you may wish to set this explicitly as each hypterthread will be identified as a core, which can result in performance loss as generally each core has only one arithmetic unit associated with it, leading to congestion in the core as the two threads share the same arithmetic-logic unit.

-g "<computeGroupName>,<computeGroupPassword>"

This will tell the worker to join a compute group to take jobs addressed to that group. A worker can join multiple groups, with -g "groupInfo" -g "groupInfo"

--no-global

This instructs the worker to not take jobs destined for the default public group. This is intended for people running workers that are being dedicated to a particular set of work being created and who do not wish them to take public work if the compute group runs out of work temporarily.

Tag summary

Content type

Image

Digest

sha256:751615ec1…

Size

499 MB

Last updated

2 months ago

docker pull distributivenetwork/dcp-worker