aztechian/ncodr repository overview

⁠Ncodr

Ncodr (en-code-er) is a web-based, distributed application for controlling a ripping and encoding queue. It's intention is to support DVD and BD (Blu-Ray Disc) ripping and encoding to MPEG-4. By default, it encodes to an "iTunes" compatible m4v container with h264 video codec. Audio encoding can be easily changed from the options field that is submitted with a job.

Utilizing bull⁠ for the queue control, this app will submit jobs to the respective queue, and provide workers to process the queued jobs. Bull is backed by a Redis⁠ database for persistent queue information storage. For a production environment, Redis should be available as an external service to Ncodr. A persistent storage for Redis is not provided from this project.

Additionally, Ncodr supplies a RESTful API around the bull queues, so that jobs may be easily submitted via HTTP interfaces.

The tools doing the hard part (dvdbackup, makemkv and HandBrakeCLI) are expected to be available on the $PATH for Ncodr to use. The primary packaging for this app is a Docker Moby container, which provides all the tools needed, and makes Ncodr the entry point. The simplest way to get running is by using the production docker-compose⁠. This uses a pre-built docker image that is published to docker hub⁠.

Alternatively, zip files of the compiled app with configuration, package.json and node_modules for production is available from the releases⁠ page. Given a Node.js installation on the target system, it should run with a simple npm start command.

As long as the app is configured to look at the redis location, as many containers as necessary can be started to process jobs for parallelizing encoding work. Generally, this would be one container per physical host, as HandBrake can be multi-threaded across CPUs - even in a container.

/dev/sr0 detection will let a container know if it can process ripping queue jobs. Be aware that you may need to run a container that will rip discs as root so that it has permission to access device files directly. There are some examples of additional permissions needed for device usage in the docker-compose⁠. By default, Ncodr runs as a high-numbered UID (not root) so that it is usable on container management systems (Kubernetes, OpenShift). If you are running separate encoding containers, they will likely work just fine as the default, non-root user, or you can specify another UID for the container to run as.

Because Bull is used for queueing, it's centralized persistence with Redis means that you may run many instances of Ncodr on your network (or, really anywhere) to process jobs. All that is required is connectivity to the Redis database (and likely a centralized storage). There are 3 components that make up Ncodr:

• Queue Processing (ripping and encoding can be controlled separately)

• REST API

• Web UI

Because of this, it is possible to deploy many instances of Ncodr in any combination of those components. Some example use cases:

I have 4 physical hosts on my network that could be used for encoding. Only one of those hosts has an optical drive.

I have 1 physical host and another physical host with 2 virtual machines on it. The physical host has an optical drive. I have external network availability to one of the VMs, and so I want to access the UI from it.

⁠Development

The quickest way to get started with Ncodr is with docker-compose⁠: ln -s docker-compose.dev.yml docker-compose.override.yml docker-compose up

Ensure docker and docker-compose are available on your system. This will start a local Redis container, matched with a nodeJS based container for executing Ncodr in. Since this setup is meant for development, the Ncodr application is bind mounted into the app container so that real-time changes to the code can be made, which should trigger an application restart to pick up those changes. In this way, it is easy to develop quickly.

The benefit of using docker-compose as described above is that the node environment is exactly the same as what is run in production. However, it is of course possible to run Ncodr locally on a development workstation. You will need NodeJS version >=8.x installed, with a recent version on NPM. Then, do the usual npm install and npm run serve. This will start a process on your workstation listening (by default) on port 2000.

Other commands for developers to know:

You will notice there are a few variations of docker-compose files available: docker-compose⁠ is an example of a production deployment of Ncodr docker-compose.dev⁠ is for use when developing Ncodr on a local workstation docker-compose.ci⁠ is to stub out some production settings when testing in Travis CI pipelines

⁠Options

See the config⁠ for what variables can be set from environment variables. These generally follow the format of <SECTION>_<KEY>, so to set Ripper -> Device you would set the environment variable like: RIPPER_DEVICE=/dev/sr2 For a complete set of options that can be tweaked, see the defaults⁠. The configuration of Ncodr is based on node-config⁠. You can also see its documentation on how things may be configured.

⁠Environment Variables

Below are some of the more important settings that can be used for configuring Ncodr. Default values are in parenthesis