Git web server, built on top of gitolite + cgit + webserver and with additional goodies
1.9K
Git web server in a Docker, built on top of gitolite
Geet Web Service (GWS) is a Docker image which integrates and coordinates a set of applications which together create your personal, ultra-lightweight Git workspace. It resembles integrated web services for Git (like Github, Gitlab or Gitea), but more barebone and infinitely more hackable.
Heavy-lifting is done by Gitolite. It is the piece which is responsible for management of repositories and access control. Everything else is built on top of it. If you're unfamiliar with it, see the following pages:
Make sure to at least understand what role a special gitolite-admin repository plays in management of your Git server. There are some examples at the end of this README file, but Gitolite documentation is much more comprehensive.
With default GWS installation you'll be managing the following things through gitolite-admin:
/repositories - your git repositories. It is probably the most important
directory, so make sure to make and test a regular backup of it./config - root directory for all GWS configuration files./config/gws.toml - configuration file for GWS (see Configuration
section)./config/git - home directory of git user. This directory contains all the
necessary configuration files for gitolite and cgit./config/nginx - nginx configuration./config/sshd - ssh configuration, together with server keys.All of the following environment variables (docker run -e) are used for
initial setup only. Once installed, they are no longer used.
GWS requires setting at least GWS_SETUP_SSH_PUBKEY.
GWS_SETUP_SSH_PUBKEY: public SSH key (contents, not path to it) used for
initial setup of gitolite. Can be set only during the initial run of
container. No default is provided, so the initial setup will fail if it
is not set.GWS_SETUP_ADMIN_NAME: name under which admin will be known. Default:
adminGWS_SETUP_ROOT_URL: initial root URL under which it is available (for
example git.example.com). Default: example.comGWS_SETUP_HTTP_PORT: HTTP port under which GWS is available. Default: 8888GWS_SETUP_SSH_PORT: SSH port under which GWS is available. Default: 2222/config/gws.toml is a single configuration file for all GWS services. GWS
will use values in this file to generate specific configuration files. Rule
of thumb is that any changes in gws.toml require a container restart to be
effective.
If you need more control over the configuration files, you may create a new
directory /config/templates and place your configuration templates in this
directory. GWS will pick them over the built-ins.
There are some rules for these templates:
.in extension. For example template
for nginx.conf is nginx.conf.in;The container runs several services:
Services run as unprivileged user "git". Thi user can be configured during the
build time. Published images set its $UID and $GID to 1000. You may change
these values by setting GWS_GIT_UID and GWS_GIT_GID build arguments.
Before the first run make sure to set GWS_SETUP_SSH_PUBKEY environment
variable to SSH public key which will be admin's SSH key. You can do it like
this:
GWS_SETUP_SSH_PUBKEY="$(cat ~/.ssh/id_rsa.pub)"
After that, running GWS is just a matter of running docker or
docker-compose command with ordinary options that you want. I encourage you
to adapt and use docker-compose.yml from this repository. If you don't want
to do that, you can run it like this:
$ docker build \
--build-arg=GWS_GIT_UID=$(id -u) \
--build-arg=GWS_GIT_GID=$(id -g) \
. \
-t gws:latest
$ mkdir config repositories
$ docker build gws -t gws:latest
$ docker run \
-v "${PWD}/config:/config" \
-v "${PWD}/repositories:/repositories" \
-p "127.0.0.1:2222:2222" \
-p "127.0.0.1:8888:8888" \
-e GWS_SETUP_SSH_PUBKEY="$(cat ~/.ssh/id_rsa.pub)" \
--rm \
--name gws \
gws:latest
After the first run, you may want to inspect and do some changes to the generated gws.toml file. Changes in this file require restart of the container.
It's possible to run GWS with podman. GWS runs with a different user and
Podman, by default, maps only root (UID=0) to the user on the host. Other
users within the container are mapped according to the rules in
/etc/subuid. If you create a bind mount, this may lead to file permission
errors, because ID of git user inside the container won't match the ID of
your user on the host.
There are few things which you can do, but the simplest is to force podman to map the ids:
podman run --userns=keep-id to map host user id to the same UID
within container.podman run --userns=keep-id:uid=1000,gid=1000 to map host user to UID
and GID=100 within container. This is correct technique for example when
you use a pre-built image on the account with different UID.Alternatively (NOT RECOMMENDED AND NOT TESTED) you may try to run podman run --user 0:0 to override the user within the container and force podman to map
it to the current user.
Example docker-compose file is inside this repository in
docker-compose.yml. It requires setting GWS_SETUP_SSH_PUBKEY environment
variable either in docker-compose.yml file itself or in environment from
which it will be passed to docker-compose:
$ GWS_SETUP_SSH_PUBKEY="$(cat ~/.ssh/id_rsa.pub)" docker-compose up
version: "3"
services:
gws:
container_name: gws
image: mgoral/gws:latest
build:
context: gws
args:
GWS_GIT_UID: "${GWS_GIT_UID}"
GWS_GIT_GID: "${GWS_GIT_GID}"
volumes:
- ./.volumes/config:/config
- ./.volumes/repositories:/repositories
ports:
- "127.0.0.1:2222:2222"
- "127.0.0.1:8888:8888"
environment:
GWS_SETUP_SSH_PUBKEY: "${GWS_SETUP_SSH_PUBKEY}"
restart: on-failure
If you're using podman-compose, remember to add userns_mode: "keep-id", or
run like this: podman-compose --podman-run-args="--userns=keep-id. See
Podman section for the details on available options to correctly map host
and container users.
Users and repositories are managed with gitolite-admin repository.
To add new users, copy their SSH public keys to keydir directory. Commit and push changes.
To create or remove repositories, edit conf/gitolite.conf file. You can
also configure wild
repositories, which allow users to automatically create repositories
on-demand during the first clone. To remove wild repos, GWS enables D
subcommand: ssh git@host D.
GWS comes with few pre-configured commands and hooks which are described in
different parts of this README. These are for example password command and
webhook repository hook. You can extend your GWS installation by adding
your own scripts and programs in a similar fashion.
This feature is built on top on how Gitolite can be extended.
To enable this feature set local-code-mode option in [gitserver] section
of gws.toml to either "gitolite-admin" or "server" and list scripts to
enable in local-code option. For example:
[gitserver]
local-code-mode = "server"
local-code = ["foo", "bar"]
local-code-mode accepts 2 values, which tell GWS where to search for the
scripts defined in local-code:
"gitolite-admin": user code is managed inside the gitolite-admin
repository. GWS sets gitolite-admin with appropriate, but inactive
directory structure. Make sure to read the Important Security Note
section before you enable this mode."server": user code is managed on a server in /config/git/local-user
directory. IMPORTANT: it is local-user, NOT local.At certain points GWS local code and user local code merge inside a
/config/git/local directory. User local-code is always prioritized, so if
there are name conflicts, user ones will be used. To "revert" to
GWS script, just remove or rename user scripts. This approach allows
extending and overwriting GWS' local code.
If you use local-code-mode = "gitolite-admin", changes in local directory
take effect after the successful push to gitolite-admin. In server mode,
you must run gitolite setup to apply your changes, or restart GWS container.
GWS and Gitolite expect a certain directory structure for local code:
.
├── commands
├── hooks
│ ├── common
│ └── repo-specific
├── lib
│ └── Gitolite
│ └── Triggers
├── syntactic-sugar
├── triggers
└── VREF
Command to create it:
mkdir -p \
commands \
hooks/common \
hooks/repo-specific \
lib/Gitolite/Triggers \
syntactic-sugar \
triggers \
VREF
/config/git/local-user and gitolite-admin/local are pre-configured with this structure.
If you manage you use gitolite-admin mode for local code, then anyone
with push access to gitolite-admin repository can run any command inside
the container as git user! It is probably suitable for most small and
personal repositories, but it might be a problem for bigger organizations
where shell and git administration should be separate roles.
To enable git HTTP access (e.g. cloning, pushing by via HTTP) for a certain
repository, enable daemon feature in gws.toml and allow read access for a
special daemon user:
repo foo
RW+ = admin
RW = read_write_user
R = read_only_user
R = daemon
Users set HTTP passwords for themselves. To set them, they can use a special
password command (enabled by default by gitserver.features in gws.toml):
$ ssh git@host password
Without arguments it asks to input the password, but you can also pass it as the first argument.
NOTE: Gitolite disables ssh's port forwarding, which in turn forbids allocating PTY. It means that typed password will be visible on the screen!
NOTE: Of course, server admins can change user passwords as well. It is done through ordinary htpasswd command, pointed at /config/git/htpasswd file.
NOTE: Authentication is done by Basic HTTP Authentication as decribed in RFC2617. It means that passwords are sent in plain text. To avoid Man in the Middle attacs, GWS should be behind web server (reverse proxy) with configured HTTPS access.
To show repository in web interface, you must enable a gitweb feature in
gws.toml and add read access to a special gitweb user. To enable both HTTP
cloning and web view, you must add read access for both users:
repo foo
RW+ = admin
R = daemon gitweb
By default all repositories are "private": they can be cloned only by authenticated SSH users and are hidden from web view.
If you enabled HTTP access for daemon user (see above), they can be cloned only by authenticated users who have read access to them. If you enable Web interface on top of that, by enabling read access for gitweb user, repository is considered public and can be cloned by anyone anonymously.
NOTE: Anonymous writes (e.g. pushes) are forbidden.
For example:
repo foo
# enable read and write access for user
RW+ = user
# enable anonymous read access
R = daemon gitweb
If all what you need is to remove a repository from web interface, remove read access for gitweb user. If you want to only hide it from the list, you can configure it like that:
repo foo
RW+ = admin
R = gitweb daemon
config cgit.hide = "1"
NOTE: This merely removes a repository from the list of projects. You can still type the full URL and cgit will show it to anyone. You can still anonymously clone such repository.
If you ever want to "unhide" the repository, you must set config cgit.hide
option to empty string or 0. Deleting it won't have any effect, unless you
manually remove this part of repo's git-config on you server.
Many people use HTTP access as an easy way to access Git repositories when working behind a HTTP proxy. I'm not a huge expert in how proxies work, but from my experience, SSH access is still possible when SSH client is configured properly.
In terms of OpenSSH, it doesn't support ordinary http_proxy environment
variables, so even if Git manual says that it supports all env variables
which curl supports, it ultimately delegates establishing the connection to
ssh. For ssh to work with a proxy server, you must configure a ProxyCommand
option.
It's the easiest to add it to ~/.ssh.config. For a single domain this would be:
Host example.com
ProxyCommand nc -X connect -x proxy.example.com %h %p
To configure it for all hosts, replace Host example.com with Host *.
With this knowledge, you can evaluate if you need authenticated HTTP access to your repositories.
GWS implements a git hook which sends webhook messages on certain events. GWS aims to make it reasonabily compatible with Github/Gogs/Gitea ones. To enable it, add the following lines to repository configuration:
option hook.post-receive = webhook
option ENV.WEBHOOK_RECIPIENTS = "r1:https://example.com:8080 r2:https://example.com:9090"
option ENV.WEBHOOK_KEYS = "r1:foobar r2:baz"
# The following 2 variables set filters which decide whether push events for
# certain tags and branches should trigger the webhook.
option ENV.WEBHOOK_ENABLE_TAGS = "r1:* r2:*"
option ENV.WEBHOOK_ENABLE_BRANCHES = "r1:master r2:'master devbranch'"
The hook.post-receive = webhook line enables webhook for a particular
repository. It is important that post-receive git hook is used for the
webhook, because the implementation of webhook assumes certain post-receive
specific logic.
WEBHOOK_RECIPIENTS sets recipients of webhook events and maps them to a
certain name. Here we have 2 recipients: r1 and r2. Next, with WEBHOOK_KEYS
we assign a key for each recipient. The key is used for hashing the JSON
payload of webhook (with a HMAC hashing algorithm). To validate message
authenticity, webhook receiver should validate payload against its known,
non-transmitted key.
IMPORTANT: WEBHOOK_KEYS is stored plain text in gitolite-admin
repository. Everyone who has a read access to this repository will be able to
forge and send fake message. By defaut only admin has access to it. It is
also only accessible via the ssh connection and not visible in cgit.
WEBHOOK_ENABLE_TAGS and WEBHOOK_ENABLE_BRANCHES are filters which enable
webhook only for certain tags and branches. If neither of them is set, or if
they are empty, then webhook will be disabled. You may set several branches
or tags for a single recipient by quoting them and separating with a space.
Branches and tag names accept glob expressions. Allowed wildcards are:
* - matches everything? - matches any single character[seq] - matches any character in seq[!seq] - matches any character not in seqAll webhook-related options support shell-like quoting and escaping.
(commits[].username is empty by design)
Content-Type: application/json
X-GitHub-Delivery: f5cf1e4b-553e-4baa-a252-d60ba5017d34
X-GitHub-Event: push
X-GitHub-Event-Type: push
X-Gitea-Delivery: f5cf1e4b-553e-4baa-a252-d60ba5017d34
X-Gitea-Event: push
X-Gitea-Event-Type: push
X-Gitea-Signature: 65d9025fde0d480a92cc08d13ae743505f51c4f161f1cca1a8900fa29c28ddfa
X-Gogs-Delivery: f5cf1e4b-553e-4baa-a252-d60ba5017d34
X-Gogs-Event: push
X-Gogs-Event-Type: push
X-Gogs-Signature: 65d9025fde0d480a92cc08d13ae743505f51c4f161f1cca1a8900fa29c28ddfa
X-Hub-Signature: sha1=3d3f9896603ab88352dba92c2f4916564bddd66d
X-Hub-Signature-256: sha256=65d9025fde0d480a92cc08d13ae743505f51c4f161f1cca1a8900fa29c28ddfa
Content-Length: 1036
{
"ref": "refs/heads/master",
"before": "15d6f88f0ef8cf5dc0c902b7917b0c714d2f3cec",
"after": "275b421642772e8b081175331a30a568e6447f60",
"commits": [
{
"id": "275b421642772e8b081175331a30a568e6447f60",
"message": "Added 1 new file(s)",
"author": {
"name": "Doug Doe",
"email": "[email protected]",
"username": ""
},
"committer": {
"name": "Doug Doe",
"email": "[email protected]",
"username": ""
},
"timestamp": "2023-01-24T15:22:17+01:00"
}
],
"head_commit": {
"id": "275b421642772e8b081175331a30a568e6447f60",
"message": "Added 1 new file(s)",
"author": {
"name": "Doug Doe",
"email": "[email protected]",
"username": ""
},
"committer": {
"name": "Doug Doe",
"email": "[email protected]",
"username": ""
},
"timestamp": "2023-01-24T15:22:17+01:00"
},
"repository": {
"name": "repo",
"full_name": "subdir/repo",
"ssh_url": "ssh://example.com:2220/subdir/repo.git",
"clone_url": "https://example.com/subdir/repo.git",
"html_url": "https://example.com/subdir/repo"
},
"pusher": {
"login": "alice",
"username": "alice"
}
}
Gitolite admins (users who have "+" access to gitolite-admin repository)
can run ssh git@host import-repo <url> command which mirrors remote
repository and prints a configuration which can be added to gitolite.conf
to enable downloaded repository. It never updates configuration by
itself.
If you would like to use the generated configuration automatically, you can use it like this:
$ cd gitolite-admin
$ url=<type your url>
$ ssh git@host import-repo "${url}" >> conf/gitolite.conf \
&& git add conf/gitolite.conf \
&& git commit -m "Mirror ${url}" \
&& git push
See import-repo --help for additional options.
Alternative to import-repo command is configure-missing-repos. It is
useful if you already have a directory with bare repositories which you want
to migrate and don't want to download them again. Here are the steps which
you should make to migrate all of them at once:
chown -R 1000:1000 volumes/repositories.ssh git@host configure-missing-repos and add the output of this
command to gitolite.conf.If you're using a bind-mount to preserve /repositories on a local file system, then can use the following script to safely copy repositories:
copyfrom=/path/to/migration/directory
volume=/path/to/volume/repository
for repo in $(find "${copyfrom}" -name '*.git' -type d); do
dst=${volume}/${repo#${copyfrom}/}
if [[ -e "${dst}" ]]; then
echo "already exists: ${dst}"
continue
fi
base=$(dirname -- "${dst}")
sudo mkdir -p "${base}"
sudo chown 1000:1000 "${base}"
sudo cp -r "${repo}" "${dst}"
sudo chown -R 1000:1000 "${dst}"
done
TIP: It might be easier for you to reverse the process and move your fresh gitolite-admin.git bare repository into the old directory and then mount it as /repositories.
WARNING: Gitolite might destroy your server-side hooks in the process! Make sure to align your configuration to preserve them.
WARNING: On the other hand, Gitolite might not touch your server-side hooks which are no longer necessary! :) For example, when I was migrating from Gitea, all my repositories had a lot of proc-receive, pre-receive, update and post-receive hooks set up by Gitea. Some of them called /usr/bin/gitea, which obviously isn't available for GWS, causing failures for some operations.
You may use authorized_keys from the host by simply mounting it to
/config/git/.ssh (for example: -v /home/git/.ssh:/config/git/.ssh. This is
useful when you don't expose the internal GWS SHH server but instead "proxy"
all the requests by the SSH server running on your host.
In this case, you should create a "git" user on your host, which should match
UID and GID of the user in container (default 1000, but you may change it
with docker run --user). You should then create a script which will act as
a shell for that user, which will redirect all incoming requests to GWS SSH
server:
$ cat <<"EOF" | sudo tee /home/git/ssh-shell
#!/bin/sh
shift
ssh -p 2222 -o StrictHostKeyChecking=no [email protected] "SSH_ORIGINAL_COMMAND=\"$SSH_ORIGINAL_COMMAND\" $@"
EOF
$ sudo chmod +x /home/git/ssh-shell
$ sudo usermod -s /home/git/ssh-shell git
You must also create a SSH key-pair on the host, which you must then add to the authorized_keys. This key will be used to authenticate the host call to ssh:
ssh-rsa <your pubkey>
# gitolite start
command="/usr/share/gitolite3/gitolite-shell admin" ...
# gitolite end
If you're using mount binds, make sure to create all intermediate directories
first, or docker will create them for you with wrong permissions: mkdir my-volumes/config/git.
Git™ and the Git logo are either registered trademarks or trademarks of Software Freedom Conservancy, Inc., corporate home of the Git Project, in the United States and/or other countries. See: Git Trademark Policy
Thus, no Git in the name.
Copyright © 2023 Michał Góral
SPDX-License-Identifier: GPL-3.0-only
Content type
Image
Digest
sha256:d8c261637…
Size
100.2 MB
Last updated
11 months ago
docker pull mgoral/gws