Posts tagged 'ops'

Systemd Unit to activate loopback devices before LVM

In a Debian server I'm using LVM to create a single logical volume from multiple different volumes. One of the volumes is a loop-back device which refers to a file in another filesystem.

The loop-back device needs to be activated before the LVM service starts or the later will fail due to missing volumes. To do so a special systemd unit needs to be created which will not have the default dependencies of units and will get executed before lvm2-activation-early service.

Systemd will set a number of dependencies for all units by default to bring the system into a usable …

Remove users from git-crypt enabled repository

Git crypt is a neat git extension to encrypt some files - if not all - in a git repository. Integrates nicely with git using filters and it's use is completely transparent once you have unlocked a repository.

Using git-crypt you can still share a repository in public and maintain a set of files with secrets that are accessible to a limited number of users. Especially useful for open source projects.

At some point maybe you'll need to remove one of the users who have access to the encrypted files. Git-crypt does not provide a command to remove users (yet) because it's …

Takis - A util that blocks until a port is open.

Over at Mozilla's Engagement Engineering we use Docker to ship our websites. We build the docker images in CI and then we run tests against them. Our tests usually need a database or a cache server which you can get it running simply with a single command:

docker run -d mariadb

The problem is that this container will take some time to initialize and become available to accept connections. Depending on what your test and how you run your tests this delay can cause a test failure to due database connection timeouts.

We used to wait on executing our tests …

Static site hosting on Dokku and Deis

Deis and Dokku, the open source Heroku-like PaaS, can be used for hosting static sites too. Since they both support Dockerfile based deployments all we need is an Docker Image with Nginx.

I created giorgos/dokku-static-site which uses the ONBUILD instruction. To use it create a Dockerfile at the root directory of your static site with only one line:

FROM giorgos/dokku-static-site

and then place all your files under the html directory. If moving your website files to another directory isn't in your plans, you can alternatively create a symbolic link

ln -s . html

Then push to your Deis / Dokku …

Host your own music server with supysonic

I like to self-host services to make things fit my needs instead of the other way around and to satisfy my need to control my data. Also remember that self hosting is really important for the health of the open web and internet.

There are a couple of options for self-hosted media servers. I always go for FLOSS solutions, so Plex and friends are not an option. A couple of years ago I tried Subsonic, including the MadSonic and Musicabinet forks. Subsonic looked really outdated at that time, so that didn't last long. Musicabinet was second in my evaluation and …

Fleet job to remove unused docker images

Engagement Engineering, the team that I'm part of at Mozilla, runs two Deis clusters on AWS to host important websites including

Deis is a Heroku-inspired PaaS which utilizes CoreOS and Docker. It's a great open-source project, developed in the public, with a great Community and commercially backed by Engine Yard.

Apps on Deis run within Docker containers which run on CoreOS machines that form the Deis cluster. Each new release of your code, i.e. each new deis pull or git push, creates a new Docker image that is stored in the internal Deis Docker Registry and …

List the extensions of files under a directory

I want to list the extensions of filenames in a directory, sorted and unique.

First list all the files, leaving out directories using:

find -type f .

Then use awk to get the last three characters of each line, i.e. filename.

find . -type f | awk '{ print substr( $0, length($0) - 3, 4) }'

AWK's substr function extracts four characters (last argument) of string $0 (first argument) starting at full lenth of the string minus three characters (second argument).

This will list the extensions from all files in the current directory and all directories below.

Now sort and uniq the output of …

Duplicity scp backend weirdness

I'm a big fan of Duplicity and I use to backup my laptop to my office based server over scp which in its turn encrypts everything with EncFS and syncs everything up to my SpiderOak account. I was going under a typical maintenance today and I run duplicity collection-status to see how many backup sets I have.

Surprisingly duplicity returned only one full backup set more than a year old and no other backup sets. After some digging I found that the returned backup set was indeed the only full set I had but besides that I had many incremental …