• Damn I Love Docker: OpenVPN Server

    Dec 31, 2018 •

    Table of Contents

    Installing OpenVPN

    AMD64

    NOTE: The following quick start install is from kylemanna’s documentation1 and uses his docker image.2

    • Pick a name for the $OVPN_DATA data volume container. It’s recommended to use the ovpn-data- prefix to operate seamlessly with the reference systemd service. Users are encourage to replace example with a descriptive name of their choosing.

        OVPN_DATA="ovpn-data-example"

    • Initialize the $OVPN_DATA container that will hold the configuration files and certificates. The container will prompt for a passphrase to protect the private key used by the newly generated certificate authority.

        docker volume create --name $OVPN_DATA
  docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm kylemanna/openvpn ovpn_genconfig -u udp://VPN.SERVERNAME.COM
  docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it kylemanna/openvpn ovpn_initpki

    • Start OpenVPN server process

        docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN kylemanna/openvpn

    • Generate a client certificate without a passphrase

        docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it kylemanna/openvpn easyrsa build-client-full CLIENTNAME nopass

    • Retrieve the client configuration with embedded certificates

        docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm kylemanna/openvpn ovpn_getclient CLIENTNAME > CLIENTNAME.ovpn

    ARMhf

    NOTE: The following is the same install instructions as the AMD64, but use an armhf image for the Raspberry Pi.

    • Pick a name for the $OVPN_DATA data volume container. It’s recommended to use the ovpn-data- prefix to operate seamlessly with the reference systemd service. Users are encourage to replace example with a descriptive name of their choosing.

        OVPN_DATA="ovpn-data-example"

    • Initialize the $OVPN_DATA container that will hold the configuration files and certificates. The container will prompt for a passphrase to protect the private key used by the newly generated certificate authority.

        docker volume create --name $OVPN_DATA
  docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm tigerj/rpi-ovpn ovpn_genconfig -u udp://VPN.SERVERNAME.COM
  docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it tigerj/rpi-ovpn ovpn_initpki

    • Start OpenVPN server process

        docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN tigerj/rpi-ovpn

    • Generate a client certificate without a passphrase

        docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it tigerj/rpi-ovpn easyrsa build-client-full CLIENTNAME nopass

    • Retrieve the client configuration with embedded certificates

        docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm tigerj/rpi-ovpn ovpn_getclient CLIENTNAME > CLIENTNAME.ovpn

    References

    1. Dockerized OVPN AMD64 ↩

    2. Dockerhub kylemanna/openvpn ↩

  • Reformatting Disk Drive on Linux to ext4: parted command

    Dec 8, 2018 •

    Table of Contents

    Introduction

    This walkthrough for reformatting a drive (already formatted) to ext4 on Linux is inspired heavily from the sources found in the reference section. The only difference is some added clarity and detail for the reformatting process

    Walkthrough

    What follows is a short walkthrough for finding a disk connected to the system, removing a partition, and making a new partition.

    Find Device Name

    First, physically connect the drive to the system, either through USB (for external drives like on the RPi) or through SATA on the motherboard (i.e. for PCs). Once connected, the device can be found using the lsblk command like so:

    $ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  4.6T  0 disk
└─sda1        8:1    0  4.6T  0 part
mmcblk0     179:0    0 59.6G  0 disk
├─mmcblk0p1 179:1    0 41.8M  0 part /boot
└─mmcblk0p2 179:2    0 59.6G  0 part /

    The above example output was taken from a Raspberry Pi 2. The device labeled sda is a Seagate 5TB External Hard Drive, whereas the mmclk0 is the micro SD card that is formatted with the Raspibian OS. The important thing here, is we now know which device is our Hard Drive.

    If we do a quick ls /dev, we can see ALL devices connected to the system:

    $ ls /dev
autofs           loop6               ram5     tty18  tty43      uhid
block            loop7               ram6     tty19  tty44      uinput
bsg              loop-control        ram7     tty2   tty45      urandom
btrfs-control    mapper              ram8     tty20  tty46      vchiq
bus              mem                 ram9     tty21  tty47      vcio
cachefiles       memory_bandwidth    random   tty22  tty48      vc-mem
char             mmcblk0             raw      tty23  tty49      vcs
console          mmcblk0p1           rfkill   tty24  tty5       vcs1
cpu_dma_latency  mmcblk0p2           sda      tty25  tty50      vcs2
cuse             mqueue              sda1     tty26  tty51      vcs3
disk             net                 serial0  tty27  tty52      vcs4
fb0              network_latency     sg0      tty28  tty53      vcs5
fd               network_throughput  shm      tty29  tty54      vcs6
full             null                snd      tty3   tty55      vcs7
fuse             ppp                 stderr   tty30  tty56      vcsa
gpiochip0        ptmx                stdin    tty31  tty57      vcsa1
gpiomem          pts                 stdout   tty32  tty58      vcsa2
hwrng            ram0                tty      tty33  tty59      vcsa3
initctl          ram1                tty0     tty34  tty6       vcsa4
input            ram10               tty1     tty35  tty60      vcsa5
kmsg             ram11               tty10    tty36  tty61      vcsa6
log              ram12               tty11    tty37  tty62      vcsa7
loop0            ram13               tty12    tty38  tty63      vcsm
loop1            ram14               tty13    tty39  tty7       vhci
loop2            ram15               tty14    tty4   tty8       watchdog
loop3            ram2                tty15    tty40  tty9       watchdog0
loop4            ram3                tty16    tty41  ttyAMA0    zero
loop5            ram4                tty17    tty42  ttyprintk

    Without going into great detail about what all these “devices” are, we can see that in the third column our device sda is located. WARNING: If you have more than one drive connected, the label they are given may change. This means that you cannot count on the drive to always be labeled sda. If there are two drives connected, one will be labeled sda and the other sdb, BUT there is no guarantee which will get which label … it can change between reboots etc … Always check with lsblk or other commands (i.e. sudo fdisk -l) which drives have been labeled which device files.

    Now we can move on to using the parted command.

    Parted Command1

    Now that we are clear we have the right drive, fire up an interactive parted session by launching the following (using the device file name we found for our drive earlier):

    $ sudo parted /dev/sda
GNU Parted 3.2
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted)

    If we type help we can see the potential reformatting options:

    (parted) help
align-check TYPE N                        check partition N for TYPE(min|opt)
        alignment
  help [COMMAND]                           print general help, or help on
        COMMAND
  mklabel,mktable LABEL-TYPE               create a new disklabel (partition
        table)
  mkpart PART-TYPE [FS-TYPE] START END     make a partition
  name NUMBER NAME                         name partition NUMBER as NAME
  print [devices|free|list,all|NUMBER]     display the partition table,
        available devices, free space, all found partitions, or a particular
        partition
  quit                                     exit program
  rescue START END                         rescue a lost partition near START
        and END
  resizepart NUMBER END                    resize partition NUMBER
  rm NUMBER                                delete partition NUMBER
  select DEVICE                            choose the device to edit
  disk_set FLAG STATE                      change the FLAG on selected device
  disk_toggle [FLAG]                       toggle the state of FLAG on selected
        device
  set NUMBER FLAG STATE                    change the FLAG on partition NUMBER
  toggle [NUMBER [FLAG]]                   toggle the state of FLAG on partition
        NUMBER
  unit UNIT                                set the default unit to UNIT
  version                                  display the version number and
        copyright information of GNU Parted

    Remove Partition

    We can see that rm NUMBER will delete a partition, but how do we know what are the partitions? Simple, we run print like so:

    (parted) print
Model: Seagate Expansion Desk (scsi)
Disk /dev/sda: 5001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  5001GB  5001GB  ext4         primary

    Here we see that we have one partition. To remove it simply type:

    (parted) rm 1

    Simply repeat this with all partitions you want to remove (i.e. rm 2, rm 3). Now let us look at adding a new partition.

    Adding Partition2

    First to add a new partition, we can exit the interactive session and simply call parted and pass the commands to it. We will do this to set the new partition standard (this assumes you have a fresh drive, or have already removed all the partitions before using rm NUMBER):

    $ sudo parted /dev/sda mklabel gpt

    Now we can create a new partition spanning the whole disk:

    $ sudo parted -a opt /dev/sda mkpart primary ext4 0% 100%

    This will create one primary partition with the ext4 format (which parted does not support formatting). Now we are ready to add an ext4 file system to this partition.

    Formatting ext4

    Finally we only need to run a simple command to complete the reformatting of the drive, with a new partition in the ext4 format:

    $ sudo mkfs.ext4 -L OPTIONAL_DISK_NAME /dev/sda1

    This will create the ext4 file system on the partition at /dev/sda1 we created earlier. The -L flag will set the label of the partition to whatever you set as OPTIONAL_DISK_NAME. This is optional and not necessary.

    Summary

    The drive is now completely reformatted with a fresh partition. You can now mount this drive:

    $ sudo mount -t ext4 /dev/sda1 /mnt/DISK_NAME

    At last, your new drive is ready to use!

    References

    1. Remove Partition Linux ↩

    2. Format Drive Linux ↩

  • Damn I Love Docker: UniFi Controller

    Nov 23, 2018 •

    Table of Contents

    Installing UniFi Controller

    AMD64

    docker create \
  --name=unifi \
  -v <path to data>:/config \
  -e PGID=<gid> -e PUID=<uid>  \
  -p 3478:3478/udp \
  -p 10001:10001/udp \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 8443:8443 \
  -p 8843:8843 \
  -p 8880:8880 \
  -p 6789:6789 \
  linuxserver/unifi

    ARMhf

    To run the dockerized UniFi Controller on an armhf device (like a Raspberry Pi),

    docker create \
  --name=unifi \
  -v <path to data>:/config \
  -e PGID=<gid> -e PUID=<uid>  \
  -p 3478:3478/udp \
  -p 10001:10001/udp \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 8443:8443 \
  -p 8843:8843 \
  -p 8880:8880 \
  -p 6789:6789 \
  lsioarmhf/unifi

    References

  • Damn I Love Docker: Plex Media Server

    Nov 23, 2018 •

    Table of Contents

    Introduction

    When you begin to research “how” to stream your media collection to your devices, you will find one simple conclusion is unavoidable: the most common way to stream media to your devices is Plex.

    Plex is everywhere, and it is used ALL the time !!!

    Installing Plex

    In the below sections, you will find the commands for creating a docker container of the LinuxServer.io1 Plex Media Server.2 Thanks to the introduction of docker manifest you do not need to worry about the architecture of your docker host (e.g. amd64 vs. arm), just execute the following commands on your docker host.3

    Before looking at the command, it is useful to clarify and briefly explain some of the options and variables passed to the docker create4 command:

    • --net=host - Shares host networking with container, required.
    • -v /config - Plex library location. This can grow very large, 50gb+ is likely for a large collection.
    • -v /data/xyz - Media goes here. Add as many as needed e.g. /data/movies, /data/tv, etc.
    • -e VERSION=latest - Set whether to update plex or not - see Setting up application section.
    • -e PGID= for for GroupID - see below for explanation
    • -e PUID= for for UserID - see below for explanation
    • -e TZ - for timezone information eg Europe/London, Asia/Shanghai, etc

    Sometimes when using data volumes (-v flags) permissions issues can arise between the host OS and the container. This can be avoided by specifying the user PUID and group PGID. Ensure the data volume directory on the host is owned by the same user you specify and it will “just work.”

    To find yours use id $USER as below:

    $ id $USER
    uid=1001(dockeruser) gid=1001(dockergroup) groups=1001(dockergroup)

    So from this information, we can see that uid=1001, and gid=1001. Hence when we run the docker create command below, we will set PUID=1001 and PGID=1001.

    What follows is the docker command for pulling the LinuxServer.io1 docker image and creating a container to run the Plex Media Server.

    docker create \
--name=plex \
--net=host \
-e PUID=<UID> -e PGID=<GID> \
-v </path/to/library>:/config \
-v <path/to/tvseries>:/data/tvshows \
-v </path/to/movies>:/data/movies \
linuxserver/plex

    NOTE: As an example for the Raspberry Pi, we will likely want to consider using an external drive to store the Movies and the Plex configuration directories. This is due to the more robust I/O capabilities of a Hard Disk Drive vs. a micro SD card.5 More information on the formatting process of the drive can be found in another post6 devoted to reformatting a hard disk drive in ext4.

    Once we have our external drive ready (in this case mounted to /mnt/PIDRIVE), we can create the container as follows:

    docker create \
--name=plex \
--net=host \
-e VERSION=latest \
-e PUID=1001 \
-e PGID=1001 \
-e TZ='America/Chicago' \
-v /mnt/PIDRIVE/plex_config:/config \
-v /mnt/PIDRIVE/movies:/data/movies \
linuxserver/plex

    Notice here that we have PUID=1001 and PGID=1001, TZ=America/Chicago, /mnt/PIDRIVE/plex_config:/config, and /mnt/PIDRIVE/plex_config:/data/movies. Basically, the results of the id $USER showed uid=1001 and gid=1001, hence the user set PUID=1001 and PGID=1001, the user set the timezone to America/Chicago, and has created their Plex config dir on their external drive at /mnt/PIDRIVE/plex_config with all of their media on the same external drive at /mnt/DATADRIVE/movies.

    Running Plex

    Now that our docker container has been created, we can simply run it with:

    $ docker start plex

    This will start the container, and run it as a daemon (i.e. in the background).

    Accessing Plex

    Now that the docker container for Plex has been built and is running on your server, you can access it. The most basic way to access the Plex application is through the WebUI by navigating in your browser to the following URL:

    HOST_IP_ADDRESS:32400/web

    This could be something like:

    raspberrypi.local:32400/web

    Or maybe:

    gnosis.local:32400/web

    Or simply use the local IP address of the host:

    192.168.10.20:32400/web

    Initial Configuration

    Without going into too much detail, the initial setup will require two parts:

    • 1: You will be prompted to login or create a Plex account
    • 2: You must select where the media is stored that will be used with Plex

    Basically after creating a Plex account, or logging in to an existing account just make sure to add a Movie library, and use the path to /data/movies as the path to where the media are located. More detailed information can be found in the official Plex post7

    Advanced Configurations

    In this section we will discuss the various advanced features of both Plex, and the host operating system for configuring your system for various needs (e.g. backing up your media, downloading subtitles automatically, etc …)

    Backing Up Media and Data

    One of the best ways to backup any Unix-like system is through a utility known as cron. While we will not discuss in detail how to edit a crontab, we will refer you to a previous post on the subject.

    Instead we will show you an example of a crontab entry that will backup the external drive containing all your media files and Plex data to another external drive:

    $ sudo crontab -e
...
...
...
# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,$
# |  |  |  |  |
# *  *  *  *  * user-name  command to be executed

# backup PIDRIVE to PIBACKUP
01 01 * * * sudo rsync -havP /mnt/PIDRIVE/ /mnt/PIBACKUP/

    What the above example shows, is that the rsync command will run with super user privileges at 01:01 am or one minute past one in the morning (or late at night if you are not a morning person). When it runs it will backup the contents of the drive at /mnt/PIDRIVE to /mnt/PIBACKUP. This will ensure that every night at one minute past one in the morning, the drive with all your Plex data and media, will be copied to the drive at /mnt/PIBACKUP

    References

    1. LinuxServer.io ↩ ↩2

    2. Plex Media Server ↩

    3. Docker Manifest ↩

    4. Docker Create ↩

    5. HDD vs. Micro SD Card ↩

    6. ext4 Reformatting in Linux ↩

    7. Plex Basic Setup ↩

  • Meta Solution: The Structure of Problem Solving

    Sep 3, 2018 •

    Table of Contents


    Introduction

    In my personal experience with problem solving I have found that time and time again, there is no real “philosophy” of problem solving.1 And with that lack of literature, I have been tasked with defining my own philosophy of sorts for understanding, and thinking about problem solving.

    To that purpose, I have frequently found my self thinking most determinedly on a singular question: Is there such a process that will ensure, know matter what is known or unknown to you, the acquisition of the solution to the problem you are faced with? The short answer is yes: structure. The long answer is where we shall focus our explication.

    The Nature of Problems

    Searching Algorithms

    References

    1. Problem Solving Wiki ↩