May 222016
 

Synology introduced Docker to the DSM in version 5+, which has opened up new ways of running some applications in Synology. Not all Synology boxes have enough resources to run docker, check if your Synology NAS can run Docker here.  Using Mike Tabor’s excellent tutorial post I was able to get CrashPlan up and running on my 713+ DiskStation. However, his tutorial only covers the basic CrashPlan application, and not CrashPlanPRO. I’ve written up a quick and dirty guide to get CrashPlan or CrashPlanPRO working on docker in Synology.

Prepare the Synology NAS for Docker

  • Login to Synology, go to Package Center
  • Uninstall CrashPlan from Package Center (if previously installed)
  • Install Docker in Package Center
  • Connect to Synology via SSH as a user with Admin privileges

Get the latest Docker Images for CrashPlan or CrashPlanPRO

First, we will pull a CrashPlan or CrashPlanPRO image from docker hub.  Note that if connected via DSM6, we will need to start the session with “sudo su”, or add “sudo” before each of the docker commands:

If using CrashPlan

If using CrashPlanPRO:

Create and run CrashPlan container using the image.

If multiple volumes need to be specified, add another -v statement after the first one e.g. “-v /volume2/:/volume2”

If using CrashPlan:

If using CrashPlanPRO:

Wait until the container has fully started up (typically less than 2 minutes)

Get the GUID from CrashPlan

In the same SSH session, enter into a bash session for the container:

The command prompt will now take place inside the container. You may see an error which is safe to ignore for now: “ERRO[0000] Unable to connect to local syslog daemon”. In the bash session, enter the following command to get the headless key

It will output something similar to the following. The first number is the port used, 4243. After the comma, the guid key is listed, then the IP for the Synology NAS. In this example below, the “4fx72835-3fcd-4319-afd5-698212c77ba7” key should be recorded to connect to the system with CrashPlan from a desktop machine

4243,4fx72835-3fcd-4319-afd5-698212c77ba7,0.0.0.0

Update the workstation to connect to CrashPlan running in Docker

.ui_info

If using Windows, update C:\ProgramData\CrashPlan\.ui_info with the guid recorded in the command above, and the IP address of the Synology NAS. If on a Mac, the path is /Library/Application Support/CrashPlan/.ui_info.  For example:

4243,4fx72835-3fcd-4319-afd5-698212c77ba7,10.0.0.99

ui.properties

If using Windows, update C:\Program Files\CrashPlan\conf\ui.properties and ensure serviceHost is enabled and pointing to the Synology NAS. On a Mac, this path is typically /Library/Application Support/CrashPlan/conf/.ui_info.  For example:

serviceHost=10.0.0.99

Connect to CrashPlan

After connecting successfully to the CrashPlan instance, adopt a previous computer (if applicable) to ensure the backup history and settings are retained.

If you have large backup sets and memory needs to be adjusted, double click the CrashPlan logo in the top right of the program and apply a custom command. In the example command below, the memory usage is increased to 2GB in order to work with very large file sets. Do not increase this value beyond what is available on the Synology NAS.  More information on this command can be found here.

java mx 2048, restart

Troubleshooting

CrashPlan on Docker can’t see my files

Sometimes the mapping for the root volume doesn’t seem to stick for sub shares in the container. To fix this, stop the container and edit it in Docker. Add each share used in CrashPlan backups to the container and mount them the same way they were referenced originally. For example, if CrashPlan backs up files on a share named “Photo” on volume1, then add that share to the container with a mount path of “/volume1/Photo”.

CrashPlan Updated to v4.8 and now my docker image is broken

The following reply by reddit user ajr6037 has a solution: https://www.reddit.com/r/synology/comments/550806/best_crashplan_docker_image/d87beo9

Jul 022015
 

CrashPlan Issue For Synology – Fixing the 4.3.0 update

If your Synology NAS can run docker, you may want to consider an alternative to the CrashPlan package. See here: CrashPlan and CrashPlan Pro on Synology using Docker

Note, see bottom of page for information on DSM 6 and CrashPlan 4.6.0

Recently CrashPlan updated to version 1427864400430, which broke the CrashPlan package Patters had created for Synology installations again. They then (July 2nd?) released 1427864410430 as well, which means that if your CrashPlan installation on Synology broke with version 1427864400430, you will need to upgrade, wait for it to break again, and then fix with version 1427864410430. In general the fix process has been the same over the last few breaks, and I’ve outlined those steps below for the latest version.

There is one major change introduced from CrashPlan. Starting with the 1427864400430 update, a guid (think of it as a key) is needed to securely log into the headless client. Thanks to Mathew Stocks, I’ve made those instructions available as well.

The log for the failing CrashPlan package would look something like this for a failing 1427864400430 upgrade:

Upgrades available at central.crashplan.com:443
Downloading a new version of CrashPlan.
Download of upgrade complete – version 1427864400430.
Installing upgrade – version 1427864400430
Upgrade installed – version 1427864400430
CrashPlan stopped, version 4.2.0, GUID 559750658046558476

If the package was attempted to start again, it would write a line similar to the following, and then stop:

Synology repairing upgrade in /var/packages/CrashPlan/target/upgrade/1427864400430.

Note that the following instructions are for the 1427864410430 upgrade, which is the latest (as of July 2nd, 2015) update. If your Synology didn’t make it to this version, run the following steps, but substitute “1427864400430” where there are any references to “1427864410430”.

Connect as root to SSH

First, connect to Synology using SSH and the root account (uses admin password).  Connecting as the admin account will give you permission failures!

In SSH, run commands to extract the update

In SSH, run unique command to cleanup the package

The next command will be different for everyone, as part of the file path is randomized.  In this case, you will need to see what your path is first by running ls, and then editing the second statement below, replacing whatevervalue with the rest of  your path:

Start CrashPlan package, check log

After the above completes, you should be able to start the CrashPlan package again in Synology and get it running once more. The log should show:

CrashPlan started, version 4.3.0 GUID 559750658046558476

Get the headless key from the .ui_info file

While connected to SSH as root, run the following command to get the headless key file. The funky formatting of this command is because the file does not contain the special newline character:

The command will output a line similar to the following (everyone’s will be different):
4243,13d436c0-230a-4242-b258-574e60e62a9f

Update .ui_info on the computer connecting to Synology

On the computer where you are connecting to CrashPlan on Synology, go to the .ui_info file under the CrashPlan directory. NOTE: I had to manually upgrade my computer’s version of CrashPlan (via https://www.code42.com/crashplan/thankyou/) in order to see the .ui_info file.

Typically on Windows, this will be found under:
C:\ProgramData\CrashPlan\.ui_info

Or on a Mac in the following location (need to have finder set to show hidden files):
/Library/Application Support/CrashPlan/.ui_info

Enter in the new key in the file, replacing the current value, and save it. You should now able able to connect to the headless client. Note that you may need to also update the values in ui.properties again if you had to upgrade CrashPlan.

CrashPlan 4.6.0

There are some tools referenced in the comments for this page which might make upgrades easier for some people. But I’ll also give the latest files & paths using the original instructions above – the following commands can be used for v4.6.0:

unzip -o /var/packages/CrashPlan/target/upgrade/1435813200460_382.jar "*.jar" -d /var/packages/CrashPlan/target/lib/
unzip -o /var/packages/CrashPlan/target/upgrade/1435813200460_382.jar "lang/*" -d /var/packages/CrashPlan/target/

Find latest package folder:
ls -l /var/packages/CrashPlan/target/upgrade/1435813200460_382.*

Replace the whatevervalue with the folder name:
mv /var/packages/CrashPlan/target/upgrade/1435813200460_382.whatevervalue/upgrade.sh /var/packages/CrashPlan/target/upgrade/1435813200460_382.whatevervalue/upgrade.sh.old

Synology DSM v6

I don’t recommend upgrading to v6 at this time (March 2016) if you do not have an Intel based CPU in your Synology NAS as there have been issues reported with the java packages in the comments here as well as on Patter’s page. If you do have an Intel based CPU you will need to use 7z commands on DSM6 instead of zip commands. Examples for CrashPlan 4.6.0:
sudo 7z e -y -o/var/packages/CrashPlan/target/lib /var/packages/CrashPlan/target/upgrade/1435813200460_382.jar "*.jar"
sudo 7z e -y -o/var/packages/CrashPlan/target/lang /var/packages/CrashPlan/target/upgrade/1435813200460_382.jar "lang/*"

May 122015
 

The latest round of updates from CrashPlan (4.3.0) has broken the CrashPlan package on Synology again. Please see http://chrisnelson.ca/2015/07/02/fixing-crashplan-4-3-0-on-synology/ for the newest instructions.

Archived information for 4.2:

After installing Synology DSM 5.2, CrashPlan recently pushed an update (4.2.0) which broke the CrashPlan package Patters had created for Synology. The log for the failing CrashPlan package would look something like this:

CrashPlan started, version 3.7.0, GUID 559750658046558476
Upgrades available at central.crashplan.com:443
Downloading a new version of CrashPlan.
Download of upgrade complete – version 1425276000420.
Installing upgrade – version 1425276000420
Upgrade installed – version 1425276000420
CrashPlan stopped, version 3.7.0, GUID 559750658046558476

If the package was attempted to start again, it would write a line similar to the following, and then stop:

Synology repairing upgrade in /var/packages/CrashPlan/target/upgrade/1425276000420.

This seemed to be similar to the previous issue seen in 3.7.0, so I used the following steps to fix crashplan on my synology box: Note that this will likely be fixed by Patters soon, this is just a workaround if your CrashPlan install has stopped as a result of the 4.2.0 update.

Connect as root to SSH

First, connect to Synology using SSH and the root account (uses admin password).  Connecting as the admin account will not give you enough permissions.

In SSH, run commands to extract the update

In SSH, run unique command to cleanup the package

The next command will be different for everyone, as part of the file path is randomized.  In this case, you will need to see what your path is first by running ls, and then editing the second statement below, replacing whatevervalue with the rest of  your path:

Start CrashPlan package, check log

After the above completes, you should be able to start the CrashPlan package again in Synology and get it running once more. The log should show:

CrashPlan started, version 4.2.0 GUID 559750658046558476

Jan 102015
 

If you recently upgraded to DSM 5.2 and had CrashPlan break, this is likely due to the CrashPlan 4.2.0 update. Please see http://chrisnelson.ca/2015/05/12/fixing-crashplan-4-2-0-on-synology-after-dsm-5-2-update/

NOTE: This manual fix should no longer be needed after the 0029 update from Patters:

0029 Updated to CrashPlan version 3.7.0, improved detection of temp folder (prevent use of /var/@tmp), added support for Annapurna Alpine AL514 CPU (armhf) in DS2015xs, added support for Marvell Armada 375 CPU (armhf) in DS215j, abandoned practical efforts to try to support Code42’s upgrade scripts, abandoned inotify support (realtime backup) on PowerPC after many failed attempts with self-built and pre-built jtux and jna libraries, back-merged older libffi support for old PowerPC binaries after it was removed in 0028 re-write

Archive content:
CrashPlan recently pushed an update (3.7.0) to all clients which broke the CrashPlan package Patters had created for Synology. The log for the failing CrashPlan package would look something like this:

CrashPlan started, version 3.6.4, GUID 574956158631543214
Upgrades available at central.crashplan.com:443
Downloading a new version of CrashPlan.
Download of upgrade complete – version 1388728800370.
Installing upgrade – version 1388728800370
Upgrade installed – version 1388728800370
CrashPlan stopped, version 3.6.4, GUID 574956158631543214

If the package was attempted to start again, it would write a line similar to the following, and then stop:

Synology repairing upgrade in /var/packages/CrashPlan/target/upgrade/1388728800370.

After some investigation by Stig, Casper, and Harv on the comments for the package a working solution was found to manually install the update files and disable the failing upgrade.sh file. Note that this will likely be fixed by Patters soon, this is just a workaround if your CrashPlan install has stopped as a result of the 3.7.0 update.

1. First, connect to Synology using SSH and the root account (uses admin password).  Connecting as the admin account will not give you enough permissions.
2. Run the following commands:

 
3. The next command will be different for everyone, as part of the file path is randomized.  In this case, you will need to see what your path is first by running ls, and then editing the second statement below, replacing whatevervalue with the rest of  your path:

4. After the above completes, you should be able to start the CrashPlan package again in Synology and get it running once more.

Apr 222014
 

What is Monit?

The simplest way of describing the value Monit is that it is a powerful monitoring tool, which can take action in the event of failures or alerts. Monit is able to keep an eye on processes, files (size, permissions, checksum), the system (CPU, RAM, Load), as well as protocols (HTTP, FTP, SMTP, SSH, etc). If any of the monitored items change in specific ways, Monit can spring into action with an alert, run separate programs, or attempt to correct the problem with a prescribed action (e.g. restart a service).

Monit web interface

How do I install Monit?

Installing Monit is fairly easy these days, as the majority of Linux repositories include the package. If you were running Ubuntu, you could issue the following command to install the package:

If you were running Redhat/CentOS, you could issue the following command to install the package:

How do I configure Monit?

After Monit has been installed, all configuration changes can be made via the monitrc file via text editor, in this case I’m using vi. Note that all of the paths for pid files and other items in my examples are for Ubuntu – you may need to do some edits if you are on a different version of Linux.

Once the file has been loaded in the text editor, you will see that there are already a number of options set (e.g. Monit is set to check every 2 minutes by default) in addition to a number of items which are commented out. For this example I’m not changing any of the default values, and you can move to the bottom of the file and begin adding new entries to be monitored by Monit.

Add email notification

First I will configure Monit to use a the local system as the default email server, set the format of the alert/status emails to be delivered, and the email address I want email alerts to go to. If you are able to, it is best to setup an alias email address to use for monitoring (e.g. monitoring@yourdomain.com), that way you can easily add additional rules or filters for your email based on the email address.

Add the Monit web interface

Monit has a nice web interface included, but is disabled by default. One important thing to note – the web interface allows the user to start, stop, and restart monitored services, as well as make monitoring changes. As such, I highly recommend using an extremely long and complex password in addition to a non standard user name for the account specified in this step to enable the web interface (format: allow accountname:password). It may also be in your best interest to limit the port used (2812 in this example) via firewall to your own IP address.

Monitor disk usage

Monit has the ability to monitor disk usage for a system, in addition to specific files. In this case, the monitor has been configured to watch the free space available on the /dev/vda drive, and send an alert if it is 95% full. The group tag seen in this example is used to organize different monitors, it doesn’t impact the usage in any way.

Monitor system load

Monit can also monitor the host system’s load – in this case sending an alert if the load average over the course of 5 minutes is larger than 3.

Monitor MySQL

In this example, Monit is monitoring MySQL in two different ways. The first way is via the specified pid file – Monit is reading that file, determining the process id, and then keeping an eye on that process on the system. The second way is via service testing – Monit is checking that MySQL has port 3306 open with the mysql specific protocol (it supports a number of options). If the process fails either of those monitoring checks, then the stop & start program scripts are run, restarting MySQL.

Monitor SSH

In this example Monit is checking the SSH daemon via pid and SSH protocol service response.

Monitor Apache2

In this example, the Apache web server is being checked four different ways. The first two have already been shown in previous examples, monitoring via pid and service protocol (in this case http). However, just because the Apache webserver is running doesn’t mean it is properly serving files. In this example, Monit is asking for a specific URL, “/monit/token” from the web server to verify it is able to serve static content. Monit is also keeping an eye of the cpu usage of this process – it is alerting when CPU usage is high, but it could also restart the service on extended CPU usage if desired.

One thing to note about the above monitor – it will fail unless the Apache web server serves back a monitor token file – we need to create that file for the web server to use first:

Monitor crashplan backup service

I’m a fan of CrashPlan and I use it on all of my machines, including my servers. However, I’ve found that the CrashPlan backup engine can be unstable on some Linux distros when on a server with limited memory, so I’ve always added it as a monitored service to keep an eye on it and restart it if it shuts down:

Implement the Monit changes

After making changes to the Monit configuration file, save it and have Monit reload to see the changes. If you are not sure if the Monit service is not already running (e.g. it is a new install), then the following command can be used:

If Monit is already running, then you can just tell it to reload the configuration file:

Hopefully this introduction is enough to get your creative monitoring juices flowing. There are lots of other examples for monitoring different things using Monit, and the platform is flexible enough that it can almost do anything you need.