Apr 222014

What is Monit?

The simplest way of describing the value Monit is that it is a powerful monitoring tool, which can take action in the event of failures or alerts. Monit is able to keep an eye on processes, files (size, permissions, checksum), the system (CPU, RAM, Load), as well as protocols (HTTP, FTP, SMTP, SSH, etc). If any of the monitored items change in specific ways, Monit can spring into action with an alert, run separate programs, or attempt to correct the problem with a prescribed action (e.g. restart a service).

Monit web interface

How do I install Monit?

Installing Monit is fairly easy these days, as the majority of Linux repositories include the package. If you were running Ubuntu, you could issue the following command to install the package:

If you were running Redhat/CentOS, you could issue the following command to install the package:

How do I configure Monit?

After Monit has been installed, all configuration changes can be made via the monitrc file via text editor, in this case I’m using vi. Note that all of the paths for pid files and other items in my examples are for Ubuntu – you may need to do some edits if you are on a different version of Linux.

Once the file has been loaded in the text editor, you will see that there are already a number of options set (e.g. Monit is set to check every 2 minutes by default) in addition to a number of items which are commented out. For this example I’m not changing any of the default values, and you can move to the bottom of the file and begin adding new entries to be monitored by Monit.

Add email notification

First I will configure Monit to use a the local system as the default email server, set the format of the alert/status emails to be delivered, and the email address I want email alerts to go to. If you are able to, it is best to setup an alias email address to use for monitoring (e.g. monitoring@yourdomain.com), that way you can easily add additional rules or filters for your email based on the email address.

Add the Monit web interface

Monit has a nice web interface included, but is disabled by default. One important thing to note – the web interface allows the user to start, stop, and restart monitored services, as well as make monitoring changes. As such, I highly recommend using an extremely long and complex password in addition to a non standard user name for the account specified in this step to enable the web interface (format: allow accountname:password). It may also be in your best interest to limit the port used (2812 in this example) via firewall to your own IP address.

Monitor disk usage

Monit has the ability to monitor disk usage for a system, in addition to specific files. In this case, the monitor has been configured to watch the free space available on the /dev/vda drive, and send an alert if it is 95% full. The group tag seen in this example is used to organize different monitors, it doesn’t impact the usage in any way.

Monitor system load

Monit can also monitor the host system’s load – in this case sending an alert if the load average over the course of 5 minutes is larger than 3.

Monitor MySQL

In this example, Monit is monitoring MySQL in two different ways. The first way is via the specified pid file – Monit is reading that file, determining the process id, and then keeping an eye on that process on the system. The second way is via service testing – Monit is checking that MySQL has port 3306 open with the mysql specific protocol (it supports a number of options). If the process fails either of those monitoring checks, then the stop & start program scripts are run, restarting MySQL.

Monitor SSH

In this example Monit is checking the SSH daemon via pid and SSH protocol service response.

Monitor Apache2

In this example, the Apache web server is being checked four different ways. The first two have already been shown in previous examples, monitoring via pid and service protocol (in this case http). However, just because the Apache webserver is running doesn’t mean it is properly serving files. In this example, Monit is asking for a specific URL, “/monit/token” from the web server to verify it is able to serve static content. Monit is also keeping an eye of the cpu usage of this process – it is alerting when CPU usage is high, but it could also restart the service on extended CPU usage if desired.

One thing to note about the above monitor – it will fail unless the Apache web server serves back a monitor token file – we need to create that file for the web server to use first:

Monitor crashplan backup service

I’m a fan of CrashPlan and I use it on all of my machines, including my servers. However, I’ve found that the CrashPlan backup engine can be unstable on some Linux distros when on a server with limited memory, so I’ve always added it as a monitored service to keep an eye on it and restart it if it shuts down:

Implement the Monit changes

After making changes to the Monit configuration file, save it and have Monit reload to see the changes. If you are not sure if the Monit service is not already running (e.g. it is a new install), then the following command can be used:

If Monit is already running, then you can just tell it to reload the configuration file:

Hopefully this introduction is enough to get your creative monitoring juices flowing. There are lots of other examples for monitoring different things using Monit, and the platform is flexible enough that it can almost do anything you need.

Mar 182014

I can’t say this about many things, but I agree with Pat Buchanan’s analysis of Putin, Russia, and Ukraine in his posting: Is Putin the Irrational One?

If we Americans want out of Afghanistan, why would Putin want to go back into Uzbekistan? Why would he want to annex Western Ukraine where hatred of Russia dates back to the forced famine of the Stalin era?

Since the viewpoint repeated in western media these days regarding Russia is very one sided, it is worth taking a look at things from another perspective.

Mar 052014

I finally got around to listening to Dan Carlin’s podcast titled Common Sense #270 – Poking the Bear which was published last week. It gets off to a slow start, but I think he does a great job of portraying the larger scope and the lead up which brings us to the situation today around Russia, Ukraine, and the USA. I highly recommend taking a listen if you are interested in learning more about the current environment and motivations.

Feb 222014

DER SPIEGEL has a good overview of some of the results published recently regarding the DNA analysis of a body found in Montana on private land. Its location means it is one of the only sets of remains which have been permitted to be tested:

The characteristic fluting of the stone weapons serve as archaeological evidence that the boy, who died some 12,600 years ago, came from the Clovis culture. It was one of the earliest New World groups, disappearing mysteriously a few centuries after the child’s burial in present day Montana. …Now a team of scientists led by the Danish geneticist Eske Willerslev has analyzed the boy’s origins and discovered that he descends from a Siberian tribe with roots tracing back to Europe. Some of the boy’s ancestors are likely even to have lived in present-day Germany. Their findings go even further: More than 80 percent of all native peoples in the Americas — from the Alaska’s Aleuts to the Maya of Yucatan to the Aymaras along the Andes — are descended from Montana boy’s lineage.

The article also touches on some of the concerns in the native community around DNA testing – there is no appetite for being linked to Europe populations, and very real concerns that DNA testing could be used in tribal disputes over who shares in the economic bounty from casinos that operate on the sovereign reservations.

Feb 172014

While visiting the Caribbean island of Saint Barthélemy (aka St Barts, aka St Barth) by ship we were told by our friends that they had explored an abandoned house years ago in the Anse a Colombier area of St Barth. When we arrived in the bay we could see the house structures were still there and we were keen to check it out. I packed my camera in a dry bag and we made our way to shore.

On the shore one could see that there at some point had been a nice cement construction of a wave break and stairs down to the ocean. Following those stairs up we came to a fence, which was easily bypassed. Wandering through the overgrown bush we eventually came to the top of the hill and saw the top of wave shaped roof. It was clear that nature had mostly taken over, but other areas were in pretty good shape – including a road to the property which seemed to be maintained.

Click here to see all photos for the abandoned Rockefeller House on Saint Barthélemy Island

The patio at one point must have had an amazing view, but the vegetation meant standing on your toes to see the view of Colombier beach. What I assume was the main living room had outdoor furniture packed up inside, as if it was ready to be put out after a rain storm. Strangely there was a new looking outdoor play set for kids, perhaps caretakers had set something up for keeping their kids happy while they worked?

Following the road leading from the front of the house down the other side of the hill we came to what looked to be a boathouse at some point in time. On the other side of the ridge was the strange lighthouse/guesthouse structure. It was in quite bad shape and the wood structure was heavily weathered. Three active bee hives sat on the structures patio, so someone has clearly been visiting. There were also a number of what looked to be fruit trees, though it wasn’t clear if they were leftovers from the occupants or new with the beekeeper.

When we returned home my thoughts turned to the house again. Clearly someone had spent a tremendous amount of money on the property, and it was in an incredibly desirable location. After researching further on the internet I eventually linked the house to David Rockefeller. There is a fair bit of speculation on what exactly happened, but the following is the best I could piece together through various sources:

The Colombier/Rockefeller land was purchased in the mid 60′s by David Rockefeller after seeing it from a sailboat. David Rockefeller built the house and guesthouse (the lighthouse structure) with designs from his cousin architect Nelson W. Aldrich in a “Caribbean parabolic” style. There were no services so the property was built to be self sufficient. In 1983 the house was sold to a NY developer named Jim Harrison. Harrison planned to build 24 houses but was subsequently denied building permits after a long court battle involving Mr Daniel Blanchard (served as mayor of the island for almost 15 years). The legal battle ended 1990 and Mr Harrison put the property for sale early 1991. It was sold in 1996 to the present owner Mr Horn. Mr Horn is getting older so there is much speculation about its sale to the many billionaires (like Roman Abramovich) who frequent the island and bay.

The home was also profiled in a 1983 Architectural Digest, and additional information about it is included in the article text:

I’ve always been fascinated by abandoned buildings and towns. Some are more interesting than others, when it is clear that significant time and money was spent. It is a bit like dismantling a failed relationship, what changed so that the fate shifted to neglect? Clearly in this case the building was sold and expected to be bulldozed, instead it is still there. There are more photos of the property which are not included above, Click here to see all photos for the abandoned Rockefeller House on Saint Barthélemy Island

Jan 202014

Tijuana, Ensenada, Rosarito, Puerto Nuevo, Guadalupe Valley. Just a few of the Baja, Mexico destinations which used to be easy and popular day trips from San Diego. In the space of a few years major changes completely changed tourism near the border.  The first was increased security at the border by the Department of Homeland Security.  The pool of available tourists was dramatically lowered by requiring passports to cross into the USA from Mexico as roughly 1/3rd of Americans hold a passport (though growing).  This increased security also lead to an increase in border wait times.  Instead of spending an hour or perhaps two at the worst waiting at the border, there began to be an increase in three and four hour waits.  The second major impact to cross border tourism was an outburst of drug war related violence.  Though mostly targeted towards narcos and those working with them, this bloody war spilled over in several cases and fed fear and general distrust of Tijuana and other border cities.

Though the narco violence subsided in this area years ago, memories take much longer to dissipate.  Given time things are starting to turn around. Without the corruption of short term (and usually debauched) cash along Revolución, Tijuana and others have looked inward to reinvent themselves.  This change over the last few years has lead to them becoming a bit of a destination for foodies and culture lovers – attracting the like of Bourdain and others to explore the new Baja.

We used to visit the coast of Baja (Rosarito, Puerto Nuevo, and occasionally Guadalupe or even further south like Bahia de los Angeles) on a regular basis, going down for lunch and shopping before returning for the day.  With the border waits we had fallen out of the habit some time ago and had yet to pick it back up again – We finally got around to visiting one of our old standards with family on Sunday, lunch in Puerto Nuevo. The toll road was washed out so we spent time on the free road driving down. I was pleasantly surprised to see that much of the route was four lanes wide and in excellent shape. The shops along the road were in mixed shape – some still seeming to be going strong, others didn’t seem to have made it through the drought.  We spoke with a few shopkeepers who said business had been slowly picking up and they were hopeful for the future.  I think we will be picking our habit back up – Baja offers some excellent opportunities for day trips.

Link to the full gallery of photos