Tuesday, 14 October 2008

Switching from CVS to Baazar Version Control System



(note : this post will be improved soon)

I'm changing my server to a new machine as the existing one as capacitor leak issue on the motherboard...
I switch from sme server to ubuntu server.
And as I was almost finishing the migration of my cvs repository to the new server, I asked myself : and what about SVN ? and Bazaar on which I've read about on mysql developer site ? And git, the VCS used for the linux kernel made by Linus Torvald himself (GIT and the Kernel)

Here is a site that help me to have an idea of the feature of each VCS :

http://www.versioncontrolblog.com/comparison/Bazaar/CVS/Git/Subversion/index.html

I've made a try with GIT, but due to a poor eclipse plugin which development seems to have stalled. I've chosen to try Bazaar.

And here is links that answer my main concern :


I've also mainly choose Bazaar because I already know cvs quite well, and made some try on SVN, but didn't know Bazaar at all (And I'm curious).

I really like to thanks the people on the Bazaar mailing list for their help.

I've started to install the eclipse plugin to see if ssh connection was possible.

Eclipse plugin install



I've installed it by following the install instruction.

which says :
  1. Install bazaar where you're running eclipse
  2. Install bazaar xml output plugin
  3. Install the Eclipse plugin

I've downloaded the latest available version here (window version for now) :

https://launchpad.net/bzr/+download

check md5 with my portableapps WinMD5sum

Run the installer, choose the install path.

Next I downloaded the bzr-XMLOutput windows plugin installer and make a md5 sum check.
Run the installer, use the default directory.

Then I've installed the eclipse plugin, Help->Software Update, change to "Available software" tab, click on add site button, paste

http://verterok.com.ar/bzr-eclipse/update-site/

Click Ok, expand the newly added site, tick the "Eclipse plugin for Bazaar" and click the Install button in the upper right corner.


Once installed, I've setup the server side on my new ubuntu server for the CVS to Bazaar conversion.

CVS to Bazaar conversion



In order to have a fairly recent release of Bazaar on ubuntu you have to modify the apt sources list, or you'll only get the 1.3.0 version of Bazaar whereas the 1.8.0 is about to be released.

sudo vi /etc/apt/sources.list

Add the following two lines to the file :
deb http://ppa.launchpad.net/bzr/ubuntu hardy main
deb-src http://ppa.launchpad.net/bzr/ubuntu hardy main


And then run :
sudo apt-get update
sudo apt-get install svn bzr bzrtools
mkdir -p ~/.bazaar/plugins
cd ~/.bazaar/plugins
bzr branch lp:bzr-fastimport fastimport #(1)
bzr checkout lp:bzr-cvsps-import cvsps_import #(2)

(1) is for conversion with CVS2SVN
(2) is for conversion with cvs2ps


(svn is needed to get one conversion tool)

Next you need to tell Bazaar who you are :

bzr whoami 'Thomas <yourmail@mailprovider.com>'


Now, here is the conversion part.

As I'm a soooooo lucky man, I had a filename encoding issue : a few file filename happen to be encoded in a non UTF-8/non ASCII encoding and this was really a big problem to the conversion tools of bazaar.
So I've tryed almost every conversion solution ;)

Unfortunately, the conversion instruction are a bit laconic. Thanks to the Bazaar mailing list and it's large and active community, I've been able to try each solution and finally get something working with my encoding issue (Hard to be French, I tell you ;)

CVS2SVN conversion tool



This tools seems to be the more accurate according to what I've read on the mailing list. (Although, this one crashed on my filename encoding issue)
There's no mistake, it's "CVS2SVN" not "CVS2BZR" or something.

This tool was initialy developed for CVS 2 SVN conversion, but it evolved to output a fileformat understandable by GIT (yes GIT...), and Bzr conversion tool also understand it as well ;)

Get CVS2SVN tool



Ubuntu Hardy Heron apt package is also old, so you should get it directly from the repository :



mkdir ~/temp
cd ~/temp
svn co --username=guest http://cvs2svn.tigris.org/svn/cvs2svn/trunk cvs2svn-trunk

simply press enter when asked for a password.

Now get your CVS repository backup file (tar jcf of the repository directory. If you can't have this try another solution)

cd
mkdir -p temp/cvsrepo
cd temp/cvsrepo
scp root@10.0.0.99:/backups/local/files/20081004_ibay_cvs.tar.bz2 .
tar jxf 20081004_ibay_cvs.tar.bz2
cd ..


Now your CVS Repository is here : ~/temp/cvsrepo and one of its subdirectory must be CVSROOT.
And the conversion tool is here : ~/temp/cvs2svn
The current working directory is ~/temp

Now we need to get some configuration file sample.
I won't give away my configuration files because, the tool maintainer says on its site that this configuration file format often change.

cp cvs2svn/cvs2svn-trunk/cvs2svn-example.options .
cp cvs2svn/cvs2svn-trunk/test-data/main-cvsrepos/cvs2svn-git.options .


In cvs2svn-git.options file, there's the path to the CVS repository to change :
look around the line 122 where you should see : r'test-data/main-cvsrepos',

and change it to the relative path to your cvs repo.
As my working directory will be ~/temp and that the cvs repo is in ~/temp/cvsrepo I've just put cvsrepo :

# Now add a project to be converted to git:
run_options.add_project(
# The path to the part of the CVS repository (*not* a CVS working
# copy) that should be converted. This may be a subdirectory
# (i.e., a module) within a larger CVS repository.
r'cvsrepo',

# See cvs2svn-example.options for more docume


Next I had to edit cvs2svn-example.options
because my commit comment were in latin1 encoding, not UTF-8 encoding, so in order to get rid of conversion Warning :

# How to convert author names, log messages, and filenames to unicode.
# The first argument to CVSTextDecoder is a list of encoders that are
# tried in order in 'strict' mode until one of them succeeds. If none
# of those succeeds, then fallback_encoder is used in lossy 'replace'
# mode (if it is specified). Setting a fallback encoder ensures that
# the encoder always succeeds, but it can cause information loss.
ctx.cvs_author_decoder = CVSTextDecoder(
[
'latin1',
#'utf8',
#'ascii',
],
fallback_encoding='ascii'
)
ctx.cvs_log_decoder = CVSTextDecoder(
[
'latin1',
#'utf8',
#'ascii',
],
fallback_encoding='ascii'
)


Then run the tool :

./cvs2svn/cvs2svn-trunk/cvs2svn --options cvs2svn-git.options


If all goes well, you should see a ~/temp/cvs2svn-tmp directory with 2 files in it :

thomas@home:~/temp/cvs2svn-tmp$ ll
total 238720
drwxr-xr-x 2 thomas thomas 4096 2008-10-07 17:48 .
drwxr-xr-x 11 thomas thomas 4096 2008-10-13 13:40 ..
-rw-r--r-- 1 thomas thomas 241749759 2008-10-07 17:47 git-blob.dat
-rw-r--r-- 1 thomas thomas 2444025 2008-10-07 17:48 git-dump.dat


That's what GIT and Bazaar conversion tools understand.

Here is how you convert to Bazaar :
mkdir ~/temp/bzr
cd ~/temp/bzr
bzr init-repo .
cat ../cvs2svn-tmp/git-blob.dat ../cvs2svn-tmp/git-dump.dat | bzr fast-import -

After this command has been executed, your directory should have been filled with your files. And the revisions of these files in .bzr directory.

Here was some clue for the instructions found on fast import page:

bzr init-repo .
front-end | bzr fast-import -


Too easy ;)


CVS2PS import



Install some needed tools ( you still need the bzr cvs2ps plugin (2))

sudo apt-get install cvs cvsps rcs

(Note: you can do without rcs, with --use-cvs but it's slower)

Your cvs repo is here : ~/temp/cvsrepo

mkdir ~/temp/bazaar
cd ~/temp
bzr cvsps-import cvsrepo . bazaar


It should convert your entire repository (~/temp/cvsrepo) into ~/temp/bazaar directory.
If you want to convert only one module, use your module name instead of '.' in the command.

Tailor



The last tool I tried to convert my CVS repo to Bazaar is Tailor.
It's this tools that succeed in converting my repository that have filename encoding.
Instead of crashing on files that have issue (only 2 for what I know), Tailor trap the exception and continue the conversion.
The two files I lost was doc files with nothing really important.

Thanks to Colin D Bennett (who apparently works on Grub) who gave me it's configuration files for Tailor I did try tailor and succeed. (I was about to give up the history of my project and import as a new project).

Tailor connect to your CVS repository and do not need to have a direct access to the repository files.

On my linux box, I use CVS through SSH, I'm not using a pserver.

Here is the content of one of my tailor configuration files :

thomas@home:~/temp$ cat crf-irp.tailor
[crf]
patch-name-format = None
source = cvs:crf-irp
target = bzr:crf-irp

[cvs:crf-irp]
repository = :extssh:thomas@my.server.com:/cvs
module = crf-irp
encoding = iso-8859-1

[bzr:crf-irp]




The first [crf] is a name you pick.
Next, source & target will be the name between bracket.
In the source section, you put your cvs configuration.
the target section doesn't need more instruction (bzr: seems to be enough).

I advise you to use a public key ssh authentication as it will ask you for the password several times.

Repeat for each module of your repository.

Publication of my (newly converted to Bazaar) project to Launchpad



Before Bazaar, I've set up on my own server a CVS server, JIRA/Confluence/Fisheye for the project handling.
As I've made no release yet (it's should happen soon), JIRA/Confluence was not really used.

Now I've migrate to Bazaar, I could set up a Bazaar server. But I did loose a huge amount of time on the Bazaar conversion (because of this ***king filename encoding issue) and I found it would be more efficient and more reliable to use Launchpad.net plateform.

So, I've created my project (http://launchpad.net/crf-irp), update my profile with my public key, load my private key into pagent.
and ran the following command from windows (My laptop unfortunately run under vista)

bzr push bzr+ssh://mylogin@bazaar.launchpad.net/~mylogin/crf-irp/crf-irp

bzr push bzr+ssh://mylogin@bazaar.launchpad.net/~mylogin/crf-irp/crf-irp-model

bzr push bzr+ssh://mylogin@bazaar.launchpad.net/~mylogin/crf-irp/crf-utilities


It took some times as my repo was about 300MB.


Next Blog post : How to import a project with the eclipse plugin.

Monday, 13 October 2008

Convert your CVS repository to GIT


While I was on sick list, this last two week, I've search to get away from the venerable CVS and it's painful merge capabilities. (Also I was curious to try something else).

Here is how I converted my CVS repository to git :

Install GIT :
sudo apt-get install git-core git-cvs


Install cvs for the conversion :

sudo apt-get install cvs cvsps



Get the CVS repository backup ( a tar jcf of the cvs root dir) :
mkdir ~/temp/cvs2git
cd ~/temp/cvs2git
scp root@10.0.0.99:/backups/local/files/20081006_ibay_cvs.tar.bz2 .
tar jxf 20081006_ibay_cvs.tar.bz2


Launch the conversion :

export CVSROOT=/home/thomas/temp/cvs2git/cvs/files

git cvsimport -C /home/thomas/temp/cvs2gitOutput/crf-irp           crf-irp
git cvsimport -C /home/thomas/temp/cvs2gitOutput/crf-irp-model     crf-irp-model
git cvsimport -C /home/thomas/temp/cvs2gitOutput/crf-irp-monitor   crf-irp-monitor
git cvsimport -C /home/thomas/temp/cvs2gitOutput/crf-irp-portail   crf-irp-portail
git cvsimport -C /home/thomas/temp/cvs2gitOutput/crf-irp-utilities crf-irp-utilities


It worked perfectly.

There's an eclipse plug-in for GIT, the install site is http://www.jgit.org/update-site

But it's not working well enough to be used. Also its development seems to have stalled.

That's why I tried Bazaar... but that's for another post.

Sunday, 12 October 2008

mount a NTFS partition on hardware RAID0 controller under linux


My main computer has a major hardware issue (probably the motherboard, maybe the cpu).
I can't re-install windows, it crashes in the middle of the installation.

To complicate things, I've installed my windows on a (fake)hardware RAID0. It's fake because the raid part is handle by the windows driver. That's why you can still see the hard drives under linux as /dev/sda /dev/sdb etc... even after having setup your RAID0/1/5 Array in the bios.

As I'm not completely dumb, I've stored no critical data on the RAID array...
But still, I wanted to check that fact (I'm not dumb ;) and get some non critical files, such as game save (I don't play that much recently... so sad, but it may change once Diablo III or Starcraft II is released).

To get my data back, I've used a linux Distribution and use it's DMRAID capabilities.

I've downloaded and burned the Gentoo Live CD.
BTW, it's a really great distribution. I loved to installed it from scratch and compile all from sources.

boot onto it.
At grub prompt I typed the following to get DMRAID support :

boot: gentoo dodmraid


Once the livecd start up (when it could with my hardware issues) with a nice 1920*1200 interface, I opened a shell and do the following :

sudo -s                                   #(0)
alias ll="ls -la"                         #this one should be always set by default !
mkdir /a                                  #(1)
ll /dev/mapper                            #(2)
total 0
drwxr-xr-x  2 root root    180 Oct 13 00:18 .
drwxr-xr-x 17 root root   4320 Oct 13 00:18 ..
lrwxrwxrwx  1 root root     16 Oct 13 00:18 control -> ../device-mapper
brw-rw----  1 root disk 253, 2 Oct 13 00:18 nvidia_dacaiefa
brw-rw----  1 root disk 253, 5 Oct 13 00:18 nvidia_dacaiefa1
brw-rw----  1 root disk 253, 1 Oct 13 00:18 pdc_bjahfddb
brw-rw----  1 root disk 253, 4 Oct 13 00:18 pdc_bjahfddb1
brw-rw----  1 root disk 253, 0 Oct 13 00:18 pdc_caafigga
brw-rw----  1 root disk 253, 3 Oct 13 00:18 pdc_caafigga1



mount /dev/mapper/nvidia_dacaiefa1 /a     #(3)
cd /a
ll


So,
  • First: get root access
  • Next : create a simple directory within the root (remember it's a livecd, it's in memory modification). I needed the shortest path possible to not loose time typing long path as my system was really not stable.
  • Next : find the name of the device representing the Fake RAID. For me, it was nvidia_*. nvidia_dacaiefa is the (virtual raid0) hard drive, and nvidia_dacaiefa1 it's first and only NTFS partition.
  • Next : Mount the ntfs partition in the filesystem.
Finally, I used command such as :
tar jxf /tmp/BunchOfFiles.tar.bz2 path1 path2
and
scp /tmp/BunchOfFiles.tar.bz2 toto@my.server.com:/home/toto/tmp


to get my data out of this sick computer !

Thursday, 9 October 2008

What I allways install/change on my ubuntu servers

I'm installing several ubuntu servers for various purpose (home server, Secondary DNS server, new primary DNS server, web servers, development servers).

For each one, I write the setup documentation (which are stored on a google apps premium account, which is by the way a must have for all the cooperative stuff).

Here is what I do for all servers, no matter the final use :

useful software




sudo -s # switch to root account as at setup I do a lot of root stuff
apt-get update # update the apt-get package list
apt-get upgrade # upgrade all package that are installed by default
apt-get install vim-full; # A lot of dependencies comes along, but vim is much better after (need some config modification)
apt-get install sysstat; # To monitor Hard drive activities
apt-get install whois; # contains the mkpasswd command
apt-get install slocate; # slocate, to find file
apt-get install nmap; # nmap, a port scanner which help to see if your firewall is properly set up
apt-get install debian-helper-scripts; # install the 'service' command (ex: sudo service mysqld restart instead of /etc/init.d/mysqld restart, I'm used to the service command)
apt-get install ntp ntp-doc; # to keep the clock up to date (all computer tends to loose time because of Interruption)
apt-get install lynx # a text browser that help sometime to do some stuff
apt-get install unzip # unzip
apt-get install screen # detach a shell, to logout, re-logon, retrieve your work
apt-get install ndisc6 tcptraceroute # network diagnostic tools (like tcptraceroute, tcptraceroute6)
exit


Colored prompt and aliases



vi .bashrc and uncomment

force_colored_prompt=yes


if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi


vi .bash_aliases

and put :

alias ps="ps eaxjf" #ps as a tree, all system process

# enable color support of ls and also add handy aliases
if [ "$TERM" != "dumb" ] && [ -x /usr/bin/dircolors ]; then
eval "`dircolors -b`"
alias ls='ls --color=auto'
alias dir='ls --color=auto --format=vertical'
alias vdir='ls --color=auto --format=long'

alias grep='grep --color=auto'
alias fgrep='fgrep --color=auto'
alias egrep='egrep --color=auto'
fi

alias ll='ls -la'
alias cd..="cd .." #type error I do often
alias uncd="cd $OLDPWD" #go to the previous directory (which can be different than cd .. ;)



Change the default home directory permission :



sudo vi /etc/adduser.conf

change DIR_MODE to :

DIR_MODE=0750

Change permission on the root directory



cd /
sudo chmod 700 root


CTRL+ALT+SUPPR disabling


sudo vi /etc/event.d/control-alt-delete

comment the line :
#exec /sbin/shutdown -r now "Control-Alt-Delete pressed"


Configuration of NTP


choose the closest NTP server from your server if you know one, or your country NTP server from pool.ntp.org

sudo vi /etc/ntp.conf

# You do need to talk to an NTP server or two (or three).
server mafreebox.free.fr # The triple play modem from my ISP (free) is also a NTP server
server ntp.ubuntu.com # default server
server fr.pool.ntp.org # NTP server for my country from the pool.ntp.org



Firewall configuration



IPv6 activation



sudo vi /etc/default/ufw

change to
IPV6=yes

Rules



Home server (ssh/http/samba)


sudo ufw logging off
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow 137/udp
sudo ufw allow 138/udp
sudo ufw allow 139/tcp
sudo ufw allow 445/tcp
sudo ufw disable;sudo ufw enable;



Production name server (DNS/HTTP/SSH for some hosts)



sudo ufw logging on

sudo ufw allow proto tcp from XX.XX.XX.XX to any port 22
sudo ufw allow proto tcp from XX.XX.XX.XX to any port 22
sudo ufw deny ssh # deny SSH for all host except the two above

sudo ufw allow domain # DNS port 53
sudo ufw allow http # http Port
sudo ufw disable;sudo ufw enable;


To allow an additional host to connect with ssh :

sudo ufw delete deny ssh                             #delete the rule : deny SSH for all
sudo ufw allow proto tcp from X.X.X.X to any port 22 #Add the IP for the additional host
sudo ufw deny ssh #deny ssh for all others (ie deny for all except the 2 existing hosts + the new one)
sudo ufw disable;sudo ufw enable; #reload the firewall



Firewall state example :

thomas@ns1:~$ sudo ufw status
Firewall loaded

To Action From
-- ------ ----
53:tcp ALLOW Anywhere
53:udp ALLOW Anywhere
80:tcp ALLOW Anywhere
22:tcp ALLOW X.X.X.X
22:tcp ALLOW Y.Y.Y.Y
22:tcp DENY Anywhere
22:udp DENY Anywhere
53:tcp ALLOW Anywhere (v6)
53:udp ALLOW Anywhere (v6)
80:tcp ALLOW Anywhere (v6)
22:tcp DENY Anywhere (v6)
22:udp DENY Anywhere (v6)



VIM configuration



(I found the mouse activation too much disturbing... and anyway... the mouse slow you down more than being drunk ;)

sudo vim /etc/vim/vimrc

" All system-wide defaults are set in $VIMRUNTIME/debian.vim (usually just
" /usr/share/vim/vimcurrent/debian.vim) and sourced by the call to :runtime
" you can find below. If you wish to change any of those settings, you should
" do it in this file (/etc/vim/vimrc), since debian.vim will be overwritten
" everytime an upgrade of the vim packages is performed. It is recommended to
" make changes after sourcing debian.vim since it alters the value of the
" 'compatible' option.

" This line should not be removed as it ensures that various options are
" properly set to work with the Vim-related packages available in Debian.
runtime! debian.vim

" Uncomment the next line to make Vim more Vi-compatible
" NOTE: debian.vim sets 'nocompatible'. Setting 'compatible' changes numerous
" options, so any other options should be set AFTER setting 'compatible'.
"set compatible

" Vim5 and later versions support syntax highlighting. Uncommenting the next
" line enables syntax highlighting by default.
syntax on

" If using a dark background within the editing area and syntax highlighting
" turn on this option as well
set background=dark

" Uncomment the following to have Vim jump to the last position when
" reopening a file
if has("autocmd")
au BufReadPost * if line("'\"") > 0 && line("'\"") <= line("$") \| exe "normal g'\"" | endif endif " Uncomment the following to have Vim load indentation rules according to the " detected filetype. Per default Debian Vim only load filetype specific " plugins. if has("autocmd") filetype indent on endif " The following are commented out as they cause vim to behave a lot " differently from regular Vi. They are highly recommended though. set showcmd " Show (partial) command in status line. set showmatch " Show matching brackets. set ignorecase " Do case insensitive matching set smartcase " Do smart case matching set incsearch " Incremental search set autowrite " Automatically save before commands like :next and :make set hidden " Hide buffers when they are abandoned "set mouse=a " Enable mouse usage (all modes) in terminals " Source a global configuration file if available " XXX Deprecated, please move your changes here in /etc/vim/vimrc if filereadable("/etc/vim/vimrc.local") source /etc/vim/vimrc.local endif


Log Rotate config



Change to 52 weeks of log retention as it's the new law in France.
And compress it.

sudo vim /etc/logrotate.conf

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 52

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
missingok
monthly
create 0664 root utmp
rotate 1
}

/var/log/btmp {
missingok
monthly
create 0664 root utmp
rotate 1
}

# system-specific logs may be configured here

Wednesday, 8 October 2008

Setting up UPS link with an ubuntu server


I've bought a new server (build from parts) to replace the old one that has capacitor leak issues and died †

And with it, I bought a new UPS (Uninterruptible Power Supply) to replace the old one that non longer work either...



I've googled internet to setup this UPS with my linux server (ubuntu server 8.0.4.1) and here is the result of my search and work :

When the UPS detect a power outage, it wait some times
(configured depending on the capacity of the UPS and the
power consumption of the server), and then send a mail
with the UPS state just before the shutdown,
and then shutdown the server.

While waiting some time, if the power is back,
the shutdown is cancelled.


The UPS state is useful as it shows you the health of your UPS when it goes on battery.

  1. If the charge is really low when the server is shutdown, you might want to reduce the time it wait on battery before shutting down the server or replace the battery (or the UPS).

  2. If the charge is really close to 100, you might want to increase the time on battery.



Script to send mail are located in my home account /home/thomas.
My UPS is a MGE Ellipse 750VA connected trough USB.
(a serial cable is also available)


NUT install & configure



NUT is the piece of software that will communicate with the UPS, and shut your server down properly.
I've read that MGE is involved in NUT development, which is worth to be mentioned as it's not so often a company do that.

NUT documentation is for nuts ;) (easy joke) Well not that easy to get into.
Instead I've read the following french tutorial written by Olivier Van Hoof and this one also helped : http://wiki.monserveurperso.com/wakka.php?wiki=NutInstall (still in French)

In all configuration files/script excerpt, I'll use some colours to show the link between configuration files. So if you want to change one value to match your config, change all string with the same colour.


Let's install NUT


a piece of cake :
sudo apt-get install nut


Edit configuration files



Note you can download a copy of my configuration files here : http://mansonthomas.com/blogger/download/nut.tar.bz2

Configuration files are in /etc/nut
No configuration files are created by default. If you want example, you can find some in /usr/share/doc/nut/examples/

First we need to tell the system to start nut's daemon when the system starts :

sudo vi /etc/default/nut

change the first two 'no' to 'yes' to do so.

# start upsd
START_UPSD=yes

# start upsmon
START_UPSMON=yes


Now lets declare our UPS :

sudo vi /etc/nut/ups.conf
[MGE-Ellipse750]
driver = usbhid-ups
port = auto
desc = "MGE UPS Systems UPS"
Within the bracket, you can set your UPS name (no space allowed).
The most important thing is to find your driver that handle your UPS.
You can find it here : http://eu1.networkupstools.org/compat/stable.html
(beware, this is for the last stable version of NUT which might not be the version installed on your server)

For an USB connection, the port is 'auto'.

If you use a serial cable instead, you should do this :
chmod 0600 /dev/ttyS0
chown nut:nut /dev/ttyS0


and put /dev/ttyS0 instead of auto.


sudo vi /etc/nut/upsd.conf
ACL all 0.0.0.0/0
ACL localhost 127.0.0.1/32

ACCEPT localhost
REJECT all


This file defines who is able to connect. In my case, only localhost.
I guess this files is involved when you have several computer connected to one big UPS.

sudo vi /etc/nut/upsd.users
[thomas]
password = test
allowfrom = localhost
upsmon master
This file defines a user 'thomas' with a password 'test' allowed to connect on localhost, and is allowed to control the UPS (upsmon master)

sudo vi /etc/nut/upsmon.conf
# define the ups to monitor and the permissions
MONITOR MGE-Ellipse750@localhost 1 thomas test master
# define the shutdown comand
SHUTDOWNCMD "/sbin/shutdown -h now"

# launch the upssched program to execute script with delay
NOTIFYCMD /sbin/upssched
NOTIFYFLAG ONBATT SYSLOG+WALL+EXEC
NOTIFYFLAG ONLINE SYSLOG+WALL+EXEC


This files basically tells NUT what to do on power outage.
The first instruction defines on which UPS is monitored, on which machine, the user/password used to connect, and the type : this is the master server.
All the line is defined from the previous configuration files we created, so if you change them, change this line too.

The SHUTDOWNCMD is what will be executed to shutdown the server.

NOTIFYCMD is called when the power status change (OB : on battery, OL : on line).
Here we call /sbin/upssched which will allow to say : Ok, if in X seconds power is not back, we do something
Events that trigger the NOTIFYCMD is the two last line (ONBATT and ONLINE)

sudo vi /etc/nut/upssched.conf
# the script to be executed
CMDSCRIPT /home/thomas/scripts/alertAndShutdown.php

# mandatory fields that must be set before AT commands
PIPEFN /var/run/nut/upssched.pipe
LOCKFN /var/run/nut/upssched.lock

# the timers, here 30 sec after the ONBATT (ups on battery) event
AT ONBATT * START-TIMER onbatt 30

# cancel the countdown is power is back
AT ONLINE * CANCEL-TIMER onbatt


In this file, we define, the script called upon the end of timer. (we'll see later this script)
Define 2 technical resources.
The two last line define the behaviour of NUT considering the events, which give in plain english :

On ONBATT event, start a time called onbatt for 30 secs, when those 30 secs are ellapsed the CMDSCRIPT will be called, unless a ONLINE event is recieved.


Now that we've defined our configuration files, we need to secure them :
sudo chown root:nut /etc/nut/*
sudo chmod 640 /etc/nut/*



At first I create a dedicated directory in /var/run named upsshed, but upon server restart, the directory is deleted... so instead I used the existing /var/run/nut directory.
If the directory is not there you get this error :

Oct  8 19:29:27 home upssched[5324]: Failed to connect to parent and failed to create parent: No such file or directory

Restart the NUT daemon



I recommend you to open to console, the first doing this :

sudo tail -f /var/log/daemon.log


that will display what is appended to the daemon.log file.
It's usefull to understand what's going wrong.

Then restart :

sudo service nut stop; sudo service nut start


which gave for me :

the stop :
Oct  8 21:13:48 home upsd[5171]: Signal 15: exiting
Oct 8 21:13:48 home upsmon[5174]: Signal 15: exiting
Oct 8 21:13:48 home upsmon[5173]: upsmon parent: read
Oct 8 21:13:48 home usbhid-ups[5169]: Signal 15: exiting


the start :
Oct  8 21:14:26 home usbhid-ups[5498]: Startup successful
Oct 8 21:14:26 home upsd[5499]: listening on 0.0.0.0 port 3493
Oct 8 21:14:26 home upsd[5499]: Connected to UPS [MGE-Ellipse750]: usbhid-ups-MGE-Ellipse750
Oct 8 21:14:26 home upsd[5500]: Startup successful
Oct 8 21:14:26 home upsmon[5502]: Startup successful
Oct 8 21:14:26 home upsd[5500]: Connection from 127.0.0.1
Oct 8 21:14:26 home upsd[5500]: Client thomas@127.0.0.1 logged into UPS [MGE-Ellipse750]



As I was doing many other thing while I was setting up my UPS, the following may not be exact :
But I had stop/start several time to see thing change upon my tries. (but as I say, maybe I simply just forgot to save the configuration files (many time))


Test the connection with the UPS



Now that NUT is started successfully, we can test that the UPS is here :


thomas@home:/etc/nut$ upsc MGE-Ellipse750@localhost
battery.charge: 100
battery.charge.low: 30
battery.runtime: 2818
battery.type: PbAc
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.version: 2.2.1-
driver.version.data: MGE HID 1.01
driver.version.internal: 0.32
input.transfer.high: 264
input.transfer.low: 184
outlet.0.desc: Main Outlet
outlet.0.id: 1
outlet.0.switchable: no
outlet.1.desc: PowerShare Outlet 1
outlet.1.id: 2
outlet.1.status: on
outlet.1.switchable: no
outlet.2.desc: PowerShare Outlet 2
output.frequency.nominal: 50
output.voltage: 230.0
output.voltage.nominal: 230
ups.beeper.status: enabled
ups.delay.shutdown: -1
ups.delay.start: -10
ups.load: 0
ups.mfr: MGE OPS SYSTEMS
ups.model: Ellipse 750
ups.power.nominal: 750
ups.productid: ffff
ups.serial: BDCJ2303A
ups.status: OL CHRG
ups.vendorid: 0463


if you want to test the SHUTDOWNCMD instruction, to trigger the event that run shutdown command :
(beware, this will shut down your server if the SHUTDOWNCMD is properly setted)
/sbin/upsmon -c fsd



This should stop the computer.


Now, about the /home/thomas/scripts/alertAndShutdown.php script.

You can find here the set of script used :
http://mansonthomas.com/blogger/download/UPS-scripts.tar.bz2

Download the file, have a look in it to see there nothing bad there (you should always check ;)
cd
tar jxf UPS-scripts.tar.bz2


This would create a scripts dir in your home directory.

First : I use php as my linux script engine because it's simple but yet powerful, and I know this language very well and it's already installed for other need on my linux box.
To get it :
sudo apt-get install php5 php5-cli


Before we proceed we need to add a group.
As I run several scripts with several user (root/thomas/nut etc...), in which there is some private data such as password (linux account/MySQL etc...) we need to do something so that all user that run these scripts can read these files, and only them.

To do so, we'll create a group, and add all those user to this group, and change the file permission so that the file belong to this group.
The script would be in a better place in /usr/local or something but I personally prefer to have them in my home directory on my small personal server. (well maybe I'll move them into /usr/local and create a link to it, it would be far more clean ;)

groupadd scriptExecutor
chmod 770 /home/thomas
sudo chown thomas:scriptExecutor /home/thomas
sudo chown -R thomas:scriptExecutor /home/thomas/scripts
sudo chmod 640 /home/thomas/scripts/config/config.php
sudo chmod 770 /home/thomas/scripts/alertAndShutdown.php
sudo chmod 640 /home/thomas/scripts/mail.php
sudo chmod 640 /home/thomas/scripts/lib/*

#thomas is a sudoers so you need to add admin otherwise you won't be able to sudo again.
sudo usermod -G scriptExecutor,admin thomas
sudo usermod -G scriptExecutor root
sudo usermod -G scriptExecutor nut


Note : if something is wrong in the script permission (from the / to the script, check permission and ownership on each directory) then you'll get this kind of warning :
Oct  8 01:12:48 home upssched[5355]: exec_cmd(/home/thomas/scripts/alertAndShutdown.php onbatt) returned 126



I use phpMailer to send mail :

http://phpmailer.codeworxtech.com/

my mail function is based on the gmail example within the phpMail archive.


What's worth to be notice in these scripts :

  1. the leading #!/usr/bin/php in alertAndShutdown.php along with chmod 770 allow you to execute the php interpreter and run this script at one time instead of doing #php alertAndShutdown.php
  2. Ok the core of the script : build an html message with the output of the command upsc
  3. Write to /var/log/ups.log the fact that a shutdown has been processed by NUT (you can find this information in /var/log/daemon.log, but it contains other logs soo...)
  4. And the last command which is utterly needed otherwise your server won't stop : "/sbin/upsmon -c fsd" which will tel NUT to run the shutdown cmd.



Test carefully your configuration

To test the script, you have no choice but unplug your UPS (between UPS and your house power outlet. (not between the server and the UPS as I did the first time (lack of sleep is terrible))
While you do so, use a console with the "tail -f /var/log/daemon.log" running in it.

Make the following test :

  1. Unplug the UPS, wait to see in logs that this as been seen by NUT, replug the UPS, and see that the timer is canceled, your server shouldn't stop.

  2. Unplug the UPS, wait more than 30 seconds (the time we set in upsshed.conf), see that you recieved a mail and that our computer shut down.

  3. Check that your UPS is fully charged, (battery.charge). Comment the last line of the alertAndShutdown.php script, unplug the UPS, and see how much time your server need to suck all the power of your UPS. Then take 80% of that time, et set it in upsshed.conf in seconds

  4. Uncomment the last line of alertAndShutdown.php, re run test 3.



For me it gave :

First test :

Broadcast Message from nut@my.server.com
(somewhere) at 22:57 ...

UPS MGE-Ellipse750@localhost on battery

Oct 8 22:57:37 home upsmon[6221]: UPS MGE-Ellipse750@localhost on battery
Oct 8 22:57:37 home upssched[6302]: Timer daemon started
Oct 8 22:57:38 home upssched[6302]: New timer: onbatt (30 seconds)

Broadcast Message from nut@my.server.com
(somewhere) at 22:57 ...

UPS MGE-Ellipse750@localhost on line power

Oct 8 22:57:57 home upsmon[6221]: UPS MGE-Ellipse750@localhost on line power
Oct 8 22:57:57 home upssched[6302]: Cancelling timer: onbatt




Second test :

Broadcast Message from nut@my.server.com
(somewhere) at 23:01 ...

UPS MGE-Ellipse750@localhost on battery

Oct 8 23:01:57 home upsmon[6221]: UPS MGE-Ellipse750@localhost on battery
Oct 8 23:01:57 home upssched[6315]: Timer daemon started
Oct 8 23:01:58 home upssched[6315]: New timer: onbatt (30 seconds)
Oct 8 23:02:28 home upssched[6315]: Event: onbatt
Oct 8 23:02:29 home upsd[6218]: Connection from 127.0.0.1
Oct 8 23:02:29 home upsd[6218]: Client on 127.0.0.1 logged out

Broadcast Message from nut@my.server.com
(somewhere) at 23:02 ...

Executing automatic power-fail shutdown



Broadcast Message from nut@my.server.com
(somewhere) at 23:02 ...

Auto logout and shutdown proceeding

Oct 8 23:02:31 home upsmon[6221]: Signal 10: User requested FSD
Oct 8 23:02:31 home upsd[6218]: Client thomas@127.0.0.1 set FSD on UPS [MGE-Ellipse750]
Oct 8 23:02:31 home upsmon[6221]: Executing automatic power-fail shutdown
Oct 8 23:02:31 home upsmon[6221]: Auto logout and shutdown proceeding

Broadcast message from root@my.server.com
(unknown) at 23:02 ...

The system is going down for halt NOW!
Oct 8 23:02:36 home init: tty4 main process (4928) killed by TERM signal
Oct 8 23:02:36 home init: tty5 main process (4929) killed by TERM signal
Oct 8 23:02:36 home init: tty2 main process (4931) killed by TERM signal
Oct 8 23:02:36 home init: tty3 main process (4934) killed by TERM signal
Oct 8 23:02:36 home init: tty6 main process (4935) killed by TERM signal
Oct 8 23:02:36 home init: tty1 main process (5240) killed by TERM signal


and the mail :




Third test :

My UPS with my server last 14 minutes with an initial battery state at 92% of full charge before reaching the 30% low limit (on which NUT execute the SHUTDOWNCMD anyway).
(so about 15 minutes with full charge)

My server parts are designed to be low power consumer, so the result is not that nice.
Details about server parts can be found here if you're interested : http://spreadsheets.google.com/pub?key=pw1pftHbgb8QakIjz1z_7Ew

I'll do some tweak, like processor speed change (if possible) and stop the harddrive if not used and mount /var/log on an USB key (so that the hard drives can stop)

Final thought


  1. recheck your UPS from time to time, like every 6 months. UPS lifetime can be bad.

  2. Of course, all above information is to be used at your own risk and may certainly be improved in many way. If you do so, share ;)