Saturday, 17 October 2009

Tune the way linux use the swap with the swappiness parameter


On linux, you can tune the way the system use the swap via the 'swappiness' parameter.


Check how the swap is used on your machine:

You can print how the system is using the swap with the vmstat command :

thomas@home:~$ vmstat 2
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0  25192  81956  93052 2383988    0    0     8    15    1    1  1  1 98  0
 6  0  25192  71076  93100 2384224    0    0    10   272 1148 1265  8 10 80  2
 0  0  25192  97492  93016 2368148    0    0   132  1186 1380 1845 26 35 39  0
 0  0  25192  96648  93032 2368740    0    0    70   188 1121 1066  2  2 96  1
 0  0  25192  96400  93044 2368976    0    0     0   188 1117 1082  1  1 98  0

the '2' parameter stands for the refresh time in seconds.

si stands for Swap IN (byte written to the swap)
so stands for Swap OUT (byte removed from the swap)

On a server which has enough RAM, I observed that there was regularly swap IN activity.
So that the server use more the RAM and less the swap, we need to tune the swappiness parameter.


By default, on ubuntu, it is set to '60'.
Basically, 0 means do not use the swapp, 100 means use it a lot.

You can check your current system value with this command :

cat /proc/sys/vm/swappiness

You can set it on the fly with the following command :

echo 40 > /proc/sys/vm/swappiness

run vmstat again and see if it changes anything on the swap activity.
On my server, with this parameter, the swap IN activity stopped.

But with this way of setting the swappiness, the parameter value you set won't last after a reboot, it will be reset to default value.

To change definitively the swappiness value, you need to edit the /etc/sysctl.conf
and add the following line at the end of the file :

#swapping tendancy 0 :no swap, 100 : full swapping
vm.swappiness = 40

save your change and then run sysctl like this :

sudo sysctl -p

it should print
vm.swappiness = 40

reboot and recheck the
cat /proc/sys/vm/swappiness

the parameter should have keep the new value.

Recheck vmstat output to see how the situation has evolved.

picture from http://www.icon-king.com/

Wednesday, 2 September 2009

Reset Bacula database and files


Today I need to purge my bacula for several reasons.

Here is how I did.

First backup the database and files used in your previous setup, one never knows you might actually need it after the purge...

/usr/bin/mysqldump -aecqQ bacula > bacula_before_purge.sql


and copy the volumes and bsr files.


Stop bacula :

service bacula-director stop
service bacula-fd       stop
service bacula-sd       stop


then :

mysql -u root -p
drop database bacula;
create database bacula;
quit


next we need to recreate bacula tables. There's a bacula script to do that, but the file needs to be changed to set the correct database name.

cp /usr/share/bacula-director/make_mysql_tables ~/
vi ~/make_mysql_tables

change XXX_DBNAME_XXX by your bacula database (let's say 'bacula')
save and exit.

then run as root the script
~/make_mysql_tables


it should have recreated the tables :
mysql -u root -p
use bacula;
show tables;
+------------------+
| Tables_in_bacula |
+------------------+
| BaseFiles        |
| CDImages         |
| Client           |
| Counters         |
| Device           |
| File             |
| FileSet          |
| Filename         |
| Job              |
| JobMedia         |
| Location         |
| LocationLog      |
| Log              |
| Media            |
| MediaType        |
| Path             |
| Pool             |
| Status           |
| Storage          |
| UnsavedFiles     |
| Version          |
+------------------+
21 rows in set (0.00 sec)
quit


next you need to delete the existing volume and bsr files.
In my case I just change the location of the bacup (bacula-sd.conf 'Archive Device')

restart your bacula services and run a job to check everything is OK.
service bacula-director start
service bacula-fd       start
service bacula-sd       start 


Image from http://www.everaldo.com/

Wednesday, 19 August 2009

locale configuration issue on Ubuntu 9.04


I've rent a new server by dedibox.fr and the ubuntu server version is tunned by the dedibox team...



On the Ubuntu Server 9.04 64bit/english version, there's a locale configuration issue.
whenever you do a aptitude update or run the 'locale' command you get a warning like this :

thomas@sd1:~$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.ISO-8859-15
LANGUAGE=en_US:en:en_GB:en
LC_CTYPE="en_US.ISO-8859-15"
LC_NUMERIC="en_US.ISO-8859-15"
LC_TIME="en_US.ISO-8859-15"
LC_COLLATE="en_US.ISO-8859-15"
LC_MONETARY="en_US.ISO-8859-15"
LC_MESSAGES="en_US.ISO-8859-15"
LC_PAPER="en_US.ISO-8859-15"
LC_NAME="en_US.ISO-8859-15"
LC_ADDRESS="en_US.ISO-8859-15"
LC_TELEPHONE="en_US.ISO-8859-15"
LC_MEASUREMENT="en_US.ISO-8859-15"
LC_IDENTIFICATION="en_US.ISO-8859-15"
LC_ALL=


or

thomas@sd1:~$ sudo  dpkg-reconfigure locales
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = "en_US:en:en_GB:en",
LC_ALL = (unset),
LANG = "en_US.ISO-8859-15"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Generating locales...
en_AU.UTF-8... up-to-date
en_BW.UTF-8... up-to-date
en_CA.UTF-8... up-to-date
en_DK.UTF-8... up-to-date
en_GB.UTF-8... up-to-date
en_HK.UTF-8... up-to-date
en_IE.UTF-8... up-to-date
en_IN.UTF-8... up-to-date
en_NG.UTF-8... up-to-date
en_NZ.UTF-8... up-to-date
en_PH.UTF-8... up-to-date
en_SG.UTF-8... up-to-date
en_US.ISO-8859-1... up-to-date
en_US.UTF-8... up-to-date
en_ZA.UTF-8... up-to-date
en_ZW.UTF-8... up-to-date
Generation complete.



I've asked the support for help, and I get a refusal arguing that software is not part of the support... but as you can install only a few Operating System provided by dedibox, which operating system has been tunned/preconfigured for their servers, I feel that they are responsible for it, if it doesn't works out of the box. (of course when you're installing new software or start editing the configuration, it's your responsibility).

Anyway, with dedibox, the support is so lame, takes so much time to answer your call (10 days to get a KVM over IP on a production system which was blocked at boot time), to finally not solve your issue

One can say that there is NO support by dedibox at all... One would be absolutely right!

So, build a redundant architecture if you decide to play with dedibox servers.

Anyway....


What is wrong here is that the current locale configured on the system (en_US.ISO-8859-15) does not exists on the system (see the one generated, you have en_US.ISO-8859-1 but not en_US.ISO-8859-15).

So to change the current locale of the system do this :

vi /etc/default/locale

and change it to this (en_US.UTF-8 is listed while regenerating the locales, so it should be good)

to :

LANGUAGE="en_US:en:en_GB:en"
LANG="en_US.UTF-8"

and then

vi /etc/environment

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
LANGUAGE="en_US:en:en_GB:en"
LANG="en_US.UTF-8"

then run this command :

thomas@sd1:~$ sudo  dpkg-reconfigure locales
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = "en_US:en:en_GB:en",
LC_ALL = (unset),
LANG = "en_US.ISO-8859-15"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Generating locales...
en_AU.UTF-8... up-to-date
en_BW.UTF-8... up-to-date
en_CA.UTF-8... up-to-date
en_DK.UTF-8... up-to-date
en_GB.UTF-8... up-to-date
en_HK.UTF-8... up-to-date
en_IE.UTF-8... up-to-date
en_IN.UTF-8... up-to-date
en_NG.UTF-8... up-to-date
en_NZ.UTF-8... up-to-date
en_PH.UTF-8... up-to-date
en_SG.UTF-8... up-to-date
en_US.ISO-8859-1... up-to-date
en_US.UTF-8... up-to-date
en_ZA.UTF-8... up-to-date
en_ZW.UTF-8... up-to-date
Generation complete.

You still have the warning which is normal since at the execution, the system is still ill-configured.
In order to complete the reconfiguration, you must to reboot !

after reboot, type

thomas@sd1:~$ locale
LANG=en_US.UTF-8
LANGUAGE=en_US:en:en_GB:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

the current locale is set to en_US.UTF-8 and there is no more warning.

Note : to add a particular charset you can use the following command :

locale-gen en_US.UTF-8 en_GB.UTF-8


You might also find localepurge usefull to cleanup unused locale :
aptitude install localepurge

Monday, 10 August 2009

Add 2 disk for RAID 1 setup


On the dev server I spoke previously, we add 2 hard drives of 500GB in order to have 500GB in RAID1 as a network storage.

Here is the setup procedure:

Check the disk

sudo fdisk -l

thomas@dev:~$ sudo fdisk -l

Disk /dev/sda: 82.3 GB, 82348277760 bytes
255 heads, 63 sectors/track, 10011 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00059bfb

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          36      289138+  fd  Linux raid autodetect
/dev/sda2              37         218     1461915   fd  Linux raid autodetect
/dev/sda3             219         826     4883760   fd  Linux raid autodetect
/dev/sda4             827       10011    73778512+  fd  Linux raid autodetect

Disk /dev/sdb: 82.3 GB, 82348277760 bytes
255 heads, 63 sectors/track, 10011 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000d96f2

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          36      289138+  fd  Linux raid autodetect
/dev/sdb2              37         218     1461915   fd  Linux raid autodetect
/dev/sdb3             219         826     4883760   fd  Linux raid autodetect
/dev/sdb4             827       10011    73778512+  fd  Linux raid autodetect

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xffffffff

Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       60801   488384001   83  Linux

Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xffffffff

Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1       60801   488384001   83  Linux

Disk /dev/sde: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x44fdfe06


will list all drives and partitions.

Here the drives are /dev/sdc and /dev/sdd

remove existing partition

thomas@dev:~$ sudo fdisk /dev/sdd

The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): p

Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xffffffff

Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1       60801   488384001   83  Linux

Command (m for help): d
Selected partition 1

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.


and do the same for the other disk.

Create RAID Partition

Create partition for RAID and change it for Linux Raid

thomas@dev:~$ sudo fdisk /dev/sdc

The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n
Command action
e   extended
p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-60801, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-60801, default 60801):
Using default value 60801

Command (m for help): p

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xffffffff

Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       60801   488384001   83  Linux

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.


Choose a number for the md device

thomas@dev:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb4[1] sda4[0]
73778432 blocks [2/2] [UU]

md2 : active raid1 sdb3[1] sda3[0]
4883648 blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
1461824 blocks [2/2] [UU]

md0 : active raid1 sdb1[1] sda1[0]
289024 blocks [2/2] [UU]

unused devices: none


here the number 4 is free... so it will be /dev/md4




Create the array
(ignore the warning, I've created a filesystem before the creation of the md device, which is wrong)
thomas@dev:~$ mdadm --create /dev/md4 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1
mdadm: /dev/sdc1 appears to contain an ext2fs file system
size=488384000K  mtime=Thu Jan  1 01:00:00 1970
mdadm: /dev/sdd1 appears to contain an ext2fs file system
size=488384000K  mtime=Thu Jan  1 01:00:00 1970
Continue creating array? y
mdadm: array /dev/md4 started.


thomas@dev:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md4 : active raid1 sdd1[1] sdc1[0]
488383936 blocks [2/2] [UU]
[>....................]  resync =  0.1% (821888/488383936) finish=98.8min speed=82188K/sec

md3 : active raid1 sda4[0] sdb4[1]
73778432 blocks [2/2] [UU]

md2 : active raid1 sda3[0] sdb3[1]
4883648 blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
1461824 blocks [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]
289024 blocks [2/2] [UU]

unused devices: none


with the second command you can see the progress of the build.

Create the filesystem

sudo mkfs.ext4 /dev/md4

mke2fs 1.41.4 (27-Jan-2009)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
61054976 inodes, 244189984 blocks
12209499 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
7453 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
 102400000, 214990848

done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 22 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.



Mount & Check

Now we can mount and check that we can use the file (even if the sync is not done yet... although, I won't copy several gigs on it, but a simple touch..

sudo mkdir /mnt/md4-sdc-sdd-500GB-RAID1
sudo mount /dev/md4 /mnt/md4-sdc-sdd-500GB-RAID1
cd /mnt/md4-sdc-sdd-500GB-RAID1
sudo touch test
sudo rm test


It seems allright...

updating /etc/mdadm/mdadm.conf


In order to auto-reassemble the new raid device on reboot, you need to persist it's configuration in /etc/mdadm/mdadm.conf

the following command make a backup copy and append in mdadm.conf the configuration of the new array :
sudo cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf.origin
sudo mdadm --misc --detail --brief /dev/md4 | sudo tee -a /etc/mdadm/mdadm.conf


here is the result :
thomas@dev:/mnt/md4-sdc-sdd-500GB-RAID1$ tail /etc/mdadm/mdadm.conf

# definitions of existing MD arrays
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=8f382586:5dd86ec3:6e6f1681:980b2774
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=6c35c2c6:96c41f8a:596e7711:e9498056
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=e35a7ea9:d90afbda:526e888e:d5c311af
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=6c7f5b4f:27e96317:a44ec9ef:2b057faa

# This file was auto-generated on Thu, 04 Sep 2008 21:25:35 +0000
# by mkconf $Id$
ARRAY /dev/md4 level=raid1 num-devices=2 metadata=00.90 UUID=25b3acb8:99375aa9:2eeb9ed0:acf0d8fd




Updating the fstab

Now, in order to auto remount the filesystem on /dev/md4, you need to update the fstab.

The UUID you find in mdadm.conf is not the one to report in /etc/fstab (I don't know why.... if someone knows... let me know)
The UUID you need for the /etc/fstab can be found here :

thomas@dev:/mnt/md4-sdc-sdd-500GB-RAID1$ ls -l /dev/disk/by-uuid/ | grep md
lrwxrwxrwx 1 root root  9 2009-08-10 23:42 242d3f42-2cdb-4626-989b-70b4e25ed5d5 -> ../../md2
lrwxrwxrwx 1 root root  9 2009-08-10 23:42 468fca8d-c048-41ec-a1d8-172afd8d081c -> ../../md0
lrwxrwxrwx 1 root root  9 2009-08-10 23:42 a670d86f-6c57-48b7-ba92-13ecf14cc8c2 -> ../../md1
lrwxrwxrwx 1 root root  9 2009-08-10 23:42 bf44c30e-7c04-41ca-b7c8-610d279a4135 -> ../../md3
lrwxrwxrwx 1 root root  9 2009-08-10 23:55 cc55dd26-a913-43c1-a67a-479c4d6eb29c -> ../../md4


sudo vi /etc/fstab


append the following line:

# /dev/md4
UUID=cc55dd26-a913-43c1-a67a-479c4d6eb29c /mnt/md4-sdc-sdd-500GB-RAID1 ext3     defaults,relatime        1       1


be sure to replace the UUID by your UUID (ls -l /dev/disk/by-uuid/ | grep md)

Now we will check that it's correct :
unmount /dev/md4 first :

cd /mnt
sudo umount /dev/md4

then try to remount all partition defined in /etc/fstab :

thomas@dev:/mnt$ sudo mount -a


check if it's mounted :

thomas@dev:/mnt$ sudo mount -l
/dev/md3 on / type ext3 (rw,relatime,errors=remount-ro) [/]
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
/proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,nosuid,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
lrm on /lib/modules/2.6.27-14-server/volatile type tmpfs (rw,mode=755)
/dev/md0 on /boot type ext2 (rw,relatime) [/boot]
/dev/md2 on /var type ext3 (rw,relatime) [/var]
securityfs on /sys/kernel/security type securityfs (rw)
/dev/md4 on /mnt/md4-sdc-sdd-500GB-RAID1 type ext3 (rw,relatime) []

thomas@dev:/mnt$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md3               70G   61G  6.3G  91% /
tmpfs                 251M     0  251M   0% /lib/init/rw
varrun                251M  292K  251M   1% /var/run
varlock               251M     0  251M   0% /var/lock
udev                  251M  2.7M  249M   2% /dev
tmpfs                 251M     0  251M   0% /dev/shm
lrm                   251M  2.2M  249M   1% /lib/modules/2.6.27-14-server/volatile
/dev/md0              265M   54M  197M  22% /boot
/dev/md2              4.7G  1.1G  3.4G  23% /var
/dev/md4              459G  199M  435G   1% /mnt/md4-sdc-sdd-500GB-RAID1


Now the raid is configured ;)


Ref. used for this blog post :
http://www.linuxpedia.fr/doku.php/expert/mdadm by
http://www.linuxpedia.fr/doku.php/expert/systeme_conversion_raid1

Raid on linux - re-add a drive



Today, one of the developer of 123Monsite.com mess up with one server, and this lead to this :


They configure fake hardware on the mother board while it's a software raid... noticing it didn't work, they remove a drive to boot on a degraded raid... and try to find what's wrong... that's when I interrupt the massacre

this lead to this :
the partitions of the removed drive was marked as deleted (even if it has been replugged and fake-hardware disabled) :

thomas@dev:~$ sudo mdadm --query --detail /dev/md3
/dev/md3:
Version : 00.90
Creation Time : Thu Sep  4 23:15:23 2008
Raid Level : raid1
Array Size : 73778432 (70.36 GiB 75.55 GB)
Used Dev Size : 73778432 (70.36 GiB 75.55 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 3
Persistence : Superblock is persistent

Update Time : Mon Aug 10 22:45:33 2009
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

UUID : 6c7f5b4f:27e96317:a44ec9ef:2b057faa
Events : 0.296

Number   Major   Minor   RaidDevice State
0       8        4        0      active sync   /dev/sda4
1       0        0        1      removed


In order to re-activate the missing drive on each mdX I run the following command :

sudo mdadm --manage -a /dev/md3 /dev/sdb4
mdadm: re-added /dev/sdb4


which re-enable the partition of the sdb disk and when needed rebuild of the faulty partition :

thomas@dev:~$ sudo mdadm --query --detail /dev/md3
/dev/md3:
Version : 00.90
Creation Time : Thu Sep  4 23:15:23 2008
Raid Level : raid1
Array Size : 73778432 (70.36 GiB 75.55 GB)
Used Dev Size : 73778432 (70.36 GiB 75.55 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 3
Persistence : Superblock is persistent

Update Time : Mon Aug 10 22:56:35 2009
State : clean, degraded
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1

UUID : 6c7f5b4f:27e96317:a44ec9ef:2b057faa
Events : 0.300

Number   Major   Minor   RaidDevice State
0       8        4        0      active sync   /dev/sda4
2       8       20        1      spare rebuilding   /dev/sdb4

thomas@dev:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb4[2] sda4[0]
73778432 blocks [2/1] [U_]
[===>.................]  recovery = 16.8% (12412928/73778432) finish=18.6min speed=54688K/sec

md2 : active raid1 sdb3[1] sda3[0]
4883648 blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
1461824 blocks [2/2] [UU]

md0 : active raid1 sdb1[1] sda1[0]
289024 blocks [2/2] [UU]

unused devices: 

Monday, 8 June 2009

Week Numbers in google calendar


I've found this website :

http://sites.google.com/site/gcalweeknumbers/googlecalendarweeknumbers

that provide a calendar that displays week numbers in you calendar which is very usefull as at work (at least in France) where we refer to week number for planning.

MySQL backup

In this post, I'll setup an automatic backup script that create an sql compressed with bz2 file per database.


sudo -s


Autologin with MySQL :




sudo -s
vi /root/.my.cnf
[client]
user=root
password=TheRootPassword
protocol=tcp

chmod 400 /root/.my.cnf


Test :
[root@dell1 ~]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 870724 to server version: 4.1.22

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> quit
Bye
[root@dell1 ~]#


Script



vi /root/scripts/cron/backupDatabaseJob.sh

#!/bin/bash
TIME=`date`
echo "Starting MySQL Backup at $TIME"

BACKUPLOCATION=/home/backup/databases
CURRENTDATE=`date +%Y%m%d`
CURRENTLOCATION=$BACKUPLOCATION/MySQL_$CURRENTDATE

if [ ! -d $BACKUPLOCATION ]
then
echo "create directory for database saves $BACKUPLOCATION"
mkdir $BACKUPLOCATION
fi

echo "Databases saves at $CURRENTLOCATION";
mkdir -p $CURRENTLOCATION


mysql --defaults-extra-file=/root/.my.cnf -B -N -e "show databases" | while read db
do
echo "Backing up $db into $CURRENTLOCATION/${CURRENTDATE}_$db.sql"
/usr/bin/mysqldump --defaults-extra-file=/root/.my.cnf -aecqQ $db > $CURRENTLOCATION/${CURRENTDATE}_$db.sql
done

ls $CURRENTLOCATION | while read dbBackup
do
echo "compressing $CURRENTLOCATION/$dbBackup"
bzip2 -9 $CURRENTLOCATION/$dbBackup
done

echo "setting rights on files"
chmod 640 $CURRENTLOCATION/*
chmod 750 $CURRENTLOCATION

echo "changing ownership to backup:backup on files"
chown -R backup:bacula $CURRENTLOCATION
cd $BACKUPLOCATION
rm -f $BACKUPLOCATION/lastbackup
ln -s $CURRENTLOCATION $BACKUPLOCATION/lastbackup
cd $OLDPWD

TIME=`date`
echo "End of MySQL Backup at $TIME"


Crontab job



crontab -e
20      1       *       *       *       /root/scripts/cron/backupDatabaseJob.sh


And you'll get something like this :

[root@dell1 databases]# ll
total 1820
drwxr-xr-x  107 root   root   57344 Jun  7 01:20 .
drwxr-xr-x    5 root   root    4096 Jun  7 01:28 ..
-rw-r--r--    1 root   root       0 Apr  4 14:08 .bacula_exclude
lrwxrwxrwx    1 root   root      37 Apr  4 01:27 lastbackup -> /home/backup/databases/MySQL_20090607
drwxr-x---    2 backup bacula 12288 Feb 24 21:06 MySQL_20090221
drwxr-x---    2 backup bacula  4096 Feb 24 21:37 MySQL_20090224
drwxr-x---    2 backup bacula 12288 Feb 25 01:26 MySQL_20090225
...

[root@dell1 databases]# ll MySQL_20090607/
total 66424
drwxr-x---    2 backup bacula    12288 Jun  7 01:28 .
drwxr-xr-x  107 root   root      57344 Jun  7 01:20 ..
-rw-r-----    1 backup bacula      415 Jun  7 01:20 20090607_db1.sql.bz2
-rw-r-----    1 backup bacula      417 Jun  7 01:20 20090607_db2.sql.bz2
-rw-r-----    1 backup bacula    97078 Jun  7 01:20 20090607_db3.sql.bz2

...

Sunday, 7 June 2009

setup tomcat6 (with native library) with apache2 integration



In this post, I'll setup a apache2 with php5 and tomcat6 integrated with apache2.

Since Tomcat 5, you don't need a front http server anymore, but since I want to have php5 and tomcat6 on port 80, I need the tomcat6-apache2 integration with modjk


since a lot of operation need the root privilege, switch to root :

sudo -s

Apache2 & PHP5 installation



The following command install php5, php5 command line interface (CLI to use php as shell script) and some common php extension.

aptitude install php5 php5-cli apache2 php5-xmlrpc php5-mysql php5-gd php5-imagick php-pear php5-mcrypt
a2enmod headers #Activation of Mod headers
a2enmod deflate #Activation of Mod deflate


Setup the timezone :

vi /etc/php5/apache2/php.ini
[Date]
; Defines the default timezone used by the date functions
date.timezone = Europe/Paris


Java installation

aptitude install sun-java6-jdk
vi /etc/profile

Append the following line :

export JAVA_HOME=/usr/lib/jvm/java-6-sun

reload your .profile :

(command to type is "dot space dot profile")
. .profile

check :

echo $JAVA_HOME
/usr/lib/jvm/java-6-sun
root@rdp:/home/special/tomcat# java -version
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed mode)


Tomcat6 installation



In this installation, I install the tomcat binaries in /usr/local and separate the CATALINA_HOME and the CATALINA_BASE

CATALINA_HOME is in /usr/local/tomcat (the engine)
CATALINA_BASE is in /home/special/tomcat (the configuration files to configure tomcat for your app)

Like this I can have several CATALINA_BASE (with differnent tomcat config) with one HOME (which is a link to the last current version, upgrading to the next version will mean change a link, switch back to the previous version : change the link to its original point).

mkdir ~/download
cd ~/download
wget http://apache.cict.fr/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.zip
unzip apache-tomcat-6.0.20.zip
mv apache-tomcat-6.0.20 /usr/local/
ln -s /usr/local/apache-tomcat-6.0.20 /usr/local/tomcat
chmod 755 /usr/local/tomcat/bin/*.sh
mkdir /home/special/tomcat
cp -r /usr/local/tomcat/* /home/special/tomcat
cd /home/special/tomcat
rm -r bin/ lib/ LICENSE NOTICE RELEASE-NOTES RUNNING.txt webapps/*


Disable the SSL Engine

as I won't use it :

switch the value from 'on' to 'off' on this line :
<listener classname="org.apache.catalina.core.AprLifecycleListener" sslengine="off"/>


Edit tomcat init script

with this script you'll be able to start/stop/restart tomcat with the service command.

It's in this script I'll specify where the CATALINA_HOME and CATALINA_BASE are.
If you want another instance of tomcat with another CATALINA_BASE location, write a second script with a different CATALINA_BASE.

In this script I set the jvm in server mode "-server" and specify some memory settings.

vi /etc/init.d/tomcat
# Tomcat auto-start
#
# description: Auto-starts tomcat
# processname: tomcat
# pidfile: /var/run/tomcat.pid

JAVA_HOME=/usr/lib/jvm/java-6-sun; export JAVA_HOME
JAVA_OPTS="-server -Xms100m -Xmx512m"; export JAVA_OPTS
CATALINA_HOME=/usr/local/tomcat; export CATALINA_HOME
CATALINA_BASE=/home/special/tomcat; export CATALINA_BASE


case $1 in
start)
sh $CATALINA_HOME/bin/startup.sh
;;
stop)
sh $CATALINA_HOME/bin/shutdown.sh
;;
restart)
sh $CATALINA_HOME/bin/shutdown.sh
sh $CATALINA_HOME/bin/startup.sh
;;
*)
echo $"Usage: $0 {start|stop|restart}"
exit 1
esac
exit 0


chmod 755 /etc/init.d/tomcat

#auto start & stop with the machine :
ln -s /etc/init.d/tomcat /etc/rc1.d/K99tomcat
ln -s /etc/init.d/tomcat /etc/rc2.d/S99tomcat


MOD_JK installation


MOD_JK is here to link apache2 and tomcat.

aptitude install libapache2-mod-jk

Edit the jk.conf file, which is here to setup general config option of mod_jk

vi /etc/apache2/mods-available/jk.conf
JkWorkersFile      /etc/apache2/workers.properties
JkLogFile "|/usr/bin/rotatelogs /var/log/apache2/mod_jk.log 86400"
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkRequestLogFormat "%w %V %T"
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

Activation of modjk :
cd /etc/apache2/mods-enabled
sudo ln -s ../mods-available/jk.conf

In workers.properties we specify the links between apache2 and J2EE servers (tomcat/jetty etc..)
You can have multiple J2EE servers declared in this file.

here I setup one J2EE server which is on the same server (yes, the tomcat could have been on another server)

vi /etc/apache2/workers.properties

worker.list=tomcat6

worker.tomcat6.type=ajp13
worker.tomcat6.host=localhost
worker.tomcat6.port=8009


Apache2 virtualhost configuration


Here we add a virtualhost for the tomcat application.
Based on the servername sent by the browser, apache will select the virtualhost to send the http request.
(At the end of this post I'll explain how to use this)

let's say the application is named "crf-rdp"

mkdir -p /home/special/tomcat/www/crf-rdp
Add a virtual host:
vi /etc/apache2/sites-available/crf-rdp
<VirtualHost *>
ServerAdmin webmaster@localhost
DocumentRoot /home/special/tomcat/www/crf-rdp

ServerName crf-rdp.server.com

<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>

<Directory /home/special/tomcat/www/crf-rdp>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>


ErrorLog "|/usr/local/apache/bin/rotatelogs /var/log/apache2/error_crf-rdp.log 86400"
CustomLog "|/usr/local/apache/bin/rotatelogs /var/log/apache2/access_crf-rdp.log 86400" combined

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
ServerSignature On

RewriteEngine on
RewriteLog /var/log/apache2/mod_rewrite_crf-rdp.log
RewriteLogLevel 1
RewriteCond %{REQUEST_URI} !^/crf-rdp
RewriteRule ^(.*) /crf-rdp$1 [PT,L]

JkMount /crf-rdp/* tomcat6


</VirtualHost>

With the rewrite rule and the JkMount all http request sent to crf-rdp.domain.com will be forwarded to the tomcat webapp.


Set the new virtual host as available :
ln -s /etc/apache2/sites-available/crf-rdp 001-crf-rdp


Installation of Apache Tomcat Native Library


This "library allows optimal performance in production environment".

Note : libapr1 is installed with apache2

aptitude  install make gcc gcc-4.2 gcc-4.2-locales gcc-4.2-multilib libapr1-dev
cd /usr/local/apache-tomcat-6.0.20/bin/
tar zxf tomcat-native.tar.gz
cd /usr/local/apache-tomcat-6.0.20/bin/tomcat-native-1.1.16-src/jni/native
export JAVA_HOME=/usr/lib/jvm/java-6-sun
./configure --with-apr=/usr/bin/apr-config; make; make install
cd /usr/lib
ln -s /usr/local/apr/lib/libtcnative-1.so.0.1.16 libtcnative-1.so


when your tomcat will start, you can check the catalina.out log file and this the following :

thomas@home:/home/special/tomcat/logs$ cat catalina.out
Oct 18, 2008 9:13:02 PM org.apache.catalina.core.AprLifecycleListener init
INFO: Loaded APR based Apache Tomcat Native library 1.1.16.
Oct 18, 2008 9:13:02 PM org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
Oct 18, 2008 9:13:02 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: Failed to initialize the SSLEngine.
Oct 18, 2008 9:13:02 PM org.apache.coyote.http11.Http11AprProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8080
Oct 18, 2008 9:13:02 PM org.apache.coyote.ajp.AjpAprProtocol init
INFO: Initializing Coyote AJP/1.3 on ajp-8009
Oct 18, 2008 9:13:02 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 708 ms
Oct 18, 2008 9:13:02 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Oct 18, 2008 9:13:02 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Oct 18, 2008 9:13:02 PM org.apache.coyote.http11.Http11AprProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
Oct 18, 2008 9:13:02 PM org.apache.coyote.ajp.AjpAprProtocol start
INFO: Starting Coyote AJP/1.3 on ajp-8009
Oct 18, 2008 9:13:02 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 109 ms



Now you should have a working installation with tomcat6 linked with apache2.

You start/stop/restart tomcat with
service tomcat start
service tomcat stop
service tomcat restart



Now here is some useful tips you may find valuable :

Define a dataSource in tomcat


It's useful and more secure to define a dataSource (database connection) inside the J2EE server instead of inside the application itself.

Because developpers should know production credentials and with credentials inside your application properties files, it goes into your cvs/svn/bazaar/git source control.
While if it's a dataSource that you use as a JDNI ressource, the person who operate tomcat (the server admin) is the only one knows the dataSource credentials.

You first need to get the jdbc driver of the datasource.
The example is here for mysql (but works too with other db servers)

cd ~/download
wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.7.tar.gz/from/http://mir2.ovh.net/ftp.mysql.com/
tar zxf mysql-connector-java-5.1.7.tar.gz
cp mysql-connector-java-5.1.7/mysql-connector-java-5.1.7-bin.jar /usr/local/tomcat/lib/

cd /home/special/tomcat/conf
mkdir -p Catalina/localhost

Example for the CRF-RDP application :



(where crf-rdp is the context-path of the webapp, see first chapter here http://tomcat.apache.org/tomcat-6.0-doc/config/context.html)

vi /home/special/tomcat/conf/Catalina/localhost/crf-rdp.xml
<Context path="crf-rdp" docBase="/home/special/tomcat/crf/crf-rdp.war" debug="0">

<Resource name="jdbc/crfirpDS" auth="Container" type="javax.sql.DataSource"
username="crfirpUser"
password="***********"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/crfirpDb?autoReconnect=true&useUnicode=true&characterEncoding=UTF8"
validationQuery="Select 1"
/>

</Context>

here we setup a datasource named "crfrdpDS" for the webapplication "crf-rdp" which has its war file here "/home/special/tomcat/crf/crf-rdp.war".
It connects to the localhost mysql database, with 'crfrdpUser' database user, on crfrdpDb database schema.

In the application, the datasource should be named 'java:comp/env/jdbc/crfrdpDS' (without quote) (and not 'jdbc/crfrdpDS' as you may think it would be... to easy ;))

Notice : about the password, don't use < or & since it's XML special characters (it should be encoded)

A virtualhost for phpMyAdmin



PhpMyAdmin is a great tool to operate your mysql databases (even more usefull than mysql-query-browser in many situation).
But your database is your most precious thing...
So expose a phpMyAdmin over internet is not the best thing to do.

But you can still enjoy phpMyAdmin in a secure way :

Create a virtualhost for phpMyAdmin that accept http connection only for localhost client,
and then connect to your server using ssh with a tunnel so that Apache see localhost connection.


mkdir /home/special/phpMyAdmin
cd /home/special/phpMyAdmin
wget http://freefr.dl.sourceforge.net/sourceforge/phpmyadmin/phpMyAdmin-3.1.5-all-languages.tar.bz2
tar jxf phpMyAdmin-3.1.5-all-languages.tar.bz2
mv phpMyAdmin-3.1.5-all-languages/* .
rm -r phpMyAdmin-3.1.5-all-languages phpMyAdmin-3.1.5-all-languages.tar.bz2


vi /etc/apache2/sites-available/phpmyadmin
<VirtualHost 127.0.0.1:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/phpmyadmin

ServerName localhost

<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>

<Directory /var/phpmyadmin/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

ErrorLog /var/log/apache2/phpmyadmin/error.log
CustomLog /var/log/apache2/phpmyadmin/access.log combined

ServerSignature On
</VirtualHost>

cd /etc/apache2/sites-enabled
ln -s ../sites-available/phpmyadmin 001-phpmyadmin
mkdir /var/log/apache2/phpmyadmin
service apache2 restart


Now, connect to your server using this command :

sudo ssh -lthomas -L 81:localhost:80 my.server.com


(you can do the same with putty SSH->Tunnel)

and type in your browser :

http://localhost:81

you should see phpMyAdmin.


Virtualhost and DNS trick



Sometime you need to access you website by its full domain name while the domain is not yet setup or the website is not open yet (you're still developping it...).


So, you either access to your site with it's IP or maybe mainserverDNSName.com/~username

or if you block the access you may have set up your virtualhost like the phpMyAdmin one with <virtualhost 127.0.0.1:80/> and ServerName localhost

To access you website as if it were already with its production url (which is www.mydomain.com) do this :


Edit on your client machine the etc/hosts file :

[c:\windows\system32\drivers]/etc/hosts file, add

127.0.0.1 www.mydomain.com
#blank line
(replace 127.0.0.1 with the server ip if you were accessing your server by its IP)


open an ssh connection with a tunnel

sudo ssh -lthomas -L 80:localhost:80 serverIp

(or do the same with putty in windows)

type in your browser
http://www.mydomain.com

What happen :

Your local client machine will resolve http://www.mydomain.com as localhost because of your /etc/hosts file.
The http connection will go through your ssh tunnel and hit your web server as a local(from the server point of view) connection with a servername in the http request "www.mydomain.com" so that Apache will select the right virtualhost.

(It might not be that clear... I'll maybe update this part with some more detailed explanation).

DHCP server with ubuntu

first install the correct package :
sudo aptitude install dhcp3-server
then edit the configuration file :
vim /etc/dhcp3/dhcpd.conf
ddns-update-style none;

# option definitions common to all supported networks...

option domain-name "domain.tld"
option domain-name-servers 80.10.246.3, 80.10.246.1, 80.10.246.129;

default-lease-time 600;
max-lease-time 7200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
#authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).

log-facility local7;

# No service will be given on this subnet, but declaring it helps the
# DHCP server to understand the network topology.


# eth0 subnet configuration

subnet 192.168.2.0 netmask 255.255.255.0 {
range 192.168.2.100 192.168.2.200;
option routers 192.168.2.254;
option broadcast-address 192.168.2.255;
}
Replace domain.tld by your local domain (keep the quotes around the domaine).
Replace the nameservers by your ISP domain servers (or your local DNS server)
then modify the subnet configuration according to your network.


Here I setup a dhcp server on network 192.168.2.0 (gateway : 192.168.2.254).

The dhcp server will attribute ip from 192.168.2.100 to 192.168.2.200.

then start the server :

sudo service dhcp3-server start

Friday, 10 April 2009

Upgrade from kubuntu 8.10 to 9.04 with raid disk : no block devices found

Well... I could have waited 15 more days to have the stable version of Ubuntu 9.04 available for upgrade... but no... It seems I wasn't aware i've so much time to loose.







So I've upgraded my kubuntu 8.10 (64bit) to the 9.04 version.

All went smoothly until I reboot where I get this nice message :

no block devices found (4 times)
Gave up waiting for root device.
ALERT! /dev/md3 does not exist. dropping to a shell!

Apparently the raid configuration has been dropped during the upgrade.

I've filled a bug here : https://bugs.launchpad.net/ubuntu/+bug/358054

I don't know exactly what in my setup causes this, but it seems that it's a bug of the last kernel.
Probably my raid setup.

My computer is made of :

P5N32-E Sli plus motherboard
C2D 6250
4GB of RAM
2x 250GB SATA, RAID1 software for linux (sdc, sdd)
2x150GB Raptor SATA Raid0 for windows (fake raid from motherboard) (sda, sdb)


To get my system back on line I just need to do this in the busybox shell :

mdadm -As
exit


The first line reassemble my RAID1 array, and the second line leave busybox, which lead the system to try to resume the boot process. (well i've found this at almost 3AM grrr...)

I didn't figure out on how to fix this (ie no more drop to a shell).
Apparently, booting with a previous version of the kernel solves the issue, but during the upgrade process, it has erased all my previous kernel ;)


Also I've been able to chroot to my system (I tryed to reinstall grub with no luck) and I did the following :

mdadm --assemble --scan

mkdir /mnt

mount -t ext3 /dev/md3 /mnt

mount -o bind /dev /mnt/dev
mount -t proc none /mnt/proc
mount -t sysfs none /mnt/sys
mount -t ext2 /dev/md0 /mnt/boot
mount -t ext3 /dev/md2 /mnt/var
chroot /mnt /bin/bash


Don't try to startX from there, it will start but with no mouse & keyboard you can't do that much...

I'll update if I found a fix...

18/04/2009 : update, the last kernel 2.6.28.11 solves the issue.

Saturday, 4 April 2009

Send mail through gmail smtp server with Exim4 - What to do when gmail account has been disabled


To send mail from a machine using gmail smtp server see this tutorial :

http://www.manu-j.com/blog/wordpress-exim4-ubuntu-gmail-smtp/75/

I've just ran into a problem with this config.

I've created a gmail account just for this purpose because you need to set the gmail account password in exim4 configuration file on each machine.
I didn't want to use my personnal gmail account for obvious reasons.

As it's a technical account, I never log into it through http://gmail.com web interface.

Which lead to the account beeing disabled by gmail.
As a result, all mail that exim4 tryed to send gives this warning message in /var/log/exim4/mainlog :

R=send_via_gmail T=gmail_smtp defer (-53): retry time not reached for any host

To solve this issue, you need to

  1. Login to http://gmail.com with the account you use with exim4. It will ask you to login a second time with a captcha... do so
  2. login on all of your machine that use this technique and issue the following command :

    sudo exim -qff
and you should now recieve all mail which was unsent.
In exim4 logs you should see the following line :

2009-04-04 17:07:44 1Lq7Jf-0007WM-VJ => user@domain.com R=send_via_gmail T=gmail_smtp H=gmail-smtp-msa.l.google.com [74.125.79.109] X=TLS1.0:RSA_ARCFOUR_MD5:16 DN="C=US,ST=California,L=Mountain View,O=Google Inc,CN=smtp.gmail.com"
2009-04-04 17:07:44 1Lq7Jf-0007WM-VJ Completed


One more thing, don't forget to login once in a while on your gmail account so that it's not disabled. A solution is maybe a firefox extension like gmail manager.

Friday, 13 March 2009

Holidays...


Hi,

I'm in holdiays until the end of march, far far away from any computers ;)

Thomas.

Friday, 6 February 2009

Gears on linux x64 ...


gmail, google docs and google calendar now offers offline support with google Gears ! Great... but...

Gears is not available for x64 linux... I've ask the google support for a release date of an official x64 build... but meanwhile, I've found (yes... google is my friend) this blog : http://nielspeen.com/blog/2009/02/google-gears-64-bit/ that gives a x64 build of gears that actually works ;)

Great !

Btw, I can understand that one loves Yahoo! webmail which is great, but Microsoft Live mail... seriously... it's ugly, slow, so poor in functionality... why use it when you have access to an awesome webmail like gmail?? ;)

Update : new version of the x64 version is available on niespeen blog ( 0.5.18.0 version (instead of the 0.5.16.0 version proposed by the firefox updater))

Thursday, 5 February 2009

Limit the upload bandwidth of your apache webserver with mod_bw


I've a server at home on a ADSL connection which has 100kBytes/s of upload and 1.2MBytes/s of download as bandwidth.

If someone download at full speed some large file on my web server, my internet connection become unusable.

To avoid this, I start to ask my friends to use a download manager to limit their download speed... but it's somewhat far too tricky for a lot of them...

So I use mod_bw developed by Ivan Barrera http://www.ivn.cl/apache/ (sourceforge : http://bwmod.sourceforge.net/).

Here is a quick way to limit the bandwidth used by Apache :

Install & enable (on Ubuntu):
sudo apt-get install libapache2-mod-bw
sudo a2enmod bw


Configure (your apache virtualhost conf file like /etc/apache2/sites-available/default) :
<virtualhost *:80>

#... other config stuf for the virtual host

BandWidthModule On
ForceBandWidthModule On
BandWidth 192.168.0.0/24 0
BandWidth all 80000
</virtualhost>


Explaination :
  • line 1 : Activate the mod for the virtualhost.
  • line 2 : Set a directive so that the mod limit the bandwidth on all http request. (you can use an alternate config to limit only some mime type)
  • line 3 : set no limit for computer on your local area network.
  • line 4 : set a limit to 80KB/s to all other computer


Restart Apache so that the config is loaded and test :
service apache2 restart (or apachectl restart)


Test with Firefox on a large file. From my computer, I've first commented with a heading # the third line (BandWidth 192.168.0.0/24 0) and when it works, I uncommented it.

Saturday, 24 January 2009

How to solve GPG warning about apt/aptitude repository update


When you add a repository in your /etc/apt/sources.list and run a apt-update you'll get the update of the newly added repository but also a warning about a GPG key like the following one :

Hit http://ppa.launchpad.net intrepid/main Sources
Hit http://fr.archive.ubuntu.com intrepid-proposed/universe Packages
Hit http://ppa.launchpad.net intrepid/main Packages
Fetched 309B in 0s (519B/s)
Reading package lists... Done
W: GPG error: http://ppa.launchpad.net intrepid Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY D702BF6B8C6C1EFD
W: You may want to run apt-get update to correct these problems


Running again apt-update won't solve the issue.
This warning can be safely ignored, but I don't like to leave warning when there is a solution to get rid of it... because when something is going wrong, it would complicate to sort out what is safe to be ignored and what tells about a real issue.

So, to get rid of this warning here is the two command you need to execute :

gpg --keyserver subkeys.pgp.net --recv D702BF6B8C6C1EFD
gpg --export --armor D702BF6B8C6C1EFD | sudo apt-key add -
sudo apt-get update


which will give this output :



thomas@daisybox:~$ gpg --keyserver subkeys.pgp.net --recv D702BF6B8C6C1EFD
gpg: directory `/home/thomas/.gnupg' created
gpg: new configuration file `/home/thomas/.gnupg/gpg.conf' created
gpg: WARNING: options in `/home/thomas/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/home/thomas/.gnupg/secring.gpg' created
gpg: keyring `/home/thomas/.gnupg/pubring.gpg' created
gpg: requesting key 8C6C1EFD from hkp server subkeys.pgp.net
gpg: /home/thomas/.gnupg/trustdb.gpg: trustdb created
gpg: key 8C6C1EFD: public key "Launchpad PPA for Bazaar Developers" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
thomas@daisybox:~$ gpg --export --armor D702BF6B8C6C1EFD | sudo apt-key add -
OK
thomas@daisybox:~$ sudo apt-get update
Hit http://fr.archive.ubuntu.com intrepid Release.gpg
....
Hit http://packages.medibuntu.org intrepid/free Sources
Hit http://packages.medibuntu.org intrepid/non-free Sources
Fetched 309B in 0s (478B/s)
Reading package lists... Done
thomas@daisybox:~$


No more warning ;)

Saturday, 3 January 2009

Thoughs about Facebook


Today I've converted one of my friends to Facebook...


This lead me to think about why I like Facebook so much and why I think it's awesome :


Basic facts :
  1. It allows you to keep in touch with your acquaintance, see what's happening in their lives without actually spying them ;)
  2. Give updates about you in the same way.
  3. Use some useful apps like 'Birthday Calendar' that help you not to forget you're friends birthday ;)
  4. Get in deeper touch with some friends of your friends that you find utterly interesting in a way far less obvious than match.com like website ;)
    You can be helped with some apps like 'Flirtable' or 'Are YOU Interested'
  5. Still on this friends and/or flirt side, when you go back from holidays and want to keep in touch, it's far easier to retrieve somebody with Facebook than exchanging email, or worse... postal address...
  6. And again, you can force your lover to put 'In relationship with you' on their information page, so that everybody knows you're off the market :oP
  7. Retrieve some old acquaintance... I've been retrieved by a girl of high school on whom I had a big crush at this time. It was really nice to chat again with this lovely person who is now married and mother of a child and live in south of france... (too bad for me;)
  8. Make some advertising to your friends of project you're doing. My brother in law launches an independant music label and use Facebook to advertise on event of his group... quite a nice way to do it.
  9. There's loads of apps on which you can spent really lot of time when you're bored...
  10. For the geek that I am, I can see some really nice idea used on this website... (Ok, that doesn't interest that many people...)

Deeper thoughts : Some of the things that happen on Facebook makes you feel worthwhile, and that's probably why Facebook is so popular.
  1. Seeing people reacting to you're Status update/photos and other activities make you feel important
  2. How important you are: you have soooo much friends !
  3. Being able to show the world you're belief by subscribing to some Cause like fighting AIDES, increase this feeling of being worthwhile.
  4. In some other minor and funny way, subscribe to some 'stupid' groups like 'I'd Throw A Shoe At G.W. Bush If I Could...' or 'My Friends Are Getting Married. I'm Just Getting Drunk.' shows some of your interest and feeling.

Also, Facebook can be dangerous as one may not weight the consequence of what they are doing on Facebook :
  1. Inviting you're immediate superior of work would be generally a mistake...

    What happen in your private life, should remain private for them.
    If you have to have them as friend (so that they won't be upset by a refusal), you should specify some specific privacy settings for them so that they can't read your status update, photo of you...

    What if you want to say : 'I'm fucking bored of my work, and want to change of it' or 'I was totally drunked all the week end long...'. Or you're tagged in some funny situation in some photos...
    Well, this might have some unexpected consequences at work.

    Also if you're too talkative on your issues on Facebook status, and you're a manager (or something like this), these facts can weaken you're position at work : hey... no you're not the flawless/indestructible person everybody think you are... I've read some stories about sharing private things at work that ends this way...
  2. Don't forget with Facebook you're publishing your life to earth (well... that part wealthy enough to have access to Facebook). Every time you do something, think 'Do I really want everybody to know that [picture/text etc...]
  3. All your friends has Facebook to... when you go at party... Digital Camera works a lot... For 2009 new year's eve, we were 6, we took near 500 photos... Photo is cheap... but some can be embarrassing... So... yes... you now must behave at party ;)
  4. Little annoying : Generate bunch of mail alert... you'd better to create a filter in gmail that archive Facebook mail so that it doesn't spoil your inbox. (gmail is just the best webmail client ever created...)

But still.... Facebook is awesome and I really love it.