Switching from CVS to Baazar Version Control System

(note : this post will be improved soon)

I'm changing my server to a new machine as the existing one as capacitor leak issue on the motherboard...
I switch from sme server to ubuntu server.
And as I was almost finishing the migration of my cvs repository to the new server, I asked myself : and what about SVN ? and Bazaar on which I've read about on mysql developer site ? And git, the VCS used for the linux kernel made by Linus Torvald himself (GIT and the Kernel)

Here is a site that help me to have an idea of the feature of each VCS :

http://www.versioncontrolblog.com/comparison/Bazaar/CVS/Git/Subversion/index.html

I've made a try with GIT, but due to a poor eclipse plugin which development seems to have stalled. I've chosen to try Bazaar.

And here is links that answer my main concern :

• CVS to Bazaar migration
• Through SSH connection
• Eclipse plug-in

I've also mainly choose Bazaar because I already know cvs quite well, and made some try on SVN, but didn't know Bazaar at all (And I'm curious).

I really like to thanks the people on the Bazaar mailing list for their help.

I've started to install the eclipse plugin to see if ssh connection was possible.

Eclipse plugin install

I've installed it by following the install instruction.

which says :

1. Install bazaar where you're running eclipse
2. Install bazaar xml output plugin
3. Install the Eclipse plugin

I've downloaded the latest available version here (window version for now) :

https://launchpad.net/bzr/+download

check md5 with my portableapps WinMD5sum

Run the installer, choose the install path.

Next I downloaded the bzr-XMLOutput windows plugin installer and make a md5 sum check.
Run the installer, use the default directory.

Then I've installed the eclipse plugin, Help->Software Update, change to "Available software" tab, click on add site button, paste

http://verterok.com.ar/bzr-eclipse/update-site/

Click Ok, expand the newly added site, tick the "Eclipse plugin for Bazaar" and click the Install button in the upper right corner.


Once installed, I've setup the server side on my new ubuntu server for the CVS to Bazaar conversion.

CVS to Bazaar conversion

In order to have a fairly recent release of Bazaar on ubuntu you have to modify the apt sources list, or you'll only get the 1.3.0 version of Bazaar whereas the 1.8.0 is about to be released.

sudo vi /etc/apt/sources.list

Add the following two lines to the file :

deb http://ppa.launchpad.net/bzr/ubuntu hardy main
deb-src http://ppa.launchpad.net/bzr/ubuntu hardy main

And then run :

sudo apt-get update
sudo apt-get install svn bzr bzrtools
mkdir -p ~/.bazaar/plugins
cd ~/.bazaar/plugins
bzr branch lp:bzr-fastimport fastimport        #(1)
bzr checkout lp:bzr-cvsps-import cvsps_import  #(2)

(1) is for conversion with CVS2SVN
(2) is for conversion with cvs2ps


(svn is needed to get one conversion tool)

Next you need to tell Bazaar who you are :

bzr whoami 'Thomas <yourmail@mailprovider.com>'

Now, here is the conversion part.

As I'm a soooooo lucky man, I had a filename encoding issue : a few file filename happen to be encoded in a non UTF-8/non ASCII encoding and this was really a big problem to the conversion tools of bazaar.
So I've tryed almost every conversion solution ;)

Unfortunately, the conversion instruction are a bit laconic. Thanks to the Bazaar mailing list and it's large and active community, I've been able to try each solution and finally get something working with my encoding issue (Hard to be French, I tell you ;)

CVS2SVN conversion tool

This tools seems to be the more accurate according to what I've read on the mailing list. (Although, this one crashed on my filename encoding issue)
There's no mistake, it's "CVS2SVN" not "CVS2BZR" or something.

This tool was initialy developed for CVS 2 SVN conversion, but it evolved to output a fileformat understandable by GIT (yes GIT...), and Bzr conversion tool also understand it as well ;)

Get CVS2SVN tool

Ubuntu Hardy Heron apt package is also old, so you should get it directly from the repository :

mkdir ~/temp
cd ~/temp
svn co --username=guest http://cvs2svn.tigris.org/svn/cvs2svn/trunk cvs2svn-trunk

simply press enter when asked for a password.

Now get your CVS repository backup file (tar jcf of the repository directory. If you can't have this try another solution)

cd
mkdir -p temp/cvsrepo
cd temp/cvsrepo
scp root@10.0.0.99:/backups/local/files/20081004_ibay_cvs.tar.bz2 .
tar jxf 20081004_ibay_cvs.tar.bz2
cd ..

Now your CVS Repository is here : ~/temp/cvsrepo and one of its subdirectory must be CVSROOT.
And the conversion tool is here : ~/temp/cvs2svn
The current working directory is ~/temp

Now we need to get some configuration file sample.
I won't give away my configuration files because, the tool maintainer says on its site that this configuration file format often change.

cp cvs2svn/cvs2svn-trunk/cvs2svn-example.options .
cp cvs2svn/cvs2svn-trunk/test-data/main-cvsrepos/cvs2svn-git.options .

In cvs2svn-git.options file, there's the path to the CVS repository to change :
look around the line 122 where you should see : r'test-data/main-cvsrepos',

and change it to the relative path to your cvs repo.
As my working directory will be ~/temp and that the cvs repo is in ~/temp/cvsrepo I've just put cvsrepo :

# Now add a project to be converted to git:
run_options.add_project(
# The path to the part of the CVS repository (*not* a CVS working
# copy) that should be converted. This may be a subdirectory
# (i.e., a module) within a larger CVS repository.
r'cvsrepo',

# See cvs2svn-example.options for more docume

Next I had to edit cvs2svn-example.options
because my commit comment were in latin1 encoding, not UTF-8 encoding, so in order to get rid of conversion Warning :

# How to convert author names, log messages, and filenames to unicode.
# The first argument to CVSTextDecoder is a list of encoders that are
# tried in order in 'strict' mode until one of them succeeds. If none
# of those succeeds, then fallback_encoder is used in lossy 'replace'
# mode (if it is specified). Setting a fallback encoder ensures that
# the encoder always succeeds, but it can cause information loss.
ctx.cvs_author_decoder = CVSTextDecoder(
[
'latin1',
#'utf8',
#'ascii',
],
fallback_encoding='ascii'
)
ctx.cvs_log_decoder = CVSTextDecoder(
[
'latin1',
#'utf8',
#'ascii',
],
fallback_encoding='ascii'
)

Then run the tool :

./cvs2svn/cvs2svn-trunk/cvs2svn --options cvs2svn-git.options

If all goes well, you should see a ~/temp/cvs2svn-tmp directory with 2 files in it :

thomas@home:~/temp/cvs2svn-tmp$ ll
total 238720
drwxr-xr-x  2 thomas thomas      4096 2008-10-07 17:48 .
drwxr-xr-x 11 thomas thomas      4096 2008-10-13 13:40 ..
-rw-r--r--  1 thomas thomas 241749759 2008-10-07 17:47 git-blob.dat
-rw-r--r--  1 thomas thomas   2444025 2008-10-07 17:48 git-dump.dat

That's what GIT and Bazaar conversion tools understand.

Here is how you convert to Bazaar :
mkdir ~/temp/bzr
cd ~/temp/bzr
bzr init-repo .
cat ../cvs2svn-tmp/git-blob.dat ../cvs2svn-tmp/git-dump.dat | bzr fast-import -

After this command has been executed, your directory should have been filled with your files. And the revisions of these files in .bzr directory.

Here was some clue for the instructions found on fast import page:

bzr init-repo .
front-end | bzr fast-import -

Too easy ;)

CVS2PS import

Install some needed tools ( you still need the bzr cvs2ps plugin (2))

sudo apt-get install cvs cvsps rcs

(Note: you can do without rcs, with --use-cvs but it's slower)

Your cvs repo is here : ~/temp/cvsrepo

mkdir ~/temp/bazaar
cd ~/temp
bzr cvsps-import cvsrepo . bazaar

It should convert your entire repository (~/temp/cvsrepo) into ~/temp/bazaar directory.
If you want to convert only one module, use your module name instead of '.' in the command.

Tailor

The last tool I tried to convert my CVS repo to Bazaar is Tailor.
It's this tools that succeed in converting my repository that have filename encoding.
Instead of crashing on files that have issue (only 2 for what I know), Tailor trap the exception and continue the conversion.
The two files I lost was doc files with nothing really important.

Thanks to Colin D Bennett (who apparently works on Grub) who gave me it's configuration files for Tailor I did try tailor and succeed. (I was about to give up the history of my project and import as a new project).

Tailor connect to your CVS repository and do not need to have a direct access to the repository files.

On my linux box, I use CVS through SSH, I'm not using a pserver.

Here is the content of one of my tailor configuration files :

thomas@home:~/temp$ cat crf-irp.tailor
[crf]
patch-name-format = None
source = cvs:crf-irp
target = bzr:crf-irp

[cvs:crf-irp]
repository = :extssh:thomas@my.server.com:/cvs
module = crf-irp
encoding = iso-8859-1

[bzr:crf-irp]

The first [crf] is a name you pick.
Next, source & target will be the name between bracket.
In the source section, you put your cvs configuration.
the target section doesn't need more instruction (bzr: seems to be enough).

I advise you to use a public key ssh authentication as it will ask you for the password several times.

Repeat for each module of your repository.

Publication of my (newly converted to Bazaar) project to Launchpad

Before Bazaar, I've set up on my own server a CVS server, JIRA/Confluence/Fisheye for the project handling.
As I've made no release yet (it's should happen soon), JIRA/Confluence was not really used.

Now I've migrate to Bazaar, I could set up a Bazaar server. But I did loose a huge amount of time on the Bazaar conversion (because of this ***king filename encoding issue) and I found it would be more efficient and more reliable to use Launchpad.net plateform.

So, I've created my project (http://launchpad.net/crf-irp), update my profile with my public key, load my private key into pagent.
and ran the following command from windows (My laptop unfortunately run under vista)

bzr push bzr+ssh://mylogin@bazaar.launchpad.net/~mylogin/crf-irp/crf-irp

bzr push bzr+ssh://mylogin@bazaar.launchpad.net/~mylogin/crf-irp/crf-irp-model

bzr push bzr+ssh://mylogin@bazaar.launchpad.net/~mylogin/crf-irp/crf-utilities

It took some times as my repo was about 300MB.


Next Blog post : How to import a project with the eclipse plugin.