Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Tuesday, January 31, 2012

The Headache of a Separate /usr Partition

I installed Arch Linux inside a VirtualBox. Originally I made a bad decision and only allocated 20 GB for the whole Linux hard disk, among which root partition occupies 8.3 GB. Later on, I found that I need to squeeze disk spaces since the root partition is quickly occupied by ever increasing packages. Then I decided that I need to add another disk to balance the load of root partition. Which directory shall I put on the separate partition? After looking at the disk usage, /usr seems a nice candidate. So I created another virtual disk, moved all /usr content to that disk, and mount it as /usr. There were problems in this process since /usr contains many important binaries, but I overcome these issues. Finally I have enough space in my root partition, although everytime Arch Linux boot process complains that separate /usr partition is not supported. I didn't care since everything went fine.

Yesterday I rebooted Linux. During boot process, Linux complained that /dev/sdb1 is mounted. e2fsck: Cannot continue, aboring. I was given a typical panic screen to either enter root password and fix the problem, or hit Control-D to continue. /dev/sdb1 is where my current /usr partition. So I knew that I was finally hit by the separate /usr partition problem.

How to solve the problem? After Googling around, the solution is to disable boot time disk checking, as discussed here. So I entered root password, and mounted root partition as writable by typing mount -o remount,rw /. Then I edit /etc/fstab. The trick here is to change the last parameter (pass) to 0 for partition /dev/sdb1. The line

UUID=adbe588d-fd82-4ab1-b935-5ea245cf89fa /usr ext4 defaults,noatime,nodiratime 0 2
is changed to
UUID=adbe588d-fd82-4ab1-b935-5ea245cf89fa /usr ext4 defaults,noatime,nodiratime 0 0

There are issues with such solution though. Boot time disk checking is useful to ensure data consistency. The solution bypasses such safety guard. But currently I cannot find any other solutions to boot the Linux box. Since I cannot check the partition after booting (since /usr is busy), I may need other ways to check the partition (maybe boot via CD-ROM and check the partition manually). The lesson here is that a small decision error at the beginning may have large implications at the end.

Monday, November 7, 2011

Displaying ANSI Escape Sequences in Windows

I installed Busybox on my Samsung Galaxy S II to enjoy the Swiss Army Knife of Embedded Linux. When I want to look at my phone SD card content, I often use adb if the phone is connected with my laptop. So I typed the following command at Windows command prompt:

adb shell ls /sdcard

What I got is somehow scary as shown below (only first line is shown):

←[1;34mAndroid←[0m ←[1;34mTitaniumBackup←[0m

After a while, I realized that what is shown is actually ANSI escape codes. The ls command provided by Busybox is actually showing color codes by default.

So what shall I do? I know option --color=never can suppress ANSI escape sequences but I don't like typing such long keystrokes. On the other hand, getting useful information out of those ANSI codes is very difficult. My first reaction is trying to see whether Windows shell can show those ANSI codes. I remember my old friend DOS can display ANSI codes, so a modern Windows 7 should be able to do so. But I cannot find relevant configuration in Properties dialog of command prompt. After reading Wikipedia page on ANSI escape codes, I found the following sentence Console windows in all versions of Windows do not support ANSI escape sequences at all. Therefore the only solution is to find some console replacements which can show ANSI codes.

I do found some by Googling. But before trying one, I asked myself why I should install one software for just such a trivial task? I suddenly thought: how about Emacs shell? Maybe Emacs folks have already solved this issue? So I fired up Emacs, typed M-x shell, and typed adb shell ls /sdcard in Emacs shell. Bingo! All ANSI color codes are correctly shown, just as in an Linux box when typing ls --color.

Ubiquitous Emacs to the rescue again.

Saturday, December 25, 2010

Switching From Gentoo Linux to Arch Linux

Introduction

Although I have been using Gentoo Linux for several years, I started to consider switching to other Linux distributions. The main drawback of Gentoo is that too much time is spent on compiling. It is really cool to build a cutting-edge system, however the time spent on building it and maintaining it is enormous. Therefore I am searching a minimalist's Linux distribution with binary packages. Arch Linux seems to be quite good. I gave it a try under VirtualBox 4.0. The reference I used was Beginner's Guide.


Install the base system

The overall installation steps are quite similar to Gentoo: you need to configure many things by hand. The difference is that the installation is significantly faster.

If possible, one should select the fastest mirror since downloading time is the most significant component of installing Arch Linux packages. For me, I select the mirror listed here with the best score for my location. After determining the mirror, you can download the ISO image from that mirror, and add it to your /etc/pacman.conf after installation. For example by adding Server = http://mirror.url/archlinux/$repo/os/x86_64 under the repo names, or uncomment preferred mirros in /etc/pacman.d/mirrorlist (which is preferred). As an alternative, one may also try reflector after installing Arch Linux. Yet another option (I'm currently using) is to rely on script rankmirrors (according to here) to select the fastest mirrors. Note that for rankmirrors option, file /etc/pacman.conf needs one line Architecture = x86_64.

To enable the compatibility with 32-bit legacy programs (like Android emulator), multilib should be enabled in pacman.conf as below:

[multilib]
Include = /etc/pacman.d/mirrorlist

Simply download the latest ISO image. I used "core" image since it contains all the base packages for installation. The latest version as writing is archlinux-2011.08.19-core-x86_64.iso. Download and mount it under VirtualBox. Then boot the virtual machine.

For help, open a virtual terminal (by typing Alt+F2) and type less /arch/beginnersguide.txt

The installation steps are as follows:

  1. Login as root.
  2. Type /arch/setup to start the installer.
  3. Select the source as "core-local".
  4. Select Editor according to your taste.
  5. Set the clock by selecting region and timezone according to your location. You should know whether your hardware clock is UTC or local (my settings in VirtualBox is UTC). The recommended setting is UTC.
  6. For "Prepare Hard Drive(s)", firslty I choose "Manually Partition Hard Drives" and then "Manually Configure block devices, file systems and mount points.". I only setup two partitions, /boot partition (200 MB, ext2), and root partition (remaining disk, ext4.)". For menu "Partition Access Method", select "uuid".
  7. After completing partitioning, go back to main menu and select "Select Packages. For bootloaders, choose "grub". Then for package, select "base-devel" meta-package as well, and select following packages in addition: gpm, inetutils, linux-header, mlocate, ncurses, openssh, pacman-mirrorlist, readline, sqlite3, sqlite3-doc, sudo, xinetd.
  8. Select "Install Packages" from main menu. Select "Yes" to keep packages in pacman cache.
  9. Select "Configure the System" from main menu.
    • Modify /etc/rc.conf. Change "foo" to your desired host name.
      # NETWORKING
      
      interface=eth0
      
      I also removed netfs from DAEMONS line.
    • Write your desired host name to /etc/hostname.
    • Modify /etc/fstab by adding options noatime and nodiratime to partition /.
    • Modify /etc/locale.gen by uncommenting the lines starting as "zh_CN" since I'm a Chinese.
    • Modify /etc/pacman.conf by uncommenting the Xfercommand line in /etc/pacman.conf for wget.
    • Modify /etc/pacman.d/mirrorlist by uncomment your preferred mirrors as discussed above.
    • Change root password.
  10. Back to main menu, select "Install Boot loader". Add option "vga=0x360" for customed VESA mode provided by VirtualBox (as discussed below). Install the bootloader to master boot record (sda in my case).
  11. Finally, select "Exit Install" and type reboot to turn on the new system.

Configuration and Graphics User Interface

This section documents further installation and customization I made.

VirtualBox related settings

Install custom VESA mode to support your PC/Laptop's screen resolution according to VirtualBox manual, if necessary.

Install VirtualBox guest additions as shown below (note that we need to install kernel headers first to compile the additions). Reboot is needed to make additions work. Note that if you want to install Xorg, you can run VBoxLinuxAdditions.run after completing pacman -S xorg. Optionally package xorg-xinit can be installed for script startx and package rxvt-unicode for a nice terminal.

# mkdir /mnt/cdrom
# mount -t iso9660 /dev/cdrom /mnt/cdrom
# cd /mnt/cdrom
# ./VBoxLinuxAdditions.run
# reboot

Proxy: for proxy, add export http_proxy=proxy.url to .bashrc of root. If subversion was later installed, oen need to modifiy ~/.subversion/servers for lines containing http-proxy-host and http-proxy-port. When using sudo to update pacman, one need to add the following line after typing visudo: Defaults env_keep += "http_proxy https_proxy".

To share folders between host and Linux guest, one need to first setup the sharing in VirtualBox GUI. Then after installing Guest Additions as discussed above, the sharing can be enabled like below (assuming the shared folder name is bar and you want to mount it in your Linux guest directory /mnt/bar):

# mkdir /mnt/bar
# mount -t vboxsf bar /mnt/bar

Add additional user

# useradd -m -G audio,optical,storage,video,wheel foo
# passwd foo

Change foo to your desired name.

Run visudo to edit file /etc/sudoers. Uncomment the 1st line containing wheel to enable running root commands from wheel group. One may add lines like Defaults env_keep += "http_proxy" to pass environment variables.

Handling of security

Ru nthe following command: cp /etc/pacman.conf.pacnew /etc/pacman.conf. Apply all previously mentioned changes to /etc/pacman.conf to the overwrittine file. Then uncomment following line:

SigLevel = Optional TrustAll

Run pacman-key --init to initialize keyrings.

Cleanup unused packages

Since some packages are not useful to me, I remove them to avoid future waste of time and bandwidth to update them. Note that you may use pacman -Qe to list the explicitly installed packages. Warning: please make sure you do understand what you're doing: removing important system packages may break your system. As shown below, I only remove those packages related to file systems I'm not using.

# pacman -Rsn jfsutils reiserfsprogs xfsprogs

ABS

Run the following command:

# pacman -S abs

Edit /etc/abs.conf, For REPOS line, add ! before testing.

AUR helper

There are lots of AUR helpers, and I chose pacaur after trying various helpers (e.g. pacaur support proxy seamlessly pacman). To install pacaur, one needs to first install yajl and expac using pacman, then install cower and pacaur by running the following command.

$ sudo pacman -S yajl
$ wget http://aur.archlinux.org/packages/co/cower/PKGBUILD ; makepkg -si
$ wget http://aur.archlinux.org/packages/pa/pacaur/PKGBUILD ; makepkg -si

To install packages from AUR, simply run pacaur -S pkg.

SSHD

If one want to run SSHD server, one need to add sshd to DAEMON line of /etc/rc.conf. One may modify /etc/hosts.allow and/or /etc/hosts.deny. Default is that all connections are denied.

Xorg

Install Xorg by simply running pacman -S xorg. Accept the default (although there are many drivers not needed). Afterward, one need to install VirtualBox Guest Addition for video drivers, as discussed above.

Generally, there is no need to modify /etc/X11/xorg.conf since VirtualBox can handle the file for you when installing Guest additions. There is no need to specify screen resolutions since VirtualBox can seamless adjust it according to host resolution. However, if one wants to restrict available resolutions (e.g. to use StumpWM which is not compatible with VirtualBox's seamleass operation), one can add the following line in Section Screen, Subsection Display. Below example sets up two resolutions with 60 Hz refresh rate.

  Modes "1680x1050_60.00" "1440x900_60.00"

One may install package mesa-demos to benchmark OpenGL performance with glxgears.

For better font rendering, one may install freetype2-infinality, fontconfig-infinality, ttf-google-webfonts, and libxft-lcd from AUR.

GNOME

Install GNOME by typing the command pacman -S gnome. Then add dbus to DAEMONS array in /etc/rc.conf.

Add the following line to ~/.xinitrc:

#!/bin/sh

# Make sure it is at the beginnig of the file, especially before "exec gnome-session"
[ -f /etc/xprofile ] && . /etc/profile
[ -f ~/.xprofile ] && . ~/.xprofile
exec gnome-session

For VirtualBox, type the following to use GNOME 3:

# ln -s /usr/lib/xorg/modules/dri/vboxvideo_dri.so /usr/lib/dri
# pacman -S libltdl # Such reinstallation is needed.
$ gnome-shell --replace

To start GNOME, simply type startx.

I installed following packages in addition:

gconf-editor
- Edit GNOME settings
gnome-system-tools
- contains GNOME admin tools
gksu
- allows normal user to use system tools
gamin
- allows changes to files and directories to be detected immediately.
gnome-media-pulse
- enables volume control as tray notification
faenza-icon-theme
- cool icon theme
baobab
- disk usage

To select fonts suitable for LCD display, right click mouse on desktop, then select Change Desktop Background -> Font, select "Subpixel smoothing (LCDs)" for Rendering.

Update (2011-05-03): I installed login manager GDM by typing pacman -S gdm, and perform following changes to /etc/inittab:

  • comment line id:3:initdefault:
  • uncomment line id:5:initdefault:
  • comment line x:5:respawn:/usr/bin/xdm -nodaemon
  • uncomment line x:5:respawn:/usr/sbin/gdm -nodaemon

Audio

In VirtualBox settings for Audio Controller, I select Intel HD Audio. The sound car can be automatically detected by udev. So what needs to be done is to unmute some channels and let alsa store and restore mixer settings.

# pacman -S alsa-utils
# amixer set Master 90% unmute
# amixer set PCM 85% unmute
# aplay /usr/share/sounds/alsa/Front_Center.wav
# alsactl -f /var/lib/alsa/asound.state store

Finally, edit /etc/rc.conf and add "alsa" to DAEMONS array to store on shutdown and restore mixer settings on boot.


Customization and Applications

Note: this part is quite personal. The purpose I wrote it is mainly for documenting the steps so I myself may use it later.

Applications

Following is a list of applications installed by category. Some packages are only available from AUR and are marked by suffix [A].

  • Development tools: gdb, strace, cmake, valgrind, gprof2dot-git[A], bochs, qemu, mmixware [A], ghc, cabal-install, sbcl, racket, fluxus-git[A], squeak-vm[A], squeak-image[A], ruby, python, ipython, ipython2, pylint, pychecker, pyflakes, pep8[A], scipy, python2-pybrain [A], python2-nltk, wxpython, python-httplib2, python-pygame, ropemacs[A], matplotlib, libgnomeui, python2-pandas, django, nginx, gunicorn[A], postgresql, python2-psycopg2, auto-complete[A], jdk7-openjdk, visualvm[A], eclipse, eclipse-findbugs[A], android-sdk[A], android-ndk[A], android-sdk-platform-tools[A], emacs-android-git[A], scala, scala-docs, scala-sources, sbt, r, emacs-ess-svn[A], nasm, clojure, subversion, rapidsvn, ctags, git, perl-term-readkey, tig, tk, perl-libwww, mercurial, darcs[A], xmlstarlet, cconv[A], emacs-markdown-mode-git[A]
  • Documentation: texlive-most, pdftk[A], aspell-en, multimarkdown-git
  • Font: ttf-dejavu, ttf-inconsolata, terminus-font, wqy-bitmapfont, wqy-zenhei, wqy-microhei[A], ttf-arphic-uming, ttf-ms-fonts[A], ttf-vista-fonts[A], ttf-ms-fonts-zh_cn[A]
  • Multimedia: mplayer, mencoder, ffmpeg, codecs, cdparanoia, lame, cuetools, flac, shntool, wavpack, mac, mp3info, vorbis-tools
  • Web: lynx, chromium, flashplugin-prerelease, w3m, curl, amule
  • Utilities: gnupg, unzip, zip, p7zip
  • Graphics: gnuplot, inkscape, processing, graphviz, gimp, blender, imagemagick, darktable
  • Reader: djview4 , evince, kchmviewer, fbreader, calibre, pdf2djvu, catdoc, tesseract, tesseract-data-eng
  • Network: whois, nfs-utils, xinetd
  • Mathematics: maxima
  • Education: timidity++, solfege
  • X: xscreensaver
  • Fun: povray
  • Libraries: gsl, atlas-lapack[A], libmtp
  • Science: pymol, biopython, stellarium, celestia

Chinese Input

I install ibus as below:

# pacman -S ibus ibus-pinyin

Then add the following lines into ~/.xprofile:

export XMODIFIERS=@im=ibus
export GTK_IM_MODULE=ibus
export QT_IM_MODULE=ibus
ibus-daemon -d -x

Update (2011-01-06): switch from SCIM to ibus.

Emacs

For Emacs package management, el-get is interesting to try. The installation method is very unique and there is no need for an ArchLinux package.

Install scala-mode as below:

$ cd ~/.emacs.d/plugins
$ svn co http://lampsvn.epfl.ch/svn-repos/scala/scala-tool-support/trunk/src/emacs/ scala-mode

Get anything.el by wget http://www.emacswiki.org/emacs/download/anything.el.

Install following packages: auctex, slime-cvs, emacs-w3m-cvs, pymacs.

Install clojure related modes.

$ cd ~/.emacs.d/plugins
$ git clone https://github.com/technomancy/clojure-mode.git clojure-mode
$ git clone https://github.com/technomancy/swank-clojure.git
$ mkdir ~/.swank-clojure
$ cd ~/.swank-clojure
## Get three jars in https://github.com/technomancy/swank-clojure/blog/master/swank-clojure.el manually.

Install Hyperspec for Common Lisp

Run the following commands:

# wget ftp://ftp.lispworks.com/pub/software_tools/reference/HyperSpec-7-0.tar.gz
# tar xzvf HyperSpec-7-0.tar.gz
# mkdir /usr/share/doc/hyperspec
# mv HyperSpec !$

Stumpwm

First invoke sbcl, then type the following into REPL. Note that ASDF install is used other than AUR. Note that installation of trivial-gray-streams, flexi-streams, and cl-unicode might not be needed.

(require 'asdf)
(require 'asdf-install)
(asdf-install:install "http://common-lisp.net/project/cl-plus-ssl/download/trivial-gray-streams.tar.gz")
(asdf-install:install "http://weitz.de/files/flexi-streams.tar.gz")
(asdf-install:install "http://weitz.de/files/cl-unicode.tar.gz")
(asdf-install:install 'clx)
(asdf-install:install 'cl-ppcre)

When prompted by "Install where", it is recommended to select "Personal Installation" since it does not require root privilege. Note that some packages do not have GPG signature, so if SBCL drops you into debugger, simply select skip GPG check. If SBCL complaints and exited (due to in --disable-debugger mode), type the following after enter sbcl (sb-ext:enable-debugger) or add it to /etc/sbclrc as system initialize file for SBCL.

After installation of the dependencies, quit SBCL by type (sb-ext:quit) and run the following at CLI:

$ git clone http://git.sv.gnu.org/r/stumpwm.git stumpwm
$ cd stumpwm
$ ./autogen.sh
$ ./configure
$ make

After copy stumpwm to a directory in PATH, one can add "exec stumpwm" at the end of ~/.xinitrc. Then typing startx will bring up Stumpwm. Since I installed both GNOME and StumpWM, I decides to write two scrips to automate the changes to .~/.xinitrc. With these scripts, I can type startg for GNOME or starts for StumpWM.

Contents of script startg:

#!/usr/bin/env bash
sed -e 's/exec.*$/exec gnome-session/' -i ~/.xinitrc
startx

Contents of script starts:

#!/usr/bin/env bash
echo "exec stumpwm" > ~/.xinitrc
startx

Miscellaneous

For root, add one line in ~/.bash_profile to enable .bashrc
. $HOME/.bashrc

To run indexing daily, run crontab -e to add the following line (Update (2010-12-28): this step is not needed since updatedb is already added in /etc/cron.daily during installation of mlocate):

11 11 * * * updatedb

To keep boot messages on tty1 instead of clearing screen, add option for tty1 in /etc/inittab like below:

c1:2345:respawn:/sbin/agetty -i -8 38400 tty1 linux

Conclusion

After using Arch Linux for a long time, I have to say that I'm extremely satisfied. Your system may not be extremely fast as Gentoo boxes, but what you gain is significant: invaluable time.

Wednesday, December 24, 2008

Adjusting Photos for Web Albums in Batch Mode

I use Picasa Web Album to store my photos online. To save the precious storage at server, I have to change jpeg files before uploading them even for those taken by my old PowerShot S30 (which outputs photos with resolution 2048x1536, which is beyond the display capacity of most monitors today). To reduce file size, one normally need to reduce both resolution and image quality.

So how to adjust images in batch mode? There are various solutions of course. The one I take is to use the combination of find/xargs/mogrify (Note that mogrify modifies files in place. Therefore one may use convert instead of mogrify to preserve original files.) in Linux. Note that mogrify is one of ImageMagick utilities and may need separate installation depending on distribution. Following is my work flow when publishing images online.

Firstly, I select images with my preferred viewer gthumb. Images are rotated if necessary. During browsing, I copy the chosen files to another folder and run the following command in that folder:

find . -iname "*.jpg" | xargs mogrify -resize resize-option -quality quality-option

(I find the above command from here.)

Several notes regarding the above command:

  • resize-option could be either absolute (e.g. 800x600) or relative (e.g. 50%). For my S30, I use the latter which reduce the photo resolution to 1/4 of its original size.
  • quality-option is JPEG compression level. The larger the value, the better the quality, but the larger the file size. I often set the value as 60 to save space.

So for photos from my S30, I use the following command:

find . -iname "*.jpg" | xargs mogrify -resize 50% -quality 60

After running above command, photo file size can be reduced around 10 times. This means that I can put more images into my web album.

Wednesday, August 20, 2008

W(S)XGA+ Resolution in VirtualBox

Note: this post is related to a workaround for one specific problem encountered by the author.

Recently I switched from coLinux to VirtualBox. VirtualBox provides guest addition which includes enhanced support for Video. Originally it worked fine for me. Due to unknown reason, the problem arises: I could not use resolution 1440x900 (WXGA+) or 1680x1050 (WSXGA+) any more. WXGA+ is the native resolution of my laptop, while WSXGA+ is the resolution of the external LCD when the laptop is placed in a dock. After tweaking for a long time, following is the solution I found so far (I installed Gentoo Linux as guest OS).

1. Revert video driver from VirtualBox to Vesa

This is done by editing /etc/X11/xorg.conf, change one line in Section Device from Driver "vboxvideo" to Driver "vesa".

2. Install support of 1440x900 and 1680x1050 to VESA BIOS

Add parameter vga=864 as Linux bootup parameter (e.g. in /boot/grub/grub.conf) for resolution 1440x900, or vga=865 for 1680x1050.

When Virtual machine is turned off, run the following two commands at Windows command line prompt, assuming virtual machine is named "Gentoo":

VBoxManage setextradata "Gentoo" "CustomeVideoMode1" "1440x900x24"
VBoxManage setextradata "Gentoo" "CustomeVideoMode2" "1680x1050x24"

Start virtual machine, select resolution 1440x900 or 1680x1050 for console display. Note that the resolution for console is not necessarily the same resolution for X, i.e. you can use 1440x900 for console, and 1680x1050 for X.

3. Add modelines for resolution 1440x900 and 1680x1050.

I use gtf to generate needed modelines. For example for resolution 1440x900 with 60 Hz refresh rate, I use the following command:

$ gtf 1440 900 60

which give the following line:

  Modeline "1440x900_60.00"  106.47  1440 1520 1672 1904  900 901 904 932  -HSync +Vsync

I do not like the suffix _60.00, therefore what finally end up in section Monitor of /etc/X11/xorg.conf is (with addition of modeline for 1680x1050):

 modeline "1440x900" 106.47 1440 1520 1672 1904 900 901 904 932 -HSync +Vsync
 modeline  "1680x1050"  147.14  1680 1784 1968 2256  1050 1051 1054 1087  -HSync +Vsync

In addition, the two modes should be added in section Screen of /etc/X11/xorg.conf as well, my current setting is:

Section "Screen"
 Identifier "Screen0"
 Device     "Card0"
 Monitor    "Monitor0"
 DefaultDepth     24
 SubSection "Display"
  Viewport   0 0
  Depth     24
  Modes    "1680x1050" "1440x900"
 EndSubSection
EndSection

4. Select the resolution

Now the toughest part. You may have noticed that in section Monitor, there are two lines about HorizSync and VertRefresh. After experimenting, my conclusion is that one should comment out them when using 1680x1050 while enable them for 1440x900. Weird.

Since I need to switch between these two resolutions frequently, manually editing file /etc/X11/xorg.conf is boring. Therefore I write one script to automate the task. I named the script as xres and invoke it as sudo xres 1440 for resolution 1440x900 and sudo xres 1680 for 1680x1050. Following is its content:

#!/bin/bash
case $1 in
    1440)
        sed -i 's/^#    HorizSync    31.5 - 64.3/       HorizSync    31.5 - 64.3/' /etc/X11/xorg.conf
        sed -i 's/^#    VertRefresh  50.0 - 70.0/       VertRefresh  50.0 - 70.0/' /etc/X11/xorg.conf
        ;;
    1680)
        sed -i 's/^     HorizSync    31.5 - 64.3/#      HorizSync    31.5 - 64.3/' /etc/X11/xorg.conf
        sed -i 's/^     VertRefresh  50.0 - 70.0/#      VertRefresh  50.0 - 70.0/' /etc/X11/xorg.conf
        ;;
    *)
        echo "Supported horizontal resolution: 1440 and 1680."
        exit
        ;;
esac

Thursday, July 24, 2008

Experiencing Stumpwm

What Emacs is to editors, Stumpwm is to window managers.
--Bill Clementson (link)

Introduction

Recently I switched from GNOME to Stumpwm (wiki), which means that I jumped out of a desktop environment to a simple window manager. So far, I'm very satisfied with such a change and never looked back. Stumpwm, as its website says, is a tiling, keyboard driven X11 Window Manager written entirely in Common Lisp. This definition summarizes the reasons why Stumpwm is so suitable for me.

First, Stumpwm is a tiling, keyboard driven window manager. This is very useful to boost productivity. When working, it is desirable to minimize context switching. Tilting combined with keyboard driven enables working with several applications simultaneously as if your are dealing with one single program. Let me give an example. As described in previous post, with Emacs as development environment, it is easy to fire up one browser within Emacs itself to preview blog post. However it is painful to switch between Emacs and browser. Before using Stumpwm, I am considering ways to integrate Emacs and Firefox together. With Stumpwm, such dream is just trivial to fulfill: one can show Emacs and browser side by side, with a few keystrokes to switch between them. Everytime you make changes in Emacs and request the results to be shown in Firefox, they are shown simultaneously. No more need to leave your hand for mouse to click back and forth between Emacs and Firefox. This gives you the feeling of integrated environment. Note that I just use Emacs and Firefox as an example, and such convenience is applicable for every application. Currently, the problem is that I'm so accustomed to Stumpwm keystrokes that I somehow confused the keystrokes of switching between windows in Stumpwm and Emacs!

Second, Stumpwm is written in Common Lisp. which means that you can build Stumpwm with your favorite CL implementation and you have a Lisp runtime when you work under Stumpwm. A powerful programming language is just embedded within your window manager and you can invoke it at any time. The other benefit is that you can configure your window manager as you like, even when it is running!

You may get a feeling on how Stumpwm works by watching this nice video.

Installation

Given the above lengthy introduction of Stumpwm, you may wonder how to install this gem. Here we assume that you have some knowledge about Common Lisp. Basically, installing Stumpwm is to compile it with your preferred CL implementation, and tell your system to run Stumpwm as your window manager.

Installing Stumpwm on Gentoo Linux is straightforward. Throughout this section, we will assume that SBCL is the CL implementation. Stumpwm website suggests turn off threading support in SBCL (disable USE flag threads) for better performance. To install, simply type emerge stumpwm. Note that at the time of writing, there is also an ebuild called stumpwm-cvs. Simply ignore it since it is actually a very old version, not the bleeding edge version suggested by its name. Advanced users might consider to get git version for latest cool features.

There are mainly two ways to invoke Stumpwm. One is to run it by calling SBCL, the other is to dump a core image containing Stumpwm within SBCL and invoke that image. We will take the latter approach. To proceed, first start SBCL by typing sbcl. Next issue the following in REPL sequentially:

(asdf:oos 'asdf:load-op :stumpwm)
(sb-ext:save-lisp-and-die "stumpwm" :executable t
                          :toplevel #'(lambda () (stumpwm:stumpwm ":0")))

Next, put the generated executable stumpwm somewhere in PATH (I put it under /usr/local/bin). Since I always start X Window by typing startx, following command is used to use Stumpwm as my window manager:

$ echo "exec stumpwm" >> ~/.xinitrc

Using Stumpwm

Configuration

All stumpwm configurations are stored in file ~/.stumpwmrc, which is written in Common Lisp. Just like .emacs, this file allows you to fully customize Stumpwm. Following is my current configuration:

;;;; -*- Mode: Lisp -*-

(in-package :stumpwm)

;; Load swank.
(load "/usr/share/emacs/site-lisp/slime/swank-loader.lisp")
(swank-loader:init)
(define-stumpwm-command "swank" ()
  (setf stumpwm:*top-level-error-action* :break)
  (swank:create-server :port 4005
                       :style swank:*communication-style*
                       :dont-close t)
  (echo-string (current-screen) "Starting swank."))
(define-key *root-map* (kbd "C-s") "swank")       

;; Customize bars and modeline.
(setf *message-window-gravity* :center)
(setf *input-window-gravity* :center)
;; Turn on mode line.
(toggle-mode-line (current-screen) (current-head))
(setf *screen-mode-line-format* 
      (list "%w | "
            '(:eval (run-shell-command "date | tr -d '[:cntrl:]'" t))))

(set-prefix-key (kbd "C-i"))
(define-key *root-map* (kbd "c") 
  "exec urxvt +sb -fn \"xft:Bitstream Vera Sans Mono:pixelsize=20\"")

(define-stumpwm-command "firefox" ()
  "Run or switch to firefox."
  (run-or-raise "firefox" '(:class "Firefox")))
(define-key *root-map* (kbd "f") "firefox")

Some explanations of my configuration:

  • Prefix key: I use C-i instead of default C-t. The reason is that C-t is used in Firefox to open a new tab and also in Emacs for transpose. Then why choose C-i? I'd like to admit that it is quite difficult to select a prefix key for Emacs user. Before settle down on C-i, I fired up Emacs to see whether there is any key binding C-x where x from a to z is not used by Emacs. Unfortunately (fortunately?), Emacs binds every combination. So I can only choose one prefix key which is easy to type and I uses infrequently in Emacs. Then C-i is selected. Note that you can send C-i to application like Emacs by typing C-i i. For the following sections, please replace C-i with your favorite prefix key.
  • SLIME: the section staring with comments Load swank provides ways to load swank. Type C-i C-s to start swank. To connect to swank, simply run Emacs, and type slime-connect within Emacs, and type RET and RET to accept default host (127.0.0.1) and default port (4005). Then you can play with Stumpwm as you wish: change parameters, add your own functions etc. Note that I do not start swank automatically for security reasons.
  • Key bindings: I use C-i c to start console: urxvt instead of xterm. Note that I have also set the font for urxvt. In addition I have setup using C-i f to start Firefox in case Firefox is not started, or bring Firefox window to front if it is already running.

Key Bindings

Following is a list of key bindings I used frequently. For simplicity, I have omitted the prefix key.

  • ?: Stumpwm help
  • ;: Run Stumpwm commands
  • :: Send commands to Common Lisp interpreter.
  • Space: Go to next window
  • c: Run X terminal
  • e: Run Emacs or raise it if it is already running
  • f: Run Firefox or raise it if it is already running
  • k: Kill current window
  • g c: Create a new group
  • g k: Kill current group
  • g m: Move current window to a specified group
  • g Space: Next group
  • o: Focus shifts to next frame
  • Q: Remove all splits
  • s: Vertical split
  • S: Horizontal split

Wednesday, May 21, 2008

Using PulseAudio for OSS Applications

In my previous post, I discussed how to use PulseAudio to enable sound for coLinux. This works fine for programs like MPlayer, but has issues with some programs e.g. Solfege.

The solution is simple: if the problematic applications have support for OSS, then emerge them with OSS USE flag on. In addition, PulseAudio should also be emerged with OSS USE flag (to enable the building of padsp, a PulseAudio OSS wrapper, which will be discussed below). Under Gentoo Linux, one way is to simply add OSS USE flag to /etc/make.conf. Alternative way is to enable the USE flag application by application. For example, to enable OSS USE flag for Solfege, type the following:

# echo "media-sound/solfege oss" >> /etc/portage/package.use

When running these applications, prefix the command line with padsp like the following:

$ padsp solfege

Note that you may need additional configurations in the application to use OSS for audio output.

Friday, May 2, 2008

Automatic Backup

We need to backup our stuff in case that some disasters might happen. One way is to backup to some external medias like tape, USB drives, or CD/DVDs. However working with them is somehow tedious and manual intervention is needed. People tend to be lazy therefore we are reluctant to backup if the process is inconvenient. If we have one additional machine with link in between (throughout this article, we assume the remote machine is named as remotebox), we can setup an automatic backup mechanism.

Overview

We use crontab to schedule rsync jobs to backup our files to another machine. Since rsync is secured with SSH, to suppress the password prompt, we have to setup SSH keychain.

Setup SSH Keychain

Generate Keys

This section mainly follows this nice guide.

We use the DSA approach.

$ ssh-keygen -t dsa
$ scp ~/.ssh/id_dsa.pub user@remotebox

Then log in to remotebox and append the public key to ~/.ssh/authorized_keys file like so:

cat id_dsa.pub >> ~/.ssh/authorized_keys
rm id_dsa.pub

Install Keychain

Keychain is a nice package which prompt you for your passphrase once you log into the system, and automatically provides passwords for later SSH session. Note this incurs security risks (there are always tradeoffs between security and convenience) therefore attention is needed.

This part mainly follows this guide.

First install keychain by typing emerge keychain.

Then add following lines to your ~/.bash_profile.

/keychain id_dsa
. ~/.keychain/$HOSTNAME-sh

Setup Crontab Job

We can then setup our automatic backup system now. Since there are almost infinite number of combinations of crontab and rsync patterns, I just shown one example below (many interesting examples of rsync usage can be found by typing man rsync).

0 9,16 * * 1-5 rsync -az -e ssh --delete ~/work/ remotebox:~/backup/

The above example synchronizes local directory ~/work to remote directory ~/backup on host remotebox, on 9 a.m. and 4 p.m. of every working day. Note that I have used option --delete, which will try to delete extraneous files from destination directory. This can ensure that the destination directory is an exact copy of source directory but it would be dangerous if the destination directory contains some files already. So test this option before you actually use it.

Thursday, April 10, 2008

Using Imaxima with Texlive in Gentoo Linux

Imaxima is an Emacs extension to interface with Maxima (see here for a nice tutorial). To use it, one need a LaTeX distribution. However with default Texlive installation in Gentoo, imaxima cannot work. Following is a sample session:

(%i1) integrate(x,x);
LaTex error in: \ifracn{x^{2}}{2}

The problem here is that there is another package breqn needed. Breqn is inlcuded in Texlive package texlive-mathextra, which is not installed when you emerge texlive. The package is also masked by default, therefore you need to unmask it before emerging. Following is a list of commands to install the package (substitute x86 with amd64 for x86-64 systems).

# echo "dev-texlive/texlive-mathextra ~x86" >> /etc/portage/package.keywords
# emerge texlive-mathextra

After installation, fire up your Emacs and enjoy (I)maxima!

Update: if you are not comfortable with the default equation size (as in my case), you can add the following line into your .emacs (adjust the number to suit your own taste):

(setq imaxima-scale-factor 2.0)

Tuesday, April 8, 2008

Copy and Paste between Windows and Linux

Many people use Windows as primary working machine and login to remote Linux/Unix (we will use Linux as shorthand afterwards) boxes. There are times when we need to copy and paste information between Windows and Linux. Below we are going to discuss convenient ways to do so.

Tools

PuTTY and Xming are two popular programs to run in Windows host to facilitate communication to Linux. PuTTY provides console connection while Xming X Window connection. For Xming, Ensure you start Xming with parameter -clipboard (which is enabled by default). Our discussion below is focused on how to copy/paste within PuTTY/Xming. Copy/paste for other Windows applications is quite general (Ctrl+C and Ctrl+V) and should be familiar to the reader, therefore omitted.

Ways

PuTTY

Copy to clipboard: click-and-drag to highlight. Just select, no need to press any key.

Paste from clipboard: right mouse button.

Xming

For Xming applications, copy/paste follows the X window standard as discussed below.

Copy to clipboard: click-and-drag to highlight. Same as PuTTY.

Paste from clipboard: middle mouse button.

Monday, April 7, 2008

Installation of Gentoo Linux 2008.0 Beta 1 with LiveCD

I just started yet another process of installing Gentoo Linux, and following is my record of the installation step by step. Previously I preferred to install from stage 1/3 following this guide. I decided to try LiveCD this time.

1. Boot the machine.

Download the 2008.0 LiveCD from here. Burn the CD and boot the machine with it.

Caveat: in my case, the boot process is stalled. I solved it (according to this article) by appending kernel options during GRUB screen (press e to edit and append all-generic-ide irqpoll pci=nommconf).

2. Installation.

After boot, I click icon "Gentoo Linux Installation (GTK+)" in the desktop.

Partitioning: I selected Recommended Layout.

Local Mounts: accept the defaults without any changes.

Timezone: Asia/Shanghai.

Networking: Dhcp is used. Appropriate host and domain name are provided.

Users: Just add the regular user for daily usage.

Extra Packages: I selected screen and slocate. In my first installation I also selected xorg-x11, gdm, firefox etc. However the installation simply failed when emerging x11-apps/xinit (and I cannot rescue the system since even /etc/fstab is not correctly setup), therefore my second try is quite conservative.

Startup services: I selected sshd.

Other Settings: Although I have not installed X11 or Gnome yet, I selected as following:

  • Display Manager: gdm
  • Clock: local
  • XSession: Gnome

3. Post Installation.

This section is similar as what I have described for coLinux (see here), but is modified specifically for a standalone Gentoo Linux installation.

3.0 Tiny Cleanups.

After rebooting, perform the following steps. Note that if your system was stalled as described before, you can use same trick in GRUB screen and you should add the kernel options to /boot/grub/grub.conf after login.

To avoid some warning, you might need to perform following steps.

# mkdir -p /usr/local/portage

To change host name (according to Gentoo Handbook), edit file /etc/conf.d/hostname, if you call your coLinux machine as tux, then set the following line.

HOSTNAME="tux"

In addition, remove string .\O from file /etc/issue to get rid of the annoying unknown_domain in the welcome message "This is host.unknown_domain ...".

3.1 Network

We have already specified using DHCP. It works but the configuration files is using old syntax. Let's clean it up to avoid future complaints. Edit file /etc/conf.d/net as follows.

config_eth0=( "dhcp" )
3.1.1 Proxy

If you connect to Internet directly, you can skip this section.

To use proxy server, add the following lines into /etc/env.d/99local:

http_proxy="http://proxy.server.com:8080"
ftp_proxy="http://proxy.server.com:8080"

Replace the port and domain name according to your situation. Then run:

env-update && source /etc/profile

3.2 Add additional locales

Modify file /etc/locale.gen, select appropriate locales. For me, the locales are:

en_US.UTF-8 UTF-8
zh_CN.UTF-8 UTF-8

Then run command locale-gen.

3.3 Modify /etc/make.conf

This is specific for your computer type and other factors. Following is my configuration (I'm using a Core 2 Duo CPU). Please refer to Gentoo documentation for your own reference. GENTOO_MIRRORS reflects the fastest mirrors for my connection, as discussed below.

CFLAGS="-march=nocona -O2 -fomit-frame-pointer -pipe -fno-ident"
CXXFLAGS="${CFLAGS}"
CHOST="x86_64-pc-linux-gnu"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j3"
ACCEPT_KEYWORDS="amd64"
FEATURES="parallel-fetch"
LINGUAS="en zh zh_CN"
INPUT_DEVICES="keyboard mouse"
VIDEO_CARDS="i810"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.kaist.ac.kr/pub/gentoo/"

USE_DEV="bash-completion doc emacs latex ruby sbcl source spell"
USE_HW="cpudetection mmx sse sse2"
USE_LIB="curl gd glibc-omitfp ncurses readline zlib"
USE_MM="aac alsa encode ffmpeg flac midi mp3 ogg pulseaudio quicktime theora xvid"
USE_PIC="gif jpeg png raw svg tiff xpm"
USE_SECURITY="crypt pam ssl tcpd"
USE_SW="bzip2"
USE_SYS="cjk nls nptl nptlonly opengl truetype unicode xinerama xml"
USE_X11="cairo gnome gtk X xft xorg"
USE="-* ${USE_DEV} ${USE_HW} ${USE_LIB} ${USE_MM} \
     ${USE_PIC} ${USE_SECURITY} ${USE_SW} ${USE_SYS} ${USE_X11}"

3.4 Synchronize Portage

Use either emerge --sync or emerge-webrsync -v to update the portage.

3.5 Mirror selection

First run emerge mirrorselect. To select the fastest mirror. I use the following command mirrorselect -D -H -s2 to select the mirror. It means that:

  • -D: actual file will be downloaded from the mirror to test speed.
  • -H: test http mirrors only.
  • -s2: only select 2 mirrors.

After running the command, original /etc/make.conf will be backed up as /etc/make.conf.backup and your selected mirrors will be reflected in make.conf. Type mirrorselect -h for details.

3.6 Update system

Run the following suites of commands (we need the -p argument to check what the commands are supposed to do since they're somehow "dangerous"):

emerge -uDNpv world
emerge -uDNv world
emerge --depclean -pv 
emerge --depclean -v
revdep-rebuild -pv
revdep-rebuild -v

Caveat: if you encounter circular dependency when emerge -uDNv world, try to remove gtk USE flag from /etc/make.conf. After successful emerging, add gtk flag again and run emerge -uDNv world again.

After update, run dispatch-conf to update the configuration files.

3.7 Install X11 and GNOME.

Run command emerge xorg-x11 to install X11. Then use xorgcfg -textmode to configure X server.

I'm using a LCD with resolution of 1680x1050, which is not listed in standard configurations. To use it, first run gtf 1680 1050 60, copy the output to the Monitor section of /etc/X11/xorg.conf, and modify section Screen to the resolution 1680x1050.

For GNOME, I prefer to install gnome-light to avoid the gigantic full GNOME installation. To do so, simply run emerge gnome-light.

If one use startx to run X11, one can simply type echo "exec gnome-session" > ~/.xinitrc to configure so.

To make gdm the default login screen, run the following commands:

# emerge gdm
# rc-update add xdm default

And then edit /etc/conf.d/xdm, changing DISPLAYMANAGER to gdm.

3.8 Install Other programs.

This section is left empty intentionally :-)


Conclusion

It is quite convenient to install with LiveCD. There are some hassles during installation, but I assume the reason is that I'm using a beta version. Since beta version is not stable, my suggestion is to install minimal components therefore you can quickly finish the installation and boot from hard drive.

Although GRP install is not so optimal, the pre-compiled packages should be quite close to optimal ones for 64-bit machines. Also as a Gentoo user, I'm always in the process of customization the OS to suit my needs, therefore a not-so-ideal-initial-setup is not an issue. In summary, livecd approach can save a lot of time, with almost similar end result as installing from stage 1/3.

Ripping CD to FLAC in Gentoo Linux

This post is just a simple note, inspired from here.

First, install cdparanoia and flac:

# emerge cdparanoia flac

Next, issue the following commands (I deleted the original WAV files after conversion). All tracks are extracted and converted to FLAC format.

# cdparanoia -B
# find . -name "*.wav" -print0 | xargs -0 flac
# rm *.wav

Then enjoy the music with your favorite player. Since I'm using mplayer, I call Ruby to generate the playlist and invoke mplayer with it (in the 1st line below, the last track number is 16. Change it according to your own situation).

$ ruby -e '(0..16).map{|i| printf("track%02d.cdda.flac\n", i)}' > pl
$ mplayer -playlist pl

Wednesday, April 2, 2008

Enable Sound for coLinux

In my previous post about installing coLinux with Gentoo, audio part is not touched. Here comes the solution on how to use PulseAudio to bridge coLinux to sound world. This is adapted from here with contents modified for Gentoo Linux.

Windows side

We need to install PulseAudio server on Windows. Frist download binaries for Windows from here. Extract to one directory (c:\bin\pulseaudio in my case).

In the directory of the binaries, create a text config file named default.pa with the content:

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24
load-module module-esound-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24
load-module module-detect

We start PulseAudio automatically when coLinux starts, by adding the following line to the coLinux configuration file:

exec0=c:\bin\pulseaudio\pulseaudio.exe

Linux side

Install and configure PulseAudio client

Install PulseAudio by emerge pulseaudio. Then edit /etc/pulse/client.conf to set the address of the Windows host that runs the PulseAudio server:

default-server = 192.168.0.1

where 192.168.0.1 is the IP address of the Windows host.

Make sure to add USE flag alsa to /etc/make.conf. If you just add this USE flag, update your system with emerge -uDNv world. ALSA would be automatically emerged.

Install alsa-plugins

In addition, we should install alsa-plugins:

# echo "media-plugins/alsa-plugins pulseaudio" >> /etc/portage/package.use
# emerge alsa-plugins

Edit ~/.asoundrc with the following contents.

pcm.!default {
    type pulse
}

ctl.!default {
    type pulse
}

pcm.pulse {
    type pulse
}

ctl.pulse {
    type pulse
}

Test

As a final step, you can test with an ALSA player:

aplay /usr/share/sounds/alsa/Front_Center.wav

You should hear a voice saying "Front Center". Enjoy!

Sunday, March 16, 2008

Install Gentoo with coLinux

coLinux is gaining momentum as a way to run Linux programs on Windows, and I decided to try it. As a user of Gentoo, my first choice is to use Gentoo with coLinux. The following description documents what I have done to run Gentoo cooperatively in Windows laptop. I mainly follows up the Gentoo Wiki Howto article and other coLinux resources.

1. Installation

First download the latest coLinux snapshot (you can also get it from SourceForge). The file I used is coLinux-0.7.2.exe from SourceForge. Run the installer. Some notes during installation:

  • In screen Choose Components, leave the default settings on (i.e. selecting all components), but uncheck Root FileSystem image download (we will download it separately).
  • In addition, change the Destination folder to c:\bin\coLinux (note that all the description below assumes coLinux is installed into this directory. If you prefer other directory, please modify accordingly in steps below).
  • Installer prompts to download and install WinPCap separately, but my network configuration does not need WinPCap therefore I don't install it.

Then download Gentoo-colinux-i686-2007-03-03.7z from SourceForge, extract the archive into c:\bin\coLinux, with a gentoo subdirectory created during extraction. Copy file gentoo.conf in gentoo to the parent directory and modify it as follows:

kernel=vmlinux
cobd0=C:\bin\coLinux\gentoo\gentoo-root
cobd1=C:\bin\coLinux\gentoo\linux-swap
root=/dev/cobd0
hdc=\Device\Cdrom0
initrd=initrd.gz
mem=1024
eth0=tuntap
cofs0=c:\linux

Some explanations of the above file, details will be covered in later sections.

  • For mem, I set it to 1G. Choose according to your own situation.
  • For network, I only select TAP driver.
  • I shared Windows folder c:\linux to coLinux.
  • CD/DVD-Drive is mounted.

To test drive, open a command prompt, change to the directory c:\bin\coLinux, and run the following command:

colinux-daemon @gentoo.conf

coLinux console should run. If there is something wrong, please check previous steps. After some time, you are prompted for log in. You can now login as root with password. Change your password immediately.

Congratulations! You just successfully run Gentoo in coLinux! You can now fool around with the system. Type halt when you are tired. There is still a long road ahead to fully harness the power. Following is two optional steps, but highly recommended.

Run coLinux during system startup

First, we need to install coLinux as a service. Open a command prompt, change to the directory c:\bin\coLinux, and run the following command:

colinux-daemon.exe @gentoo.conf --install-service "CoGentoo Linux" 

Then open Control Panel, first select Administrative Tools, then Services, click service CoGentoo Linux, and change its Startup type to Automatic.

Enlarge the image file

The original image size is 2G. This might not be sufficient. To enlarge it, you need to download the tools from here. Extract the archive and run file toporesize.bat. Use the program to resize file c:\bin\coLinux\gentoo\gentoo-root to whatever size you want. Make sure you uncheck "resize file only no resize2fs".

2. Configuration

2.1 Timezone

Since I'm located in Beijing, I set the corresponding info as follows. The related configuration files is /etc/conf.d/clock. I changed two lines as follows.

CLOCK="local"
TIMEZONE="Asia/Shanghai"

Also run command ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime.

It is interesting that Beijing cannot manage to snap one place inside /usr/share/zoneinfo/Asia.

2.2 Network

Based on coLinux wiki article.

2.2.1 One network interface

This configuration only enable TAP, which is used to enable both the high-speed private connection between Windows host and Linux guest, and also to bridge the connection to Internet. One drawback of this setup is that when you Internet connection changes (e.g. from ADSL to WiFi, you have to adjust bridging and some parameters manually).

File gentoo.conf discussed above already contains the setup for coLinux. Next steps are as follows.

On hosting OS (Windows), configure TAP-Win32 Adapter V8 (coLinux) with following properties (make sure to leave Default Gateway as blank):

Ethernet adapter:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : TAP-Win32 Adapter V8 (coLinux)
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.0.1
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Default Gateway . . . . . . . . . :

We need to bridge the Internet connection with TAP interface. For example, assume you're using an ADSL PPPoE connection. In Windows host, select Control Panel -> Network Connections, right click the PPPoE interface, and select Properties. In tab Advanced, for Internet Connection Sharing, check Allow other network users to connect through this computer's Internet connection, and for the Home networking connection, select the TAP interface (e.g. Local Area Connection 2). When Windows prompts that TAP interface IP address will be set to 192.168.0.1, choose OK.

Under coLinux, edit file /etc/conf.d/net as follows.

config_eth0=( "192.168.0.20/24" )
routes_eth0=( "default via 192.168.0.1" )
dns_servers_eth0=(" xxx.xxx.xxx.xxx " )

Please change the DNS server information (shown as xxx above) to those obtained in windows (e.g. via ipconfig).

We have to emphasize here that in this setup, Linux guest has IP address 192.168.0.20 (this is arbitrarily selected, could be other numbers like 192.168.0.x when 2 <= x <= 254), and Windows host has IP address 192.168.0.1 interfacing Linux (of course Windows host has external IP addresses as well).

2.2.2 For two network interfaces

Note this part is NOT compatible with other sections. The purpose of this section is only to document alternatives to above solution. If one interface solution is fine for you, you can skip this section.

This configuration enables both SLiRP and TAP connections. SLiRP is used to connect outside world while TAP enables the high-speed private connection between Windows host and Linux guest, which is desirable for X protocol. The benefits of this configuration includes improved security and that SLiRP can automatically use current Windows Internet connection for outgoing connections (i.e. you do not need to enter DNS server information) without reconfigurations. This works fine for Wireless LANs, but I did meet problems for ADSL and finally decided to use TAP only.

Anyway, let's look at how to setup two network interfaces.

First, we need to modify the aforementioned file gentoo.conf located in coLinux directory. Change the line eth0=tuntap with two lines as below.

eth0=slirp
eth1=tuntap

Then edit file /etc/conf.d/net on Linux guest. This enables both SLiRP and TAP.

config_eth0=( "dhcp" )
config_eth1=( "192.168.37.20/24" )

We then need to setup DHCP client as following:

emerge dhcpcd
rc-update add dhcpcd default
/etc/init.d/dhcpcd start

Since only eth0 device is generated, we need to generate eth1 device by

cd /etc/init.d
ln -s net.lo net.eth1

On hosting OS (Windows), configure TAP-Win32 Adapter V8 (coLinux) with following properties (make sure to leave Default Gateway as blank):

Ethernet adapter:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : TAP-Win32 Adapter V8 (coLinux)
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.37.10
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Default Gateway . . . . . . . . . :
2.2.3 Proxy

If you connect to Internet directly, you can skip this section.

To use proxy server, add the following lines into /etc/env.d/99local:

http_proxy="http://proxy.server.com:8080"
ftp_proxy="http://proxy.server.com:8080"

Replace the port and domain name according to your situation. Then run:

env-update && source /etc/profile

2.3 Add additional locales

Modify file /etc/locale.gen, select appropriate locales. For me, the locales are:

en_US.UTF-8 UTF-8
zh_CN.UTF-8 UTF-8

Then run command locale-gen.

2.4 Mirror selection

First run emerge mirrorselect. To select the fastest mirror. I use the following command mirrorselect -D -H -s2 to select the mirror. It means that:

  • -D: actual file will be downloaded from the mirror to test speed.
  • -H: test http mirrors only.
  • -s2: only select 2 mirrors.

After running the command, original /etc/make.conf will be backed up as /etc/make.conf.backup and your selected mirrors will be reflected in make.conf. Type mirrorselect -h for details.

2.5 Modify /etc/make.conf

This is specific for your computer type and other factors. Following is my configuration (I'm using a Core 2 Duo CPU on 32 bit Windows). Please refer to Gentoo documentation for your own reference. GENTOO_MIRRORS reflects the fastest mirrors for my connection, so you may skip that line if your have already performed mirrorselect above.

CFLAGS="-march=prescott -O2 -fomit-frame-pointer -pipe -fno-ident"
CXXFLAGS="${CFLAGS}"
CHOST="i686-pc-linux-gnu"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
ACCEPT_KEYWORDS="x86"
FEATURES="parallel-fetch"
LINGUAS="en zh"
USE="-* auctex bash-completion bzip2 cjk crypt curl doc emacs gd gif glibc-omitfp gtk \
     javascript jpeg mmx mozilla ncurses nls nptl nptlonly opengl pam png readline \
     ruby sbcl source spell sse sse2 ssl svg tcpd tiff truetype \
     unicode X xft xml xorg xpm zlib"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.kaist.ac.kr/pub/gentoo/"

2.6 Update system

Since our original system is somehow outdated, we need to update it. First run emerge --sync or emerge-webrsync -v to synchronize portage, then run the following suites of commands (we need the -p argument to check what the commands are supposed to do since they're somehow "dangerous"):

emerge -uDNpv world
emerge -uDNv world
emerge --depclean -pv 
emerge --depclean -v
revdep-rebuild -pv
revdep-rebuild -v

Caveat: if you encounter circular dependency when emerge -uDNv world, try to remove gtk USE flag from /etc/make.conf. After successful emerging, add gtk flag again and run emerge -uDNv world again.

After update, run dispatch-conf to update the configuration files (you need to avoid the update of /bin/setfont. If you accidentally updated the file, you can remedy the situation as discussed below):

Caveat: if during booting, you meet the error message putfont: PIO_FONT: Function not supported, Solve the problem as follows (as discussed here):

First, run command mv /bin/setfont /bin/setfont.old

Create a new file /bin/setfont with the following content:

#!/bin/bash

if ! uname -r | grep -q -e "-co-"
then
        /bin/setfont.old $@
fi

and make it executable by running chmod +x /bin/setfont

3. Run X

X.org is already installed in coLinux machine when we update the system as discussed in section 2.6 (we have include X USE flag). Therefore the remaining steps are just to install X server in Windows host.

We choose Xming considering it is lightweight and widely used. To install Xming, Download Xming installer, font installer and Xming-tools-and-clients. You may use Xming-mesa installer to replace Xming installer for stable OpenGL operation (only do so if you meet problems with Xming installer).

My settings during installation of Xming are as follows:

  • Xming
    • Destination directory: changed to c:\bin\Xming.
  • Xming-fonts
    • Destination directory: changed to the same directory as Xming (c:\bin\Xming)
    • Fonts to be installed: select according to your own taste.
  • Xming-tools-and-clients
    • extract the zip file of Xming-tools-and-clients to the directory where Xming is installed (c:\bin\Xming in this case).

Settings on Windows host

For file x0.hosts in Xming directory, make sure the IP address of coLinux machines (192.168.0.20) is added.

If you want to run the tools and clients bundled with Xming in Windows host, you need to type set DISPLAY=127.0.0.1:0 in your Windows host.

Following steps are used to enable Windows fonts to be used by X application. This step is optional.

Open a command prompt window, change to directory c:\bin\Xming, and type the following commands:

mkfontscale C:/WINDOWS/Fonts
mkfontscale -b -s -l C:/WINDOWS/Fonts

Make sure that directory C:\Windows\Fonts appeared in the file font-dirs in Xming directory.

On coLinux machine

Add the following line into /etc/profile.

export DISPLAY=192.168.0.1:0

After start Xming (preferably to start it automatically when Windows start), you can simply invoke any command inside Linux console like:

xterm &

4. Miscellaneous

Change host name

According to Gentoo Handbook.

Edit file /etc/conf.d/hostname, if you call your coLinux machine as tux, then set the following line.

HOSTNAME="tux"

Edit file /etc/conf.d/net, if you call your domain as homenetwork, then set the following line.

dns_domain_lo="homenetwork"

Remove string .\O from file /etc/issue to get rid of the annoying unknown_domain in the welcome message "This is host.unknown_domain ...".

Create regular user accounts

With the following command, I created a new user named foo as a powerful user (with the ability to su to root and also invoke some portage related commands). Pleas substitute foo above with your desired user name.

useradd foo -m -G users,wheel,portage
passwd foo

Access Windows folder

When we edit gentoo.conf above, we have already enabled coLinux to share folder c:\linux. To make our regular user to access this directory in Linux, edit file /etc/fstab and add the following line:

cofs00  /home/foo/windows  cofs  uid=foo,gid=foo,dmask=0755,fmask=0644 0 0

The above line will enable the Windows folder to be shared in Linux as a folder /home/foo/windows. You can modify above line for your own user.

Additional programs

Following is a list of commands to install my own favorite programs. Your situation would be of course different.

# Some system utilities
emerge gentoolkit slocate

# Development tools
echo "dev-lang/erlang hipe" >> /etc/portage/package.use
emerge aspell ruby sbcl erlang dev-util/git maxima

# Gfx (GLEidescope is one of my favorite XScreenSaver)
emerge inkscape blender xscreensaver

# Netowork
emerge mozilla-firefox

# X tools
emerge xlsfonts xfontsel

Before emerge texlive, I have to add a bunch of keywords for dependencies in /etc/portage/package.keywords as follows.

app-text/texlive ~x86
app-text/texlive-core ~x86
dev-texlive/texlive-metapost ~x86
dev-texlive/texlive-basic ~x86
dev-texlive/texlive-documentation-base ~x86
dev-texlive/texlive-psutils ~x86
dev-texlive/texlive-langcjk ~x86
dev-tex/cjk-latex ~x86
dev-texlive/texlive-documentation-english ~x86
dev-texlive/texlive-genericrecommended ~x86
dev-texlive/texlive-latexrecommended ~x86
dev-texlive/texlive-latex ~x86
dev-texlive/texlive-texinfo ~x86
dev-texlive/texlive-latex3 ~x86
dev-texlive/texlive-htmlxml ~x86
app-text/jadetex ~x86
dev-texlive/texlive-fontsrecommended ~x86
app-text/dvipdfm ~x86
dev-texlive/texlive-documentation-chinese ~x86
dev-texlive/texlive-langmanju ~x86

Use CVS version of Emacs

Run the following commands:

echo "app-editors/emacs-cvs ~x86" >> /etc/portage/package.keywords
emerge emacs-cvs

If you want to use this bleeding-edge version only, you can uninstall the previous version of Emacs (if you have installed) and type:

echo "virtual/emacs ~x86" >> /etc/portage/package.keywords

Otherwise (i.e. you have multiple versions of Emacs), when you emerge packages requiring Emacs, a stable version (e.g. v22.1) will be installed. For the multiple-version-installation, you can use eselect emacs list to see a list of available version. To use Emacs 23 just installed, you can type eselect emacs set emacs-23. For details, please consult here.

Emacs extensions

I run the following commands to install a few Emacs extensions. Note that in Gentoo, many extensions (e.g. those related to development) are automatically installed for you when you emerge the related tools and enable emacs USE flag.

echo "app-emacs/gentoo-syntax ~x86" >> /etc/portage/package.keywords
echo "app-emacs/imaxima ~x86" >> /etc/portage/package.keywords
emerge htmlize imaxima app-emacs/gentoo-syntax keywiz

Enable SSH server

You may want to enable SSH access to your coLinux machine. There are some benefits. Examples are shown below (assuming PuTTY is used):

  • More flexibilities to configure the client (e.g. pretty fonts) and nice scrollbars.
  • The title of PuTTY indicates what coLinux is doing (such information might be useful during a long emerge process).

To start SSH server, type the following commands.

rc-update add sshd default
/etc/init.d/sshd start

When you use PuTTY client, you should fill 192.168.0.20 as the host address to utilize the high speed internal connection.

GTK configrations

Change the default GTK+ font. Edit file ~/.gtkrc-2.0, add the following line, and change the font name and size to your preferred one.

gtk-font-name = "Arial 15"

Mount CD

Simply type the following

# mount -t iso9660 -o ro /dev/hdc /mnt/cdrom

Final Words

coLinux is an interesting way to run Linux applications in Windows host. One drawback of coLinux is that the daemon process is pinned to one processor only. This avoids some bizarre issues, however it does not utilize the potentials of modern multi-core systems. This is disappointing especially for a Gentoo user who requires lots of computation power. Hope the situation could be improved in the future.

Update: please refer this post on how to enable sound.

Saturday, January 12, 2008

Gentoo's Crisis

Sad to learn Gentoo's crisis from Slashdot. I unsubscribed from Gentoo newsletter several months ago (even the newsletter itself has not been updated for some time), and I do not have any clue on this crisis. But according to my knowledge, there is no Gentoo 2007.1 release, which is a dangerous signal. Although the periodical release in Gentoo itself is not as important as for other distributions (actually there is no impact for users who have already installed Gentoo), bleeding-edge release is a must for newcomers.

As a fan of Gentoo, I do hope that Robbins's offer will be accepted and all the issues will be sorted out quickly.

Friday, December 28, 2007

Install Proggy Fonts for Emacs in Gentoo

Coding horror discussed various nice programming fonts (see here and here). Intrigued by these posts, I decided to try these fonts on my Gentoo box. Following is a record of what I have done.

Install Fonts in Gentoo

I decided to use Proggy Clean font (website is here). This section is mainly related to this particular font, but I will discuss at the end of this section how to handle other fonts in the same site.

Instead of manually installing the font, I determined to writer one ebuild. Main rationale is to rely on font eclass to automate font installation work. The ebuild could be easily reused and even incorporate into Gentoo portage (hopefully). The other reason is that I want to practice a little bit on portage overlay and to write my first (trivial) ebuild.

Create portage overlay

If you have already setup a overlay, you may skip this part. However it should be noted that all the description in this post assumes that your overlay is located in /usr/local/portage.

Run the following commands.

# mkdir -p /usr/local/portage
# echo 'PORTDIR_OVERLAY="/usr/local/portage"' >> /etc/make.conf    

Create the ebuild

First, create the directories.

mkdir -p /usr/local/portage/media-fonts/ttf-proggy-clean

Next, create a file named ttf-proggy-clean-1.0.ebuild under the directory just created. Fill the contents as below.

inherit font

DESCRIPTION="Proggy Clean TTF font"
HOMEPAGE="http://www.proggyfonts.com/"
SRC_URI="http://www.proggyfonts.com/download/download_bridge.php?get=ProggyClean.ttf.zip"

LICENSE="Proggy"
SLOT="0"
KEYWORDS="alpha amd64 arm ia64 ppc s390 sh sparc x86"
IUSE=""

DEPEND=""

S="${WORKDIR}"
FONT_S=${S}
FONT_SUFFIX="ttf"
DOCS="Licence.txt Readme.txt"

RESTRICT="mirror" 

Some explanations of the ebuild:

  • Version: I believe the font is quite stable, therefore version 1.0 should be OK :-).
  • License: the license in the package is quite permissive, but since I cannot link it to any existing license, I category it simply as Proggy.

Run the following command to sign the ebuild.

# ebuild /usr/local/portage/media-fonts/ttf-proggy-clean/ttf-proggy-clean-1.0.ebuild digest

BTW, I refers a lot to this when writing this first ebuild (of course, the skeleton is from man font.eclass).

Install font

Now comes our familiar emerge.

# emerge media-fonts/ttf-proggy-clean

Now we need to add the font path. Open file /etc/X11/xorg.conf, find the section Section "Files", and add the following line within the section:

  FontPath "/usr/share/fonts/ttf-proggy-clean"

Then press Ctrl-Alt-Backspace to restart X server. After login again, you may issue the following command (if you have not installed xlsfonts, please do so by typing emerge xlsfonts).

# xlsfonts | grep proggy

You should see several lines looking like -altsys-proggycleantt.... If not, there is something wrong to fix (please check previous typings).

Caveat: before I add the FontPath line in section Files, that section is actually empty! Therefore X simply cannot restart. The solution is to add some default paths into that section, e.g. /usr/share/fonts/100dpi, /usr/share/fonts/misc.

How to use other fonts

Following is a guideline for other fonts listed in the same website.

  • Use appropriate directory and file name to represent the font. For example, for Proggy Square X font, one might setup directory /usr/local/portage/media-fonts/proggy-square, and create a ebuild called proggy-square-1.0.ebuild. The directory should be added into /etc/X11/xorg.conf as described above.
  • Modify the ebuild:
    • Specify appropriate DESCRIPTION.
    • Use correct SRC_URI.
    • Specify DOCS correctly. I found that X font package does not contain Readme.txt.
    • Specify FONT_SUFFIX. For X fonts, use gz; use ttf (like above) for TTF fonts.

Configure Emacs

Compared with the installation described above, this part is rather easy. Assume that you want to use the font -altsys-proggycleantt-medium-r-normal--0-0-0-0-m-0-iso8859-1 (you may select one from the output of xlsfonts as described above)d, simply add the following line in your .emacs:

(set-default-font "-altsys-proggycleantt-medium-r-normal--0-0-0-0-m-0-iso8859-1")

Friday, December 7, 2007

Nicer Fonts for Gnuplot

Gnuplot is a great visualization tool to use. However the default font is not so pleasant for me. Following is the record on my adventure for alternative fonts. The description is mainly Gentoo oriented.

Which font to use?

I'd like to try TTF font for nicer looking, and Bitstream Vera bundled with Gnome just fits my taste. In case that you have not installed Gnome (e.g. you're pretty much satisfied with the plain terminal, or you use X only without any desktop environment, or you are a fan of other desktop environments like KDE or E17), you may install the font with one command in Gentoo: emerge ttf-bitstream-vera.

Build Gnuplot

Gnuplot should be built with libgd support. To do so, you should enable the USE flag "gd". You may add gd to your make.conf, or use the following command to add it for gnuplot alone:

echo "sci-visualization/gnuplot gd" >> /etc/portage/package.use

After that, build gnuplot by typing emerge gnuplot. The version I'm using for this post is 4.2.2.

Configure environment variable

Add the following line to one of the shell profiles (e.g. ~/.bash_profile):

export GDFONTPATH=/usr/share/fonts/ttf-bitstream-vera

Try it out

Now launch gnuplot. To test whether our settings take effect, type the following at gnuplot prompt:

gnuplot> set term png enhanced font "Vera,12"
Terminal type set to 'png'
Options are 'nocrop enhanced font Vera 12 '
gnuplot> set output "test.png"
gnuplot> plot sin(x)
gnuplot> quit

Now start your favorite photo viewer to examine test.png (I use Emacs in this case). Does the figure looks prettier than before? (Note: it seems that the font name is case sensitive and should be consistent with the actual file name of the font: "Vera" works in above example while "vera" not.)

One limitation is that TTF fonts are not available to X11 terminal therefore you cannot directly enjoy the effect within gnuplot. Emacs users using gnuplot-mode might not feel too much switching pain in this case. Another solution is: if you have ImageMagick installed, you can use png terminal, and pipe the output to ImageMagick by typing set output '|display png:-' instead of plain test.png as shown above.

Wednesday, November 7, 2007

Is Gentoo Linux the Choice for Minimalists?

Merriam-Webster defines minimalist as: one who favors restricting the functions and powers of a political organization or the achievement of a set of goals to a minimum. I tends to believe that myself is a minimalist, especially in the areas of computer usage. So what should be my choice for Linux distribution?

There are hundreds of Linux distributions listed in distrowatch while my favorite is Gentoo. To discuss whether it is suitable for minimalists, we need one example for comparison. Here I will choose popular Ubuntu, without any intention to start a flamewar between the two distributions.

If one is the minimalist in terms of installation procedure, Ubuntu seems to be a better choice: the setup procedure may take less than half an hour on modern systems, and during setup, one only needs to make a few selections. Gentoo, on the contrary, requires a rather long process: if you perform a stage 1/3 install, you need to type many commands, setup some configuration files, and modify other configuration files; you need a decent Internet connection, and the compilation process might be quite long (if not longer than LFS). On a not-so-modern setup, to build a system with similar functionalities as what Ubuntu offered, you may need several days. So why bother?

The reasons are:

  • Minimal Components: you can build a minimal system out of your own taste with Gentoo: you only need to install programs you like to use (and their dependencies of course, note that you also have some freedom to choose which dependencies to install by control the USE flag), you also specify the minimum number of services to start at boot time. In the end, you have setup your own system with minimal components.
  • Minimal Upgrade: The upgrade system is also minimal. After installation, Gentoo is versionless. For example, you may use your Gentoo box for a long time without any upgrading. But one day, you may be tempted to upgrade one program, say Firefox, since the new features attract you. Then after synchronization of portage, one simple emerge command can bring this particular program up to date (this may update some dependencies or install new dependencies if needed).
  • Minimal Security Risk: you are less prone to security problems since you have less programs installed.

In summary, the installation for Gentoo can not be labelled as minimal, however this is only needed once for one computer. After installation, you can enjoy all the other benefits from Gentoo. The customization capability of Gentoo tends to encourage me to link it with Emacs: if you like Emacs and want to customize everything, Gentoo might be your favorite distribution.

Tuesday, October 9, 2007

Tag Local Files

Nowadays we use tags for almost every online service: blogs, photos, videos etc. They are quite helpful for organizing contents. How about our local files? We still need them for various reasons e.g. archive, privacy, security and so on. I'm going to discuss a simple way to tag local files. We assume you are using a Unix-like system (Linux in the example), where soft link is an inherent system feature.

The idea is quite simple: we setup directory ~/tags and make a subdirectory for each tag. For every file/directory with tags, one soft link to the original file is put under each tag subdirectory. To make tags, we use a command called tag which has the following synopsis:

tag src tag-name...

where src is the path to the original file/directory, and tag-name... are the list of tags to be applied delimited with spaces.

For example, assume there is a image file ~/photo/wonderful.jpg. When you change to directory ~/photo and type the command

tag wonderful.jpg france paris 2007 october

(assume that the image is taken in Paris in October 2007), there will be four directories created: ~/tags/france, ~/tags/paris, ~/tags/2007, ~/tags/october, and each directory contains a soft link to the original file.

Following is a snippet of the content of script tag.

#!/usr/bin/env bash

src=$1
dir=`pwd`
shift 1
for x in "$@"
do
  dst=~/tags/${x}/
  mkdir -p $dst
  ln -s $dir\/$src $dst
done

In the above script, we create directory for each tag and then make soft link within that directory.

To install this script, simply make it executable (by typing command chmod 755 tag), and copy it into a directory included in your search path (e.g. /usr/local/bin, you might need to su to root to gain access for these system directories).

In addition to the operation of adding tags as shown above, you might wonder how can we handle other operations on tags, e.g. deletion, search, and renaming. Fortunately, we don't need additional scripts any more. The reason is that tags are themselves ordinary directories, therefore we can use normal console commands to perform these tasks (e.g. rm for tag deletion, find for search and mv for renaming).

Enjoy organizing your local files with tags!

Update (October 10, 2007) As pointed out by aspir8or: The original script creates links whose paths are missing. He also provides a correction to the issue. The current script is based on his update, with slight changes purely from the author's own preference on variable naming.