Wednesday, December 24, 2008

Adjusting Photos for Web Albums in Batch Mode

I use Picasa Web Album to store my photos online. To save the precious storage at server, I have to change jpeg files before uploading them even for those taken by my old PowerShot S30 (which outputs photos with resolution 2048x1536, which is beyond the display capacity of most monitors today). To reduce file size, one normally need to reduce both resolution and image quality.

So how to adjust images in batch mode? There are various solutions of course. The one I take is to use the combination of find/xargs/mogrify (Note that mogrify modifies files in place. Therefore one may use convert instead of mogrify to preserve original files.) in Linux. Note that mogrify is one of ImageMagick utilities and may need separate installation depending on distribution. Following is my work flow when publishing images online.

Firstly, I select images with my preferred viewer gthumb. Images are rotated if necessary. During browsing, I copy the chosen files to another folder and run the following command in that folder:

find . -iname "*.jpg" | xargs mogrify -resize resize-option -quality quality-option

(I find the above command from here.)

Several notes regarding the above command:

  • resize-option could be either absolute (e.g. 800x600) or relative (e.g. 50%). For my S30, I use the latter which reduce the photo resolution to 1/4 of its original size.
  • quality-option is JPEG compression level. The larger the value, the better the quality, but the larger the file size. I often set the value as 60 to save space.

So for photos from my S30, I use the following command:

find . -iname "*.jpg" | xargs mogrify -resize 50% -quality 60

After running above command, photo file size can be reduced around 10 times. This means that I can put more images into my web album.

Wednesday, August 20, 2008

W(S)XGA+ Resolution in VirtualBox

Note: this post is related to a workaround for one specific problem encountered by the author.

Recently I switched from coLinux to VirtualBox. VirtualBox provides guest addition which includes enhanced support for Video. Originally it worked fine for me. Due to unknown reason, the problem arises: I could not use resolution 1440x900 (WXGA+) or 1680x1050 (WSXGA+) any more. WXGA+ is the native resolution of my laptop, while WSXGA+ is the resolution of the external LCD when the laptop is placed in a dock. After tweaking for a long time, following is the solution I found so far (I installed Gentoo Linux as guest OS).

1. Revert video driver from VirtualBox to Vesa

This is done by editing /etc/X11/xorg.conf, change one line in Section Device from Driver "vboxvideo" to Driver "vesa".

2. Install support of 1440x900 and 1680x1050 to VESA BIOS

Add parameter vga=864 as Linux bootup parameter (e.g. in /boot/grub/grub.conf) for resolution 1440x900, or vga=865 for 1680x1050.

When Virtual machine is turned off, run the following two commands at Windows command line prompt, assuming virtual machine is named "Gentoo":

VBoxManage setextradata "Gentoo" "CustomeVideoMode1" "1440x900x24"
VBoxManage setextradata "Gentoo" "CustomeVideoMode2" "1680x1050x24"

Start virtual machine, select resolution 1440x900 or 1680x1050 for console display. Note that the resolution for console is not necessarily the same resolution for X, i.e. you can use 1440x900 for console, and 1680x1050 for X.

3. Add modelines for resolution 1440x900 and 1680x1050.

I use gtf to generate needed modelines. For example for resolution 1440x900 with 60 Hz refresh rate, I use the following command:

$ gtf 1440 900 60

which give the following line:

  Modeline "1440x900_60.00"  106.47  1440 1520 1672 1904  900 901 904 932  -HSync +Vsync

I do not like the suffix _60.00, therefore what finally end up in section Monitor of /etc/X11/xorg.conf is (with addition of modeline for 1680x1050):

 modeline "1440x900" 106.47 1440 1520 1672 1904 900 901 904 932 -HSync +Vsync
 modeline  "1680x1050"  147.14  1680 1784 1968 2256  1050 1051 1054 1087  -HSync +Vsync

In addition, the two modes should be added in section Screen of /etc/X11/xorg.conf as well, my current setting is:

Section "Screen"
 Identifier "Screen0"
 Device     "Card0"
 Monitor    "Monitor0"
 DefaultDepth     24
 SubSection "Display"
  Viewport   0 0
  Depth     24
  Modes    "1680x1050" "1440x900"
 EndSubSection
EndSection

4. Select the resolution

Now the toughest part. You may have noticed that in section Monitor, there are two lines about HorizSync and VertRefresh. After experimenting, my conclusion is that one should comment out them when using 1680x1050 while enable them for 1440x900. Weird.

Since I need to switch between these two resolutions frequently, manually editing file /etc/X11/xorg.conf is boring. Therefore I write one script to automate the task. I named the script as xres and invoke it as sudo xres 1440 for resolution 1440x900 and sudo xres 1680 for 1680x1050. Following is its content:

#!/bin/bash
case $1 in
    1440)
        sed -i 's/^#    HorizSync    31.5 - 64.3/       HorizSync    31.5 - 64.3/' /etc/X11/xorg.conf
        sed -i 's/^#    VertRefresh  50.0 - 70.0/       VertRefresh  50.0 - 70.0/' /etc/X11/xorg.conf
        ;;
    1680)
        sed -i 's/^     HorizSync    31.5 - 64.3/#      HorizSync    31.5 - 64.3/' /etc/X11/xorg.conf
        sed -i 's/^     VertRefresh  50.0 - 70.0/#      VertRefresh  50.0 - 70.0/' /etc/X11/xorg.conf
        ;;
    *)
        echo "Supported horizontal resolution: 1440 and 1680."
        exit
        ;;
esac

Thursday, July 24, 2008

Experiencing Stumpwm

What Emacs is to editors, Stumpwm is to window managers.
--Bill Clementson (link)

Introduction

Recently I switched from GNOME to Stumpwm (wiki), which means that I jumped out of a desktop environment to a simple window manager. So far, I'm very satisfied with such a change and never looked back. Stumpwm, as its website says, is a tiling, keyboard driven X11 Window Manager written entirely in Common Lisp. This definition summarizes the reasons why Stumpwm is so suitable for me.

First, Stumpwm is a tiling, keyboard driven window manager. This is very useful to boost productivity. When working, it is desirable to minimize context switching. Tilting combined with keyboard driven enables working with several applications simultaneously as if your are dealing with one single program. Let me give an example. As described in previous post, with Emacs as development environment, it is easy to fire up one browser within Emacs itself to preview blog post. However it is painful to switch between Emacs and browser. Before using Stumpwm, I am considering ways to integrate Emacs and Firefox together. With Stumpwm, such dream is just trivial to fulfill: one can show Emacs and browser side by side, with a few keystrokes to switch between them. Everytime you make changes in Emacs and request the results to be shown in Firefox, they are shown simultaneously. No more need to leave your hand for mouse to click back and forth between Emacs and Firefox. This gives you the feeling of integrated environment. Note that I just use Emacs and Firefox as an example, and such convenience is applicable for every application. Currently, the problem is that I'm so accustomed to Stumpwm keystrokes that I somehow confused the keystrokes of switching between windows in Stumpwm and Emacs!

Second, Stumpwm is written in Common Lisp. which means that you can build Stumpwm with your favorite CL implementation and you have a Lisp runtime when you work under Stumpwm. A powerful programming language is just embedded within your window manager and you can invoke it at any time. The other benefit is that you can configure your window manager as you like, even when it is running!

You may get a feeling on how Stumpwm works by watching this nice video.

Installation

Given the above lengthy introduction of Stumpwm, you may wonder how to install this gem. Here we assume that you have some knowledge about Common Lisp. Basically, installing Stumpwm is to compile it with your preferred CL implementation, and tell your system to run Stumpwm as your window manager.

Installing Stumpwm on Gentoo Linux is straightforward. Throughout this section, we will assume that SBCL is the CL implementation. Stumpwm website suggests turn off threading support in SBCL (disable USE flag threads) for better performance. To install, simply type emerge stumpwm. Note that at the time of writing, there is also an ebuild called stumpwm-cvs. Simply ignore it since it is actually a very old version, not the bleeding edge version suggested by its name. Advanced users might consider to get git version for latest cool features.

There are mainly two ways to invoke Stumpwm. One is to run it by calling SBCL, the other is to dump a core image containing Stumpwm within SBCL and invoke that image. We will take the latter approach. To proceed, first start SBCL by typing sbcl. Next issue the following in REPL sequentially:

(asdf:oos 'asdf:load-op :stumpwm)
(sb-ext:save-lisp-and-die "stumpwm" :executable t
                          :toplevel #'(lambda () (stumpwm:stumpwm ":0")))

Next, put the generated executable stumpwm somewhere in PATH (I put it under /usr/local/bin). Since I always start X Window by typing startx, following command is used to use Stumpwm as my window manager:

$ echo "exec stumpwm" >> ~/.xinitrc

Using Stumpwm

Configuration

All stumpwm configurations are stored in file ~/.stumpwmrc, which is written in Common Lisp. Just like .emacs, this file allows you to fully customize Stumpwm. Following is my current configuration:

;;;; -*- Mode: Lisp -*-

(in-package :stumpwm)

;; Load swank.
(load "/usr/share/emacs/site-lisp/slime/swank-loader.lisp")
(swank-loader:init)
(define-stumpwm-command "swank" ()
  (setf stumpwm:*top-level-error-action* :break)
  (swank:create-server :port 4005
                       :style swank:*communication-style*
                       :dont-close t)
  (echo-string (current-screen) "Starting swank."))
(define-key *root-map* (kbd "C-s") "swank")       

;; Customize bars and modeline.
(setf *message-window-gravity* :center)
(setf *input-window-gravity* :center)
;; Turn on mode line.
(toggle-mode-line (current-screen) (current-head))
(setf *screen-mode-line-format* 
      (list "%w | "
            '(:eval (run-shell-command "date | tr -d '[:cntrl:]'" t))))

(set-prefix-key (kbd "C-i"))
(define-key *root-map* (kbd "c") 
  "exec urxvt +sb -fn \"xft:Bitstream Vera Sans Mono:pixelsize=20\"")

(define-stumpwm-command "firefox" ()
  "Run or switch to firefox."
  (run-or-raise "firefox" '(:class "Firefox")))
(define-key *root-map* (kbd "f") "firefox")

Some explanations of my configuration:

  • Prefix key: I use C-i instead of default C-t. The reason is that C-t is used in Firefox to open a new tab and also in Emacs for transpose. Then why choose C-i? I'd like to admit that it is quite difficult to select a prefix key for Emacs user. Before settle down on C-i, I fired up Emacs to see whether there is any key binding C-x where x from a to z is not used by Emacs. Unfortunately (fortunately?), Emacs binds every combination. So I can only choose one prefix key which is easy to type and I uses infrequently in Emacs. Then C-i is selected. Note that you can send C-i to application like Emacs by typing C-i i. For the following sections, please replace C-i with your favorite prefix key.
  • SLIME: the section staring with comments Load swank provides ways to load swank. Type C-i C-s to start swank. To connect to swank, simply run Emacs, and type slime-connect within Emacs, and type RET and RET to accept default host (127.0.0.1) and default port (4005). Then you can play with Stumpwm as you wish: change parameters, add your own functions etc. Note that I do not start swank automatically for security reasons.
  • Key bindings: I use C-i c to start console: urxvt instead of xterm. Note that I have also set the font for urxvt. In addition I have setup using C-i f to start Firefox in case Firefox is not started, or bring Firefox window to front if it is already running.

Key Bindings

Following is a list of key bindings I used frequently. For simplicity, I have omitted the prefix key.

  • ?: Stumpwm help
  • ;: Run Stumpwm commands
  • :: Send commands to Common Lisp interpreter.
  • Space: Go to next window
  • c: Run X terminal
  • e: Run Emacs or raise it if it is already running
  • f: Run Firefox or raise it if it is already running
  • k: Kill current window
  • g c: Create a new group
  • g k: Kill current group
  • g m: Move current window to a specified group
  • g Space: Next group
  • o: Focus shifts to next frame
  • Q: Remove all splits
  • s: Vertical split
  • S: Horizontal split

Wednesday, May 21, 2008

Using PulseAudio for OSS Applications

In my previous post, I discussed how to use PulseAudio to enable sound for coLinux. This works fine for programs like MPlayer, but has issues with some programs e.g. Solfege.

The solution is simple: if the problematic applications have support for OSS, then emerge them with OSS USE flag on. In addition, PulseAudio should also be emerged with OSS USE flag (to enable the building of padsp, a PulseAudio OSS wrapper, which will be discussed below). Under Gentoo Linux, one way is to simply add OSS USE flag to /etc/make.conf. Alternative way is to enable the USE flag application by application. For example, to enable OSS USE flag for Solfege, type the following:

# echo "media-sound/solfege oss" >> /etc/portage/package.use

When running these applications, prefix the command line with padsp like the following:

$ padsp solfege

Note that you may need additional configurations in the application to use OSS for audio output.

Monday, May 5, 2008

HV30: Camcorder and Accessories

I just bought Canon HV30 (Amazon, Review) recently for my baby. Before buying HV30, I was agonizing over the selection between HV30 and Canon HF10 (Amazon, Review) (although I know that HF100 is better than HF10 from money point of view since the internal 16GB flash drive of HF10 does not justify the price difference, I do like the black paint job of HF10 and HV30).

Comparison of HV30 vs. HF10

So why I chose HV30 over HF10? The following is my comparison.

FeaturesHV30HF10Comments
Price MSRP price is $999 in US and ¥10580.00 (about $1514) in China MSRP price is $1099 in US and ¥14080.00 (about $2014) in China The price difference in China is quite significant, which is true even for street price.
Performance 1/2.7 in. sensor 1/3.2 in. sensor According to the review, HV30 has slightly better performance over HF10 thanks to larger sensor, however the gain seems to be marginal for most cases.
Functionalities Viewfinder, Zebra, Peaking
Accessories Standard accessory shoe Canon proprietary accessory shoe
Backup Tapes are cheap for backup Additional backup is needed since SDHC cards are expensive and will be reused.
Community A mature community (looking at hv20.com for example) Situation will improve
Form Factor 88x82x138 mm (3.5x3.2x5.4 in.), 535 g (1.2 lb.) 73x64x129 mm (2.9x2.5x5.1 in.), 380 g (13.4 oz) HF10 is small and light, which is ideal for traveling. Normally the camcorder which you takes out often is the most useful one.
Media Tape SDHC card + internal Flash Flash card is really convenient: just like small DCs. No hassles for rewinding and no worry for overriding.
Zoom 10x optical zoom 12x optical zoom
Noise Tape noise can be heard in silent environment No such noise since flash card is used.

Accessories

What I have:

  • Battery and charger: compatible ones
  • Camera bag: Lowepro Edit 120+
  • CPL filter: Kenko 43mm
  • HDMI cable
  • IEEE 1394 cable: Belkin 4-pin to 4-pin
  • Lens cap
  • miniSD: Transcend 2G. Not sure whether 4G could be used (the manual only states that up to 2G has been tested).
  • Tape: Panasonic AY-DVM63PQ
  • UV filter: Kenko MC-UV 43mm

For future purchase:

  • 35mm adapter
  • Close-up lens: +1/+2/+4
  • Mic: Canon DM-50, or combination of Rode Videomic and stereo mic. Maybe XLR adapters with XLR Mics?
  • Neutral density filters: ND2/4/8
  • Shoulder strap
  • Steadicam Merlin
  • Telephoto lens: Canon TL-H43 (any black version?)
  • Tripod and fluid head: Manfrotto 190XPROB (tripod) + 701RC2 (fluid head) + MBAG70 (bag)
  • Video light: Canon VL-3
  • Wide angle lens: Canon WD-H43 (any black version?)
  • Wrist strap

Final Words

After over 1 month's usage of HV30, I'm pretty satisfied with it. There is a lot of things for me to learn: how to shoot, how to edit, just to name a few. But all the efforts are worthy when you view the video clips produced from it.

Friday, May 2, 2008

Automatic Backup

We need to backup our stuff in case that some disasters might happen. One way is to backup to some external medias like tape, USB drives, or CD/DVDs. However working with them is somehow tedious and manual intervention is needed. People tend to be lazy therefore we are reluctant to backup if the process is inconvenient. If we have one additional machine with link in between (throughout this article, we assume the remote machine is named as remotebox), we can setup an automatic backup mechanism.

Overview

We use crontab to schedule rsync jobs to backup our files to another machine. Since rsync is secured with SSH, to suppress the password prompt, we have to setup SSH keychain.

Setup SSH Keychain

Generate Keys

This section mainly follows this nice guide.

We use the DSA approach.

$ ssh-keygen -t dsa
$ scp ~/.ssh/id_dsa.pub user@remotebox

Then log in to remotebox and append the public key to ~/.ssh/authorized_keys file like so:

cat id_dsa.pub >> ~/.ssh/authorized_keys
rm id_dsa.pub

Install Keychain

Keychain is a nice package which prompt you for your passphrase once you log into the system, and automatically provides passwords for later SSH session. Note this incurs security risks (there are always tradeoffs between security and convenience) therefore attention is needed.

This part mainly follows this guide.

First install keychain by typing emerge keychain.

Then add following lines to your ~/.bash_profile.

/keychain id_dsa
. ~/.keychain/$HOSTNAME-sh

Setup Crontab Job

We can then setup our automatic backup system now. Since there are almost infinite number of combinations of crontab and rsync patterns, I just shown one example below (many interesting examples of rsync usage can be found by typing man rsync).

0 9,16 * * 1-5 rsync -az -e ssh --delete ~/work/ remotebox:~/backup/

The above example synchronizes local directory ~/work to remote directory ~/backup on host remotebox, on 9 a.m. and 4 p.m. of every working day. Note that I have used option --delete, which will try to delete extraneous files from destination directory. This can ensure that the destination directory is an exact copy of source directory but it would be dangerous if the destination directory contains some files already. So test this option before you actually use it.

Switching Browsers in Emacs

Recently, I have met one dilemma in Emacs. Since I'm using Emacs for my blog writing, Firefox is used for previewing. Also as an Common Lisp user, I use emacs-w3m to read HyperSpec. Here comes the problem: I can only set one browser for browse-url-browser-function. How can I suit my requirements simultaneously?

Fortunately, Emacs is highly extensible. I just write one simple interactive function toggle-browser, which can toggle between Firefox and emacs-w3m when pressing M-x toggle-browser. Following is the code:

(defun toggle-browser ()
  "Toggle browser between Firefox and emacs-w3m."
  (interactive)
  (setq browse-url-browser-function 
        (if (eql browse-url-browser-function 'browse-url-firefox)
            'w3m-browse-url
          'browse-url-firefox))
  (message "%s" browse-url-browser-function))

Update: please look at the 1st comment from pedro for a better solution.

Thursday, April 10, 2008

Using Imaxima with Texlive in Gentoo Linux

Imaxima is an Emacs extension to interface with Maxima (see here for a nice tutorial). To use it, one need a LaTeX distribution. However with default Texlive installation in Gentoo, imaxima cannot work. Following is a sample session:

(%i1) integrate(x,x);
LaTex error in: \ifracn{x^{2}}{2}

The problem here is that there is another package breqn needed. Breqn is inlcuded in Texlive package texlive-mathextra, which is not installed when you emerge texlive. The package is also masked by default, therefore you need to unmask it before emerging. Following is a list of commands to install the package (substitute x86 with amd64 for x86-64 systems).

# echo "dev-texlive/texlive-mathextra ~x86" >> /etc/portage/package.keywords
# emerge texlive-mathextra

After installation, fire up your Emacs and enjoy (I)maxima!

Update: if you are not comfortable with the default equation size (as in my case), you can add the following line into your .emacs (adjust the number to suit your own taste):

(setq imaxima-scale-factor 2.0)

Tuesday, April 8, 2008

Copy and Paste between Windows and Linux

Many people use Windows as primary working machine and login to remote Linux/Unix (we will use Linux as shorthand afterwards) boxes. There are times when we need to copy and paste information between Windows and Linux. Below we are going to discuss convenient ways to do so.

Tools

PuTTY and Xming are two popular programs to run in Windows host to facilitate communication to Linux. PuTTY provides console connection while Xming X Window connection. For Xming, Ensure you start Xming with parameter -clipboard (which is enabled by default). Our discussion below is focused on how to copy/paste within PuTTY/Xming. Copy/paste for other Windows applications is quite general (Ctrl+C and Ctrl+V) and should be familiar to the reader, therefore omitted.

Ways

PuTTY

Copy to clipboard: click-and-drag to highlight. Just select, no need to press any key.

Paste from clipboard: right mouse button.

Xming

For Xming applications, copy/paste follows the X window standard as discussed below.

Copy to clipboard: click-and-drag to highlight. Same as PuTTY.

Paste from clipboard: middle mouse button.

Monday, April 7, 2008

Installation of Gentoo Linux 2008.0 Beta 1 with LiveCD

I just started yet another process of installing Gentoo Linux, and following is my record of the installation step by step. Previously I preferred to install from stage 1/3 following this guide. I decided to try LiveCD this time.

1. Boot the machine.

Download the 2008.0 LiveCD from here. Burn the CD and boot the machine with it.

Caveat: in my case, the boot process is stalled. I solved it (according to this article) by appending kernel options during GRUB screen (press e to edit and append all-generic-ide irqpoll pci=nommconf).

2. Installation.

After boot, I click icon "Gentoo Linux Installation (GTK+)" in the desktop.

Partitioning: I selected Recommended Layout.

Local Mounts: accept the defaults without any changes.

Timezone: Asia/Shanghai.

Networking: Dhcp is used. Appropriate host and domain name are provided.

Users: Just add the regular user for daily usage.

Extra Packages: I selected screen and slocate. In my first installation I also selected xorg-x11, gdm, firefox etc. However the installation simply failed when emerging x11-apps/xinit (and I cannot rescue the system since even /etc/fstab is not correctly setup), therefore my second try is quite conservative.

Startup services: I selected sshd.

Other Settings: Although I have not installed X11 or Gnome yet, I selected as following:

  • Display Manager: gdm
  • Clock: local
  • XSession: Gnome

3. Post Installation.

This section is similar as what I have described for coLinux (see here), but is modified specifically for a standalone Gentoo Linux installation.

3.0 Tiny Cleanups.

After rebooting, perform the following steps. Note that if your system was stalled as described before, you can use same trick in GRUB screen and you should add the kernel options to /boot/grub/grub.conf after login.

To avoid some warning, you might need to perform following steps.

# mkdir -p /usr/local/portage

To change host name (according to Gentoo Handbook), edit file /etc/conf.d/hostname, if you call your coLinux machine as tux, then set the following line.

HOSTNAME="tux"

In addition, remove string .\O from file /etc/issue to get rid of the annoying unknown_domain in the welcome message "This is host.unknown_domain ...".

3.1 Network

We have already specified using DHCP. It works but the configuration files is using old syntax. Let's clean it up to avoid future complaints. Edit file /etc/conf.d/net as follows.

config_eth0=( "dhcp" )
3.1.1 Proxy

If you connect to Internet directly, you can skip this section.

To use proxy server, add the following lines into /etc/env.d/99local:

http_proxy="http://proxy.server.com:8080"
ftp_proxy="http://proxy.server.com:8080"

Replace the port and domain name according to your situation. Then run:

env-update && source /etc/profile

3.2 Add additional locales

Modify file /etc/locale.gen, select appropriate locales. For me, the locales are:

en_US.UTF-8 UTF-8
zh_CN.UTF-8 UTF-8

Then run command locale-gen.

3.3 Modify /etc/make.conf

This is specific for your computer type and other factors. Following is my configuration (I'm using a Core 2 Duo CPU). Please refer to Gentoo documentation for your own reference. GENTOO_MIRRORS reflects the fastest mirrors for my connection, as discussed below.

CFLAGS="-march=nocona -O2 -fomit-frame-pointer -pipe -fno-ident"
CXXFLAGS="${CFLAGS}"
CHOST="x86_64-pc-linux-gnu"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j3"
ACCEPT_KEYWORDS="amd64"
FEATURES="parallel-fetch"
LINGUAS="en zh zh_CN"
INPUT_DEVICES="keyboard mouse"
VIDEO_CARDS="i810"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.kaist.ac.kr/pub/gentoo/"

USE_DEV="bash-completion doc emacs latex ruby sbcl source spell"
USE_HW="cpudetection mmx sse sse2"
USE_LIB="curl gd glibc-omitfp ncurses readline zlib"
USE_MM="aac alsa encode ffmpeg flac midi mp3 ogg pulseaudio quicktime theora xvid"
USE_PIC="gif jpeg png raw svg tiff xpm"
USE_SECURITY="crypt pam ssl tcpd"
USE_SW="bzip2"
USE_SYS="cjk nls nptl nptlonly opengl truetype unicode xinerama xml"
USE_X11="cairo gnome gtk X xft xorg"
USE="-* ${USE_DEV} ${USE_HW} ${USE_LIB} ${USE_MM} \
     ${USE_PIC} ${USE_SECURITY} ${USE_SW} ${USE_SYS} ${USE_X11}"

3.4 Synchronize Portage

Use either emerge --sync or emerge-webrsync -v to update the portage.

3.5 Mirror selection

First run emerge mirrorselect. To select the fastest mirror. I use the following command mirrorselect -D -H -s2 to select the mirror. It means that:

  • -D: actual file will be downloaded from the mirror to test speed.
  • -H: test http mirrors only.
  • -s2: only select 2 mirrors.

After running the command, original /etc/make.conf will be backed up as /etc/make.conf.backup and your selected mirrors will be reflected in make.conf. Type mirrorselect -h for details.

3.6 Update system

Run the following suites of commands (we need the -p argument to check what the commands are supposed to do since they're somehow "dangerous"):

emerge -uDNpv world
emerge -uDNv world
emerge --depclean -pv 
emerge --depclean -v
revdep-rebuild -pv
revdep-rebuild -v

Caveat: if you encounter circular dependency when emerge -uDNv world, try to remove gtk USE flag from /etc/make.conf. After successful emerging, add gtk flag again and run emerge -uDNv world again.

After update, run dispatch-conf to update the configuration files.

3.7 Install X11 and GNOME.

Run command emerge xorg-x11 to install X11. Then use xorgcfg -textmode to configure X server.

I'm using a LCD with resolution of 1680x1050, which is not listed in standard configurations. To use it, first run gtf 1680 1050 60, copy the output to the Monitor section of /etc/X11/xorg.conf, and modify section Screen to the resolution 1680x1050.

For GNOME, I prefer to install gnome-light to avoid the gigantic full GNOME installation. To do so, simply run emerge gnome-light.

If one use startx to run X11, one can simply type echo "exec gnome-session" > ~/.xinitrc to configure so.

To make gdm the default login screen, run the following commands:

# emerge gdm
# rc-update add xdm default

And then edit /etc/conf.d/xdm, changing DISPLAYMANAGER to gdm.

3.8 Install Other programs.

This section is left empty intentionally :-)


Conclusion

It is quite convenient to install with LiveCD. There are some hassles during installation, but I assume the reason is that I'm using a beta version. Since beta version is not stable, my suggestion is to install minimal components therefore you can quickly finish the installation and boot from hard drive.

Although GRP install is not so optimal, the pre-compiled packages should be quite close to optimal ones for 64-bit machines. Also as a Gentoo user, I'm always in the process of customization the OS to suit my needs, therefore a not-so-ideal-initial-setup is not an issue. In summary, livecd approach can save a lot of time, with almost similar end result as installing from stage 1/3.

Ripping CD to FLAC in Gentoo Linux

This post is just a simple note, inspired from here.

First, install cdparanoia and flac:

# emerge cdparanoia flac

Next, issue the following commands (I deleted the original WAV files after conversion). All tracks are extracted and converted to FLAC format.

# cdparanoia -B
# find . -name "*.wav" -print0 | xargs -0 flac
# rm *.wav

Then enjoy the music with your favorite player. Since I'm using mplayer, I call Ruby to generate the playlist and invoke mplayer with it (in the 1st line below, the last track number is 16. Change it according to your own situation).

$ ruby -e '(0..16).map{|i| printf("track%02d.cdda.flac\n", i)}' > pl
$ mplayer -playlist pl

Wednesday, April 2, 2008

Enable Sound for coLinux

In my previous post about installing coLinux with Gentoo, audio part is not touched. Here comes the solution on how to use PulseAudio to bridge coLinux to sound world. This is adapted from here with contents modified for Gentoo Linux.

Windows side

We need to install PulseAudio server on Windows. Frist download binaries for Windows from here. Extract to one directory (c:\bin\pulseaudio in my case).

In the directory of the binaries, create a text config file named default.pa with the content:

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24
load-module module-esound-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24
load-module module-detect

We start PulseAudio automatically when coLinux starts, by adding the following line to the coLinux configuration file:

exec0=c:\bin\pulseaudio\pulseaudio.exe

Linux side

Install and configure PulseAudio client

Install PulseAudio by emerge pulseaudio. Then edit /etc/pulse/client.conf to set the address of the Windows host that runs the PulseAudio server:

default-server = 192.168.0.1

where 192.168.0.1 is the IP address of the Windows host.

Make sure to add USE flag alsa to /etc/make.conf. If you just add this USE flag, update your system with emerge -uDNv world. ALSA would be automatically emerged.

Install alsa-plugins

In addition, we should install alsa-plugins:

# echo "media-plugins/alsa-plugins pulseaudio" >> /etc/portage/package.use
# emerge alsa-plugins

Edit ~/.asoundrc with the following contents.

pcm.!default {
    type pulse
}

ctl.!default {
    type pulse
}

pcm.pulse {
    type pulse
}

ctl.pulse {
    type pulse
}

Test

As a final step, you can test with an ALSA player:

aplay /usr/share/sounds/alsa/Front_Center.wav

You should hear a voice saying "Front Center". Enjoy!

Sunday, March 16, 2008

Install Gentoo with coLinux

coLinux is gaining momentum as a way to run Linux programs on Windows, and I decided to try it. As a user of Gentoo, my first choice is to use Gentoo with coLinux. The following description documents what I have done to run Gentoo cooperatively in Windows laptop. I mainly follows up the Gentoo Wiki Howto article and other coLinux resources.

1. Installation

First download the latest coLinux snapshot (you can also get it from SourceForge). The file I used is coLinux-0.7.2.exe from SourceForge. Run the installer. Some notes during installation:

  • In screen Choose Components, leave the default settings on (i.e. selecting all components), but uncheck Root FileSystem image download (we will download it separately).
  • In addition, change the Destination folder to c:\bin\coLinux (note that all the description below assumes coLinux is installed into this directory. If you prefer other directory, please modify accordingly in steps below).
  • Installer prompts to download and install WinPCap separately, but my network configuration does not need WinPCap therefore I don't install it.

Then download Gentoo-colinux-i686-2007-03-03.7z from SourceForge, extract the archive into c:\bin\coLinux, with a gentoo subdirectory created during extraction. Copy file gentoo.conf in gentoo to the parent directory and modify it as follows:

kernel=vmlinux
cobd0=C:\bin\coLinux\gentoo\gentoo-root
cobd1=C:\bin\coLinux\gentoo\linux-swap
root=/dev/cobd0
hdc=\Device\Cdrom0
initrd=initrd.gz
mem=1024
eth0=tuntap
cofs0=c:\linux

Some explanations of the above file, details will be covered in later sections.

  • For mem, I set it to 1G. Choose according to your own situation.
  • For network, I only select TAP driver.
  • I shared Windows folder c:\linux to coLinux.
  • CD/DVD-Drive is mounted.

To test drive, open a command prompt, change to the directory c:\bin\coLinux, and run the following command:

colinux-daemon @gentoo.conf

coLinux console should run. If there is something wrong, please check previous steps. After some time, you are prompted for log in. You can now login as root with password. Change your password immediately.

Congratulations! You just successfully run Gentoo in coLinux! You can now fool around with the system. Type halt when you are tired. There is still a long road ahead to fully harness the power. Following is two optional steps, but highly recommended.

Run coLinux during system startup

First, we need to install coLinux as a service. Open a command prompt, change to the directory c:\bin\coLinux, and run the following command:

colinux-daemon.exe @gentoo.conf --install-service "CoGentoo Linux" 

Then open Control Panel, first select Administrative Tools, then Services, click service CoGentoo Linux, and change its Startup type to Automatic.

Enlarge the image file

The original image size is 2G. This might not be sufficient. To enlarge it, you need to download the tools from here. Extract the archive and run file toporesize.bat. Use the program to resize file c:\bin\coLinux\gentoo\gentoo-root to whatever size you want. Make sure you uncheck "resize file only no resize2fs".

2. Configuration

2.1 Timezone

Since I'm located in Beijing, I set the corresponding info as follows. The related configuration files is /etc/conf.d/clock. I changed two lines as follows.

CLOCK="local"
TIMEZONE="Asia/Shanghai"

Also run command ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime.

It is interesting that Beijing cannot manage to snap one place inside /usr/share/zoneinfo/Asia.

2.2 Network

Based on coLinux wiki article.

2.2.1 One network interface

This configuration only enable TAP, which is used to enable both the high-speed private connection between Windows host and Linux guest, and also to bridge the connection to Internet. One drawback of this setup is that when you Internet connection changes (e.g. from ADSL to WiFi, you have to adjust bridging and some parameters manually).

File gentoo.conf discussed above already contains the setup for coLinux. Next steps are as follows.

On hosting OS (Windows), configure TAP-Win32 Adapter V8 (coLinux) with following properties (make sure to leave Default Gateway as blank):

Ethernet adapter:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : TAP-Win32 Adapter V8 (coLinux)
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.0.1
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Default Gateway . . . . . . . . . :

We need to bridge the Internet connection with TAP interface. For example, assume you're using an ADSL PPPoE connection. In Windows host, select Control Panel -> Network Connections, right click the PPPoE interface, and select Properties. In tab Advanced, for Internet Connection Sharing, check Allow other network users to connect through this computer's Internet connection, and for the Home networking connection, select the TAP interface (e.g. Local Area Connection 2). When Windows prompts that TAP interface IP address will be set to 192.168.0.1, choose OK.

Under coLinux, edit file /etc/conf.d/net as follows.

config_eth0=( "192.168.0.20/24" )
routes_eth0=( "default via 192.168.0.1" )
dns_servers_eth0=(" xxx.xxx.xxx.xxx " )

Please change the DNS server information (shown as xxx above) to those obtained in windows (e.g. via ipconfig).

We have to emphasize here that in this setup, Linux guest has IP address 192.168.0.20 (this is arbitrarily selected, could be other numbers like 192.168.0.x when 2 <= x <= 254), and Windows host has IP address 192.168.0.1 interfacing Linux (of course Windows host has external IP addresses as well).

2.2.2 For two network interfaces

Note this part is NOT compatible with other sections. The purpose of this section is only to document alternatives to above solution. If one interface solution is fine for you, you can skip this section.

This configuration enables both SLiRP and TAP connections. SLiRP is used to connect outside world while TAP enables the high-speed private connection between Windows host and Linux guest, which is desirable for X protocol. The benefits of this configuration includes improved security and that SLiRP can automatically use current Windows Internet connection for outgoing connections (i.e. you do not need to enter DNS server information) without reconfigurations. This works fine for Wireless LANs, but I did meet problems for ADSL and finally decided to use TAP only.

Anyway, let's look at how to setup two network interfaces.

First, we need to modify the aforementioned file gentoo.conf located in coLinux directory. Change the line eth0=tuntap with two lines as below.

eth0=slirp
eth1=tuntap

Then edit file /etc/conf.d/net on Linux guest. This enables both SLiRP and TAP.

config_eth0=( "dhcp" )
config_eth1=( "192.168.37.20/24" )

We then need to setup DHCP client as following:

emerge dhcpcd
rc-update add dhcpcd default
/etc/init.d/dhcpcd start

Since only eth0 device is generated, we need to generate eth1 device by

cd /etc/init.d
ln -s net.lo net.eth1

On hosting OS (Windows), configure TAP-Win32 Adapter V8 (coLinux) with following properties (make sure to leave Default Gateway as blank):

Ethernet adapter:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : TAP-Win32 Adapter V8 (coLinux)
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.37.10
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Default Gateway . . . . . . . . . :
2.2.3 Proxy

If you connect to Internet directly, you can skip this section.

To use proxy server, add the following lines into /etc/env.d/99local:

http_proxy="http://proxy.server.com:8080"
ftp_proxy="http://proxy.server.com:8080"

Replace the port and domain name according to your situation. Then run:

env-update && source /etc/profile

2.3 Add additional locales

Modify file /etc/locale.gen, select appropriate locales. For me, the locales are:

en_US.UTF-8 UTF-8
zh_CN.UTF-8 UTF-8

Then run command locale-gen.

2.4 Mirror selection

First run emerge mirrorselect. To select the fastest mirror. I use the following command mirrorselect -D -H -s2 to select the mirror. It means that:

  • -D: actual file will be downloaded from the mirror to test speed.
  • -H: test http mirrors only.
  • -s2: only select 2 mirrors.

After running the command, original /etc/make.conf will be backed up as /etc/make.conf.backup and your selected mirrors will be reflected in make.conf. Type mirrorselect -h for details.

2.5 Modify /etc/make.conf

This is specific for your computer type and other factors. Following is my configuration (I'm using a Core 2 Duo CPU on 32 bit Windows). Please refer to Gentoo documentation for your own reference. GENTOO_MIRRORS reflects the fastest mirrors for my connection, so you may skip that line if your have already performed mirrorselect above.

CFLAGS="-march=prescott -O2 -fomit-frame-pointer -pipe -fno-ident"
CXXFLAGS="${CFLAGS}"
CHOST="i686-pc-linux-gnu"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
ACCEPT_KEYWORDS="x86"
FEATURES="parallel-fetch"
LINGUAS="en zh"
USE="-* auctex bash-completion bzip2 cjk crypt curl doc emacs gd gif glibc-omitfp gtk \
     javascript jpeg mmx mozilla ncurses nls nptl nptlonly opengl pam png readline \
     ruby sbcl source spell sse sse2 ssl svg tcpd tiff truetype \
     unicode X xft xml xorg xpm zlib"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.kaist.ac.kr/pub/gentoo/"

2.6 Update system

Since our original system is somehow outdated, we need to update it. First run emerge --sync or emerge-webrsync -v to synchronize portage, then run the following suites of commands (we need the -p argument to check what the commands are supposed to do since they're somehow "dangerous"):

emerge -uDNpv world
emerge -uDNv world
emerge --depclean -pv 
emerge --depclean -v
revdep-rebuild -pv
revdep-rebuild -v

Caveat: if you encounter circular dependency when emerge -uDNv world, try to remove gtk USE flag from /etc/make.conf. After successful emerging, add gtk flag again and run emerge -uDNv world again.

After update, run dispatch-conf to update the configuration files (you need to avoid the update of /bin/setfont. If you accidentally updated the file, you can remedy the situation as discussed below):

Caveat: if during booting, you meet the error message putfont: PIO_FONT: Function not supported, Solve the problem as follows (as discussed here):

First, run command mv /bin/setfont /bin/setfont.old

Create a new file /bin/setfont with the following content:

#!/bin/bash

if ! uname -r | grep -q -e "-co-"
then
        /bin/setfont.old $@
fi

and make it executable by running chmod +x /bin/setfont

3. Run X

X.org is already installed in coLinux machine when we update the system as discussed in section 2.6 (we have include X USE flag). Therefore the remaining steps are just to install X server in Windows host.

We choose Xming considering it is lightweight and widely used. To install Xming, Download Xming installer, font installer and Xming-tools-and-clients. You may use Xming-mesa installer to replace Xming installer for stable OpenGL operation (only do so if you meet problems with Xming installer).

My settings during installation of Xming are as follows:

  • Xming
    • Destination directory: changed to c:\bin\Xming.
  • Xming-fonts
    • Destination directory: changed to the same directory as Xming (c:\bin\Xming)
    • Fonts to be installed: select according to your own taste.
  • Xming-tools-and-clients
    • extract the zip file of Xming-tools-and-clients to the directory where Xming is installed (c:\bin\Xming in this case).

Settings on Windows host

For file x0.hosts in Xming directory, make sure the IP address of coLinux machines (192.168.0.20) is added.

If you want to run the tools and clients bundled with Xming in Windows host, you need to type set DISPLAY=127.0.0.1:0 in your Windows host.

Following steps are used to enable Windows fonts to be used by X application. This step is optional.

Open a command prompt window, change to directory c:\bin\Xming, and type the following commands:

mkfontscale C:/WINDOWS/Fonts
mkfontscale -b -s -l C:/WINDOWS/Fonts

Make sure that directory C:\Windows\Fonts appeared in the file font-dirs in Xming directory.

On coLinux machine

Add the following line into /etc/profile.

export DISPLAY=192.168.0.1:0

After start Xming (preferably to start it automatically when Windows start), you can simply invoke any command inside Linux console like:

xterm &

4. Miscellaneous

Change host name

According to Gentoo Handbook.

Edit file /etc/conf.d/hostname, if you call your coLinux machine as tux, then set the following line.

HOSTNAME="tux"

Edit file /etc/conf.d/net, if you call your domain as homenetwork, then set the following line.

dns_domain_lo="homenetwork"

Remove string .\O from file /etc/issue to get rid of the annoying unknown_domain in the welcome message "This is host.unknown_domain ...".

Create regular user accounts

With the following command, I created a new user named foo as a powerful user (with the ability to su to root and also invoke some portage related commands). Pleas substitute foo above with your desired user name.

useradd foo -m -G users,wheel,portage
passwd foo

Access Windows folder

When we edit gentoo.conf above, we have already enabled coLinux to share folder c:\linux. To make our regular user to access this directory in Linux, edit file /etc/fstab and add the following line:

cofs00  /home/foo/windows  cofs  uid=foo,gid=foo,dmask=0755,fmask=0644 0 0

The above line will enable the Windows folder to be shared in Linux as a folder /home/foo/windows. You can modify above line for your own user.

Additional programs

Following is a list of commands to install my own favorite programs. Your situation would be of course different.

# Some system utilities
emerge gentoolkit slocate

# Development tools
echo "dev-lang/erlang hipe" >> /etc/portage/package.use
emerge aspell ruby sbcl erlang dev-util/git maxima

# Gfx (GLEidescope is one of my favorite XScreenSaver)
emerge inkscape blender xscreensaver

# Netowork
emerge mozilla-firefox

# X tools
emerge xlsfonts xfontsel

Before emerge texlive, I have to add a bunch of keywords for dependencies in /etc/portage/package.keywords as follows.

app-text/texlive ~x86
app-text/texlive-core ~x86
dev-texlive/texlive-metapost ~x86
dev-texlive/texlive-basic ~x86
dev-texlive/texlive-documentation-base ~x86
dev-texlive/texlive-psutils ~x86
dev-texlive/texlive-langcjk ~x86
dev-tex/cjk-latex ~x86
dev-texlive/texlive-documentation-english ~x86
dev-texlive/texlive-genericrecommended ~x86
dev-texlive/texlive-latexrecommended ~x86
dev-texlive/texlive-latex ~x86
dev-texlive/texlive-texinfo ~x86
dev-texlive/texlive-latex3 ~x86
dev-texlive/texlive-htmlxml ~x86
app-text/jadetex ~x86
dev-texlive/texlive-fontsrecommended ~x86
app-text/dvipdfm ~x86
dev-texlive/texlive-documentation-chinese ~x86
dev-texlive/texlive-langmanju ~x86

Use CVS version of Emacs

Run the following commands:

echo "app-editors/emacs-cvs ~x86" >> /etc/portage/package.keywords
emerge emacs-cvs

If you want to use this bleeding-edge version only, you can uninstall the previous version of Emacs (if you have installed) and type:

echo "virtual/emacs ~x86" >> /etc/portage/package.keywords

Otherwise (i.e. you have multiple versions of Emacs), when you emerge packages requiring Emacs, a stable version (e.g. v22.1) will be installed. For the multiple-version-installation, you can use eselect emacs list to see a list of available version. To use Emacs 23 just installed, you can type eselect emacs set emacs-23. For details, please consult here.

Emacs extensions

I run the following commands to install a few Emacs extensions. Note that in Gentoo, many extensions (e.g. those related to development) are automatically installed for you when you emerge the related tools and enable emacs USE flag.

echo "app-emacs/gentoo-syntax ~x86" >> /etc/portage/package.keywords
echo "app-emacs/imaxima ~x86" >> /etc/portage/package.keywords
emerge htmlize imaxima app-emacs/gentoo-syntax keywiz

Enable SSH server

You may want to enable SSH access to your coLinux machine. There are some benefits. Examples are shown below (assuming PuTTY is used):

  • More flexibilities to configure the client (e.g. pretty fonts) and nice scrollbars.
  • The title of PuTTY indicates what coLinux is doing (such information might be useful during a long emerge process).

To start SSH server, type the following commands.

rc-update add sshd default
/etc/init.d/sshd start

When you use PuTTY client, you should fill 192.168.0.20 as the host address to utilize the high speed internal connection.

GTK configrations

Change the default GTK+ font. Edit file ~/.gtkrc-2.0, add the following line, and change the font name and size to your preferred one.

gtk-font-name = "Arial 15"

Mount CD

Simply type the following

# mount -t iso9660 -o ro /dev/hdc /mnt/cdrom

Final Words

coLinux is an interesting way to run Linux applications in Windows host. One drawback of coLinux is that the daemon process is pinned to one processor only. This avoids some bizarre issues, however it does not utilize the potentials of modern multi-core systems. This is disappointing especially for a Gentoo user who requires lots of computation power. Hope the situation could be improved in the future.

Update: please refer this post on how to enable sound.

Thursday, January 17, 2008

Reading List

My previous post discussed my thoughts on how to read programming books, here comes my reading list of programming/computer science books. The books I have completed reading after this post are listed for each year, afterwards come the books I am reading and those I plan to read (all books I have completed before this post is not listed here). Note that this is a living document.


Reading

To Read


Update (2008-8-18): started to add books other than CS/programming, e.g. mathematics, physics, literature, communications.

Update (2009-2-25): replaced table format with lists (lists with year heading consists books completed in that year), partly because lists are easier to update, but mainly because that I tend to sneak more books into reading queue when no free "slots" available, which makes the table format itself somehow "cheating".

Update (2009-3-13): started to use Amazon widget; correct the 1st paragraph to reflect the status of reading.

Wednesday, January 16, 2008

Read Programming Books

Although learning programming can be achieved via online tutorials, reading books is still the preferred way for systematic learning (see some discussions here, here, and here from author's point of view). Following list is gathered along my adventure on reading programming books, especially on programming language books (for books not requiring active participations, it is recommended that you follow advice from Charles Petzold). Some suggestions are easier to be said than to be done, therefore the list is also a reminder to myself.

1. Select the right book

Reading a programming book requires enormous time and energy. Before embarking on the hundreds-hour-adventure, it is justified to spend sometime to select the correct path. Sometimes starting reading an excellent book itself is an achievement. So which book to read? The simplest solution is to search in Google or Amazon and select those with enthsiastic reviews. Google/Amazon are always handy since you can get more details/reviews on books. There are alternatives of course, as we will discuss below.

If you just want to read great programming books without any specific topics, you are lucky since this important question has already been discussed again and again. For example, you may refer to this Reddit thread, or you can consult the opinions of some outstanding programmers/bloggers (e.g. Steve Yegge's recommendations here and here, Joel Spolsky's advice, and Jeff Atwood's list).

If you do have some topics in mind (e.g. one particular programming language), there are various ways in addition to Google/Amazon approach.

  • Lookup in Wikipedia. Typically for one programming language, there will be one related article. At the end of the article, there is one References section, where you can find some interesting links to books.
  • Consult the FAQ of the language newsgroup. Normally, language X has one corresponding newsgroup like comp.lang.X. Generally the FAQ (sometimes quite outdated) of the newsgroup contains some book recommendations.
  • For some languages with official website (e.g. Haskell, Perl, and Python), you can easily find book recommendations in the site.

Let's name a few commonly recommended books for some programming languages: SICP (a.k.a. Wizard Book) for Scheme, Practical Common Lisp for Common Lisp, Programming Ruby (a.k.a. The PickAxe Book) for Ruby, and Javascript, the Definitive Guide (a.k.a. The Rhino Book) for the Next Big Language.

2. Setup programming environment and prepare related resources

You can't learn programming without getting your hands dirty. To study one programming language, you have to install necessary package(s). "Modern" programming books (e.g. Practical Common Lisp, Programming Ruby, Javascript, the Definitive Guide, and Programming Erlang) typically have one chapter getting you started. If your book does not contain such part, you can always consult online tutorials to setup the environment.

Typically the package contains some kind of editor or IDE for developing. You need to familiarize yourself with them either by reading your book or tutorial. For Emacs users, you can install related mode and simply fire up Emacs.

If the book is not so cutting-edge, you can find abundant resources in the web: source code, discussion on exercises, and even video lectures. You can bookmark or download these resources for your future reference. SICP is an excellent example here: great video lectures, and nice wiki to discuss solutions to exercises.

3. Practice, practice, and practice

You can't learn a language without sustained practice. We will focus on the examples and exercises contained in the book here.

When you read one example, simply reading is not enough. It is better if you type the example and run it. Note: type, not copy & paste. The reason is the same as when you write programs. After compilation, you think your code has done what it's supposed to do. But typically you need several iterations to make it right. Similar thing happens for reading. You think you understand the code on paper, but actually you do not. You should type it in and run it to really understand it. When you type, you can think about the meaning of the code, syntax and idioms associated simultaneously. A more paranoid yet effective way is that you first read the example, and then enter it without even referring to the book. You then run the code, analyze the results, and compare your code with the example supplied in the book.

Now for exercises. Do them all if you can. Excellent books come with excellent exercises, if any. You could gain more insight when you employ the techniques you just learned. (If you have any doubt on the value of exercises, you can look at Steve Yegge's example on how doing exercises can change a person's view about a language.) If you cannot solve one problem with serious try, you can leave it for a while and attack it later. If you really cannot handle it, you can refer to the solutions, which you can typically find online. Believe me, you can easily find the answers to these hard problems of excellent books in the web.

4. Develop hobby project

When you have already read several chapters of the book and feel comfortable to write a few small programs with the language you just learned, you may consider to setup some hobby projects. You can use the project to automate some day-to-day tasks, either in home or at work. Or you may try to implement/interface some (emerging) cool things (e.g. Google Chart API or Ray tracing) with the language you are learning. Make sure the skills to complete the project match the skills you just learned. You may meet some problems you cannot handle with the skills you just acquired. Don't panic. Just peep the later chapters of the book you are reading, or Google around. Try to finish the project if you can (and then you can blog about it if nobody has not done so or your way is quite interesting). If you cannot finish it, you can leave it for a later attempt. You may try several such projects when you progress in the reading. The further you progress, the more ambitious task you can try. Such hobby project itself can be viewed as extended exercises, but designed by you, instead of the author.

(Note: this section is largely influenced by a nice article/post related to reading On Lisp. Unfortunately I cannot find the URL now.)

5. Never give up

The last but not the least advice is that you should read the book from the beginning to the end. Spending a lot of time on one book but giving it up when approaching say one third of it is just wasting time. Since you only touched the surface of the book, which is easier to learn and also easier to forget. Authors arrange the books in such a way that you could linearly read along, which typically means that advanced topics come later. You may get stuck with one chapter and cannot understand the topics discussed there. At this point, don't give up. Remember that you can only achieve a higher level of understanding of one language/technique if you can understand the difficult part of it. Just read it, understand it, try it, or Google around to find different (maybe better) treatment of the topic. Attack this gnarly problem and you will gain lots from it. You have now learned some unique things of the language, you can now look at the language in a different yet more insightful view, and of course, you can now proceed further to read the great book and learn more.

Happy reading!

Update (2010-2-21): editorial corrections (thank you justin for your suggestions).

Monday, January 14, 2008

MID for Programmers

Lots of MIDs were showed off in CES 2008. However, none of them satisfy my requirements. What I need is a device for web surfing, RSS feed consuming, programming, and of course blogging. The one closest to my criterial is from LG, however I shunned away from Windows. Following is a list of features I expect an MID to have, from a programmer's perspective.

Mandatory factors:

  • Around 5 inch touchscreen. Size matters. The screen size is the most difficult factor when making tradeoffs. We want it smaller for portability (otherwise why not a gorgeous 17 inch MacBook Pro?), but not so small to hurt our eyes. 5 inch seems to strike a nice balance. At this size, 800x600 resolution seems to be preferred.
  • Full keyboard (similar as that of laptop in terms of keys equipped). Interface is critical. I am an Emacs fan, and the main usage of the MID is to try interesting programming ideas within Emacs and to compose blogs as well. Therefore I need the keyboard to have Ctrl and Alt equipped as a minimum. I also hope numerical keys have their independent positions (e.g. not shared with QWERTYUIOP as my Dopod 838, which is really painful). Keyboard from LG prototype satisfies all requirements here, therefore I do hope other vendors could follow suit.
  • Fast CPU. Gentoo is my favorite Linux distribution, therefore I may try to install it (somebody has already done so on Q1 Ultra). In addition to satisfy a power-demanding distribution, a powerful CPU is also a right for programmer even on such a tiny gadget.
  • Linux based OS. Linux will shine on MID, which is actually an old news. However, there are also devices running Windows at least for Menlow based MIDs, e.g. the one from LG. The reason for me to mandate Linux here is that I can easily install other flavors of Linux distributions if the device itself comes with one. With built-in Linux, we can make sure that source code for hardware is readily accessible: every vendor should learn something from the event of Eee PC GPL issues.
  • WiFi access. Connection is important. Without WiFi access, I guess we can remove Internet from the name of MID.
  • Miscellaneous. We should match the fast CPU with at least 512 MB RAM. 8 GB internal storage is a minimum to fool around. SD card slot is a must to connect the MID with other devices, and Bluetooth with A2DP support is necessary to avoid hairy wires.

Optional features:

  • Phone capability. This feature is actually desperately needed. However I doubt whether I could find one MID if I insist on this feature. There are various reasons for vendors to drop this feature: maturity of the needed software, regulation aspects, and marketing issues (imagine the impact on N95 if N810 is a phone) etc.
  • GPS. This is always a nice feature to enjoy.
  • Nice camera. Just imagine the wow moment when you use your 5-inch-LCD-digital-camera.

Finally, is it possible for MID + Android?

Saturday, January 12, 2008

Gentoo's Crisis

Sad to learn Gentoo's crisis from Slashdot. I unsubscribed from Gentoo newsletter several months ago (even the newsletter itself has not been updated for some time), and I do not have any clue on this crisis. But according to my knowledge, there is no Gentoo 2007.1 release, which is a dangerous signal. Although the periodical release in Gentoo itself is not as important as for other distributions (actually there is no impact for users who have already installed Gentoo), bleeding-edge release is a must for newcomers.

As a fan of Gentoo, I do hope that Robbins's offer will be accepted and all the issues will be sorted out quickly.