How to compile an experimental X server

Some functions of the X.org server, such as 3D acceleration for newer cards, are under constant developments. To test the latest features, you need to build the X server and related software directly from the developers' CVS and GIT repositories. Since several projects and dozens of sub-projects are involved, this is a complicated task.

The following script automates fetching, compilation and installation of the following:

• Modular X.org server+libraries from CVS and GIT (see http://wiki.x.org/wiki/ModularDevelopersGuide and http://wiki.x.org/wiki/GitPage)
• libDRM and DRM kernel modules (see http://dri.freedesktop.org/wiki/Building)
• Mesa libraries and DRI drivers (see http://www.mesa3d.org/install.html and http://www.mesa3d.org/cvs_access.html)

By default all compilation happens in ~/3d-pit and everything is installed into ~/3d-pit/install. The built DRM kernel module is installed to /lib/modules/.... No other paths are touched (but run as a non-root user, just in case).

To save time not all X libraries are built, so you'll need the usual X development environment packages installed (see above links).

Currently only the ATI radeon X.org drivers and r300 DRI driver are fetched and compiled. If you need different drivers, search for "radeon" and "r300" and modify as needed.

The script

#!/bin/bash
# Script to fetch and build X.org, DRI and Mesa from CVS and GIT.
# If you want to skip some tasks, comment them out at the bottom of the script.
# Source http://thinkwiki.org/wiki/How_to_compile_an_experimental_X_server
# See also:
#   http://wiki.x.org/wiki/ModularDevelopersGuide
#   http://wiki.x.org/wiki/GitPage
#   http://gitweb.freedesktop.org
#   http://dri.freedesktop.org/wiki/Building
#   http://www.mesa3d.org/cvs_access.html
#   http://www.mesa3d.org/install.html

#############################################
# Init

PIT=$HOME/3d-pit       # Download and build here
DEST=$PIT/install      # Install here
# MESA_DATE=2006-06-29 # Set this to get a historical Mesa CVS snapshot
FLAGS='-O2 -g -march=pentium-m'

trap "echo Aborting.; exit 1" ERR

die() {
  echo "$*"
  exit 1
}
run() {
  echo "@ $@"
  "$@" || { echo "ERROR: Command \"$*\" failed in `pwd`"; exit 1; }
}

#############################################
# Fetching and building libDRM and DRM kernel drivers

fetch_drm() {
  echo "########### Fetch DRM ###########"
  DRI_CVS=:pserver:anonymous@anoncvs.freedesktop.org:2401/cvs/dri
  run mkdir -p $PIT
  run cd $PIT
  grep -qF $DRI_CVS ~/.cvspass || run cvs -d $DRI_CVS login
  [ -d drm ] || run cvs -z3 -d $DRI_CVS co drm
  run cd drm
  run cvs -z3 update -ACPd
}

build_libdrm() {
  echo "########### Build libdrm ###########"
  run cd $PIT/drm
  run ./autogen.sh
  CFLAGS="$FLAGS" run ./configure --prefix=$DEST --quiet
  run make clean
  run make
  run make install
}

build_drm_modules() {
  echo "########### Build DRM kernel modules ###########"
  run cd $PIT/drm/linux-core
  run make DRM_MODULES="radeon"
  run sudo sh -c \
    'rm -fv /lib/modules/`uname -r`/kernel/drivers/char/drm/*.ko; \
     cp -v *.ko /lib/modules/`uname -r`/extra/; \
     /sbin/depmod -a'
}

#############################################
# Fetching Mesa

fetch_mesa() {
  echo "########### Fetching Mesa ###########"
  MESA_CVS=:pserver:anonymous@anoncvs.freedesktop.org:2401/cvs/mesa
  run mkdir -p $PIT
  run cd $PIT
  grep -qF $MESA_CVS ~/.cvspass || run cvs -d $MESA_CVS login
  [ -d Mesa ] || run cvs -z3 -d $MESA_CVS co ${MESA_DATE:+-D $MESA_DATE} Mesa
  run cd $PIT/Mesa
  run cvs -z3 update -ACPd ${MESA_DATE:+-D $MESA_DATE}
}

patch_mesa() {
  # patch to make Google Earth use acceleration:
  run perl -i -pe 'print "//DISABLE " if m/^\s*FALLBACK_IF.*Line\.SmoothFlag/' \
    cd $PIT/Mesa/src/mesa/drivers/dri/r300/r300_render.c
}

#############################################
# Building Mesa (do this *after* installing X.org)

build_mesa() { (
  echo "########### Building Mesa ###########"
  # This uses both old-style and new-style vars, to support older Mesa snapshots.
  run cd $PIT/Mesa
  export DRM_SOURCE_PATH=$PIT/drm
  export PKG_CONFIG_PATH="$DEST/lib/pkgconfig:$PKG_CONFIG_PATH"
  export DRM_SOURCE_PATH="$PIT/drm"
  export DESTDIR=$DEST  INSTALL_DIR=$DEST
  export DRI_DRIVER_INSTALL_DIR=$DEST/lib/dri
  export DRI_DIRS=r300
  export OPT_FLAGS="$CFLAGS"
  export PATH="$DEST/bin:$PATH"
  run mkdir -p $DRI_DRIVER_INSTALL_DIR
  run make -e linux-dri-x86
  run make -e install
  run cp -v lib/r300_dri.so $DRI_DRIVER_INSTALL_DIR
) }

#############################################
# X.org setup
# This is a list of modules that are in flux and likely to be related to
# whatever you're testing (or are required for the build), sorted by
# build order (see util/modular/build.sh for dependencies).
# Check CVS and the above URLs for other modules.

XORG_DIRS=(
  util/macros
  proto/{bigreqs,composite,fixes,damage,gl,input,kb,pm,randr,resource}proto
  proto/{render,scrnsaver,video}proto
  proto/{x11,xf86dga,xf86dri,xf86misc,xf86vidmode,xinerama}proto
  lib/lib{xtrans,X11,Xt,Xmu,Xcomposite,Xrender,Xdamage,Xcursor,Xi,Xinerama}
  lib/lib{Xrandr,Xv,XvMC,Xxf86dga,Xxf86misc,Xxf86vm}
  xserver
  driver/xf86-video-ati
  driver/xf86-input-mouse driver/xf86-input-keyboard
  data/cursors
  util/cf util/makedepend util/gccmakedep util/lndir
)

#############################################
# Fetching X.org

fetch_xorg() {
  echo "########### Fetching (selected) X.org modules ###########"
  XORG_GIT=git://anongit.freedesktop.org/git/xorg
  run mkdir -p $PIT/xorg
  run cd $PIT/xorg
  for D in "${XORG_DIRS[@]}"; do
    if [ -d $PIT/xorg/$D ]; then 
      run cd $PIT/xorg/$D
      run git pull
    else
      run cd $PIT/xorg 
      run git clone $XORG_GIT/$D $D
    fi
  done
}

#############################################
# Building and installing X.org
# (On repeated runs we avoid rerunning autotools if nothing changed.)

LAST_HASH=last_dir_hash
dir_hash() {
  find . \( -path ./$LAST_HASH -o -path ./.git -prune \) -o -ls | sort | md5sum
}
should_autogen() {
  [ ! -f $LAST_HASH ] || [ "`dir_hash`" != "`cat $LAST_HASH`" ]
}
done_build() {
  dir_hash > $LAST_HASH
}
build_x_module() { (
  D=$1
  echo "########### Building xorg/$D ###########"
  export PATH="$DEST/bin:$PATH"
  export PKG_CONFIG_PATH="$DEST/lib/pkgconfig:$PKG_CONFIG_PATH"
  export ACLOCAL="${AC_LOCAL:-aclocal} -I $DEST/share/aclocal"
  export LD_LIBRARY_PATH="$DEST/lib:$LD_LIBRARY_PATH"
  if [ $D == "xserver" ]; then
    OPT="--with-mesa-source=$PIT/Mesa"
  elif [ $D == "lib/libX11" ]; then
    OPT="--with-xcb=no"
  else
    OPT=
  fi
  run cd $PIT/xorg/$D
  if should_autogen; then
    CFLAGS="$FLAGS" run sh autogen.sh --prefix=$DEST --quiet --cache-file=$PIT/xorg/autoconf.cache $OPT
  fi
  run make
  done_build
  run make install
) }

build_xorg() {
  run cd $PIT/xorg
  run mkdir -p $DEST/share/aclocal
  run mkdir -p $DEST/var/log
  for D in "${XORG_DIRS[@]}"; do 
    build_x_module $D || return 1
  done
  run sudo sh -c "chown -v root $DEST/bin/Xorg; \
                  chmod -v 4711 $DEST/bin/Xorg"
}

#############################################
# Creating a script which sets env vars

create_xsetenv() {
  cat <<EOF > $DEST/bin/xsetenv || die "Can't create $DEST/bin/xsetenv"
export PATH="$DEST/bin:$PATH" 
export LD_LIBRARY_PATH="$DEST/lib:$LD_LIBRARY_PATH"
export LIBGL_DRIVERS_PATH=$DEST/lib/dri
EOF
  echo "Run '. $DEST/bin/xsetenv' to point env vars to this installation."
}

#############################################
# Do it:

fetch_drm
build_libdrm
build_drm_modules
fetch_mesa
patch_mesa
fetch_xorg
build_xorg
build_mesa
create_xsetenv

echo 'Done.'

Running

Just run the above. If prompted for a CVS password, press Enter. There are two sudo invocations for which you may need to provide a password.

Prepare a config file called (say) /etc/X11/xorg.dri-test.conf with DRI enabled.

Then, exit your current X server (e.g., via # init 3) and run something like: $ . ~/3d-pit/install/bin/xsetenv; X -config xorg.dri-test.conf :0 >& xout.log & ( export DISPLAY=:0; sleep 3; kwin& xterm& )

If all goes well, you'll get a minimal X session with just an xterm open. Some things to do now:

• $ xdriinfo
• $ LIBGL_DEBUG=verbose glxinfo 2>&1 | less
• $ glxgears
• $ sudo less ~3d-pit/install/var/log/Xorg.0.log
• $ ppracer (and report on the associated bug including the relevant # lspci -v lines)
• $ googleearth (Linux version here)

To keep up with the latest CVS and GIT versions, just run the script again. You don't need to (and shouldn't) delete ~/3d-pit beforehand.