Howto build a 'nodefactory' based on FreeBSD 7.2

---

Procedure

Please note that you can find the supporting files in the subdirectory nanobsd. Also keep a close look at the INFO WARN ERR directives as they guide you through some common pitfalls. This is the original guide written for FreeBSD 7.2. As there are a lot of (positive) changes in the wireless stack, we now prefer the use of FreeBSD 8.0.

A. Setup a FreeBSD host

---

WARN: Make sure /usr is at least 5GB in size building as building images require quite some space
WARN: Make sure you install the i386 release of FreeBSD also when your system does support amd64, as cross compiling can give some nasty surprises

---

Get yourself a fresh i386 freebsd host with ports and subversion installed as follows:

A.1. Run the basic CD installer of 7.2.

The procedure below has been tested with 7.2-RELEASE (standard developer install - no ports - will fit our needs). Installing FreeBSD is outside the scope of this document, take a look into ​the FreeBSD handbook Chapter 2 Installing FreeBSD if you do not know the details.

Please do mind that all commands below need to be executed as root, due to the many mounts and unmounts done in various phases.

Internet connection is required.

Set correct date/time, e.g.

$ ntpdate -s pool.ntp.org

A.2. get latest sources

$ csup  -h cvsup.nl.freebsd.org /usr/share/examples/cvsup/standard-supfile

A.3 fetch ports

$ portsnap fetch extract

A.4 set some usefull variables

Edit the shell configuration file .cshrc:

vi /root/.cshrc

check whether ftp is set to passive mode, to avoid potential firewall issues:

setenv FTP_PASSIVE_MODE YES

set a default password for the images that you will produce

setenv CFG_ROOT_PASSWORD [default password]

and define the svn download directory:

setenv R /root/nanobsd

A.5. install subversion . NOTE: Install all packages via ports to prevent issues later on!

$ cd /usr/ports/devel/subversion; make install clean BATCH=yes

A.6. OPTIONAL, every developer has his own preferences, these are mine ;-)

$ (cd /usr/ports/editors/vim-lite; make install clean BATCH=yes)
$ (cd /usr/ports/security/sudo; make install clean BATCH=yes)
$ (cd /usr/ports/sysutils/screen; make install clean BATCH=yes)

B. Build environment

B.1 Download the environment from the Wireless Leiden svn repository

$ 
$ svn checkout http://svn.wirelessleiden.nl/svn/projects/iris/nanobsd $R
$ cd $R

Note: if svn is not found: svn is in /usr/local/bin, alternatively log out and in, or use rehash in a csh shell to make it available.

B.2. Compile all required packages using

$ /root/nanobsd/tools/package-build.sh

B.3. Set your favorite root password to be used in the image

Note: you can skip this step if you are satisfied with the default password set in step A.4 above.

$ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15`
$ echo $CFG_ROOT_PASSWORD

If you like a simple password, substitute the `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15` with your password.

B.4 Apply kernel patch for pcmcia LAN cards and USB-LAN adapter

If you want to be able to use pcmcia LAN cards, like the Xircom RealPort2 Cardbus Ethernet10/100 R2BE-100, you have to apply the patch to the cardbus driver before compiling the kernel:

cd /usr/src/sys/dev/cardbus
patch < /root/nanobsd/misc/patches/cardbus_cis.c.patch

For use of a Davicom DM9601 USB network controler you have to patch usbdevs and the if_udav driver:

cd /usr/src/sys/dev/usb/
patch < /root/nanobsd/misc/patches/usbdevs.patch
patch < /root/nanobsd/misc/patches/if_udav.c.patch

B.5. Build nanobsd (make sure to prepare some coffee;-) )

$ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -c /root/nanobsd/cfg/nanobsd.wleiden 

Note 1: Take a coffee of go for a hike, this normally takes 2 - 8 hours depending on the machine configuration

Note 2: even this little script got flags, check the output of sh /usr/src/tools/tools/nanobsd/nanobsd.sh -h

-b suppress builds (both kernel and world)
-k suppress buildkernel
-w suppress buildworld
-c specify config file
...

C. Fetch node configuration onto image, write to CF disk or remotely update

C.1. Fetch configuration

$ /root/nanobsd/tools/config-image.sh 

The script is connecting to the ​Wireless Leiden 'genesis' database. First make sure that the configuration file is up to date by clicking the 'update' button on ​http://wlconfig.wirelessleiden.nl/freebsd/iris/ Pick your situation!

You can inspect the image bij mounting as memory disk:

mdconfig -a -t vnode -f /usr/obj/nanobsd.wleiden/_.disk.full
mount /dev/md0s1a /mnt
ls /mnt
umount /mnt
mount /dev/md0s3 /mnt
ls /mnt
umount /mnt
mdconfig -d -u 0

C.2. Write the correct image to CF (media based on SLC and not MLC flash seem to perform much better).

---

NOTE: _.disk.full is required for new CF cards as it contains two base systems and one configuration.

_.disk.image on the other hand can be used to update an existing CF card

---

1. New image: Put full image on compact flash disk (attach a card reader/writer with a CF disk of minimum 1 GB)

   $ dd bs=64k if=/usr/obj/nanobsd.wleiden/_.disk.full of=/dev/da0
   

   (assuming /dev/da0 is your compact flash entry; this takes about 15 minutes; you may wish to check afterwards with fdisk whether there are three partitions on the disk, you can also mount /dev/da0s1a and /dev/da0s3 and check the configuration)

1. Existing image: Put partial image on slice (slice 2) (attach a card reader/writer with the CF disk containing the existing image)

   $ dd bs=64k if=/usr/obj/nanobsd.wleiden/_.disk.image of=/dev/da0s2
   

1. Existing image remote update (slice 2) (network connection to machine 172.x.y.z required)

For remotely updating an existing configuration use:

dd if=/usr/obj/nanobsd.wleiden/_.disk.image bs=10k | ssh root@172.x.y.z /tools/updatep2

You may adjust the block size to make the transfer more efficient (additionally ssh -C is possible) and you can use Ctrl-T to check progress.

NOTE: in case of a remote update from 7.2 to 8.0 you also have to run the configuration script (/tools/wl-config) because the configuration file rc.node.local is different.

D. Check the cf card and apply last minute changes

D.1. Check cf card

While the cf card is still in your flash card reader you can check whether the image has been written correctly. Check whether you can mount the partitions, e.g.

mount /dev/da0s1a /mnt
ls /mnt
umount /mnt

and the configuration partition:

mount /dev/da0s3 /mnt
ls /mnt

If you cannot mount the partitions, take a fresh cf card and start again with writing the image.

D.2 apply wi-driver hack, if applicable

If you are using wi-drivers in master mode (hostap) you may have to adjust the rc.node.local file on /dev/da0s3. There is a bug in the wi-driver: the channel is always set to 1. You can set the correct channel by adding a line to rc.node.local like:

ifconfig_wi0_alias0="channel {channel number}"

D.3 adjust lvrouted conf

If the node has a /28 subnet that should be routable modify the /conf/base/etc/rc.conf file:

lvrouted_flags="-u -m 28 -s s00p3rs3kr3t"

(for a /24 subnet use -m 24)

E. All done! Load the machine

Place the new CF disk in the machine and boot it up (existing nodes: just reboot), you should be good to go! Especially first time booting can take a long time, depending also on hardware. In a Soekris4521 this takes about ten minutes. You can follow the boot process using e.g. minicom or tip, via a serial connection (communication parameters: 9600 8N1).

Note 1. For subsequent nodes you can skip the build and go directly for step C, or take a look at development if you have to rebuild the image (after small changes).

Note 2. Check your bios version in case of booting problems (note: default communication setting for alix is 38400 8N1, bios should be 0.99h)

F. Applying Updates

For small changes there is a shortcut in the build (to save you some coffee ;-)). Pick the situation applicable

1. First lets pump it's source to the latest version

   $ svn up /root/nanobsd
   

2. Set your favorite root password to be used in the image

   $ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15`; echo $CFG_ROOT_PASSWORD
   

   If you like a simple password, substitute the `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15` with your password.
3. Next step depends on the kind of changes:
   1. Only changes in kernel options - kernel.wleiden altered - ETA 0:45

      $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -w -c /root/nanobsd/cfg/nanobsd.wleiden
      

   2. Only changes in world options - CONF_{BUILD,COMMON,INSTALL} in nanobsd.wleiden altered - ETA 2:00 - 6:00

      $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -k -c /root/nanobsd/cfg/nanobsd.wleiden
      

   3. _NO_ changes in build options - any other file altered - ETA 0:10

      $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -b -c /root/nanobsd/cfg/nanobsd.wleiden
      

G, Development

Handy tools

Some handy tools are available in the /tools directory, such as

• /tools/wl-version : to get information on the configuration that is running
• /tools/image-build.sh for building images (replaces long command lines and prevents unintentional, time-consuming rebuilds; do not forget to setenv the root password!)

Light based packages suggestions

• dnsmasq has been implemented instead of isc-dhcp30-server and bind
• thttpd instead of apache

Further research

• stumber /usr/src/tools/tools/net80211/stumber

Background

• Packages needed are to be found in source:/nanobsd/tools/package-build.sh i.e.

   $PORTSDIR/dns/dnsmasq
   $PORTSDIR/www/py-cherrypy
   $PORTSDIR/www/tinyproxy
   $PORTSDIR/editors/vim-lite
   $PORTSDIR/net-mgmt/net-snmp
   $PORTSDIR/net-mgmt/nrpe2
   $PORTSDIR/benchmarks/iperf
   $PORTSDIR/net/pen
   $PORTSDIR/net/rsync
   # Wireless Leiden ports at $WL_PORTSDIR
   $WL_PORTSDIR/net/lvrouted
   $WL_PORTSDIR/net/transproxy
  

Testing Images

• KnownErrorMessages
• TestingViaNFS
• TestMatrix

Hardware

• Soekris 4521, bios version 1.23, 1.31b, 1.32, 1.33
• Alix 2D3, only use latest bios: 0.99h

Note on updating Soekris bios (CTRL+P to enter): no luck with minicom, used cu on linux (Ubuntu: sudo apt-get install lrzsz cu):

chown uucp /dev/ttyS0
cu -l /dev/ttyS0 -s 9600
> download -
Start sending file using XMODEM/CRC protocol.
~+sx -X b4501_133.bin
Sending b4501_133.bin, 608 blocks: Give your local XMODEM receive command now.
Bytes Sent:  77824   BPS:892
Transfer complete
File downloaded succesfully, size 608 Blocks.

reboot

H. Hints

• You could always decide to install your FreeBSD base instance into a ​virtual environment
• cvsup.nl.freebsd.org is the local dutch cvsup mirror, replace nl with your proper country code
• It seems tempting to follow /usr/share/examples/cvsup/stable-supfile instead of /usr/share/examples/cvsup/standard-supfile please DO NOT unless your like big trouble as stable-supfile is the stable DEVELOPMENT branch e.g. upcoming stable.
• Default username/password = root/<blank>, so please do mind, _when no password is set ssh login is disabled_.
• Always use cu or putty for serial communication minicom, screen, putty all issues of some kind.

Comments, questions or remarks?

Feel free to edit this page or send an email to our ​techniek mailinglist

• Handy .cshrc hints

  alias quicknano sh /usr/src/tools/tools/nanobsd/nanobsd.sh -b -c /root/nanobsd/cfg/nanobsd.wleiden
  alias slownano  sh /usr/src/tools/tools/nanobsd/nanobsd.sh -c /root/nanobsd/cfg/nanobsd.wleiden
  alias slownewkernelnano sh /usr/src/tools/tools/nanobsd/nanobsd.sh -w -c /root/nanobsd/cfg/nanobsd.wleiden
  alias slownewworldnano sh /usr/src/tools/tools/nanobsd/nanobsd.sh -k -c /root/nanobsd/cfg/nanobsd.wleiden
  alias prepare-nfs  /root/nanobsd/tools/prepare-nfs.sh -f -n