TOC

Welcome to 'building the WirelessLeiden NanoBSD image'

As an introduction you may wish to check out ​FreeBSD default method for installing ​FreeBSD on embedded devices, like ​Soekris, ​Alix and others. The manual described below is (loosely) based on the ​basic concept plan of Tim Baas, but also consists of many hints by ​official 'Introduction to NanoBSD' manual and other stuff found on the net ;-). The advantages of using Nanobsd have been nicely summarized recently on the ​pfsense website.

'Have to remember' links

• ​http://people.freebsd.org/~phk/nanobsd/soekris_4x26/make.soekris_4x26.conf
• Active work in progress node images at ​http://webfolder.wirelessleiden.nl/rick/nodefactory/
• ​Getting Xircom pcmcia network adapter to work
• ​How to add a USB vendor
• ​LIACS wiki

Procedure

Please note that you can find the supporting files in the ​svn/code subdirectory. There are different versions available for node- and gateway (proxy) configurations and also 'trunk' (operational) and 'branches' (development) versions. Also keep a close look at the INFO WARNING ERR directives as they guide you through some common pitfalls. This guide was originally written for FreeBSD 7.2, this page is still available FreeBSD7.2_Guide. As there were a lot of (positive) changes in the wireless stack, we moved to FreeBSD 8.2 which was implemented on all nodes in the course of 2011 and to 9.0 beginning of 2012.

We now use exclusively a FreeBSD 9 version. Code can be found in svn: ​http://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-9/ . Note: there is an issue with usb-lan adapters in 9.1-release (9.0-release is OK).

Changelog

For every new release of FreeBSD we also tend to release a new version a few weeks after it, containing all our latest fixes. But bear in mind that we also do bugfixes in the meanwhile and that the changelog is not always kept super up2date, so keep an eye on svn log ;-).

11.1-RELEASE:

• see FreeBSD11_Guide

10.2-RELEASE (obsolete):

• see FreeBSD10_Guide

9.1-RELEASE:

• Merged Proxy and Node configurations in one configuration, such that we need fewer boards.
• Using ISC-DHCPD instead of DNSMASQ, because we have more memory available now.
• Using BIND instead of MARADNS, maradns support for slaves was hard to maintain.
• Made captive portal more robust and user friendly for mobile devices.
• Implemented incoming RDR support, to allow accessing some WL devices from the internet.

9.0-RELEASE:

• use of pf instead of ipfw - because of rich feature set.
• Better design of captive portal, faster response.
• More monitoring support, also monitoring of nanostations
• Better maintenance tools and overviews (dynamic motd).
• gformat support for genesis nodes database.
• Overhaul of the DNS infrastructure master/slave on the nodes with named.
• Better support for PEN (more than three proxies) and better sorting.
• Ensure that the fastest nameservers are automagically ranked first in /etc/resolv.conf to shorten DNS response time.

The procedure to setup and configure a 9.1 nodefactory is similar to the 9.0 and 8.2 procedure described below.

A. Setup a FreeBSD host

---

Warning: 1) Make sure /usr is at least 5GB in size, as building images requires quite some space. 2) Make sure you install the 32bit i386 release of FreeBSD also when your system does support amd64, as cross compiling can give some nasty surprises. Tip: use a separate hard disk, mounted on /usr/obj to speed up the compilation process.

---

Get yourself a fresh i386 freebsd host with ports and subversion installed as follows:

A.1. Run the basic CD installer

The procedure below has been tested with 9.0-RELEASE (standard developer install - no ports -, e.g. with default partitioning will fit our needs). Installing FreeBSD is outside the scope of this document, take a look into ​the FreeBSD handbook Chapter 2 Installing FreeBSD if you do not know the details.

Please do mind that all commands below need to be executed as root, because of the many mounts and unmounts done in various phases. It maybe convenient to permit ssh root login (modify /etc/ssh/sshd_config accordingly).

Internet connection is required.

Set correct date/time, e.g: build# ntpdate -s 0.nl.pool.ntp.org

(You may also wish to add ntpd_enable="YES" to /etc/rc.conf)

A.2 fetch ports

build# portsnap fetch extract

A.3 set some useful variables

Alter the shell configuration file:/root/.cshrc:

Ensure ftp is set to passive mode, to avoid potential firewall issues: build# echo 'setenv FTP_PASSIVE_MODE YES' >> /root/.cshrc

Set a default password for the images that you will produce build# echo 'setenv CFG_ROOT_PASSWORD DefaultPassword12!' >> /root/.cshrc Define your nanobsd (svn) working directory: NOTE: All commands at later stages will refer to this so you better get it right! build# echo 'setenv R /root/nanobsd' >> /root/.cshrc

Next load your file (or login again): build# source /root/.cshrc

A.4. install subversion and ocaml

Note: Install all packages via ports to prevent issues later on.

build# make -C /usr/ports/devel/subversion install clean BATCH=yes

to compile lvrouted you need ocaml support, without X11, without TOOLBOX:

build# make -C /usr/ports/devel/ocaml-findlib install clean WITHOUT="TOOLBOX X11 TK" BATCH=YES

build# make -C /usr/ports/ports-mgmt/pkg install clean BATCH=yes

build# make -C /usr/ports/dns/bind-tools/ install clean BATCH=yes #(required for nagios-plugins in FreeBSD10)

A.5. OPTIONAL, every developer has his own preferences, these are mine

build# make -C /usr/ports/editors/vim-lite install clean BATCH=yes build# make -C /usr/ports/security/sudo install clean BATCH=yes build# make -C /usr/ports/sysutils/screen install clean BATCH=yes

A.6. get latest sources

build# svn co ​svn://svn.freebsd.org/base/releng/9.0 /usr/src or build# svn co ​svn://svn.freebsd.org/base/releng/10.0 /usr/src or build# svn co ​svn://svn.freebsd.org/base/stable/10.0 /usr/src or use mirror svn0.eu.FreeBSD.org

B. Build environment

B.1 Download the environment from the Wireless Leiden svn repository

 $ svn checkout \
   https://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-9/nanobsd $R
(or   https://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-10/nanobsd/ $R )
(what about https://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-9.2/nanobsd $R)
 $ cd $R

Note: if svn is not found: svn is in /usr/local/bin, alternatively log out and in, or use rehash in a csh shell to make it available.

B.2. Compile all required packages using

$ $R/tools/package-build.sh

B.3. Set your favorite root password to be used in the image

Note: you can skip this step if you are satisfied with the default password set in step A.3 above.

$ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 |\
  tr -cd '[a-zA-Z0-9]' | cut -c -15`
$ echo $CFG_ROOT_PASSWORD

If you like a simple password, substitute the `dd if=/dev/random bs=10k count=10 | tr -cd '[a-zA-Z0-9]' | cut -c -15` with your password.

B.4 Build nanobsd (make sure to prepare some coffee;-) )

$ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -c $R/cfg/nanobsd.wleiden 

Note 1: Take a coffee of go for a hike, this normally takes 2 - 8 hours depending on the machine configuration. If you like to save some power use the script provided by Rick (​http://rickvanderzwet.nl/svn/personal/misc/power-saver). This script is mainly used on a home server for building FreeBSD world and kernels. As soon it is done it can shutdown if not being used anymore. The system has Wake-On-Lan support and can thus be activated again from remote.

Note 2: even this little script got flags, check the output of sh /usr/src/tools/tools/nanobsd/nanobsd.sh -h

-b suppress builds (both kernel and world)
-k suppress buildkernel
-w suppress buildworld
-c specify config file
...

Note 3: The geometry of the cf card is defined in the nanobsd configuration file $R/cfg/nanobsd.wleiden The 'default' values are for a Peak 1 GB card. They also work on an Alix2D3 board with a 1 GB PCEngines 'blanc' cf card and 1 or 2 GB Transcend cf cards although those geometries are different. There are issues with Soekris boards, depending on BIOS version.

C. Fetch node configuration onto image, write to CF disk or remotely update

C.1. Fetch configuration

$ $R/tools/image config for <HybridNodename>

The script is connecting to the ​Wireless Leiden 'genesis' database. First make sure that the configuration file is up to date by clicking the 'update' button on ​http://sunfire.wirelessleiden.nl/wleiden/config/ Pick your situation! Note: proxy configurations are ​here.

You can inspect the image bij mounting as memory disk:

mdconfig -a -t vnode -f /usr/obj/nanobsd.wleiden-hybrid/_.disk.full
mount /dev/md0s1a /mnt
ls /mnt
umount /mnt
mount /dev/md0s3 /mnt
ls /mnt
umount /mnt
mdconfig -d -u 0

Editing can also be done using the image script:

$R/tools/image edit

C.2. Write the correct image to CF (media based on SLC and not MLC flash seem to perform much better).

---

NOTE: _.disk.full is required for new CF cards as it contains two base systems and one configuration.

_.disk.image on the other hand can be used to update an existing CF card

---

1. New image: Put full image on compact flash disk (attach a card reader/writer with a CF disk of minimum 1 GB) Use the script

   $R/tools/write-image.sh
   

or the command line:

$ dd bs=64k if=/usr/obj/nanobsd.wleiden-hybrid/_.disk.full of=/dev/da0

(assuming /dev/da0 is your compact flash entry; this takes about 15 minutes, check progress by typing Ctrl-T; you may wish to check afterwards with fdisk whether there are three partitions on the disk, you can also mount /dev/da0s1a and /dev/da0s3 and check the configuration)

1. Existing image: Put partial image on slice (slice 2) (attach a card reader/writer with the CF disk containing the existing image)

   $ dd bs=64k if=/usr/obj/nanobsd.wleiden-hybrid/_.disk.image of=/dev/da0s2
   

1. Existing image remote update (slice 2) (network connection to machine 172.x.y.z required)

For remotely updating an existing configuration use: the image script

$R/tools/image deploy on <node> [and reboot]

or manually:

dd if=/usr/obj/nanobsd.wleiden-hybrid/_.disk.image bs=10k | ssh root@172.x.y.z /tools/updatep2

You may adjust the block size to make the transfer more efficient (additionally ssh -C is possible) and you can use Ctrl-T to check progress.

WARNING: in case of a remote update from 7.2 to 8.0 you must scp the new rc.conf.local to the node before reboot because the interface configuration statements are quite different. If you do not adjust the configuration before reboot the interfaces will not be configured after reboot and you will not be able to reach the node!(see F.4 below)

D. Check the cf card and apply last minute changes

D.1. Check cf card

While the cf card is still in your flash card reader you can check whether the image has been written correctly. Check whether you can mount the partitions, e.g.

mount /dev/da0s1a /mnt
ls /mnt
umount /mnt

and the configuration partition:

mount /dev/da0s3 /mnt
ls /mnt

If you cannot mount the partitions, take a fresh cf card and start again with writing the image.

D.2 adjust lvrouted conf

If the node has a /28 subnet that should be routable modify the /conf/base/etc/rc.conf file:

lvrouted_flags="...... -m 28 "

(for a /24 subnet use -m 24)

Note. For an 'iLeiden'-type node you have to add also the -z flag (pointing to iLeiden internet gateway ip-addresses, separated by comma's and no spaces) here. This is now automagically done in the config step.

D.3 adjust memory disk sizes

For non-standard hard ware you may change the memory disk sizes by modifying the rc.conf.local configuration file. /dev/md0 /etc /dev/md1 /var /dev/md2 /tmp For instance a Soekris4521 has only 64 MB RAM, add for instance the following values

varsize="2m"
tmpsize="2m"

D.4 * now obsolete

Customized captive portal page can be added as /cfg/local/captive/include.htm The default captive portal page is defined at the end of /usr/local/www/wlportal/index.cgi

E. All done! Load the machine

Place the new CF disk in the machine and boot it up (existing nodes: just reboot), you should be good to go! Especially first time booting can take a long time, depending also on hardware. In a Soekris4521 this takes about ten minutes. You can follow the boot process using e.g. minicom or tip, via a serial connection (communication parameters: 9600 8N1).

Note 1. For subsequent nodes you can skip the build and go directly for step C, or take a look at development if you have to rebuild the image (after small changes).

Note 2. Check your bios version in case of booting problems (note: default communication setting for alix is 38400 8N1, bios should be 0.99h)

F. Applying Updates

For small changes there is a shortcut in the build (to save you some coffee ;-)). Pick the situation applicable

1. First lets pump its source to the latest version

   $ svn up /root/nanobsd
   

2. Set your favorite root password to be used in the image - if different from the password defined in your setup file (see A.4 above):

   $ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15`; echo $CFG_ROOT_PASSWORD
   

   If you like a simple password, substitute the `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15` with your password.
3. Next step depends on the kind of changes:
   1. Only changes in kernel options - kernel.wleiden altered - ETA 0:45

      $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -w -c $R/cfg/nanobsd.wleiden
      

   2. Only changes in world options - CONF_{BUILD,COMMON,INSTALL} in nanobsd.wleiden altered - ETA 2:00 - 6:00

      $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -k -c $R/cfg/nanobsd.wleiden
      

   3. _NO_ changes in build options - any other file altered - ETA 0:10

      $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -b -c $R/cfg/nanobsd.wleiden
      

   4. ONLY different configuration

      $ $R/tools/image config [for node] 
      

4. Upgrading from 7.2 to 8.2 is tricky because the interface configuration statements have changed. You have to update the config file on slice 3 as well in this case. Mount the image as memory disk:

   mdconfig -a -t vnode -f /usr/obj/nanobsd.wleiden/_.disk.full
   mount /dev/md0s1a /mnt
   

   and apply changes to lvrouted flags in rc.conf if needed, see D.2 above

   umount /mnt
   

   Mount the configuration slice:

   mount /dev/md0s3 /mnt
   

   scp the rc.conf.local file to the /cfg directory on the node running the old 7.2 configuration

   umount /mnt
   mdconfig -d -u 0
   

5. Partition switch bug in FreeBSD8.0

There appears to be a bug in FreeBSD8.0 upgrade procedure. A work around for the partition switch problem in the upgrade procedure that works for Wireless Leiden:

• reboot ( unclear why this is needed, but otherwise it doesnot work, i.e. no switch of the active partition)
• updatep2 (or p1)
• log in, check whether fdisk has activated p2 (or p1).
• If positive go to runlevel 6 (init 6), log out.
• After a couple of minutes you can log in, welcome on the new slice!

Note we have recently modified the /tools/update scripts to use gpart instead of fdisk and boot0cfg, see the /tools/updatep scripts for details. This solved the problem.

G, Development

Handy tools

Some handy tools are available in the /tools directory, such as

• /tools/wl-version : to get information on the configuration that is running
• /tools/image-build.sh for building images (replaces long command lines and prevents unintentional, time-consuming rebuilds; do not forget to setenv the root password!)

Further research

• stumber /usr/src/tools/tools/net80211/stumber

Background

• Packages needed are to be found in the nanobsd/tools/package-build.sh script i.e.

   $PORTSDIR/dns/dnsmasq
   $PORTSDIR/www/py-cherrypy
   $PORTSDIR/www/tinyproxy
   $PORTSDIR/editors/vim-lite
   $PORTSDIR/net-mgmt/net-snmp
   $PORTSDIR/net-mgmt/nrpe2
   $PORTSDIR/benchmarks/iperf
   $PORTSDIR/net/pen
   $PORTSDIR/net/rsync
   # Wireless Leiden ports at $WL_PORTSDIR
   $WL_PORTSDIR/net/lvrouted
   $WL_PORTSDIR/net/transproxy
  

Testing Images

• KnownErrorMessages
• TestingViaNFS
• TestMatrix

Hardware

• Soekris 4521, 4801, 5501, bios version 1.23, 1.31b, 1.32, 1.33
• Alix 2D3, only use latest bios: 0.99h

Note on updating Soekris bios (CTRL+P to enter): no luck with minicom, used cu on linux (Ubuntu: sudo apt-get install lrzsz cu):

chown uucp /dev/ttyS0
cu -l /dev/ttyS0 -s 9600
> download -
Start sending file using XMODEM/CRC protocol.
~+sx -X b4501_133.bin
Sending b4501_133.bin, 608 blocks: Give your local XMODEM receive command now.
Bytes Sent:  77824   BPS:892
Transfer complete
File downloaded succesfully, size 608 Blocks.

flashupdate reboot

What changed for FreeBSD 8.0 ?

All modifications required for 8.0 are incorporated in the nodefactory version that can be found in svn: ​http://svn.wirelessleiden.nl/svn/code/iris/trunk/nanobsd/

Some modifications for freebsd-8.0 RELEASE going from 7.2 to 8.0:

• use of rc.conf.local in stead of rc.node.local to be more in-line with standard FreeBSD config
• apply only patch for usbdevs and if_udav.c to make Davicom usb-lan adapter work;
• no cardbus patch
• modify kernel.wleiden configuration file, see ​http://svn.wirelessleiden.nl/svn/projects/proxy/2.0/nanobsd/nanobsd/cfg/kernel.wleiden

   # options     CLK_USE_I8254_CALIBRATION
   # options     ADAPTIVE_GIANT  # Giant mutex is adaptive.
   # device      wlan_scan_ap    # 802.11 AP mode scanning.
   # device      wlan_scan_sta   # 802.11 STA mode scanning.
   # device      ppp             # Kernel PPP
   # device      ugen            # Generic
   # device      sio
  

• modify ttys-nanobsd.patch file
• wireless interface configuration is quite different (configuration script was changed) modify rc.conf.local somewhat like this:

   wlans_ath0="wlan0"
   create_args_wlan0="wlanmode hostap mode 11b ssid ap-omni.huub.wleiden.net channel 7"
   ifconfig_wlan0="inet 172.17.16.1/26"
  

  note that interface has to be 'down' to create_args_wlan0!
• usb-lan convertor interface for Digitus adapter is now ue (not aue)
• known issues (see freebsd-embedded mailinglist): geometry mismatch between cf-card and disklable; boot-sector/upgrade-procedure.

H. Hints

• You could always decide to install your FreeBSD base instance into a ​virtual environment
• cvsup.nl.freebsd.org is the local dutch cvsup mirror, replace nl with your proper country code. An alternative way to update sources is svn, possibly with a symlink for use of different releases, e.g.:

  $ svn co http://svn.freebsd.org/base/release/8.2.0/ /usr/src-release-8.2.0
  $ ln -s /usr/src-release-8.2.0 /usr/src
  

• updating ports:

  # cd hybrid-releng-9.0-nanobsd
  # svn up .
  # pkg_delete /var/db/pkg/*
  # rm -R /usr/local/* /var/db/ports/* /var/db/pkg/*
  # ./tools/image ports update
  

  check list of installed packages:

  # echo pkg/All/* | wc -w
  

• It seems tempting to follow /usr/share/examples/cvsup/stable-supfile instead of /usr/share/examples/cvsup/standard-supfile please DO NOT unless your like big trouble as stable-supfile is the stable DEVELOPMENT branch e.g. upcoming stable.
• Default username/password = root/<blank>, so please do mind, _when no password is set ssh login is disabled_.
• Always use cu or putty for serial communication minicom, screen, putty all have issues of some kind.
• After writing an image to cf-card using a usb-card-reader/writer you may encounter a problem dd-ing another image. Rebooting will help. Alternatively you can apply a power-cycle to the usb device:

  usbconfig -u 1 -a 2 power_off
  usbconfig -u 1 -a 2 power_on
  

  (unit and address may vary, see usbconfig list).

J. Comments, questions or remarks?

Feel free to edit this page or send an email to our ​techniek mailinglist