Welcome to 'building the WirelessLeiden NanoBSD image'
As an introduction you may wish to check out FreeBSD default method for installing FreeBSD on embedded devices, like Soekris, Alix and others. The manual described below is (loosely) based on the basic concept plan of Tim Baas, but also consists of many hints by official 'Introduction to NanoBSD' manual and other stuff found on the net ;-). The advantages of using Nanobsd have been nicely summarized recently on the pfsense website.
'Have to remember' links
- Active work in progress node images at http://webfolder.wirelessleiden.nl/rick/nodefactory/
- Getting Xircom pcmcia network adapter to work
- How to add a USB vendor
- LIACS wiki
Please note that you can find the supporting files in the svn/code subdirectory. There are different versions available for node- and gateway (proxy) configurations and also 'trunk' (operational) and 'branches' (development) versions. Also keep a close look at the INFO WARNING ERR directives as they guide you through some common pitfalls. This guide was originally written for FreeBSD 7.2, this page is still available FreeBSD7.2_Guide. As there were a lot of (positive) changes in the wireless stack, we moved to FreeBSD 8.2 which was implemented on all nodes in the course of 2011 and to 9.0 beginning of 2012.
We now use exclusively a FreeBSD 9 version. Code can be found in svn: http://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-9/ . Note: there is an issue with usb-lan adapters in 9.1-release (9.0-release is OK).
For every new release of FreeBSD we also tend to release a new version a few weeks after it, containing all our latest fixes. But bear in mind that we also do bugfixes in the meanwhile and that the changelog is not always kept super up2date, so keep an eye on
svn log ;-).
- see FreeBSD11_Guide
- see FreeBSD10_Guide
- Merged Proxy and Node configurations in one configuration, such that we need fewer boards.
- Using ISC-DHCPD instead of DNSMASQ, because we have more memory available now.
- Using BIND instead of MARADNS, maradns support for slaves was hard to maintain.
- Made captive portal more robust and user friendly for mobile devices.
- Implemented incoming RDR support, to allow accessing some WL devices from the internet.
- use of pf instead of ipfw - because of rich feature set.
- Better design of captive portal, faster response.
- More monitoring support, also monitoring of nanostations
- Better maintenance tools and overviews (dynamic motd).
- gformat support for genesis nodes database.
- Overhaul of the DNS infrastructure master/slave on the nodes with named.
- Better support for PEN (more than three proxies) and better sorting.
- Ensure that the fastest nameservers are automagically ranked first in /etc/resolv.conf to shorten DNS response time.
The procedure to setup and configure a 9.1 nodefactory is similar to the 9.0 and 8.2 procedure described below.
A. Setup a FreeBSD host
Warning: 1) Make sure /usr is at least 5GB in size, as building images requires quite some space. 2) Make sure you install the 32bit i386 release of FreeBSD also when your system does support amd64, as cross compiling can give some nasty surprises. Tip: use a separate hard disk, mounted on /usr/obj to speed up the compilation process.
Get yourself a fresh i386 freebsd host with ports and subversion installed as follows:
A.1. Run the basic CD installer
The procedure below has been tested with 9.0-RELEASE (standard developer install - no ports -, e.g. with default partitioning will fit our needs). Installing FreeBSD is outside the scope of this document, take a look into the FreeBSD handbook Chapter 2 Installing FreeBSD if you do not know the details.
Please do mind that all commands below need to be executed as root, because of the many mounts and unmounts done in various phases. It maybe convenient to permit ssh root login (modify /etc/ssh/sshd_config accordingly).
Internet connection is required.
Set correct date/time, e.g: build# ntpdate -s 0.nl.pool.ntp.org
(You may also wish to add ntpd_enable="YES" to /etc/rc.conf)
A.2 fetch ports
build# portsnap fetch extract
A.3 set some useful variables
Alter the shell configuration file:/root/.cshrc:
Ensure ftp is set to passive mode, to avoid potential firewall issues: build# echo 'setenv FTP_PASSIVE_MODE YES' >> /root/.cshrc
Set a default password for the images that you will produce build# echo 'setenv CFG_ROOT_PASSWORD DefaultPassword12!' >> /root/.cshrc Define your nanobsd (svn) working directory: NOTE: All commands at later stages will refer to this so you better get it right! build# echo 'setenv R /root/nanobsd' >> /root/.cshrc
Next load your file (or login again): build# source /root/.cshrc
A.4. install subversion and ocaml
Note: Install all packages via ports to prevent issues later on.
build# make -C /usr/ports/devel/subversion install clean BATCH=yes
to compile lvrouted you need ocaml support, without X11, without TOOLBOX:
build# make -C /usr/ports/devel/ocaml-findlib install clean WITHOUT="TOOLBOX X11 TK" BATCH=YES
build# make -C /usr/ports/ports-mgmt/pkg install clean BATCH=yes
build# make -C /usr/ports/dns/bind-tools/ install clean BATCH=yes #(required for nagios-plugins in FreeBSD10)
A.5. OPTIONAL, every developer has his own preferences, these are mine
build# make -C /usr/ports/editors/vim-lite install clean BATCH=yes build# make -C /usr/ports/security/sudo install clean BATCH=yes build# make -C /usr/ports/sysutils/screen install clean BATCH=yes
A.6. get latest sources
build# svn co svn://svn.freebsd.org/base/releng/9.0 /usr/src or build# svn co svn://svn.freebsd.org/base/releng/10.0 /usr/src or build# svn co svn://svn.freebsd.org/base/stable/10.0 /usr/src or use mirror svn0.eu.FreeBSD.org
B. Build environment
B.1 Download the environment from the Wireless Leiden svn repository
$ svn checkout \ https://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-9/nanobsd $R (or https://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-10/nanobsd/ $R ) (what about https://svn.wirelessleiden.nl/svn/code/hybrid/branches/releng-9.2/nanobsd $R) $ cd $R
Note: if svn is not found: svn is in /usr/local/bin, alternatively log out and in, or use
cshshell to make it available.
B.2. Compile all required packages using
B.3. Set your favorite root password to be used in the image
Note: you can skip this step if you are satisfied with the default password set in step A.3 above.
$ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 |\ tr -cd '[a-zA-Z0-9]' | cut -c -15` $ echo $CFG_ROOT_PASSWORD
If you like a simple password, substitute the
`dd if=/dev/random bs=10k count=10 | tr -cd '[a-zA-Z0-9]' | cut -c -15`with your password.
B.4 Build nanobsd (make sure to prepare some coffee;-) )
$ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -c $R/cfg/nanobsd.wleiden
Note 1: Take a coffee of go for a hike, this normally takes 2 - 8 hours depending on the machine configuration. If you like to save some power use the script provided by Rick (http://rickvanderzwet.nl/svn/personal/misc/power-saver). This script is mainly used on a home server for building FreeBSD world and kernels. As soon it is done it can shutdown if not being used anymore. The system has Wake-On-Lan support and can thus be activated again from remote.
Note 2: even this little script got flags, check the output of
sh /usr/src/tools/tools/nanobsd/nanobsd.sh -h-b suppress builds (both kernel and world) -k suppress buildkernel -w suppress buildworld -c specify config file ...
Note 3: The geometry of the cf card is defined in the nanobsd configuration file $R/cfg/nanobsd.wleiden The 'default' values are for a Peak 1 GB card. They also work on an Alix2D3 board with a 1 GB PCEngines 'blanc' cf card and 1 or 2 GB Transcend cf cards although those geometries are different. There are issues with Soekris boards, depending on BIOS version.
C. Fetch node configuration onto image, write to CF disk or remotely update
C.1. Fetch configuration
$ $R/tools/image config for <HybridNodename>
The script is connecting to the Wireless Leiden 'genesis' database. First make sure that the configuration file is up to date by clicking the 'update' button on http://sunfire.wirelessleiden.nl/wleiden/config/ Pick your situation! Note: proxy configurations are here.
You can inspect the image bij mounting as memory disk:
mdconfig -a -t vnode -f /usr/obj/nanobsd.wleiden-hybrid/_.disk.full mount /dev/md0s1a /mnt ls /mnt umount /mnt mount /dev/md0s3 /mnt ls /mnt umount /mnt mdconfig -d -u 0
Editing can also be done using the image script:
C.2. Write the correct image to CF (media based on SLC and not MLC flash seem to perform much better).
NOTE: _.disk.full is required for new CF cards as it contains two base systems and one configuration.
_.disk.image on the other hand can be used to update an existing CF card
- New image: Put full image on compact flash disk (attach a card reader/writer with a CF disk of minimum 1 GB)
Use the script
or the command line:
$ dd bs=64k if=/usr/obj/nanobsd.wleiden-hybrid/_.disk.full of=/dev/da0
/dev/da0is your compact flash entry; this takes about 15 minutes, check progress by typing Ctrl-T; you may wish to check afterwards with fdisk whether there are three partitions on the disk, you can also mount /dev/da0s1a and /dev/da0s3 and check the configuration)
- Existing image: Put partial image on slice (slice 2) (attach a card reader/writer with the CF disk containing the existing image)
$ dd bs=64k if=/usr/obj/nanobsd.wleiden-hybrid/_.disk.image of=/dev/da0s2
- Existing image remote update (slice 2) (network connection to machine 172.x.y.z required)
For remotely updating an existing configuration use: the image script$R/tools/image deploy on <node> [and reboot]
or manually:dd if=/usr/obj/nanobsd.wleiden-hybrid/_.disk.image bs=10k | ssh email@example.com /tools/updatep2
You may adjust the block size to make the transfer more efficient (additionally ssh -C is possible) and you can use Ctrl-T to check progress.
WARNING: in case of a remote update from 7.2 to 8.0 you must scp the new rc.conf.local to the node before reboot because the interface configuration statements are quite different. If you do not adjust the configuration before reboot the interfaces will not be configured after reboot and you will not be able to reach the node!(see F.4 below)
D. Check the cf card and apply last minute changes
D.1. Check cf card
While the cf card is still in your flash card reader you can check whether the image has been written correctly. Check whether you can mount the partitions, e.g.
mount /dev/da0s1a /mnt ls /mnt umount /mnt
and the configuration partition:
mount /dev/da0s3 /mnt ls /mnt
If you cannot mount the partitions, take a fresh cf card and start again with writing the image.
D.2 adjust lvrouted conf
If the node has a /28 subnet that should be routable modify the /conf/base/etc/rc.conf file:
lvrouted_flags="...... -m 28 "
(for a /24 subnet use -m 24)
Note. For an 'iLeiden'-type node you have to add also the -z flag (pointing to iLeiden internet gateway ip-addresses, separated by comma's and no spaces) here. This is now automagically done in the config step.
D.3 adjust memory disk sizes
For non-standard hard ware you may change the memory disk sizes by modifying the rc.conf.local configuration file. /dev/md0 /etc /dev/md1 /var /dev/md2 /tmp For instance a Soekris4521 has only 64 MB RAM, add for instance the following values
D.4 * now obsolete
Customized captive portal page can be added as /cfg/local/captive/include.htm The default captive portal page is defined at the end of /usr/local/www/wlportal/index.cgi
E. All done! Load the machine
Place the new CF disk in the machine and boot it up (existing nodes: just reboot), you should be good to go! Especially first time booting can take a long time, depending also on hardware. In a Soekris4521 this takes about ten minutes. You can follow the boot process using e.g. minicom or tip, via a serial connection (communication parameters: 9600 8N1).
Note 1. For subsequent nodes you can skip the build and go directly for step C, or take a look at development if you have to rebuild the image (after small changes).
Note 2. Check your bios version in case of booting problems (note: default communication setting for alix is 38400 8N1, bios should be 0.99h)
F. Applying Updates
For small changes there is a shortcut in the build (to save you some coffee ;-)). Pick the situation applicable
- First lets pump its source to the latest version
$ svn up /root/nanobsd
- Set your favorite root password to be used in the image - if different from the password defined in your setup file (see A.4 above):
$ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15`; echo $CFG_ROOT_PASSWORDIf you like a simple password, substitute the
`dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15`with your password.
- Next step depends on the kind of changes:
- Only changes in kernel options -
kernel.wleidenaltered - ETA 0:45
$ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -w -c $R/cfg/nanobsd.wleiden
- Only changes in world options -
nanobsd.wleidenaltered - ETA 2:00 - 6:00
$ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -k -c $R/cfg/nanobsd.wleiden
- _NO_ changes in build options - any other file altered - ETA 0:10
$ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -b -c $R/cfg/nanobsd.wleiden
- ONLY different configuration
$ $R/tools/image config [for node]
- Only changes in kernel options -
- Upgrading from 7.2 to 8.2 is tricky because the interface configuration statements have changed. You have to update the config file on slice 3 as well in this case.
Mount the image as memory disk:
mdconfig -a -t vnode -f /usr/obj/nanobsd.wleiden/_.disk.full mount /dev/md0s1a /mntand apply changes to lvrouted flags in rc.conf if needed, see D.2 above
umount /mntMount the configuration slice:
mount /dev/md0s3 /mntscp the rc.conf.local file to the /cfg directory on the node running the old 7.2 configuration
umount /mnt mdconfig -d -u 0
- Partition switch bug in FreeBSD8.0
There appears to be a bug in FreeBSD8.0 upgrade procedure. A work around for the partition switch problem in the upgrade procedure that works for Wireless Leiden:
- reboot ( unclear why this is needed, but otherwise it doesnot work, i.e. no switch of the active partition)
- updatep2 (or p1)
- log in, check whether fdisk has activated p2 (or p1).
- If positive go to runlevel 6 (init 6), log out.
- After a couple of minutes you can log in, welcome on the new slice!
Note we have recently modified the /tools/update scripts to use gpart instead of fdisk and boot0cfg, see the /tools/updatep scripts for details. This solved the problem.
Some handy tools are available in the /tools directory, such as
- /tools/wl-version : to get information on the configuration that is running
- /tools/image-build.sh for building images (replaces long command lines and prevents unintentional, time-consuming rebuilds; do not forget to setenv the root password!)
- Packages needed are to be found in the nanobsd/tools/package-build.sh script
$PORTSDIR/dns/dnsmasq $PORTSDIR/www/py-cherrypy $PORTSDIR/www/tinyproxy $PORTSDIR/editors/vim-lite $PORTSDIR/net-mgmt/net-snmp $PORTSDIR/net-mgmt/nrpe2 $PORTSDIR/benchmarks/iperf $PORTSDIR/net/pen $PORTSDIR/net/rsync # Wireless Leiden ports at $WL_PORTSDIR $WL_PORTSDIR/net/lvrouted $WL_PORTSDIR/net/transproxy
- Soekris 4521, 4801, 5501, bios version 1.23, 1.31b, 1.32, 1.33
- Alix 2D3, only use latest bios: 0.99h
Note on updating Soekris bios (CTRL+P to enter): no luck with minicom, used cu on linux (Ubuntu: sudo apt-get install lrzsz cu):chown uucp /dev/ttyS0 cu -l /dev/ttyS0 -s 9600 > download - Start sending file using XMODEM/CRC protocol. ~+sx -X b4501_133.bin Sending b4501_133.bin, 608 blocks: Give your local XMODEM receive command now. Bytes Sent: 77824 BPS:892 Transfer complete File downloaded succesfully, size 608 Blocks.
What changed for FreeBSD 8.0 ?
All modifications required for 8.0 are incorporated in the nodefactory version that can be found in svn: http://svn.wirelessleiden.nl/svn/code/iris/trunk/nanobsd/
Some modifications for freebsd-8.0 RELEASE going from 7.2 to 8.0:
- use of rc.conf.local in stead of rc.node.local to be more in-line with standard FreeBSD config
- apply only patch for usbdevs and if_udav.c to make Davicom usb-lan adapter work;
- no cardbus patch
- modify kernel.wleiden configuration file, see http://svn.wirelessleiden.nl/svn/projects/proxy/2.0/nanobsd/nanobsd/cfg/kernel.wleiden
# options CLK_USE_I8254_CALIBRATION # options ADAPTIVE_GIANT # Giant mutex is adaptive. # device wlan_scan_ap # 802.11 AP mode scanning. # device wlan_scan_sta # 802.11 STA mode scanning. # device ppp # Kernel PPP # device ugen # Generic # device sio
- modify ttys-nanobsd.patch file
- wireless interface configuration is quite different (configuration script was changed)
modify rc.conf.local somewhat like this:
wlans_ath0="wlan0" create_args_wlan0="wlanmode hostap mode 11b ssid ap-omni.huub.wleiden.net channel 7" ifconfig_wlan0="inet 172.17.16.1/26"note that interface has to be 'down' to create_args_wlan0!
- usb-lan convertor interface for Digitus adapter is now ue (not aue)
- known issues (see freebsd-embedded mailinglist): geometry mismatch between cf-card and disklable; boot-sector/upgrade-procedure.
- You could always decide to install your FreeBSD base instance into a virtual environment
cvsup.nl.freebsd.orgis the local dutch cvsup mirror, replace
nlwith your proper country code. An alternative way to update sources is svn, possibly with a symlink for use of different releases, e.g.:
$ svn co http://svn.freebsd.org/base/release/8.2.0/ /usr/src-release-8.2.0 $ ln -s /usr/src-release-8.2.0 /usr/src
- updating ports:
# cd hybrid-releng-9.0-nanobsd # svn up . # pkg_delete /var/db/pkg/* # rm -R /usr/local/* /var/db/ports/* /var/db/pkg/* # ./tools/image ports updatecheck list of installed packages:
# echo pkg/All/* | wc -w
- It seems tempting to follow
/usr/share/examples/cvsup/standard-supfileplease DO NOT unless your like big trouble as stable-supfile is the stable DEVELOPMENT branch e.g. upcoming stable.
- Default username/password = root/<blank>, so please do mind, _when no password is set ssh login is disabled_.
- Always use
puttyfor serial communication minicom, screen, putty all have issues of some kind.
- After writing an image to cf-card using a usb-card-reader/writer you may encounter a problem dd-ing another image. Rebooting will help. Alternatively you can apply a power-cycle to the usb device:
usbconfig -u 1 -a 2 power_off usbconfig -u 1 -a 2 power_on(unit and address may vary, see usbconfig list).
J. Comments, questions or remarks?
Feel free to edit this page or send an email to our techniek mailinglist