= Welcome to the Proxy Factory project =
== Objective ==
The aim of this project is to build a 'factory' based on Nanobsd to generate proxy images, similar to the 'nodefactory' that is used for generating node-images. We will try to incorporate the results of the [http://www.genericproxy.org/ Hogeschool Leiden project 'generic proxy'].

== Functionality ==

 * proxy server for internet browsing from wleiden.net (no access to local subnet if present), e.g. tinyproxy
 * name server, use of captive portal on nodes requires resolution of internet names, so recursive dns is required.
 * ntpd for wleiden.net
 * sshd for administrative use
 * snmpd  for monitoring
 * no browsing from wleiden.net of local webserver(s)
 * access to wleiden.net from local network
 * SSH tunnel to dellphi (backdoor and used for monitoring (nagios)
 * nmap for monitoring
 
== Previous (and other ongoing) work ==
Basic work on configuration of a proxy-PC has been done by Dennis, see http://svn.wirelessleiden.nl/svn/projects/proxy/.

There is also a howto-description for building proxy-images on the basis of the (outdated) FreeBSD6.0 configuration: http://svn.wirelessleiden.nl/svn/projects/proxy/proxy_op_soekris.txt.

Recently Richard has provided some notes on building a proxy for a Soekris-compact flash card: http://svn.wirelessleiden.nl/svn/projects/proxy/2.0/

Hogeschool Leiden is carrying out the GenericProxy project, see also [http://www.genericproxy.org Generic Proxy project website].


== Approach ==
Files will be stored in a dedicated [http://svn.wirelessleiden.nl/svn/projects/proxy/2.0 svn directory].

We will use the NodeFactory as the basis for this project, based on nanobsd, but build on FreeBSD-8.0.
Tinyproxy, ntpd, sshd, snmpd are already included in the packagelist.
 * rebuilding the nodefactory on the basis of FreeBSD-8.0
   No surprises here. We do not need the patches to the driver modules, only usbdevs.patch is still required.
 * kernel configuration needs some adjustments for 8.0. 
   notably following options should be commented out:
    * # options     CLK_USE_I8254_CALIBRATION
    * # options     ADAPTIVE_GIANT  # Giant mutex is adaptive.
    * # device      wlan_scan_ap    # 802.11 AP mode scanning.
    * # device      wlan_scan_sta   # 802.11 STA mode scanning.
    * # device      ppp             # Kernel PPP
    * # device      ugen            # Generic
    * # device      sio

 * adjustments of nanobsd.wleiden config file (required for kernel to build):
 add the option NANO_PARALLEL_MAKE="1"

 * To the genesis-configuration we have added the variable $internalroute. This is the ip-address of the node interface at the proxy site. It is uses as a
 summary route to the wleiden-network.

 * Name server: the proxies use Google nameservers e.g. 8.8.8.8 by default and search wleiden.net. Initially we have used nsd, but this has no recursive nameserver capabilities. Maradns is now under evaluation.

 * The location specific configuration file has been named rc.conf.local (in stead of rc.node.local which is used in the nodefactory) and the configuration scripts have been modified where required. The reference to rc.node.local in the rc.conf file has been removed as rc.conf.local is read by default (see rc_conf_files variable in the /etc/defaults/rc.conf file).

 * To access the wleiden netwerk from the local network one can set up a ssh-tunnel, i.e. to access the host 192.168.1.20 at some node   one can use:
   {{{
   ssh -L 8888:192.168.1.20:80 <node-ip> 
   (for webinterface: http://localhost:8888)
   or
   ssh -L 2222:192.168.1.20:22 <node-ip> 
   (for ssh connection: ssh -p 2222 root@localhost) 
   Every application (port) requires a separate ssh tunnel.
   }}}
   We have decided not to use NAT as this requires the installation of a firewall which complicates the setup. In most cases access to wleiden.net will not be required.


= Building a Proxy Factory =

Please note that you can find the supporting files in the subdirectory [source:nanobsd]. Also keep a close look at the ''INFO'' ''WARN'' ''ERR'' directives as they guide you trough the common pitfalls of the guide.  

==  A. Setup a FreeBSD host ==
----
WARN: Make sure ''/usr'' is '''at least 5GB''' in size building as building images require quite some space [[BR]]
WARN: Make sure you install the '''i386''' release of FreeBSD also when your system does support amd64, as cross compiling can give some nasty surprises [[BR]]
----
Get yourself a fresh i386 freebsd host with ports and subversion installed as follows:

 A.1. Run the basic CD installer of 8.0. 
The procedure below has been tested with 8.0-RELEASE (standard developer install - no ports - will fit our needs). Installing FreeBSD is outside the scope of this document, take a look into [http://www.freebsd.org/doc/en/books/handbook/install.html the FreeBSD handbook Chapter 2 Installing FreeBSD] if you do not know the details.

 Please do mind that all commands below need to be executed as root, due to the many mounts and unmounts done in various phases.

 Internet connection is required.

 Set correct date/time, e.g. 
 {{{
 $ ntpdate -s pool.ntp.org
 }}}

 A.2. get latest sources 
 {{{ 
 $ csup  -h cvsup.nl.freebsd.org /usr/share/examples/cvsup/standard-supfile
 }}}

 
 A.3 fetch ports
 {{{
 $ portsnap fetch extract
 }}}

 A.4 set some usefull variables 
 
 Edit the shell configuration file .cshrc:
 {{{
 vi /root/.cshrc
 }}}
 check whether ftp is set to passive mode, to avoid potential firewall issues:
 {{{
 setenv FTP_PASSIVE_MODE YES
 }}}
 set a default password for the images that you will produce
 {{{
 setenv CFG_ROOT_PASSWORD [default password]
 }}}
 and define the svn download directory:
 {{{
 setenv R /root/nanobsd
 }}}

 A.5. install subversion . 
 NOTE: Install all packages via ports to prevent issues later on!
 {{{
 $ cd /usr/ports/devel/subversion; make install clean BATCH=yes
 }}}

 A.6. OPTIONAL, every developer has his own preferences, these are mine ;-)
 {{{
 $ (cd /usr/ports/editors/vim-lite; make install clean BATCH=yes)
 $ (cd /usr/ports/security/sudo; make install clean BATCH=yes)
 $ (cd /usr/ports/sysutils/screen; make install clean BATCH=yes)
 }}}

== B. Build environment ==

 B.1 Download the environment from the Wireless Leiden svn repository
 {{{
 $ 
 $ svn checkout http://svn.wirelessleiden.nl/svn/projects/proxy/2.0/nanobsd/nanobsd $R
 $ cd $R
 }}}
 Note: if svn is not found: svn is in /usr/local/bin, alternatively log out and in, or use {{{rehash}}} in a {{{csh}}} shell to make it available.

 B.2. Compile all required packages using
 {{{
 $ /root/nanobsd/tools/package-build.sh
 }}}

 B.3. Set your favorite root password to be used in the image 

 Note: you can skip this step if you are satisfied with the default password set in step A.4 above.

 {{{ 
 $ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15`
 $ echo $CFG_ROOT_PASSWORD
 }}} 
 If you like a simple password, substitute the {{{ `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15` }}} with your password.

 B.4 Apply kernel patch for USB-LAN adapter

 For use of a Davicom DM9601 USB network controler you have to patch usbdevs and the if_udav driver:
 {{{
 cd /usr/src/sys/dev/usb/
 patch < /root/nanobsd/misc/patches/usbdevs.patch
 patch < /root/nanobsd/misc/patches/if_udav.c.patch
 }}}

 B.5. Build nanobsd (make sure to prepare some coffee;-) )
 {{{ 
 $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -c /root/nanobsd/cfg/nanobsd.wleiden 
 }}}
 Note 1: Take a coffee of go for a hike, this normally takes 2 - 8 hours depending on the machine configuration

 Note 2: even this little script got flags, check the output of {{{sh /usr/src/tools/tools/nanobsd/nanobsd.sh -h}}}
 {{{
-b suppress builds (both kernel and world)
-k suppress buildkernel
-w suppress buildworld
-c specify config file
...
 }}}

 Note 3: The geometry of the cf card is defined in the nanobsd configuration file /root/nanobsd/cfg/nanobsd.wleiden
 The 'default' values are for a Peak 1 GB card. They also work on an Alix board (but ''' NOT ON Soekris! ''' ) for a 1 GB PCEngines 'blanc' cf card although its geometry is slightly different. 

== C. Fetch proxy configuration onto image, write to CF disk or remotely update ==

C.1. Fetch the configuration
 First make sure that the configuration file is up to date by clicking the 'update' button on http://wlconfig.wirelessleiden.nl/freebsd/proxy-8.0/ .

 {{{ 
 $ /root/nanobsd/tools/config-image.sh 
 }}}

The script is connecting to the [http://svn.wirelessleiden.nl/svn/node-config/genesis/nodes/ Wireless Leiden 'genesis' database]. Pick your situation!

You can inspect the image bij mounting as memory disk:
{{{
mdconfig -a -t vnode -f /usr/obj/nanobsd.wleiden/_.disk.full
mount /dev/md0s1a /mnt
ls /mnt
umount /mnt
mount /dev/md0s3 /mnt
ls /mnt
umount /mnt
mdconfig -d -u 0
}}}
After this you may have to reboot the machine to release the md.

C.2. Write the correct image to CF
----
    ''NOTE:'' '''''_.disk.full''''' is required for '''new CF cards''' as it contains two base systems and one configuration. [[BR]]
              '''''_.disk.image''''' on the other hand can be used to update an '''existing CF card''' 
----

    a. '''New image''': Put full image on compact flash disk (attach a card reader/writer with a CF disk of minimum 1 GB)
    {{{
    $ dd bs=64k if=/usr/obj/nanobsd.wleiden/_.disk.full of=/dev/da0
    }}} 
    (assuming {{{/dev/da0}}} is your compact flash entry; this takes about 15 minutes; you may wish to check afterwards with fdisk whether there 
    are three partitions on the disk, you can also mount /dev/da0s1a and /dev/da0s3 and check the configuration)

    a. '''Existing image''': Put partial image on slice (slice 2) (attach a card reader/writer with the CF disk containing the existing image)
    {{{
    $ dd bs=64k if=/usr/obj/nanobsd.wleiden/_.disk.image of=/dev/da0s2
    }}}

    a. '''Existing image remote update''' (slice 2) (network connection to machine 172.x.y.z required)

    For remotely updating an existing configuration use:
    {{{
    dd if=/usr/obj/nanobsd.wleiden/_.disk.image bs=10k | ssh root@172.x.y.z /tools/updatep2
    }}}
  
   You may adjust the block size to make the transfer more efficient (additionally ssh -C is possible) and you can use Ctrl-T to check progress.
 
== D. Check the cf card and apply last minute changes ==

While the cf card is still in your flash card reader you can check whether the image has been written correctly. Check whether you can mount the partitions, e.g.
{{{
mount /dev/da0s1a /mnt
ls /mnt
umount /mnt
}}}
and the configuration partition:
{{{
mount /dev/da0s3 /mnt
ls /mnt
}}}

If you can not mount the partitions, take a fresh cf card and start again with writing the image.


 == E. All done! Load the machine ==
 Place the new CF disk in the machine and boot it up (existing nodes: just reboot), you should be good to go! Especially first time booting can take a long time (ssh-keys are generated), depending also on hardware. In a Soekris4521 this takes about ten minutes. You can follow the boot process using e.g. minicom or tip, via a serial connection (communication parameters: 9600 8N1).

  Note 1. For subsequent nodes you can skip the build and go directly for step C, or take a look at development if you have to rebuild the image (after small changes).

  Note 2. Check your bios version in case of booting problems (note: default communication setting for alix is 38400 8N1, bios should be 0.99h) 
 
== F. Applying Updates ==
 For small changes there is a shortcut in the build (to save you some coffee ;-)). Pick the situation applicable
 1. First lets pump it's source to the latest version 
 {{{
 $ svn up /root/nanobsd
 }}}
 2. Set your favorite root password to be used in the image 
 {{{ 
 $ setenv CFG_ROOT_PASSWORD `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15`; echo $CFG_ROOT_PASSWORD
 }}} 
 If you like a simple password, substitute the {{{ `dd if=/dev/random bs=10k count=10 | & tr -cd '[a-zA-Z0-9]' | cut -c -15` }}} with your password.
 3. Next step depends on the kind of changes:
    a. Only changes in kernel options - `kernel.wleiden` altered - ETA 0:45
    {{{
    $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -w -c /root/nanobsd/cfg/nanobsd.wleiden
    }}}
    a. Only changes in world options - `CONF_{BUILD,COMMON,INSTALL}` in  `nanobsd.wleiden` altered - ETA 2:00 - 6:00
    {{{
    $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -k -c /root/nanobsd/cfg/nanobsd.wleiden
    }}}
    a. _NO_ changes in build options - any other file altered - ETA 0:10
    {{{
    $ sh /usr/src/tools/tools/nanobsd/nanobsd.sh -b -c /root/nanobsd/cfg/nanobsd.wleiden
    }}} 

= G, Development =

== Handy tools ==
Some handy tools are available in the /tools directory, such as
 * /tools/wl-version : to get information on the configuration that is running
 * /tools/image-build.sh for building images (replaces long command lines and prevents unintentional, time-consuming rebuilds; do not forget to '''setenv the root password'''!) 



== About Trac ==


 * TracGuide --  Built-in Documentation
 * [http://trac.edgewall.org/ The Trac project] -- Trac Open Source Project
 * [http://trac.edgewall.org/wiki/TracFaq Trac FAQ] -- Frequently Asked Questions
 * TracSupport --  Trac Support

For a complete list of local wiki pages, see TitleIndex.