|
|
|
Minimal Linux Proxy
Introduction
I've always liked the idea of having a seperate internet gateway system or proxy server. When you have a few systems networked together, such a setup is quite useful. I recently upgraded to a higher speed ISP connection. It is a terrestrial microwave connection. The basics of it are:
- You have a PC with a modem and an internal satellite card.
- The modem dials the ISP and connects as usual. All outbound requests go via the modem. However, most inbound traffic is sent back to you via a dish on the side of your house ... and then into your satellite card (I think DNS lookups actually come back via the modem though). This gives you high download speed and crappy modem speed for uploads.
The use of an internal satellite card was bound to make this difficult to do in Linux but the card manufacturer Telemann do make Linux drivers available. In addition to that I wanted to use an internal Lucent Winmodem for my modem. If this complicated setup weren't enough, I also wanted to achieve this proxy on an old P75 with 16MB and a 500mb hard disk. Impossible?? (I think Lucent recommend at least a P133).
First steps
Initially I loaded Slackware 7.1 onto the P75 system. I prefer Slackware and you can be quite selective about what gets installed. I loaded the basics of a console only system (no X) with gcc. I think it came in around 200mb from memory. From their I knew I was going to have to update the kernel. I downloaded the 2.2.16 kernel since I knew that I could set up the satellite card drivers for it. After the kernel compile I was up close to 300mb. The P75 initially had an ISA 3c509 LAN card, but has since had a PCI 100Mbps RTL8139 put in (as well).
I used the freely available 5.68 Lucent modem driver. Now this does not work on kernels above 2.2.13 (or .14??), so I had to modify the tty.h include in the kernel, and move the poll_wait to the end of the structure.
I set up the LAN cards and ppp driver to load as modules. The sm200d (satellite card) driver loads as a module and the Lucent driver as well.
I won't go on any further about the ISP specific setup, suffice to say I got it all going. I even set up pppd demand dialling so I just had to ping a remote address and the modem would start dialling. I also set up ipchains so that it would restrict access across the gateway (now firewall).
Towards a minimal install
The Slackware gateway I'd setup was fine. It did the job and seemed to work OK. However, there were a few further goals to achieve:
- Use something other than ext2 for the filesystem so that you can just 'turn it on and off'.
- Cut down the size of the required disk space, just so that its easy to rebuild.
An alternative file system
I had pretty much made my mind up to try ReiserFS with this. With its journalling, I thought you should theoretically be able to just turn it off and the next time it starts, it rolls back and does what it needs to do (quickly) to get the FS to a consistent state.
Being a 2.2.16 kernel, I needed to directly patch the kernel using the patches from the ReiserFS site. I also downloaded and compiled the ReiserFS utils so that I could do a 'mkreiserfs'. My 500mb hard drive was filling up quickly with all this compiling, but I was able to use GNU Parted to shrink the main ext2 partition down by 50mb so I could create my new ReiserFS partition to play with.
The first thing to do was to format my new 50mb partition with mkreiserfs. I did this, but got quite a surprise when I mounted it for the first time. Out of my 50mb partition only 20mb was available for files. 30mb had been chewed up by ReiserFS's 'overheads'. I accepted this as the cost of having journalling and moved on. I also guessed that the total filespace requirement for my gateway should be quite a bit less than 50mb anyway.
Creating the reduced build
Having mucked around with building my own distribution a bit, I knew quite well what was required to get a Linux system 'bootable'. At this point the easiest way was to use my existing setup in Slackware as a base and just copy across the required files.
To simplify I copied my entire /lib tree and /dev tree and then hand picked files from /bin, /usr/bin, /sbin and /etc as required. An important consideration was to setup the correct directory structures under /var (eg. /var/lock to allow pppd to do locking correctly).
At the end of my hand copying I had only used up about 10mb. If I took more time and used small versions of certain files (eg. Busybox versions), then I'm sure I could get it much smaller. Since I'm only experimenting at this stage it would have to do.
I had some fun initially getting it to boot. I had the requisite LILO 21-6.1 (??). I can't quite remember what I did, but I got it to boot and it all seemed to go quite well.
ReiserFS log rollbacks!
Unfortunately, my ideas of simply turning it off and on at my leisure weren't quite successful. Sure I could turn it off and it would start up correctly after I turned it on, BUT ReiserFS decided to take quite a long time to rollback and correct itself. I had always thought that Journalling filesystems could rollback quickly and get going. Not in my case. With my setup, a crappy old 500mb harddrive and slow P75, the rolling back seemed to be taking forever. It would have been just as quick having a ext2 filesystem fsck itself.
Using a read-only Filesystem
Given that my ReiserFS idea wasn't so succesful, I decided on trying an ext2 filesystem, but only mounting it read-only. This way there would definately be no file damage when I turned it off. Even though logging to disk would be nice, in the multitude of uses, I thought no logging would be fine.
Problems with running Linux read-only
A complete Linux system doesn't really like having no write access to its filesystems. The main case for this is device files. For example, when a user logs in on the tty1 virtual console on a Linux system, the login process tries to 'chown' the /dev/tty1 device to the user that is logging in. If the /dev/tty1 file is in a read-only filesystem, the user won't be allowed to log in. Quite a problem really. Another case is the syslogd daemon. When it starts up it actually creates the /dev/log device (which is obviously not possible in a read only file system. And I knew that pppd usually likes to create a lock file under /var/lock.
Using a partial ram disk
My solution to all this was the following:
- Use the same basic file structure as my ReiserFS based build
- Use an ext2 filesystem for /
- Leave the / filesystem mounted read-only when the system comes up (ie. don't remount / read/write)
- Create a small RAM disk as the system boots and mount it as /var
- Create the appropriate directory structure under /var
- Create a /var/dev to put specific read/writable device files into
- Replace the read/write devices with soft-links into /var/dev (this needs to be done from a seperate session prior to the first boot).
eg.
ls -l /dev/tty1
lrwxrwxrwx 1 root root 13 Apr 6 10:08 /dev/tty1 -> /var/dev/tty1
ls -l /var/dev/tty1
crw--w--w- 1 root root 4, 1 Apr 7 11:13 /var/dev/tty1
- Similarly for /dev/log, Just create a soft-link in /dev, but use the -p option of syslogd in order to create its device file in a different place.
eg.
ls -l /dev/log
lrwxrwxrwx 1 root root 12 Apr 6 10:55 /dev/log -> /var/dev/log
Start syslogd as:
/sbin/syslogd -p /var/dev/log
Success
Using the read-only filesystem setup works quite well. I still fsck the filesystem as the system boots, but it never finds any faults so it boots quite quickly.
Directory trees for the Read-Only Build
/dev
lrwxrwxrwx 1 root root 4 Apr 6 10:20 X0R -> null
crw------- 1 root tty 5, 1 Apr 7 11:13 console
lrwxrwxrwx 1 root root 11 Apr 6 10:29 core -> /proc/kcore
lrwxrwxrwx 1 root root 13 Apr 6 10:18 fd -> /proc/self/fd
brw-r----- 1 root disk 3, 0 Mar 25 23:28 hda
brw-r----- 1 root disk 3, 1 Mar 25 23:28 hda1
brw-r----- 1 root disk 3, 2 Mar 25 23:28 hda2
brw-r----- 1 root disk 3, 20 Mar 25 23:28 hda20
brw-r----- 1 root disk 3, 3 Mar 25 23:28 hda3
brw-r----- 1 root disk 3, 4 Mar 25 23:28 hda4
brw-r----- 1 root disk 3, 5 Mar 25 23:28 hda5
brw-r----- 1 root disk 3, 6 Mar 25 23:28 hda6
brw-r----- 1 root disk 3, 7 Mar 25 23:28 hda7
brw-r----- 1 root disk 3, 8 Mar 25 23:28 hda8
brw-r----- 1 root disk 3, 9 Mar 25 23:28 hda9
prw------- 1 root root 0 Apr 6 10:53 initctl
crw-r----- 1 root kmem 1, 2 Mar 25 23:28 kmem
lrwxrwxrwx 1 root root 12 Apr 6 10:55 log -> /var/dev/log
crw-r----- 1 root kmem 1, 1 Mar 25 23:28 mem
crw-r--r-- 1 root root 1, 3 Mar 25 23:28 null
crw-r--r-- 1 root root 5, 2 Apr 7 11:48 ptmx
drwxr-xr-x 2 root root 0 Apr 7 2001 pts
brw-r----- 1 root disk 1, 0 Mar 25 23:28 ram0
brw-r----- 1 root disk 1, 1 May 15 1996 ram1
lrwxrwxrwx 1 root root 4 Apr 6 10:18 stderr -> fd/2
lrwxrwxrwx 1 root root 4 Apr 6 10:19 stdin -> fd/0
lrwxrwxrwx 1 root root 4 Apr 6 10:19 stdout -> fd/1
lrwxrwxrwx 1 root root 7 Apr 6 10:19 systty -> console
crw-r--r-- 1 root tty 5, 0 Apr 7 11:15 tty
lrwxrwxrwx 1 root root 13 Apr 6 10:08 tty1 -> /var/dev/tty1
lrwxrwxrwx 1 root root 13 Apr 6 10:51 tty2 -> /var/dev/tty2
lrwxrwxrwx 1 root root 15 Apr 6 10:09 ttyS14 -> /var/dev/ttyS14
crw-r--r-- 1 root root 1, 5 Mar 25 23:28 zero
Notes: I have hda1 to 9, but I only have 3 partitions, hda1 to 3 would have been OK. I'm using the /dev/pts file system, so I have ptmx and the pts mount point.ttyS14 is the device used by the Lucent modem. At the moment only ram0 is used.
/bin
-rwxr-xr-x 1 root root 477692 Mar 25 22:32 bash
-rwxr-xr-x 1 root root 10332 Mar 25 23:58 cat
-rwxr-xr-x 1 root root 11540 Mar 26 09:42 chmod
-rwxr-xr-x 1 root root 27188 Mar 25 22:46 cp
-rwxr-xr-x 1 root root 12436 Mar 25 23:59 cut
-rwxr-xr-x 1 root root 20492 Mar 25 22:33 df
-rwxr-xr-x 1 root root 8760 Mar 26 00:18 hostname
-rwxr-xr-x 1 root root 13904 Mar 25 22:45 ln
-rwxr-xr-x 1 root root 47880 Mar 25 22:35 login
-rwxr-xr-x 1 root root 47876 Mar 25 22:33 ls
-rwxr-xr-x 1 root root 11356 Mar 26 09:49 mkdir
-rwxr-xr-x 1 root root 10108 Apr 5 12:38 mknod
-r-sr-xr-x 1 root root 14772 Mar 26 09:58 ping
-rwxr-xr-x 1 root root 58140 Mar 26 09:55 ps
-rwxr-xr-x 1 root root 19340 Mar 25 23:58 rm
-rwxr-xr-x 1 root root 21012 Mar 26 00:18 setterm
lrwxrwxrwx 1 root root 4 Apr 5 11:53 sh -> bash
-rwxr-xr-x 1 root root 5528 Mar 26 09:42 sync
-rwxr-xr-x 1 root root 93188 Mar 26 09:53 telnet
-rwxr-xr-x 1 root root 6876 Mar 25 23:58 uname
Notes: I decided to use bash out of meer convenience (a command like history is handy). Busybox could have easily been used here instead.
/etc
-rw-r--r-- 1 root root 5 Mar 26 00:19 HOSTNAME
-rw-r--r-- 1 root root 28 Mar 26 00:00 adjtime
-rw-r--r-- 1 root root 0 Mar 26 00:23 fastboot
-rw-r--r-- 1 root root 226 Apr 5 11:55 fstab
-rw-r--r-- 1 root root 316 Mar 25 22:36 group
-rw-r--r-- 1 root root 745 Apr 6 10:31 hosts
-rw-r--r-- 1 root root 318 Mar 26 09:26 hosts.allow
-rw-r--r-- 1 root root 124 Mar 26 09:21 inetd.conf
-rw-r--r-- 1 root root 2781 Mar 25 22:34 inittab
-rw------- 1 root root 60 Apr 5 11:58 ioctl.save
-rw-r--r-- 1 root root 27 Apr 5 11:58 issue
-rw-r--r-- 1 root root 3276 Apr 5 11:58 ld.so.cache
-rw-r--r-- 1 root root 223 Mar 25 23:49 lilo.conf
lrwxrwxrwx 1 root root 22 Apr 5 11:53 localtime -> /usr/share/z
oneinfo/NZ
-rw-r--r-- 1 root root 10214 Mar 25 22:43 login.defs
-rw-r--r-- 1 root root 98 Mar 26 10:48 minirc.dfl
-rw-r--r-- 1 root root 14 Apr 5 11:58 motd
-rw-r--r-- 1 root root 24 Apr 5 11:59 mtab
-rw-r--r-- 1 root root 232 Mar 25 22:43 networks
-rw-r--r-- 1 root root 1108 Mar 25 22:37 nsswitch.conf
-rw-r--r-- 1 root root 28 Mar 26 15:58 passwd
drwxr-xr-x 2 root root 1024 Mar 26 16:02 ppp
-rw-r--r-- 1 root root 1101 Mar 26 00:02 profile
-rw-r--r-- 1 root root 595 Mar 25 22:43 protocols
drwxr-xr-x 2 root root 1024 Apr 5 12:02 rc.d
-rw-r--r-- 1 root root 92 Apr 5 17:29 resolv.conf
-rw-r--r-- 1 root root 5924 Mar 26 09:43 services
-rw-r--r-- 1 root root 52 Apr 5 12:14 shadow
-rw------- 1 root root 52 Mar 26 15:58 shadow-
-rw-r----- 1 root root 619 Mar 26 09:30 syslog.conf
-rw-r--r-- 1 root root 7881 Mar 25 23:04 termcap
Notes: The contents of many of these config files is shown later in this document.
/etc/rc.d
-rwxr-xr-x 1 root root 374 Mar 26 09:55 rc.6
-rwxr-xr-x 1 root root 1316 Apr 6 10:34 rc.M
-rwxr-xr-x 1 root root 1199 Apr 6 10:32 rc.S
-rwxr-xr-x 1 root root 360 Mar 26 15:48 rc.local
-rwxr-xr-x 1 root root 404 Mar 26 00:09 rc.modules
/etc/ppp
-rw------- 1 root root 263 Apr 5 12:50 chap-secrets
-rw------- 1 root root 0 Jan 12 08:55 connect-errors
-rwxr-xr-x 1 root root 1209 Dec 14 14:38 ip-down
-rwxr-xr-x 1 root root 1209 Dec 14 11:53 ip-down.OLD
-rwxr-xr-x 1 root root 1995 Mar 16 10:55 ip-up
-rwxr-xr-x 1 root root 1946 Dec 14 11:53 ip-up.OLD
-rw------- 1 root root 548 Mar 26 16:02 options
-rw------- 1 root root 704 Apr 5 12:49 options.demand
-rw------- 1 root root 211 Apr 5 12:50 pap-secrets
-rw------- 1 root root 127 Mar 25 17:30 pppscript
/lib
At the moment, /lib is simply a complete copy of all the libs under /lib on a standard Slackware 7.1 install. /lib/modules contains modules for the lan cards; 3c509 and rtl8139, the ppp modules, the ltmodem module and all the ipmasq helper modules.
/sbin
-rwxr-xr-x 1 root root 13844 Mar 25 22:35 agetty
-rwxr-xr-x 1 root root 39456 Mar 26 00:10 depmod
-rwxr-xr-x 1 root root 75860 Apr 5 12:08 e2fsck
-rwxr-xr-x 1 root root 12900 Apr 5 12:09 fsck
-rwxr-xr-x 1 root root 36 Apr 5 12:09 fsck.ext2
-rwxr-xr-x 1 root root 34952 Mar 25 23:58 hwclock
-rwxr-xr-x 1 root root 31992 Mar 26 00:26 ifconfig
-rwxr-xr-x 1 root root 339576 Mar 25 22:30 init
-rwxr-xr-x 1 root root 72860 Mar 26 09:59 insmod
-rwxr-xr-x 1 root root 38008 Mar 26 10:07 ipchains
-rwxr-xr-x 1 root root 8644 Mar 26 09:54 killall5
-rwxr-xr-x 1 root root 109628 Mar 26 09:49 ldconfig
-rwxr-xr-x 1 root root 87924 Mar 25 23:02 lilo
-rwxr-xr-x 1 root root 72860 Mar 26 00:10 lsmod
-rwxr-xr-x 1 root root 19812 Apr 5 11:54 mke2fs
-rwxr-xr-x 1 root root 72860 Mar 26 00:10 modprobe
-rwsr-xr-x 1 root root 60912 Mar 25 22:34 mount
-rwxr-xr-x 1 root root 7576 Mar 26 00:17 reboot
-rwxr-xr-x 1 root root 31064 Mar 26 00:26 route
-rwxr-xr-x 1 root root 14744 Mar 26 00:17 shutdown
-rwxr-xr-x 1 root root 16416 Apr 5 12:06 sulogin
-rwxr-xr-x 1 root root 6956 Mar 26 09:41 swapoff
-rwxr-xr-x 1 root root 6956 Mar 26 00:05 swapon
-rwsr-xr-x 1 root root 28588 Mar 25 22:34 umount
/usr/bin
-rws--x--x 1 root root 29572 Mar 26 15:38 chfn
-rwxr-xr-x 1 root root 303420 Mar 25 22:46 elvis
-rwx--s--x 1 root root 164312 Mar 26 10:18 minicom
-rws--x--x 1 root root 35620 Mar 25 22:36 passwd
-rwxr-xr-x 1 root root 6584 Mar 26 09:25 sleep
lrwxrwxrwx 1 root root 5 Apr 5 11:53 vi -> elvis
-rwxr-xr-x 1 root root 2129 Mar 26 15:37 adduser
-rwxr-xr-x 1 root root 16916 Mar 26 10:10 chat
-rwxr-xr-x 1 root root 31276 Mar 26 09:53 in.telnetd
-rwxr-xr-x 1 root root 21224 Mar 26 09:19 inetd
-rwxr-xr-x 1 root root 20280 Mar 26 09:25 klogd
-rwxr-xr-x 1 root root 144980 Mar 26 10:09 pppd
-rwxr-xr-x 1 root root 27684 Mar 26 09:25 syslogd
-rwxr-xr-x 1 root root 21744 Mar 26 09:22 tcpd
-rwxr-xr-x 1 root root 50548 Mar 26 15:37 useradd
Notes: The useradd, adduser is left over from when I was trying to set up the root user password. tcpd is there to allow restricted telnet access.
Startup Files
inittab
id:3:initdefault:
si:S:sysinit:/etc/rc.d/rc.S
su:1S:wait:/etc/rc.d/rc.K
rc:2345:wait:/etc/rc.d/rc.M
ca::ctrlaltdel:/sbin/shutdown -t5 -rf now
l0:0:wait:/etc/rc.d/rc.0
l6:6:wait:/etc/rc.d/rc.6
c1:1235:respawn:/sbin/agetty 38400 tty1 linux
c2:1235:respawn:/sbin/agetty 38400 tty2 linux
Notes: The standard progression is to run rc.S then rc.M. rc.0 and rc.6 are actually the same script, soft linked together. i really should create a rc.K, as going to single user is really the only way of being able to remount the / filesystem read/write.
/etc/rc.d/rc.S
PATH=/sbin:/usr/sbin:/bin:/usr/bin
# enable swapping
/sbin/swapon -a
echo "junk" >/tagfile 2>/dev/null
if [ -f /tagfile ];then
echo "Root is already mounted readwrite"
rm -f /tagfile
else
fsck -A -a
fi
echo -n "Make /var ... "
/sbin/mke2fs -m 0 /dev/ram0 >/dev/null 2>&1
if [ $? -eq 0 ];then
echo "OK"
mount -t ext2 /dev/ram0 /var
chmod 777 /var
mkdir /var/run /var/log /var/adm /var/tmp /var/lock /var/dev
mknod /var/dev/tty1 c 4 1
mknod /var/dev/tty2 c 4 2
mknod /var/dev/ttyS14 c 62 78
else
echo "FAIL"
fi
# mount file systems in fstab (and create an entry for /)
# but not NFS because TCP/IP is not yet configured
/sbin/mount -a -v -t nonfs
cat /dev/null > /var/run/utmp
echo "Set system time from hardware clock ... "
/sbin/hwclock --hctosys
[ $? -eq 0 ] && echo "OK" || echo "FAIL"
# This loads any kernel modules that are needed. These might be required to
# use your CD-ROM drive, bus mouse, ethernet card, or other optional hardware.
if [ -x /etc/rc.d/rc.modules ]; then
. /etc/rc.d/rc.modules
fi
Notes: I've compiled into the 2.2.16 kernel to use 512Kbyte ram disks rather than the default of 4096Kbyte. There is no point in wasting the small amount of ram available to the gateway. Creating a 512K file system takes a fraction of a second and all that needs to be done is to create the directory structure and devices.
/etc/rc.d/rc.M
#!/bin/sh
echo "Going multiuser..."
# Screen blanks after 15 minutes idle time.
/bin/setterm -blank 15
# Set the hostname. This might not work correctly if TCP/IP is not
# compiled in the kernel.
/bin/hostname `cat /etc/HOSTNAME | cut -f1 -d .`
HOSTNAME="`cat /etc/HOSTNAME`"
/sbin/ifconfig lo 127.0.0.1
/sbin/route add -net 127.0.0.0 netmask 255.0.0.0 lo
/sbin/ifconfig eth0 10.0.0.5 broadcast 10.0.0.255 netmask 255.255.255.0
[ $? -eq 0 ] && echo "eth0 configured OK" || echo "eth0 FAIL"
/sbin/ifconfig eth1 10.0.1.5 broadcast 10.0.1.255 netmask 255.255.255.0
[ $? -eq 0 ] && echo "eth1 configured OK" || echo "eth1 FAIL"
echo "Activate IPv4 packet forwarding"
echo 1 > /proc/sys/net/ipv4/ip_forward
echo -n "Start syslogd ... "
/usr/sbin/syslogd -p /var/dev/log
[ $? -eq 0 ] && echo "OK" || echo "FAIL"
sleep 1
echo -n "Start klogd ... "
/usr/sbin/klogd -c 3
[ $? -eq 0 ] && echo "OK" || echo "FAIL"
echo -n "Start inetd ... "
/usr/sbin/inetd
[ $? -eq 0 ] && echo "OK" || echo "FAIL"
# Remove stale locks and junk files (must be done after mount -a!)
/bin/rm -f /var/lock/* /tmp/.X*lock /tmp/core 1> /dev/null 2> /dev/null
# Update all the shared library links automatically
#/sbin/ldconfig
# Start the local setup procedure.
if [ -x /etc/rc.d/rc.local ]; then
. /etc/rc.d/rc.local
fi
# All done.
Notes: Here the network interfaces are set up (remember I have two lan cards), syslogd and klogd are started and the last part is to run rc.local
/etc/rc.d/rc.local
#!/bin/sh
#
# /etc/rc.d/rc.local: Local system initialization script.
#
# Put any local setup commands in here:
echo "Starting ltmodem"
/sbin/insmod -f ltmodem
echo "Starting Skymedia"
/usr/local/sm200d/skymedia start
echo "Starting IP Masquerade and Firewall"
/usr/local/bin/set-ipchains
echo "Starting pppd dial on demand"
pppd file /etc/ppp/options.demand
Notes:Add the modem module, start the sm200d satellite driver, set some basic firewall rules and start pppd in demand dialing mode.
Apr 23,2001
|