5/5/2003 What we're going to put in here now is the update of how the system works now. The base test hardware is the PCM-5923 from Advantech. We built a standard LinuxBIOS for this board that uses Etherboot to load the 9load image from Compact Flash. I can send you the flash ROMIMAGE on request so that rather than fool around with LinuxBIOS you can just use our image.

We've had to make fixes to 9load and we are now trying to back those changes out as much as possible to minimize 9load variance.

The parts you need to replicate our setup:

• Advantech PCM-5823
• Any kind of Compact Flash > 1 MB in size
• Replacement flash part (SST 28SF040 is what we use)
• If you are building LinuxBIOS, a Linux box. If you are not, a way to reprogram the FLASH and a copy of our flash image.

If you just the flash image from us, we can also send the working 9load image (which you burn onto the CF) and a working Plan 9 kernel image which you tftp download. This kernel knows how to use CMOS for its nvram.

Base document follows:

LinuxBIOS can load lots of different "payloads", i.e. ELF images in FLASH. In this document we will go over how we got 9load into FLASH, and thus made nodes that can boot as dedicated Plan 9 CPU nodes.

NOTE: also check (http://www.acl.lanl.gov/plan9/flash/(broken link!)) for additional information!

This documentation pertains to the setup, configuration and means of getting Plan 9 booted with LinuxBIOS. It consists of the following parts:

• 0. Preamble
• 1. Development setup -- getting started. file organization in Plan9 and Linux
• 2. Source code explanation -- diffs with 4th edition p9 source with explanation
• 3. Source code -- tarred & gzipped

• 1.1 Plan9 network (including file servers, cpu servers and terminals (http://pages.cpsc.ucalgary.ca/~mirtchov/lanlp9/)
• 1.1.1 It is possible to work from a standalone P9 terminal, though this setup would be trickier
• 1.2 Plan9 source (usually comes with access to the P9 network and/or a P9 installation)
• 1.3 A Linux machine (used to compile the LinuxBIOS part)
• 1.4 Some hardware that LinuxBIOS works with
• 1.(5-9) Lots of patience

Plan9 setup:

The setup for P9 uses the fact that the operating system supports union directories. What that gives us is the ability to store only those files we modify in our local directory, and bind them when necessary on top of the standard kernel tree (in /sys/src/)... In fact this results in a carefully crafted namespace that could be used in a single terminal window (or, if so willing in an entire rio session on a cpu server) to present a decent development environment, while the rest of the screen, session or terminals keep the common, non-development view of the system. Ultimately this allows for several different developments to go on at the same time, (example will be shown in a bit) -- compiling kernels in one window, 9load bootloaders in another window, while being able to see the normal, unmodified kernel tree in a third window for comparison. All in the same directory.

How it actually works:

In brief, LinuxBIOS starts up on the hardware, find and loads Plan9's bootloader (9load masked as an elf32 image so it can be recognized by LinuxBIOS), 9load then prompts for kernel or attempts to etherboot one, downloads it, uncompresses it and jumps to it. Voila.

Details follow:

You compile (make) a 9load image tailored to work with

LinuxBIOS:

p9% lbbind # bind the development tree...
p9% mk 9loadlb # in the mkfile there's a special
8a -DLinuxBios -o ll.8 l.s # configuration for linuxbios
8c -FVw -I. alarm.c # kernels carrying the necessary
8c -FVw -I. cga.c # #define's and linking 9load
8c -FVw -I. clock.c # at 0x800000
8c -FVw -I. console.c
8c -FVw -I. dosboot.c
8c -FVw -I. donprint.c
8c -FVw -I. devfloppy.c
8c -FVw -I. dma.c
8c -FVw -I. ilock.c
8c -FVw -I. kbd.c
8c -FVw -I. queue.c
8c -FVw -I. trap.c
8c -FVw -I. 8250.c
warning: 8250.c:130 param declared and not used: arg
8c -FVw -I. apm.c
8c -FVw -I. boot.c
8c -FVw -I. devpccard.c
8c -FVw -I. conf.c
8c -FVw -I. devi82365.c
8c -FVw -I. devsd.c
8c -FVw -I. inflate.c
8c -FVw -I. load.c
warning: load.c:159 no return at end of function: myboot
load.c:280 function args not checked: pcihinv

8c -FVw -I. memory.c
8c -FVw -I. part.c
8c -FVw -I. pci.c
8c -FVw -I. sdata.c
8c -FVw -I. sdmylex.c
8c -FVw -I. sd53c8xx.c
8c -FVw -I. sdscsi.c
8c -FVw -I. bootp.c
8c -FVw -I. ether.c
8c -FVw -I. ether2114x.c
8c -FVw -I. ether2000.c
8c -FVw -I. ether589.c
8c -FVw -I. ether79c970.c
8c -FVw -I. ether8003.c
8c -FVw -I. ether8139.c
8c -FVw -I. ether82557.c
8c -FVw -I. ether83815.c
8c -FVw -I. ether8390.c
8c -FVw -I. etherec2t.c
8c -FVw -I. etherelnk3.c

8l -o 9loadlb -H3 -T0x80800000 -l ll.8 alarm.8 cga.8 clock.8
console.8 dosboot.8 donprint.8 devfloppy.8 dma.8 ilock.8 kbd.8
queue.8 trap.8 8250.8 apm.8 boot.8 devpccard.8 conf.8 devi82365.8
devsd.8 inflate.8 load.8 memory.8 part.8 pci.8 sdata.8 sdmylex.8
sd53c8xx.8 sdscsi.8 bootp.8 ether.8 ether2114x.8 ether2000.8
ether589.8 ether79c970.8 ether8003.8 ether8139.8 ether82557.8
ether83815.8 ether8390.8 etherec2t.8 etherelnk3.8 -lflate -lc

ls -l 9loadlb

--rwxr-xr-x M 9 andrey andrey 178320 Jun 20 11:34 9loadlb

p9%

9loadlb is a special configuration in the mkfile that will

#define 'LinuxBios' during the compilation and will link 9loadlb at
0x800000 (that's where LinuxBIOS will jump to).

Next step is to copy 9loadlb to the linux machine (usually you would
scp it a level below the mkelf/ directory in snaresland):

fbsd$ scp plan9:src/boot/pc/9loadlb ../9load.800000

andrey@plan9.acl.lanl.gov's password:

9loadlb 100% |*****************************| 174 KB 00:00

fbsd$

Then run the 'mymk' script in mkelf:

snares$ pwd
/users/andrey/kern/mkelf
snares$ ./mymk
#!/bin/sh -v
cd 9load
cp ~/kern/9load.800000 .
make all4
./mkas < 9load.800000 > 9l.s
as -o 9l.o 9l.s
ld -T elfImage.lds.800000 -o 9l.elf 9l.o
ld: warning: cannot find entry symbol startup_32; defaulting to
00800000
cp 9l.elf ..
cd ..

bin/mkelfImage --kernel=9l.elf

Running gcc -O2 -DDEFAULT_ROOT_DEV='(((0x3<<8)| 0))'
-DDEFAULT_COMMAND_LINE='""' -DDEFAULT_PROGRAM_VERSION='""' -c
/users/andrey/kern/mkelf/share/mkelfImage/elf32-i386/convert_params.c -o /am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/convert_params_6384.o

Running gcc -O2 -DDEFAULT_ROOT_DEV='(((0x3<<8)| 0))'
-DDEFAULT_COMMAND_LINE='""' -DDEFAULT_PROGRAM_VERSION='""' -c
/users/andrey/kern/mkelf/share/mkelfImage/elf32-i386/head.S -o
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/head_6384.o

Running ld -o elfImage.fat -T
/users/andrey/kern/mkelf/share/mkelfImage/elf32-i386/elfImage.lds
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/kernel_piggy_6384.o
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/ramdisk_piggy_6384.o
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/convert_params_6384.o
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/head_6384.o Running
ld -o elfImage.fat -T
/users/andrey/kern/mkelf/share/mkelfImage/elf32-i386/elfImage.lds
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/kernel_piggy_6384.o
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/ramdisk_piggy_6384.o
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/convert_params_6384.o
/am/mother-v2-e5/vol/vol0/u0/andrey/kern/mkelf/head_6384.oBFD:
elfImage.fat: warning: Empty loadable segment detected

checksum: dc7c

echo done

done

The script is a bit ugly, but we strive not for beauty... You'll find it below, or in the linux bundle... Next step is to scp back the 9load (masked as an elfImage) to a place where it could be burned in flash, or if you want to debug, downloaded by an etherboot LinuxBIOS:

fbsd$ scp elfImage plan9:/386/9pcblah
andrey@plan9.acl.lanl.gov's password:
elfImage 100% |*****************************| 190 KB 00:00
scp: can't wstat /386/9pcblah: wstat -- attempt to change qid.vers
fbsd$

Ignore the error message if you see it... In this case, the dhcp server has been instructed to provide /386/9pcblah as the kernel to a bootp request from the machine we work with (based on that machine's ethernet address, ask dpx@acl for more details).

NOTE: if you think that the 9load will work fine, just tell Ron to burn it in flash for you.. In this case you need to put the 9pccpu kernel in /386/9pcblah instead :)

File organization:

create a $home/src directory to store all the modified files in. the then in $home/src recreate all the directories present in /sys/src, which pertain in your current development. in my case it was /sys/src/9, /sys/src/boot, /sys/src/libauthconfig, and then recursively recreate all other directories present there. here's a real-life example:

p9% du /usr/andrey/src | awk '{print $2}'
/usr/andrey/src/9/pc
/usr/andrey/src/9/ip
/usr/andrey/src/9/alphapc
/usr/andrey/src/9/bitsy
/usr/andrey/src/9/boot
/usr/andrey/src/9/port
/usr/andrey/src/9/mtx
/usr/andrey/src/9
/usr/andrey/src/libauthsrv
/usr/andrey/src/boot/alphapc
/usr/andrey/src/boot/arm
/usr/andrey/src/boot/pc
/usr/andrey/src/boot
/usr/andrey/src/cmd/aux/vga
/usr/andrey/src/cmd/aux
/usr/andrey/src/cmd/auth/factotum
/usr/andrey/src/cmd/auth/lib
/usr/andrey/src/cmd/auth/secstore
/usr/andrey/src/cmd/auth
/usr/andrey/src/cmd
/usr/andrey/src
p9%

Then if you want to edit /sys/src/9/pc/pci.c just do:

cp /sys/src/9/pc/pci.c $home/src/9/pc/pci.c

Scripts:

Several scripts are used to automate the binding process and to make your life easier. Remember that after the boot-up the machine is in a non-development state, i.e. it does not know that you have made any modification to the kernel source -- you have to tell it to change the namespace accordingly, so that when you type 'mk' in /sys/src/9/pc it will compile your own files instead of somebody else's... The scripts will do the modification to the namespace for you, just execute them before you start working on the code (think of it as a 'start your engines' of some sort). Here are the scripts (they differ because one does binding of the $home/src/9 as the kernel tree -- something I use for writing drivers to support several graphics cards, i.e. non-LinuxBIOS kernels) and the other binds $home/src/linuxbios/ which is the same as $home/src/9, but the files therein are LinuxBIOS-related. Copy those files in $home/bin/rc and chmod 755 them:

p9% cat /usr/andrey/bin/rc/kbind
#!/bin/rc
# this does bindings for the non-linuxbios kernel sources
# and the 9load linuxbios sources

# 9
bind -cb $home/src/9/pc /sys/src/9/pc
bind -cb $home/src/9/ip /sys/src/9/ip
bind -cb $home/src/9/alphapc /sys/src/9/alphapc
bind -cb $home/src/9/bitsy /sys/src/9/bitsy
bind -cb $home/src/9/boot /sys/src/9/boot
bind -cb $home/src/9/port /sys/src/9/port
bind -cb $home/src/9/mtx /sys/src/9/mtx

#boot
bind -cb $home/src/boot/alphapc /sys/src/boot/alphapc
bind -cb $home/src/boot/arm /sys/src/boot/arm
bind -cb $home/src/boot/pc /sys/src/boot/pc

#fs
bind -cb $home/src/fs/port /sys/src/fs/port
bind -cb $home/src/fs/choline /sys/src/fs/choline
bind -cb $home/src/fs/dev /sys/src/fs/dev
bind -cb $home/src/fs/emelie /sys/src/fs/emelie
bind -cb $home/src/fs/ip /sys/src/fs/ip
bind -cb $home/src/fs/pc /sys/src/fs/pc
bind -cb $home/src/fs/roro /sys/src/fs/roro
bind -cb $home/src/fs/sony /sys/src/fs/sony

#cmd

bind -cb $home/src/cmd /sys/src/cmd

p9%

The #cmd part is not really necessary, but i keep in there

stuff i do with P9 commands...

p9% cat /usr/andrey/bin/rc/lbbind
#!/bin/rc
# linuxbios stuff

# 9
bind -cb $home/src/linuxbios/pc /sys/src/9/pc
bind -cb $home/src/linuxbios/ip /sys/src/9/ip
bind -cb $home/src/linuxbios/alphapc /sys/src/9/alphapc
bind -cb $home/src/linuxbios/bitsy /sys/src/9/bitsy
bind -cb $home/src/linuxbios/boot /sys/src/9/boot
bind -cb $home/src/linuxbios/port /sys/src/9/port
bind -cb $home/src/linuxbios/mtx /sys/src/9/mtx

#boot
bind -cb $home/src/boot/alphapc /sys/src/boot/alphapc
bind -cb $home/src/boot/arm /sys/src/boot/arm
bind -cb $home/src/boot/pc /sys/src/boot/pc

#fs
bind -cb $home/src/fs/port /sys/src/fs/port
bind -cb $home/src/fs/choline /sys/src/fs/choline
bind -cb $home/src/fs/dev /sys/src/fs/dev
bind -cb $home/src/fs/emelie /sys/src/fs/emelie
bind -cb $home/src/fs/ip /sys/src/fs/ip
bind -cb $home/src/fs/pc /sys/src/fs/pc
bind -cb $home/src/fs/roro /sys/src/fs/roro
bind -cb $home/src/fs/sony /sys/src/fs/sony

#cmd
bind -cb $home/src/cmd /sys/src/cmd

p9%

Actually, only the lbbind command is needed -- that'll bind

all sources from the source tree

An example of a typical P9 session follows:

p9% lbbind

p9% cd /sys/src/9/pc

# edit some files, make some changes, etc...

#make a cpu kernel

p9% mk 'CONF=pccpu'

8c -FVw ether79c970.c

8c -FVw vgabt485.c

# lots of files being compiled, here's where your errors will happen
#
# now you have a decent kernel, copy it somewhere where it could be etherbooted

p9% cp 9pccpu

# clean up -- not really necessary, but sometimes fun :)

p9% mk clean

rm -f *.[v4586xq7] *.root.s cfs.h fs.h init.h conf.h *.out

for(i in pc pccpu pcdisk)

mk $i.clean

rm -f pc.c [9bz]pc [9bz]pc.gz bootpc.*

rm -f pccpu.c [9bz]pccpu [9bz]pccpu.gz bootpccpu.*

rm -f pcdisk.c [9bz]pcdisk [9bz]pcdisk.gz bootpcdisk.*

p9%

# to stop a development session just delete the terminal window,
# your files will be saved there :)

Here you'll need ron's help! CVS update the latest from the freebios distribution, then guntar the mkelf.tgz file included, then get Ron to make LinuxBIOS proper so you can combine it with 9loadlb... I pretty much know nothing about this process. As far as I am concerned I may as well be sitting around waving a chicket at the screen while Ron does everything... After the setup, just use the 'mymk' script as above and enjoy :)

p9% diff boot/pc/mkfile /sys/src/boot/pc/mkfile
7d6
< 9loadlb\
95,98d93
< 9loadlb: ll.$O $CORE $LOAD $ETHER
< $LD -o $target -H3 -T0x80800000 -l $prereq -lflate -lc
< ls -l $target
<
118,120d112
<
< ll.$O: l.s
< $AS -DLinuxBios -o ll.$O l.s
p9%

MOV AX, SS

MOVL $0x80, DX
MOVB $0x19, AL
OUTB

p9% diff boot/pc/l.s /sys/src/boot/pc/l.s
20,24d19
<
< MOVL $0x80, DX
< MOVB $0x11, AL
< OUTB
<
27c22
< #ifndef LinuxBios
---
>
200,202d194
<
<
<
209c201
< MOVL $1,AX
---
> LWI(1, rAX)
213,216d204
< /* MOVL CR0,AX
< ORL $1,AX
< MOVL AX,CR0
< */
227,238c215,218
<
< #endif /* LinuxBios */
< /* begin andrey messing around */
<
< MOVL $0x80, DX
< MOVB $0x11, AL
< OUTB
<
< MOVL tgdtptr-KZERO(SB),GDTR /**/
<
<
< MOVW $SELECTOR(1, SELGDT, 0),AX /**/
---
> /* MOVW $SELECTOR(1, SELGDT, 0),AX /**/
> BYTE $0xc7
> BYTE $0xc0
> WORD $SELECTOR(1, SELGDT, 0)
239a220
> MOVW AX,SS
243d223
< MOVW AX,SS
245,249d224
< MOVL $0x80, DX
< MOVB $0x12, AL
< OUTB
<
< #ifndef LinuxBios
255,256c230,231
< #endif /* linuxbios */
< TEXT mode32bit(SB),$0
---
>
> TEXT mode32bit(SB),$0
266,270d240
<
< MOVL $0x80, DX
< MOVB $0x13, AL
< OUTB
<
274,277d243
< MOVL $0x80, DX
< MOVB $0x14, AL
< OUTB
<
290,292d255
<
<
<
308d270
<
320d281
<
345,348d305
< MOVL $0x80, DX
< MOVB $0x17, AL
< OUTB
<
361,363d317
< MOVL $0x80, DX
< MOVB $0x18, AL
< OUTB
371,374d324
<
< MOVL $0x80, DX
< MOVB $0x19, AL
< OUTB
p9%

p9% diff boot/pc/ether.c /sys/src/boot/pc/ether.c
41d40
<
62,63c61,62
< // if(isaconfig("ether", ctlrno, ctlr) == 0)
< // continue;
---
> if(isaconfig("ether", ctlrno, ctlr) == 0)
> continue;
66,67d64
< memset(ctlr, 0, sizeof(Ether));
< strcpy(ctlr->type, ethercards[n].type);
73,74c70
< i = (*ethercards[n].reset)(ctlr);
< if(i < 0) {
---
> if((*ethercards[n].reset)(ctlr)){
76,78d71
< continue;
< } else if(i) {
< splx(x);
86d78
< //ctlr->irq = 9;
p9%

p9% diff boot/pc/load.c /sys/src/boot/pc/load.c
125d124
<
149,162d147
< static int
< myboot(char* file)
< {
< char *eth = "ether0";
< Type *tp;
< Medium *mp;
<
< for(tp = types; tp->type != Tnil; tp++)
< for(mp = tp->media; mp; mp = mp->next)
< if(strcmp(mp->name, eth) == 0)
< boot(mp, file);
<
< }
<
252,254d236
< delay(100000);
<
< outb(0x80, 0x20);
256d237
< outb(0x80, 0x21);
258,264d238
< outb(0x80, 0x22);
<
< outb(0x80, 0x27);
< consinit("0", "115200");
< outb(0x80, 0x28);
< kbdinit();
<
266d239
< outb(0x80, 0x23);
268d240
< outb(0x80, 0x24);
270d241
< outb(0x80, 0x25);
272d242
< outb(0x80, 0x26);
274,277c244
< outb(0x80, 0x27);
< // consinit("0", "115200");
< outb(0x80, 0x28);
< // kbdinit();
---
> kbdinit();
279,283c246,247
< print("hi!\n");
< pcihinv(nil);
< outb(0x80, 0x29);
< // if((ulong)&end > (KZERO|(640*1024)))
< // panic("i'm too big\n");
---
> if((ulong)&end > (KZERO|(640*1024)))
> panic("i'm too big\n");
285d248
< outb(0x80, 0x30);
287d249
< outb(0x80, 0x31);
297d258
< outb(0x80, 0x32);
299d259
< outb(0x80, 0x33);
301,304c261,262
< // if((p = getconf("console")) != nil)
< // consinit(p, getconf("baud"));
<
< outb(0x80, 0x34);
---
> if((p = getconf("console")) != nil)
> consinit(p, getconf("baud"));
306d263
< outb(0x80, 0x35);
308d264
< outb(0x80, 0x36);
318d273
< outb(0x80, 0x37);
322d276
< outb(0x80, 0x38);
337d290
< outb(0x80, 0x39);
365,368d317
< outb(0x80, 0x40);
< myboot(file); /* hardcoded ether0 boot */
< outb(0x80, 0x41);
<
443a393,395
> extern void diff(char*);
>
> ulong palloc;
447,448d398
<
< static ulong palloc;
464d413
<
p9%

p9% diff linuxbios/pc/main.c /sys/src/9/pc/main.c
76d75
< outb(0x80, 0xA0);
78d76
< outb(0x80, 0xA0);
80d77
< outb(0x80, 0xA1);
82,83d78
< outb(0x80, 0xA2);
< kbdinit();
85d79
< outb(0x80, 0xA3);
88d81
< outb(0x80, 0xA4);
91c84
< outb(0x80, 0xA6);
---
> kbdinit();
93d85
< outb(0x80, 0xA7);
95d86
< outb(0x80, 0xA8);
97d87
< outb(0x80, 0xA9);
99d88
< outb(0x80, 0xAa);
101d89
< outb(0x80, 0xAb);
103d90
< outb(0x80, 0xAc);
105d91
< outb(0x80, 0xAd);
107d92
< outb(0x80, 0xAe);
109d93
< outb(0x80, 0xAf);
111d94
< outb(0x80, 0xb0);
113d95
< outb(0x80, 0xb1);
116d97
< outb(0x80, 0xb2);
118d98
< outb(0x80, 0xb3);
120d99
< outb(0x80, 0xb4