Building a "Kernel" in PCI Expansion ROM
Intro
This article is positioned as the official documentation of my Operating System On a PCI Expansion ROM project. Previously, this file is only a change log to the OS source code distribution, but later I decide to make it a complete article and here we are now :). Due to this "history", every detail explanation in this article will use the OS source code as its reference. You can download the latest source code distribution in the attachment of this article (scroll to the end of the article to see the link to download it).
The current source code distribution is a makefile based source code. Hence, it's not very easy to comprehend how the source code works. Also, there are several things that need to be explained pertaining to how the code gets executed, as this OS is not like the current "conventional Operating System" which is loaded from HDD or other types of "conventional Boot Device".
Preliminary Information About The Code
To fully comprehend the explanations int this article, you have to be good at the following subjects:
Knowing how to code in x86 assembly.
Knowing how to code in C language.
Have a quite indepth knowledge in the x86 hardware area, especially the BIOS. The most important subject to master in the BIOS is the Bootstrap Entry Vector (BEV) which is used in the resulting binary to "redirect" the booting process, so that the routine that gets executed is our OS initialization routine. A more thorough explanation on BEV can be found in the PnP BIOS Boot specification version 1.0A and other relevant publication.
These stuff would help you better understand what actually happen here :).
The source code actually built a two stage kernel boot loader. It changes the direction of booting from LAN facility in your computer BIOS into booting your own routine inside of the flashrom/eeprom of your LAN card or another card that you hack into a " LAN card indisguise", to be used to boot your kernel which is located inside the flashrom. A more thorough explanation about expansion ROM can be be found in the 2nd section of my "Preliminary BIOS Modification Guide" article.
How The Code Works
The principle of the execution of the code is based upon the BEV(Bootstrap Entry Vector) mechanism as explained in the Plug and Play BIOS Spec 1.0A.
Some important terms :
BEV = Bootstrap Entry Vector
BCV = Boot Connection Vector
IPL = Initial Program Loader. Device used as the boot device, so to speak
The basic run-down of what happens as follows:
During POST (Power On Self Test), the system BIOS (original.tmp in award bios) will scan every PCI devices ROM BAR (ROM Base Address Register). If this BAR is valid, i.e. implemented (consumed address space),then option/expansion ROMs exist. System bios then will copy the expansion ROM contents to RAM execute it there, as stated in PCI spec, PCI expansion ROM have to be executed from RAM. At this point, expansion-rom-os PCI initialization code i.e. labeled with entry point INIT: in /loader/loader1.asm. In this procedure, we only set the PCI ROM image indicator to zero and signaled the system bios that this is an IPL (Initial Program Loader) device by setting AX register value as needed upon return from this PCI initialization routine. By doing so, the system BIOS will then call the bootstrap entry point in our device ( BEV or BCV), if it is selected in the BIOS setting as the default boot device (in this case boot from LAN device). In this source code I choose to use the BEV mechanism as explained in PnP BIOS 1A spec.
After the system bios successfully runs POST, it will search for the IPL. In our case I assumed that we have choosen the LAN card as the default IPL. In this scheme, the BEV will be executed. In the source code, the BEV is the address labeled Start_OS: in /loader/loader1.asm. From there, the execution flows accordingly. The code will copy the remaining OS component, i.e. loader2.asm plus all the C-compiled code in /kernel to RAM starting at address 0000:7C00h and jumped there. The code jumped into is the start of /loader/loader2.asm
/loader/loader2.asm then switch the machine to protected mode and jump into the C compiled code in /kernel/main.c.
If you are new to this subject, then you might need the PCI specification 2.1 or newer and the Plug And Play BIOS Specification v1.0A so that the explanation here will be more obvious.
There is one thing that still obscure here: How exactly the BEV executed? is the system bios jump into it directly in the PCI device address space (in ROM) that's mapped into the system address space or the system bios load the entire expansion rom to RAM and execute the BEV in RAM ? This is still a mystery to be solved.
That's all for the current code version.
I plan to port all of the nasm code to GAS in the future and integrate the whole code using GNU tools so that I can use linker script to ease code maintenance. I hope this explanation is enough to someone that might read this.
"Shocking" fact about PCI Expansion ROM
Recently, I'm frustrated to find a flash rom chip for my new realtek RTL8139 NIC to continue my "OS_in_PCI_Expansion-ROM" project. Being unable to find such a chip that would work with my new card, I came across the idea of really hacking a different kind of PCI card that initially have flash rom chip from it's manufacturer to ease the continuation of my project. PCI spec 2.1 and 2.2 also the PnP Bios Boot spec says that the motherboard BIOS boot from LAN feature "only" (read--not so sure until I try it) checks the feature from the expansion ROM of the "NIC" (he..he.. or a "hacked into NIC" PCI expansion card) and proceeds accordingly, through int 19h interface (please correct me if I'm wrong). Then the hypothetical conclusion is this:
"ANY PCI expansion card can boot just like a typical bootable LAN Card provided we have the 'right' PCI expansion ROM BIOS".
So, I give it a try, since I don't have another cheap solution to this problem. And voila', it works very nicely, the mainboard BIOS is fooled to boot from my hacked PCI expansion rom card, here's the detail: Adaptec AHA-2940U SCSI controller card (VendorID = 9004, DeviceID = 8178), with soldered PLCC SST29C512 flashrom (64KByte). The binary flashed using unnoficial flash program (flash4.exe). The result is awesome and a bit weird, no matter how I changed the BIOS setup, the PCI initialization routine always get called during POST. I think this is due to the controller's chip Subclass Code and Interface Code, which is a SCSI controller/boot device in the Adaptec controller chip (AICXXX). Hence, it's always called by the mainboard BIOS at boot. The hacked BIOS make it behave as if it's a real PCI NIC except for the peculiarity mentioned above, my system boot from the card (through it's BEV routine) if I select boot from LAN in the BIOS setup of my mainboard. One thing to notice is I've changed the PCI vendor ID and device ID to match the ID's in the adaptec controller chip from my previous OS source code (for the Realtek based NIC), otherwise it won't boot (I think this what would happen, I haven't try this ).
Successfully "Hacked" PCI CARDs
The following cards have been successfully "implanted" with the resulting binary from this source code with little modification(s):
Realtek 8139A NIC (VendorID = 10EC, DeviceID = 8139), with Atmel AT29C512 flashrom (64KByte). The binary flashed using flash program provided by Realtek website (rtflash.exe). First, set the flashrom window size with rset8139.exe (also from realtek website), just read Realtek's README file.
Adaptec AHA-2940U SCSI controller card (VendorID = 9004, DeviceID = 8178), with soldered PLCC SST29C512 flashrom (64KByte). The binary flashed using unnoficial flash program (flash4.exe). The result is awesome and a bit weird, no matter how I changed the BIOS setup, the PCI initialization routine always get called (I think this is due to the controller's chip Subclass Code and Interface Code, which is a SCSI controller/boot device). The hacked BIOS make it behave as if it's a real PCI NIC except for the peculiarity mentioned above, my system boot from the card (through it's BEV routine) if I select boot from LAN in the BIOS setup of my mainboard. Also note that the flash program for this card ONLY ACCEPT BINARY FILE OF LENGTH 64KB, if you fail to do so, the flash program will not flash the binary at all :(.
Building The OS from The Latest Source Code
The latest source code distribution already supported compilation using makefile. You only need to invoke:
make
in the main/root directory of the source code distribution to make the OS and invoke :
make clean
to clean up all of the files generated.
Note: You have to provide the following program in an executable path within your "shell":
nasm
gcc
I used Linux (with bash shell) as my development environment, and it works just fine. I've tried using MinGW32 and MSys and the makefile works just fine but I don't know why gcc unable to output the "pure binary" file of the kernel (kernel.bin) correctly. If you have any suggestion please mail me.
Building The OS From Scratch (a Detailed Insight into The Source Code Structure)
This is the explanation of how to build the very first version which place all the source files in one directory. It is cumbersome, but it works. It is still here to give insight of how actually the code works. Rather than reading all the "twisty" makefile, it's better to read this file to know how exactly this thing works.
The following guide explains how to build a working "very simple kernel" from the sources in one directory.
Step 0: The assumptions in the following steps
You have to put all the needed sources back into one directory, then invoke all the "command" mentioned here from within that directory if you wish to do this.
Every binary/executable tool mentioned here are built from their respective source and are named the same as their source file, except for the extension, i.e. in the binary/executable tool no file extension used at all, while in the source I'm using *.c extension.
You already knows the PCI vendor ID and Device ID of the card you are using. They are needed since if you are using a different card than mine, YOU HAVE TO CHANGE THOSE IDs in the source code to match your card. It is in the first stage kernel loader, i.e. loader1.asm, which is located in the loader directory. If you failed to do so, there's a big chance that the resulting binary ROM file routine will not be executed at all during the booting process, which means our kernel is not executed at all.
You have the compiler and assembler needed. Nasm and GCC are required to build the sources that I provided.
Step 1: Build the tools needed
The sources needed to build all of the tools are provided in the utility directory. If you are using gcc, then just invoke it as follows (for each file):
gcc [source_filename] -o [target_filename]
Explanation of the tools
mergebin, this tools is used to combine 2 binary file (actually anyfile) into a single file. It's sensitive to position of the input parameters, i.e. the input filenames, the second input filename will be appended to the first input filename and the third input filename is the target binary file that we're building.
zeroextend, this tool is used to append zero(s) (0h) into a file until the file matches the size we're targeting (in bytes), which is the input parameter. For example to "zeroextend" a file into 1024 byte invoke:
zeroextend [input_filename] 1024
patch2pnprom, this tool is used to patch the 8-bit checksum of a "pseudo PCI ROM" file into a valid PCI pnprom. Frankly, it calculates the checksums and patches the needed header format as needed.
Step 2: Build the kernel loader
The source files are in the loader directory. Here's the explanation of the files :
loader1.asm . This file contains the PCI PnP rom header of the rom to be built, and some loader routines.Its function is to load the operating system code from ROM to RAM during the int 19h, which invokes the BEV that we set in this ROM source code. Its size is 512 bytes, after assembled.
loader2.asm . this file contains the assembly code to switch the machine from real to protected mode and also contains a jump into the C-compiled kernel code.Its size is 512bytes, after assembled.
To build the kernel loader, proceed with the following steps :
Assemble loader1.asm and loaader2.asm by invoking:
nasm -fbin [filename] -o [target filename]
in the command line.
Combine the resulting binary from previous step (above). To merge the file I use mergebin utility. Invoke :
mergebin loader1.bin loader2.bin loader.bin
in the command line to obtain it. Becareful not to swap the filename position since mergebin will put the first filename argument in the beginning of the resulting file, and so forth.
Step 3: Build the C kernel code
Compile and link the C sources for the kernel which are located in the kernel directory. To do so, invoke the following command:
gcc -c video.c -o video.o
gcc -c ports.c -o ports.o
ld -o kernel.bin -Ttext 0x7E00 -e main -N --oformat binary main.o video.o ports.o
Note: The last line means, link the files with main() function as entry point, with plain binary format and the code will begin at 0x7E00 when executed, since the first 512 bytes from 0x7C00 is used by loader2.bin, and with no page alignment (one page is 4Kbyte).
We're not done yet !!!
Then use zeroextend utility to extend the file into multiple of 512 bytes (since we're building a ROM file here d00d) as follows:
zeroextend kernel.bin 1024
Note: I'm using 1024 bytes as the "extended" file size for the C kernel binary here.
Step 4: Merge the kernel loader and the C kernel
Merge the C compiled code (kernel.bin) and the assembly code (loader.bin). Invoke the following command:
mergebin loader.bin kernel.bin boot.bin
Note: Again, take care of the position of the parameters! mergebin is sensitive to it.
Step 5: Patch the needed checksums
Invoke the following command:
patch2pnprom boot.bin
to patch all the wrong checksums in the binary so that boot.bin will become a valid ROM file. This file is the "ready to burn ROM file", use rtflash or another flashing tool to burn it into your LAN/NIC card (or another PCI expansion card) flash rom chip.
Yeah, we're done. I believe that the majority of the reader of this article is people who attracted in OS development. Thus, I assume that you are a programmer and just let the source code speak for additional technical information. I'm pretty much busy lately, so I haven't been able to continue the development of this experimental OS.
-- Update --
For more complete explanation, go to this article.
-- 31 Jan 2017 Update --
The Expansion ROM OS code is now hosted in GitHub: https://github.com/pinczakko/PCI-Expansion-ROM-OS
Copyright © Darmawan M S a.k.a Pinczakko