Thursday, April 26, 2007

evolving to OKL4

Well, I was out of town for almost one week, now that I'm back, I've been informed that OKL4 has been released at Open Kernel Labs, and now getting it up and running seems much more simplier, than old iguana-devel:

wget http://portal.ok-labs.com/_assets/downloads/okl4_release_1.4.1.1.tar.gz
tar xvfz
okl4_release_1.4.1.1.tar.gz
./tools/build.py machine=ia32_pc99 project=iguana wombat=True simulate

I've noted some changes:

there is no lua script and init for iguana, therefore, I have a few questions:

What is the l4 to wombat init sequence, and where can I customize it?

The other thing: I'm trying to understand the iguana/server/src/main.c

but I can't find the server_loop() function code.

Here is some output:
jetorres@dolce:~/cool/okl4_release_1.4.1.1/iguana$ grep -r server_loop * ../libs/*
naming/src/naming_server.c: server_loop();
serial/src/main.c: server_loop();
server/src/main.c:extern void server_loop(void);
server/src/main.c: server_loop();
timer/src/main.c: server_loop();

Wednesday, April 11, 2007

Genesis II

Various processes running in the same computer implies a risk of having one process corrupting some other process memory, because of that; memory protection concept evolved into the allocation of distinct pages of memory for each process into segments, its limitations are defined by two hardware registers, hence, if a process tries to access memory outside the defined segment a protection mechanism is used, such as Intel's general protection faults. Having that; the concept of memory management unit (MMU) appeared as the set of components responsible for handling memory access from the CPU.

Time-sharing has also had a very interesting evolution into what we know as multitasking, which defines scheduling policies that decide which task runs at any given time, and establishes a mechanism known as context switching that reassigns CPU from one process to another. Concept began at computing early days, where peripherals (devices hooked up to the CPU) became much more slower than processing units, so when a process needed access to a peripheral, the CPU had to stop executing instructions while such peripherals were busy. Thus, multiprogramming idea came to scene by context switching every time a process was waiting, but that gave no guarantee of process to be run at proper times, since one could run for very long periods of time without triggering a context switch. That was absolutely not suitable for interactive computing needs, therefore cooperative multitasking was designed; there, processes ceded time to each other in a voluntary manner, however multitasking dangerously relied on each process to efficiently give time to the others, consequently the preemptive multitasking idea was born about assigning operating time slices to each process. Only then; processes were cataloged into two different groups; CPU-bounded ones as those who were fully utilizing CPU, and I/O bound processes as those waiting for input or output from peripherals. A perfect Martini came out of blending preemptive multitasking with hardware interrupts, which would rather than inefficient I/O bound process to constantly check for resource availability. It gives a call to subroutines handling the interrupt event of resource availability, changing I/O bound processes state from on hold or blocked to unblocked as it may correspond, meaning that blocked processes would properly return to execution.

At that point processes were so handy, that programmers started use them by assigning different tasks to different processes, which then was generally understud as a set of cooperating processes that people started to reffer as applications, the main issue behind it, was finding appropriate ways to exchange information between application's processes. As a result of that search; Threads were proposed as cooperating lightweight processes coexisting in a shared memory space or context, thus, another advantage was found in their favor, and it was that memory context switching was not necessary when context switching between threads.

Friday, March 30, 2007

בראשית-Bərêšîth-Genesis

I decided to start from the beginning, and here it goes my first undergrad chapter:

In the beginning, the earth was without form and void, and darkness was upon the face of the deep [Torah]; many, many years later; the sons of the sons of the sons......of Noah said; let it be a CPU and there was a Central processing Unit, or simply processor, whose job was to process data and to interpret computer programs instructions, hence an important difference between processes and programs: were programs are the set of instructions which we call a process once they are being processed by the processor (makes sense doesn't it?), in other words a process is a program under execution. And Noah's grand grand... children saw that the processor was good, but for this processor to be of some real use; it had to interact with some computer resources. And as systems went larger and complex, managing computer resources and processing interactions became a development complicated issue. Thus, operating systems (OSs) were born as a set of programs developed to provide standardized solutions for managing computer software and hardware.

Having operative systems and their services; a new world of concepts such as process and memory management, device drivers, disk and file systems, networking, etc. started to have a meaning and reason for existence.

As long as the computer is a Harvard or von Neumann architecture like the one the reader certainly has at home, only one process can run per CPU core at a time. As a solution for that; the time-sharing magic appeared at early 1960's, enabling concurrent execution of many processes at once per processor core, the trick behind it is called process management; and it is simply done by quick-switching processes, therefore distributing CPU time, which will be explained in deatail.

Having many concurrent executions dramatically increases memory management complexity, since it has to coordinate how memory is used by processes. Keep in mind that now days system's memory, is divided in various types, depending on its access speed. we have registers, CPU cache, random access memory (RAM) and at last: disk storage. The memory manager deals with those various types of memory by determining how to move data between them. To help with that task, virtual memory management was invented, there, memory is divided in virtual and physical addresses, having the virtual ones as those unique to processes, and real addresses as those unique to memory manager and CPU, that provides a separation between the physical memory and addresses used by programs. It is important to understand that in memory; programs consist mainly of two things: ``Text'' where program's running instructions are stored, and ``data'' where hardwired and volatile information such as string constants and variable values are stored, but as programs become a little bigger, the use of functions or subroutines is necessary, so there is a need for storing current process state before they call a function, consequently there must be some specific place to store such information, which is called the process's stack. For security reasons some operative systems split text, data and stack sections into what is called segments. Having that clear, when a program is about to be run, a virtual memory space for its text, data and stack information is created, that means that at process point of view, virtual memory its the only memory available, which is organized , but as process's need for more memory arises, data segment can change its size dynamically, by reallocating memory from unused memory areas called heap, that method is known as heap-based memory allocation.

It is common to hear people talking about 64-Bits now days, due to AMD's successful publicity, what this number is telling us is the largest number an AMD-64 can work with at a time, called word-size; which in that specific case is 64 bits. There are many implications from it, one important comes if we think about addressable addresses in memory, for example the common 32-Bit architectures have top of 2^32-1 bits = 4GB to work with. Therefore having a virtual address space from 0x00000000 to 0xffffffff, which is divided in peaces called pages. Same thing happens to physical memory but divisions are called frames. So a program segment can be located in virtual address A found by its page and offset numbers but it really corresponds to a physical address B, located by its frame and offset numbers, and what the virtual memory manager does is to translate virtual into physical addresses, with the use of a page table containing page-frame mappings, but address translation by software rutines is time consuming, and therefore reduces system's eficiency, so they decided to fasten this by storing most recent page-frame mappings into a very fast content addressable memory, known as the translation lookaside buffer (TLB), in which the search key is the virtual address and the search result it's corresponding real address. If no match ocured then a software address translation is trigged, improving virtual memory management speed.



to be continued.....

Wednesday, March 28, 2007

Good old make times before scons


What tha f*&%k have I gotten my self into?, sent this paper signing my own death penalty, I was very wrong thinking, if I was planning to finish my undergrad project in no longer term than a month, hence I could start backpacking Latin America for as long as my wallet would've supported it. why didn't I do some stupid piece of dummy borland software, like every f**&cking else. Did I mentioned that I never liked python at all?, well surprise, I'm starting to hate scons build system, it makes it dificult for me to understand things, and I also feel frustated because of me not understanding as quick as I expected the iguana-devel sources, too late to figure out that it is not as simple as understanding the papers.

WHEREEEEEEEEEEEEE THE HELL IS THE SCHEDULE FUNCTION EXECUTED FOR THE FIRST TIME IN WOMBAT????????

Any how, just for the record: To find a function code: Must first check on the file you're reading, then if nothing is there; cd to libs dir, and grep -r "FunctionName" *.

Oh, also some useful documents I've found, for "code understanding"

L4 Development
L4-X2 Reference manual (outdated, for what I'm working on, since Im using the N series :)

also found this on the mailing list:

> Can anyone help me and describe the execute process of L4:pistachio mainly? Thanks.

See the "Teaching" section of the L4Ka web. There are some very
interesting slides like:

http://i30www.ira.uka.de/teaching/coursedocuments/80/mkc-03.pdf

template env for lua

I cant understand exactly where is it that the lua env variables values are set, for the iguana startup script. I'm not too much into python, but as I can see there is something at the IGUANA Init SConstruct, but still I don't understand how scons args.get works, and where such args are set.

Tuesday, March 27, 2007

Wombat memory

I now understand that system's memory size is passed to wombat by the iguana init server as Hal Ashburner said in his last e-mail, my question is about the LINUX_HEAP_SIZE definition, is that the amount of RAM memory assigned to wombat L4's thread?, or is it the parameter that iguana passes to wombat informing about memory virtual size, or both?
Hal Ashburner:
This is the size of the iguana memsection that is created and backed byphysical memory that is passed to wombat that wombat can then use for
its heap.

As far as wombat is concerned this is all the memory available for its heap, it knows of no more and can't use anything else.


And what is the L4_ZONE_SIZE?,


Oh, yes and about my last e-mail, does somebody know why is it that I have to use the -nographic parameter with qemu in order to run the c.img image?

Hal Ashburner:
Because there is no vga driver enabled in the version of wombat you are
using.


Thank you very much,

Thursday, March 22, 2007

L4 embedded + Wombat Linux in 3 simple steps, Magic?

No theres no magic, thank god I use debian, here I will describe how to get your wombat up&running within few commands:

First you have to get the required software + plus some other useful things:

apt-get remove --purge gcc g++
apt-get install gcc-3.3 g++-3.3
apt-get install python
apt-get install scons
apt-get install sudo
apt-get install qemu
apt-get install qemuctl
vgabios qemu-launcher kqemu-common
apt-get install kqemu-modules-
apt-get install bazaar


WARNING gcc-3.3 and g++-3.3 are required!

Now that we have what we need, we will get the last version of iguana-devel into some particular directory, check out this link for further explanation.

mkdir wombat-rep
cd wombat-rep
baz register-archive http://www.ertos.nicta.com.au/arch/public/
baz get ertos@nicta.com.au--public/iguana-project--devel
cd iguana-project--devel--1.2--version-0/
baz build-config packages


Now go get your self something to eat!!

scons machine=pc99 simulate
or
tools/scons.py machine=pc99 simulate

Results on the build folder, and if getting hands dirty is wanted, there's a doc folder with iguana framework information.


That's it!! Special thanks to the ERTOS NICTA group for making this as simple as it is!!!