almost done

It's been some time since the last post, I've been very busy, I'm almost done, so this may be my last post about this undergrad project specifically, I will edit this with project's download regarding information, I'll probably will start a sourceforge.net project, so for now on this blog, will tell things about different things that happen in my life, that only matter to me and I just want to keep on for the record.

So what is it that you're doing Jorge?

Well, sometimes Images have more words than bytes (joke?), anyhow it's called MPSched, a multi personality scheduler.

Problems breathing?, get some Doxygen

Well, I found this project which really can give some Oxygen

http://www.stack.nl/~dimitri/doxygen/

It helps people around the globe on documenting code.

ICONTEC up my ASS!!

Well, AFTER HARD WORKING ON IT, I STILL CAN'T MAKE MY DOCUMENT LOOK AS UGLY AS ICONTEC WANTS ME TO!: Story begins yesterday when I lost all my day working on something I couldn't avoid to do; unfortunately any UPB's undergrad project documentat should be properly delivered in compliance with ICONTEC norms for documents, thing is, I'm using LATEX and as you can imagine there is not a single LATEX class for ICONTEC on the searchable planet (google), meaning that I have two options: redo on M$ word, or .cls DOITYOURSELF, since such animal was climbing my leg directly to my ass, I decided to put an end to it yesterday. Well, after a complete day of tweaking the standard neat book class; I STILL CAN'T DEFEAT IT PROPERLY INTO ICONTEC, IT STILL LOOKS SOMEHOW NICE!!!!!!!!!!!!!, once I get it done, I'll sure share such .cls file to the world.

Dresden's TUDOS

It seems that not everything in Dresden is about neonazis, people at UT Dresden, have been working hard for the last years on L4 fiasco, among of which I will want to have in record:

• Driver Reuse In The L4 Environment

• Qt for the L4 Environment

Three thumbs up for UT-Dresden!!!,
too bad I still feel scared about the idea of visiting some day.

evolving to OKL4

Well, I was out of town for almost one week, now that I'm back, I've been informed that OKL4 has been released at Open Kernel Labs, and now getting it up and running seems much more simplier, than old iguana-devel:

wget http://portal.ok-labs.com/_assets/downloads/okl4_release_1.4.1.1.tar.gz
tar xvfz
okl4_release_1.4.1.1.tar.gz
./tools/build.py machine=ia32_pc99 project=iguana wombat=True simulate

I've noted some changes:

there is no lua script and init for iguana, therefore, I have a few questions:

What is the l4 to wombat init sequence, and where can I customize it?

The other thing: I'm trying to understand the iguana/server/src/main.c

but I can't find the server_loop() function code.

Here is some output:
jetorres@dolce:~/cool/okl4_release_1.4.1.1/iguana$ grep -r server_loop * ../libs/*

naming/src/naming_server.c: server_loop();
serial/src/main.c: server_loop();
server/src/main.c:extern void server_loop(void);
server/src/main.c: server_loop();
timer/src/main.c: server_loop();

Genesis II

Various processes running in the same computer implies a risk of having one process corrupting some other process memory, because of that; memory protection concept evolved into the allocation of distinct pages of memory for each process into segments, its limitations are defined by two hardware registers, hence, if a process tries to access memory outside the defined segment a protection mechanism is used, such as Intel's general protection faults. Having that; the concept of memory management unit (MMU) appeared as the set of components responsible for handling memory access from the CPU.

Time-sharing has also had a very interesting evolution into what we know as multitasking, which defines scheduling policies that decide which task runs at any given time, and establishes a mechanism known as context switching that reassigns CPU from one process to another. Concept began at computing early days, where peripherals (devices hooked up to the CPU) became much more slower than processing units, so when a process needed access to a peripheral, the CPU had to stop executing instructions while such peripherals were busy. Thus, multiprogramming idea came to scene by context switching every time a process was waiting, but that gave no guarantee of process to be run at proper times, since one could run for very long periods of time without triggering a context switch. That was absolutely not suitable for interactive computing needs, therefore cooperative multitasking was designed; there, processes ceded time to each other in a voluntary manner, however multitasking dangerously relied on each process to efficiently give time to the others, consequently the preemptive multitasking idea was born about assigning operating time slices to each process. Only then; processes were cataloged into two different groups; CPU-bounded ones as those who were fully utilizing CPU, and I/O bound processes as those waiting for input or output from peripherals. A perfect Martini came out of blending preemptive multitasking with hardware interrupts, which would rather than inefficient I/O bound process to constantly check for resource availability. It gives a call to subroutines handling the interrupt event of resource availability, changing I/O bound processes state from on hold or blocked to unblocked as it may correspond, meaning that blocked processes would properly return to execution.

At that point processes were so handy, that programmers started use them by assigning different tasks to different processes, which then was generally understud as a set of cooperating processes that people started to reffer as applications, the main issue behind it, was finding appropriate ways to exchange information between application's processes. As a result of that search; Threads were proposed as cooperating lightweight processes coexisting in a shared memory space or context, thus, another advantage was found in their favor, and it was that memory context switching was not necessary when context switching between threads.

בראשית-Bərêšîth-Genesis

I decided to start from the beginning, and here it goes my first undergrad chapter:

In the beginning, the earth was without form and void, and darkness was upon the face of the deep [Torah]; many, many years later; the sons of the sons of the sons......of Noah said; let it be a CPU and there was a Central processing Unit, or simply processor, whose job was to process data and to interpret computer programs instructions, hence an important difference between processes and programs: were programs are the set of instructions which we call a process once they are being processed by the processor (makes sense doesn't it?), in other words a process is a program under execution. And Noah's grand grand... children saw that the processor was good, but for this processor to be of some real use; it had to interact with some computer resources. And as systems went larger and complex, managing computer resources and processing interactions became a development complicated issue. Thus, operating systems (OSs) were born as a set of programs developed to provide standardized solutions for managing computer software and hardware.

Having operative systems and their services; a new world of concepts such as process and memory management, device drivers, disk and file systems, networking, etc. started to have a meaning and reason for existence.

As long as the computer is a Harvard or von Neumann architecture like the one the reader certainly has at home, only one process can run per CPU core at a time. As a solution for that; the time-sharing magic appeared at early 1960's, enabling concurrent execution of many processes at once per processor core, the trick behind it is called process management; and it is simply done by quick-switching processes, therefore distributing CPU time, which will be explained in deatail.

Having many concurrent executions dramatically increases memory management complexity, since it has to coordinate how memory is used by processes. Keep in mind that now days system's memory, is divided in various types, depending on its access speed. we have registers, CPU cache, random access memory (RAM) and at last: disk storage. The memory manager deals with those various types of memory by determining how to move data between them. To help with that task, virtual memory management was invented, there, memory is divided in virtual and physical addresses, having the virtual ones as those unique to processes, and real addresses as those unique to memory manager and CPU, that provides a separation between the physical memory and addresses used by programs. It is important to understand that in memory; programs consist mainly of two things: ``Text'' where program's running instructions are stored, and ``data'' where hardwired and volatile information such as string constants and variable values are stored, but as programs become a little bigger, the use of functions or subroutines is necessary, so there is a need for storing current process state before they call a function, consequently there must be some specific place to store such information, which is called the process's stack. For security reasons some operative systems split text, data and stack sections into what is called segments. Having that clear, when a program is about to be run, a virtual memory space for its text, data and stack information is created, that means that at process point of view, virtual memory its the only memory available, which is organized , but as process's need for more memory arises, data segment can change its size dynamically, by reallocating memory from unused memory areas called heap, that method is known as heap-based memory allocation.

It is common to hear people talking about 64-Bits now days, due to AMD's successful publicity, what this number is telling us is the largest number an AMD-64 can work with at a time, called word-size; which in that specific case is 64 bits. There are many implications from it, one important comes if we think about addressable addresses in memory, for example the common 32-Bit architectures have top of 2^32-1 bits = 4GB to work with. Therefore having a virtual address space from 0x00000000 to 0xffffffff, which is divided in peaces called pages. Same thing happens to physical memory but divisions are called frames. So a program segment can be located in virtual address A found by its page and offset numbers but it really corresponds to a physical address B, located by its frame and offset numbers, and what the virtual memory manager does is to translate virtual into physical addresses, with the use of a page table containing page-frame mappings, but address translation by software rutines is time consuming, and therefore reduces system's eficiency, so they decided to fasten this by storing most recent page-frame mappings into a very fast content addressable memory, known as the translation lookaside buffer (TLB), in which the search key is the virtual address and the search result it's corresponding real address. If no match ocured then a software address translation is trigged, improving virtual memory management speed.



to be continued.....

Good old make times before scons

What tha f*&%k have I gotten my self into?, sent this paper signing my own death penalty, I was very wrong thinking, if I was planning to finish my undergrad project in no longer term than a month, hence I could start backpacking Latin America for as long as my wallet would've supported it. why didn't I do some stupid piece of dummy borland software, like every f**&cking else. Did I mentioned that I never liked python at all?, well surprise, I'm starting to hate scons build system, it makes it dificult for me to understand things, and I also feel frustated because of me not understanding as quick as I expected the iguana-devel sources, too late to figure out that it is not as simple as understanding the papers.

WHEREEEEEEEEEEEEE THE HELL IS THE SCHEDULE FUNCTION EXECUTED FOR THE FIRST TIME IN WOMBAT????????

Any how, just for the record: To find a function code: Must first check on the file you're reading, then if nothing is there; cd to libs dir, and grep -r "FunctionName" *.

Oh, also some useful documents I've found, for "code understanding"

L4 Development
L4-X2 Reference manual (outdated, for what I'm working on, since Im using the N series :)

also found this on the mailing list:

> Can anyone help me and describe the execute process of L4:pistachio mainly? Thanks.

See the "Teaching" section of the L4Ka web. There are some very
interesting slides like:

http://i30www.ira.uka.de/teaching/coursedocuments/80/mkc-03.pdf

template env for lua

I cant understand exactly where is it that the lua env variables values are set, for the iguana startup script. I'm not too much into python, but as I can see there is something at the IGUANA Init SConstruct, but still I don't understand how scons args.get works, and where such args are set.

Wombat memory

I now understand that system's memory size is passed to wombat by the iguana init server as Hal Ashburner said in his last e-mail, my question is about the LINUX_HEAP_SIZE definition, is that the amount of RAM memory assigned to wombat L4's thread?, or is it the parameter that iguana passes to wombat informing about memory virtual size, or both?

Hal Ashburner:

This is the size of the iguana memsection that is created and backed byphysical memory that is passed to wombat that wombat can then use for
its heap.

As far as wombat is concerned this is all the memory available for its heap, it knows of no more and can't use anything else.

And what is the L4_ZONE_SIZE?,

Oh, yes and about my last e-mail, does somebody know why is it that I have to use the -nographic parameter with qemu in order to run the c.img image?

Hal Ashburner:

Because there is no vga driver enabled in the version of wombat you are
using.

Thank you very much,

L4 embedded + Wombat Linux in 3 simple steps, Magic?

No theres no magic, thank god I use debian, here I will describe how to get your wombat up&running within few commands:

First you have to get the required software + plus some other useful things:

apt-get remove --purge gcc g++
apt-get install gcc-3.3 g++-3.3
apt-get install python
apt-get install scons
apt-get install sudo
apt-get install qemu
apt-get install qemuctl vgabios qemu-launcher kqemu-common
apt-get install kqemu-modules-
apt-get install bazaar

WARNING gcc-3.3 and g++-3.3 are required!

Now that we have what we need, we will get the last version of iguana-devel into some particular directory, check out this link for further explanation.

mkdir wombat-rep
cd wombat-rep
baz register-archive http://www.ertos.nicta.com.au/arch/public/
baz get ertos@nicta.com.au--public/iguana-project--devel
cd iguana-project--devel--1.2--version-0/
baz build-config packages

Now go get your self something to eat!!

scons machine=pc99 simulate
or
tools/scons.py machine=pc99 simulate

Results on the build folder, and if getting hands dirty is wanted, there's a doc folder with iguana framework information.

That's it!! Special thanks to the ERTOS NICTA group for making this as simple as it is!!!

Just in case jorge sees it!

A few days ago (like 60!), I wrote Ben Leslie about sigma0 and its out of scene, and I forgot to post his answer, I beleve my friend Jorge, will find this useful for his project:


>Well and the other question, this removal of sigma0 need, happens at the L4::N1 implementation or is it a very recent l4:Pistachio improvemnet?

I think it is N1. The change is that MapControl allows the roottask to insert explicit mappings, rather than relying on the address space of sigma0.

close your eyes.... OPEN UP!!

What's it gonna be?, Last time I was talking about Gernot's idea of a user-level scheduler implementation for L4, but now I've come up with something using it, I've found what Sergio Ruocco wrote: User-Level Fine-Grained Adaptive Real-Time Scheduling via Temporal Reflection, also he said that:

A decision on releasing the framework has not been made (yet), but you
can read more about it in this paper

It seems that I can implemented it my self plus some other things I believe I can add to it. The question is: What do I want it for?, well I wrote long time ago at the kenge users list about me wanting to merge down the wombat scheduler into a L4 embedded thread. so that's precisely what I have in mind. one important thing is that I cannot shoot my own foot, so I have to make my goal something like this:

TITLE (which describes the goal): Framework for realtime Wombat Linux scheduler

Back on top of the saddle

Well, as a dream come true, there is my self knocking on wood and more than happy to tell I'm jobless. Today is my first day back in Medellin, to get my hands dirty with my undergrad project.

Fist things first:

Find me an advisor: I called my friend Rafael Gonzalez, a very busy fluidsignal worker, teacher at the UPB and a very nice guy, told him it was going to be related to L4, so he accepted. Now I have to write a project description, and e-mail it, lets see how he likes it!

(ANTEPROYECTO) Project description: I have promised my self that this will be finished before Friday night!.

rafa:~# declare -x LIFE="Don't Become a Scientist"

I don´t know where exactly would this post match the better, but here it comes: I DON´T FEEL like writing Italian at this moment, I'll probably come with something out of it after I get to chew and swallow, what my close friend rafa advises, DON´T BECOME A SCIENTIST, I remember late at fluidsignal's "working" nights, long talking to him about the dreamer idea of becoming one of those he is now telling to do not touch!!, I have only one thing to add, I do have dualities: entrepreneur or scientist: which could translate (like it or not) at the very end into money and knowledge of the art of making money or knowledge and just passion for making something that has nothing to do with money, very contradicting one from the other??, I really don't know but sometimes I say to my self, money can wait, plus it is not just about becoming a scientist, it´s about the experience, new countries, new people. I don't want to be a millionaire, once dead no buddy remembers them!! (will my grandsons remember Bill Gates???) I truly believe that the reason I want to become a so called entrepreneur, it´s just because I want to get into that level at which I don't have to care about money, and dedicate my life to something much more interesting, which I still have to find!.

B licenses

Kind Gerwin, wrote:

> Hi Jorge,
>
> Raf has a copy of his formal B model available for download at:
> http://xaph.net/x/APInov3.tgz
>
> As for B licenses: we don't really know.
>
> Raf says he would advise against B, unless you or your institute already
> have some licenses available. You can always ask the manufacturers, though.
> The main tools are Atelier B and the B Tool, I think. There is also a new
> version of B coming from the RODIN project, but I don't think Raf's
> formalisation will be compatible with that.
>
> Cheers,
> Gerwin
>

This is my response:

Hi Gerwin,

Thank you very much, Raf´s advise against B, would be great, since we dont have any licenses.

Why is it that you think that Raf's formalization wont be compatible with the B version from RODIN?, anyhow, I've seen the website: http://rodin-b-sharp.sourceforge.net/, but it seems that project stoped six months ago.

Thank you very much,

Jorge

PLEB-> Testbed for L4

So I came across with PLEB, an XSCALE single board computer designed at NICTA. They have everithing one will need to get it done, form hardware designs, til software code for BIOS and the rest. They even have one with CAN node integrated,
More info about it:
http://www.ertos.nicta.com.au/hardware/pleb/
lets see if I cant talk Jorge to get his hands on it.

where did sigmas go?

I wrote back:

This removal of sigma0 need, happens at the L4::N1 implementation or is it a very recent l4:Pistachio improvemnet?
Now I'm a little more confused than last time. When you mean that you've removed the sigma0 need, you mean that it is not a provileged thread anymore?, giving freedom of implementation for memory management, so there is no thread memory space asingment at kernel level, and that task is now performed at a user level, at same level of memory manager? which makes sense. If so, would it be better having memory manager implementations wich will do both deliver memory for threads(sigma0) and manage memory?, is your simple sigma0 implementation http://www.ertos.nicta.com.au/software/kenge/sigma0/devel/, a demo of it?.


Just as a reminder, I havent found any information about the l4::N1, I'll have to search some more.

L4 User-level reflection scheduler

I've found this article:
http://www.ertos.nicta.com.au/publications/papers/Ruocco_RTSS_06.pdf
In which Sergio Ruocoo describes his 100 loc implementation of a user-level reflection scheduler, for realtime implementations without cahnges at l4 kernel, I assume it is about a L4 Thread with the SCHEDULLER attribute set to 0, any how, there's the article, I'll write about it once I finish reding it. For now:

Computational reflection: “the activity performed by a computational system when doing computation about (and by that possibily affecting) its own state and computations”.

BenLeslie says: Sorry to tell, but sigma0 died!

Well, what can I say, unexpected answer from the kenge users list!. Friendly Ben Leslie answered about sigma1:

Hi Jorge,

We have actually removed the need for sigma1 (and sigma0). On our
latest version it does not exist.

Further e-mail about this I'll write. And about B-toolkit, he was kind enough to CC to the L4 formal method author. And about scheduler I was correct.

Few questions I have for my self, and for Ben off course.

distributed or disturbed notion of sigma0

Well, I had this project with my friend Jorge Cardona, about distributing an operative system trough a network of computers, so if you get to run a Virtual Machine on top of it, it would be transparent for the host OS, in other words a transparent cluster configuration.

How about making sigma0 distributed over a network?, in that way VM machine threads memory on top of L4 would be distributed, and then, it wouldnt be too complicated to make a thread to manage distrubuted interrups and make virtualized wombat to use such thread, thus having wombat working under multiple computers at the same time, and one could do cool things such as active-active failover OS.

Reading this B-method implementation i found this:

Sigma0 is the default system pager. Its job is to take up all possible non-kernel memory on start up, and give it out to threads as it is requested. Sigma0 is not a memory manager, and does not accept any form of request to take back (free) the memory that has once been given out. When other special threads load during the kernel loading procedure, memory is taken from sigma0. After this is done, a memory manager should take the remaining memory from sigma0 and manage it appropriately. Sigma0 gets its own address space, and is a privileged thread.

If I understand OK; in order to make it real one would have to worry aboout sigma0 and a memory manager.

echo mailto kenge-users >> /dev/null ??

I DONT know why, every time I write to the kenge-users list I dont get any answers, this time I worte about some questions I had.

to: kenge-users@ertos.nicta.com.au
Subject: sigma1 and L4 formal model

Hi you all,

I'm confused about what sigma1 is for?, I've found that it is about task persistency, but still it isn't clear to me, is it an atempt of having a swap memory manager?

Another question, is there any CVS for the Rafal Kolanski formal description of L4 using the B-method?, are there any kind of B-toolkit research licences, for thesis projects, I want to use it for a formal model of L4 scheduler so that a privileged thread such as sigma0 but for handling time interrupts which will IPC to a user space thread in which schedulling dessisions will be taken.

I understand that the internal scheduler is not a thread it self, it is just a function that chooses next thread to execute based on scheduller attribute.

Thank you very much,


Jorge

Gernot thesis supervisor?

En el otro lado del charco, which textually translates, at the other side of the pool, is Gernot's mail user agent (MUA), and suddenly a weird mail from: jorge torres GH126: User-level scheduling in L4:

Professor Gernot Heiser,

I understand that you are a very busy person, so please receive my apologies for taking some of your time. I have seen your Thesis Topics list for undergraduate students, and I am convinced that User-level scheduling in L4 is what I want to do for my undergraduate thesis project. I have only one question: what can I do (if possible) In order for you to be my thesis Supervisor?,

Thank you very much for taking the time to read this,

Sincerely,

ME,

That was more than a week ago, so he probably wasnt interested at all, and decided to send it to the bulk mail, or to mark it as spam, since he didn't even answer something like: sorry, you are right I'm very busy.

Even if it's his idea or not, due to its under public domain, and under the freedom of choice I'm proud of having to tell, if I don't find something more interesting in L4, that will be my thesis project whether Gernot likes it or not!

B->AMN

Abstract machine notation, I meet with a friend of mine, "tocayo" Jorge Eduardo Cardona and between beers, he told me about this thing he was reading, the B-method, so it came in to my curiosity todo reading list, and as i got to talk to him tody again i remembered, I found this site:
http://www.b-core.com/aboutb_method.html
and documentation about it:
http://www.b-core.com/ONLINEDOC/Contents.html
Jorge tells me that there is this guy at NICTA who came up with the L4 microkernel implementation using the B-method:
http://cgi.cse.unsw.edu.au/%7Erafalk/pubs/b-l4-api.pdf
So having that on the table why not having a zero defect thesis, as b-core promises?

The B Method, developed by Jean-Raymond Abrial [1], is a system of formal development from the initial high-level specification all the way to implementation via a process called refinement. It is one of the few formal methods to do so. The idea is to abstract away the implementation and concentrate on pure functional requirements at the top level. Then, with each refinement step, provide more information on how exactly the system fulfils those requirements, until enough information exists to implement the system. Each refinement step requires proof of consistency with the previous step.

Rafal Kolansk

L4 thesis

I was never worried about my undergraduate thesis project, but time passes, and suddenly I found my self working as the information security officer of the biggest stock market company in Colombia, INTERBOLSA, in a job that takes almost 16 hours/day from me.
With that on the scene it is very easy to understand that my undergrad thesis is something I really need to worry about. As my family says to me: after 5 years of study, You cannot end up as a "tegua", which in my country means: some one how knows a job, but has learned in a empiric way. But that is not my consternation, I beleve teguas are people to admire. thing is I REALLY WANT to become a PhD some day, so finishing my undergraduate studies is my big MUST for this semester. That's why I've taken two decisions: QUIT INTERBOLSA, and start what at this moment is important for my life plans!.