Archive for February, 2011

Some current projects...

Feb 23, 2011 by olsner

A bit of a mixed bag.

I've started working on a compiler for my programming language, see m3.git. Since code generation is boring, it targets LLVM. And anything related to "M++" is gone, and so are all object oriented features. This is simply an imperative language with modules, and maybe some type-system features.

That hasn't been moving for some time though, due to my replacement project: an operating system. This is going to be a dirt simple but (at least theoretically) fully functional operating system for x86-64/amd64/em64t, written in Assembly. I've decided that it's not going to be running C code until I finish support for loadable processes and/or kernel modules. As a general rule, I'm aiming for a microkernel structure. In part because microkernels are cool, in part because Assembly is hard to write so I really want to keep the "kernel" part as small as possible!

Now, the kernel thingy needs to do some memory management internally to keep track of some data structures where most of them are not as big as a page. I have a simple page-frame allocator, but now I need virtual memory management (which will reside inside the microkernel part of the OS) and processes, and to do that conveniently I need a kernel malloc to allocate my metadata.

There goes the third interrupting project: the malloc. In order to have something working in the kernel, written in assembly, I thought it best to start by even implementing malloc at all. So this is a malloc that you build into a .so (you can also link to it statically, if you compile it for that yourself) that loads into any program using LD_PRELOAD to replace the C library's malloc. That is, on Linux. Haven't tried it on anything else :)

I've stolen most of the ideas from a paper about phkmalloc, so I guess it's not very interesting in terms of malloc design. And it's single-threaded (protected by a single lock, if thread safety is enabled) so there will be no scalability contest between this and e.g. jemalloc! Then again, it shouldn't be excessively hard to take this allocator and essentially have one heap per thread/core, with a shared pagepool in the bottom somewhere.

My malloc currently "works" with any programs I've thrown at it, except that it doesn't actually free anything back to the OS. Properly implementing that seems to require a few changes to the underlying data structures. Oops! Which is precisely the reason I'm doing it at all - to find these problems out so I can redo it correctly in a write-once language like assembly :)

Anyway, after the malloc works, the next step is to translate it into the kernel code, implement virtual memory management in the kernel and to use it instead of the hardcoded page tables that it currently sets up for user processes. Then it'll be time to implement dynamically allocating processes based on an empty or existing address space, starting them up, scheduling them, ending them, and waiting for them! That'll be exciting :)