Now, back to my original question: I’ve done a bit of reading on how to parallelize raytracing, especially given the memory constraints of the cell. There are three approaches I’ve discovered so far:

1. Distribute the rays: Each node runs a full raytracer on the full scene, and processes a subset of the rays from start to finish
2. Distribute the processing: One node handles ray creation, another ray intersection, another shading, another shadows. Every node needs access to different aspects of the entire scene.
3. Distribute the scene: Each node handles every ray segment that intersects it’s slice of the scene.

For my purposes, I’m interested in two things:

1. How well this will work on the Cell processor, where each node has only 256K of memory, and
2. How well the approach maps to the E Concurrency model , which I also call the vat model, below.

Distribute the Rays

The first approach is unquestionably the best when the entire scene fits into local memory of each node, as it is pretty easy on network bandwidth. Using this approach, once the scene is availible to all nodes, the raytracing problem is embarassingly parallel: every ray shooting out of the camera can be processed with zero communication among the nodes. However, if the scene does not fit in memory, the node will have to fetch parts of it and cache them locally.

Distribute the Computation

The second approach is closest to what the Cell was meant to do: act as a stream processor. However, this does not scale up to any number of nodes, nor does it scale down; it requires each node to be developed as part of a stream. This is rather network-intensive; most rays will make at least one pass through every node.

Distribute the Scene

The third is the closest to how vats natively operate. Scenes are usually stored in a tree structure, with the root pointing to a number of children. In this scheme, Many of the child pointers in the tree would be local pointers, but some would be far refs; entire subtrees of the scene graph would be seperated from the root and reside in a seperate vat. This would be very easy to program for.

A raytracer would be started on the vat with the root node of the scene, and as the program progressed, it would send remote messages to the parts of the scene stored in other vats, distributing the computation as a side-effect of sending regular, asynchronous messages to various parts of the scene. This is network intensive; every ray passes through several slices of the scene, and it is also unfair; slices with a light will have to process nearly every ray, while slices in the corner may get only a handful of rays.

The E Language has an interesting and very nice concurrency model. It is related to the Croquet/Tweak messaging model. This is an outline of it. I also made an animated demo of how promises work

Objects

• Everything is an OBJECT.
• Every object is associated with exactly one MAILBOX

There are two ways to initiate an action:

• CALL a METHOD of an object (synchronous, immediate)
• SEND a MESSAGE to an object (asynchronous, eventual)

Calling a method:

• Pauses the currently executing method
• Runs the called method until it completes
• Returns the computation result to the calling method
• Resumes executing the calling method

Sending a message:

• Creates a message object
• Places the message the receiver object’s mailbox
• Returns a PROMISE to the sending method

A message knows about:

• The promise created when the message was sent
• The object that will receive the message (the receiver)
• The method to call
• The arguments to the method

Promises

• A promise has a private mailbox
• Messages sent to a promise will merely be stuffed in its private mailbox
• A promise will turn into the return value of its associated message when it completes (the promise RESOLVES)
• A promise will put the messages in it’s private mailbox on the public mailbox when it resolves

A thread executes the following loop until terminated:

• take the earliest message from an object’s mailbox
• call the method named in the message
• when the method finishes, resolve its promise with the method’s return value

A method CANNOT wait for a promise to resolve, or for any other event. It may only place messages in a promise’s mailbox, to be run after the promise resolves. Thus, waiting is impossible, and, therefore, deadlock is impossible.

A thread, the group of objects associated with that thread, and the mailbox associated with those objects, is collectively called a VAT (called “Islands” in Tweak). Vats are the basic unit of concurrency in E. The number of active vats is the upper bound on the number of methods that are executing at once.

A vat is:

• A thread
• A mailbox
• A group of objects

All objects belong to exactly one vat

• a message sent to an object is enqueued in the object’s vat’s mailbox (unless the receiver is a promise)
• a method called on an object is executed in the context of the object’s vat’s thread
• an object can call methods of another object only if the callee object and the calling object are in the same vat (and therefore the methods execute in the same thread)
• an object can send a message to any object it knows about, no matter if it is in the local vat, in another vat, or is a promise

A vat’s thread is only ever processing one message at any time, and it runs to completion. And since an object may only be in one vat, it’s state is only ever being touched by at most one thread. Hence, memory contention is not present, and locking is not necessary. Locking is impossible anyway, as waiting is impossible.

Keywords:

• Object: An entity with state and behavior
• Method: A unit of behavior defined on an object
• Call: To execute a method in the current thread synchronously
• Message: A request to execute a method at some point in the future
• Mailbox: A queue for messages that have not yet been serviced
• Send: To place a message in an object’s mailbox
• Vat: A set of objects all sharing one thread and mailbox
• Promise: A handle for an object that has not been computed yet, since it is the result of a message that has not yet been serviced
• Resolve: To turn a promise into the object it was standing-in for. This happens when the message associated with the promise completes

This model is somewhat simplified. It does not address:

• Exception handling
• Broken promises
• Pass-by-copy objects
• Global message ordering

For more information, see chapter 3 of the Walnut book

I don’t update my blog nearly enough. I released DeltaStreams (with Goran’s approval), nearly a month ago (announcement). Since then, I’ve been working on a few improvements in my (currently copious) spare time. I’ve eliminated (I believe) the dependence on ToolBuilder and PlusTools, which should allow a Universe release and a 3.8 release. I also fixed a number of bugs, increasing the number of passing unit tests from 87 to 107 (out of 131). The release process for 0.1 was done manually, and it was a pain, and will only get worse as the number of supported Squeak versions and the number of release avenues increases. So, the last step of 0.1.1 will be an automated build and release script, which will (hopefully) let me build the .sar release files and upload them to squeakmap and universes with a single command.

It may be the start of a build system (like the Sake idea), or just a DSL for publishing packages (akin to Installer); I don’t know.

Stephan Wessels just finished an excellent tutorial for Squeak Smalltalk. Covers everything from installing squeak to building a fun game with Morphic. It focuses on test driven development throughout the entire process. It also shows how to use Monticello to manage code.

A great accomplishment. Thank you, Steve

As I was working on adding email notification to SqueakSource commits, I had the debugger open to a method with two very interesting objects. I wanted to send a few test messages involving those objects to see what kind of information I could coax out of them. But I couldn’t think of a way to have them both open in a workspace or inspector at the same time! How could I test them out?

Then it hit me. You can edit the method here in the debugger, test them out however I want, then save the messages I want into this method, and erase the ones I don’t. It is more than what I wanted, and I had to press zero buttons. I like interfaces like that

I have decided to update ThingLab so that it runs on Squeak 3.8. I am doing this partly to prepare for Peek, my Google Summer of Code proposal, and partly because ThingLab is one of the coolest projects I have ever seen and actually exists. So far, I have it partially running under MVC in Squeak 3.8. I will get it fully working under MVC before I even attempt to get it running under Morphic, probably via the Omni framework. A lot of it works already, but the nice features, like constraint merging and deletion, tend to bring up the emergency evaluator at odd times.

I am also running it under Squeak 2.3, which was the target of the partial update done by Reinier van Loon in 1998. It works a bit better under Squeak 2.3, but constraint deletion brings up the emergency evaluator even there.

I will port the code to Monticello soon. Check back at http://www.squeaksource.com/ThingLab

In my previous post, I told about my experience trying to port a
windows-only C++ program first to the cross-platform glib toolkit, then
to the Squeak environment. I gave up because I thought it would take too
long, and because I was not willing to ask for help using Smallapack, a
Squeak/VisualWorks matrix library. Now I have finished the short project
assigned to me and can look at what happened in retrospect.

The code was even harder to work with than I had imagined it to be. In
spite of being written in C++, the program has no modularity whatsoever,
since all the work is done in a single instance of a single class, and a
mighty fat class it is. The class contains three separate, but heavily
related, algorithms to analyze a stream of data using overlapping window
analysis. So, from the very beginning, there is extreme code
duplication, with up to four functions with nearly the same role; only 3
of the functions were actually shared by all three algorithms.

But what of it? This code is a maintenance nightmare, but will it be
maintained? No! So no maintenance means no nightmare. Harvey, my
adviser, said that he had the same problem I had when he first started
making disposable systems: He had to learn how to write bad code.

So what did I do? I just erased the functions that did something they
did not need to do and made them do what I wanted them to do. Sure; I
accidentally deleted a bit of key functionality, but I could just look at
an old version of the code in subversion to put it back in, and then I
had it: an ugly working program, built from another working program;
neither worse than the other. Each separate functional unit is a
separate program in a separate subversion branch; never to be merged
with the trunk. Again; a maintenance nightmare, but no maintenance
means it is all good.

So is my experience applicable in the real world where code lives on?
I hope not. My code has no future. If it did, a refactoring would be
necessary.

So I wonder, is this more of a problem with early-bound, bottom-up
systems like C++ than it is late-bound, top-down systems like smalltalk?
I don’t know. I started the port with the goal of portability and
maintainability in mind, and chose smalltalk because I wanted to see if
it really was easier to use. So was it?

Refactoring was very easy in Squeak, thanks to the refactoring browser.
However, one problem I found was I could not find a way to limit the
scope of a method rename to just my own classes. I managed to trash the
image twice by renaming a message globally.

Backtracking in Smalltalk was easier than in Subversion, thanks to the
ability to view all changes to each method rather than a coarse
commits. However, I did have a little trouble importing changes after a
crash; the change viewer seemed to file in a range of changes in a
non-temporal order, then abort because it tried to define a method on a
non-existent class. That is probably easy to fix.

The smalltalk debugger actually works. The MSVC debugger just crashed. I
had to resort to printf’s and debugging macros. C++ got totally owned in
this respect.

Squeak does not have a stable matrix library. I sincerely wish I had
been able to Smallapack; the only thing that stopped me was the
deadline pressure. It was not the deadline itself, but the pressure of
“just get it working, now” that prevented me from debugging the very
nice Smallapack library. Without that, Smallapack would be slightly more
stable, and Squeak a pre-alpha dataflow framework.

So, what have I learned? The best way to make a deadline is to cheat.
Therefore, I am going to do whatever it takes to make my next project be
free of milestone deadlines, where functionality always trumps code
sharing. This project is open-source in spirit, especially within our
research department, but the focus on the present seems to prevent
the code from having a future, other than as mutant forms of itself.
Luckily, my internship at Intel seems, at least right now, to be free of
milestone deadlines. I have created milestones to track my progress, but
not for scheduling purposes.

Three weeks ago, I received an assignment from my research adviser to
add a new, simple motion model to our path tracing program. Quite a
simple assignment. However, I do not like the code base for several
reasons:

1. It only works under Visual Studio on Windows
2. It is monolithic
3. It is not modular

So I thought, why not get around to cleaning up this code? I’ve been
wanting to do it all of last semester. First order of business is to get
it to work under Linux, as I didn’t have easy access to Windows or
Visual Studio at home. So I figured out how to use Autoconf and
Automake, and had the program building under Cygwin (but not linking) by
the end of the day. I then went home, thinking I could get the rest done
at home on Linux.

So I started examining what was preventing the program from running
anywhere. I found three things:

1. A windows-only C++ threading package
2. A windows/Mac only UDP Multicast client and server
3. A possibly non-portable Mersenne Twister RNG

Being the over-confident programmer that I was, I figured I could solve
all these problems by refactoring to use glibmm, which had portable
implementations of all these things.

At the thought of refactoring, I had a bright idea: This would be so
much easier in Squeak Smalltalk! Why don’t I translate the program into
Squeak, then I could do the refactoring really fast! Also, I could do
the required visualizations in eToys and Morphic rather than learn how
to program in Max/MSP. So I started systematically translating the
relevant parts of the C++ code into Smalltalk.

By this time, I was about a week into the assignment. I had read enough
of the code to be able to decipher what was going on. Since I had this
understanding, why not make a simple, extensible framework that does the
same thing! So I installed the Smallapack matrix library for Squeak.
Then I set about creating the framework that does what the C++ code did
as a bunch of nested loops.

Two weeks go by, and I finally have a framework to send in data, process
it using the Matrix library, and draws a simple visualization. So I try
running it. Everything breaks. It seems that Smallapack for Squeak is
still in an early version, and diagonal matrices are unusably broken. So
I fixed that, and I find that I still have many of the calculations
wrong, and matrix inversion does not quite work in Smallapack. I did a
bit of Smallapack debugging, but by this time, I am tired of fixing
code, and just want to get it working. But also by this time, I have
spent so much time on this project that I have a few pending assignments
I need to work on.

About this time, Intel gave me a new Windows laptop. Well, that kind of
invalidated the entire reason I had been porting this software in the
first place. So I stopped working on this software for a week and worked
on the other projects I had to do.

That is what I have been up to for the past four weeks. I have the new
motion models in the Squeak version, but that version is broken, and
does not yet integrate into the rest of Smallab, although that would be
pretty easy to add.

So what do I have to show? I have a little bit of broken code with a
fraction of the functionality I started with. Sure, it is cleaner and
a bit easier to work with, but I don’t see an easy way to get it
functional without debugging Smallapack.

I now see that I went about the problem completely wrong:

• I tried to refactor before I even solved the problem
• I tried to change the build system while refactoring
• I tried to change the platform out from under the system while
  refactoring
• I tried to translate the program just to make the refactoring I hadn’t
  even started yet easier
• I tried to rewrite the program on an unstable platform,
  with no commitment to fix the platform
• I tried to fix the program with no way to run it or test it until the
  very end.

This was my biggest mistake.What I should have done is to suck in my arrogance, stay at the lab, and
use Visual Studio to add the new model to the C++ code base. I probably
would then have had a bit of time to go about refactoring the code in a
more leisurely manner. That would have prevented me from trying to do
the entire jump from Windows to glib or Squeak in one step. If I wanted
to port it to Squeak, I should have made the C++ code into a Squeak
plug-in, and wrote some tests for it. Then I should have translated it
method by method, refactoring as I translate, and keep running the
tests.

Finally, I would like to apologize to my adviser: Harvey, I am sorry I
spent my time taking the long path. I know that you value my time and I
see now that all we need is something that works, so that should be
given top priority. Again, I apologize for not following your advice, I
will most definitely have the new model and visualization working by
next Wednesday.

Yesterday was my first day on the internship at Intel. It was a lot of fun, and the cafeteria is amazing. But I found out what I had been dreading to hear: I am to develop proprietary software for Intel to use in-house.

I was considering dropping the internship for that reason, but I talked to my father about it first. We talked about the values of free software (for the first time), and I decided that the cost of developing the proprietary software in-house is not very high, as opposed to distributed proprietary software, and that I will only work on proprietary software if it is not distributed.

Here is how I weighed the cost:

Proprietary software costs everyone who uses it a certain amount of freedom. It costs everyone who does not use it the ability to use it, which is not too bad. The software I will be making is a Python program to string together and record the results of proprietary Intel testing and measuring equipment, and so has very little use outside of Intel. That is why I say that denying everyone outside of Intel the right to use it is not too bad: they will not want or use it anyway.

Now consider the cost to the user: the freedom to share it. First, all users are being paid by Intel,  and second, nobody wants it anyway. Thus the users are compensated, and the non-users have a negligible interest in the software, so I think it is OK to work on this proprietary program.  None of this would be true if the program was distributed; thus I will never work on distributed proprietary software.

At church tonight, there was a presentation regarding the New Mexico Christian Children’s Home. My church has donated food, laundry detergent, and money to that place for as long as I have been there, but this was the first time I ever saw anything about it. I am very impressed; it is definitely a place worth investing in. They raise about 50 kids at a time, and train them all to be excellent people.

The families have times for Bible Study, times for homework, and times for play. The children are taught economics through assigned chores, and raising crops and animals. Each year, they attend the local fair, and they present and auction the animals, and sell the vegetables.  It seemed to be very effective.

I think I believe that a farm is the only place to raise children, and I got a confirming glimpse of that during this presentation. When I become a parent, I will need to keep in mind three things:

- Always set a Godly example for the others. The best way to know how to be Godly is to see others do it.

- Teach the value of work, time, and money. Do not waste much of them, because they are quite valuable, especially work and time.

- Put the children in a community of parents and peers. The more friends anyone has, the better.  I think the same could be said of parents. The more adults a child knows and trusts, the more wisdom and knowledge they can gain.

And I should definitely put this place in my will.