Well, that’s done.0
rzwitserloot posted in me on March 3rd, 2006

Mission Update: Objectives completed.

I received a 9. I heard from a little birdie that I would have gotten a 10 if only no one had actually read my thesis thoroughly (it was inconsistent in parts and I knew it, but there wasn’t really time to fix it in the 2 weeks I had to put it together).

Everyone was either lyrical about the presentation or at least enjoyed it. I’ll say it one more time, Garr Reynolds knows what he’s talking about! While I’m handing out props, Cristiano’s presentation zen-like presenting skills are the origin of the cube transition for switching contexts. I just shamelessly ripped it off.

Audience Pong went more than great. Only Oliver apparently wasn’t fooled, in that a single person probably can beat the room, and my point was the opposite of this. It’s just too much fun to let mere facts get in the way of having a good time.

My parents were very impressed which is a good thing; after all, they were probably beginning to wonder what the hell I’ve been doing these last 8 years. They’ve paid my rent and university tuition so far, so they deserved it.

A big THANK YOU! to everyone who attended and made it a day I won’t forget for a very long time.

UPDATE: Look at the photos alper made!.

Open Source - good for a startup?4
rzwitserloot posted in startup, programming on March 5th, 2006

I like open source. I operate 2 linux servers which do their job very very well, and I’ve personally looked through some code of the JVM (that’s ’shared source’ but the point is there) which helped me figure something out. I even patched mplayer to do stuff I wanted it to do, which pretty much covers all 3 main ideas behind open source from a personal perspective: Use, Looking, and Modifying.

And yet, I don’t think it’s the future for software development as a whole.

Look at webservices like Reddit, basecamp, del.icio.us. None of them are ‘open source’. Yet they all extensively use open source and are in general written by upcoming, creative programmers. Depending on who you ask, web-based services are the future of software. I don’t see any Open Source happening there.

Similar sad story in the desktop world. Linux desktop still sucks (see rant on X window server below*), windows isn’t losing significant marketshare, and Os X in general beats the pants off of either and proves that it is possible to write something that’s just plain better. Why, exactly, do both Gnome (taskbar) and KDE (start menu) look so much like Windows?

Yet, looking at the development world, everything is open source, or at least shared source. You don’t stand a chance without it. Eclipse is open source. So is python and perl. java is a combination of open and shared source. Heck, even large parts of C# are more or less open source. All serious web platforms are open source (i.e.: jboss and jetty, Ruby on Rails, and django).

Might it be that Open Source works well in some areas but just doesn’t work in others?

Even kernel development is an iffy subject. Linux and FreeBSD are pretty neat, but solaris seems to be a lot more flexible in the kernel department. Given the general stability of linux and personally experience in how debian can update a box almost entirely without even letting the webserver go down, I’ll give this one to linux, though I’d love to see some sort of solaris/debian appear.

I think its because of what open source does to a programmer. Open Source is important first, making something cool is important second. That’s backwards. (I recall that gnome was started because KDE is based on QT, which is not GPL). Also, open source offers no obvious advantages:

Eric raymond said, Given enough eyeballs, all bugs are shallow. Yet it doesn’t seem to be true. I’m getting the feeling that a development team unfettered by bureaucracy and endowed with a caring attitude toward their product makes linux security bugs get fixed so quickly. Not its open source nature.

In general open source also doesn’t mean you can scratch your own itch. Many people have whined about X, it’s a very favourite thing to blast. Yet there’s still no one just getting rid of it. The only project I managed to actually patch was mplayer, which had some extremely nice and readable code. Damien Katz argues that Source is a horrible form of documentation and he’s right of course.

Open Source works in programming because the majority of serious users of any programming tools know how to derive benefit from the product’s open source nature. Anywhere else it doesn’t matter; there just needs to be a programming team with sufficient skill and drive to make a great product.

From the perspective of the programmers: Just because you open source your product, doesn’t mean you get a barrage of bug and security audits and patches sent to you. Products get security audited whether or not they are open source - they just need to be used lots.

To summarize, whether to open source or not? Irrelevant. As a client, go with a product backed by a programming team that you seem to trust. As a programmer, if you are making any development-related tool, open source it. If not, don’t. If it gets popular, you can. If you think you can sell it at some point, keep the option open.

Another interesting angle analysing the beneficial economics of open source (in the sense that in goes with free-as-in-beer most of the time) from spolsky here. It doesn’t explain what eclipse has to gain from being open source, but it certainly does explain what IBM has to gain from eclipse being free.

*) For some strange and as yet unknown reason, there are still no serious projects going on to just drop the X window system. Os X has an X11 grafted on as an afterthought. Heck, even windows has an X version. Why isn’t there a KDE or Gnome version that works straight off the framebuffer, no X required? Then emulate an X server like Os X and Windows do.

Micropayments…1
rzwitserloot posted in startup on March 9th, 2006

A one-sentence description of the failure of micropayments convinced me to cook up the idea for tipjar, but a bit more robust treatise on why your average run of the mill micropayment scheme is doomed to fail can be found here (Clay Shirky).

Always nice to read a thorough report on why your future company’s basis is going to fail, especially if every relevant problem doesn’t actually apply to aforementioned company.

It’s so going to work!

(via Alper)

Is that a bubble I see?1
rzwitserloot posted in startup on March 9th, 2006

Writely has just been acquired. By one of the big five - google to be exact.

I wonder how long it’s going to last, but, for now, aiming to build something cool appears to be enough to get acquired by one of the 5 (Yahoo, Ebay/Paypal, Amazon, Google, Microsoft).

Good stuff - makes it real easy to focus. As far as I know, writely hasn’t been around very long. I guess having an actual monetize strategy is just a bonus, then, these days.

Modernized java1
rzwitserloot posted in programming on March 11th, 2006

I’m less than pleased with ruby and python so far (tried RoR and django). Yet, in pure language syntax both are miles ahead of java. It’s the enforced type-lessness of it all which really bothers me, and the reason it does so is because I keep having to resort to external documentation to figure out the proper names for all sorts of things, instead of having my IDE just pop up my choices as it occurs in eclipse or netbeans (java).

Learning to use new toolkits is tons easier with such popups, and so far I am still not convinced that you can just throw compile-time checks out the window as long as you have proper unit tests. Given that any serious project spends as much time, if not more time, debugging compared to creating, any move that makes debugger more difficult has got to be a bad one. There’s also the implicit compile-time checked documentation. These beat comments hands down, simply because comments are both difficult to machine read, and they can get out of date.

I’m really looking for a best-of-both-worlds language. Something with static typing (but with an option to escape it when it is unwieldy) yet with the language improvements of python and ruby.

A new language which compiles to Java bytecode and is such as directly ‘pluggable’ into any java system has shown up, and it’s called groovy. Java 1.6 (Mustang) will even support it out of the box. In many ways its trying to be this panacea language.

I’ll first explain what groovy is trying to do, then I’ll dive into some improvements that groovy doesn’t have yet.

Remember first that java is a lot of things: It’s a language (which groovy tries to make better), but also a runtime library, a distribution model, a virtual machine, and a security model. Groovy just tries to fix the language it self, but in my opinion all the other stuff is pretty good already.

Let’s start with a simple custom sort example. This example shows function objects, as well as some basic collections API (creating and manipulating lists of data).

python:

def reverseNumeric(a, b):
    return b-a
a = [5, 1, 3, 2]
a.sort(reverseNumeric)
print a

JAVA5:

Comparator reverseNumeric = new Comparator<Integer>() {
    public int compare(Integer a, Integer b) {
        return b-a;
    }
};
List<Integer> a = Arrays.asList(5, 1, 3, 2);
Collections.sort(a, reverseNumeric);
System.out.println(a);

These little fragments of code do the exact same thing, and in the exact same way. I think it highlights exactly what’s wrong with both python and java. First things first: notice how ‘reverseNumeric’ is a function object in both examples. Suggestions that java doesn’t support function objects are misleading; java doesn’t support functions as direct objects, but you can declare new classes on the fly, which amounts to the same thing. The syntax for this is ridicuously wordy, but we’ll get to that.

What’s wrong with the java code: Arrays are supported ‘natively’ but lists and maps aren’t, leading to ugly hacks like the new Arrays.asList method. There is of course the wordy function object. A third problem arises in java’s strictness - mixin functionality is difficult to pull in java, so we have a Collections.sort, which is a bad thing - it should have been a.sort(reverseNumeric), somehow. I sort of understand why sun hasn’t done it that way, but this does indicate a clear weakness of the language. The fact that it was difficult to implement sort into all relevant collection classes is not good.

The python problem is much easier to identify: It’s all dynamic. There’s less namespacing going on (module members have a namespace in java, ie Gun.shoot() is something else compared to Camera.shoot(), assuming Gun and Camera are classes or interfaces, whereas in python that’s not important and there’s just shoot(). I take it the wording of the method and classes illustrates why this can sometimes lead to shooting yourself in the foot). Also, the ubiquitous autocomplete of modern java IDEs such as eclipse and netbeans, which lists all possible parameter choices/method names/whatever applies cannot, by definition, work nearly as well in a dynamic language. This is problematic in large projects, and doubly so if everyone mixes and matches camel case and underscores, like for example python.

Fixing the java code is mostly purely a language problem though. The underlying JVM, basics, or static nature aren’t to blame. Here’s my first attempt to fix the java fragment. Let’s call it wishful java:

Comparator<Integer> reverseNumeric =
   new Comparator(Integer a, Integer b) {
    return b - a;
};
List<Integer> a = [5, 1, 3, 2];
Collections.sort(a, reverseNumeric);  //still ugly
System.out.println(a);

It’s trivial to write a program that compiles this example to the previous fragment. yet this code is much, much cleaner. It’s still more wordy, but this time it’s not overly wordy. The extra words spent (mostly in naming the List as explicitly being both a List and of type Integer, and same story for the comparator) give us something in return which is usually worth the trouble: compiler-checkable certainty. Eclipse or netbeans users know why it’s so damn handy to make sure your compiler/IDE can know exactly what parameters you want.

We’ve added a native syntax for creating lists. We’ve also made it possible to implement single-method interfaces such as Comparator in one swoop. this ‘works’ because there is only one method defined by the java.util.Comparator interface, namely the compare method. If there were more, this kind of syntax couldn’t work, but then again, if there were more, a simple function object can’t get the job done anyway. Note that if the <Integer> markers bother you, you could allow an aggresive type inference strategy into the language, where lack of explicit generic types means they get inferred (from the parameter types in the first case, and from the contents of the list literal in the second case) if possible, but overriding is still possible.

This new example still has the power of java. For example, if we explicitly make our list, semantically, a list of objects:

List<Object> a = [5, 1, 3, 2];

We get a compile-time (read: development time, you get an instant red underline if you try, assuming you use a proper IDE) error, because our comparator is not set up to deal with comparing objects, only integers. We’d have to either make the list integer-only, or adjust our comparator to be capable of handling any objects. This is good - it allows you to catch bugs early, and it adds implicit documentation. In other words, for the same reason that naming your variable ‘userName’ instead of ‘a’ is generally a good idea, so is explicitly documenting that a list is restricted to certain types. This beats comments any day, because comments get out of date. Compile-time checked restrictions do not, because the moment they fall out of date (ie: because we allow our list to contain any object instead of just integers), we get compile-time errors.

While so far we’ve got a pretty nice language, our first draft of wishful java has two remaining problems.
First, there’s still the ugly Collections.sort. We’ll try to solve that in a later post. The second problem is that, while usually all that explicit typing is very useful, sometimes it’s superfluous. Think, for example, of unit tests. Unit tests are never part of an API and never will be, the code tends to be very trivial, and it needs to be quickly written. Entering in all the type information may just be too much work. Also, when writing your unit tests before your actual code, as is suggested by most experts on the subject, it can be useful to not explicitly type stuff - you’ll figure out such details later.

We could solve this typing problem in our wishful java, actually. Here’s the second draft of wishful java, in case this code occurs in a place where we don’t want or need explicit typing:

reverseNumeric = function compare(a, b) {
    return b - a;
};
a = [5, 1, 3, 2];
Collections.sort(a, reverseNumeric);  //still ugly
System.out.println(a);

We could go even further and eliminate the ‘compare’ string, but as the java runtime library is still strongly and strictly typed, we could only use such unnamed function objects within our own code. Just adding the ‘compare’ text is no big deal, though, I’d say. The reason this can still be compiled to either ‘java’ or JVM bytecode is due to the reflections API.

Turns out all this ‘wishful’ java is very much like Groovy. Groovy does almost exactly what I suggested earlier: It compiles groovy to ordinary java bytecode, which can freely interact with plain old vanilla java5 code. Just like our two examples of wishful java, in groovy you can pick - you can type stuff, or you don’t.

Groovy also goes on with the nice syntactic sugar, beyond what ruby and python have to offer in some cases. For example, groovy supports native regexps. These generate compile-time errors if the regexps are wrong. Valid groovy:

Pattern p = /(foo)?[bar]/;

The nice thing about this is that in theory it’s faster in the extremely unlikely event speed matters, but mostly the syntax is more obvious (// always means a regular expression) and you get a compile-time/development-time (if your IDE supports it) error when there is a syntax error in your regexp. Again, the benefit of having the code document itself is very important.

Groovy is new and I expect they’ll be doing more and more nice stuff. Of course, unfortunately, in some ways they’ve gone too far. Groovy makes a big deal out of supporting straight closures, but in practice you can’t use these with any existing java code. It does not (yet?) support the ‘function interface’ syntax introduced in the first example, either. Will Groovy coalesce to the same level of robustness of java, combined with the comfortable programming style of python and ruby? We can only hope so!

---

Extra treatise - feel free to skip:

We’re still stuck with that ugly Collections.sort. The reason why it works this way is somewhat unclear, but let me try to explain anyway: First off, making some sort of feature that automatically calls the (static) Collections.sort method anytime:

a.sort(comparator);

shows up would take either mixins or macros, neither of which are supported by java. As far as macros goes - basically a good thing. If you allow macros like this you’ll quickly degenerate into either C preprocessor hell, or the compiler doesn’t know what’s going on and then you lose all the static typing advantages. Mixins would be much better but so far no one has come up with any serious candidate for syntax.

The basic java to do this would be to add sort() to the List interface, and then let each implementation figure out how to implement sort. Probably just by writing:

public void sort(Comparator comparator) {
    Collections.sort(this, comparator);}

A java1.6 feature is that interfaces can have static methods, which would mean this code could be cleaned up some to:

public void sort(Comparator comparator) {
    sort(this, comparator);}

by moving the sort static method into the List interface itself. Unfortunately, this tactic still has two problems: For one it’s sort of ugly that all implementations have to write explicit code just to implement default behaviour, and secondly, if you imagine for a moment that this sort thing is an upgrade from a previous version, then all third party list implementations just became broken, because they do not have this explicit sort method covered.

Still, sort has been added in java1.2 if not earlier, and by now if I were sun I would have just stuck a sort method into the List interface. Just because it’s a bit awkward to pull of syntactically doesn’t mean that you have to bother users with a godawful Collections.sort syntax.

Here’s a very nice solution: automated delegation. This particular trick is so powerful, it can actually allow an object that implements both Camera and Gun to have two different shoot() methods. The reason it can work is because all calls in java have a parent type.

Current situation:

public interface Camera {public void shoot();}
public interface Gun {public void shoot();}
public class CamGun implements Camera, Gun {
    public void shoot() {
        System.out.println("This can't be good!");
    }
}

In java it’s impossible to do this properly and fortunately at least this is trivially obvious when trying to program our CamGun. I submit that this is a better option compared to silently allowing it, but due to the way java calls are always typed it is in theory possible to figure out which method is intended in some cases. Example:

//Clearly, fire the gun:
Gun camgun = new CamGun(); camgun.shoot();

//Clearly, make a picture:
Camera camgun = new CamGun(); camgun.shoot();

//Don't know what to do:
CamGun camgun = new CamGun(); camgun.shoot();

//Fire the gun:
CamGun camgun = new CamGun(); ((Gun)camgun).shoot();

In all but the third scenario, the intention is obvious. Forcing a programmer to pick a supertype when the intention is unclear, such as in the fourth example, is also acceptable, especially because this situation will in practice not occur all that often anyway.

Any solution to this problem tends to solve the multiple inheritance problem as well - where two implementations of the same method are inherited, and the compiler has to prioritize one over the other, which in practice (see C++) is a recipe for disaster.

In python or ruby, where functions are first class objects, this problem can never be resolved properly. They do not have the notion of an interface or a supertype nearly as clearly as in java, and the notion of casts or types are foreign to the python/ruby class of programming languages regardless.

The only extra actual syntax required to support separate methods is this:

• Allow interface methods to have a method body, which counts as the default implementation
• Allow any method declaration to explicitly specify what it is overriding. There’s already an @Override annotation, which can simply be extended to take 1 optional class parameter

An example of this:

public interface Camera {
    public void shoot() {
        System.out.println("BANG!");
    }
}

public interface Gun {
    public void shoot() {
        System.out.println("CLICK!");
    }
}

public class FlashingCamGun implements Camera, Gun {
    public @Override(Camera) shoot() {
        System.out.println("FLASH!");
        Camera.shoot();  //just 'super' would be unclear.
    }
}

This code does everything we want, neatly sidesteps (in that it generates compiler errors, and fail-first is a good idea for this sort of thing, otherwise you have a hard to trace bug on your hands) any multiple inheritance or namespace overlap issues, yet would make implementing the Collections.sort thing an absolute breeze: Just add a default sort() implementation to java.util.List.

As a bonus we get a language which allows CameraGuns. No mainsteam languages I know of can handle cameraguns. So far cameraguns have just been just as an example where explicit type checking is good because it allows you to detect them. This cameragun is actually implementable.

---

I’ve listed a bunch of other possible java improvements a while ago here.

Tipjar - we gotta move!4
rzwitserloot posted in startup, programming on March 14th, 2006

Random check at the success of the tipjar idea: It’s currently the highest rated comment on an article by Shirky about how (mandatory) micropayments will always fail. I didn’t make that comment, someone else did.

I’m more and more convinced that we’re “on the clock” as it were. This idea is so great, it’s really just a matter of time before someone else runs with it.

In other words, I’m so utterly convinced we need to move move move on this, I’ve given in; Cris wants PHP or some similar slapdash language - we’ll go slapdash language. Anything to get a prototype out the door. Once a real use case crystallises out of what users actually do with a tipjar service, I’ll just rewrite it in hibernate/trails, RoR, or whatever robust simple web-framework is available then.

I’m mostly concerned about banking and security issues. We are dealing with payment here. Any code stuck together with spit and a prayer offends the security hobbyist inside me greatly.

Speaking of robustness, just read an interesting article about the horrors of ORM libraries. Makes one wonder how the ORM of i.e. django can ever scale up far enough to stick with tipjar in the event that it does take off big.

The answer is probably simply that any toolkit made by oracle is such a bureaucratic debacle that they managed to turn a relatively manageable problem into an utter mess, but ORM -sounds- like it should be difficult to build in such a way that it scales well.

Tipjar in PHP…0
rzwitserloot posted in startup, programming on March 17th, 2006

Spent last night learning PHP. A decidedly unsexy language in that it doesn’t introduce any new concepts whatsoever, it’s proven to be even easier to pick up compared to python or even javascript.

Fortunately a lot of features introduced in PHP5 allows pushing the language quite far before you hit the unmaintainable dregs usually contributed to PHP-based projects that outgrew their language.

Right now I’m just building ORM on-the-fly, creating PHP classes that do the rough SQL work. It’s not much work at all, and any fancy dbase manipulation work for optimization purposes can always be added in later without having to learn anything but SQL which I mostly already know.

Any project that fits securely in a LAMP stack and works decidedly top-down in nature (most projects are but something like writely probably isn’t) can be built with this stuff without too much fuss. And for small/simple projects that’s really all that matters, I guess.

Evangelising Open Courseware0
rzwitserloot posted in education on March 17th, 2006

While we’ve left the concepts of education made easy behind, apparently the educational world is (slowly) picking up on the economy of scale effect inherent in digital media.

Of course, The UK and The US are leading the way in innovation, again.

The Early Vote2
rzwitserloot posted in politics on March 19th, 2006

I’ve noticed an interesting and disturbing side-effect of today’s connected society:

Early votes have significantly more impact compared to later votes, sometimes exponentially so.

‘votes’, here, applies to almost anything. Reviews, popularity, memes, political parties, blog articles, etcetera.

This hypothesis is a bit hard to prove, but I’ll try anyway.

On ‘aggregator’ websites such as reddit.com and digg.com, users ‘vote’ on submitted articles, and those articles with the most votes get displayed on the front page.

Because people tend to want to watch high-rated articles, entries with higher ratings see more traffic. Assuming that the article is not a complete turd, on average each view generates a slight positive change, resulting in the whole thing spiralling out of control fast.

In other words, early votes cause more votes, which cause more votes, etcetera, making those early votes pivotal.

This very trick was used in a stock market manipulation scam recently.

Full story (if you already know it, feel free to skip on to the horizontal divider):

---

Check out the ratings of articles on reddit and especially digg (note that the ‘this article is under review’ banner has appeared only weeks after the initial storm of diggs).

Interestingly, the digg item on the scam, here, has only 650 diggs; less than half of what the original article has. Still overwhelming and more than enough to make front-page, though.

The short story here: 6 people worked together to get the article 6 quick diggs. The apparent legitimacy that these initial votes imply combined with pushing the right buttons (digg sheeple are known to vote anything with ‘google’ in it up) combines to launch this story into stardom.

---

Similar negative effects can also be observed - post an item at the wrong time, and receive some early negative votes, and the article drops off the radar, never to be seen again, regardless of quality. At least reddit is trying to do something about it.

These trends aren’t unique to reddit or digg, though they are most striking there because they can be quantified somewhat.

The year 2000 craze back in the late 90s, and currently, the peak oil theory are showing up all over the place.

Peak oil has been around for decades, and aside from its introduction, it has never ever correctly predicted the oil price. Not even close. From time to time, however, peak oil surges, makes it into major papers, and even otherwise sane people write blogs about it. Not to say that such people drink that koolaid, but the point is - lots of people are devoting attention to this issue, linking to each other for legitimacy.

Clearly, attention garners more attention. A vicious spiral.

That’s not all though; not only attention is controlled by the early vote - legitimacy is another area which relies on that crucial early vote.

Reputable news organizations and the lights of the blogging community do not normally blindly post every rumour they hear. They research. Unfortunately, research is becoming more and more difficult. Crafting fake news is easier than ever. The average internet user I confront about such things tends to believe that google news, wikipedia, digg, reddit, slashdot, and just about any other popular site is a reputable source of news, yet time and time again they fall for fake news. Even mainstream media have often been the butt end of practical jokes. You could try spending a lot of time, money and effort being extremely diligent in chasing down your news links, but apparently no one cares, and in this age of soundbytes and instant gratification, if you aren’t as good as the other guy, you wither away into oblivion. the only site I really know of that is popular in its community yet has a great diligence record is think secret, a site dedicated to rumours surrounding apple.

In short - once a story seems initially legit, the blogosphere, the aggregator world, and the entire news world in general will automatically turn it into a major story for you.

A spectacular breakdown of the notion that crowds are smart, in other words (The author of the book linked does explain that independent review works best, but so far few seem to get that).

The lesson here? When you vie for the attention of the world, make sure you’ve got a bunch of cronies lined up to evangelise the heck out of your product, preferably all at the same time, for about a day. Then just sit back, relax, and bake in the spotlight.

Of course, with some luck at some point people will wise up to this and develop a strong sense of skepticism. Here’s to it!

New java puzzler4
rzwitserloot posted in programming on March 22nd, 2006

Oliver, a friend of mine, decided to upgrade his java knowledge from java1.4 to java1.5. He managed to stumble into a neat java puzzler* almost immediatly.

What does the following code print, when compiled and run with java 1.5?

import java.util.Arrays;

public class Puzzler {
    public static void main(String... args) {
        test1();
        test2();
    }

    public static void test1() {
        int[] array = {1, 2, 3};
        System.out.println(
         Arrays.asList(array).size());
    }

    public static void test2() {
        Integer[] array = {1, 2, 3};
        System.out.println(
         Arrays.asList(array).size());
    }
}

Answer and explanation/discussion is in the first comment.

---

*) For those of you who don’t know what puzzlers are: Josh Bloch and Neal Gafter, both java bigwigs (incidentally also both working for google these days instead of sun) wrote a book called Java Puzzlers. It contains a lot of code snippets with a catch. Aside from testing your knowledge of java ideosyncracies, they also reveal design mistakes in the java language and in that way teach you what to avoid. In fact, most puzzlers come with an alternative ‘right’ way that avoids the problem or weird thing that makes the code snippet puzzling.