... indistinguishable from magic
effing the ineffable since 1977


Show All

Recent Posts


Taking advantage

Thanks to everyone who's emailed or commented supportively. Jeff in particular, thank you for a much needed laugh, and I too hope that what I actually have is Nullable<Cancer>. Also Jeroen and Mark for the thoughtful emails, Jim for the comment on his own blog, and everyone at work and everyone I know in person for their thoughts and prayers (I may not believe in prayer personally, but I appreciate the thought from people who do).

I arrived at work this morning to find that lots of people were sick with colds, headaches, etc - and that's not including the people who were out sick. The conversation went something like...

Coworker 1: "We're all a bunch of invalids today..."
Me: "Well, I have cancer -- I win!"
Coworker 2: "My husband's sick and he's also having a colonoscopy"
Me: "I have cancer -- I still win!"
Coworker 2: "Fair enough"

Normally when I or members of the family are sick I'll struggle through and work from home, or sometimes feel guilty and leave Janene to suffer while I go into the office because there's stuff that simply needs me to do it. But right now even when I'm in the office I can't really focus, and besides, if there's anything in life that entitles you to take advantage and take a little bit of a break to recuperate, it's having cancer.

So for the rest of this week I've pledged that I'm not going to feel obligated to get any work done. That's not to say I won't do anything that will benefit my work, but I'll focus on stuff I want to do with long-term benefits, rather than the never-ending stream of kludgy customer-specific fixes that drive my stress levels through the roof at the best of times.

(By the way, this means among other things that I won't be receiving any email - if you want to reach me, use the gmail address at the bottom of every page of my site)

So here's a list of projects, work-related and not, that I intend to attempt over the next few days:
  • Get japitools handling some JDK5.0 features. I've started this already - I have a version of japicompat that can theoretically cope with a lot of the "interim" japi file spec version 0.9.7 that supports some, but not all, of the 5.0 features. Unfortunately I don't have any way of creating japi files in that format: Jeroen, if you're reading this, do you have any tips on how to get the necessary metadata out of the class files?
  • Get nrdo integrated into the new Visual Studio 2005 beta in the cleanest possible way. This means using List<T> everywhere, nullable types everywhere (an act of faith that these will be adequate by final release) and somehow hooking it into the build system in such a way that, hopefully, we don't require two separate extra project files and to rebuild the whole thing twice just to pick up the generated code.
  • Produce a release of NRobot to include the new security code, and announce it in enough places that perhaps some people will try producing robot implementations...
  • Watch all three LotR extended editions, especially RotK which I've never seen even the standard edition of.
  • Continue to push the nullable type issue with Microsoft any way I can find.
  • Learn as much as possible about Visual Studio 2005 and how the migration will impact cmScribe. I think that actually doing a migration will take longer than the few days I have, but hopefully I can at least figure out what the biggest issues will be.
  • Catch up on DVR'd TV shows that I haven't watched yet.
  • Oh yeah, recover from the surgery...


"Possible tumor"

Well, yesterday I went for an ultrasound. While it was being taken, the person taking it told me that the results would be sent to my doctor "by late tomorrow". So when the radiologist told me that they'd been sent before I even left the office, I had a strong suspicion that something serious was wrong.

So I called the doctor's office as soon as I got home and the conversation went a little bit like this:

"Hi, I just had an ultrasound and I was told they'd sent you the results."

"You just had it today?"


"Well, they don't usually send in the results that soon."

"I know that, but they told me they'd sent them in."

I think she got the hint from my tone of voice with that last comment and went to look. Then the remainder of the conversation went a little bit like this:

"Yes, your results came back ABnormal, which means it could be an infection, or it could be a tumor or a mass. They'll need to do a biopsy. You'll need to come and see the doctor tomorrow. What time is good for you?"

"Is 7pm okay?"

"Okay, and this is very important, so don't miss that appointment."


Thanks for the wealth of detail, bitch.

Soo... at 7pm today I'll go and find out what happens next. Since I wasn't given any instructions not to eat or anything, I doubt they're doing the biopsy today, which means that all they're going to do today is probably to talk to me about what they're going to some other time. Which leaves me wondering... why waste time having me go in today at all? Why couldn't they just tell me whatever they're going to tell me over the phone, so that we can get on with the actual business of getting the biopsy done and finding out whether I'm going to die or not?

My coping strategy in stressful situations like this is to focus on something comparatively inconsequential and enjoyable. On previous occasions I've been known to throw myself into reading books (this has limited success because I finish a typical book in about 3-4 hours) or programming. Yesterday I should probably have thrown myself into working, but somehow I didn't have the motivation for that (I wonder why?). Instead I threw myself into my mini-campaign to get nullable types fixed in C# 2.0, posting that mammoth blog posting (which I'd already written the bulk of, but needed some reformatting and lookup of links) and making sure that I linked to it[1] from everywhere I could think of - especially Cyrus's "please give me feedback" blog posting, since he'd already promised to forward my concerns to the language design team. Actually, if it ends up making a difference, the time spent doing this will result long term in my company avoiding way more in lost productivity due to bugs than the few hours I spent doing it.

Today I'm going to be back in work. I'm already running an hour late getting there in the first place (I'm writing this post on the train), and I need to leave early tonight to make sure I make that appointment. And I have no idea how productive I'm going to be while I am there. I'll do my best, and hopefully they'll understand why I may be a little distracted.

[1] Actually I managed to screw up the link and link to an internal-only URL all over the place. Fortunately I got at least one link right and someone at microsoft corrected all the rest for me. Needless to say, "D'oh!".


How to completely screw up a good idea: Nullable types in C# 2.0

(Update way too many months later: the final release of 2.0 fixed most (but not quite all) of these issues. Eventually I'll get around to producing an updated table to show how things improved and what didn't. Sorry to everyone at MS who worked on this for not making this update sooner)

So Visual Studio 2005, aka Whidbey, Beta 2 has finally been released, so I guess this is as good a time as any to rant about my biggest pet peeve in the new release. For the record, on the whole I'm thrilled and excited and can't wait for my (3.5Gb!) download to finish to start working with it (and for Mono to catch up with the new features so I can use them in Free Software too). C# is hands down the nicest language I've ever worked with and the 2.0 release takes a good thing and makes it even better.

EXCEPT for one of the new features.

When I first heard that C# 2.0 would have nullable types built into the language, I was delighted. The lack of built-in support for nullable types was one of the very first downsides I ever encountered in the language, and one of my first tasks when porting nrdo from Java to C# was to attempt to rectify this lack, at least for the datatypes that nrdo supports. Fortunately C# is expressive enough that I was able to implement Nint, Nbool, NDateTime, etc classes which get reasonably close to providing the desired behavior. There have always been some limits to how close I could get to the ideal, though: there are some scenarios where you simply need some help at the language level. So with C# 2.0 we finally get that help, right?

You can guess the answer. No, we don't.

In fact, Microsoft's team of language designers (who as we've already established are pretty good, having produced such a kickass language in the first place) have managed to come up with something worse than what I was able to put together without any compiler changes using the language that they invented, over two years ago.

Let's review the behavior you want from a nullable types feature. It's pretty simple -- in fact, it's already there in the language. Reference types (classes, interfaces, delegates, etc) are already nullable. The whole concept of nullability and how it should behave is already established. The goal is, or should be, to capture that concept and allow it to apply to value types (structs, enums, and the builtin types like int and bool, which are actually structs under the hood) with as few changes as possible to programmer expectations. Bonus points if you can micro-optimize for close-to-the-metal performance, but this is a secondary concern.

(Some people may argue that point about performance. To me, it's self-evident: every use case I've ever seen given for nullable types has a pre-existing bottleneck in disk or network IO, so a few extra CPU cycles handling the nullability is going to make absolutely zero difference in practice. This is a moot point, however: I'll show later that it's perfectly possible to get the behavior right without any performance penalty.)

The goal of capturing the existing behavior of reference types and applying it to value types is pretty much sufficient to define exactly how an ideal nullable type feature should work, because for any given code construct using nullable types, you can just subsitute in an existing reference type and specify that the behavior should be the same. Here are some examples using ints and strings.

Reference type Nullable type (ideal) Nint C# 2.0 Comments
string x = "hello"; int? x = 1; Nint x = 1; int? x = 1; Pretty straightforward so far.
string n = null; int? n = null; Nint n = null; int? n = null; It's hardly nullable if you can't put null in it.
if (n == null)... if (n == null)... if (n == null)... if (n == null)... Again hard to get wrong.
x.ToString() x.ToString() x.ToString() x.ToString() 2.0 and Nint both cheat here by providing ToString() methods that call ToString() on the underlying int. A tie - it's cheating, but it works.
n.ToString() throws NullReferenceException n.ToString() throws NullReferenceException n.ToString() throws NullReferenceException n.ToString() returns an undefined value Since 2.0 doesn't really store null as null, it ends up invoking ToString() on an undefined value. No exception, just a meaningless result. In my book, invoking a method on null is the definition of what should be a NullReferenceException. Advantage Nint, but admittedly this isn't a terribly big deal.
object o1 = x; object o1 = x; object o1 = x; object o1 = x; This looks like it worked...
object o2 = n; object o2 = n; object o2 = n; object o2 = n; And so does this...
if (o1 == null) is false if (o1 == null) is false if (o1 == null) is false if (o1 == null) is false And so does this...
if (o2 == null) is true if (o2 == null) is true if (o2 == null) is true if (o2 == null) is FALSE? Nint got this right, but C# 2.0 inexplicably thinks that returning false is a good idea here. If you understand the implementation, there's a perfectly good explanation for why this happens, but a low level explanation isn't an excuse for the language flat-out lying to me. In order to make this work right we need to change the last few lines...
object o3 = x; object o3 = x; object o3 = x; object o3 = Nullable.ToObject(x); Holy verbosity batman! And not even a warning if we forget to do this, which we certainly will most of the time.
object o4 = n; object o4 = n; object o4 = n; object o4 = Nullable.ToObject(n); Okay, so once we've got them into objects, we can easily cast them back, right...?
string s = (string) o3; int? s = (int?) o3; Nint s = (Nint) o3; int? s = Nullable.FromObject<int>(o3); As if the ToObject line wasn't bad enough, now we have to remember to stick <int> in there at the right place for no obvious reason. Actually, the natural approach with casting would work as long as we didn't use ToObject and lived with a null that isn't actually null. But if you want sane behavior, you need this insane syntax.
string s1 = (string) o1;
string s2 = (string) o3;
int s1 = (int) o1;
int s2 = (int) o3;
int s1 = (int) (Nint) o1;
int s2 = (int) (Nint) o3;
int s1 = (int) (int?) o1;
int s2 = (int) o3;
2.0 actually gets this one right for o3, but only because we had to jump through hoops to create o3 in the first place. With Nint (and 2.0 if you forget to call ToObject) you have to perform this bizarre double-cast because the value in o3 is actually not an int. In theory, advantage 2.0; in practice it's a wash because you can't get to the correct behaviour without working around the wrong behavior first (and actually, Nint has ToObject and FromObject methods as well, but nobody ever calls them - with null behaving correctly, it turns out to be easier not to bother and just use the double-cast when necessary). Notice that by this point neither version is matching the ideal.
string y = t ? "hello" : null; int? y = t ? 1 : null; Nint y = t ? 1 : (Nint) null; int? y = t ? 1 : (int?) null; This one is truly a genuine tie. It's an extremely common construct when using nullable types and it's a huge pain to always have to remember the cast - I can tell you this from bitter experience. This is one of the most obvious places where the language could have helped us all out, but they didn't bother.
IComparable c = x; IComparable c = x; IComparable c = x; IComparable c = x.Value; Nint cheated here - because the underlying type is hardcoded I didn't need to do any magic to implement the same set of interfaces. Still, it gives the right result. 2.0 makes no attempt to even bother.
s.ToString(fmt); DateTime? dt;
NDateTime dt;
DateTime? dt;
Again, my implementation cheated but got the right result; again, 2.0 doesn't bother.
Hopefully it's becoming clear by this point that although Nint, NDateTime and company have serious limitations compared to the ideal, the behavior of 2.0 is significantly worse. So what were the creators of 2.0 thinking? These are clearly smart people, how did they get it so badly wrong?

Well, I can only speculate, but my guess based on their public statements is that they made two fatal mistakes:
  1. Deciding on an implementation first and then fitting the behavior to that implementation, rather than designing the behavior first and then finding a way to implement it.
  2. Treating performance as if it were the critical factor and correctness and intuitiveness were entirely subservient to that overriding need.
It seems to me that the most likely route they took to the current state is by first making the decision that, "for performance reasons", the Nullable<T> type must be a value type. Once they'd made that decision they designed the entire behavior around what was easiest and most natural to do with value types, and never even thought to try to match the behavior of reference types.

The most bitter irony is that in fact it would have been perfectly possible to get the right behavior while still keeping the performance of a value type. If they'd provided a custom attribute which allowed a value type to override the standard "boxing" behavior, all the right behaviors would naturally fall into place. The problem wasn't with the fact that they decided to use a value type, but rather that they made that decision too soon and let it drive their design.

I, and others, have reported these issues to Microsoft in their Product Feedback Center. Here are some links: Notice in particular Microsoft's response to the last one as an exercise in missing the point. In response to the complaint that using "(int?)null" is hard to remember, confusing, and needlessly verbose when compared to just "null", what did they suggest as an alternative? "default(int?)"! At least my suggestion has null in it somewhere. And having the casts to and from object work right would be "confusing" because it would be different from the way other value types work. Earth to Microsoft: nobody understands how value types work. Nobody cares. The reason the language did such a great job in general is because you don't have to care, because it just works the way you'd expect. This doesn't -- not even close -- and that's what's confusing.

In retrospect I should have probably argued harder about these issues when I first filed or discovered them. My style of argument is usually to simply try to get people to realize the manifestly obvious rightness of what I'm saying ( ;) ), rather than attempt to use any credentials of my own, because I don't really have any that would impress the world's largest software company. So when I saw that response that missed the point so utterly, I gave up -- if they couldn't see the manifestly obvious wrongness of their own position, they'd never see the rightness of mine. But what never occurred to me is that in this area I actually do have some fairly unique credentials - I've implemented, worked with, and led a team of 4-5 people using, my own implementation of nullable types over several years. When I say "people will find this confusing", it's not just a guess: I've actually presented it to people and seen them find it confusing.

Unfortunately, it'll probably never get fixed now. Backward compatibility and all that. I'm trying to point Microsoft people to this blog entry in hopes that some of the worst misfeatures might still get fixed; one example is here, and Cyrus has responded by forwarding my issues to the language design team. There may yet be hope... (oh, and you can find the code to Nint and my other nullable type wrappers here).


Goo-tri-Matic redux

When writing my previous post, I wondered whether the paragraph I quoted was an intentional HHTG homage or just the same idea invented independently. I just realized that if the reference was intentional, they embedded an even more subtle hint, because DNA can stand for Douglas Noel Adams as well as Deoxyribonucleic acid.




From the "Google Gulp" home page:

Think a DNA scanner embedded in the lip of your bottle reading all 3 gigabytes of your base pair genetic data in a fraction of a second, fine-tuning your individual hormonal cocktail in real time using our patented Auto-Drinkā„¢ technology, and slamming a truckload of electrolytic neurotransmitter smart-drug stimulants past the blood-brain barrier to achieve maximum optimization of your soon-to-be-grateful cerebral cortex.

However, no one knows quite why it does this because it invariably delivers a cupful of liquid that is almost, but not quite, entirely unlike tea.