A plea to the powers-that-be at Google

Would you please pretty please with sugar on top fix group search?

You get what you pay for

There have been a lot of stories about drywall from China used to rebuild the Gulf coast after hurricane Katrina producing a host of unpleasant problems including foul odors, corrosion in air conditioning systems, and a variety of health problems. As if that weren't enough, now it turns out the stuff might be radioactive.

Sarah Palin: Quitter

Sarah Palin has announced that she is going to resign as Governor of Alaska to pursue the presidency in 2012.

Palin was quoted as saying, "Some are going to question the timing of this."

Gee, ya think?

By leaving office early, Ms. Palin, a 45-year-old mother of five, will be able to travel around the country more freely and not be constrained by the duties and responsibilities of being a governor.

Excuse me? She doesn't want to be constrained by duties and responsibilities? And she wants to run for president? I wonder, will she resign the presidency mid-term because the duties and responsibilities of that office make her miss out on all the good shoe sales at Bloomingdales?

Good grief.

More progress

India has decriminalized homosexuality.

Say what?

From the twisted mind of Fox news comes this mind-bending pretzel of twisted logic:

Michael Scheuer : The only chance we have as a country right now is for Osama Bin Laden to deploy and detonate a major weapon in the United States because it's going to take a grass-roots bottom-up pressure because these politicians prize their office, prize the praise of the media and the Europeans ... it's an absurd situation again: only Osama can execute an attack which will force Americans will demand that their government protect them effectively, consistently, and with as much violence as necessary.

Glen Beck: If I were him that would be the last thing I would do right now.

So let me get this straight: According to them, we're in a situation where we're safe from everyone except Osama Bin Laden, and it's in Osama's strategic interests not to attack us.

And they see this as a problem? The only chance we have is for Osama to deploy a major weapon? Which he probably won't do? And this is somehow bad? Un fleepin' believable.

Gay marriage redux (or is that reduced?)

For those of you confused by the gay marriage debate, Patrick Farley offers this handy reference guide.

My Dinner with Erik

This post is about a man named Erik Naggum, who died in Oslo, Norway on June 20, 2009, apparently after a long bout with chronic illness. I say "apparently" because I didn't know Erik very well, and the official accounts of his death have been light on details (which is fine -- it's not really anyone else's business).

The reason I'm writing this is because despite the fact that I didn't know him very well, we had a very close, um, association over the years (it will become clear presently why I had to struggle a bit to find the right word to use). My name, for better or worse, has become inextricably linked with his in certain circles, and so I thought it would be appropriate for me to write something on the occasion of his passing. Death often provides an occasion for reflection.

I would like to start by setting the record straight about a couple of things. Erik and I were often very publicly at loggerheads and a lot of people believe that I had a personal dislike for the man. I didn't. I met him on only one occasion -- a story which I've never told, but which I will tell here shortly -- and we got along reasonably well, to the shock of many witnesses. To be sure we had deep and profound disagreements, but beyond those disagreements I knew (and to this day know) very little about him. I have no reason to believe that he was anything other than an honorable, hard-working man, but beyond that he was always a cipher to me. I mourn his passing because now that can never change.

My relationship with Erik is unique in that, with only a few exceptions, every interaction we ever had took place on a Usenet newsgroup called comp.lang.lisp. As a result, our relationship is chronicled in unusual detail, thanks to the eternal memory of the World Wide Web in general, and Google Groups in particular. Unfortunately, when I went to consult the record to prepare this piece I found to my dismay that Google Groups seems to be coming down with Alzheimer's. I would occasionally go there for a trip down memory lane, so I know it was working fine up until a few months ago. But last week, when I learned of Erik's death, I went to search for some of his old postings and found that Google's index of several Usenet groups has apparently been corrupted. The original raw data is all still there as far as I can tell, but the index that lets you search it has big gaping holes in it. As a result, I had to do the research for this piece manually, which is why it has taken me so long to finish this, and why I may well be leaving out some pertinent details. If you think I've missed something important, you're probably right.

To set the stage for people who may be reading this who aren't already familiar with the background, I have to tell you a little bit about comp.lang.lisp, or CLL as it is known amongst its adherents. CLL is a Usenet newsgroup, one of the earliest forms of on-line communities on what would eventually become the World Wide Web. Usenet itself goes back to 1979, and CLL to 1986.

My first post (as far as I can tell) was in February of 1990. Time Berners-Lee was still ten months away from writing the world's first web browser. The general public did not yet have access to the Internet, was not generally even aware of its existence, so everyone there was a computer geek. On-line communities (they weren't thought of as such back in those days) tended to organize around technical topics of mutual interest. CLL focused (and does to this day, at least ostensibly) on a programming language called Lisp, whose fans (and I was definitely one of them) tend to be unusually passionate about it. Lisp inspires love and devotion among programming language geeks the way that Apple's products do among gadget hounds. I won't get into why that is here because that would take me far afield. Just take my word for it.

Because of this common interest, and because Lisp tends to be somewhat of a black sheep among programming languages, CLL fostered a kind of camaraderie that went beyond its nominal scope, to the point that in March of 2003 when I felt the need to vent about the upcoming and IMO ill-advised invasion of Iraq, CLL is where I went. (Reasonable people can differ about whether or not this was appropriate. The point is, this is an illustration of how *I* felt about CLL. And I was apparently not alone because my post spawned a very long thread. But I digress.)

Erik Naggum joined CLL (again, as far as I can tell with search not working properly) in August 1994. (N.B. I'm actually pretty sure this is *not* Erik's first post because I have a vague recollection of looking for it several years ago when search was working properly and coming up with a different posting that was quite a bit earlier than this. But searching manually I couldn't find it.)

BTW, by drawing attention to the fact that Erik joined CLL several years after I did I in no way mean to imply that I had some kind of seniority privilege or anything like that. I am only pointing this out because I want to tell the whole story, and part of the story is that I've been observing Erik for a long, long time.

The thing that I remember first catching my attention was Erik's habit of not capitalizing the first words of his sentences. The second thing that I noticed about him was that he had a very no-nonsense take-no-prisoners style. And the third thing that I noticed was that he seemed very smart. But that was pretty much it. It was five years (again AFAICT) before Erik and I interacted in any way. My first participation in a thread involving Erik in any way was in February of 1999 and my first direct response to him was in March. If you take the time to follow those links you will see that there is nothing whatsoever remarkable about them, which is pretty remarkable in light of how things ultimately turned out.

Things didn't start to get tense between Erik and me until late August of 1999. This is not to say that things weren't tense. I was not the first to clash with Erik (but as fate would have it I would be the last). Erik was already getting into regular tussles like this one.

At this point I have to digress for a moment and say a word about my background. I am not doing this to be self-aggrandizing, but because the conflict between me and Erik ultimately hinged on differences of opinion, and I'm going to make some comments on those differences of opinion, and I want to provide a basis for you the reader to assess how much stock to put in my opinion. For example, one of the things that Erik wrote in the post I cite above was:

"what? a customer of a product has _invested_ in the company?! this is too fucking nuts to bother discussing."

The opinion I wish to express is that the position that Erik is dismissing is defensible. Not necessarily *correct*, but not so wrong that it can reasonably be dismissed as "too fucking nuts to bother discussing."

So let me tell you why I feel qualified to make such an assessment. One of the things I did in my career is to convince the powers-that-be at NASA to fly Lisp on a spacecraft, which was not an easy thing to do. The culture at NASA is extremely conservative when it comes to software technology, and Lisp was viewed with extreme suspicion. But I did it, and along the way I learned a lot about how managers think, and one of the things that managers think is that they *do* make an investment in a company when they buy that company's product, particularly a software product. Right or wrong, that's what they think.

I'm walking on eggshells a bit here because I don't want to start an argument with a man who is no long here here to hold up his end of it. I only want to point this out as an example of Erik's habit of taking very strong positions on matters that ultimately hinged on subjective issues, like what the word "investment" really means.

What bothered me about Erik's behavior was not that he was wrong. In fact, it was exactly the opposite. The problem was that he was usually right, and often people would try to argue with him when he was right, and often they would find themselves on the receiving end of some very harsh rhetoric. I saw two problems with this. First, it made it very hard to tell when Erik was right and when he was wrong, because taking issue with him could be so emotionally draining. And second, CLL was turning into not a very nice place to hang out.

It's important to keep in mind that I was not the only person who felt this way. A number of people had similar opinions. A few of them expressed their views openly if a bit inartfully.

And then there was this where Erik wrote (among other things):

"one of the things you will experience is that the community is very different from the Windows communities, whence it appears you come. for instance, the Common Lisp market is not marketing-driven, it is not a pyramid game that requires ever new people nor a bug-and-upgrade scam, and it is not leveraging its operational costs across a huge volume of sales. rather, it is a pretty mature market of long-term partnerships with a steady growth..."

Now, this really bothered me, not only because he was taking it upon himself to speak for the community (and I felt like I was part of the community and I never agreed to have him speak on my behalf) but also because I thought he was wrong. The Lisp market was *not* experiencing steady growth. I believed it was in fact in slow, steady decline (a belief which, sadly, has been at least partially vindicated by subsequent history). I also thought that Erik's very visible acerbic nature might be contributing to that decline even if only in a small way. And finally, I just found the man's arrogance to be incredibly annoying.

In August of 1999 I decided to try to do something about it. Having observed a lot of interactions between Erik and people who disagreed with him I knew that simply confronting him with my views would not work, so I chose a different tactic: I decided to try to engage him and model my behavior as closely as I could on his, expecting that this would lead to a confrontation. I would then reveal (privately of course) what I had done, and hopefully he would see that his style, having been on the receiving end of it, was counterproductive.

The exchange took place in this thread. My first engagement was in this message and it escalated from there, as I expected it would, at one point fulfilling Godwin's law.

I don't want to rehash the controversy over that exchange (there was a lot of it) but there are a couple of things that I think are important from a historical perspective. First, there were only four postings from me out of 183 total in the thread. I pushed only as hard as I thought I needed to to make the point. Nonetheless, by the end of our exchange, Erik was threatening to sue me, so I decided that things had gone far enough, and took the conversation to email.

I don't have those emails any more, and I wouldn't reveal their contents even if I did out of respect to Erik. (I did find this message from Erik in which he references that exchange just to show that I'm not making this up.) Suffice it to say, it turned out that I had misjudged the situation quite badly. My mental model of Erik had been very wrong, and my plan had, to put it mildly, not worked.

It was in this context that Erik and I had our first and only face-to-face meeting. It took place at the 1999 Lisp User Group Meeting (LUGM99) in San Francisco. This was the conference at which Erik delivered his now legendary "Long Painful History of Time" paper, as far as I can tell the only peer-reviewed work he ever published. To make a long story short, I managed to convince him to talk to me, and we had dinner together on the first evening of the conference. No one else was present.

When I was outlining this essay I had planned to write a paragraph at this point about my physical impression of Erik, which was nothing like what I had been expecting. But I've changed my mind for a couple of reasons. First, it's not relevant. Second, Erik was a very private person. He never published details of his personal life, nor as far as I know ever put a photograph of himself on the web. So I will respect his obvious desire for privacy and say simply that in person he was soft-spoken and very polite.

We reached an understanding at that dinner. I don't remember the details of what the understanding was -- this was ten years ago and it's not like we signed a peace treaty or anything like that. But by the time dinner was over, Erik understood -- or at least acted like he understood -- what I had done and why I had done it. The only evidence I have for this was that we got along for the rest of the conference, and this was witnessed by all the other attendees. I could have sworn someone wrote up an account of this on CLL because many people remarked on it at the time, but I can't find it. (Maybe it was in an email.)

(Ironically, at that same conference I also approached Kenny Tilton, with whom I had also been having some friction on CLL. He refused to speak to me, and to this day I have never had a real conversation with Kenny.)

I thought that would be the end of it. Things were quiet for a while. Erik and I didn't interact again until December, in a very unremarkable exchange, and then not again until February.

It all started to unravel in late February of 2000, when I posted this. Detente was not breached immediately. It was a slow, steady unwinding, and it's all there if you want to read it. I'll just note this where Erik wrote:

"I also vote that somebody write "the complete idiot's guide to special variables" instead of proposing silly language changes."

The result of that suggestion was this often-cited paper which owes its existence and its title to Erik.

It took a very long time for things to really deteriorate. Things were tense but still professional for a long time. As far as I can make out, the real turning point came in May, which ultimately led to this.

Shortly thereafter I quit JPL to go work for Google. I was absent from CLL for over a year, during which time Xah Lee stepped up to the plate. If ever there were two people who deserved each other, it was Erik and Xah. Watching those two go at each other was a truly tragic comedy.

I returned to JPL and CLL in August of 2001. My first post after my long hiatus was this one, which again is remarkable only for how unremarkable it is.

Things apparently hadn't changed much in my absence. I watched with some dismay asJohn Foderaro was driven from CLL. I thought this in particular was a little over the top. The understanding we had reached in 1999 seemed to be long forgotten.

I myself didn't enter the fray again until late August. This thread is notable because for about half a dozen messages I responded to Erik by cutting-and-pasting exact quotes from earlier messages of his. I was hoping this "replay attack" (a term of art from computer security) would remind him of our conversation at LUGM99. No such luck.

In retrospect I find it fascinating how the level of tension on CLL seemed to suddenly build in the days before the World Trade Center was attacked. Maybe there was something in the air. (For the record, I didn't participate in that thread.)

Erik and I never managed to rebuild the fragile truce we had forged in San Francisco. Our last exchange was this one after which, as far as I could tell, he not only dropped out of CLL but stopped posting in English altogether. (He posted a couple of things about me in Norwegian but I don't speak Norwegian so I have no idea what he said. Frankly, I think I'm just as happy not knowing.)

So that's my Erik Naggum story. I've told it as accurately as I can. There might be mistakes. I did the best I could under the circumstances. And I think Erik did too.

So farewell, Erik Naggum. Notwithstanding our differences, I actually learned a lot from you, and not least among those lessons is that life is short. So I will always remember our meeting in October of 1999 with great fondness. Everything else is lost in the noise.

First Iowa, now Ireland

Vying to replace Iowa in the the top spot on the list of last-places-in-the-world-I'd-expect-to-see-gay-rights-recognized Ireland has granted limited rights to same-sex couples.

In other news, temperatures in hell continue their plunge to unseasonal lows.

Best. Ad. Ever.

Who would have thought that potato chips could be so compelling?

Microsoft does something cool. Frost sightings reported in hell.

I used to say that Microsoft never produced a technological innovation in its history, preferring instead to make money by stealing other people's ideas and squashing the competition with unfair business practices. But I have to admit that this looks pretty frickin' cool.

Barbarism in Iran

Not that they were ever great even before the "election" (there hasn't been real democracy in Iran since 1953 when the U.S. helped depose Mohammed Mosaddeq and install the Shah), but the situation in Iran is apparently getting really bad. Warning: there's a very graphic image on that link.

A summary for the squeamish:

An Iranian blogger (whose URL I will not publish) live blogging from Baharestan Square in central Tehran today captures but brief glimpses of the unimaginable horror that took place today. Bus loads of protesters were stopped and unloaded from their buses by "black-clad police" and literally herded. When the massing was sufficient, as the barely controllably distraught Tehran caller to CNN described first hand, hundreds of the regime's Basij thugs poured out of an adjoining mosque and commenced a massacre with axes, clubs, guns and gas.

Obama, Iran, torture, and hypocrisy

Someone asked me the other day if I was "pissed" at Obama. No, I'm not, at least not yet. But I am very disappointed.

The opposite of prohibition is not permission

Ron's First Law states, in a self-deprecating and hopefully somewhat humorous strange loop, that all extreme positions are wrong. But that is only a useful aphorism if you actually know what the extreme positions are. Unfortunately, many people mistake moderate positions for extreme one simply because the extreme positions are so extreme (and so obviously wrong) that they don't even enter the conversation.

Take gay marriage for example. This is framed as an argument between prohibiting gay marriage and allowing gay marriage, with the moderate compromise being civil unions or some such thing. But this misses a very important point: permission is already a moderate position. The opposite extreme of prohibition is not permission, it is requirement. The counterpart to prohibiting gay marriage is not allowing gay marriage, it is prohibiting straight marriage. Those are the two "extreme positions" that Ron's First Law warns against.

The same applies to abortion: the opposite of prohibiting abortion is not allowing abortions but requiring them.

There's another important point that people often miss: permission is not endorsement. Just because the law permits something does not meant that it's right or desirable, just that we've decided to place the responsibility of making the decision with the individual rather than the government. I personally think that it's a terrible thing to, say, raise a child as a creationist. But if there were ever a law proposed to forbid the teaching of creationism (in private -- public schools are different) I would be first in line to oppose it.

So you can be vehemently opposed to abortion and gay marriage and still oppose laws that prohibit them on the grounds of moderation, restraint, and (dare I say it?) even conservatism. Because once you set the precedent that it is acceptable to pass laws that support extreme positions, the only thing protecting you from the opposite extreme is your status as a member of the majority. And majorities are often fleeting, as the Mullah's in Iran are lately learning to their dismay.

More on placebos

They really do work.

A response to quantamos

Quantamos took up my challenge and posted a response to my earlier post about gay marriage. It was long enough and well-written enough that it raised sufficiently legitimte points that it deserved a considered response, hence this separate posting.


fine. i'll comment. we'll see what you can do to assuage my queasy twaddle. good luck!


Thanks.


historically, fecundity was a matter of national security, since it was important to get lots of children to fill your battle lines, factories, voting booths, and to provide social security for the elderly.


It was actually even more important than that. Until not too long ago (a few tens of thousands of years) fecundity was vital not just to the survival of nations (because there weren't any) but to the survival of the entire species. That is why we have instincts hard-wired into our brains that drive humans -- even gay ones (I'll get to that in a moment) -- to reproduce. Our not-so-distant ancestors who lacked these drives didn't leave many offspring. Halting reproduction is not an idea that reproduces well.

But, to understate the case considerably, times change. The human species has in a very short span of time (a few thousand years) gone from being just another contender in the game of life to being its undisputed champion. The survival of our species is no longer under serious threat from any quarter except ourselves. To the contrary, we have been so successful that we are the predominant threat to the survival of nearly every other species on earth. And indeed, increasing our numbers beyond the carrying capacity of the planet is a very real possibility (some think we're already there).

So reproduction need not be attended to with the same urgency as it once was. Unfortunately, we are still left with the instincts, intuitions and social norms that evolved back in the day.

But even leaving that aside, your next assertion:

gay marriage takes two families out of the pool.

is simply not true. First, gay people can and do reproduce. Gay women can do so through artificial insemination, and gay men can do so through surrogacy, both of which are widely accepted means of heterosexual reproduction in cases of infertility. But this misses a much more important point: producing a baby is only the first (and arguably the easiest) step in a much longer and more difficult process. Human babies require years of care before they are able to survive on their own. That's the reason marriage is supposed to be a long-term commitment. It has nothing to do with making babies; it has everything to do with raising them. And gays are just as capable (in my experience often more so) of raising children as straight people. So the truth is the exact opposite of what you say: the shortage of resources is not babies; we've got babies aplenty. What there is a shortage of is couples who are ready, willing and able to raise those babies to adulthood. So it is not gay marriage that "takes families out of the pool", is is the prohibition on gay marriage that has this effect!

By the way, if you really believe that reproduction is such an urgent societal imperative that it should override people's right to marry whom they love, then it follows that having children ought to be required of all married couples. But this is obviously a political non-starter, which proves that virtually no one really takes this argument seriously.


i understand that japanese, europeans, and israelis worry about their demographics all the time, and in some sense, they're a forecast for our future. our country currently has the benefit of imported labor, but how much of the financial burden for our elderly will they be willing to bear? the pyramid scheme we call Social Security is already scheduled to blow up. in your previous post you said "Both are slowly but surely dying, and good riddance." (emphasis mine). surely you do not intend to propose that as a solution to the health care questions i'm raising. as a very wealthy man in a very wealthy country who has no concerns about your future, have you considered that you're committing "selection bias?"


Absolutely. But this is a separate issue. As I've already pointed out, when the topic at hand is gay marriage, reproduction is a red herring. I've actually been meaning to write about this. Maybe this will motivate me to finally do it. (It's a very complicated topic.)


here's what I'm queasy about. nobody's figured out a (sustainable) way to draw more money out of a pool than you put into it, no matter how big or complicated that pool is.


That's not true either. Economics is not a zero-sum game. If I have something that you want and I don't, and you have something that I want and you don't, and we make an exchange, then we have magically created value out of thin air. Likewise, technology can create more value than the sum of the parts that went into it.


this is why many gov'ts provide incentives to encourage the production of children of their own, because migrant workers are maybe not going to be willing to foot the bill. i'm not sure that it's fair to allow gay people to get those incentives, or to gain full access to social security if they've been unwilling to contribute as much as others through the hard work of raising children. it makes me suspect that gay marriage is fundamentally a selfish act. it's double jeopardy; they won't have children to bear some of the direct responsibilities so their care will cost more, and they won't have children paying into the system as a part of their indirect responsibilities.


Here again, if you really want to make this argument, you have to take into account the fact that many gay couples raise children and many straight couples (my wife and me included) don't. In any case, the remedy for this is not to make gay marriage illegal, but to change the rules about who is eligible for social security (and I think you'll find that's a political non-starter as well).


so will gay marriage raise my taxes to pay for elderly gays who draw more from the system than they put in?


No more than straight marriage raises your taxes for elderly straights who draw more from the system than they put in. Being gay has nothing to do with it.


the other historical purpose for marriage was to provide protection for women; financial, legal, and physical. our individualistic society has, it seems, obviated the need for this to some extent. but is it gone and will it stay away? i'm not sure, but i don't know enough to comment more on this.


I think that's wise ;-)


i'm worried that this whole issue is merely foolish populist rage at a time when scientists are prohibited from objectively studying and producing facts about the long term effects of this on society. so we do agree that there is "no demonstrable harm in allowing gays to marry" because the one thing activist groups accomplish for sure is to eliminate objectivity.


Now you're sounding like a conspiracy theorist. You acknowledge that there is no evidence that gay marriage harms society, but attribute this to unknown dark forces that prevent scientists from studying the matter and uncovering the diabolical truth. If there were a diabolical truth to be discovered there is nothing preventing the Mormon church from taking the hundreds of millions of dollars it puts into political campaigns and instead funding scientific studies to uncover it.


thank God we have bigots who stand in the way when crowd mentality takes over! i WANT the law to move at a glacial pace.


Are you familiar with Martin Luther King's famous letter from the Birmingham jail? I'll quote the relevant part for you:

We know through painful experience that freedom is never voluntarily given by the oppressor; it must be demanded by the oppressed. Frankly I have never yet engaged in a direct action movement that was "well timed," according to the timetable of those who have not suffered unduly from the disease of segregation. For years now I have heard the word "Wait!" It rings in the ear of every Negro with a piercing familiarity. This "wait" has almost always meant "never." We must come to see with the distinguished jurist of yesterday that "justice too long delayed is justice denied."

finally, i don't understand why people think it's so important for the gov't to play a regulatory role in their interpersonal relationships. what's the point in demanding the right to give up your rights? let gay couples sign contracts to accomplish the same thing! maybe you're not worried about what i just wrote, and you don't believe the gov't has a stake in the institution of marriage. if you believe that the historical motivations for marriage are obsolete, then shouldn't we pull the gov't out of it all together?


I'd personally be all for abolishing government recognition of marriage altogether and just having civil unions for everyone, but that's yet another political non-starter.


thanks for giving me the opportunity to solidify some of my thinking on this issue.


My pleasure. Thanks for having the courage to stand up for your beliefs.

Google Usenet archive data lossage

According to Google Groups today, there were only 227 articles posted to comp.lang.c between 1981 and 1995.

The effect seems to extend broadly across the comp.lang.* hierarchy. I haven't investigated beyond that. I hope this is a temporary glitch and not permanent data loss.

It's a little scary how much the world has come to rely on Google for historical data archiving.

Am I a bigot?

In my earlier post about the recent California Supreme Court ruling on Proposition 8 I wrote:


The overarching social trend in the United States is clear: the bigotry that drives people to deny gays the right to marry the people they love resides mostly in the older generation. Both are slowly but surely dying, and good riddance.


For this I was accused of bigotry. I freely confess to and apologize for a very poorly constructed sentence that left the impression that I was happy to see all old people die because some old people are bigots, or because statistically bigots tend to be old. I rejoice only at the prospect of the death of bigotry, not individual bigots. But that wasn't enough to satisfy my accuser, who responded:


You are tarring the "older generation" with the brush of bigotry. I hope you can see the irony.


So just to be clear: it was absolutely not my intent to "tar the 'older generation' with the brush of bigotry". It is simply a fact that prejudice against gays is more prevalent among older people than younger ones, and that the margin is substantial enough that it is a virtual certainty on simple actuarial grounds that shameful laws like Proposition 8 will eventually get repealed. That is all I intended to say.

There is, however, one form of bigotry to which I do subscribe: I am bigoted against bigots. I am intolerant of intolerance. I am prejudiced against prejudice. And in this I am an extremist. You may be entitled to your opinions, but you are not entitled to have those opinions respected merely because you hold them. Moreover, if you tacitly endorse bigotry through inaction or feigned injury (like, say, pretending that *you* are somehow hurt by the alleged damage done to the "institution" of marriage if gay people are allowed to marry each other, or by someone "insulting" your religion, or whatever) then you are in my book part of the problem. If you don't stand up for other people's rights then your forfeit your own. It makes mighty good rhetoric, but the sad fact of the matter is that we are not in fact endowed by our Creator with certain inalienable rights. We have rights because we as a society agree -- sometimes only at the end of painful and violent struggle -- to grant each other those rights for our net mutual benefit. Part of the cost of freedom is letting others be free.

If you want to be free to worship your god, if you want to be free to marry who you want, if you want to be free to speak your mind, but you are unwilling to grant those same freedoms to others, then yes, I am stubbornly and completely intolerant of you. If that makes me a bigot then I will proudly don the scarlet B.

Change? What change?

Barack Obama, taking another page from Dick Cheney's playbook, has blocked an MSNBC request to see the White House visitor logs. It seems transparency is just another piece of empty rhetoric.

UPDATE: Glen Greenwald has a much more comprehensive analysis of the Obama administration's record on transparency. It's not pretty.

Why health care reform matters

A friend of mine recently sent the following message to a mailing list I subscribe to:


We have a family friend who was diagnosed with 2nd stage breast cancer yesterday.

Several months ago she lost her job that provided health insurance. She continued her health insurance after losing her job, and set up automatic payment on a credit card. But then the card expired, and she didn't check the bills carefully and the insurance company dropped her.

Because she had a very expensive and difficult third pregnancy, no insurance company wanted her and the one that was legally required to provide the insurance was pretty happy to be rid of her. She spent a few months trying to get insurance, but then gave up.

Now she has cancer....

I'm curious - what happens to people like her.


The answer is, sadly: they go broke, at least in the U.S. Then they qualify for medicaid.

Or they die.

Or, in many cases, both.

My friend goes on to explore various options for this unfortunate woman, but the sad fact of the matter is that she's screwed, a victim of, depending on your political leanings, the private insurance system, or her own poor choices. Once you get a life-threatening illness while you are uninsured under our current system, that's it. Game over. You can never get insurance again, at least not for that illness. And it makes sense that you should not be able to, otherwise there would be an incentive for people to wait until they got sick to get insured. It would be like waiting to get auto insurance until after you had an accident. The entire system would collapse.

[UPDATE: it turns out that the situation isn't quite as bad as I thought. I learned, to my astonishment, that you can sometimes get coverage for pre-existing conditions after a waiting period of, typically, 12-18 months. But that's a long time to let a stage II breast cancer go untreated.]

So here we have the entire health care debate in a horrible nutshell. The conservative position is that people should be free to make their own choices, and must assume the responsibility that goes along with that freedom, and bear the consequences if they make bad choices or fail to pay their bills on time. The liberal position is that the consequences of getting those choices wrong are so severe both on an individual and a societal level that government intervention is necessary.

The U.S. is gearing up to have this debate (again!) right now. If you live here, you should pay attention, and urge everyone you know to pay attention too, because it really matters.

Worst president ever: Bush... or Reagan?

Robert Parry makes a pretty convincing case that the Gipper ought to at least be a contender for the crown.

China breaks off diplomatic relations with North Korea (maybe)

According to Bloomberg News, China has suspended diplomatic relations with North Korea.

I don't normally post bare links to regular news items here, but this story is remarkable in that I can't find a single other news outlet that has picked it up (and neither, apparently, can Google). Normally in a situation like this I would just discount it as an Internet hoax, but Bloomberg is not exactly the Drudge Report. If it's actually true, this is a huge development in international politics. It completely changes the political landscape surrounding North Korea and their nuclear ambitions. I'm no journalist, but it seems to me that this is more important than, say, Susan Boyle taking second place on Britain's Got Talent.

So I'm not sure what is more interesting here, the story itself, or the fact that only Bloomberg seems to be reporting it.

Proposition 8 stands. And that's a good thing.

Minutes ago the California Supreme Court upheld Proposition 8, the voter-approved measure that bans gay marriage. Anyone who reads my blog regularly knows that I stand foursquare for allowing gays to marry, so this may come as something of a shock, but I think that in the long run this will prove to be a good thing for gay rights.

The overarching social trend in the United States is clear: the bigotry that drives people to deny gays the right to marry the people they love resides mostly in the older generation. Both are slowly but surely dying, and good riddance. It is therefore a matter of when, not if, Proposition 8 is ultimately reversed by the people and this shameful episode is relegated to the history books. But if the court had overturned Proposition 8 that would have energized the forces of bigotry and caused them to redouble their efforts. The fight would have been long and drawn out, probably over decades, just as it has been with abortion rights.

This way, it is those who support tolerance and freedom who are energized, as they should have been but weren't last year. Early polls showing Proposition 8 failing by 10 points caused supporters of gay rights to become complacent that the earlier California Supreme Court decision legalizing gay marriage would simply stand. But this was naive. Court decisions enforcing radical social changes rarely stand the test of time. At the end of the day (or the decade as the case may be) it is the people who decide.

So what will happen now is that there will be another proposition on the ballot in 2010 to overturn Proposition 8, and it will pass. It will pass by an overwhelmingly greater margin than did prop 8. It will pass because this time supporters of gay rights will not be complacent. It will pass because by November of 2010 the tide of history will be clearly turning against bigotry. While California denied its citizens equal rights, three other states have granted them and two others are poised to do so in the coming months. Furthermore, Californians will have been living for two years with 18,000 married gay couples in their midst and they will see that the sky has not fallen. It will be impossible in 2010 to paint gay marriage as the bogeyman because the bogeyman lives only in the abstract. But gay marriage is here, and the unraveling of the social fabric that was supposed to accompany it is not. In 2010 it will be impossible to condemn gay marriage without appearing manifestly delusional.

Proposition 8 will be overturned in 2010 because the forces of righteousness will be on the march as they were not in 2008 and the bigots will have nowhere left to turn. So hang in there, my gay brothers and sisters, justice is coming. And it is coming a lot sooner and with greater finality than it would have if the court decision had gone the other way.

It's the helium, stupid

Cold fusion is suddenly in the news again because of a recent 60 minutes story. For anyone who thinks that there is anything to this, I have only one word:

Helium.

Excess energy is a red herring. There might be excess energy coming from somewhere, but if you're fusing hydrogen, then you produce helium. No helium, no fusion. It's as simple as that. And I'm not talking about the trace amounts that have been reported, there should be gobs and gobs of it, certainly enough to float a toy balloon and remove all doubt. There's also the problem that all known hydrogen-based fusion reactions produce energetic neutrons. Really, cold fusion researchers should be dropping dead left and right from radiation poisoning, but failing that the radiation should at least be detectable on a geiger counter. So where are the videos of the cold fusion apparatus with a geiger counter buzzing away next to it?

If there's no fusion, there could still be excess energy, but it's almost certainly coming from some prosaic source, and it's almost certainly consuming palladium somehow. Palladium might be an economically viable catalyst, but it makes a mighty expensive fuel.

UPDATE: Here's the math.

The specific heat of deuterium fusion is about 1 MeV, which is about 10^-13 joules. To produce enough helium to fill a balloon (let's say a liter at standard temperature and pressure, or about 0.05 mole) we'd need to produce ~5x10^7 joules of excess heat. That's actually quite a lot, enough to raise the temperature of a cubic meter of water by 50 degrees C. Not enough to vaporize the lab, but definitely enough to get someone's attention.

But here's the thing: some of these experiments run for months at a time. Measuring small amounts of excess heat over long periods of time is very hard. But collecting and storing even small amounts of putative excess helium is very easy. Let's say that the excess heat is only one watt, it would only take a year to accumulate a liter of helium, and the fact helium was being produced would be inarguable after only a few days. So I still say if it were there someone would have exhibited it by now.

UPDATE 2:

I made a mistake in my calculations. There are actually two different deuterium fusion reactions. One produces 3He (helium 3, with two protons and one neutron) and an extra neutron, i.e. radiation. The other produces 4He, with two protons and two neutrons, and no extra neutrons. The second "clean" reaction also produces 23 times as much energy as the first one (or, equivalently, only about 4% as much helium per unit energy produced). It is also very rare. As far as I have been able to determine (which at this point means looking at a few Wikipedia articles) it does not occur in nature, and has not been achieved in more traditional "hot fusion" experiments. So I wrote off the possibility of achieving this reaction at room temperature as too good to be true. However, there are some plausible-looking reports of 4He production correlated with excess heat production in about the right amounts. So at this point I have to say that the possibility cannot be definitively ruled out. I still wouldn't bet my life savings on it, but if a policy maker were to ask me at this point I think I would recommend funding some of this research. These experiments are dirt-cheap by contemporary standards, and if it can be made to work it would be the single greatest technological breakthrough in the history of mankind. So it's worth putting some resources into it to determine definitively what, if anything, is going on.

The trouble with shadow photons

Chapter 7 of David Deutsch's "The Fabric of Reality" should be considered required reading for anyone who wants to understand the scientific process. Unfortunately, it is embedded in a book that is full of some crazy ideas, at least one of which is wrong even by Deutsch's own standards. This one mistake, which Deutsch makes very early on, infects the rest of his reasoning like a virus. This is very unfortunate because I think this is at least in part to blame for the fact that the book as a whole has not gotten the attention it deserves.

It makes a very interesting intellectual exercise to try to uncover Deutsch's mistake on your own. You don't need to be a physicist to find it. Here are two hints to get you started: the mistake is in chapter 2, and everything you need to know in order to figure it out is in this paper, which is written to be accessibly to a lay audience.

So at the risk of sounding like a broken record, get ye over to Amazon and buy a copy of Deutsch's book. Read chapter 7. Then read chapter 2 and try to solve this puzzle for yourself before you read any further. I promise you it will be time well spent. This post is not just about pointing a finger at David Deutsch and saying "ha! ha!" I want to make a much deeper point about how challenging it can be to figure out what is "true". It will help if you have the visceral experience of reading Deutsch's argument and trying to figure out for yourself what is wrong with it even if (perhaps especially if) you don't succeed.

Last warning. Spoiler alert.

Deutsch's mistake is in this paragraph, which in my copy is on page 43:

"Could it be that the photon splits into fragments which, after passing through the slits, change course and recombine? We can rule out that possibility too. If, again, we fire one photon through the apparatus, but use four detectors, one at each slit, then at most one of them ever registers anything. Since in such an experiment we never observe two of the detectors going off at once, we can tell that the entities that they detect are not splitting up."

In my "Quantum Mysteries Disentangled" paper I use the metaphor of a magic trick to describe how popular accounts of QM spread confusion. Most people think that magic is all about gimmicks and skillful sleight-of-hand, but the heart of a good magic trick is a narrative that seems plausible but in fact does not reflect the underlying reality. The elements of this narrative can be quite subtle. I take the Queen of Spades and I turn it face down and I put it, uh, over here. But in fact the card that I'm putting over here is not the queen of spades at all; I've already switched it out for a different card. Much later when the true identity of the card is revealed, you will swear on a stack of Bibles that that card was the queen of spades. You saw it with your own eyes. And you were watching it like a hawk the whole time. Except that by the time you started watching like a hawk the trick was already done. It happened during that little moment of confusion, when I lost my focus and couldn't quite remember what I was supposed to do next. In fact, I didn't lose my focus. Every move was scripted in minute detail. I say "uh" to make it look like I've lost my train of thought. That makes you take your guard down because if I've lost my train of thought then I'm not doing the trick. I look around trying to decide what I'm supposed to do with the card, and your gaze follows mine because deep down in your brain there are instincts that say when a member of your tribe is focusing their attention somewhere you should check it out too because they might be looking at a saber-tooth tiger. In that instant, I absent-mindedly put the card briefly back on the top of the deck before I put it over here. And in that instant I make the switch. Except that I don't even have to make a switch because in fact that card that I put on top of the deck was not the queen of spades either, it was two cards, with the queen of spades on the bottom.

Deutsch's mistake is a sin of omission: he fails to mention the crucial fact that when you do this experiment with the four detectors the interference goes away. This is a devastating fact for Deutsch's theory of shadow photons, but before I explain why there is another important thing to take note of, namely, that at this point in the argument Deutsch has not yet introduced the theory of shadow photons. This is what makes it so easy to miss that Deutsch has omitted an important fact.

The fact that adding detectors destroys interference is fatal for the theory of shadow photons. To understand why, we need to recap exactly what it is that the theory of shadow photons says: something influences the behavior of "real" (what Deutsch calls "tangible") photons to produce interference. Whatever that "something" is behaves just like "real" photons, except that it is "detectable only indirectly through ... interference effects ...". So we might as well call them (shadow) photons. Since shadow photons exist (because they have observable effects), it must be the case that entire shadow universes exist spanning an incomprehensible gamut of possible configurations, including some (actually incomprehensibly many) where David Deutsch does not exist, and therefore -- and this is crucial -- some in which the detectors on the slits do not exist.

This is the odd thing about shadow photons: they behave just like photons that exist in parallel universes except that their movements correspond exactly to the movements of photons in this universe. There are only two possibilities. One is that our universe is somehow privileged, and what we do here produces miraculous effects in the myriad shadow universes. If we decide to cover our slits then the photons in all the shadow universes are blocked, even those in shadow universes where the slits are still open. Imagine how puzzled those poor scientists must be trying to figure out why light behaves as it does. There is no correspondence between how light behaves in their universe and the physical configuration of their universe. Sometimes light passes through blocked slits, and sometimes it fails to pass through open slits. From this behavior they could likewise infer the presence of what is to them a shadow universe (which to us is the real universe) that governs the behavior of photons in their universe, but that just begs the question of why we happen to inhabit the one privileged universe whose macroscopic configuration governs the behavior of light in all of the shadow universes.

The other possibility is to postulate a new rule for shadow photons: only those shadow photons that come from shadow universes whose macroscopic configuration is the same as the "real" universe are allowed to influence "real" photons. But if we postulate that then it immediately follows that shadow universes with different macroscopic configurations have no influence on our universe whatsoever and therefore, by Deutsch's own criteria, do not exist.

One way or another, there is no escaping the fact that our universe is somehow special.

Now, the interesting this about this to me is not so much that Deutsch was wrong per se, but the process by which he came to be wrong. Unlike a magician, I'm pretty sure Deutsch did not set out intentionally to deceive. I'm also pretty sure that Deutsch didn't get it wrong because he was simply being stupid. To the contrary, I think Deutsch is probably brilliant, a lot smarter than me. For one thing, he's actually a physicist, which I'm pretty sure even now still makes an effective filter for idiocy. So what went wrong?

I have no way of knowing for sure, of course, but here's my best guess: I think Deutsch actually managed to fool himself. Deutch's book is, notwithstanding that much of it is actually wrong, a model of clear thinking. To find the flaw in his reasoning is not easy, in no small measure because of subtle details of how he presents his argument. This is why I urged you to actually do the exercise yourself.

The real point of this essay is not to harsh on David Deutsch, but to show that even very smart people, even scientists (sometimes especially scientists) can and do get things wrong, and sorting out the resulting mess is not always easy. This is one of many reasons why both scientists and Scientists ought always to maintain a healthy measure of humility.

UPDATE:

Here's what David Deutsch had to say about this (via email -- I'm pretty sure he hasn't actually read this post):

RG: I know you probably hear this a lot, but I believe I have found a serious flaw in the theory of shadow photons. In a nutshell, the rules of motion for shadow photons are governed by the macroscopic configuration of *our* universe. If a slit is open in *our* universe, both tangible and shadow photons pass through. If a slit is blocked in *our* universe, both tangible and shadow photons are blocked. It seems to me there are only two possibilities: either the macroscopic configuration of *our* universe governs the movement of shadow photons (in which case shadow-scientists must be mightily puzzled why their tangible photons sometimes pass through solid objects, and sometimes fail to pass through open slits), or we have to postulate that only shadow photons from universes whose macroscopic configuration matches our own can interact with our tangible photons.

DD: The latter is, to a good approximation, the case.

RG: But if we postulate this, then shadow photons from universes whose macroscopic configurations do not match ours cannot interact with our universe in any way, and therefore by your own criteria do not exist. Or have I missed something?

DD: You're claiming there's an inadequacy in the criterion for existing that I presented, not a flaw in the 'theory of shadow photons' (which is aka quantum theory). However, that criterion was not intended as a criterion of what *doesn't* exist. If it were used in that way, then we would have to classify all the photons that have left the sun, and passed the Earth, and are never going to strike anything in the future, as being nonexistent.

My response:

1. I find it disingenuous to claim that the theory of shadow photons is "a.k.a. quantum theory." The theory of shadow photons is in fact a.k.a. High Everett's relative state formulation of quantum mechanics, which was later renamed the many-worlds interpretation by Bryce DeWitt. To be sure it is taken seriously by a great many people, but to say without qualification that it is quantum theory is just flat-out wrong. I appreciate Deutsch's intellectual honesty in essentially admitting that my criticism is valid, but I'm puzzled by how he can rail against arbitrary complexity in scientific explanations on the one hand, and then accept as essentially inarguable the proposition that parallel universes are causally connected on a microscopic level by virtue of their macroscopic configurations. It seems plain to me that such an "explanation" of QM gets you exactly nowhere. But I suppose we'll just have to agree to disagree about that.

2. John Cramer's transactional interpretation of quantum mechanics actually does consider photons that never interact with anything to be non-existent, so this is not as outlandish a notion as Deutsch implies.

UPDATE 2:

Deutsch just referred me to this paper which is the more formal formulation of his multiple-worlds theory. I must confess that on a cursory read it seems to be a compelling argument. So I may have to rethink this whole thing.

Just for the record, my (current) disagreement with Deutsch is not huge. It's clear from the structure of QM that the proposition that there is only one (classical) universe is untenable. But that still leaves you with the choice of an infinite number of universes (Deutsch's position) or zero (mine). It has always seemed to me that zero was the more parsimonious choice, but I'm now beginning to have some doubts.

Why I am not a Unicornian

The title of this post is a play on Bertrand Russel's classic essay, "Why I am not a Christian." But the content is modeled after Chapter 7 of David Deutsch's highly under-appreciated book, "The Fabric of Reality". Following Deutsch (who himself followed John Worrall) the essay takes the form of a dialog between me (Ron) and a Unicornian, one who believes in Unicorns.

A disclaimer: this piece is a poor paraphrase of Deutsch's argument. If it weren't protected by copyright, I would just cut-and-paste the entirety of chapter 7 of TFOR here and leave it at that. I asked Deutsch once to put Chapter 7 on the web as a service to humanity but he declined. I cannot hope to reproduce the clarity and completeness of Deutch's argument (which is really Popper's argument), though I'll certainly give it my best shot.

I will lift one passage verbatim from Deutsch as fair-use, which is his statement of the thesis, so at least I won't be able to screw that up:


Science seeks better explanations. A scientific explanation accounts for our observations by postulating something about what reality is like and how it works. We deem an explanation to be better if it leaves fewer loose ends (such as entities whose properties are themselves unexplained), requires fewer and simpler postulates, is more general, meshes more easily with good explanations in other fields and so on. But why should a better explanation be what we always assume it to be in practice, namely the token of a truer theory? Why, for that matter, should a downright bad explanation (one that has none of the above attributes, say) necessarily be false? There is indeed no logically necessary connection between truth and explanatory power. A bad explanation ... may be true. Even the best and truest available theory may make a false prediction in particular cases, and those might be the very cases in which we rely on the theory. No valid form of reasoning can logically rule out such possibilities, or even prove them unlikely [emphasis added]. ... [Nonetheless] I believe that we can justify our expectation that [good scientific theories make accurate predictions].


So here's my humble attempt at channeling Deutsch. But seriously, get a copy of the book and read the original. It's available in paperback.

The setting is, as with Deutsch and Worrall, the top of the Eiffel tower. The characters are Ron and a Unicornian. Ron has just eaten some bad mussels and come down with a very serious case of food poisoning.

---

Ron: I'm not feeling so good. I think I'd better go to the hospital to be checked out.

Unicornian: You know, the line at the elevator is awfully long. Why don't we see if we can find a unicorn instead? Their horns have curative powers you know.

R: If it's all the same to you, I think I'll take my chances with the elevator line.

U: You don't believe in unicorns, do you.

R: That's right, I am an a-unicornist.

U: I feel sorry for you. You are going to risk dying from salmonella just because you stubbornly cling to your faith.

R: I don't really think of a-unicornism as faith. My non-belief in unicorns follows from my belief in science, which is to say, in my belief that experiment and rational argument are the best paths to Truth.

U: But science can't disprove he existence of unicorns.

R: That's right, it can't. Science can't disprove the existence of anything. What science can do is to show, in a philosophically justifiable way, that certain things are extremely unlikely.

U: I don't see how that's possible. After all, any argument you could possibly make has to start with some unprovable and untestable assumptions. Even the belief that experiment is a reliable guide to Truth (with a capital T) is unprovable and untestable. So I don't see how one argument that rests on unprovable and untestable assumptions can be any more philosophically justifiable than any other argument that rests on unprovable and untestable assumptions. You have your faith, I have mine, and there's no way to determine who is right and who is wrong (except that I know that you are wrong).

R: No, that's not true. But explaining why is rather subtle and complicated.

U: The elevator line has hardly moved at all since we started this little chat, so it would seem that as long as you cling to your misguided beliefs about hospitals and modern medicine we have time.

R: OK, let's see how far we get. Have you read David Deutsch's book, "The Fabric of Reality"?

U: Nope, never even heard of it.

R: If you really want to understand this stuff you really ought to read it, especially chapter 7. But I'll do my best to explain it. I'm in a fairly dire situation here. Let us assume for the sake of argument that whatever it is that can save my life, be it unicorns or antibiotics, is only available down there at ground level. After all, if I'm not mistaken, unicorns have many magical properties, but they can't fly.

U: No, they can't fly, but they can ride elevators. There might be one up here. We really ought to look.

R: But then we'd risk losing our place in line.

U: Hm, good point. Maybe I should go looking and you could stay here?

R: If you don't mind I'd rather you stay. If I suddenly collapse I'd just as soon have the company.

U: It's your funeral.

R: Not yet, I hope. But there is another possible way of getting to the ground. We could jump over the side.

U: That seems like a bad plan.

R: Indeed it does. But *why* does it seem like a bad plan?

U: Duh, because you'd fall to your death.

R: And how do you know that? Maybe a unicorn would save me.

U: Dude, unicorns don't work that way.

R: I thought unicorns were magic.

U: They are, but their magic is limited. Their horns have curative powers (which is the reason I think you should seek one out) and they can, of course, become invisible (which is what makes them challenging to locate). But they can't fly, and they don't make particularly good backstops for a hard fall because of their horn and all.

R: And how do you know all that?

U: Because the Unicorn Bible says so.

R: And how do you know that what the Unicorn Bible says is true?

U: Because the Unicorn Bible is the Revealed Word of Uni, the omniscient and infallible unicorn who created the universe.

R: I see. Maybe if I jump over the side he'll break my fall?

U: Alas, no, that is not possible. Because while Uni is omniscient and infallible, he is alas not omnipotent.

R: How inconvenient. Though that does have the beneficial side-effect that you don't have to worry about the problem of theodicy.

U: Indeed not. All evil is the work of men, not unicorns.

R: OK, but even if Uni won't save me, something might. You can't prove that if I jumped over the side that I would fall to my death.

U: Sure I can. There are no exceptions to the law of gravity.

R: How do you know?

U: Because none has ever been observed.

R: But I've never seen a unicorn. You've never seen a unicorn. No one alive has ever seen a unicorn...

U: Duh, because they're invisible. I explained that already.

R: ...and yet you believe in unicorns. So just because something hasn't been observed doesn't prove it doesn't exist. How can we be sure that there isn't an invisible construction elevator just over there that would lower me gently to the ground?

U: Because that's ridiculous.

R: Is it? It seems no more ridiculous to me than the idea of invisible one-horned horses roaming the earth. Why should your intuitions about what is and is not ridiculous be so much more reliable than mine?

U: Because my intuitions are informed by the Revealed Word of Uni and yours are not.

R: That's not true. I've read the Unicorn Bible. I just happen not to believe that it is the Revealed Word of Uni.

U: "The fool sayeth in his heart: there are no unicorns."

R: Are you familiar with the concept of a circular argument?

U: Fair enough. But it seems to me that we have gotten exactly nowhere. It still seems to me that you base your beliefs on a set of unprovable untestable assumptions.

R: No. I base my beliefs on experimental evidence and rational argument.

U: Not so. No one has (as far as I know) ever conducted the experiment of jumping off the Eiffel tower, so you have no basis for believing that you would fall to your death. Sure, you may have seen lots of other things fall under the force of gravity, but to extrapolate from those experiments to this situation requires a leap of faith.

R: No, it doesn't.

U: Why not?

R: Because of the explanatory power of scientific theories.

U: What does that have to do with anything?

R: Scientific theories have value for two reasons. First, they allow you to make predictions about the world which can serve as useful guides for action. (For example, a fairly complex array of scientific theories, ranging from gravity to the biology, is currently guiding me to wait for the elevator so I can go to the hospital rather than leaping over the side.) Second, and more important, they provide explanations of how the universe *really* works (for some value of "really"). The quality of a scientific theory is judged not merely by how well its predictions match experimental observation, but also by the quality of the explanatory mechanism that it provides. For example, the problem with your theory of using unicorn horn to treat food poisoning is not that it is wrong, it is that it provides no explanatory power. It merely stipulates that unicorn horns are an antidote against poison, it doesn't say anything about the mechanism by which these curative powers arise.

U: Sure it does. Unicorns are magic.

R: But magic is not an explanation. Magic -- real magic, not the "fake" magic practiced by professional magicians -- by definition cannot be explained. That's what makes it magic. As soon as you explain it, it ceases to be magic and becomes science. Calling something "magic" offers no more of an explanation than saying that you simply don't know.

U: OK, but you still have to assume with no proof that "explanatory power" is a reliable guide to Truth. So your beliefs are still, at root, based on faith.

R: No, I don't. This is the really cool thing about science. It just turns out that theories with better explanatory power invariably make better, more reliable, more accurate, more precise predictions about the world than theories with poorer explanatory power. Why this should be is indeed a great mystery, but it is an incontrovertible fact.

U: Nothing is incontrovertible.

R: To controvert this you would have to resort to quite the extremes of intellectual dishonesty (some would say you'd have to be delusional or even clinically insane, though I'm not sure I'd go quite that far). The evidence that theories with better explanatory power make better predictions (mind you this is a very different thing from saying that these theories are actually true) pervades every aspect of modern life. Our modern technological society is utterly dependent on being able to make all manner of accurate predictions about what will happen if things like cars and cell phones and bridges are built in certain ways and not others. My life currently depends on my deciding whether or not to jump down to the ground or wait for the elevator. Historically, people who base such decisions on scientific theories that both 1) agree with experimental evidence and 2) have explanatory power have vastly better track records (in terms of the objective material outcomes of their decisions) than people who base their decisions on anything else. And over the centuries people have tried lots and lots of different things, prayed to lots and lots of different gods, listened to countless prophets, tried to gleaned meaning from myriad signs and portents. Compared to science, nothing else even comes close.

U: OK, but this still isn't proof that scientific theories are actually true.

R: I never said that scientific theories are actually true, only that they are reliable guides to action. In fact, one of the fundamental tenets of science is that nearly all scientific theories are almost certain to be false. For example, there are two theories on which all of modern physics rests: general relativity and quantum mechanics. Both of these theories have enormous predictive and explanatory power. Neither theory has ever made a prediction that was not confirmed by experiment. And yet the two theories are fundamentally incompatible with each other so we actually know that at least one of them must be "wrong" in some sense. But (and this is very important) scientific theories, when they are wrong, are almost invariably wrong in a very particular way. The way in which they are wrong is rather tricky to describe. Isaac Asimov did an excellent job of describing how scientific theories are wrong in his essay, The Relativity of Wrong, which happily has been published on the Internet.

U: I'll be sure to read it at the earliest opportunity. But it still seems to me that none of this has anything to do with Truth (with a capital T). It may be that the methods of science could be effective even if they are completely wrong in a metaphysical sense.

R: Yes, that's possible, but it's unlikely. It's much more likely that the reason that scientific theories are as reliable as they are is that there really is a metaphysical reality "out there", and that our scientific theories really are a somewhat accurate description of that reality. Moreover, there is actually a sound scientific reason to believe that this is the case. You can actually treat the metaphysics of science as a scientific theory and test its predictions. For example, one of the predictions made by the theory that science is an accurate reflection of metaphysical reality is that we ourselves are a part of that metaphysical reality. This has consequences in terms of our ability to construct scientific theories. I won't get into the details here except to say that the theory of scientific metaphysical reality makes certain predictions about the structure of scientific theories, all of which turn out (so far) to be true. (For example, it predicts that scientific theories will be constructed by physical entities that emulate Turing machines.) So there really are some very powerful reasons to believe that science is a not-wholly-inaccurate description of metaphysical reality.

U: Holy cow.

R: Yes, it's rather a lot to take in, isn't it?

U: It certainly is.

R: And actually I've barely begun to scratch the surface. Scientific metaphysics is intimately related to neuroscience, computer science and information theory, all of which are fields of study that are only a few decades old. The entire enterprise of science only goes back a few hundred years. Who knows what we'll know a few decades or centuries from now?

U: OK, how do you account for all the Unicornians out there? All the early martys who died for their beliefs? People's personal experience with Uni and his minions? Do you really think we're all idiots?

R: "Idiot" is putting it a bit strongly. I certainly think you're (almost certainly) wrong in a metaphysical sense. But interestingly, there is a very satisfying (at least to me) scientific explanation of why so many people believe in unicorns. Unicorns may not be real, but the belief in unicorns is very real, and can have actual physical effects on the body by virtue of the placebo effect (which is a real, physiological, scientifically verified phenomenon). It is not even out of the question that if I could summon a sufficiently strong belief in unicorns that that belief alone could cure me or at least make me feel better.

U: So why don't you give yourself over to the power of Uni?

R: Because I can't. The placebo effect relies on believing in the efficacy of the placebo, and I know too much to ever be able to fool myself into thinking that unicorns are real in the absence of compelling evidence to the contrary. Belief is very powerful, but its power is predicated on a level of ignorance that I am no longer able to muster. I don't know how to "unlearn" my Scientific worldview.

U: I'll pray for you nonetheless.

R: I appreciate the gesture, and accept it in the spirit in which it is offered. Hm, it looks like the line is starting to move. And you know what? I'm actually feeling a little better. Maybe it was just a touch of gas, not food poisoning after all.

U: Or maybe Uni just worked a miracle in your life.

R: Maybe. We could put it to the test and order another round of mussels.

U: I would think you would be reluctant to put yourself at risk like that.

R: I meant for you, not for me. I've had quite enough of this kind of experiment for one day.

U: But that wouldn't be a valid experiment according to you because even according to you I can summon the power of Uni by virtue of my belief even though you can't.

R: Good point. But the placebo effect will only get you so far. If these mussels really are contaminated with salmonella I think even the strongest belief in Uni will be impotent. You really will need antibiotics. (Christian Scientists occasionally do this kind of experiment on themselves or their kids, usually with unhappy results.) So while it would not be conclusive, I do think it would yield an interesting data point. Shall I place an order?

U: I think the elevator is here. We should go.

R: Yes, I suppose we should.

Hooked on God: Religion as a Drug

Karl Marx famously quipped that religion was the opium of the masses. I submit that this aphorism should be taken more seriously, that considering religion as a drug can be a constructive framework for understanding religion both for believers and, more to the point, non-believers. Non-believers tend to focus on religious claims rather than the effects of believing (or even going through the motions of believing) in those claims, and thus tend to miss the forest for the trees. The result is endless cycles of useless thrashing, wasted emotional energy, and a lot of unnecessary pain.

To begin, let me be precise about exactly what I mean by "religion is a drug." I mean for this view to be taken literally, not metaphorically. Now, obviously I do not mean that religion is a chemical. Religion is a placebo, but that emphatically does not mean that it's "imaginary" or "all in your head." The placebo effect is a real, scientifically observable physical phenomenon with actual physiological effects. The placebo effect can, for example, measurably reduce physical pain, sometimes in cases where all other treatments are ineffective.

It is important to emphasize that the placebo effect is not mystical mumbo-jumbo. It is as well established a scientific fact as you cold hope to find. That double-blind studies are the gold standard of scientific proof in medicine is an explicit acknowledgement of the power of the placebo effect. To demonstrate the effectiveness of a pharmaceutical you have to explicitly and laboriously exclude the possibility that any observed benefit was due to the placebo effect. Doing a properly controlled double-blind study is difficult precisely because the mechanism by which the placebo effect operates is belief, which is very difficult to control with precision.

But religion is not just a placebo; it is in some sense the ultimate placebo, honed and refined over thousands of years to be vastly more potent and powerful than a mere sugar pill. Whereas the curative powers of a sugar pill can be undermined simply by revealing that it is a sugar pill, religions have built-in defenses against such anti-revelations, sometimes layer upon layer of defenses. Whereas sugar pills derive their power from the (presumed) authority of a medical doctor, religions derive theirs, if not from God Himself, at least from His authority. This is one of the most important things that non-believers miss about religion: because placebos derive their power from belief, it doesn't matter whether the thing being believed in is real or not. As long as the belief is genuine, it works. So it doesn't matter if the power of religion actually derives from God or merely from the idea of God; for the purpose of inducing a placebo effect, they are the same thing.

Placebos are particularly effective at producing relief from emotional pain, which is no less real and can be no less intense that actual physical pain. To this I can bear personal witness. I have (like many technically inclined people, I think) suffered from periodic bouts of severe, debilitating depression, even leading on one occasion back in grad school to a half-hearted suicide attempt. Even as a life-long confirmed non-believer, I find the odd prayer occasionally to be palliative. (I often tell people with tongue only half-way in cheek that I'm an unusual atheist: I talk to God. Moreover, He talks back. I tell Him that I don't believe in Him, and He tells me that's OK, that He's here for me when I need Him, but that He can get along without me.)

Within this framework a lot of seemingly intractable problems simply evaporate. Theodicy, for example, becomes a complete non-issue. The existence of evil is only a philosophical problem if one assumes that religious beliefs exist to describe objective reality. On the religion-as-a-drug view, that is not the reason religious beliefs exist at all, they exist to produce a placebo effect. The idea of an all-powerful all-knowing all-just all-loving God prevails not because it's (objectively) true, but because it's therapeutically effective. Moreover, it's a lot more effective if you believe it's actually, objectively true. So people believe it's actually objectively true. And they defend those beliefs against all reason because to do otherwise is painful, in many cases too painful to bear. This is not a sign of intellectual weakness or duplicity. It is simply an aspect of being human. We are both rational and emotional creatures, yin and yang. Militant atheism fails because it ignores this inconvenient truth. (It is no surprise to me that Christopher Hitchens is a chain-smoking alcoholic. Very few people can summon the courage to face the awful realities of life without some form of chemical intervention.)

The religion-as-drug framework helps make sense of many other normally vexing (to atheists) aspects of religious belief. Like any drug, it can be both beneficial (if it alleviates pain) and harmful (if it is abused). It comes in different forms with varying potencies and side-effects. It can be bought and sold. Because it's legal (mostly), there is a thriving economy in religious ideas catering to all segments of a huge and diverse market. It can be addictive, which is good if you make your living as a dealer, not so good if you become a junkie.

And, of course, like other psychoactive drugs, religion can make you crazy, even violent if not taken judiciously.

Finally, the religion-as-drug framework offers some useful insight into the likely outcomes of efforts to rid the world of religion, which are
as likely to succeed as efforts to rid the world of other drugs have been. In particular, religion is very unlikely to yield to either reason, ridicule, or prohibition. It is important to understand that people don't adopt religious beliefs because they are stupid, they do it because self-deception can, via the placebo effect, actually improve quality of life. Until this is understood and accepted, the secular and religious will continue to be at loggerheads.

Kids ruin marriage. Who knew?

Via LiveScience:

"An eight-year study of 218 couples found 90 percent experienced a decrease in marital satisfaction once the first child was born."

Imagine that.

Is it marriage or prostitution?

Scientists have discovered that Chimpanzees exchange meat for sex.

Dispatches from the jury assembly room

I've lived in Los Angeles county for twenty-one years now and I have never served on a jury. I've been summoned for jury duty probably a dozen times, but I've never actually been on a jury. In fact, until about two years ago I never even made it to the courthouse. I was always in the middle of some new startup or other that let me get excused from duty. They've cracked down on that kind of thing now and made it much harder to be excused. On the plus side, they've instituted a new system where if you don't actually get on a jury on your first day then you're done. They also have a new (and well hidden) way of going through the juror orientation on-line, which allows you to show up at the courthouse at 9:30 instead of 7:30. And the juror assembly room has free wireless. So all in all, while it's now nearly impossible to avoid, jury duty is not nearly the onerous chore that it once was.

Which is not to say that it's not annoying. I still have to sit here all day knowing full well that I will never be empaneled on a jury. I know this because of what happened to me the last time I went through this, about two years ago. I sat around all day, and just before quittin' time I and about thirty of my fellow unfortunate prospective jurors had our names called. We all filed out and went to where we'd been instructed (a process that took about half an hour because one of the elevators was broken and the staircases are inaccessible except in emergencies "for security reasons"). There was just enough time to give us yet another pep talk about how wonderful it was to serve on a jury and participate in this unique American institution, and that we had to show up again the next day for the actual start of voir dire.

The next day, after several hours of sitting around waiting for some delay whose nature was never revealed, they started questioning us. That went on for several hours. A few jurors got dismissed by peremptory challenge but I was still there at the end of the day. So on day three I shlep myself to downtown LA yet again, wait around for a few more hours for more mysterious delays, before finally being summoned back into the courtroom. The very first thing that happens then is that the prosecution attorney dismisses me by peremptory challenge. That's it. I'm done. Great, but couldn't you have figured that out yesterday and saved me this trip?

Since I had already set the day aside anyway I decided to stick around and watch our justice system at work. I met a family attending the trial of a man accused of killing a family member. It struck me how different this reality was from the way it's portrayed on TV. The drama and pain are the same, but real life moves at a glacial pace by comparison to television, and even if the killer is convicted, the victim is still dead. And this time it's a real flesh-and-blood person, not a thinly drawn character who evaporates with the closing credits.

During one of the breaks I asked the prosecutor why she'd gotten rid of me, though I was pretty sure I already knew. She confirmed my suspicions: One of the voir dire questions was, "Would you have a problem convicting someone if the only evidence against them was eyewitness testimony?" I had to answer that yes, I would, because I had done a lot of reading about the unreliability of eyewitness testimony.

I'm not sure which I find more annoying, that I was barred from serving for being too well informed, or that I now have to spend a day going through this whole process over again with no reason to believe that the outcome will be any different this time around. I'm no less informed now than I was then. The evidence for the unreliability of eyewitness testimony has, if anything, only grown stronger. I've also been the victim of a violent crime, which I'm told makes me tainted goods for defense attorneys as well. I should be able to get some kind of permanent undesirable-juror designation.

There are two ironies in this situation that really drive me bonkers. First, unlike most of my colleagues in the juror assembly room who view being placed on a jury as drawing the short straw, I actually wouldn't mind serving on a jury. I'm not sure I'd go so far as to say I want to do it, but I'm certainly willing to do it. I do believe in the system, flawed as it is, and I believe in making the time to do my civic duty. I just wish it didn't have to be so damned inefficient. (I am also starting to have my doubts about whether or not, economically speaking, we can continue to afford to indulge ourselves in jury trials. There are an awful lot of people sitting around me being not terribly productive at the moment. And that damn video they just started showing with the sound cranked up to full volume isn't going to help end the recession any sooner either.)

The second irony: I'm pretty sure I'll never serve on a jury because both the prosecution and the defense think I'm prejudiced against them. Trick is, at least one of them must be wrong.

I wonder what would happen if they found out I believe in jury nullification.

UPDATE:

Just happened to stumble across this gem from Philip "Greenspun's tenth law" Greenspun:


"Today we have a legal system with many safeguards for defendants' rights. However, in our heart of hearts, we don't really believe that we could convict enough defendants if we actually gave all of them all of their rights. Consequently, we set nominal penalties for crimes at absurdly high levels, e.g., 'life plus 100 years.' The actual penalty received by 95% of the people who commit such crimes is in fact 12-15 years. This is what they get if they agree to a plea bargain. However, if they choose to exercise their right to trial, they face the nominal penalty of life plus 100.

Obviously having these really high penalties is more subtle than physical torture, but the basic idea is the same and probably a fair number of sensible people are pleading guilty to crimes they didn't commit."


UPDATE 2 (the following day):

Dismissed without ever making it into the jury box. A day and a half down the drain for nothing. There has to be a better way.

Another domino falls

The Burlington Free Press reports:

"Vermont has become the fourth state to legalize gay marriage — and the first to do so with a legislature’s vote."

Four down. Forty-six to go.

Also, a tip o' the hat to Iowa state senator Mike Gronstal for channeling Atticus Finch.

Counterfeiters aren't too sharp (yet)

I would never have imagined that anyone would counterfeit a razor blade, and yet the other day I had the unfortunate experience of attempting to shave with what turned out to be a counterfeit Gillette Fusion razor blade. I think most high-end consumer products are mostly hype, but I have to tell you, if you have four-gauge whiskers like me you don't know what a comfortable shave is until you've tried the Fusion. It's horrifically expensive, but in my hirsute opinion worth every penny. (And no, Gillette is not paying me to say that. But if anyone from Gillette is reading this, I'll entertain offers.)

It was immediately obvious that something was wrong as soon as I took my first swipe. Instead of the normal effortless glide across my morning stubble leaving smooth skin behind it felt like my face was being ripped off. At first I didn't know what to make of it. It didn't dawn on me that there were counterfeit blades out there until I did a Google search and discovered that not only are there fake blades, there's an entire website dedicated to helping people ferret them out. Who knew?

Contrary to the information on that site, it was *very* hard to tell the counterfeit blades. I bought a package of known-good blades a local store so I could do a side-by-side comparison, and I was stunned and how close a match they were. About the only difference was the crappy shave, and the counterfeit's lack of the tell-tale serial number on each blade, which is supposed to be the smoking gun of genuine Fusion blades.

My latest acquisition is a package of Fusion blades ordered directly from Amazon. They also lacked the serial numbers, but they shave like genuine blades. So are they real or just good counterfeits? Does it matter?

I really wonder if 1000 years from now, when the technology to synthetically generate any object, any image or any video scene is available on electronic devices so cheap that they're given away as free premiums with your morning breakfast cereal, if anyone will be able to tell truth from fiction any more.

CSS is awesome

If it's on a coffee mug it must be true.

Announcing µWiki

I've been working on a coding project in order to keep my skills from completely atrophying. It's a little wiki called µWiki (pronounced micro-wiki). I started working on it because I needed a wiki for another project and none of the available ones met my needs so I decided to roll my own. Comments and feedback are welcome.

Some perspective on a destroyed career

Someone over on Hacker news posted a link to a comp.lang.lisp post that I wrote three years ago that's generating a lot of discussion. I found I had more to say about this than would comfortably fit in a comment, so I'm putting it here instead.

The gist of the original piece was that knowing Lisp gave my career an early boost because it was such a productivity multiplier, but later turned into a handicap because it led to frustration with the languages that were becoming the industry norms. The title of the piece was "How knowing Lisp destroyed my programming career" but that's a bit misleading. To understand how it's misleading and why I wrote it that way I have to give you a little bit more background. In particular, you have to understand the context in which that piece was written.

"How knowing Lisp destroyed my programming career" was a followup to an earlier and even more inflammatory piece that called How Common Lisp Sucks. That in turn was a follow up to literally years of off-and-on arguments about whether or not the Common Lisp standard ought to be revised. I thought (and still think) that it should be, but the overwhelming consensus in the Lisp community is that everything is just fine the way it is. (The argument is not unlike the one occasioned by my recent screed on CSS and several followup posts, including this one, which I think sums up both situations quite nicely. (One bit of evidence for this: of all the posts in the series, it is the one that has garnered the fewest comments.)

The important thing to understand about that post is that it was not intended to be therapy or a confession, it was intended to be persuasive. (It failed miserably, but that's neither here nor there.) The idea was to highlight the negative impact that knowing Lisp on the one hand and its failure to penetrate the market more deeply on the other had had on my career, in the hope that that might motivate some people to take market penetration more seriously. (Like I said, it didn't work. There is a hard-core group of Lispers who think Lisp is fine and dandy just the way it is and they honestly don't care whether Lisp is popular or not. These people wield enough influence in the community to effectively halt nearly all progress. But that is not what I want to talk about here.)

Anyway, that was the idea, and that's why I chose the gloom-and-doom title. It's misleading because the fact of the matter is that I never had a career as a programmer. My career up through the year 2000 was as a researcher. My product was academic papers, and if you'll pardon a bit of immodesty, I was pretty darn good at it. In fact, I was so good at it that by the time I was in my mid-thirties I had the equivalent of a tenured faculty position, and I was the most cited computer science researcher in all of NASA. (For all I know I may even still hold that title. I still held it in 2006 despite the fact that I hadn't published anything substantive for over five years.)

I say all this not to toot my horn, but to put my "destroyed career" in perspective. I quit JPL not because I wasn't successful, but because I was bored. I was successful not so much because I was good, but because I had figured out how to game the system. I figured out how to write papers that would get published, and how to write patents that would get accepted. (I successfully prosecuted three patents, including one on an invention that violates the laws of physics.)

Mind you, nothing I did was dishonest. I did not violate any laws or rules. Everything I did was completely aboveboard. What I did was to find the weaknesses in the system and exploit them. And one of the weaknesses that I exploited was an order-of-magnitude productivity increase in writing code. That let me do in a day what would take my colleagues a week, leaving me nine days free to do other things. Like study quantum mechanics.

Whatever problems I may have, an unwillingness to learn new things is not one of them. I love to learn new things. That's one of the reasons I hate Java, because learning Java didn't teach me anything except how truly brain-damaged a language can be. (I've never learned Perl, but I've never learned how to run the deep fryer at a McDonalds either. I like learning new things, but life is short and there are some things I'm content not to know.)

My problem is that I'm lazy. I don't mind working hard, but I mind very much working harder than I have to, especially if I know a better way. This is particularly a problem at an institution like NASA which, despite its reputation for innovation, is actually quite ossified and set in its ways, especially where software is concerned. So I quit and went to work for an obscure little Silicon Valley startup company named Google. And that worked out OK, despite the fact that I felt like I was borderline incompetent. I made some money, but more importantly, I learned more in my one year at Google than in the previous thirty. And when I was done I tried to bring some of those lessons back to where I had come from, back to JPL, and back to the Lisp community. They weren't interested, and while that saddens me, it doesn't paralyze me.

I may not be a particularly good coder (though I think I can hold my own) but I've discovered that I have one very marketable skill, and that is picking winners early. I've had two big wins so far in my life, Lisp and Google. Since Lisp "destroyed" my programming career I've started a new career as an angel investor and entrepreneur. I've been involved in over a dozen startups in the last four years, sometimes as a passive investor, sometimes as a co-founder. And I feel like I'm just getting started. I'm very new at this and I have a lot to learn. But I'm having a blast, so I say bring it on! I used to fret over not being able to program in Lisp; no more. If being in the Silicon Valley taught me anything it is that the world is planted thick with opportunity, and life is too short to worry about the road not taken.

Fixing Spidermonkey's readline()

I had occasion to try out the Spidermonkey Javascript engine today and found an annoying misfeature for which I was able to generate a fix; so I thought I'd put it out here in case other people are as annoyed by this as I was.

Because Spidermonkey is designed to run as an interactive command-line shell and not in a web browser, it includes a readline() method for getting input from the user. It's a very handy feature, but it has a bug: there's no way to tell when you've reached the end of the input stream. I'm apparently not the first person to be annoyed by this. Awesome Andy tried to fix it with a horrible hack which just made me cringe and think that there has to be a better way. There is. That's the wonderful thing about open source. If you don't like the way something works you can change it.

The Right Way to fix Spidermonkey's readline is to simply have it retain the linefeed at the end of lines. That way, empty lines have length 1. When you get to the end of file (and only then) you get a result of length zero. The fix is absolutely trivial. It's a one-line change to the file js.c:

*** js.c.~3.93.2.15.~ 2007-04-20 11:45:18.000000000 -0700
--- js.c 2009-03-29 12:46:36.000000000 -0700
***************
*** 630,635 ****
--- 630,636 ----
  
          /* Are we done? */
          if (buf[buflength - 1] == '\n') {
+    buflength++;
              buf[buflength - 1] = '\0';
              break;
          }

That's it!

Different band, same old song

It's nice to be done with eight years of living under the repressive weight of George Bush and Dick Cheney's disdain for the Constitution, but don't breathe a sigh of relief just yet. Democrats are apparently no more reluctant than Republicans to jettison the Constitution when it suits them. Here's Rahm Emanuel speaking at a Brady Center event:

"If you are on the no-fly list because you are known as a possible terrorist then you cannot buy a handgun in America.... If you're on that no-fly list your access to the right to bear arms is cancelled because you're not part of the American family. You don't deserve that right. There is no right for you if you're on that terrorist list."

At least under the Republicans we could console ourselves by looking forward to the day when we could vote them out of office. But now we have voted them out of office, only to find that it might be a different band, but it's the same tired old song. Your rights end as soon as someone in the government decides you're not entitled to them any more. No due process, no right to confront your accusers (or even know who they are), no appeal.

Now what are we supposed to do?

What are the odds?

And no, these cars are not owned by the same family. "We write" is my wife's car. I have no idea who owns the van. (Besides, it's a Georgia plate. Click for the full-size original.)

A glimpse of the future

The future seems to arrive in the Silicon Valley before the rest of the world, which is why although I make my home in Los Angeles, my heart lives in Palo Alto (and it commutes to Mountain View). So every six months for the past couple of years I make a pilgrimage up north to attend Y Combinator demo day. Lately the event has gotten so popular that they repeat it two days in a row so that all interested people can attend.

This past week I was able to attend both days, once in person, and once by robot, courtesy of Trevor Blackwell's AnyBots and their new QA telepresence robot. It's like a remote-controlled Segway but made with a controller designed by Trevor. With a sleek-looking white body, it looks like something straight out of the Jetsons but without the cheese.

Although there were quite a few rough edges (mainly having to do with connectivity problems) I came away quite impressed. In particular, I was surprised at how usable the system was even with the glitches, and how much being able to move around added to the richness of the experience. It wasn't quite like being there (couldn't help myself to the buffet) but it was much closer than I was expecting.

The most interesting thing about the experience was how much more aware I suddenly became of my sensorimotor limitations than I am when I'm on, say, a phone call, or watching a video feed, or even in a teleconference. I don't know how much of this was due to novelty and how much will remain over time, but on occasion when we lost the video feed I actually felt the same kind of visceral reaction that I would have if someone had put a blindfold over my eyes. When we needed to switch off my ability to control the robot so that it could be manually moved around, I felt like a gimp in a wheelchair. When I was able to move the robot in front of a mirror (not an easy trick -- there was a three-second round-trip communications delay that made precise positioning very tough) I felt in a weird transitive sort of way that I was looking at myself for the first time.



That's "me" on the right, along with the Anybots crew: Daniel Casner, Trevor Blackwell and Benjie Nelson (from left to right).

Some refinements are certainly needed before it's really ready for prime time, but it wouldn't surprise me a bit for these things to be in common use five or ten years from now. Who knows, maybe in 20 or 30 years when we run out of jet fuel we can "travel" through these avatars instead. Maybe by then they will have figured out how to transmit smells and flavors over the internet as well as sights and sounds so it can really be just like being there.

A dog-shaped hole

In December of 1998 my wife Nancy and I moved from a cozy little townhouse into a new 2500 square foot house. No one outside of Stanford had never heard of Google. You could still take large bottles of shampoo onto airplanes in your carry-ons. Al Gore was still Vice President. And I was looking forward (or maybe "resigned" would be a more apt description) to a long career at JPL.

Now that we had some extra space, my wife wanted to get a dog. She didn't really want a dog, she wanted a cat. But I was allergic to cats, so a dog was the next best thing.

I didn't really want any furry creatures in the house besides me. We had had a cat when I was a kid, and once the initial charm wore off, taking care of it became a horrible chore. Emptying the litter box was the worst. I learned the hard way that I get terribly squeamish about the smelly realities of biology. But my wife promised to do all the dirty work and batted her eyelashes at me and I succumbed.

We made a list of the characteristics we wanted in our dog: He had to be a rescue animal. Neither one of us wanted to go to a breeder. He couldn't be too big or too small. We wanted a dog that we could take hiking with us, but not one that we couldn't pick up and carry if we had to. He had to have short hair. He had to be housebroken, and preferably already trained. No barkers, no diggers, no chewers.

We spent a year looking for the perfect dog. We must have visited every pound and rescue organization in Los Angeles. We researched all the breeds. We looked at akitas and malamutes, shepherds and sheepdogs. (Well, actually we didn't look at sheepdogs. Long hair, and all. But I couldn't pass up the alliteration.) We found out that there's a reason that all these animals end up in the pound. If someone has the perfect dog, they tend to keep them.

I finally threw up my hands in despair and agreed to get a cat and deal with taking antihistamines for the rest of my life. So we went to the Pasadena Humane Society and picked out a beautiful orange manx whom we ultimately named Purrcy. While Nancy was in the office filling out the paperwork I happened to notice a lady in the parking lot with a nice looking brown dog. To kill time I decided to go say hello. It turned out the dog wasn't her's, it was a stray that she'd found and was bringing in to the pound.

He was medium-sized.

He had short hair.

He looked at me with sad puppy eyes.

I said, "Hi boy, can you sit?"

He sat.

I said, "Can you shake?"

He held up his paw.

He never made it into the pound. We bought a leash at the humane society's store, put him in the back seat of the car and brought him home. Purrcy arrived a few days later, and after a few skirmishes the two negotiated a peace settlement that lasted for the next nine years. And I, who didn't want any animals at all, suddenly found myself with both a dog and a cat.

Nancy wanted to name him Hobo and I wanted to name him Mojo. We compromised and called him Mobo. In retrospect we should have named him Pogo because he bounced. He bounced when we fed him. He bounced when he walked. His floppy ears would wiggle up and down like Dopey and it would make me happy just to watch them as he trotted along. He was just a bouncy dog, and quite the hiker. We'd take him up into the Angeles National Forest and let him run off leash so he could chase chipmunks and squirrels and the occasional deer. (I don't know what he would have done if he ever managed to catch one.) I lost count of the number of times he vanished (which always freaked Nancy out) only to show up half a mile down the trail, panting furiously, with this huge doggy grin on his face saying, "What the heck took you guys so long to get here?"

But then, as all dogs do, he got old. His legs got weaker. We took him to the vet and learned that one of his hind legs was completely missing the ball part of its ball-and-socket joint. It was amazing that he could even walk, let alone run up and down mountains. But somehow he managed. We started giving him pain pills and that helped for a while. Then he had knee surgery. Then he started making these horrible hacking noises. This time it was a lung tumor, which turned out to be inoperable. That was two years ago.

Today, Mobo finally succumbed. He'd been in a pretty steady declined for a while. His legs were so weak that even standing was becoming a struggle. His breathing was getting raspy and labored. He was on four different kinds of medications. And then he had one horrible no good terrible very bad night and we decided it was time to say goodbye.

I never in a million years dreamed that I could become so attached to an animal. Taking him to the emergency clinic to be euthanized was the hardest thing I've ever done. (I've lived a pretty easy life.) I don't think I've felt so much emotional pain since I was a teenager and my first girlfriend dumped me. His passing has left a dog-shaped hole in my soul. I miss him more than I ever imagined that I could.

No more hikes.



No more sad puppy eyes.



I don't know how anyone who has been through an experience like this can't find at least a little bit of sympathy for people who turn to God for emotional solace. I am a non-believer to the core, and yet the temptation to ease the pain by saying, "You're going to a better place where you can chase squirrels to your heart's content" was overwhelming. And despite the fact that part of me knew it wasn't true, saying it (and I confess I did say it) actually helped. Imagine how much more effective it might have been if I actually believed it. And imagine how much more necessary it might have been if it had been a dying child I was dealing with instead of a dog. Even now I can't even begin to imagine what that must be like.

Mobo, I will miss you. You made a dog person out of me, which I didn't think was possible. Wherever you are now, I hope there are squirrels to chase.

Never turn down a sales lead

I would not have even bothered to write about this -- it's so obvious it sounds patronizing -- had I not met with a startup company the other day (not a YC company) who actually did this. Sometimes even the obvious needs to be stated, so here's a bit of free advice for startup companies. (Actually, this is good advice for any company, but startups in particular.)

Never turn down a sales lead.

When someone offers to put you in touch with a potential customer, always accept, and do it graciously. You don't have to actually follow up on the lead (although you'd be foolish not to -- even if you're absolutely sure that the lead is worthless you never know when you're going to be pleasantly surprised). But under no circumstances should you tell the person offering you the lead that you have no intention of following up even if it's true. Most of the time honesty is the best policy, but not in this case. You are much better off saying, "Thanks for the lead, I'll follow up on it ASAP" and then not doing it than you are explaining all the reasons why you think the lead is useless. In the former case you can always plead schedule overloading or amnesia or something like that (assuming you are ever in a position where you have to explain why you didn't follow up, which is extremely unlikely). But if you start trying to explain why you think the lead is useless there are only two possibilities: either you're wrong, in which you will sound like an idiot, or you are right, in which case you will make the person who offered you the lead feel like an idiot. Either way it's not going to help.

So rehearse this line and add it to your repertoire of little white lies: thanks for the lead. I really appreciate it. I'll follow up on it as soon as I can.

UPDATE: There is one exception to this rule, and that is if the lead is one you have already followed up on, in which you can (and should) say, "We already tried them and it didn't work out because of X and Y and Z."

So much for openness

The Obama administration has taken the position that the details of a treaty on intellectual property need to be kept secret in the interests of national security.

Let's hope that this is the action of some lower-level bureaucrat left over from the Bush administration who didn't get the memo from the President explicitly directing a halt to this sort of bullshit. That's not a particularly good outcome given how much trouble Obama is having keeping things under control. But it's a hell of a lot better than the alternative.

It would be funny if it weren't so sad

What does it say about our society when the best investigative journalism comes from Comedy Central?

The link goes to John Stewart's final showdown with Jim Cramer. It's in three parts. Well worth the time to watch all three.

In praise of SpamSieve

I get a ton of spam, and it was starting to overwhelm the filter built in to the OS X mail client. I've recently started running my own mail server, so I started tweaking the settings to try to cut down on the processed breakfast meat, and discovered that Microsoft Outlook has a serious bug (what a surprise). One of the ways an SMTP server can cut down on spam is to require clients to connect with a fully qualified domain name (FQDN). This will prevent many botnet machines from connecting because they tend to not be configured to send a FQDN. Unfortunately, it works a little too well. As far as I have been able to determine, it is not possible to configure Microsoft Outlook on Windows to send an FQDN. I have clients using this server who use outlook, so I had to disable the FQDN requirement.

I could have installed a content-based spam filter on the server, but the problem is that spam is personal. One man's spam is another man's hot deal of the week from his favorite on-line vendor, and the training process can get really annoying if it's not integrated into the client. I was about to throw up my hands in despair and set up a second mail server for Microsoft users, when I decided instead to try SpamSieve. I was a little skeptical that it would work much better than Apple's built-in filter, but there's a 30-day free trial so I didn't have much to lose.

I've become a believer. I've only been using (which is to say training) it for three days, but the situation is already dramatically better than it was before. I've had one false positive during the training process, and the false negatives have rapidly dwindled to 1-2 a day. My spam problem is (I almost dare not say this for fear of tempting fate) effectively solved.

So I recommend SpamSieve. It's well worth the $30 it costs.

I do have one complaint: the instructions for the initial training process are a little unclear, and in one place downright misleading. The instructions say to delete the messages in your current spam folder before setting up SpamSieve. DON'T DO THIS! The training process requires about 600 representative spam messages. The instructions suggest fishing these out of your trash, but the problem is that if your spam is in the trash it's almost certainly mixed in with a lot of good messages and now you have to manually tease them apart. So if you're going to be using SpamSieve, keep a collection of spam in a separate folder for training before you begin. You'll be glad you did.

Can OS X really suck this badly?

I was playing around with Git because that seems to be what the cool kids are doing nowadays, and stumbled onto what seems to be a rather nasty bug in OS X. Apparently, creating a lot of files with names in a random lexical order is reeeeeaaallllyyyy slllooooowwwww. Here's a demo (in Python):

import hashlib, datetime, os
l1 = [hashlib.md5(str(i)).hexdigest() for i in range(3000)]
l2 = [x for x in l1]
l2.sort()

os.mkdir('test')

def test(l):
  t0 = datetime.datetime.now()
  for x in l: open('test/%s' % x, 'w').write('test')
  os.system('sync')
  os.system('rm test/*')
  os.system('sync')
  return datetime.datetime.now()-t0

test(l1)
test(l2)

On Linux, both tests run in about the same amount of time (about a second on my machine). But on OS X, test(l1) is seven times slower than test(l2). This is enough to cause real pain when trying to deal with a large repository because Git uses the filesystem as sort of a poor man's database.

If anyone happens to know a fix for this, or how to get Apple's attention, I would be most grateful. I've reported this to Apple Feedback and also their discussion forums but I'm not holding my breath.

Fun with numerology and the King James Bible

Not sure what inspired me to do this. Maybe it was the Hand of God. But for some reason I felt moved to see how long it would take to generate a histogram of the words in the King James Bible using a modern computer. The answer turns out to be about ten minutes of coding (in Python) and a second or two of run time.

The results have some interesting features.

The King James Bible contains:

789,634 words

31,103 verses

12,808 distinct words, of which 223 are possessives. Of those 223, all but four are also found in their root form. The four possessives that are not found in their root form are "barber's", "nachon's", "stomach's", and "fisher's". (The last one is, of course, found in the plural -- seven times in fact -- but the singular "fisher" appears nowhere in the KJV.)

4046 hapax legomena, including "onions", "cries", and "moist'.

The most common word is, naturally, "the", which occurs 63,919 times. The most common word that is not a conjunction, preposition, or pronoun is "shall" with 9837 occurrences.

The word "love" occurs 310 times and "hate" 87. "Good" and "evil" are more evenly matched at 720 and 613 respectively (with "holy" adding to the margin at 611). "Lord" appears 7830 times, "God" 4442, and "Jesus" runs a distant third at 983 plus 555 occurrences of "Christ". (Interestingly for Bible-code enthusiasts, no word appears exactly 666 times. The closest are "like" and "way" flanking the Number of the Beast at 669 and 664.) "Mary" rounds out the Catholic pantheon at 54, but not all those references are to the mother of Jesus. At least twelve of the 54 are to Mary Magdalene (I say "at least" because there are some unadorned references to "Mary" that appear to my unschooled eye to be references to MM, e.g. John 12:3), and some are to an assortment of other Mary's, like Mary the sister of Martha and Mary the wife of Cleophas. If God had really intended for us to pay a lot of attention to Mary the mother of Jesus, you wouldn't know it from the histogram.

You can explore the complete list yourself here.

The CSS Holy War

I'm not sure which is more striking, the fact that people are still posting comments on my CSS article, or the degree to which the discussion has come to resemble a religious one. Computer programmers talk about having religious wars about programming languages, but the term is usually understood to be at least partially tongue-in-cheek. I used to think that no one really got religious about these things. Now I'm becoming less sure.

I was really hoping to leave this topic behind but a comment posted today by someone who calls himself Pherdnut just pushed me over the edge. I happen to have a free hour between meetings, so against my better judgement I'm going to take this on yet again.

Pherdnut writes:

This would have been a better argument if:

What you suggest is impossible wasn't possible.

I never suggested anything was impossible. Go look it up. Go ahead, I'll wait.

Did you find the word "impossible"? No, you did not, because it isn't there. What is there is an explicit acknowledgement that all these things are possible:

Of course, all of these things can be fixed. But the point is they have to be fixed!
...
It [creating a three-column layout that doesn't have one of the many problems I cited] may be possible. I don't have a mathematical proof that it's not.

The problem with CSS is not that it makes it impossible to do the things people want to do. The problem with CSS is that it makes it hard. There is nothing inherently flawed about the CSS approach. The problem is in the execution.

But I don't want to talk about CSS. I want to talk about the tenor of the discussion. Consider this comment:

I recommend CSS Mastery, which gets straight to a lot of the stuff that confuses people about CSS...

I have not read CSS Mastery. I have read a lot of other material about CSS though, including several books, a boatload of web pages, and even some of the original source material. Does it matter that I have not read CSS Mastery? Pherdnut thinks so.

It's by Eric Meyer.

Well, actually, no, it isn't. CSS Mastery is by Andy Budd. Eric Meyer wrote a book called "CSS: The Definitive Guide."

I can assure that he knows a lot more CSS than whoever hacked together those tutorials you linked to.

How can I trust your assurances when you can't even get the most basic facts straight?

(UPDATE: I made an inadvertent but crucial edit in the above exchange. Pherdnut's full quote was, "I recommend CSS Mastery, which gets straight to a lot of the stuff that confuses people about CSS and also the CSS pocket reference which very quickly gets to the point when it comes to the specifics of how things are rendered (or supposed to be rendered) by the browsers. It's by Eric Meyer." The CSS pocket reference is indeed by Eric Meyer. This was simply an oversight on my part. I just missed the second reference, and thought the whole passage was talking about one book. I regret the error, but I stand by the substance of this post.)

But the question I want to ask is not whether or not "CSS Mastery" or "CSS: The Definitive Guide" or any of the other 41,803 results from an Amazon book search for "CSS" is a higher quality source of information about CSS than any of the sources I've actually relied on to date. The question I want to ask is: how is someone who doesn't know CSS supposed to know? Let us suppose for the moment that everything I've read about CSS (including the original source material from the W3C) is crap, and only Eric Meyer (or is it Andy Budd?) can show me the One True CSS Way. How fortunate it is that Pherdnut came along to set me right! What would I have done without him? Among the over 1000 comments that this post has generated in various venues, not a single person suggested either book before now. If not for Pherdnut, I might never have seen the light.

I submit that that in an of itself is a problem. But that's not what I want to talk about either. What I want to talk about is the form of the argument, which is something along the lines of, "Your conclusions about CSS are wrong because you do not have the necessary information."

On its face this doesn't seem altogether unreasonable, but in fact it is a very problematic argument. How much information is enough, and how can you tell? Suppose I read Meyer and I still believed that on balance tables ought to be used for layout. What then? Would someone else come along and say, "Oh, don't listen to that Pherdnut character, he's a moron, and so is Meyer. The person you really want to read is Arglebargle. He'll set you straight."

Notice how similar in structure this line of reasoning is to a truly religious argument: God doesn't answer your prayers? That doesn't prove that God doesn't exist, it just shows that you don't have enough faith, or haven't read the proper holy text. Maybe if you believed just a little harder next time...

Such arguments can never be answered, because there's no way to prove that you've attained a level of faith/knowledge that would falsify the hypothesis. No amount of unanswered prayers will ever disprove the existence of God to a true believer. Likewise, no demonstration of difficulty will ever convince the CSS true believer that there's a problem. The cause will always lie in the person experiencing the problem because there will always be some book he hasn't read or some technique or hack he hasn't mastered.

I think the reason that my original CSS rant got so much attention is not necessarily because I'm right but because I introduced some actual facts into the discussion, and nothing is more threatening to the true believer than facts. Notably, that claim is a hypothesis that can be tested by introducing some more facts into the discussion and observing the responses. Let's give it a try, shall we? Here are some more facts:

1. It has been claimed by the CSS true believers that using tables for layout is bad for SEO. You would think that if this were true, Google would say so on their webmaster guidelines, but they don't. In fact, the only reference to CSS and tables that I could find on Google is this entry from the Google blog:

If you are using a complex, multi-column layout for most of the content on your site, you might wish to step back and analyze how you are achieving the desired effect. For example, using deeply-nested HTML tables makes it difficult to link together related pieces of text in a logical manner.

The same effect can often be achieved using CSS and logically organized <div> elements in HTML. As an added bonus, you will find that your site renders much faster as a result.

Note that this is not an admonition against tables, it's an admonition against deeply nested tables, which is an admonition that I thoroughly endorse.

There's another bit of evidence that the tables-destroy-SEO argument is bogus, and that is that if it were true there would be an absolutely trivial fix that Google could implement: simply standardize a class identifier that indicates that a table is being used for layout, e.g.:


<table class=for_layout_only>...


I submit that the reason they don't do this is because there isn't actually a problem to be solved.

2. A second claim made by the CSS true believers is that tables make a site inaccessible. But the W3C's own guidelines on accessibility say nothing about this, nor do the W3C recommendations on the use of tables have any mention of this. To the contrary, the latter page provides a few specific recommendations on how to make tables accessible to non-visual user agents.

Oh, and if that wasn't enough to convince you that the W3C doesn't agree with the CSS fanatics, check out their quick reference to Web Content Accessibility Guidelines. There you will find a whole pile of recommendations on how to make web sites accessible, including specific recommendations for using tables. There's also this:

Using CSS rather than tables for page layout (future link)

You'd think that if tables were so inherently awful the W3C would have given this item more attention.

You can actually test the effect of tables on accessibility yourself. There are publicly accessible screen readers out there. Pick one and run a table through it and you can see for yourself how bad the "problem" really is.

Let the experiment begin.

A new look for Rondam Ramblings

I've finally updated my blog template so I could add a TipJoy button. This is more of an exercise in market research than a serious attempt to make money. This blog is not and was never intended to make money. It's just a place for me to express my opinions so that anyone who cares to know what I think can find out. But today I got a reminder email from TipJoy that I needed to pay my bill (which I did), so I thought I'd try it out. Any money I make will be donated to charity.

I've decided not to add AdSense ads, at least for now, because I'm afraid they will prove too much of a temptation to abandon my editorial independence. Also, Google has been getting kind of evil lately.

CSS: the last word (I hope)

The flood of comments on my CSS versus tables rant has finally died down to a trickle. The final tally (as of 9 AM PST) was 146 on Hacker News, 198 on Rondam Ramblings, and a whopping 716 on Reddit (plus 921 karma points for Nicou). That smashed my previous record by a factor of four! There were also ~250 comments on my followup piece. Yowza! I wish I could say that I read every comment, but I didn't. After a while they started to get a little repetitive, and the ad hominem attacks in particular started to get annoying. But despite the fact that I didn't respond much because of the overwhelming volume, I did read a lot of them. And I wanted to thank everyone who commented, even the ones who attacked me. Notwithstanding accusations to the contrary, I did not write the piece as a publicity stunt. You will notice there are no ads on my blog, nor on my personal web page. [UPDATE: I've added a TipJoy button just to see what happens. Any money I make will be donated to charity.] I wrote it because I thought the CSS versus tables controversy could benefit from the injection of some actual data, and because I wanted to take a stand against what I saw as faulty arguments in favor of CSS. Nonetheless, as a writer it's always good to know that someone out there is paying attention, even if that attention is negative. So thanks to all of you.

I did want to take time to respond to some of the common themes that kept repeating themselves over and over in the comments. I doubt this will stop any of these memes from reproducing in the future, but everyone else is getting their two cents in so I figure I'm entitled.

Theme #1: CSS is hard, and you have no business expressing an opinion until you've paid your dues and spent five years learning all of the ins-and-outs.

I already addressed this point in a previous post but this theme kept showing up even in the comments to that post so apparently I was still being a little too subtle. I utterly reject this view, and not just when it comes to CSS, but in all aspects of life. This idea that only the experts have opinions worth considering is not merely wrong, it's corrosive. It is -- to cite but one example -- what got us into this economic mess. The idea that the crisis "could not have been foreseen" is absurd on its face. Not only could it have been forseen, it was foreseen. It was predicted at least as far back as 1992 and probably earlier than that. You don't have to be a rocket scientist nor an economist to see that if you spend more than you make, sooner or later you will wind up in trouble.

Likewise, you don't have to be a CSS expert to see 1) a lot of broken CSS on the web, 2) a lot of people tying themselves into knots trying to do conceptually simple things. You don't have to be an expert to know that if text is spilling out beyond div boundaries everywhere you look, and people are proving their CSS studliness by doing something as simple as a liquid three-column layout, something is wrong.

Now, tables are not perfect. Much of the criticism leveled at them is valid. But it is unsound reasoning to conclude that because tables have problems you should therefore use CSS. CSS has problems too. That makes the choice between tables and CSS a valid engineering tradeoff. My personal quality metric is that all else being equal the simpler solution wins. The situation could change as the technology landscape shifts, but right now, that calculus comes down on the side of tables. Reasonable people could disagree.

(BTW, if you are a fan of arguments from authority, you should keep in mind that just because someone chooses not to wield their credentials doesn't mean they don't have any.)

Theme #2: CSS does not really have the problems you think it has, you just need to study it more.

I anticipated this argument as well, but again people didn't seem to notice. I knew people would say this, which is why I used someone else's code to make my point. People still didn't get it. They attacked the code anyway and said that those people didn't know what they were talking about. Here's the thing: on that criterion, no one knows what they are talking about. There is not a single CSS-based solution to the "holy grail" problem that does not have some deficiency or other. The mere fact that people even talk about the "holy grail" in connection with CSS ought to tell you something.

I don't have anywhere near the time to deconstruct every putative solution to the three-column layout problem, so I'll pick just a few of my favorite examples. The first is ChristianZ, who I am choosing to pick on because his comments were particularly vituperative. He held out his website as an example of three-column layout in CSS. I must confess, I think his website is looks outstanding. Trick is, it's not done in CSS. Turn off Javascript and suddenly it doesn't look so hot.

The second is Matthew James Taylor's oft-cited solution. Once again I have to doff my hat to what is truly a heroic effort to rescue a hopeless situation. As a thought experiment you might want to ask yourself why it is that Taylor's solution (nor YUI grids, nor anything else) has not emerged as the definitive solution to the problem. If you can't come up with an answer, try this: take Taylor's "holy grail" solution and modify is so that the left and right columns are not always butted right up against the edges of the screen. (BTW, even Taylor's solution suffers from the ubiquitous CSS malady that text will spill out beyond the div boundaries if it's too wide to fit.)

Theme #3: CSS is not meant for dynamic content

This one I did not anticipate. It was not a common theme so it could be that the commenter was simply wrong. But I list it here just an case anyone else has something to say about it.

That's pretty much all I have to say on this. Thanks again to everyone who commented.

On the semantics of HTML

Since we're having so much fun talking about tables and CSS and whatnot I thought I'd see how long I could keep the party rolling. Yes, I'm being ironic. I actually didn't want to get sucked into this. But I made an offhand comment in my last post that needs clarification:

HTML has no semantics beyond how it is rendered!

That resulted in comments like this one:


No. You're very, very wrong about this. The whole field of SEO exists because you are wrong. The title element, depths of heading, everything is used. Just because you can't see it being used, it doesn't mean it isn't.

It's commonly said that Google is the biggest blind user in the world. If your content isn't blind-accessible, it's likely to be less Google-accessible. Accessibility is for everyone.


This is too big a misconception to let slide, so here I go again.

First, notice that I did not say that HTML has no semantics. I said that HTML has no semantics beyond how it is rendered. It turns out that this is not quite correct, but the way in which it is not quite correct is subtle and rather difficult to explain. So I'll start with an example. There are two links there. They both go to documents whose HTML content is identical. The only difference between the two documents is the style sheet. Those of you who use custom style sheets in your browser will probably need to turn them off. Those of you who are blind (do I have any blind readers?) will completely not get the point of the example.

And that is precisely the point of the example.

Back in 1995 I wrote an essay for the Risks forum called the source of semantic content. Back then it was in the context of an attempt by then-Senator Jim Exon of Nebraska to pass legislation limiting the transmission of pornography on the Internet. The key sentence of that essay is as relevant today as it was forteen years ago:

[T]he semantic content of bit streams is in the eye of the beholder, and ... the apparent correspondence between bits and semantics is the result of engineering convention and not an inherent property of the bits.

Let me try to explain what I mean by that by way of a non-computer-related example. What gives words their meaning? One answer is that their meanings are given by their dictionary definitions. (The French take this to a whole nuther level.) This would be analogous to saying that the semantics of HTML are given by the standards published by the W3C. It is not an unreasonable answer. But it is wrong.

The best example I know of to show that it is wrong is so powerful that I almost dare not use it. It is the word "niggard", with an A, no E, and a D at the end. The dictionary definition of this word is, "A stingy, grasping person; a miser." A negative connotation to be sure, but on its face less offensive than some of the epithets that people regularly sling at one another. And yet the actual semantics of the word, which is to say, its actual meaning as measured in terms of the effect it has when it is employed, is radically different from its dictionary definition. (The word "ignorant" has a similar property. So does "liberal", at least in the U.S.)

Neither words nor computer programs derive their true semantics by fiat. They derive their semantics by the effects they have on the world. And the principal effect that HTML has on the world is that it gets rendered in browsers and those renderings are viewed by humans. The W3C and CSS purists can rant and rave all they want, but the fact of the matter is that what the HTML in the above example really means depends on which style sheet you use to render it.

And that is true for any HTML document. It's fairly easy to make a CSS style sheet that will take any HTML document and cause it to render in a completely arbitrary way. (How to do it is left as an exercise for the reader.) This is not just an academic observation; techniques like this are frequently used by spammers to get past filters.

The counter to this is that these people are undermining the true semantics of HTML. They are somehow "cheating" or "ruining the web" or some such thing. I have a certain amount of sympathy for this position. I am no fan of spam. The world would be a better place if everyone followed the rules. But this is just like arguing that the world would also be a better place if everyone agreed to abide by the dictionary definitions of the word "niggard". You can argue this until you are blue in the face. That will not change the fact that if you call a person with black skin a "niggard" you will likely cause more offense than if you called her a "miser". That is reality.

The reality for HTML is that its semantics are determined primarily by how it renders on the browsers that people actually use. And at the moment that includes IE6. The H1 tag doesn't mean "top level heading" because the W3C says it does. It means top-level heading because browsers by default render it in big bold type and as a block rather than inline. And if you believe that I have the causality backwards, that browsers render H1 big and bold because it means top-level heading, imagine an alternate world where the default style for H1 was NOT big bold type, and ask yourself how many people would use <font size=+10> instead. (Actually, you don't have to imagine. Just look at how many people use the "I" tag instead of the "EM" tag, or how many web forms are out there that don't use LABELs. Actually, I use "I" instead of "EM" myself. It's partly out of habit (when I started writing HTML there was no EM tag) and partly because "I" is less typing, uses up less screen real estate, and accomplishes the same thing for my purposes.)

Now, I said at the beginning that all this was not quite true, and the thing that makes it not quite is SEO, which is to say, Google. Google imposes a set of operational semantics on HTML that are substantially different from those imposed by browsers, and it is those differences that lie at the root of many a web designer's sleepless nights, and is responsible for the existence of the SEO industry. But here's the thing: even mighty Google has to yield to the operational semantics of browser rendering. Google puts in enormous effort to try to glean how a page will render, and not just extract its apparent content as defined by the W3C. The reason they do this is simple: if they don't they will be overwhelmed with spam. If Google could generate their index by rendering every page and running OCR on it they would. The only reason they don't is that it's prohibitively expensive.

By the way, if you still doubt that the semantics of HTML are inextricably bound to rendering, consider the P tag and tell me how you define a paragraph without talking about rendering. A paragraph is an inherently visual concept. If you doubt this, do the following experiment: take an audio recording of Barack Obama's inauguration speech and transcribe it. Now compare your transcription with the original text. Count the number of places you put in a paragraph break where there was none in the original text, and the number of places there was a paragraph break in the text that you missed. Now compare that count to the number of places where you missed (or added) the end of a sentence. Now ask yourself: why is there markup for paragraphs but not for sentences?

CSS and the meaning of life

Wow, my little CSS screed seems to have really hit a nerve. In just six hours it has hit #6 on the Reddit front page (up to #5 now). I didn't even submit it to Reddit because it has been ages since I was able to get any traction over there! All those karma points, they could have been mine. Oh well.

I would love to respond to all the comments, but the volume is pretty overwhelming. But this one really stands out as my favorite:

Yes, you are an idiot.

Why mince words? I confess, I am not a CSS expert. I all but admitted as much in my piece, which is why I based my example on code that was not written by me, but by someone who holds themselves out as being a CSS expert. Maybe I'm so idiotic that I can't even tell a real expert from a snake oil salesman. But I submit that whatever the case, I am an very good company. The mere fact that the CSS wars have gone on for as long as they have is ample evidence that there are a lot of people out there who do not find CSS a model of intuitive clarity.

I don't want to re-fight the CSS wars here. Instead I want to focus on some things that surely we can all agree on, to wit -- that the CSS wars have been going on for a long time, that a lot of people have put a lot of effort into fighting them, and that very little progress has been made towards resolving the dispute -- and ask the following question: What conclusions can we draw from these observations?

One possibility, of course, is that one side is right and the other side is wrong, and that the people on the wrong side are simply dim-witted or intransigent, or both. I try (not always successfully) to take seriously the possibility that I could be wrong whenever I write anything. Maybe Dr. Howard R. Fine is right and I really am an idiot. One of the many problems with idiocy is that it does not yield readily to introspection. I take some comfort from the fact that the vast majority of the people accusing me of idiocy don't actually dispute any of the points that I made. But maybe I'm such an idiot that I'm not even worth educating (hard to tell from where I'm sitting). So the best I can do is to leave that on the table as a possibility and move on to the next option, which is, I believe, if it not a more plausible hypothesis, at least more promising as a potentially productive line of inquiry.

I humbly submit for your consideration the possibility that different people want different things out of life. It seems self-evident, but it can sometimes be surprisingly hard to wrap your brain around the fact that someone else doesn't want the same things you do. One person wants long-term maintainability while another just wants to get it done as quickly as possible. One person wants their web site to be accessible to blind people, another doesn't. One wants SEO, another doesn't care.

Underlying my conclusion that tables were the way to go was a tacitly assumed quality metric that not everyone shares. At the time I was writing I was not even consciously aware of some aspects of my own quality metric. For example, accessibility to the blind had not even entered my mind. It's just not something I have ever had to deal with, and about which I know next to nothing.

There are other aspects of my quality metric about which I was aware, but about which I was not explicit. For example, I tacitly assumed that having content spill out beyond the boundaries of its containing DIV is Bad. I think most people would agree on this and other tacit aesthetic judgements that I made, but not all would. A blind person, for example, couldn't care less where on the screen something ended up. (I'm actually guessing about this. I really know very little about how blind people navigate the web. Maybe someone can enlighten me.)

There are likewise tacit aspects of other people's quality metrics that manifest themselves in their comments. For example, a lot of people seem to think that anything that was done in the 1990's is automatically bad. It is certainly true that tables can be, and during the 90's often were, badly abused. But IMO that is no reason to throw out the baby with the bath water. It's overreactions like that that led to throwing out Lisp with the rest of AI during the 80's, a mistake from which computer engineering is still not fully recovered.

And then there is one element of my quality metric which seems to be at the heart of the controversy: I believe that computers are meant to serve people and not the other way around. That means that if something inherently simple is difficult to do, it's wrong.

Not everyone agrees with this. A surprising number of people like things to be difficult. I have never understood this, but it is a fact. Some people revel in the intellectual studliness of mastering, say, C++ template programming or x86 assembler. I don't. I think C++ is a pain in the ass. Life is short, and I would rather spend my allotted time skiing or making love to my wife than worrying about whether or not I need to define a virtual destructor in my base class in order to avoid memory leaks. And that means that when I want, say, for something to appear centered on the screen, I want to be able to do it by typing "center" rather than "margin:auto". If I can't tell a computer to center something by saying "center", then as far as I'm concerned the computer is wrong, not me.

Like I said, not everyone buys in to this worldview. I don't get it, but it is surprisingly common. I think it might have something to do with job security. I first encountered the complicated-is-beautiful mindset when I was working for NASA. I had an idea for how to simplify spacecraft sequencing and make it less error prone. I figured everyone would love it. To the contrary, the establishment fought the idea tooth-and-nail (and still does to this day). And this attitude is everywhere. It's the reason the tax code is so fucked up. It's the reason the law is so byzantine. It's the reason that languages like C++ and Perl thrive.

I like simplicity. I think the computer should have to conform to my brain and not the other way around. That's why I like tables. They are simple, and they work -- for some value of "work".

Different people have different ideas of what it means to "work". For me, the ratio of effort expended to the quality of the visual appearance on the screen is paramount. I want to put forth as little effort as possible to get something that looks decent. (Yes, I'm a lazy bum. It's my greatest strength. Laziness is a powerful motivator for finding more effective ways of doing things, and it has served me quite well.) Everything else is secondary. I don't have any blind customers. I am not in any business where SEO really matters all that much. But your situation, of course, could be different.

That said, I do believe that some of the things said in favor of CSS and against tables are just flat-out bogus, and to point out some of those was the purpose of my essay. CSS is touted as separating layout from content. It doesn't. Maybe with the new table layouts in CSS3 that will change, but one of the external constraints that I have to live with is that what I do has to work in IE6. While the popularity of IE6 is mercifully in decline, the sad fact of the matter is that it still has a significant market share, and I am not in a position to ignore that much as I would like to.

Another common thread is that "tables are for tabular data, not layout." Why? Just because they are called tables? Here's a news flash: HTML has no semantics beyond how it is rendered! (That's not quite true. Links have semantics beyond their renderings. And maybe label tags. But nothing else in HTML does.) This is the reason that the "semantic web" is still a pipe dream. HTML was supposed to have semantics, but it doesn't. Giving real semantics to a markup language is really really hard. The history of computer science is lousy with failed attempts, from KRL to SGML.

Finally, a lot of people pointed to the Zen Garden as evidence of CSS's power. I remember seeing the Zen Garden for the first time and being blown away. But if you look closely you will see that it's not really CSS that makes the Garden beautiful, it's images! Don't get me wrong, the Garden is lovely, occasionally spectacular. But how much of the loveliness is due to CSS and how much is due to photoshop is far from clear. And this is to say nothing of the fact that the Garden completely sidesteps one of the most thorny issues for CSS, namely, dealing with dynamic content. If you know ahead of time what your content is going to be it's not hard to write CSS that will make it look nice. The challenge is writing CSS that looks good if you don't know ahead of time what your content is going to be. The Garden, beautiful as it is, has nothing to say about that.

I want to point out one more thing: I am not bashing CSS. I like CSS. Its goals are the right ones, and in many ways it is well executed. You can do some really cool things with it, often without having to work very hard. But layout -- at least the three-column layout that everyone seems to want -- isn't one of them.