Getting it straight

A recent Educated Guesswork post missattributed one of my favorite sayings.

The correct story, as told to me by Peter Hart: he and Ed Feigenbaum were walking somewhere at Stanford. The great John McCarthy brushes by without acknowledging their greeting. "I wonder what resource he's conserving?" Ed says.

What becomes a programmer most?

What do Richard Stallman, James Gosling, Charles Simonyi and Bill Joy have in common?

Famous hackers all, yes, but oddly, their initial fame came from text editors: (Teco) Emacs, (Unix) Emacs, Bravo and Vi, respectively.

Of course, Cap'n Crunch wrote EasyWriter, but he was famous for other things first.

12% = 3 x 4% and I say to hell with it

Judge Richard Posner is guest blogger over at lessig blog. He's had several posts on the topic "Break Up the CIA?" which I hastened to comment upon, suggesting that he was Screwing-up the Lede. He responded, and I replied at length, which I reproduce here.

Judge Posner,

Your response to my "it's not just about the CIA" comment had two main points: 1) the CIA is the key component of the intelligence establishment and 2) a plan like Senator Robert's to reorganize the 15 agencies would be extremely disruptive.

The evidence appears to indicate that the CIA is important but not valuable. The CIA's failures of analysis are legion and are too numerous to list here. It's not hindsight to point out that well before their most recent failures in Iraq, they failed to correctly analyze the progress of North Korean ballistic missile programs, Indian nuclear weapons programs, Libyan nuclear weapons programs, the Iranian revolution, the Soviet strength in the 80's, the significance of Jihadists prior to '94, etc. The record shows an inability for CIA analysis to be relevant to US strategic interests.

WRT CIA collection, it seems clear that the agency had no significant assets in North Korea or China in the 1950s, in North Vietnam during the 1960s and early 1970s, in the Soviet Union during the entire Cold War, in Iraq in the first Gulf War, or in Jihadist organizations during the 90's.

WRT CIA operations, "covert actions" performed by the agency have too frequently been major failures and public relation disasters (not to mention yielding "blowback"). Supporters of the CIA suggest that their successes are unheralded but this claim is risible: in the credit-hungry and leak-ridden environment inside the beltway, no success, however small, is unheralded for long. Note, for instance, the heavily publicized role for CIA irregular forces in the action against the Taliban, and for missile-wielding Predators in Yemen.

Finally, to your other point, disruption. A preservationist argument can always be made, no matter how bad the current situation. The question is not what disruption will be done by change, but is change required? No doubt you would accept that a drastic reorganization of the intelligence services could be required in some circumstances, despite high transition costs. Are these the circumstances? I suspect they are.

I Want My WebTV (that doesn't suck)

I bought my Mom a WebTV^* many years ago, when it first came out. It fills a need: an appliance to access the web simple enough for everyone. At least, that should be the selling proposition.

I mean, just because she wants to use the web and send me email, Mom should not have to know how to upgrade to the latest version of Windows. Or patch. Or worry about the virus-of-the-week. To make that happen, she needs a box that isn't configurable, isn't expandable, that "just works".

WebTV would be just the ticket, if it didn't suck so much. Here's what it would have to do to not suck:

1. Work with providers other than MSN.
2. Support broadband (i.e., the freaking box needs an ethernet port).
3. Have video output that can drive a HDTV: Component video, VGA, DVI or HDMI.
4. [Extra credit] Support USB (so you can use a modern printer, and maybe CF readers).
5. [Bonus Round] Support more data types (e.g. PDF) without opening Pandora's box.

I know about Cidco's mailstation (which is apparently not available anymore), and I've seen some other boxes that could fill the bill if someone would provision and support them properly. BTW, my Mom would still happily pay the 22 bucks per month — that's what Microsoft is extorting from her now.

[*] Now it's called msnTV, of course, but I refuse to call it that.

Screwing-up the Lede, Again

The national press often gets big stories wrong. And usually even those of us who care about the particular story shrug off their errors. But I'm pretty annoyed this time, so I'm taking this one on — forgive me for making my point in a roundabout way.

Suppose that an influential Senator proposes to reorganize a major portion of the federal government. The status quo is a dozen plus agencies (depending how you count) with a total annual budget of about $40 billion, allocated as given in Figure 1 (the "F+6" bar represents seven agencies I have aggregated together). Some of these agencies report directly to a cabinet-level officer, some not, but the only common authority across all of these agencies is the President. A separate matter, but worthy of note is that Agencies N, S & D are operated by the Department of Defense.

The Senator proposes consolidating the agencies into five, with a single cabinet officer heading four of them. All of the personnel from the old agencies N, P & D would no longer be DoD employees, but would work for civilian agencies.

The consolidation plan is complex, but some high points are:

• New agency W is old N plus F and bits of S, C & D
• New agency X is old P plus bits of old F, C & D
• New agency Y is the bulk of old C & D
• New agency Z is old S plus bits of C & D
• T is unchanged

OK, I hope you're still with me. What would you say if all the major papers ran this story as Senator Proposes to Break Up Agency C.

What's that? No reporter, you say, would fixate on the relatively minor issue of the allocation of 13% of the total budget and manpower represented by Agency C. Well, that's exactly what all papers are leading with. In case you haven't guessed: Agency C = CIA; Agency F = FBI; Agency N = NSA; Agency S = (Satellites) NRO; Agency P = (Photos) NGA; Agency D = DIA without tactical groups; and Agency T = tactical military intelligence groups. A typical headline and first graph (from the NYT today):

An Angry Republican Roils Intelligence Waters

By DOUGLAS JEHL

Published: August 24, 2004

WASHINGTON, Aug. 23 - The very idea of dismantling the Central Intelligence Agency, Senator Pat Roberts concedes, is one that he could not have conceived of proposing even a year ago.

Idiots.

Notes: My source for much of the (estimated) budgetary data comes from GlobalSecurity.org. My take on how the new agencies would be organized is based on the very sketchy data gleaned from TV interviews with Senator Roberts. Also, to note my bias: I think the Senator's idea is a good one.

Channeling my inner teenager

As previously reported, last week I went to my Mars High all-class-years reunion. The reunion was originally to be held in St. Petersburg, Florida, on August 14th — the place and time, coincidently, someone had invited Hurricane Charley. We, along with the rest of the population of the Tampa bay region, were evacuated to Orlando. That's where Charley decided to go too. But enough about the weather. Let's talk cheerleaders.

Your Correspondent hadn't ever been to an reunion before, although he'd known for some time that ISB alumni organization held them periodically. I don't know anyone there, he said. I won't have anyone to talk to. I hated high school anyway, he remembers. He goes this time because people he did remember from school sent email, asking him to come. But because of the weather, for the first few days he's without them.

Vintage Jailbait One's high school reunion may be the only situation where a middle-age man can make salacious comments in public regarding pictures of 16 year-old girls without being arrested. Reunion attendees have a badge hanging around their necks with name, graduation year, and their yearbook picture. He approaches a nice lady the apparent age of his wife of 25+ years. He bends over and squints at the badge with photo. Suddenly, hunched-over, he realizes: this woman was once the blond big-breasted shiksa senior he fantasized over in his sophomore year. He drools a bit, he straightens-up. She doesn't mind! In fact, she appreciates the attention! He enjoys looking at these pictures very, very, very much. It's probably because when he's looking at them...just for a moment...the teenager inside stirs and looks at the picture too.

Bring Out Your Dead Some of the classmates who aren't there get discussed a lot. There's Steve, he was shot ("murdered") while robbing a liquor store — 20 years after high school, though. He was on PCP or something. Then the suicide — they tell the story about how Trey died. Somebody has new details on the story, they will work that into the narrative. Then a woman he doesn't know tells him a long story about how she saw Trey that day. If she had done this or that, she could have saved him. No you couldn't, he thinks, but I could have. Show of hands, everybody, who else thinks they killed Trey?

Whatever Happened To so-and-so, is a popular question, naturally. He has done some research on the matter. Three of the girls I hung out with, Barbara, Berry, and Eddie, are professors, he says. That's gotta be a record. And HKP, you have no idea how well he's doing, he practically runs Singapore. Charlie, he's a big time screenwriter, wrote K-Pax and The Mighty. Nobody responds in kind — they shame him for being so status-conscious. They tell him instead: what's-his-name became a missionary; so-and-so gave up his practice to treat AIDS in the Thai hill country, wasn't that just like him?

Instant Intimacy Which people you were buddies with isn't that relevant here. You can talk to anyone and get an immediate confession of some secret. I didn't menstruate until I was in college, one woman tells him. I've brought some Cialis with me, says some guy. He tells some of his secrets, too.

Cheerleader Coverage He keeps looking at these women, and the photos of young girls around their necks. He makes a point to talk to every one of the former cheerleaders. Of course, we remember you, Allan, they say. We watched you watching us, sitting on the bench when we practiced. We thought you were cute. And smart. Oh you were so pretty, he sighs. Then stutters: not that you're not pretty now, of course! And they discuss for a minute what they've both done in the intervening 34 years. Or ask what happened to her brother. You know, he says, we've just spoken more words to each other than we did in all of high school. Oh, they say, we were too shy to talk to you Allan, why would you want to talk to us? Sure, he says, that's your story.

Reunion Junkie His buddies finally show, complete with spouses, he follows them around like a puppy. He loses his voice; from singing Satisfaction at 3am, or from his logorrhea? He gets into two class photos, Zellig-like, thinks of trying for the other classes too. All those years, 34 years. When is the next reunion, he wonders, hope there's going to be a hurricane then too.

May you live in (cryptographically) interesting times

Eric Rescorla has a post on the consequences of rumored and actual breaks in commonly used cryptographic hash functions. Since this stuff is being reported as of August 16th, I guess it's what you call breaking news, yes?

Anyway, EKR's post says that since the breaks are in collision-resistance, rather than preimage-resistance, it doesn't have a major impact on security protocols. I don't think so.

Hash functions with weak collision-resistance would be a very bad thing for non-repudiation in signature applications.

This has been described before in motivating the so-called birthday attack^* in digital signature applications. That is, you as the originator of a signed message could generate two messages: the first a commitment that you'd like the recipient to rely upon and the second that you'd later claim you'd actually said instead which commits to you to something less than the first.

The birthday attack assumes that collision are hard and that you need to pre-compute a large collection of, for instance, good/bad contract pairs. If collisions are easy, you have a more realistic prospect of coming-up with a plausible pair of good/evil offers.

[*] The commonly-cited countermeasure for the birthday attack is for the counter-party to non-materially modify the offered message before signing. This ignores the fact that there are applications requiring non-repudiation that aren't two-party contracts.

Rich Parker, where are you?

OK, so we were evacuated from St. Petersburg and put up in a nice hotel in Orlando.

Now they tell us that Charlie is going to miss Tampa entirely and, in a surprise move, is heading straight for Orlando!

I must say, however, for a Category 4 Hurricane, nothing much is happening so far. Wait...uh, isn't that a bad thing?

Reporting Live from Hurricane Charlie

I flew into Tampa this evening. I did check the weather online before I left my house in Tahoe at 5:45 AM. When I arrived at the Tampa airport, I learned that much of Tampa Bay Region was under evacuation orders. The largest evacuation, they're saying on TV, in the area's history.

I had a lot of luck, though. The organizers of the reunion managed to get a block of rooms in Orlando (well inland) and bussed everyone who was already in St. Petersburg to the new hotel. I rented a car at the airport and now after a four hour drive (usually 90 minutes, I'm told), I can blog from the comfort of a very nice room: the Rosen Centre Hotel. They're charging me a "distressed traveler's rate" of about $48, for a room that I estimate would usually cost $200.

And there are a lot of ISB'ers milling about in the bar downstairs. So, assuming I can get home when the weekend is over, I can say this was the most pleasant emergency I've ever been in.

Hurricane watch

Leaving on a jet plane tomorrow for Florida. Why go to Florida in August, you ask, especially when there are two, count-em, two hurricanes going through.

Hard to explain, you think, but there is my high school reunion happening in St.Petersburg. Not every day do you get to see your former classmates from Mars High.

Buggy Style Sheets

OK, I just noticed I had some funky Read this! link at the end of every one of my posts. This arose from adding a feature to my blogger template to permit for abridging posts.

I think I've fixed it.

You don't tug on Superman's cape...

My long-time colleague, L. Peter Deutsch has taken on a project that might change things for Python users. LPD is, of course, the man who made Smalltalk go too fast^1,2, the reason Adobe invented Postscript Level 2 (to keep him busy), the man who wrote the Lisp 1.5 interpreter for the PDP-1 while still in short pants. LPD is going to make Python go fast. Don't bet on Psyco.

Here's a long message from Peter regarding his plans, posted here by permission (click the permalink at the end of this post to see the whole thing):

pycore is a project to create a new implementation, also called pycore, of the Python language and libraries. It has the following goals, roughly in descending order of importance:

1. Radically improve the performance of many Python programs.


2. Reimplement as many C-coded Python libraries as possible in Python while retaining acceptable performance.


3. Be able to run any Python program (some possibly slower than CPython) that does not:


1. Depend on libraries implemented in C that haven't been recoded in Python;


2. Use some of the more arcane customization facilities;


3. Depend on being able to manipulate 'int' and 'long' as separate types, rather than having the implementation choose how integers are stored;


4. Subclass any of the built-in types (bool, int, long, tuple, list, str, unicode, and possibly others).

pycore works by translating compiled Python bytecode to the bytecode of VisualWorks, the Cincom Smalltalk implementation. The VisualWorks JIT compiler is a mature, high-performance engine that is undergoing constant improvement, specifically optimized for a non-type-declared object-oriented language with inheritance: it is a good match for (the normal usage patterns of) Python.

pycore actually includes three different execution mechanisms:

• A Python bytecode interpreter;


• A Python-to-VW bytecode translator that represents all objects as dictionaries, and does explicit dictionary lookups for every attribute access (both data and method);


• A Python-to-VW translator that represents (most) data attributes as Smalltalk instance variables and (most) methods as Smalltalk methods.

The interpreter is currently complete, except for 'exec'; the simple translator is substantially behind; and the optimized translator is only at the design stage. Nevertheless, some Python programs run faster even with the pycore simple translator than with CPython, for example:

• Recursive fibonacci function, 9x faster


• Iterating over a large list of integers, 5x faster


• Creating a list element-by-element, 2x faster


• Accessing an attribute by calling a method, 2.5x faster

On the other hand, replicating a collection: (1000000 * 'x'), is 7x slower.

So there are many challenges ahead.

We know of 5 other current projects with somewhat similar goals.

1. Psyco is a fine-grained JIT compiler with dynamic customization. It should do much better than the VW JIT on numeric and string/array inner loops; however, its performance on method invocation is poor. In contrast, the VW JIT has very efficient invocation.


2. PyPy aims to recode the Python interpreter and libraries in Python, and then use unspecified compiler technology to create a fully compiled system. pycore should be able to leverage the recoded libraries.


3. Jython is a Java implementation of Python. While it compiles Python to Java, it discards most of Python's unique abilities in doing so (e.g., the ability to add attributes to any object, the ability to change the bindings of methods at run time, all the customization hooks, etc.) pycore does not need to discard any of these abilities: in principle, we believe we could support *all* of Python's extensive customization facilities without losing any performance in the usual cases.


4. IronPython is a compiled implementation of Python on top of Microsoft's Common Language Runtime (CLR). Its author recently joined the Microsoft CLR group. It is in an early stage of development.


5. Pirate is a Python compiler that targets the Parrot dynamic-language virtual machine. It is in a very early state of development. The pycore interpreter should be able to run all the test code on the Pirate Web site, and the simple compiler isn't very far behind.

There are surely others we don't know about.

pycore is currently a one-person project. Depending on what happens with the other projects listed above (especially PyPy), it may never get any bigger than this. Indeed, there's no commitment that the present person will ever deliver anything, although if he gets tired of it, he'll make sure that it gets out into the world with an Open Source license so anyone else interested can pick it up.

And when the hackers all get together at night

You know they all call Peter boss.

[1] Of course, Smalltalk was too fast already, even before Peter made it faster.

[2] I helped. See the POPL Paper.

OK, now they're making me mad!

You may have heard that there's trouble at Los Alamos National Laboratory. They've lost a few disk drives, among other things. Maybe they had bomb blueprints on 'em, I dunno.

In a unrelated development, oil prices have passed $45 per barrel.

I thought you ought to know:

1. The Bush Administration has proposed a replacement for the University of California as the manager of Los Alamos. The company they propose is Haliburton.
2. With oil prices at an all-time-high, the administration has resumed buying oil for the Strategic Petroleum Reserve.
   

As the Bolsheviks used to say: the worse, the better.

Physicist Discovers Bogon

Shahriar S. Afshar claims to have performed an experiment which contradicts Bohr's Principle of Complementarity, that is: light behaves as a either a wave or a particle depending on what sort of experimental apparatus you have — it is impossible to do an experiment which observes wave and particle properties of light simultaneously. I was taught that Complementarity was the bedrock of Quantum Mechanics, which is, as we all know, "the most well tested and successful theory in the history of physics".

Except it is apparently quite possible to perform such an experiment, and Afshar has done it.

Great edifices of philosophy have been built based on Complementarity and Bohr's Copenhagen Interpretation. My father must have been very impressed by Bohr's arguments, because he described these ideas to me when I was very young^*, I suppose to impress upon me how mind-expanding science could be.

My favorite suggestion on how to adapt to the new result is to agree that there's no such thing as a Photon.

There was a brief discussion of the experiment and what it may mean on the July 30th episode of NPR's Science Friday. The July 24th issue of New Scientist magazine had a cover story on the topic, but registration is required to read it.

[*] "Light is both made up of things that are like rocks and is like the ripples a rock makes when you throw it in the lake — how could light be like both of those?", he'd say.

Reduction to a previously-unsolved problem

Ed Felton blogging from the Meltdown Conference:

The security session descended into a series of rants about the evil of spam. Lately this seems to happen often in conference panels about security. This strikes me as odd, since spam is far from the worst security problem we face online. Don’t get me wrong; spam annoys me, just like everybody else. But I don’t think we’ll make much progress on the spam problem until we get a handle on more fundamental problems, such as how to protect ordinary machines from hijacking, and how to produce higher-quality commercial software.

Tell it, brother.

Happy Second Anniversary!

Recondite is two months old. Gosh, it seems more like sixty days!

High School on Mars

I'm going to an "all class years" reunion of my high school, weekend after next. I've never been to a high school reunion before (although I've seen the movie) — I'm a little apprehensive.¹

My high school was kind of unusual and this explains why the class sizes are too small to have a normal reunion of a single graduating class: I graduated in 1970 from the International School of Bangkok ("ISB"). When I tell people that, I typically get one of two reactions:

1. What were you doing in Bangkok? I tell the questioner I went originally on a sex tour but lost my passport.
2. No kidding, I went there too! Believe it or not, I've met several other people who went to ISB. One guy is a friend of mine in Tahoe. Another ended-up working for me. Nice to know: no matter how wierd you are, there's always company for you.²

I was there because my parents were, of course, and my parents were there because my Dad worked there. My Dad worked there because he was a civil engineer and there was a lot of work in Thailand during the Vietnam War — designing bases for the US military. We had a lot of fun, we scions of the colonizers of Asia. Essentially immune from Thai law, we were unusually liberated 12 year-olds: driving motorcycles, buying dope from street vendors and hanging-out in bars.³

Anyway, if you went to ISB too, it's not too late to register for the reunion. See you there!

[1] I'm worried the seniors are going to beat me up again behind the teen club.

[2] Would this be a good time to mention that I was the first person to be Bar Mitzvah'ed in Thailand?

[3] I haven't really researched the matter, but my understanding is that the Status of Forces agreement between the Thais and the US Military made arresting an American dependent too problematic for the Thais to attempt.