« March 2006 | Main | May 2006 »

Saturday, April 29, 2006

Les Quatre Saisons: One Year Time-Lapse Movie Posted

After a year of photography and almost a month of post-production, I have posted Les Quatre Saisons, a time-lapse movie created by taking a photograph every day (with a few gaps due to this and that) from an east-facing window of the Fourmilab conference room which, at the start of the project in March 2005, looked upon a hay-mow that has been agricultural land since the Roman Empire, but went and sprouted three new houses in the last twelvemonth. Each season of the year is set to an excerpt from Vivaldi's The Four Seasons, custom arranged from a MIDI sequence for this film. A tremendous amount of detail is captured in these images as the seasons change, the weather flaunts its fickleness, plants flower, flourish, and finally fade, and the relentless process of urbanisation continues. If you spot something interesting in the movie and want to look closer, you can visit Les Quatre Saisons: Jour par Jour, a Web document tree which lets you explore all of the individual frames of the movie as high-quality JPEG images with information about the circumstances (illumination and weather) on the day the frame was photographed. The main document contains a complete description of the production of the movie, a link to download the custom Perl programs used to produce it, and links to all of the software tools used in the process. No proprietary software was used in the creation of this movie.

The full movie is 37 Mb, so if you don't have a fast Internet connection you should give it a miss; if you're curious, you can still explore the single-frame edition. Because of the potential a file of this size has for eating my own Internet connection alive should somebody post a link to it on a public discussion board or, even worse, use it in a distributed denial of service attack, this time I'm trying something different with the hosting. While the links above and the single frame edition are served from my Web server as usual, I have placed the movie in the public domain and contributed it to the collection of the Internet Archive where it is hosted on this page. When you click the download link for the movie you're getting it from there, not here. The process of uploading the film was reasonably painless, although be very careful if your metadata contains ISO-8859 characters: it appears that if you use the browser's “Back” button, they can get accidentally transformed into UTF-8, mangling every accented letter. This may, of course, be specific to my configuration or just my customary luck with such things.

Posted at 02:06 Permalink

Wednesday, April 26, 2006

Reading List: Second Stage Lensmen

Smith, Edward E. Second Stage Lensmen. Baltimore: Old Earth Books, [1941–1942, 1953] 1998. ISBN 1-882968-13-1.
This is the fifth installment of the Lensman series, following Triplanetary, First Lensman, Galactic Patrol, and Gray Lensman. Second Stage Lensmen ran in serial form in Astounding Science Fiction from November 1941 through February 1942. This book is a facsimile of the illustrated 1953 Fantasy Press edition, which was revised from the original magazine serial.

The only thing I found disappointing when rereading this book in my fourth lifetime expedition through the Lensman saga is knowing there's only one volume of the main story remaining—but what a yarn that is. In Second Stage Lensmen, Doc Smith more overtly adopts the voice of “historian of civilisation” and from time to time departs from straight story-telling to describe off-stage action, discuss his “source material”, and grouse about Galactic Patrol secrecy depriving him of important documents. Still, there's enough rays and shields space opera action for three or four normal novels, although the focus increasingly shifts from super-weapons and shoot-em-ups to mental combat, indirection, and espionage.

It's here we first meet Nadreck, one of the most fascinating of Doc Smith's creations: a poison-breathing cryogenic being who extends into the fourth dimension and considers cowardice and sloth among his greatest virtues. His mind, however, like Kinnison's, honed to second stage Lensman capability by Mentor of Arisia, is both powerful and subtle, and Nadreck a master of boring within without the villains even suspecting his presence. He gets the job done, despite never being satisfied with his “pitifully imperfect” performance. I've known programmers like that.

Some mystery and thriller writers complain of how difficult the invention of mobile phones has made their craft. While it used to be easy for characters to be out of touch and operating with incomplete and conflicting information, now the reader immediately asks, “Why didn't she just pick up the phone and ask?” But in the Lensman universe, both the good guys and (to a lesser extent) the blackguards have instantaneous, mind-to-mind high bandwidth communication on an intergalactic scale, and such is Doc Smith's mastery of his craft that it neither reduces the suspense nor strains the plot, and he makes it look almost effortless.

Writing in an age where realistic women of any kind were rare in science fiction, Smith was known for his strong female characters—on p. 151 he observes, “Indeed, it has been argued that sexual equality is the most important criterion of that which we know as Civilization”—no postmodern multi-culti crapola here! Some critics carped that his women characters were so strong and resourceful they were just male heroes without the square jaws and broad shoulders. So here, probably in part just to show he can do it, we have Illona of Lonabar, a five-sigma airhead bimbo (albeit with black hair, not blonde), and the mind-murdering matriarchy of Lyrane, who have selectively bred their males to be sub-sentient dwarves with no function other than reproduction.

The author's inexhaustible imagination manages to keep these stories up to date, even more than half a century on. While the earlier volumes stressed what would decades later be called low-observable or stealth technology, in this outing he anticipates today's hot Pentagon buzzword, “network-centric warfare”: the grand battles here are won not by better weapons or numbers, but by the unique and top secret information technology of the Z9M9Z Directrix command vessel. The bizarre excursion into “Nth-space” may have seemed over the top to readers in the 1940s, but today it's reminiscent of another valley in the cosmic landscape of string theory.

Although there is a fifteen page foreword by the author which recaps the story to date, you don't really want to start with this volume: there's just too much background and context you'll have missed. It's best either to start at the beginning with Triplanetary or, if you'd rather defer the two slower-paced “prequels”, with Volume 3, Galactic Patrol, which was the first written and can stand alone.

Posted at 23:45 Permalink

Tuesday, April 25, 2006

Fourmilab: HotBits Secure Server Request Form Now Delivered Via HTTPS

When I recently implemented secure HotBits requests via the https: protocol, I did not use HTTPS in the link to the secure request form from the main HotBits page. I figured that since nothing in that page was secure, there wasn't any reason to burden the server by requesting it with HTTPS; the random data would be requested and returned securely, and that's what mattered.

Well, as is usually the case in questions of security, things are a bit more subtle. A few days after I posted the update to HotBits, this Handler's Diary from the SANS Internet Storm Center came to hand. While it focuses on bank login pages, the underlying issue applies to any site which receives or delivers sensitive data using HTTPS. Secure connections provide two things: protection against the data being intercepted, and authentication: confidence, based on verification of the site's certificate and the authority which issued it, that you're actually talking to the site you believe you are. While the former doesn't require the login page (or, in the case of HotBits, the request form) to be delivered via HTTPS, the latter does. Let's see how a malefactor might exploit non-secure delivery of the request form, using HotBits as an example. (Granted, the stakes with HotBits are so low this isn't likely to happen, but if you mentally replace a HotBits request with the user name and password of your on-line banking service, you'll see how serious this could be.)

Suppose some chapeau noir discovered that HotBits were being used for a security application worth compromising. They might, then, throw out bait intended to lure the unsuspecting to a bogus site, perhaps with a name which is a simple misspelling of “fourmilab” likely to entrap elbow-typers who regularly strike out when posting messages on public discussion boards: “forumilab.ch”, say. Then, they might pollute the Web with “link farm” pages which wrapped the word “HotBits” with a link to a URL pointing to this impostor site, in the hope it would pop out of search engines. When a user arrived at the bogus site, everything would look precisely like the legitimate site, but when a secure data request was received, a copy of the data returned securely to the user would be saved, along with the requester's IP address, with the intent of nefarious exploitation in the future.

By making the request form secure, the user is at least guaranteed that there is a certificate containing the name of the site they believe themselves to be accessing which their browser deems secure (or they've opted to accept after having been warned). In the case of the misspelled domain name, this doesn't help much, since if the user doesn't notice the spelling error in the URL, they probably won't notice it even if they choose to verify the certificate, but at least the information is there for the perspicacious. In the case of trolling for passwords, the stakes are much higher, since an innocent (but spelling-impaired) user might type his password into a non-secured page on a site named, say, “bankofamercia.com”, thinking they were connected to Bank of America. Indicative of the seriousness of this threat is that fact that Bank of America has registered this misspelling (among many others, including “bankofamerika.com”—hee, hee, hee) to preempt such entrapment of their customers.

Posted at 00:16 Permalink

Monday, April 24, 2006

Windows: Cygwin OpenSSH Service vs. Media Player KB911565/KB911564 Patch and iTunes Installation Hang

Microsoft issued two patches in February 2006, KB 911565 and KB 911564, to correct a serious vulnerability which permits remote code execution. These patches installed without problems on all of the Windows systems here with the exception of the Windows 2000 machine I use as the system management station for the Alcatel PBX, the Check Point Firewall software, and assorted other gizmos which run only on Windows. On that system, installation of the KB 911565 patch hung halfway through, and only killing the installer task would get rid of it. I even tried installing the patch after booting in Safe Mode (which, of course, in the interest of “safety” destroys the positioning of all your carefully-arranged icons on the desktop—feel safer now?) Nothing doing: it still hung. The hang only affected these two patches; I had no problem whatsoever installing all subsequent patches up to the present. I even tried re-installing Media Player from scratch, but Windows Update still required the patches, and attempting to apply them hung as before.

Being more or less out of ideas, I simply ignored these patches and removed them from the list to be applied when updating the system. Today, I tried to install iTunes for Windows on this system to permit downloading free “podcasts” without having to reboot my development machine into Windows and sit around while lengthy downloads complete, and the iTunes installer also hung before displaying its welcome screen. I popped up the task manager and noted that the CPU was pegged at 100% and, curiously, most of the CPU time was being consumed by the OpenSSH service installed under Cygwin. Even more curiously, when I terminated that task, the iTunes installer immediately popped up an “installation failed” dialogue. This was repeatable.

Now, there's no legitimate reason the iTunes installer ought to be using any Cygwin component, not to speak of the OpenSSH service. Since I've never actually used the SSH login facility on this machine, I opened up the Services applet under Administrative Tools and changed the run status for the service from “Automatic” to “Disabled”. After having done so, I restarted the iTunes installer and it worked perfectly. This hang was so reminiscent of the ones I'd experienced with the Media Player patch (and, after all, iTunes and QuickTime are similar in function to Media Player), I decided to try applying the two patches and guess what? They also both installed just fine.

After the obligatory reboot after patching Media Player (in the interest of brevity I shall refrain from heaping scorn on an operating system whose architecture requires a reboot to update the movie player application), the system came up normally and everything appears to work. At long last, Windows Update reports the system current, with no high priority patches outstanding. Once bitten, and having no real need for it, I decided to leave the OpenSSH service disabled. Note that this report applies to a machine running Windows 2000 with Media Player 9; as I noted in the introduction, I have not had any problems installing these patches on other Windows systems running Windows 98 and XP, most of which also have Cygwin installed. I have no reason, therefore, to suspect this problem applies to systems other than Windows 2000, but if you're installing something sound-and-light related on a machine with Cygwin and it hangs, take a peek at the task manager to see if OpenSSH is devouring your CPU.

Posted at 21:09 Permalink

Saturday, April 22, 2006

Fourmilab: WeatherCam joins FloodCam

After the Great Fourmilab Flood of February 2006, I installed the FloodCam so I (and anybody else who's curious) could keep an eye on the furnace room where the water entered. Due to the shortcomings of the Sony SNC-P5 camera I used (at least for my application), I resolved to try something different the next time, and ended up choosing an Fourmilab live camera: weather looking southward AXIS 206M megapixel network camera, in the hope its manufacturer would get it right. Did they ever! This is one of the cleanest network appliances I have ever encountered. Within five minutes after opening the box, I had live motion video streaming to my Firefox browser window on Linux, all without ever booting Windows, installing any software, or using any Microsoft products. The camera usually obtains an IP address from DHCP when you plug it into a network with the requisite server, but because I preferred to assign it a fixed address on the local LAN (behind the firewall), I set the IP address using arp and ping as described in the manual, which worked the first time. After that I simply pointed my browser at the camera's IP address, set the root password, and was able to immediately view images.

The embedded Web server has a large and well thought out tree of configuration pages, from which I was able to enable automatic time setting with NTP, set the subnet mask for the LAN, and specify the DNS servers. You can set the camera's image properties from the configuration pages, and specify extras such as including the date and time and a camera identifier in the image. Unlike the Sony camera, the AXIS permits image retrieval with a user-specified resolution (not limited to the configured default) with a simple URL. Up to 10 separate users (limited by network bandwidth) can connect and view motion video simultaneously. Separate password protected user accounts can be set up with different privileges, but since the camera is on the LAN and outside access is blocked by the firewall, I specified unrestricted read-only access by any local IP address without a password.

Maximum image resolution is 1280×1024 pixels, with about 12 frames per second of motion video in this mode (up to 30 frames per second in 640×480 and lower resolution). The CMOS image sensor provides 10 lux sensitivity. The manual focus lens has a fixed aperture of f/2.8 and 10 mm focal length, providing a field of view of 45°. Clicking the image above will take you to the Fourmilab Live! page with current images from both the Weather and Flood cameras. If the WeatherCam image is black, it's because it's dark out! You may occasionally catch headlights or taillights on the road at night, but there isn't a lot of traffic around here, especially after dark.

The camera is physically tiny: 100×55×34 mm. It is intended to be permanently mounted, and comes with a mounting stand which includes a ball-joint which permits orienting the camera as desired, including flipping it over for ceiling mounting. Unless I've missed something, there is no socket for mounting on a camera tripod, but if necessary that can be easily kludged by gluing a 1/4 inch nut to the mounting base. At the moment, the WeatherCam is on a jury-rig mounting attached to a ladder with twist ties sitting in front of a south facing window. I will eventually replace this with something more solid, probably the next time I need to use the ladder. This isn't the first time a camera has been mounted on a ladder at Fourmilab!

Posted at 20:36 Permalink

Friday, April 21, 2006

Reading List: 1968 : The Year That Rocked the World

Kurlansky, Mark. 1968 : The Year That Rocked the World. New York: Random House, 2004. ISBN 0-345-45582-7.
In the hands of an author who can make an entire book about Salt fascinating, the epochal year of 1968 abounds with people, events, and cultural phenomena which make for a compelling narrative. Many watershed events in history: war, inventions, plague, geographical discoveries, natural disasters, economic booms and busts, etc. have causes which are reasonably easy to determine. But 1968, like the wave of revolutions which swept Europe in 1848, seems to have been driven by a zeitgeist—a spirit in the air which independently inspired people to act in a common way.

The nearly simultaneous “youthquake” which shook societies as widespread and diverse as France, Poland, Mexico, Czechoslovakia, Spain, and the United States, and manifested itself in radical social movements: antiwar, feminism, black power, anti-authoritarianism, psychedelic instant enlightenment, revolutionary and subversive music, and the emergence of “the whole world is watching” wired planetary culture of live satellite television, all of which continue to reverberate today, seemed so co-ordinated that politicians from Charles de Gaulle, Mexican el presidente Díaz Ordaz, and Leonid Brezhnev were convinced it must be the result of deliberate subversion by their enemies, and were motivated to repressive actions which, in the short term, only fed the fire. In fact, most of the leaders of the various youth movements (to the extent they can be called “leaders”—in those individualistic and anarchistic days, most disdained the title) had never met, and knew about the actions of one another only from what they saw on television. Radicals in the U.S. were largely unaware of the student movement in Mexico before it exploded into televised violence in October.

However the leaders of 1968 may have viewed themselves, in retrospect they were for the most part fascinating, intelligent, well-educated, motivated by a desire to make the world a better place, and optimistic that they could—nothing like the dour, hateful, contemptuous, intolerant, and historically and culturally ignorant people one so often finds today in collectivist movements which believe themselves descended from those of 1968. Consider Mark Rudd's famous letter to Grayson Kirk, president of Columbia University, which ended with the memorable sentence, “I'll use the words of LeRoi Jones, whom I'm sure you don't like a whole lot: ‘Up against the wall, mother****er, this is a stick-up.’” (p. 197), which shocked his contemporaries with the (quoted) profanity, but strikes readers today mostly for the grammatically correct use of “whom”. Who among present-day radicals has the eloquence of Mario Savio's “There's a time when the operation of the machine becomes so odious, makes you so sick at heart, that you can't take part, you can't even tacitly take part, and you've got to put your bodies upon the gears and upon the wheels, upon the levers, upon all the apparatus, and you've got to make it stop” (p. 92), yet had the politeness to remove his shoes to avoid damaging the paint before jumping on a police car to address a crowd. In the days of the Free Speech Movement, who would have imagined some of those student radicals, tenured professors four decades later, enacting campus speech codes and enforcing an intellectual monoculture on their own students?

It is remarkable to read on p. 149 how the French soixante-huitards were “dazzled” by their German contemporaries: “We went there and they had their banners and signs and their security forces and everything with militaristic tactics. It was new to me and the other French.” One suspects they weren't paying attention when their parents spoke of the spring of 1940! Some things haven't changed: when New Left leaders from ten countries finally had the opportunity to meet one another at a conference sponsored by the London School of Economics and the BBC (p. 353), the Americans dismissed the Europeans as all talk and no action, while the Europeans mocked the U.S. radicals' propensity for charging into battle without thinking through why, what the goal was supposed to be, or how it was to be achieved.

In the introduction, the author declares his sympathy for the radical movements of 1968 and says “fairness is possible but true objectivity is not”. And, indeed, the book is written from the phrasebook of the leftist legacy media: good guys are “progressives” and “activists”, while bad guys are “right wingers”, “bigots”, or “reactionaries”. (What's “progressive” ought to depend on your idea of progress. Was SNCC's expulsion of all its white members [p. 96] on racial grounds progress?) I do not recall a single observation which would be considered outside the box on the editorial page of the New York Times. While the book provides a thorough recounting of the events and acquaintance with the principal personalities involved, for me it failed to evoke the “anything goes”, “everything is possible” spirit of those days—maybe you just had to have been there. The summation is useful for correcting false memories of 1968, which ended with both Dubček and de Gaulle still in power; the only major world leader defeated in 1968 was Lyndon Johnson, and he was succeeded by Nixon. A “whatever became of” or “where are they now” section would be a useful addition; such information, when it's given, is scattered all over the text.

One wonders whether, in our increasingly interconnected world, something like 1968 could happen again. Certainly, that's the dream of greying radicals nostalgic for their days of glory and young firebrands regretful for having been born too late. Perhaps better channels of communication and the collapse of monolithic political structures have resulted in change becoming an incremental process which adapts to the evolving public consensus before a mass movement has time to develop. It could simply be that the major battles of “liberation” have all been won, and the next major conflict will be incited by those who wish to rein them in. Or maybe it's just that we're still trying to digest the consequences of 1968 and far from ready for another round.

Posted at 22:48 Permalink

Sunday, April 16, 2006

HotBits: Secure Requests Now Available

The HotBits server has been providing random data, generated by radioactive decay, to all comers for almost a decade (it opened to the public in June of 1996). HotBits has even attracted modest press attention, with write-ups in The New York Times and, most recently, Science News. One constraint on the use of HotBits for security applications (for example, generating encryption keys or passwords) has been that the random data were returned to the requester in the clear and hence at risk of interception by a scoundrel snooping on the network connection between the user and the HotBits server.

To reduce this vulnerability, requests for HotBits may now be sent via the https: encrypted protocol with results returned the same way. The user's confidence that the request has, in fact, been processed by the Fourmilab HotBits server can be reinforced by verifying the secure server's certificate, issued by Thawte, a root certificate authority which most Web browsers consider trusted by default. To request HotBits from the secure server, use the new Secure HotBits Request page, which is XHTML 1.0 and CSS 2.1 compliant, and contains an icon which confirms the validity of the Fourmilab certificate and may be clicked to verify it. The request page is not usually delivered in secure mode (because it contains no sensitive information), but the requests it sends to the HotBits server and the data returned by them are encrypted, as the security icon in your browser will indicate.

While the secure request form has been upgraded to strict XHTML compliance, the HTML result returned from the HotBits server is unchanged (apart from being returned via the secure HTTPS protocol) classic HTML 1.0. A number of programs and subroutine libraries request HotBits data and parse the results programmatically, and changing the format of the data returned runs the risk of torpedoing these applications for no benefit other than purity of essence. Consequently, I have no intention of changing the HotBits results format in the foreseeable future.

Yes, I am aware that HTTPS/TLS/SSL encryption is not perfectly secure, nor are site certificates an absolute guarantee of the identity of the site to whom a user is submitting a request. This is the Real World, in which nothing is ever completely certain (you may be dreaming you're reading this, on a computer screen you're hallucinating due to neural stimulation by tendrils grown into your visual cortex by the alien mutant vegetable you ate for dinner). There is no positive finite number whose reciprocal does not exceed my interest in debating such matters.

Posted at 00:23 Permalink

Saturday, April 15, 2006

Reading List: Pleins feux sur. . . Columbo

Bannier, Pierre. Pleins feux sur… Columbo. Paris: Horizon illimité, 2005. ISBN 2-84787-141-1.
It seems like the most implausible formula for a successful television series: no violence, no sex, no car chases, a one-eyed hero who is the antithesis of glamorous, detests guns, and drives a beat-up Peugeot 403. In almost every episode the viewer knows “whodunit” before the detective appears on the screen, and in most cases the story doesn't revolve around his discovery of the perpetrator, but rather obtaining evidence to prove their guilt, the latter done without derring-do or scientific wizardry, but rather endless, often seemingly aimless dialogue between the killer and the tenacious inspector. Yet Columbo, which rarely deviated from this formula, worked so well it ran (including pilot episodes) for thirty-five years in two separate series (1968–1978 and 1989–1994) and subsequent telefilm specials through 2003 (a complete episode guide is available online).

Columbo, as much a morality play about persistence and cunning triumphing over the wealthy, powerful, and famous as it is a mystery (creators of the series Richard Levinson and William Link said the character was inspired by Porfiry Petrovich in Dostoyevsky's Crime and Punishment and G. K. Chesterton's Father Brown mysteries), translates well into almost any language and culture. This book provides the French perspective on the phénomène Columbo. In addition to a comprehensive history of the character and series (did you know that the character which became Columbo first appeared in a story in Alfred Hitchcock's Mystery Magazine in 1960, or that Peter Falk was neither the first nor the second, but the third actor to portray Columbo?), details specific to l'Hexagone abound: a profile of Serge Sauvion, the actor who does the uncanny French doublage of Peter Falk's voice in the series, Marc Gallier, the “French Columbo”, and the stage adaptation in 2005 of Une femme de trop (based on the original stage play by Levinson and Link which became the pilot of the television series) starring Pascal Brunner. This being a French take on popular culture, there is even a chapter (pp. 74–77) providing a Marxish analysis of class conflict in Columbo! A complete episode guide with both original English and French titles and profiles of prominent guest villains rounds out the book.

For a hardcover, glossy paper, coffee table book, many of the colour pictures are hideously reproduced; they look like they were blown up from thumbnail images found on the Internet with pixel artefacts so prominent that in some cases you can barely make out what the picture is supposed to be. Other illustrations desperately need the hue, saturation, and contrast adjustment you'd expect to be routine pre-press steps for a publication of this type and price range. There are also a number of errors in transcribing English words in the text—sadly, this is not uncommon in French publications; even Jules Verne did it.

Posted at 21:20 Permalink

Thursday, April 13, 2006

Physics: Nagging Little Discrepancies

An interesting paper was posted yesterday on arXiv, “Is the physics within the Solar system really understood?” which summarises the following apparent anomalies for which there are varying degrees of experimental evidence:
  1. Dark matter
  2. Dark energy
  3. The Pioneer anomaly
  4. Excess velocity increase of spacecraft which fly-by Earth
  5. Apparent secular increase in the astronomical unit (about 10 metres/century)
  6. Quadrupole and octupole power in the cosmic background radiation correlated and aligned with the ecliptic
I was unaware of 4, and 5 and hadn't heard much about 6 recently although it was rumoured something interesting might be in the three year WMAP data. This paper does not cite that data release.

Wouldn't be interesting if all of these effects could be explained by the choice of too large a numerical integration step in a simulated universe? Note that items 3 and 4 both involve small apparent discrepancies in the motion of man-made objects which move more rapidly than most natural bodies on such geodesics—the creator (or, perhaps I should write, more reverently, “Creator”) of a simulation who chose a time step suitable for planetary motion (for example, the 1/100th day integration step I used in the Quarter Million Year Canon computation) might just find themselves caught out by pesky in-simulation sentients who made precision measurements of high-speed gravitational assist maneuvers or objects on hyperbolic trajectories.

Here are a few questions for physicists and numerical integration experts. Is the choice of an insufficiently fine integration step likely to produce discrepancies of the sign and magnitude observed in the Pioneer, fly-by, and astronomical unit anomalies? How large an integration step would be required to produce the deviations from general relativity in each case? Are they all the same, or related in a simple way? Are there observations of solar system bodies (for example, sun-grazing comets on hyperbolic escape trajectories or close asteroidal encounters with planets) which could exclude (or provide evidence for) this hypothesis?

Posted at 14:30 Permalink

Monday, April 10, 2006

Reading List: Generation Kill

Wright, Evan. Generation Kill. New York: Berkley Caliber, 2004. ISBN 0-425-20040-X.
The author was an “embedded journalist” with Second Platoon, Bravo Company of the U.S. First Marine Reconnaissance Battalion from a week before the invasion of Iraq in March of 2003 through the entire active combat phase and subsequent garrison duty in Baghdad until the end of April. This book is an expanded edition of his National Magazine Award winning reportage in Rolling Stone. Recon Marines are the elite component of the U.S. Marine Corps—like Army Special Forces or Navy SEALs; there are only about a thousand Recon Marines in the entire 180,000 strong Corps. In the invasion of Iraq, First Recon was used—some say misused—as the point of the spear, often the lead unit in their section of the conflict, essentially inviting ambushes by advancing into suspected hostile terrain.

Wright accompanied the troops 24/7 throughout their mission, sharing their limited rations, sleeping in the same “Ranger graves”, and risking the enemy fire, incoming mortar rounds, and misdirected friendly artillery and airstrikes alongside the Marines. This is 100% grunt-level boots on the ground reportage in the tradition of Ernie Pyle and Bill Mauldin, and superbly done. If you're looking for grand strategy or “what it all means”, you won't find any of that: only the confusing and often appalling face of war as seen through the eyes of the young men sent to fight it. The impression you're left with of the troops (and recall, these are elite warriors of a military branch itself considered elite) is one of apolitical professionalism. You don't get the slightest sense they're motivated by patriotism or a belief they're defending their country or its principles; they're there to do their job, however messy and distasteful. One suspects you'd have heard much the same from the Roman legionnaires who occupied this land almost nineteen centuries ago.

The platoon's brief stay in post-conquest Baghdad provides some insight into why war-fighters, however they excel at breaking stuff and killing people, are as ill-suited to the tasks of nation building, restoring civil order, and promoting self-government as a chainsaw is for watchmaking. One begins to understand how it can be that three years after declaring victory in Iraq, a military power which was able to conquer the entire country in less than two weeks has yet to assert effective control over its capital city.

Posted at 22:09 Permalink

Sunday, April 9, 2006

Backhoe 3, Internet 0

Around 17:30 local time of Thursday, April 6th, the 2 Mbit/sec leased line which provides Fourmilab's main Internet connectivity went down. The router detected the failure and, as it is configured to do, established a dial-up ISDN connection to my ISP to serve as a backup. This connection is limited to 128 Kb, and while it's fine for E-mail, DNS, and local access to Web sites, it is hopelessly inadequate for the outbound Web traffic, so as seen from the outside the site, while never actually down, appeared to run so slowly you could barely tell the difference. I subscribe to a service which makes HTTP requests to the site every 15 minutes from a variety of peering points in Europe and the U.S. and prepares a daily response time report. Usually, 90% of these polls complete in less than half a second, with the worst case on the order of three seconds. On “Black Thursday” the maximum response time was more than twenty seconds and the mean response was almost three seconds!

hitrate_2006-04-09.png

Here is how the outage looked from the load balancer. Instead of the usual mean HTTP request rate between 6 and 8 packets per second, there's a “notch” for the duration of the leased line outage where only about two packets per second were processed. (The earlier down-spike that day is due to my restarting the Apache HTTP daemon to “cycle” the Web access log to a new file; that only causes about a 10 second interruption in service, but the load balancer spots it and marks it on the chart. An outage longer than 15 seconds will cause the load balancer to start directing requests to the hot spare server, but a log cycle isn't long enough to trigger the fail-over.)

The leased line outage was due, it turns out, to two Swisscom 20 Gbit/sec fibre optic backbone cables being cut, which darkened the Internet connectivity of some 30,000 clients, including Fourmilab. (Swisscom is not my ISP, but the line which connects Fourmilab to the ISP's point of presence in Bern is supplied by Swisscom.) Notwithstanding the newspaper story (click the thumbnail for an enlargement), which quotes Swisscom as saying the outage was three hours in duration, in fact connectivity was not restored until about midnight, as you can see from the hit rate chart. This was confirmed by the service desk at my ISP, who said that all their customers with connectivity through the affected cables came back up at the same time. Connaisseurs of technological trouble know that events like this, like quarks and macroscopic spatial dimensions, always come in threes, and won't be surprised to read that almost at the same moment the two fibre cables were cut which took down Fourmilab's connectivity, a third, completely unrelated, cable was cut about five kilometres on the other side of Fourmilab. Swisscom attributed the reports of outages by customers served by this cable to the two they were already working on, and only finally twigged to the actual problem when fixing the first two cables didn't make it go away; some people were cut off from land-line telephone service for as long as 30 hours.

When my original ISP went bankrupt in the aftermath of the dot.bomb, I considered buying half of my bandwidth from each of two different ISPs, with a router configured to load balance as long as both were up. As it turns out, both of the two substantial connectivity outages I've experienced in the last 12 years were due to cables cut by excavations which would have, in all probability, taken down both ISP connections. As they said about the ascent engine on the lunar module, “some things just have to work”.

Posted at 22:07 Permalink

Saturday, April 8, 2006

Linux: Take-Out from the Diner

For fans of the prolific work of humorist James Lileks, Friday means another broadcast from “The Diner”, a grab-bag of memories, music, and images from the bottomless dumpster of American popular culture. These are primarily audio programs, but starting in 2006, MP4 editions embellished with still images have been available. If you have one of those stylish pocket deafness appliances, it's said that you can just click the link without pausing to think but, if as I do, you play these programs on a Linux machine, it's rather difficult to download the MP4 files, which are hosted on the web.mac.com site that insists upon the presence of a QuickTime browser plug-in unavailable for Linux systems. You can download an audio-only MP3 edition, but then you miss the album covers, movie stills, and other graphics from the full version.

Digging into the HTML source code for the download page, it turns out there's an embedded object which specifies the MP4 file as a parameter interpreted by the QuickTime plug-in. If you extract this file name and cobble up a URL containing it, you can download the MP4 file and play it on your Linux system. After doing this manually several times, I bashed together a little Perl program to automate the process, which is now available for download. Extract the “Diner” program from the archive and fix the path to Perl in the first line if necessary. Then you can download the MP4 file for a broadcast to the current directory by pasting the URL of its download page on a command line which invokes the Diner program. The program also works with other video features occasionally posted by James Lileks and hosted on mac.com . You can try the Diner program with the following links, whose target URLs should be copied and pasted onto its command line, not clicked directly (unless you have the requisite QuickTime plug-in, of course, in which case you don't need the program in the first place):

In addition to Perl, the Diner program requires GNU Wget to download files from the Web; this program is included in most modern Linux distributions. Once a file has been downloaded, the Diner program's mission is accomplished. Then it's up to you to find a compatible media player application and figure out how to play the file. If you can't manage to play it, this is not my problem—it wasn't I who littered the multimedia landscape with five or six thousand mutually incompatible ways to store video and audio in computer files! I play these files with MPlayer/GMPlayer, but this may or may not work for you depending on which version you're using and the complement of audio and video CODECs you have installed. If you can't get it to work, I offer you only condolences, not consulting.

Posted at 15:51 Permalink

Friday, April 7, 2006

Reading List: Voyage à reculons en Angleterre et en Écosse

Verne, Jules. Voyage à reculons en Angleterre et en Écosse. Paris: Le Cherche Midi, 1989. ISBN 2-86274-147-7.
As a child, Jules Verne was fascinated by the stories of his ancestor who came to France from exotic Scotland to serve as an archer in the guard of Louis XI. Verne's attraction to Scotland was reinforced by his life-long love of the novels of Sir Walter Scott, and when in 1859, at age 31, he had a chance to visit that enchanting ancestral land, he jumped at the opportunity. This novel is a thinly fictionalised account of his “backwards voyage” to Scotland and England. “Backwards” («à reculons») because he and his travelling companion began their trip from Paris into the North by heading South to Bordeaux, where they had arranged economical passage on a ship bound for Liverpool, then on to Edinburgh, Glasgow, and then back by way of London and Dieppe—en sens inverse of most Parisian tourists. The theme of “backwards” surfaces regularly in the narrative, most amusingly on p. 110 where they find themselves advancing to the rear after having inadvertently wandered onto a nude beach.

So prolific was Jules Verne that more than a century and a half after he began his writing career, new manuscripts keep turning up among his voluminous papers. In the last two decades, Paris au XXe siècle, the original un-mangled version of La chasse au météore, and the present volume have finally made their way into print. Verne transformed the account of his own trip into a fictionalised travel narrative of a kind quite common in the 19th century but rarely encountered today. The fictional form gave him freedom to add humour, accentuate detail, and highlight aspects of the country and culture he was visiting without crossing the line into that other venerable literary genre, the travel tall tale. One suspects that the pub brawl in chapter 16 is an example of such embroidery, along with the remarkable steam powered contraption on p. 159 which prefigured Mrs. Tweedy's infernal machine in Chicken Run. The description of the weather, however, seems entirely authentic. Verne offered the manuscript to Hetzel, who published most of his work, but it was rejected and remained forgotten until it was discovered in a cache of Verne papers acquired by the city of Nantes in 1981. This 1989 edition is its first appearance in print, and includes six pages of notes on the history of the work and its significance in Verne's œuvre, notes on changes in the manuscript made by Verne, and a facsimile manuscript page.

What is remarkable in reading this novel is the extent to which it is a fully-developed “template” for Verne's subsequent Voyages extraordinaires: here we have an excitable and naïve voyager (think Michel Ardan or Passepartout) paired with a more stolid and knowledgeable companion (Barbicane or Phileas Fogg), the encyclopedist's exultation in enumeration, fascination with all forms of locomotion, and fun with language and dialect (particularly poor Jacques who beats the Dickens out of the language of Shakespeare). Often, when reading the early works of writers, you sense them “finding their voice”—not here. Verne is in full form, the master of his language and the art of story-telling, and fully ready, a few years later, with just a change of topic, to invent science fiction. This is not “major Verne”, and you certainly wouldn't want to start with this work, but if you've read most of Verne and are interested in how it all began, this is genuine treat.

This book is out of print. If you can't locate a used copy at a reasonable price at the Amazon link above, try abebooks.com. For comparison with copies offered for sale, the cover price in 1989 was FRF 95, which is about €14.50 at the final fixed rate.

Posted at 23:58 Permalink

Sunday, April 2, 2006

Reading List: Freakonomics

Levitt, Steven D. and Stephen J. Dubner. Freakonomics. New York: William Morrow, 2005. ISBN 0-06-073132-X.
Finally—a book about one of my favourite pastimes: mining real-world data sets for interesting correlations and searching for evidence of causality—and it's gone and become a best-seller! Steven Levitt is a University of Chicago economics professor who excels in asking questions others never think to pose such as, “If dealing crack is so profitable, why do most drug dealers live with their mothers?” and “Why do real estate agents leave their own houses on the market longer than the houses of their clients?”, then crunches the numbers to answer them, often with fascinating results. Co-author Stephen Dubner, who has written about Levitt's work for The New York Times Magazine, explains Levitt's methodologies in plain language that won't scare away readers inclined to be intimidated by terms such as “multiple regression analysis” and “confidence level”.

Topics run the gamut from correlation between the legalisation of abortion and a drop in the crime rate, cheating in sumo wrestling in Japan, tournament dynamics in advancement to managerial positions in the crack cocaine trade, Superman versus the Ku Klux Klan, the generation-long trajectory of baby names from prestigious to down-market, and the effects of campaign spending on the outcome of elections. In each case there are surprises in store, and sufficient background to understand where the results came from and the process by which they were obtained. The Internet has been a godsend for this kind of research: a wealth of public domain data in more or less machine-readable form awaits analysis by anybody curious about how it might fit together to explain something. This book is an excellent way to get your own mind asking such questions.

My only quibble with the book is the title: “Freakonomics: A Rogue Economist Explores the Hidden Side of Everything.” The only thing freaky about Levitt's work is that so few other professional economists are using the tools of their profession to ask and answer such interesting and important questions. And as to “rogue economist”, that's a rather odd term for somebody with degrees from Harvard and MIT, who is a full professor in one of the most prestigious departments of economics in the United States, recipient of the Clark Medal for best American economist under forty, and author of dozens of academic publications in the leading journals. But book titles, after all, are marketing tools, and the way this book is selling, I guess the title is doing its job quite well, thank you. A web site devoted to the book contains additional information and New York Times columns by the authors containing additional analyses.

Posted at 03:08 Permalink

Saturday, April 1, 2006

It Was Fifteen Years Ago Today. . .

…the Information Letter got away! Yes, fifteen years have come and gone since Autodesk Information Letter 14 self-replicated its way onto desktops all over Autodesk (and beyond; it was, arguably, one of the most widely-distributed “internal memos” in the history of the computing industry up to that time). That wasn't my intent—like other documents, I had circulated a draft copy to a short list of people whose opinion I value before sending the memo to senior management. Unfortunately, one of the individuals to whom I gave a draft was somewhat deficient in discretion and made copies for everybody in his department, whence they started to spread in all directions.

Information Letter 14 started the process of turning a complacent Autodesk around, which eventually resulted in the arrival of Carol Bartz as CEO a year later, in April 1992. To commemorate the fifteenth anniversary of Information Letter 14, a new, easier to read edition is now available, which is a prototype of the style to be used in the Fifth Edition of The Autodesk File which, given how much free time I have to work on it, will probably be available sometime in the twenty-sixth century. The new edition, like this Information Letter 14 pathfinder, will be 100% XHTML and CSS compatible, and use Unicode entities for typographic symbols unavailable in the Latin-1 character set. As in the Fourth Edition of The Hacker's Diet, each chapter will be a single HTML document, no longer chopped up into screen-size morsels as was done in previous editions in the interest of reducing download time for users with slow dial-up Internet access.

Posted at 21:23 Permalink