Your web-browser does not support CSS, or you
have CSS switched off. For a better web experience, I recommend
using a modern
browser. Until then, an older
version of this site will look better in your browser.
I thought I’d take a few minutes to upload this video of the Harvard
Law School building-moving
project to YouTube. As
usual, the project stretched into three or four hours, complete with
the usual triumvirate of poorly-documented software, inexplicable error
messages, and mysterious crashes.
Now, the geeky part. If you’re not a computer nerd, the rest of this
post may bore you.
I wanted to concatenate a bunch of videos I took with my camera,
rotate the ones that I took at a 90 degree angle, strip out a few
seconds of chaff, and append a title card to each end. And I wanted to
speed them up. It’s impressive to see a house creep down a street at
all, but on the Internet, no-one wants to spend more than a minute on
it.
I settled for concatenating the videos. (And speeding them up.) Here’s
what actually worked, after several zillion different things that
didn’t:
Use transcode on Linux to concatenate
the AVI files together. (Concatenating them with cat
and running them through mencoder is a bad idea
(despite what some may
suggest
) – it produced a file that crashed most of other programs I tried
to run on it. Concatenation is easy: avimerge -o out.avi -i
in1.avi in2.avi in3.avi …
Use VirtualDub” to speed up the
video. Go to Video->Frame Rate. Click “Change to ___ frames per
second” and set that to some multiple of the original frame rate (I
used 10, so 15 fps became 150 fps). To ensure that the file is not
insanely large, you also have to decimate the video. Click “Decimate
by ___” and fill in the multiple of the frame rate you used (10, in
this case.) N.B.: transcode is a Linux program and
VirtualDub is a windows program, so I needed two boxen just to
upload a video to YouTube! Of course, I’m hardly the first to
rant about this stuff.
Compress the video (Video->Compression), because YouTube only
handles videos up to 100MB. I just picked the first codec that
worked “Indeo® video 5.10”). (I also dropped the audio (Audio->no
output, for a small savings.)
Upload it. This was the easy part.
Now, for the stuff that didn’t work:
VirtualDub has a wonderful interface for a piece of free video
software. It may be the only piece of free video software with a
wonderful interface. The problem with VirtualDub is that it’s
fantastically finicky about the video it will take. This became an
issue when I tried to paste my title cards (15 fps) onto my video
(15.00015 fps). Yes, of course, it refused.
Avidemux, despite being possessed of a…utilitatian (but very
functional) GTK2 interface, didn’t refuse to concatenate video
because of a tiny difference in the frame rate. The problem was that
it would randomly stop opening the videos it produced for reasons I
never understood. I’d use it to append a bunch of mpegs
together. (These were either straight from the camera or from the
camera via VirtualDub, where I’d rotated and cropped them.) It would
happily save the concatenated video to a file…then refuse to open
it. I’d open one of those files, and the application would instantly
quit without so much as an error message.
So I switched back to Linux. I took the clips that I’d rotated and
cropped and tried assembling them using the
catandmencoder method. Flop: I couldn’t slice
out the extra frames using the VirtualDub decimator. VirtualDub
would abort 1/3 of the way in and complain that the file was
corrupt. Same deal if I used the avimerge technique.
So I switched to Linux again. My guess as to how to drop the
unnecessary frames didn’t work: I tried the
–frame_interval option to transcode…but
transcode promptly crashed. Like this:
jdb@bigbox:/mnt/max/park$ transcode -i merge.avi --frame_interval 120 -o fast.avi
transcode v1.0.3 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg
(dvd_reader.c) no support for DVD reading configured - exit.
[transcode] (probe) suggested AV correction -D 0 (0 ms) | AV 0 ms | 0 ms
[transcode] auto-probing source merge.avi (ok)
[transcode] V: import format | MJPG RIFF data, AVI (V=ffmpeg|A=avi)
[transcode] V: import frame | 320x240 1.33:1
[transcode] V: bits/pixel | 1.562
[transcode] V: decoding fps,frc | 15.000,0
[transcode] V: Y'CbCr | YV12/I420
[transcode] A: import format | 0x1 PCM [11024, 8,1] 176 kbps
[transcode] A: export | disabled
[transcode] V: encoding fps,frc | 15.000,13
[transcode] A: bytes per frame | 733 (734.933333)
[transcode] A: adjustment | 1936@1000
[transcode] V: IA32/AMD64 accel | sse (sse 3dnowext 3dnow mmxext mmx asm C)
tc_memcpy: using sse for memcpy
[transcode] warning : no option -y found, option -o ignored, writing to "/dev/null"
[transcode] V: video buffer | 10 @ 320x240
[import_avi.so] v0.4.2 (2002-05-24) (video) * | (audio) *
[import_ffmpeg.so] v0.1.12 (2004-05-07) (video) ffmpeg: MS MPEG4v1-3/MPEG4/MJPEG
[export_null.so] v0.1.2 (2001-08-17) (video) null | (audio) null
[import_avi.so] format=0x1, rate=11024 Hz, bits=8, channels=1, bitrate=176
[transcode] input is mjpeg, reducing range from YUVJ420P to YUV420P
[filter.c] Filter "levels=output=16-240:pre=1" with args (levels=output=16-240:pre=1)
[filter.c] Filter "levels=output=16-240:pre=1" not loaded. Loading ...
[filter.c] Loading (levels=output=16-240:pre=1) ..
[filter_levels.so]: v1.0.0 (2004-06-09) Luminosity level scaler #0
[filter_levels.so]: scaling 0-255 gamma 1.000000 to 16-240
[filter_levels.so]: pre-processing filter
[mjpeg @ 0xb6953c68]invalid id 23618.38 fps, EMT: 0:05:53, ( 0| 0| 0)
Segmentation fault (core dumped)
(Sharp-eyed readers will note that that command wouldn’t have done
anything anyway, because the computer complained that I didn’t pass a
-y argument, so it wrote its output to /dev/null. I
didn’t notice this until just now. I tested it with an actual
-y argument: -y mjpeg,null (because the input was
mjpeg). That also bombed:
jdb@bigbox:/mnt/max/park$ transcode -i merge.avi --frame_interval 120 -o fast.avi -y mjpeg,null
transcode v1.0.3 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg
(dvd_reader.c) no support for DVD reading configured - exit.
[transcode] (probe) suggested AV correction -D 0 (0 ms) | AV 0 ms | 0 ms
[transcode] auto-probing source merge.avi (ok)
[transcode] V: import format | MJPG RIFF data, AVI (V=ffmpeg|A=avi)
[transcode] V: import frame | 320x240 1.33:1
[transcode] V: bits/pixel | 1.562
[transcode] V: decoding fps,frc | 15.000,0
[transcode] V: Y'CbCr | YV12/I420
[transcode] A: import format | 0x1 PCM [11024, 8,1] 176 kbps
[transcode] A: export | disabled
[transcode] V: encoding fps,frc | 15.000,13
[transcode] A: bytes per frame | 733 (734.933333)
[transcode] A: adjustment | 1936@1000
[transcode] V: IA32/AMD64 accel | sse (sse 3dnowext 3dnow mmxext mmx asm C)
tc_memcpy: using sse for memcpy
[transcode] V: video buffer | 10 @ 320x240
[import_avi.so] v0.4.2 (2002-05-24) (video) * | (audio) *
[import_ffmpeg.so] v0.1.12 (2004-05-07) (video) ffmpeg: MS MPEG4v1-3/MPEG4/MJPEG
[export_null.so] v0.1.2 (2001-08-17) (video) null | (audio) null
[export_mjpeg.so] v0.0.5 (2003-07-24) (video) Motion JPEG | (audio) MPEG/AC3/PCM
[import_avi.so] format=0x1, rate=11024 Hz, bits=8, channels=1, bitrate=176
[transcode] input is mjpeg, reducing range from YUVJ420P to YUV420P
[filter.c] Filter "levels=output=16-240:pre=1" with args (levels=output=16-240:pre=1)
[filter.c] Filter "levels=output=16-240:pre=1" not loaded. Loading ...
[filter.c] Loading (levels=output=16-240:pre=1) ..
[filter_levels.so]: v1.0.0 (2004-06-09) Luminosity level scaler #0
[filter_levels.so]: scaling 0-255 gamma 1.000000 to 16-240
[filter_levels.so]: pre-processing filter
*** glibc detected *** transcode: malloc(): memory corruption: 0x080f97a8 ***
======= Backtrace: =========
/lib/libc.so.6[0xb7e8682d]
/lib/libc.so.6[0xb7e8726c]
/lib/libc.so.6(__libc_memalign+0xab)[0xb7e8812b]
/usr/lib/libavutil.so.1d(av_malloc+0x2d)[0xb7fc155d]
======= Memory map: ========
<snip>
Aborted (core dumped)
I couldn’t find an equivalent option in mencoder, so I decided that
the rotate-and-crop videos must have been corrupted. I ditched them
and went back to the the raw ones from the camera. After some
fiddling, these actually worked.
In retrospect, I probably could have tried using VirtualDub to
assemble the cropped-and-rotated videos (I didn’t before, because I
was trying to add my title cards, which it wouldn’t do – at least,
I couldn’t produce title cards that it would take.) Ah, well. Next
time.
TechCrunch has a gushing post about
Yahoo! Pipes, which is some sort of graphical RSS-feed-editing
service. You can drag little boxes around to combine news feeds,
deleting posts that match certain criteria, keeping others, and publishing the result as a new RSS feed.
I can think of some uses for this, but I’m not quite as enthusiastic
as Michael Arrington.It’s very slick, but I’m not sure who the
intended audience is.
To Yahoo’s credit, the UI is so intuitive that in a few minutes, I’d
whipped up a new feed
for Guy Kawasaki’s blog that strips out his oh-so-cute minced
oaths “bull-shitake” and
“full of sushi.” Here’s what it looks like:
Unfortunately, I couldn’t figure out how to replace the phrases—I
could only delete the offending posts in their entirety. Call it
collateral damage in the war on preciousness.
If you sent mail to any @barillari.org address between early September
and late December, it most likely did not get through. The mail server
for barillari.org stopped working in early September. Because the vast
majority of the mail sent to @barillari.org addresses is bulk mail, I
didn’t make fixing it a high priority. My apologies if you sent
anything to a @barillari.org email address during that period — it
should be working now.
As more and more entrepreneurs start building what Fred
Wilson referred to as second derivative companies, I think they run a
big risk of designing a product/service that is targeted at too small
of an audience. Too many companies are targeting an audience of
53,651. That’s how many people subscribe to Michael Arrington’s
TechCrunch blog feed. I’m a big fan of Techcrunch – and read it
every day. However, the Techcrunch audience is NOT a mainstream
America audience.
WebDAV is a nice, lightweight, widely implemented network filesystem
protocol. davfs2 is the client implementation for Linux, which lets
you mount a remote WebDAV server.
Heaven forbid that you should confuse it, though. If you confuse
davfs2 (as I did, by trying to unmount the same share twice), you
might get this delightfully cryptic and mildly ungrammatical error
message:
/sbin/mount.davfs: Process 5852 already uses device cfs0.
Maybe it's only a stale pid-file left from unclean shutdown.
Please clean up manually.
Notice the lack of any hint as to where that pidfile is. The phrase
“Maybe it’s only a stale pid-file left from unclean shutdown.” doesn’t
appear in a Google or a Yahoo search; I hope putting it here will
fix that.
I thought I’d be smart about this. “I know!,” I said, “I’ll use
strace!” (strace prints out all of the
system calls that a program makes, so you can get an idea of what its
inputs and outputs are.)
strace -o foo mount /path/to/mountpoint
Of course, strace revealed not a word about pidfiles, device cfs0, or
the long-dead process 5824. Neither did ltrace.
Eventually I realized/remembered that mount was
(fork/exec)ing mount.davfs, so strace wasn’t picking up
anything that happened after the fork—which is when all of the
interesting stuff happened.
Thankfully, strace has a switch that makes it hang on to
all child processes: -f. So,
strace -f -o foo mount /path/to/mountpoint
…revealed the last few things that mount.davfs was doing:
Note the “AHA!” above. The pidfile was in /var/run.
I have no idea how /var/ is supposed to be arranged. Some
things make sense – logfiles are in /var/log, mail boxes
live in /var/spool/mail (depending on your setup). But
some things make no sense: mysql databases live in
/var/<b>lib</b>/mysql, as if that makes any sense. (They
aren’t code libraries!) I suppose that /var/run makes
sense for pidfiles but why couldn’t it be something obvious like
/var/pids?
(I know. I know. That would be too easy.)
Update: The source of the problem that prompted me to mount and
unmount davfs2 until it died and left behind a stray pidfile was also
in /var/ – the /var/cache/davfs2
directory. As it turns out, if you remove files behind davfs2’s back,
it can get confused, spewing errors like this:
/sbin/mount.davfs: Connection failed, mounting anyway. File system will only be usable when connection comes up.
When I looked at the network dumps (using tethereal "tcp port $PORT" -i lo -d tcp.port==$PORT,http -x -w path-to-output,
which I highly recommend), it turned out that davfs was requesting
files that I’d long since nuked. Blowing away the contents of the
/var/cache/davfs2 fixed the problem. (Why the heck do
these hang around between mounts, anyway?)
This is what I do now that I no longer have CS problem sets
Snopes.com broke with the usual repertoire of urban legends and posted the following:
The English language has at least one nine-letter word
that remains a valid word as each of its letters is successively
removed.
Their solution was “startling,” with the parting note that “there may
be other equally valid solutions.”
As a computer nerd, I couldn’t resist. With a quick hack, I found
13 words of length >= 9 that can be cut down to one letter, leaving a
valid English word each time. (To be fair, I found 12, since “splitting” is a derivative
of “splittings.”)
Note that “splittings” is a whopping ten letters long.
splittings
splitting
slitting
sitting
siting
sting
ting
tin
in
i
I’m using the /usr/share/dict/american-english-large dictionary from
Debian, ignoring all case and throwing out all words with apostrophes. (N.B.: american-english-large includes some unusual plural constructs like “shes” and “hes”.) I’m only counting “a” and “i” as valid one-letter words. Counting all single-letter words as valid under the self-reference rulebumps up the total by two, adding “carousels” and “flashiest,” but adding no words of length > 9.
I used to pride myself on coming up with awful ideasbefore they appeared on the Internet.
So, when I said to myself “hey, why hasn’t someone taken this mashup silliness to its logical conclusion and made a National Sex Offender Registry/Google Maps hybrid?”, I was disappointed to discover that someone had already done that.
I’m still waiting for the crimelibrary.com/Google Maps “Jeffrey Dahmer
Walking Tour” mashup, though…
“So…,” I said to Dan around 3:30 p.m. yesterday, “I’m sorry—I’ve
got to run. I want to put this computer together so I can show the
product to people by the close of business today.”
Ha.
Thirteen hours later, I now think that I might – might! – have the
computer working.
This is why I didn’t go into hardware.
Some tips, so you don’t repeat my mistakes:
Bring the laptop. Always bring the laptop. When I took the new
computer from Boston to Cambridge (where it resides for now), the
voice of reason won out: “Joe, it’ll only slow you down. It’s
really heavy. Besides, you have another computer there. Why would
you need the laptop?” Needless to say, I answered this one around
2 a.m., when I spent an hour or so trying to sort out a DHCP
failure at two in the morning with two potentially-broken
computers.
Don’t move a CD-ROM drive while it’s spinning. I trashed a Knoppix
CD this way. I might have even trashed the drive, because it didn’t
seem to work properly after that (but it might not have been working
to begin with).
The Gigabyte GA-8I945GMF motherboard isn’t just bleeding edge, it’s
freakin’ hemorrhaging wound. Linux 2.6.15 isn’t 31337 enough for
it, apparently – to get the on-board Ethernet to work, I needed
2.6.16. (Note that I said “work”. 2.6.15 recognized that there was
an Ethernet card, but pointedly discarded any data I tried to send
through it. An ancient 3COM boomerang card (which I fortunately had
snagged from MIT reuse some months back) did work, though.
On that same note, if you’re trying to use the motherboard with both
PATA (IDE) and SATA devices, make sure you switch the “On-Chop SATA
Mode” to “Enhanced”. Otherwise, you’ll get delightful errors: after
I booted the Debian installer from CD, it would complain that it
couldn’t find a CD-ROM drive. No, I’m not making this up.
After I managed to get the OS installed and booted, the kernel
started spewing KERN_EMERG errors (“CPU0: Running in modulated clock
mode” “CPU0: Temperature above threshold”) to the console. Googling
confirmed that these are generated by the processor when it thinks
it’s overheating. A gob of thermal paste didn’t help matters, but
reseating the heatsink did. It’s not as though you have to push the
heatsink anchors down hard. You have to push them down reallyreallyreally damned hard. I could have sworn I was going to
ruin the motherboard — which would have been one of the least fun,
but not the least fun, experience I’ve had for $90. (According to
Intel, you’re supposed to mount the heatsink after installing the
motherboard in the chassis – so it’s anchored a half-inch above the
case by a bunch of screws, and has a lot more room to bend than if
it were on the floor.) After I managed to get the confirmatory
click from all of the anchors, I haven’t seen the CPU core
temperature rise above 52 degrees. (Yet. I’m sure it will spike as
soon as I publish this, just to spite me.)
I actually had a related problem with an AMD processor last year
(which is why I had the thermal paste.) I should have learned my
lesson: buy nothing bigger than a Pentium II. Any speed gains from
the speedier processor are more than consumed by the trouble of
getting the faster one not to burn up.
I have a newfound respect for the guys who immersed their entire
computer in cooking oil as a new form of liquid cooling. I’d link to
them, except that I don’t have a web browser. (I’m typing this at
the console. Did you think I was enough of a masochist to try to get
X11 working on this thing?)
I now have the urge to rig up the computer to send me an SMS message
if the CPU core temperature gets too high. I’m not sure what’s
worse: that I know how to do this, or that I am tempted to do so.
All in all, I’m amazed that there are so many Internet sites that
publish twenty-page articles comparing heat sinks or motherboards or
processors – or anything else that requires you to take apart and
reassemble the heatsink-mobo-processor combo more often than
absolutely necessary. (I mean, as far as unpleasant hobbies go, at
least being a fire-eater or a shock rocker gets you the chicks, right?
I am unaware of anyone, anywhere, of any gender, ever having been
seduced by a tricked-out PC.)
Advice for the friend of an obsessive webalizer-checker
Dear Mr. Mouthpiece,
I have a friend who obsessively checks her web site statistics and flips out
if she doesn’t get enough visitors since her last look (typically
every 5-10 minutes). I’m worried — what should I do about this?
Harried at HMS
Dear Harried,
If the site is a blog, which I suspect
it is, the obsessive stat-checking is
typically a manifestation of a condition called novel hypertext-specific spectrum disorder. It frequently occurs in bloggers who are new to the web and haven’t yet become disgusted with the whole business. It doesn’t appear in every blogger; my suspicion is that your friend wasn’t hugged enough
as a child. That said, if she really wants website traffic, post a link to her site in a forum devoted to trolling blogs. She probably won’t thank you, but at least she won’t be concerned about the lack of traffic after that.
In their 2005 paper at the Symposium on Usable Privacy and Security, Conti, et al. discuss various ways in which a criminal might interfere with network security monitoring systems. The best of these is the speculation that an adversary might cause the displays of the security software to flash quickly enough to induce epileptic seizures in the security staff. They observe:
With the trend toward larger displays and higher
resolution the situation is worsened. In our experiments we were able
to generate network traffic that caused both static and updating
patterns in our network visualization system that would possibly
induce seizures in some photosensitive epileptics, but we did not
proceed further due to safety concerns.
Some people have interpersonal drama issues. No matter what they do, or how much they try, they always seem to be involved in some kind of clusterfuck or another. You’ve known these people: everything they get near turns into a disaster, and no matter how nice they seem, they are best avoided.
Me? I have computer drama issues.
I have completely dysfunctional relationships with machines. The simplest thing, that normal people do every day, starts a big screaming fight. I only rarely find myself in situations where I’m genuinely angry at a person, and yet, computers leave me in a blind rage on a pretty regular basis. It’s been worse in the last few weeks, but there have been a couple times in very recent memory where I had to make a conscious effort to close my eyes, breathe deeply and talk myself out of smashing things…
Today, I spent several hours trying to do something *completely
simple*: install a second video card in a Linux box. Some highlights:
The obvious solution (X -configure) crashed with an inexplicable error message:
*** If unresolved symbols were reported above, they might not
*** be the reason for the server aborting.
Backtrace:
0: X(xf86SigHandler+0x88) [0x80897b8]
1: [0xffffe420]
2: X(xf86DriverlistFromCompile+0x29) [0x80813b9]
3: X(DoConfigure+0x11) [0x8073dc1]
4: X(InitOutput+0x655) [0x8071a05]
5: X(main+0x239) [0x80d6589]
6: /lib/tls/libc.so.6(__libc_start_main+0xd0) [0xb7e6ced0]
7: X [0x8070131]
Fatal server error:
Caught signal 11. Server aborting
Heaven forbid that this should be that easy. (When X -configure works, it actually works.)
I tried two different PCI video cards, then noPCI video card. No
luck. I tried upgrading the kernel (and everything else). No
luck. Signal 11. Looks like I had to do it the hard way.
So I dinked around with settings for a while before realizing that one
of these cards was justnotgoingtowork. I switched to the
steam-powered TI Permedia 2 card, for which I had an old copy of the
correct Display settings (which had been spit out by X
-configure before it decided to die on me. For those of you who
have this card:
I think I eventually gleaned it from complaints that the computer was
excreting into /var/log/Xorg.0.log (if I remember correctly, it
complained that there was no device for PCI:0:11:00, and I eventually
got the hint…)
I had to figure out the HorizSync and VertRefresh settings for the
monitor. Because I couldn’t find the manual on the Internet (there’s
no obvious relation between model number on the back and the monitors
on the manufacturer’s website, and I was too impatient to download
every 1.5MB PDF manual from the manufacturer’s dinky 30kbps Internet
link). Fortunately, the screen displayed “OVER RANGE” (instead of
blowing up) when I set the HorizSync too high, so I just
binary-searched it down to the right range, restarting X over and over
and over and over and over.
As a final kick in the crotch, I couldn’t get it to shift to the
native resolution (1024×768, I assume). Below a certain HorizSync, it
would go to 800×600, above it, it would go to an ugly 1152×864 (!?). I
couldn’t figure out how to force 1024×768, so I eventually just gave
up and left it at 800×600. At least it’s easy to read.
(Yes, I realize that switching to the Mac or Windows would have made
this easy. Of course, those platforms introduce a host of their owndisgusting problems which pushed me to Linux to begin with. I’m wondering
if I should just go back to punched cards and paper tape.)
(In context, it looks like this translates to “Privacy disasters could
hurt Google, just as security disasters could hurt Microsoft,” rather
than “Google is as concerned about privacy as Microsoft is apparently
concerned about security.” Before I read the context, though, the latter seemed more likely.)
Conspiracy of the week: subtle crypto-fascism at MSN
MSN Search has a neat little “quick facts” search service – if they
know the exact answer to a query (e.g., “king of spain”, “babe ruth
ERA”), it will display it just above the search results. A recent
release announced the addition of 10^6 more facts:
It’s not perfect, of course:
First, there’s the obvious problem that he isn’t. Second, doesn’t “Leader of
Germany” usually conjure up images, not of the prime ministership, but
of a certain historical political office?
An “AMV” or “anime music video” is a fan-produced recording made by
overlaying a soundtrack with spliced-together clips from Japanese
animated films. Needless to say, a good deal of the source material
(both audio and visual) in the “AMV scene” is used without permission,
which has raised the ire of some of the copyright holders.
Larry Lessig posted
today about
music-industry opposition to the kids who make AMVs, lamenting it as
the latest effort by Big Media to hammer the Internet into a
“consume-only” medium.
While I’m generally sympathetic to Prof. Lessig’s arguments about
copyright, the AMV scene strikes me as a particularly misleading
example. An typical AMV contains several minutes worth of a movie or
movies and the entirety of a recorded song. The defense in a
copyright-infringement suit would face an uphill battle. But there’s
no reason AMVs have to be distributed that way – it’s an issue of
technology, not a legal issue.
There is nothing about modern computers that prevents AMVs from being
distributed not as an audiovisual recording, but as a set of
instructions a computer program could follow to assemble the AMV from
the source material. If you (the viewer) have all of the source
materials on your computer, the program will reassemble and play the
AMV. Some media owners might sue, but their chances would be
considerably less likely (Galoob v. Nintendo, for instance, confirmed
a user’s right to apply patch-files to media they own.) As a bonus,
the AMV files would be much smaller – a few kilobytes, instead of
several megabytes.
If the AMV scene wants to avoid copyright problems, technological
solutions are well within reach.
Another few hours of my all-too-short youth irretrievably lost
[Warning. Hardcore nerd post.]
In the process of writing a web application with mod_python, I had
to restart my web server. A lot. Apparently, Apache 2 (the web server
in question) doesn’t like this, and starts dying on boot with error
messages along the lines of
[Mon Dec 12 02:23:17 2005] [emerg] (28)No space left on device: Couldn't create accept lock
…although the disk had plenty of free space and no shortage of inodes.
I Googled this quite a bit. A number of sites suggested looking for
zombie semaphores with ipcs and deleting them with ipcrm. But when I ran ipcs, all I got was
It took me way too long to realize that ipcs was showing me the
semaphores that I (user=jdb) owned, not the ones that the web server
(user=www-data) owned. [Slaps forehead.]
Oh well. Live and learn.
So, to actually trash all of the unneeded semaphores (why is there a
limit on these, anyway?), become root, stop apache, and type something
along the lines of:
for i in `ipcs -s|grep www-data|tr ' ' '\t' |cut -f 2`; do ipcrm -s $i; done
Paul Graham: Brought down the house with (and I paraphrase,
not so well) A metaphor is like a function with the wrong
arguments – so presumably your brain must support dynamic
typing.
Michael Mandel, Chief Economist, Business Week: “People who
start companies and are willing to fail are doing God’s work.” Loud
applause. Whooping. (I wish – wish! – someone had screamed
“Preach it, brother!” I should have.)
Woz: high-speed 30 minute speech
on the design of the Apple I and Apple II. I was rapt – my notebook
reads “Woz. Amazing.” – so was everyone else. He got a standing
ovation. I’m glad I didn’t have to follow that act.
About 99.5% of the audience (which filled Science Center D, a 500-seat
audtiorium) is male.
I just got a short email from a reader of the UN-Internet-takeover article. I tapped out
a reply, only to get this message back:
I apologize for this automatic reply to your email.
To control spam, I now allow incoming messages only from senders I
have approved beforehand.
If you would like to be added to my list of approved senders, please
fill out the short request form (see link below). Once I approve you,
I will receive your original message in my inbox. You do not need to
resend your message. I apologize for this one-time inconvenience.
Click the link below to fill out the request:
https://webmail.pas.earthlink.net/wam/addme?a=(rest snipped)
If someone sends me email expecting a reply, they ought to add my
address to that “approved” list. Any
non-brain-dead
email software would do that automatically. It looks like the writer
used a feature provided by the mega-ISP Earthlink, a company that
really should know better. (As an aside to my correspondent, my
apologies if your ISP forced this on you.)
I don’t plan to click – it just
encourages this sort of
thing.
I’m still floored that people who lament the lack of
majors/women/minorities/non-nerds/funding in academic computer science
continue to use the terms “computer science” and “information
technology” interchangably.
As I mentioned a few weeks
back,
corporate “information technology” embodies everything that’s
repelling people from the field. Computer science is what people do
in research labs and academic departments. Corporate “information
technology” is what people do in Douglas Copeland’s
Microserfs,
for instance:
“…reworking old code for something like the Norwegian
Macintosh version of Word 5.8.”
Professor Klawe worries aloud about the CS funding crunch and the
dearth of women in computing. Again, I can’t say much about the
former, but the most elequent answer to the latter came ten years ago
(a veritable eternity in this field) from Philip Greenspun:
Intelligent people with PhDs are working as C
programmers; The average engineering career lasts seven years, pays
average, and doesn’t justify an MIT education that costs $120,000;
anyone smart enough to make it as a computer scientist can make it
with less work and risk as an MD, MBA, or JD; there has been so little
progress in programming environments, systems, and computer languages
in the last three decades that programmers in India and other Third
World countries are perfectly capable of taking over the majority of
American computer science jobs.
Your January issue [of Communications of the ACM] asks
“Why are there so few women in computing?” Maybe you should do another
issue asking “Why are there so many men?”
Date: Mon, 14 May 2001 22:51:11 -0500
From: Adoh Fadduq
Newsgroups: gnu.emacs.help
Subject: Possible security issue with Emacs?
Insha Allah, I am now trying to choose an editor for my software
development and typesetting work. I have closely considered Emacs,
which fits my needs in some respects. I do, however, feel that there is
a big security issue with it for me and my brethren: Emacs was largely
developed by Jews and for Jews. Considering how cunning the Jews are,
I would not be surprised to find that they have hidden special bugs
and booby traps inside emacs, in order to spy on and disrupt work of my
Allah believing brethren. Are my concerns justified?
Computer science enrollments collapse! Carpet of toads seen in Berkeley! Locusts descend upon MIT!
Prof. Felten’s latest blog post discusses the ‘computer science malaise’, a winter of grant contractions and declining enrollments. I can’t speak to the grant situation, but I can spout a few numbers on the enrollment situation.
Below is a table of a few key figures relating to CS enrollment at
Princeton. (The registrar’s
office
supplied the data with the exception of the thesis information, which
I pulled from the Princeton senior thesis
catalog.
For reference, Princeton grants two degrees: the AB (what everyone
else calls a BA) and the BSE (BS in Engineering). Computer Science is
the only department that offers both degrees. Senior theses are
required of all AB majors and are completed by some CS BSEs. “EE,”
“MechE,” and “all eng” are the number of electrical engineering
degrees, mechanical/aerospace engineering degrees, and all engineering
degrees granted that year.
year
CS theses
AB CS
BSE CS
total CS
EE
MechE
all eng
1994
11
12
14
26
32
50
183
1995
14
11
20
31
24
55
190
1996
6
5
9
14
25
37
153
1997
9
7
29
36
35
27
178
1998
10
9
27
36
32
29
182
1999
20
13
28
47
31
28
170
2000
14
8
29
37
43
35
172
2001
20
12
36
48
26
30
162
2002
15
13
34
47
54
31
196
2003
11
9
21
30
45
31
175
2004
14
10
27
37
34
37
167
* The registrar calls this “1995-1995”, assumed to be a typo. Number of
BSE-CS majors this year seems too small — can anyone comment?
The same data, with a few columns removed for clarity:
(The y axis shows the number of degrees (or theses) granted in a
given year. Thanks to GC for pointing out the
ambiguity. Update: Graph really fixed. Thanks, GC. Mea culpa. )
Certainly, the number of majors is down 23% from its peak in 2001 at
47. Certainly, this is a limited dataset. Certainly, students have to
decide where they want to major years before graduating (although
switching from CS to operations research and financial engineering, a
popular major for pre-Wall Street types, is presumably not difficult
if one took a few ORFE prerequisites). But, fortunately, Princeton has
not seen the same 60% decline claimed by the study Prof. Felten mentions.
The percentage of incoming undergraduates indicating that
they would major in CS declined by over 60 percent between the Fall of
2000 and 2004, and is now 70 percent lower than its peak in the early
1980s.
(There’s a huge spike in interest centered around 1982 – can anyone
speculate on what prompted that?)
“Interest” figures are less interesting than actual graduation figures
– they depend on the phrasing of the question, the news heading up to
the time of the survey, the circumstances under which it was
administered, and a variety of other factors. To make matters fuzzier,
people believe a lot of strange things at 18. (Case in point: when I
was a freshman, I voted for Al Gore.) Enrollment figures are also
fuzzy: different schools require declaration at different times; some
make it harder to switch than others.
Back to Chazelle and Arora’s “Viewpoint: Is the thrill gone?”:
While computing technology is thriving and extending its
reach further into our everyday lives, computer science is facing a
crisis in the U.S., including falling undergraduate enrollment
and reduced research funding.
My 2¢? I know too little to speak about funding. As to student
enrollment, with all due respect to Profs. Chazelle, Arora, and
Felten, I’m not particularly worried. CS will always attract the
students who will advance the state of the art: those who genuinely
enjoy the field. As to those who, spurred by the dot-com gold rush,
swelled the ranks in the late-00s, I wish them the best of luck, but I
don’t think their departure for greener pastures (Econ? ORFE?)
imperils the study of computer science.
P.S.: I’m a bit puzzled by Chazelle and Arora’s use of language:
Many ongoing efforts aim to develop new high school and
college curricula that would help lead to an IT-literate work
force. While supporting them, we wish to raise another enrollment
issue critical to the field: attracting bright high school students
and undergraduates who represent the next generation of IT researchers
and educators.
IT? I’ve always used IT as a shorthand for “corporate information
systems,” or “the stuff that a nerd wouldn’t voluntarily touch with a
ten-foot pole”. ‘IT’ means ‘TPS reports.’ ‘IT’
means ‘9-to-5, Monday through Friday.’ I’m surprised that a pair of CS
profs would lament the lack of “IT researchers and educators” —
isn’t more CS researchers and educators what they want? If we’re to
point fingers, it was corporate IT that sucked the ‘thrill’ out of
computer science’s reputation among undergrads. (That, or Scott Adams. )If there’s anything truly
opposed to the “hacker” philosophy that makes CS thrilling,
it’s the reputation of corporate information technology.
(Apologies for the non-link; ACM keeps its content in a walled garden.)
Update: Mihai pointed me to an article
in the PAW that shows a massive drop in CS enrollment from the class
of ‘06 to the class of ‘07. The article is really about Dean Nancy
Malkiel’s latest edu-engineering effort to push students out of
popular departments and into unpopular ones. As an aside, I haven’t
decided if I think this is a good idea. (The drop in CS enrollment
might also be due to other factors – Shirley Tilghman’s Green Hair
Initiative,
for instance.) I’ll hold off on drawing real conclusions until ‘07
graduates — enrollment jumps around, especially when people shift
between different parts of the engineering school, but graduation is
permanent.
After an unpleasantly large Verizon Wireless bill arrived this
morning, I decided to write a program to let me know if I’m burning
too many peak minutes in a given month. If you run this script daily (as a cron job, for instance)
it will let you know if you’ve gone over a preset threshhold, either
by email, or by writing a message to a file (my RSS reader,
feedonfeeds is hacked to display
the contents of that file), or both.
If you wanted to, you could even set the email address to your
vtext.com address, so the program would send you an SMS message if you
went over. (You might want to run the program less often in that case,
because SMS messages are billable.)
This sort of thing would be trivially easy to build into the phone,
but that would interfere with the business models of cell phone
carriers – don’t hold your breath.
If you need to do any graph-wrangling, I highly recommend
NetworkX, a handy graph library
written in Python. (This is for graphs in the nodes-and-edges sense,
not graphs in the bars-and-charts sense. For the latter, I’d use
Ploticus or Gnuplot.)
From: Yahoo!Shopping <rate-merchant@store.yahoo.com>
To: joseph barillari
Subject: Yahoo! Shopping Merchant Review for ArtCity.com - order#
artcity-com-26790
Dear joseph barillari,
Thank you for your recent purchase on Yahoo! Shopping.
Our records indicate you recently made a purchase from ArtCity.com.
Please take a moment to rate and review this merchant. By rating and
reviewing merchants, you can help other customers find merchants and
provide those merchants with feedback. It's easy - just rate the
merchant and add any helpful comments.
Before you start, review our Merchant Review Guidelines at this URL:
http://shopping.yahoo.com/merchrating/general_info.html
Yahoo! reserves the right to refuse or remove any review that does not
comply with these Guidelines or the Yahoo! Terms of Service and
terminate your Yahoo! account (including email) for a
violation. Yahoo! is not responsible or liable in+any way for ratings
and reviews posted by its users.
Mmm, now there’s nothing like threats to make me want to waste five
minutes of my life reviewing a web merchant…
Inform Capital One of my intention to post to bugtraq in 24 hours
13 Apr 16:31:11 -0400
Capital One form letter arrives: “this [phishing] email has not compromised Capital One’s systems in any way,”
13 Apr 16:44:42 -0400
Reply to Capital One form letter: “this email has taken advantage of a compromised Capital One system: Capital One’s website redirects URLs without checking them….please see the note about bugtraq below”
13 Apr 16:47:15 -0400
Another form letter: “A Capital One representative will respond to your e-mail inquiry, usually within 24 - 48 hours. Please note, due to high email volumes, this timeframe may be extended to up to 72 hours”. I wonder if saying “bugtraq” provokes this response.
19 Apr 16:32:15 -0400
Four business days later (well beyond 72h), redirect is still unchecked. Post bug to bugtraq and cc Capital One
19 Apr 16:53:46 -0400
Reply to Capital One (signed by a human?) form letter: “the point is that the phishing email has exploited a flaw in Capital One’s systems. Your website permits unchecked redirects. This makes a phisher’s job much, much easier.
19 Apr 18:01:00 -0400
A bugtraq subscriber tells me that he’s emailed abuse@capitalone.com (I should have thought of that)
19 Apr 14:27:05 -0800
Another bugtraq subscriber tells me that it’s fixed. Checked myself — apparently, it is.
19 Apr 18:55:38 -0400
Send email to webinfo@, thanking them for fixing the unchecked redirect.
Gah! Capital One enables revolutionary new “phishing” feature
A phisher just sent me an email claiming to be from Capital One,
inviting me to enter my account information at his website. Unusually,
Capital One is actually faciliating this scam: they provide an
unchecked redirect feature on their website, so the URL actually
starts with http://www.capitalone.com. Captial One then redirects
the user to the phisher’s site!
If this link still redirects to Wikipedia, then the problem hasn’t
been fixed yet:
From: "Capital One Representative: Kristina Barker " <Kristina.Barker@capitalone.com>
To: 2bslashdot@barillari.org
Subject: Error: Your Capital One Account Tue, 12 Apr 2005 22:25:00 -0800
X-Spam-Score: 10.407
X-Spam-Flag: YES
X-Spam-Level: ********** (10.407)
Capital One is committed to maintaining a safe environment for its
community of buyers and sellers. To protect the security of your account,
Capital One Bank employs some of the most advanced security systems in the world
and our anti-fraud teams regularly screen the Capital One Bank system for
unusual activity.
We recently have determined that different computers have logged onto your
Capital One Banking account, and multiple password failures were present before the
logons. We now need you to re-confirm your account information to us. If this is
not completed by April 14, 2006, we will be forced to suspend your account
indefinitely, as it may have been used for fraudulent purposes. We thank you for
your cooperation in this manner.
In order to confirm your Online Bank records, we may require some specific
information from you.
Click below to verify your account
http://www.capitalone.com/redirect.html?linkid=SECURITY+VALIDATION&dest=http://24.232.117.142/bin/capitalone.com/
Thank you for your prompt attention to this matter. Please understand that this is
a security measure meant to help protect you and your account.
We apologize for any inconvenience.
If you choose to ignore our request, you leave us no choice but to temporaly suspend
your account.
Thank you for using Capital One Bank!
----- End forwarded message -----
Mail had been backing up for the past 24 hours and I hadn’t even
noticed it. It wouldn’t be the first time this had happened –
frequent wedging was one reason why I switched from exim to
nullmailer, on the principle that a simpler mta would be a more
reliable MTA (that, and Harvard firewalls port 25, so there was no
point in running a real SMTP server anyway).
Restarting nullmailer cleared the queue, but left me wondering how to
notify myself next time this happened. I couldn’t use email, for
obvious reasons. I considered having a sound play through the speakers
if the queue seemed wedged, but that would only work when I was at
home.
I settled on a cron job to periodically dump the output from
mailq to a file. My RSS reader, feedonfeeds, now prepends
this file’s contents in giant honking red letters to my RSS display
— procrastination via RSS now has a purpose!
While writing this, it occured to me that another solution would be to
alias mutt to a script that alerted me if the queue was non-empty
before running mutt:
#!/bin/sh
# warns user if WATCH's output is nonempty before running mutt
MUTT=/usr/bin/mutt
WATCH=/usr/bin/mailq
TMP=`/bin/tempfile`
$WATCH > $TMP
if [[ -s $TMP ]] ; then
less $TMP ;
fi
rm $TMP
$MUTT
How to score an above-the-title slot on the Drudge Report
By now, everyone’s heard of the website whose owner promises to
butcher and eat Toby, his pet rabbit, unless he receives US$50,000 by the end of
June. As with the infamous Bonsai
Kitten, Snopes says that it’s a
hoax.
Toby’s owner had the right idea, but he wasn’t exploitative enough:
I’d like to see a website whose owner vows to have an abortion unless
she receives some enormous amount of PayPal
donations in the next nine (eight…seven…six..five…four…)
months. (Bonus points if the “mother” fakes her photo the old-fashioned way using a
balloon, rather than just stealing pictures from Google Images or launching Photoshop.)
Maybe Drudge will dust off his flashing-police-beacon animated GIF for
this one.
So, there I am, innocently reading a Wikipedia article on a man of leading cultural significance.
Then I follow the link to the Amazon.com page for his biography. For
some reason, Amazon treats ELinks users to a “Sponsored Links” section that Firefox users don’t get to see. And one of the sponsored links, for reasons I cannot fathom was…
charact55 Maryland-Baltimore
Lately, it seems like when dealing with females, all there is is
emotional problems. But on LonleyWives, most of the women just want to
have fun, and that is music to my ears.
Nope, nosiree, I can’t imagine the crew on this site would have any
emotional problems. Seeking clandestine romantic fulfillment elsewhere
is the hallmark of a healthy relationship and stable emotional status.
I am sorely tempted to submit this as an ALOD. Or maybe I’ll just
start my own section for these: I’ll call it OMG Egregious Internet
Abuse – or OMGEAI. Hmmmm… In fact, I will do that. Watch this
category.
This site always gets more attention when I should be doing real
work – as such, revamped the photos section’s layout,
uploaded a whole mess of new photos, and even started organizing them
by keyword.
If you should be doing something useful right now, by all means,
have a look.
The barillari.org server’s motherboard (a GA-7VRXP 2.0) bought it
several weeks ago, and barillari.org has been offline ever since. My
apologies. I’ve recently resurrected it. There may be further downtime
in the future, until the server moves to Boston with me at the
beginning of September.
barillari.org is now at its summer home. The pipe here is
much narrower (cable modem vs. massive massive campus bandwidth), so
go easy on the spidering.
