wazoox 2 days ago

OK, story time. I probably mentioned already that I did the computer graphics for World Cup 98 worldwide TV broadcast. The setup was using two SGI Octane MXI, controlled from a PC running a custom application driving the two Octanes, on each of the ten stadiums across France. On each site a local team managed the system; one guy using the graphics remote control PC in the OB truck, a couple of other guys entering manually statistic data on other remote control PCs, and another monitoring everything and sorting out problems.

For a lot of reasons, the development of the remote control application (written in Visual Basic 4.2 IIRC) was a painful death march that had began a too few short weeks before WC98 went live, weeks during which I've worked 100 hours a week and more, sleeping under my desk on which were the development Octane and PCs; I suppose many can relate.

As the remote control application was so late, it evolved almost daily during the World Cup; we basically debugged it at night, compiled and tested it during the day, pushed the binaries to our central server (an Origin 2000) through ISDN, binaries that the on-site team had to download and install before each and every match, after calling me to know the eventual problems to sort out.

The main point was that all features weren't ready from the beginning; during the first few weeks, the daily release of the software could only manage the things that were supposed to be happening in the field.

So one night I was (finally) home and I suddenly saw a match starting live on TV, in Lyon. And I realised Carlos (the local team manager in Lyon) hadn't called me about the daily software update. So I called Carlos while the match was beginning : "No worries", he said. "Yesterday's software worked fine, and I know you're tired so I managed it myself". "OK, but did you install today's update?" "What for? Yesterday's release was fine". "It was all fine yesterday", I replied, "but tonight is the first match which can have extra time and penalties. Yesterday's release doesn't manage penalties so I really hope we won't need it". Yeah, you know how this ends already...

Because you know, downloading the software through ISDN and installing it wasn't really an option once the match had begun...

What do you think happened? I watched the game going on and score stuck 0 to 0... And going to extra time... Still 0 - 0 at the end of extra time... Penalties! Time for improvising, live in front of about 2 billions spectators, isn't that nice? From that moment and until the end, I was on the phone with Carlos. First, to allow for a proper display, we had to reset the score to track the penalties like actual goals.

Then at last came the tough moment, displaying the final score, by filling in all data manually. I could only see the result live, on my home TV, hoping for the best... "OK, as the remote control software won't work, you'll have to manipulate the Octane directly" (which of course was never envisioned, and he had absolutely zero experience of). "Hit exactly all the keys as I tell you : on the numpad, hit 5000, then enter. Hit tab twice, press 3. Hit tab once, press 4. Hit tab twice, press 4. Hit tab once, press 3. Hit tab 3 times, press 0. Hit tab once, press 0. Press Ctrl + *. Press Ctrl + Enter. Call the OB truck and tell them to send in the picture".

It went all right in the end, but boy it was tiring.

2
EvanAnderson 2 days ago

I love the bad-assitude of knowing a system so well you can simulate it in your head, feed inputs to a third-party who actually has eyes/hands on the real thing, and have the state in your mind and reality match. I think of it a little bit like using a human as a convoluted (and sometimes unreliable) expect script.

xunil2ycom 2 days ago

I was a telephone customer support for WordPerfect back in the early 90s. Walking through our cubicles, you'd see most of us leaned back in our chair with our eyes closed talking people through WordPerfect's menu system to fix their problems. At some point, you just don't need to interact with the software to know what's on the customer's screen.

wazoox 2 days ago

As I had sweated night and day on these systems for 3 months, unsurprisingly I knew exactly everything about them in and out :)

wazoox 1 day ago

For the sake of exactitude, this was England - Argentine in Saint-Étienne (not Lyon, but Carlos managed both sites as they're not very far away), on June 30th 1998. As these were highly respected teams and a very disputed game (actually the score at the end of extra time was 2 - 2) the stakes were very high as there were literally billions of people watching the game live. Pressure was really strong... Fortunately I was used to and comfortable working in live broadcast conditions myself; Carlos not so much :)