Law of Demos, tests and transpops

The Law of Demos is nice to learn before upcoming Tcl/Tk conference.

As the law's wiki seems to be designed for mobiles only, below it's reproduced as a plain text, with a cosmetic correction. The reading is rather merry.

The Law of Demos reads:

    In general, when x is being demonstrated, x will fail.

Thus...

Thus, the demos are very useful for testing and debugging.

In my humble experience, the most troublesome issue of GUI development is the way to catch bugs.

The stops, steps and watches of debuggers are often not too much helpful here, as their GUI events tend to interfere in a tested stuff, with bugs gladly escaping or messing all up.

Hence the indispensability of good old puts and home-made tools like log files.

Still, to catch a bug you need to reproduce it. And it's here demos may come to help.

I mean demos that are recorded as a youtube film or a screen video. Not a live presentation like the famous failed one by Bill Gates.

Example

While preparing demos of alited like this one or that one, I have been stumbling upon more or less weird bugs, in strong conformity with Law of Demos .

After some unsuccessful attempts to reproduce the bugs (demos removed to trash, with curses and donnerwetters:), I thought: why, stupid, why to remove these valuable proofs of your stupidity? why don't use them to catch the bugs?

A typical example: in this demo there is a bug at 03:15 / 04:18 time points, namely:

    cursor position of a unit is saved as 309.10 and restored as 309.1

The reason is the text widget's position arithmetic, as the program need to calculate the restored cursor position as an absolute unit's start + a relative shift. Simple expr {$start+$shift} won't fit.

The bug's cause is obvious only if you are happy enough to notice the resemblance of 309.10 and 309.1 and to bind it to the text arithmetic.

I could not probably manage to reproduce this bug without reproducing the demo and then repeating it step by step in the program. After that the debugging was trivial.

transpops package

The transpops package displays popup messages. I use it for all of my demos . It's available by the links:

The messages are read from a text file and after pressing a hotkey are shown one by one, in transparent mode.

First called, the popup message is displayed under the mouse pointer. The following messages will be displayed at the same screen coordinates if not reset with an alternative hotkey.

At clicking a popup message, it disappears. Without clicking, it disappears in a while by itself.

Runs this way:

    package require transpops
    ::transpops::run transpops.txt {<Alt-t> <Alt-y>} {.w1 .w2 .w3} ?fg? ?bg?

where:

  • transpops.txt - name of text file containing messages
  • Alt-t - hotkey event to start the pop-up of messages at current pointer position
  • Alt-y - alternative hotkey event to restart the pop-up of messages at new coordinates
  • .w1 .w2 .w3 - list of toplevel windows the events are bound to
  • fg, bg - foreground and background colors

Of course, those <Alt-t> and <Alt-y> may be substituted with other events specific for a application.

Example of a text file (the --- lines are separators for messages):

    Let's set some basic options of alited,
    mostly dimensions and colors.
    ---
    The dimensions of alited's window
    are usually set by mouse.

    Let's resize the four panes by their sizers.
    ---
    A little bit of efforts need to size
    "Row" column of the unit tree.

    Not too much, though.
    ---

Screen recorder

The screen recorder I use is SimpleScreenRecorder .

Simple, smart and very helpful.

Developed for Linux. Hopefully, for other platforms it's compiled too.


Law of Demos

In general, when x is being demonstrated, x will fail.

For examples,

  • A "bulletproof" demo will have Demo Meltdown;
  • A bug will fail to reappear in front of the programmer;
  • The hardware fails to power up;
  • The demo install you just gave to Bill!!! himself fails;
  • The brochure misspells the product name;
  • (For a different sort of failure...) Cooked demos will be bought as if they were existing products;

etc.


One would think the Law Of Demos duefully applied to itself would cause it to disappear, but that would almost be optimistic.

Not related to the Law Of Demeter.

But related to the Law Of Stubs.


I have come to the conclusion that a quantum-like psychological effect causes the Law Of Demos, especially the second strange phenomenon listed: the irreproducability of the bug that appears in front on the prospective user/customer/employer/girlfriend/other person you want to impress.

One simply acts differently when being observed by such people. It goes very deep and affects many subconscious details in the way you interact with a system that you are hopefully very familiar with. So you see bugs that you will never see again (sometimes no eventual user ever reports them).

This would imply that there are Limits To Automated Testing -- Richard Drake


There is also some correlation between the probability of an undiscovered defect surfacing and the sum of the salaries of the observers. -- Andy Morris

There are several jargon terms for these kind of bugs (Heisenbugs, schroedinbugs, etc...) that manifest what Richard is saying to be the Truth and only the Truth... -- David DeLis "fandango on core" (it's Friday)

you mean there's someone else out there that understands...?

I mean it... :-)


This sort of thing always reminds me of one of the examples in Programming Pearls. Please feel free to correct this anecdote. My recollections are somewhat hazy, having foolishly lent my copy out a few years ago...

A user complains to support that some of the keys on his/her terminal are intermittent, yet neither the user nor the support engineer can demonstrate the problem when the engineer arrives. Eventually (after a lot of head-scratching) it turns out that the problem is two transposed key caps on the keyboard. When unsupervised, the user mostly touch-types, but under demo pressure, looks at the keys. -- Frank Carver

See: The Original Folk Tale


I think it has something to do with neckties. The probability of something going wrong is proportional to the density of ties in the neighborhood of a demonstration. This is why you should never let your customer's boss peer too closely at the monitor.

Uniforms seem to spook computers, too. I once saw a demonstration crash and burn when a major general, keenly interested in what was going on, leaned forward and hit the computer's power switch with his knee. -- Tom Kreitzberg


In a recent job I did it was necessary for all of the team to do lots of demos. When things went wrong it was usually caused by one of three things:

  • The audience asked that question you were hoping no one would ask.
  • The demonstrator getting through a perfect demo, and because its going so well he/she becomes so enthusiastic they end up attempting to show off a feature they weren't planning to.
  • Not freezing the code at least 3 days in advance of the demo and actually doing a dry run on the machine that the demo will be done on.

That last one tended to be the most common mistake. -- Channing Walton

Even that last point can backfire. A number of years back a story made the rounds telling of a particularly paranoid microsoft project manager who made several dry runs on a clean machine the night before the demo. The next day, the demo failed. The dry runs had been just enough to nearly fill up the disk. -- Dave Smith


Once when working for an online company, I had to drive a demo given to some important stock analysts. The night before I installed and tested everything on the machine that was to be used for the demo. The only thing I was not able to do was test it in the room we were to give the demo in, I had to do everything in another room while various tech minions set the main room up. I left at about 2:30am finally having everything work. (Got home at 3:30 am and left at 6:30 to return) The big issue was the most of the technology being shown was from a recently acquired company, and we had to use a rival service to connect to it. There had been no time to rewrite things to work on our own system. We, of course, did not plan to show this part.

When we arrived the next morning, the presentation had started, and I discovered that the techies had not moved the computer over intact as promised. They figured that since the room had a huge projection screen, we would not need a monitor. My boss gave a very compelling little speech (thank god he was a good speaker) while I, in front of the analysts, tried to look bored while making our initial connections. As far as we know, we got away with it. And words were had with the people who decided we did not need the monitor. Was also a lesson for me about doing things myself. Next time I stay with the machine until it is truly set up, and just go without sleep....


I've got a better one...

Once, we were demonstrating our television product to a potentially big client. The program was faithfully rendering over a lot wire from upstairs to a television set at the back of the conference room. After the initial demo, our Vice President of Imagination, came in, introduced himself, and gave the Visionary Speech. At this point, a couple of us engineers wandered over to the balcony overlooking the conference room to peer in through the glass doors. You guessed it. The demo had crashed. The marketeers noticed us and tried waving us down while our vice president was giving the charismatic speech of his life to stall. We quickly rebooted the system just in time. A year later, I recall enjoying it just a little too much when I got to delete the entire codebase out of our version control system before rewriting it correctly. ;) -- Sunir Shah


Demos sabotaged by last-minute requirements

I had the misfortune of working for a start-up whose goal was to be the AOL of their city (never mind that AOL already existed). On the morning of our product's demo to the public, the hands-on, won't-take-no-for-an-answer CEO came in very excited about a new program he'd put on his home PC the previous evening. It looks fantastic, he told us; could we put it on the demo machine? It would make our demo so much better!

The program was the just-released MicroSoft Windows 95.

Sensing disaster, the lead programmer and I put up a united front. We won, and the demo ran as planned under Windows 3.1, the environment it had been written on. We were sobered by just how close we'd come to catastrophe.

See: AiKoans