In this post I will discuss video design workflows and the strategies I’ve found that work well with QLab (and most other show control software).

I’ll start with two very important axioms:

1. There is no Magic Bullet;
2. There is no Easy Way.

Whenever we create our original artwork, or just throw together a quick edit of some stock footage, we’re always looking for a way to keep moving in tech rehearsal. We don’t want to slow down the director’s (and our own) creative flow. We want collaboration to move at the speed of ideas, and our stupid computers keep getting in the way of that. Yup.

In those moments, I like to remind myself that I’m not working with film; that when I think of a “layer,” I don’t mean literally stacking negatives on top of each other, and hoping it comes out right in the water wash. That is to say, sure, encoding time and the classic, “Give me a half hour while that clip renders,” are frustrating. But sometimes it helps to remember that we’re working in a field that’s still only gotten as far as its toddler first-steps. Even with the frustration of waiting for a render, it’s still better than saying, “I’ll work that out tonight, and we can look at it in the morning.” And losing a night’s sleep to dark room chemical fumes.

It’s in those moments, when all I have to hold onto is how much less frustrating it is now that how it was a few short years ago, that I wish there were an Easy Way. One trick, that when executed correctly would mean speedy, efficient collaboration with no one staring at my tech table, wishing I could catch up. I want the Magic Bullet — that one codec that works in every situation and solves all of my jerky playback problems, and means I can bundle one show and send a disk on tour, regardless of what computer the show will see at the next venue. It’s also in those moments that I remind myself of the two above-mentioned axioms. And I steel myself for the groans, and I ask for some time. “Maybe we can come back to this in an hour?” Again.

So, how do we work around those two rules? How do we find an easier way? A practical bullet? Therein can be found the scope of this discussion.

I’ll approach this from a couple of angles, and keep in mind that all of my information is based on research and experience – that is to say, your mileage may vary. And, to extend the driving metaphor, the first challenge I’d like to wrestle with is bottlenecks. I’ll get to codecs in another post.

The Bottlenecks

When working with video for live playback from a hard disk (narrowing the scope to QLab, as opposed to DVD’s, internet video, or other delivery methods), we need to think about our hardware, and what we can do with our files to cause as few traffic jams as possible. This starts with the Hard Disk.

How fast is your hard drive? What’s the disk’s connection’s bandwidth? What other traffic is holding back your video file’s data path? In a perfect world, every computer your show will see will have a Solid State drive, a SATA connection, and every video file will be accessed independently of any other data at the moment of playback. In a practical world, we often have to satisfy ourselves with a 7,200 rpm “platter” hard disk drive, FireWire 400, and a couple of files being called for simultaneously. Hopefully, we do our best to ensure that every system we use at least fits those minimum requirements. Because here’s the thing: video files are huge. Like, imposingly large files. And getting them moving efficiently is the key to good playback.

The key here is that hard disk drives have only one point of data access at a time. Most of the data gets stored in entirely different physical locations, from one file to another, and the drive needs to rotate to that point, let the reading head grab those ones and zeros, and rotate to another point to grab the next chunk of data. If the operating system, and the QLab program, and the video files are all stored on the same disk, then there are loads of reading and writing processes being asked for simultaneously, and one poor little hard disk just can’t keep up, no matter how fast it can spin. So, step one: move your media files to a separate physical drive from your system. Two data paths means less bottlenecking.

Here’s a weird concept: video files are so huge that it’s probably more efficient to run the system off of an external drive, and store the media files on an internal. I would never setup a system like that myself, because a bumped cable would mean catastrophic failure. But it’s an interesting idea. FireWire 400 transfers data at a nominal rate of 400 million bits per second, or 400 Mbps. An internal eSATA connection, like on your MacBook Pro, transfers at a nominal 1.5 Gbps, or more than 3 times faster. The system and QLab are doing some reading and writing, but not nearly as much reading as a single video file. So, prioritizing your data stream to video is a nice idea. In practice, this means storing your media files on an external FW 400 or 800 drive, which is usually sufficient. Of course, if you’re trying to access four videos at the same time, you’ll probably benefit from putting those files on more than one disk. Keep in mind, if your FW 400 and FW 800 connection are on the same internal buss, or if you’re daisy-chaining drives, you’re only gaining fluidity in access, not transfer. The files can be read independently, but they’ll still need to get that data through the same path, which means hitting the same bottleneck. In practice, the bottleneck of the buss is less important than the gain in access speed from having multiple sources.

OK, we’ve covered the hard drive, and its associated bottlenecks: speed, the access conundrum, and the data path. We’re ready to prioritize appropriately and manage our traffic accordingly. What’s next?

The other relevant hardware all comes in one big jumble, oddly enough. The CPU, GPU, RAM and vRAM all work with each other, in several confusing ways, to get your imagery from the hard drive to the display. The basic data path, after the hard drive, goes something like this: RAM > CPU > (v)RAM > GPU > Display. I’ll start with that questionable looking fellow there, the (v)RAM.

If you’ve created one cue in QLab, referencing one video file, and you’re sending it to one display patch, your video frames travel straight from the decoding process, to the vRAM, to the display. If your cue is told to show up on more than one display patch, that breaks the data flow a little bit, and the frames have to be written to the RAM, and then distributed to the appropriate vRAM – each graphics card has its own vRAM, so if a cue only needs one display, it only needs one card, so it can go straight to vRAM. If it needs multiple displays, it needs to hit the distribution center, which slows down the process. What’s the trick here? If you want to send one file to multiple displays, you’ll benefit from creating one cue per display. Simple enough.

Now, if you’re half the geek I am, you’re wondering why poor old GPU is stuck at the end of the data path. Why isn’t the GPU, which is made for such things, tasked with decoding video files? Well, Apple only allows H.264 files to be decoded on the GPU, and even that is limited to OSX 10.6, for our purposes. Similar constraints lend to QLab being stuck as a 32-bit application for the time being. Since QLab wants to be accessible to as many of you designers as possible, it has to embrace some somewhat outdated technologies, and wait for Apple to give us the tools we need to embrace the newer, more powerful stuff. One thing you can be sure of — when we have access, we’ll be embracing it! For now, there is no hardware-accelerated playback for movie files in QLab, and it’s relegated to sharing only 4GB of RAM with all of the other 32-bit applications on the system.

“Only 4GB of RAM? But my video files are much larger than 4GB!” Fortunately, there’s a point where QuickTime, OSX’s playback engine, has prepared a nice road for us. We’re not entirely sure of the specifics, since it’s proprietary information for Apple to keep locked in a bunker, but we are sure of a few things. When QLab calls for a file to be loaded into RAM, the whole video file doesn’t get dumped in one big chunk. It’s being accessed as necessary, and as RAM becomes available. QuickTime and the OS manage that for us, and they do a pretty good job of it. There’s one point where your bottlenecks are handled for you. Also, your files have to be stored somewhere while they’re being decoded, and that location is the RAM. If they don’t have to come back to the RAM (as described above), they’ll move from there to the display pretty smoothly.

So what’s left? CPU and GPU. The CPU’s role is to decode the files you’ve made. The GPU’s role is to make your adjustments, and send the frames to the display. So, here’s where your choice of codec can be incredibly important, and where your programming can create a challenge for your system.

We’ll get into codecs in a future post, but for now, keep in mind that the more you’ve compressed a video file, the more your CPU will have to work to decompress it, and that can create a bottleneck. The less you compress it, the lighter the load on the CPU, and the smoother it will flow through this part of our data path.

The GPU then gets frames from vRAM, after they’ve been decoded and put there for holding. At this point, if you’ve made scaling adjustments, rotated your image, changed its opacity, or moved it around, the GPU has to figure out how that affects the frame that it’s sending the display, and then send it out. It’s at this point that I often create an unnecessary bottleneck for myself. And I’m going to suggest a workflow that includes sufficient tech time here, because I think QLab is actually two different tools at this stage.

I try to create my artwork at a scale that will work in the space, but obviously, what I’m seeing in my imagination, before projecting material on a set (for instance), will not exactly match my needs once in the theater. So, I love Custom Geometry in QLab, because it lets me rescale and rotate and position my artwork to fit my needs. But if I’m scaling my artwork up by a factor of 1.15, that means that at playback, poor old GPU is having to make that calculation to every frame it sends to the display. And that’s a more difficult calculation than a factor of 2 (computers really like 2′s). In tech rehearsal, I tell the director that we may see some glitches in playback for this reason, and once I’ve corrected my artwork in QLab, I’ll re-render it overnight to match the scaling adjustments I’ve made, and tomorrow, we’ll see it play back smoothly. Basically, the strategy involves being prepared for imperfection in rehearsal, and pre-rendering the adjustments I’ve made, once I’m satisfied with what I have. QLab in rehearsal is another tool in the composition process. QLab at playback should have all of its media already optimized as much as possible. And then, the GPU only has to position and fade the artwork, which it’s very comfortable with doing in real-time.

All of these strategies and workflow adjustments mean we’re looking at the equipment we have to work with, and creating media files that are optimized for the system at hand, not for the ideal system. We’re also doing everything we can to optimize our system, with the understanding that our budget and time constraints will hamper perfection in the equipment we’re able to use. So, we’re preparing as much as possible off-site, and planning at every stage for the tools we’ll have to use once we walk away from the show and hand it off to the stage manager and operator.

Understanding the hardware perspective on all of this can certainly help to create a system that will bring us closer to playback perfection. Next, how to prepare the files with the same goal. Stay tuned for Magic Bullet Part II: The Codec.

I think you’ll agree: he’s doing some amazing work on these.

Sound designer and composer Jason Knox is working on a series of video tutorials for QLab 2. (See our artist’s rendition below, simulating what experts believe to be a true-to-life simulation of a Jason Knox in his natural tutorial habitat.)

Each tutorial video will have a corresponding example workspace, so you’ll be able to watch and tinker at the same time. I’m really excited about this. We’ll be rolling out the videos as they’re produced in the coming weeks.

But wait! There’s more! We want your ideas in the tutorials too. In particular, we’d like to create a whole series of “tips and tricks”. So start brainstorming, and if you’ve got a video camera, start practicing your delivery. We want to see you teaching the community what they can do with this thing.

To whet your appetite, Jason has just finished an awesome QLab 2 teaser video. Go check it out over on the Tour page.

We’re building a running list of places, people, and shows that have used QLab. Check it out here:

QLab Wiki: QLab in Action

On the frustrating “QLab crashes on 10.3.9″ front, the news is that I have narrowed in on what I think may be a problem with CoreAudio. I have submitted sample code to the engineers at Apple to allow them to recreate the problem I’m seeing, and I am waiting to hear back. Hopefully I’ll be able to resolve the issue one way or another within the next week.

Major changes have been made to the Fade Cues, as well as some revisions to the interface. The changes to the Fade Cues require you to recreate fades in old workspaces. In the future I’ll make sure that new releases are backwards-compatible, but I don’t want a ton of cruft in the code to make special checks for developmental versions of workspace documents.

The documentation is now fairly out of date, but I’m not going to devote *too* many cycles to it until the interface is a bit more stable. It’s of little use rewriting it with every release, methinks.

To build these docs I’ve set up a handy makefile and a couple of shell scripts, so I only have to maintain one set of source files for both the web-based documentation and the application-based documentation. The content just gets wrapped with the appropriate header and footer, and away it goes to its proper spot. Saves a good bit of hassle.

Now I believe it’s time to finally get back into the code and take care of some of those bugs.