Gonzo Engineering

A geek manifesto

by Steven K. Roberts
May, 2004

Microship- a decade of nautical geekery

If you probe the interstices of an industry increasingly dominated by Big Business, you’ll discover a microculture of hackers motivated by the mad bliss of invention, surviving on the sweet contagion of creative energy. Employment bonuses mean nothing here; fancy packaging and market share are viewed with contempt if a product lacks art. Beauty, now that’s the thing — the beauty of elegant code, of a robust network, of a balanced design that “just works” without duct tape and feature bloat.

It is from this culture that the Open Source movement emerged. Less obviously, it’s also a diverse community of home-shop machinists, embedded system artists, guerilla solar experimenters, human-powered vehicle designers, robotics hobbyists, amateur radio satellite builders, and countless other independent developers. If you want to see passionate invention without the sloppy overhead of a big R&D budget or the weird constraints of maximizing shareholder value, go find a hacker… someone who gets a techno-boner from circumventing limitations and knows how to get things done.

This has been my world for 30 years — a world where fun is the bottom line and livings are made on the opportunistic spinoffs of creativity, not selling one’s life for a salary. We subsist in the dark matter between industries, trolling flea markets and dumpsters for Obtainium, mail-ordering goodies, making holy pilgrimages to the surplus Mecca of Silicon Valley, re-purposing the detritus of corporate America to our own obsessive ends. Scattered among us are conjurers, alchemists, wizards, lone-wolf inventors, quirky entrepreneurs, larger-than-life writers, and the origins of more than a few geek stereotypes.

In this parallel universe, the motivation for creating is highly personal. In industry, you can bet that any massive development effort is associated with a business plan — there’s no room for slack in a bottom-line world, and seldom are things done for fun. But here, you’ll find entire lifetimes given over to chasing quixotic dreams; you’ll see personal fortunes whittled down to marginal subsistence in the name of invention and reputation. Occasionally there’s an imagined pot o’ gold, to be sure, but most likely it’s just a reassuring fiction to keep the spousal unit calm in the face of demonic focus, Every Goddamn Night Out There in the Shed. No, our motives are usually as guileless as passion itself: chasing daydreams, building tools, realizing obsessions, shattering limits, publishing, earning grins of appreciation from the cognoscenti and accolades from neophytes.

These are things that touch the soul more than the bank account, and there’s definitely a conceit about it — our sense of security lies more in our toolsets than our 401-Ks. We feel sorry for vested employees with their BMWs and well-appointed houses, even as we decorate our labs with rusted hand-me-down office furniture and pay for system upgrades by mining our hardware boneyards through eBay. But money is not the point. It’s the exhilaration of surfing the knee of the learning curve, the almost erotic bliss of a machine flickering to life — catching the spark and glowing while the rest of the world sleeps.

The First Steps

You’ll never get a corporate middle manager to admit it, but such lunacy, driven by emotion and other unquantifiable wild cards of the psyche, lies at the heart of the design process. You can formalize tools and implement procedures all you like, but you can’t fit passion on a PERT chart; trying to do so will repel the people you need most.

The first step is one of the most fun: indulging in a fantasy rich enough to trigger secret grins of hard-core technolust. That’s the stuff that makes otherwise sensible engineers willing to devote years, if that’s what it takes, to getting it right.

One of the great secrets I’ve discovered is that even someone with stupendously bad work habits (like me) can get a prodigious amount accomplished by applying one simple and obvious technique: keep moving in the same direction for a long time. Unfortunately, that can lead one down the path of specialization — an essential part of the great symbiosis between those who dream and those who produce. Specialization along with its concomitant skills is obviously necessary to get real work done, but if you’re not careful it can become a filter through which you see the world, attenuating everything that is unrelated to your primary focus. Over time, this can cause severe perceptual distortion from which it can be difficult to recover (especially if said specialty ends up becoming an evolutionary dead end in a rapidly changing industry).

That’s an easy platitude for a self-proclaimed generalist to spout, but how do we resolve the problem? How do we hold on to a central design objective for a decade or more without becoming like one of those single-issue political or religious zealots who lose the broader context entirely and descend into extremism? It’s much easier to end up there than you might think, especially when you audaciously choose to chase a personal obsession rather than sell 40-hour weeks while hanging onto the remainder for sanity-preserving pursuits.

The trick is at once simple and fiendishly tricky: all it takes is caring so passionately about the project that it fills your daydreams, turns trade journals into treasure hunts, induces you to recruit your friends, inspires doodles, and overlays a sense of purpose onto every foray into the backwaters of the web. Unfortunately, this is a lot to ask of a job that’s been dumped on you by management, and one of our central messages here is that if this crazy-talk of passion gets you all fired up and chafing at the bonds of a career that isn’t letting you play enough, then maybe some restructuring is in order. For there is simply no way that crank-turning, even by a well-oiled department full of Really Smart People, is going to give you a sustained rush of intense creative obsession; doing that requires a suite of characteristics that are generally regarded as pathological in a corporate environment:

• Enough chutzpah to believe that you are doing something original and important, but the humility to steal shamelessly from the work of those who have preceded you

• Enough schmoozing ability to induce others to buy in to the dream, but the stubbornness to continue believing in your mad quest when associates have given up on you

• Enough optimistic naiveté to interpret catastrophic failures as steps along a continuous path, but the sensitivity to recognize the real gotchas (like your own change of heart) when they subtly appear

• Enough arrogance to ignore warnings and skepticism of people with far more experience, but the wisdom to shut up and listen to the advice of practitioners in an unrelated field

People who behave this way are often described as having attitude problems, difficulty working well with others, and a tendency to jump around and not finish assignments. These are not the things managers look for in employees.

What I’m trying to tell you here is that if you are one of these troublesome folks, you need to shape your environment to support your passions: nothing is more important than removing the barriers that our culture erects around creative madmen, and few companies are willing to customize a job description to allow your brain to frolic in its own juices. In severe cases, you might even need to jump ship and accept the insecurities that accompany working alone. (On the other hand, if you are in management and are trying to pull off the impossible, then you need to recognize and encourage the hackers in your midst, giving them the freedom to be profoundly annoying and unpredictable.)

All this is simply a contextual backdrop for the real point here, which is that massively audacious feats of creativity fall out of a way of thinking that is more a lifestyle than a toolset. I find myself smirking at books about management and team-building, when virtually every world-changing cusp in the fabric of technology can be at least partly attributed to the obsessive-compulsive behavior of some intense character who broke the rules, dropped out of school, irritated colleagues, jumped between careers, got in trouble, or, as the schoolbooks used to say about the inventors I tended to identify with, “died alone in poverty, an embittered man.”

It seems we keep returning to this theme: a lifestyle of dedication to a mad dream, with everything else shoved aside to make room for equipment, learning curves, relationships with gurus and assistants, testing phases, and the endless quest for support. It’s not necessarily profitable, nor is it particularly fun (in the amusing sense), but there is something blissful about having a raison d’etre, a central passion, an unwavering navigational objective that allows every instant of your life to be tagged unambiguously with Distance To Go, Cross-Track Error, Estimated Time of Arrival, and Speed Over Ground. Such clarity is illusory, but it beats floundering around every day, changing direction on a whim, and questioning your purpose even while working your butt off and looking forward to evenings, weekends, vacations, and retirement.

It’s also no guarantee of success. But even going spectacularly down the tubes feels kind of noble when it’s part of your life’s enduring quest.

Still, I keep wanting to overlay some kind of formality on this. Aren’t there a few rules we can apply that are a bit more useful than saying “just dream it,” like some incongruously successful relic of the 60s who became a crystal-sucker in the New Age fringes of Silicon Valley before stumbling into a founder’s pool during the can’t-fail dotcom boom? Like, it’s all about the fundamental vibrations of your creative energy, man…

Well, um, OK. But if this level of design is indeed a lifestyle, then the closest we can get to “formal tools” is a body of behaviors and attitudes. Let’s put on an engineering hat and attempt to consider the problem in that light.

Formal Tools, Briefly Considered

Sometimes I wish I could claim that Microship trimaran development had been a tightly managed progression in which, beginning with a vaporous initial concept, we generated increasingly refined formal specification documents, mapped everything onto a PERT chart to establish dependencies, used that to drive human resources and purchasing departments, then underwent a tightly scheduled fabrication and coding phase focused on milestones and design reviews. That’s how big companies claim to do it… and, hey, we even have some nifty project-management software that knows how to convert TO-DO lists into pretty pictures.

During the BEHEMOTH bicycle era in 1990, I spent a very interesting afternoon at Trimble Navigation, makers of the bike’s GPS. These weren’t colorful, user-friendly handhelds wrapped around cheap off-the-shelf chipsets back then; they were extremely complex DSP engines coupled with RF hybrid black magic that pushed just about every envelope in the book and cost a couple thousand for entry-level models. I remember being captivated by a massive floor-to-ceiling PERT chart, spanning an entire hallway, the completed boxes bright yellow, the web of interconnections revealing Deep Understanding of the design process and accurate predictions of every step remaining.

“I should do this for the bike,” I mused to my host. “It looks like a great tool.”

“Nah,” he replied. “Project management tools assign resources to tasks. You work alone. Just do something.”

He was right. Even with first-class volunteers and occasional contract help, Nomadic Research Labs is a tiny operation, a de facto non-profit, beset by overload and bad work habits, constantly challenged by such fundamental issues as demotivation, distraction, and lack of funds. A PERT chart in this environment would be masturbatory, and would presuppose a stable design.

Engineering in a Nutshell

What actually happened was much more organic, and I’ve noted with amusement that, despite protestations to the contrary among the engineering population, it’s typical of the way innovative developments usually work. Here’s how to manage a huge, complex project:

1. Accept going in that your first tentative decomposition of the fundamental concept will yield an over-simplified TO-DO list, distorted by misunderstanding of key issues.

2. Avoiding all the items labeled TBDWL (To Be Dealt With Later), dive into the well-defined parts, finishing some of the electronic design so early in the game that it is guaranteed to be obsolete before the physical substrate is built.

3. Blunder ahead on the non-obvious parts, getting pleasantly distracted by learning curves and occasional moments of certainty, only to discover basic flaws in your reasoning.

4. Now that you are forced to re-think the initial concept, map it onto newly recognized reality to yield a fresh TO-DO list (with new lab notebooks and computational tools to keep things lively) and another cycle of enthusiastic activity.

5. Repeat steps 3-4 at varying levels of abstraction ranging from the entire system down to individual components.

6. Meanwhile, since technology evolves with frightening rapidity, acknowledge the fact that any computer-based system is such a moving target that if it’s not completed quickly, it will be irrelevant by the time it ships.

7. Respond by simplifying, refining objectives and abandoning dead-end ideas while doggedly pursuing others that have come to represent too large a sunk economic or emotional investment to allow graceful retreat.

8. Bang out a few things that weren’t on the list, then add them and cross them off to make yourself feel good.

9. Get totally sidetracked a few times, and dive into development marathons to meet public deadlines like trade shows, pulling all-nighters in PFD mode (Procrastination Followed by Despair).

10. Announce new completion dates whenever a previous one has passed, and keep driving your PR engine to maintain interest during a process that is a textbook illustration of Hofstadter’s Law (“Everything takes longer than you expect, even when you take into account Hofstadter’s law.”)

Part of this development heuristic is just sloppy management, but it reflects the way we think. This is why the average completion time of a homebuilt boat is 137 years.

Perhaps the most interesting thing about this seemingly ugly process is that it’s iterative and self-correcting. Grandiose or stupid ideas may not be obvious during blue-sky analysis (when the project is glued together by wishful thinking), but it’s another story entirely when it all has to be converted into Clearly-Defined Tasks (CDTs) and drawings that make sense to machinists. Without some kind of closed-loop intellectual process to fine-tune your thinking, it would be impossible to get to the point where you can start using engineering tools to convert fantasies into contraptions.

Trying to shortcut this by starting on Day One with formal design methodologies can have the catastrophic effect of committing you to an ill-defined goal state, whereupon the end result is shaped by your toolkit more than the objective. That’s why so many products seem malformed, patched, and inelegant: management loves formal methods and looks askance upon such frivolous notions as approaching product design as a delicate blend of art and engineering. The exceptions are a joy to use. The rest miss the point, no matter how stylish their exterior or sophisticated their underlying technology.

So it appears that designing a system isn’t nearly as rigid a process as typical engineering textbooks would have you believe. Your component choices affect the shape of the thing you’re building; said shape in turn creates constraints that affect your choice of components. Such psychological race conditions can only be resolved by tweaking the granularity knob while adding inputs to your evolving mental model, until the correct solution congeals in a flash.

It’s easy, and here’s how to do it: Prop your feet up on your desk, relax, and form a fantasy of the desired results. Now turn it slowly in your head while calmly examining it from all sides, allowing input variables to float until an unanticipated combination satisfies your psychic fantasy-comparator and generates a flash of recognition. Since all your noodling is saved in a big circular buffer called short-term memory, let this recognition event pre-trigger a snapshot of the conditions that immediately preceded it (before accumulated pondering-propagation delays introduce conceptual drift). There’s your design specification. Take that and run with it!

This is probably not an engineering methodology that makes managers comfortable, though it’s a good summary of life in the trenches. There is a pervasive myth that structured methods and sequential procedures, used in isolation, will get you there… but I’ve never seen it work that way. Those tools don’t actually start to become useful until you’re quite thoroughly immersed, and that can take weeks of appearing, to outside observers, as if you are loafing.

Economics

Finally, let’s talk about money. From an engineering perspective, this can be even more annoying than time — there’s nothing like “aggressive cost minimization” to take all the fun out of a design. Fortunately, one of the intrinsic features of passionate dream-chasing is that everything else is secondary, and it’s thus easy to justify spending as much as you have (and then some). Combine this with poverty consciousness, and one can get amazingly creative at scrounging. In addition to all the expensive bits, our crazy research vessel contains thousands of bits that were donated, bought surplus, extracted from dumpsters, horse-traded, repurposed, cannibalized, or fabricated on the cheap. But one issue that never came up was worrying about manufacturability and component cost. There’s a sort of certainty here that is immensely liberating:

“This is the most important thing I can possibly be doing, so it doesn’t really matter what it costs to get the job done. I’ll afford it somehow.”

---

Art without engineering is dreaming;
Engineering without art is calculating.
— Steven K. Roberts