hacker journalist

This is more-or-less a script for an un-un-conference talk I’ll be giving shortly at ONA09.

“Imagine a news organization with only writers, and no editors. They might manage to crank out some successful stories, but without editorial controls, the failure rate would be astronomical.”
– Me, a couple of months ago.

Why we do this

You don’t adopt processes because they’re fun, you adopt them because they have special ass-saving properties. Doing it the right way may seem heavy, micro-manage-y, but when the process sings, the unbearable weight of uncertainty is lifted from your shoulders. This is freedom through tyrrany.

A seasoned developer won’t find much of what follows particularly interesting. This is elementary, but it’s stuff that seemed worth talking about…

A few baseline requirements for anyone making and releasing software

Version control

Version control software is both a safety net and a collaboration tool. It’s a place, off your machine, to keep your code, and when you update the code, it keeps your previous version(s). So, even on a one-person project, it’s essential. When your hard drive crashes, you don’t lose your work. And, when you’re working with others on a single codebase, version control gives you a central repository to coordinate everyone’s changes.

Task tracking

Task tracking is not about micromanagement (or, at least it doesn’t have to be). You’ve gotta be able to see the tasks on the docket so that you can know how deep in the weeds you really are. Also, forgetting to do something is really embarassing. You can track tasks in a spreadsheet, but that’s not very visible to the team. Instead, go low-tech — 3×5 cards pinned to the wall — or high-tech — with one of many software packages designed for the purpose.

Defect tracking

When you find a problem: log a defect. Take a screenshot, and give sufficient details to reproduce the problem. Defects are your unplanned tasks, they must be addressed — either by fixing them, or choosing to let them slide as a known defect, which is totally okay. Unknown defects, on the other hand, are the devil. Always, always record your defects, even if the very next thing you’re going to do is fix it. You *will* be distracted. You *will* forget. Defects are pickier than tasks, and are best tracked with software.

Staging environment

Like defect tracking, having a staging environment is about reducing uncertainty. It’s an environment running in parallel to production, set up as identical as you can make it to the production system. (If you’re using Amazon EC2, this is pretty much as simple as copying your production instance!) Your goal is this: knowing that, if your application works in staging, it will work in production. You can run load and performance testing against your staging environment, test your deployment scripts, and, as a bonus, it makes for a nice place to demo your work before it’s finished.

Push-button deployment

When everything is running smoothly, a multi-step deployment process (grab the latest code, ftp it all to the server, restart apache, etc.) doesn’t seem like so much of a hassle. But when the shit hits the fan, folks are freaking out, and you’ve gotta redeploy, half-drunk on a friday night, you’ll screw it up. You’ll forget something, and your minor bug will become a total clusterfuck. But if you’ve got push-button deployment, you can’t miss. If you’ve got a identical staging environment, you’re even better off, because you can develop your deployment script for staging, use it a few dozen times, and then when it’s time to roll to production, you know it’ll work.

The tools we use

• Python — Our language of choice
• Textmate — The best text editor for OSX
• pip — A better Python package installer
• virtualenv — “a tool to create isolated Python environments”
• Django — The secret sauce: a supercharged web development framework
• Fabric — Push-button deployment framework
  • Deploying Django with Fabric
  • Tools of the Modern Python Hacker: Virtualenv, Fabric and Pip
  • A recent fabfile we wrote — it’s not perfect, but it might be interesting to read
• memcached — The caching framework that makes our shit fast
• PostgreSQL — Our database of choice, primarily because of what comes next…
  • get it pre-built for OS X
• PostGIS — Gives PosegreSQL geo-magic
  • get it pre-built for OS X
• QGIS — For when you just wanna throw something on a map
  • get it pre-built for OS X — don’t forget the other frameworks!
• Amazon EC2 — Virtual servers hosted in the cloud and cheaper than my iPhone
• Unfuddle — Hosted version control, plus task and defect tracking
• Git — Our version control system of choice
  • Subversion is perfectly good too — check chapters 1 and 2 of the svn book to learn the basics.

Further reading (please, read further!):

• The craft of making software
  • Paul Graham
  • The Pragmatic Programmer

• The business of making software
  • Getting Real

• Usability
  • Jacob Nielsen
  • Usability.gov (don’t miss the guidelines!)

• Design
  • A List Apart
  • Signal vs. Noise
  • Edward Tufte

• Agile and iterative software development
  • Agile software development
  • Iterative and incremental development
  • The agile process we followed while making News Mixer