On Testing

When I started working on 1.0.10 at the start of this year, one of the things I asked myself was how to improve our patching process – or how to stop horribly game-breaking patches from happening ever again. First, let us review the Dredmor patch history:

0.9 -> 1.0.0 – The Greatest Crashing Beta Test Ever of 2011.
1.0.0 – Dredmor released with mysterious audio crashing problems, exposed by a change to the audio code at the last minute.
1.0.1 – Fixed the audio crash.
1.0.2 – Fixed the audio crash which was exposed by the fix for the first audio crash.
1.0.3 – Zillions of bugfixes.
1.0.4 – A hastily released patch after 1.0.3 broke save game compatibility with 1.0.2, and the laser-like glare of the Eye of Sauron turned upon me. (And I wept.)
1.0.5 – Was reasonably boring and nothing happened.
1.0.6 – Broke achievements; was hotfixed. (Some people consider this version a “classic” Dredmor for some reason.)
1.0.7 – Humble Bundle build; added Linux support and, also, bonus crashing.
1.0.8 – Diggle Gods build. Included things like “the n-Dimensional Lathe, a giant, crashing, unexploded bomb”, things not giving people the correct stats, crashing mysterious portals, eleventy billion broken tutorials…
1.0.9 – Patch for everything that went wrong with the Diggle Gods build. Added crashing bookcases, crashing savegames, save game corruption, and stuff not loading in the mod launcher. This also marked the start of the attempts at fixing the “belt eating problem.”
1.0.9 REV B – patch for all of these things, some of which actually stayed patch.

One of the conclusions that I reached was that a lot of the really bad problems (save game compatibility; n-dimensional lathe; hideous random audio crashes) were things that showed up when we had a thoroughly tested build, made one small change to it, and then put up the build with that one small change (which, consequently, was the change that broke everything.) As a result, we decided to instigate a new policy: any time we make any change to a Dredmor release candidate, no matter how small, the patch goes out for testing and is banged upon for at least 48 hours (preferably 72) before we ship it. Dredmor 1.0.10 is the first build we’re shipping with this new policy; we hope it works out well for all parties concerned, but it does mean that deadlines are a little hit or miss.

As always, thanks for your patience! We will try to keep the forums updated on where we stand in the QA process. If you want to grab patch RC7 and try it out, OS X and Windows versions are now available in our forums.

What's this? I don't know. Look over there!

Posted in Dungeons of Dredmor, Gaslamp, Programming | Tagged , , , , ,

18 Responses to “On Testing”

  1. A solid regression testing & smoke testing process will always pay dividends in terms of build stability. Go team!

    { reply }
  2. RaustBD says:

    Oh my. That DOES look like official art for the clockwork knights items and practical geology icons!

    { reply }
  3. Althea says:

    I like those new skill icons. Is it a new skill or a mod skill?

    { reply }
  4. Daynab says:

    Amazing. I remember all this but seeing it in list form really brings back memories.

    The new policy is a good change, honestly.

    { reply }
  5. Fax Celestis says:

    I think people think of 1.0.6 as classic because that’s what they started playing the game on: a lot of people bought it during Steam’s Christmas sale. I know that’s when I started playing.

    { reply }
  6. Alejandro Moreno says:

    Last-minute, untested changes are the bane of every product imaginable, from cars, to collectible card games, and obviously, to software.

    Kudos on sticking to your new release plan! Even if I’m still waiting for RotDG on Desura 🙂

    { reply }
  7. ChocolateWaffle says:

    If that’s a werethursty tree, it looks amazing :O Also, is that icon at the bottom a hint to dagger weapons? If so, have my babies, I’m tired of make-believe using swords when I play rogue/thief/ninja

    { reply }
  8. Werediggler says:

    Those icons do look a lot like skill icons for Practical Geology with what looks to be an icon for… a dagger skill? The items seem like they would be for Clockwork Knights.

    { reply }
  9. Knallis says:

    1.0.6 <3 <3 <3

    { reply }
  10. Snargleplax says:

    No amount of panicked last-minute manual testing can replace a solid and constantly-growing suite of automated tests. Every time you fix a bug, write the test that should have been there to prevent it in the first place.

    { reply }
    • Sure, and we’re doing that for Odin where we can. But at anything other than a very basic, functional level, games are sufficiently complex and non-linear that writing a good test suite for them is impossible. That sucks, but such is life.

      { reply }
  11. Lukas Høgh says:

    This is definitely the way to go. Dredmor is gonna rock 50% more once we get a super stable, super awesome version 1.0.10! Good decision guys.

    { reply }
  12. Roundhouse Kitty says:

    The power of ~rock~.

    { reply }
  13. Legend says:

    I REALLY wanna know what those pictures at the bottom are from. Updated art for some of the mods? Are they available? Where?

    Art from an upcoming 2nd expansion?

    Artwork for some of the mods that are now/soon going to be officially added to the game now?

    { reply }
  14. yatima2975 says:

    The owls are not what they seem? What has that got to do with anything?

    (Hooray for obscure Twin Peaks references in the tags; if you ever do a retail version, please use “wrapped in plastic”!)

    { reply }
  15. Looks like a Terraformer and/or a Thrustologist to me.

    { reply }
  16. Crunchy says:

    More stuff…ME CRUNCHY LIKE!!!

    Already 409 hours in and still playing.
    I await 1.0.10 with baited fishy-soapy breath.

    { reply }
  17. Nick says:

    A new skill in honour of the Rock beating John Cena?

    { reply }

Leave a Reply

Your email address will not be published. Required fields are marked *