Thursday, February 21, 2008

Quality sucks

Since I started with software development years ago I was taught that quality matters the most. All those software development processes focused on achieving the best quality for the least of bucks. You were taught that everything you do should be designed, tested, and so on. I believed in that. I believed that that's the best way to do. I believed that to be successful software developer I should create good code. Code that shines. With good design, tests, etc. Then after few years of me trying to be better and better I somehow managed to read a few articles that changed my way of thinking - the worse is better, there's no such thing as perfect solution and you should stop thinking about it, the good is enough - if it works that's great, don't optimize it's not worth it (before you really, really are forced to). When I thought about it then I realized that's true, you can't push yourself too far into quality because you'll lose.

But now I think I realize that still I'm focusing too much on quality ;-) When reading all those stories from entrepreneurs or talking with guys that succeeded you can see that in the early days their mindset was "do something, let it work, push it through the doors, then gather feedback and improve". Don't waste time on beautifying your code, don't waste time on sophisticated design. If you want to succeed you need to have something working (with bugs, bad design, even crashing from time to time) - here's the selling point - it should do something useful for customers and ease their life. If it brakes from time to time, or isn't so fast, or has some other problems - as long as shortcomings don't overshadow improvements your software brings you're still in the zone, you still have customers and you still have time to improve. Just listen to people - you'll notice how they react, if they have any problems react - fast and efficient.

OK, you'll argue that this way you end up with some big, badly designed application that will be hard to maintain. You can also argue that doing everything good way from the beginning is cheaper than fixing it later. Sure, it's is cheaper but no one will give you money for something that's beautifully designed and implemented as long as it doesn't bring value to the customer. The thing is that I start to think now that I still put too much into the quality. I have this habit of refactoring things until I'm satisfied. I see bad design and can't stand it. I have an urge felling to fix it. That's a bad mindset. I'm trying to get rid of it and hopefully I'll succeed.

All those 'quality people' ask you "how much do you want to spend on quality?", "do you want to pay less or more to achieve quality?", they tell that getting things right for the first time is the best option you have.

I'm asking you "is it really?", "do you really care about quality?", "do you really give a sh*t how much do you pay for quality as long as you have money to spend on it (and you have still a lot money for other things)?" ;-) Of course you do care, but I think you care a lot more about how much money you can get and when. And in case of a startup (either a company or a new product) you want to get as much money as fast as you can in the shortest time you can - because your whole life (OK, life of your company) depends on this. Am I right?

And to be honest every software sucks, every software is a nightmare to maintain after a few years and a few generations of developers. Even if you focus on quality someone will have troubles working on it. You can't avoid this, sorry. So knowing this do you want to care about it so much? Just focus on getting money and you'll be fine even if your code sucks :-)

Am I wrong?


  1. I would partially agree.

    Good quality does not bring money, but bad quality can take it away from you.
    Also having reasonably good quality from the beginning just makes your life easier:
    - If your code has good design, it is easier to add this one more single feature that will make your core customer happy
    - If you have test framework ready (not necessarily 100% code coverage upfront), you can reproduce and fix that annoying show-stopper bug fast.
    - If you have the test above automated and run from now on during every build, you have big chance of not breaking your next release (regression), and pissing off your core customer.

  2. Bad and good quality cost - that's the point I'm talking about. I think it's better to delay some efforts to shorter time when you break even.

    The idea is how much of an effort to put into quality and does it really pay off? Should you go really low on quality?

    What does matter more earning the money or having a great code? Do you think that by having better quality you can break even faster?

  3. Maybe the key to success is not TDD (Test Driven Development) or BDD (Behavioral Driven Development) but CSDD (Common Sense Driven Development). Do design your product (in your brain, on the piece of paper, etc.), do think about future improvements, do keep in mind performance, scallability, reliability, etc., etc. But don't exaggerate - focus on the output not the quality. Quality is what should be inherently in your code - but it should not be the goal itself (like famous quality measurement in % e.g.: the quality this month was 94%). There should be some golden mean and you know where it is because of your experience.

    I could write and write on this subject but I don't want to bore anyone. Last comment is that I totally agree that if something works and meets requirements is OK - my favorite example is Apache Tomcat. It's the most popular and industry-adopted Java web server instead of it's smelly code and security holes.

  4. Sounds like you've been equating quality with perfection. Any attempt at perfection in anything non-trivial is doomed to fail (to err is to be human after all), so it's unsurprising to hear you're disillusioned with quality.

    But quality isn't perfection - it's satisfying the needs of your customers as well as the needs of your business.

    Everyone gets the satisfying customer needs part, as you demonstrate when you say "it should do something useful for customers and ease their life", but you still need to test your software to know that any "shortcomings don't overshadow improvements your software brings".

    Not everyone gets the satisfying business needs part. In your example immediate revenue is a business need, therefore releasing your product ASAP by spending as little time as possible on design and writing good code doesn't demonstrate a lack of quality, but rather the opposite! However if immediate revenue isn't a business need (say you've got a business loan, sold equity to investors, inherited a fortune) then this approach may not satisfy a need and if so won't provide any quality. If this appraoch causes other needs to go unsatisfied then it will reduce the level of quality.

    Does effort spent on design, ensuring "good" code and adequate testing produce quality? It depends on your needs. If containing maintenance and support costs is a need of your business and these techniques reduce those costs, then undertaking them produces quality and you should care about doing them (even if your software inevitably becomes a nightmare to maintain as you suggest.)

  5. Hello :) In parallel, not knowing your blog post, in the same day I wrote something on the same subject, but from PM's point of view:

    Warm Regards!

  6. In my view, this is all conceptually pretty simple: 1. quality has a measurable cost, 2. lack of quality has also a measurable cost, which may or may not be greater than cost #1. It is a business decision, not a technical decision how much you are willing to spend on quality. So, if you are a project manager, what you do is you ask your sponsor or customer this very question: "how much are you willing to pay for the software to not suck".

    Now, as Slawek said in the comment above, what separates projects that are successful technical-wise (not necessarily business-wise as these measurements are often totally orthogonal) from the ones that are not, are the tools, frameworks and processes that are at use by the development team, that can bring more quality for the same amount of money, compared to lesser projects.

    An analogy would be manufacturing a car in a Toyota factory (excellent tools and processes, lots of automation, excellent skills) and manufacturing it in my garage (no tools, no processes, no skills) or at Fiat (pretty much the same thing as my garage). The reason why Toyotas are generally higher quality compared to equally priced Fiats is exactly this: their technological processes can give you more quality for a given amount of money.

  7. Blah, blah, blah.

    You still don't get it. You still focus on the "quality" part of this post. But my intension was to show all of you that if you want to create something new - some product, service, etc. You should stop worrying about good quality (whatever it is) at the beginning at all. Don't waste time on (even) thinking about it. Just do the damn thing.

    And what's hard to do - we are were taught to think about creating good code. It's hard for me and I think you too to create a code that smells :-)

  8. I do get what you mean - what you are referring to is commonly known as a "business plan". Which is just a fancy-schmancy name for "define what it is that you want to achieve and why". Depending on circumstances, business plan will or will not include a desired quality metric. Which is a fancy-schmancy name for "decide if you accept the fact that it will probably be crap". If your upfront investment is not big, this is most probably the right way to go about this. A certain student-founded startup comes to mind here :). But if you are going to invest serious monies before you see results, I bet your ass you want to think hard about how much crap you are ready to deal with just because you decided to skip on quality.

    My point here - you have to make a conscious decision about this.

  9. Marcin, I agree with you, but in fact there are two decisions to take:

    1. Whether I want to have quality manageable in my project
    If Yes, then you need to have at least informal quality management system. Then you may take the second decision:
    2. What level of quality is good enough for my project?

    If you say "No" to first decision, you won't able to say how much quality you have until customers tell it to you when you release you product.

  10. @dlx:

    ad 1. I will risk this statement: most "quality management systems" are only a little more than a bunch of hot air. They are only as good as the test suites you are using. And what you usually end up having is a fake framework that shows you 99.9% test coverage, but this number is meaningless because your tests don't test the real problems and only exist in order to show only positive results. This is because your developers are lazy :)

  11. "This is because your developers are lazy :)" - from now on you're the one writing code and we are watching and "managing" :-P

  12. >>> And what you usually end up having is a fake framework that shows you 99.9% test coverage, but this number is meaningless because your tests don't test the real problems and only exist in order to show only positive results.

    If you set a ridiculous goal, say, 100% of tests coverage, then you'll get 99.9% coverage giving you no information, that's right.
    If you set a good goal, say, report weekly and automatically a coverage level to management, then it's a chance you'll get a real information. Because there is a thing much more important than % of coverage: ability to inspect what parts of your product (in terms of requirements, files, modules) are better and worse tested.
    No, developers are not lazy. They just intuitively, and correctly, maximize the work not done. So if they know data won't be useful, or, even worse, may be used against them, then they provide just a proper, politically correct data.
    This is what managers teach them all the time ;)

  13. I would introduce 2 metrics that would give much better information about test coverage:

    1. Average amount of time required to reproduce a customer reported bug in automated test environment - testframeworkness
    2. Average number of bugs found after adding 100 new test cases to existing set - marginal bugfindness

    There probably is some correlation between those two and coverage percentage, but it's just my gut feeling.

    During development, #1 tells you to stop writing tests when you feel you have a test for each major execution path. You will write more when a concrete need arises.

    #2 tells you to stop writing tests when hey stop bringing any value.

  14. @pawel: I know that developers are lazy because I used to be one :). B ut by "lazy" I mean here "can't be bothered to take this bureaucratic shit seriously". And don't get me wrong - it is a good thing to be lazy about this. Take it from me

    @dix: sorry pal, no amount of processes, procedures, frameworks and reporting will substitute having a good team of developers that know their shit inside out and are motivated to deliver software that they can be proud of.

    @slawek: remember - there is no silver bullet. Frameworks (testing or otherwise) will only help you so much. They will not take hard work from you. Sometimes there will be bugs, they will be hard to find and hard to fix. It is just nature

  15. Two things: because fixing bugs looking at code directly is cheaper than fixing and finding bugs from user bug reports, so robust code is overall cheaper, and because maintenance, even when negligible in costs, drive developers away from writing code for the next release.