For years I was a fan of bug/issue management systems and worked with open systems like Jira or Bugzilla. I’ve used these systems during the development and production/ support phase of the products. Every time I use these systems I spend too much time registering, evaluating and sorting issues. At the end of the project I always get stuck with a dozen of unspecified issues with a vague status. Why is this? Bug tracking systems are not bad. The entire process of registering and tracking bugs is wrong.
What the most effective thing to do when you discover a bug? Registered the bug in a system and trace it? Does this solve the bug? It does’nt!!
You should be busy resolving the bug, not administrating and tacking it!
Why should you register bugs during a project? Why not fix them. The bug is discovered and immediately handed over to the developer who is able to fix it quickly and pass it over to the tester. This is of course easier in a co-located agile team than in a team with a formal testing department, handovers and sign-offs. This issue resolution process can take place in a matter of minutes, discussion about the nature and resolution options can be done directly. Complete the bug-cycle without any form of registration. With registration this process would take more time and discussion. Nobody looks at the list of resolved issues. When the issue is resolved the issue information is of no value at all.
Why do we register bugs?
- To prove you have tested the application? No! Use the test scripts and test reports for this. When you have to prove testing by handing over the list of resolved issues you have serious trust issues with your customer.
- To prove quality of the application? No! The number of issues does not prove anything. Are these all issues? Are all issues found? Or is there just one blocking issue that is not discovered yet?
- To prove you are productive? No! issue resolution is not an indicator of productivity. It is an indicator of being busy. Valuable functionality is the measure of productivity.
- To remember the issue for later resolution? No! When the issue resolution can wait is not important and you do not have to register it. This is not an issue but a feature request.
- To conserve a resolution for future usage? No! Log the resolution in a user manual or in the (technical) documentation.
- To create release notes? No! Release notes should contain a list of new or changed functionality and instructions for usage. Not a list of resolved issues. The readers of the release notes are not interested in specific bugs. They want to know if the new functionality is included. They expect issues from the previous release to be resolved.
Still no reason for registering bugs. .
An issue management system is a perfect system for hiding and losing your issues. The bugs are “registered in the system” and with the right account, access rights, filters and select criteria the issue manager is able to generate a list of the current issues. But you are not sure about the actual number of issues because the issues are “linked: resolved-in, duplicated or contain sub-issues.” The systems surely produce absolute numbers but are these really current and important issues? Issues change during the storage in the issue management system. During storage of issues in the system the issues become less important, less clear or irrelevant due to external factors (other solutions, changed applications, new suppliers or lost customers).
Bug registration is actually blocking the full value of your product.
The issue manager with a bug management system will frequently review and evaluate all bugs and assess their priority.
Firstly, the time spend on evaluating bugs is not spend on resolving them or on creating valuable functionality in your product.
Secondly, each bug is representing a concrete value. When the bug is resolved the feature / functionality with this issue becomes more valuable. When you do not resolve this bug you are holding back some (or a lot) of the value on your application. Not all bugs are immediately invaluable. But their “resolution-value” diminishes in time. Users will find a work around or just chose to use another application.
Thirdly, the resolution time of an issue/bug grows exponentially over time. When the bug is discovered in code created some hours ago the developer has still the code in mind and can resolve this quickly. When the resolution starts weeks after discovery the developer is thinking about another piece of code or (even better for him) is on vacation. Resolution will take a lot longer.
When you discover an issue, just fix it and do not spend time on mindless registration. When you need a reminder use a bright colored post-it memo and stick it on the concerning feature (or preferably on the forehead of the concerning developer).
Creating working software is fun, issue administration isn’t.
So Fix bugs .. do not register them.. !
On larger SW projects the testers are often organized in a separate team. Reasons are:
· Test bed can consist of expensive HW equipment, testing tools sometimes require expensive licenses. It would be too expensive to make all that equipment available to each (or most) of the developers. So, the test bed is centralised and used by several projects.
· Skills needed for setting the test bed and using test tools can differ from the skills that an average developer has. Therefore you are better off with a skilled testing team.
· Developers tend to be subjective towards their own code. It is good to get an independent view from a tester.
· Developers are usually familiar only with their part of SW and sometimes the parts that directly interact with it. So they can write unit tests for their code and sometimes even make some integration tests, but that’s it. You also need complete integration tests (especially if you have a system which involves several servers, clients etc.) , performance tests, reliability tests and long-run tests. Developers usually have no time, knowledge and resources to execute this kind of tests.
Testing is usually done on a release e.g. on a weekly or monthly basis (or when a project manager decides to start a testing cycle). I am not refering to usual unit tests that are done regularly by the developers. To broaden this view, customers can also be viewed as testers and they need to have a way to register the issues they found.
Having the above in mind, I would like to comment your views on issue management system:
· »To prove you have tested the application?« Yes, partially – use the bug tracking system to automatically create a part of test report (e.g. create a list of solved and unresolved issues). This would inform all involved parties about issues that were tested and what are the results. It is not only the tester and the developer that need to know what is going on, as you suggested.
· »To remember the issue for later resolution?« Yes. It can happen that you (as a developer) get several issue requests in a single day and you simply can not deal with all of them at once. And again, it is not only the tester and a developer that need to know about an issue. Also the customers can see that some issue is already known (and maybe planned for a solution) which can significantly diminish the time they spend about the issue (see the example below).
· »No, when the issue resolution can wait is not important and you do not have to register it. This is not an issue but a feature request.« I think it is Ok to use the same system for bugs AND for feature requests. Somebody else (e.g. a customer, a product manager etc.) can also decide how important an issue really is and if it should be marked as a bug or a feature request. So, as a tester, you should also register issues that you think have lower priority.
· »To create release notes? No, release notes should contain a list of new or changed functionality and instructions for usage. Not a list of resolved issues. The readers of the release notes are not interested in specific bugs. They want to know if the new functionality is included. They expect issues from the previous release to be resolved.« Well, sometimes the readers need to know which bugs were solved. How can you expect that all issues are solved with a new release? E.g. we had an issue with a piece of Java code. At the beginning it was not clear if the problem is in our code or some 3rd party library we used or in JVM itself. Pretty soon we found that issue described in Java SE issue tracking system and stopped working on the issue. In one of the next releases the issue was solved and we could see that in the release notes. In fact, I think that most of the popular libraries, frameworks and platforms have a lot of known issues and in lots of cases they are not solved in the next release.
· »The resolution of the bug will also include a unit test . This test will fail when the problem occurs the second time.« This is true only for the bugs that can be tested with a unit test. That is not always the case.
To conclude: I think it is better to fix without tracking all the bugs that are found by the developers (or by the testers that are closely connected to the development team), especially if the functionality has not yet been released to customers. In other cases it is better to track also. Without an issue management system all you have is e-mail, excel sheets or post-it memos. Which, in my experience, don’t work for larger projects.
Â
#Remark: Â Currently it is not possible to comment anonymous. You have to register and login to the blog to enter your comments. (on the top of the left menu). This bug is logged in our jira system #fail.
You’ve hit the problem head-on. Why do you want to know what has been done with a bug? I should be resolved. Not  managed and traced. The resolution of the bug will also include a unit test . This test will fail when the problem occurs the second time. When you want to preserve what was done to resolve the issue, use a wiki or a technical FAQ list. This is more effective than searching in an issue management system. Excessive administration will also lead to chaos.
“I have read it, and my opinion I fully disagree with you. How will you ever be able trace back what has been done with a bug. What will you do when in a later stage it turns out that a bug which was supposed to be solved, is still in the software? How will you trace back what had been done to fix the problem the first time. How can you prevent that the second time that a programmer is trying to fix the problem, he is not making the same mistake as the first one if you have not registered anything? Chaos leads to chaos is my opinion…. “
Amici, I completely disagree regarding this blogpost regarding the points of not registring / tracking any bugs anymore. What I read here in this post that you have a problem regarding that bug tracking is not “efficiently” done, but I don’t read any reasons why you shouldn’t.
I my mind bug tracking, also in the fast lifecycle of agile projects, can add to the quality of work due to that you also “measure” (!) those issues via a system. The reasons you mention for complete dismissal are reasons that are based on the fact that the WAY we do it  isn’t efficient. So optimize that.
Complete dismisal doesn’t measure, give you the data about why bugs always happen on the same topic and maybe indicate a member in your team with the need for some extra training and/or  support in certain area’s. And by the way “creating functionality” that everytime has bugs is still a “non functionality” being created. A bug tracking system gives you insight in processes in your team and your development live cycle. Bugs should be closed with solutions in whatever setting so people remember how it was solved and how it maybe effect other erea’s of the product and or product livecycle.
Also it is a way of communicating with a customer, a developer, a project member about how this should work, why it doesn’t work, what is actually expected in the realworld. I am a human and I forget things. A bug tracking system is can be used for the single point of truth regarding what was agreed.
Of course there are other means of doing some of this stuff, but as said, you have only written down IMHO why a bug registration is inefficient, not agile enough, not fast enough, not good enough for communication. So why not just improve the software instead of completely dismissing the reasons people created a bug tracking system/process software in the first place.
Administrator on whatever level have also to keep the system/software alive after it has been handed over. A bug tracking system is sometimes invaluable regarding the information how things were done the way it was done.
The Oracle database has a “wait interface”, extra code which keeps track in the Oracle database how long things need to execute and finish. Someone from Oracle was once asked, “Why was it implemented in the first place, doesn’t it make the database slower all this extra book keeping ? This time and processing could have been used for the actual code to finish”. His reply was along the lines of “The database is nowadays a factor XX faster because it helped us to measure, identify and improve what we  needed to improve”.
If you don’t measure, you will never know if your agile approach is better than the “other” way of doing it.
I say. Automate, improve the way of gathering this information and its quality. I know for sure that software support and administration people would greatly benefit from all this information gathered during the software cycle way things were done the way they were done while the agile team was communicating and discussing the functionality they have delivered.
There was a reason why people started creating bug tracking systems; there are people out there that earn a living creating it. If it didn’t sell, than there wouldn’t be a reason. I say, find out the reason, implement and improve the way of doing it or dismiss the reason within a certain context, but…
If you don’t measure, you don’t know and if you don’t know my arguement is as solid as yours regarding pro or contra. Don’t guess, prove it.