A recent Methods & Tools survey tried to evaluate the usage of tools to automate execution of functional software tests. A similar poll was conducted in 2005 and it was interesting to compare the results.
| 2009 | 2005 | |
| My organisation has no tool for functional software tests | 37% | 38%; |
| My organisation has tools, but my project or I do not use them | 19% | 26% |
| I use tools for functional tests | 44% | 36% |
Participants: 268 (147)
Ending date: August 2009 (August 2005)
There is an improvement of tool usage since 2005, but the results seem still to show an ambiguous situation to me: 63 % of companies have functional software tools, but 56% of participants don’t use any tool for functional testing. In my personal experience as a developer in (non-agile) business software development organizations, the developers had to be able to understand functional requirements and to test them before their delivery to the user for acceptance testing.
Asking questions about this situation to members of the software development community, I discovered many causes for the non-usage of functional testing tools by developers. The first problem is the awareness problem of the testing activity. Testing is still often considered as something that you do before the deadline of a project…. if you have some time left. Functional testing is even more at the end of the testing sequence from the developer side. Automation of this activity is also costly. Besides the price of the commercial tools that can be very expensive, you have also to set-up a production-like environment (software, data) to perform meaningful tests. But this doesn’t completely get to the point, as all answering organizations were performing functional tests.
Every functional test tools has its own language, so you cannot transfer easily the knowledge you have with one tool to another. It is difficult to maintain automated scripts of functional testing tools. If the application to test evolves too quickly, scripts become obsolete and are not used anymore. It is also mentions that tools are bought sometimes for specific project and may not be useful for other that are based on different software technologies.
However, a majority of the remarks about the difference between tools adoption and tools usage pointed out that the functional testing phase marks in many organizations the division between the developer and tester responsibilities. Despite the fact that agile approaches aim at reducing work specialization with the developer discussing directly with the user, it seems that there is still an important number of organizations where QA specialists perform functional testing. This could be seen also a consequence of the specialization of the functional testing tools, but I don’t think they are more difficult to learn than unit testing tools.
This specialization, or non-responsibility, hides the global aim of the projects to the developer. From my point of view, this situation is similar to creating tires not knowing if they will be used for a car or a truck, and not being able to test drive the vehicle that will use them to check if they are working efficiently. Some will tell that software developers are not interested to talk to users, but we have to realize that we do not produce software for ourselves. It make me think of the operations guys that seemed sometimes to want computers with only operating systems running on them. They blame all the problems on the internal developers that supply software for the company users. Good developers should be able to understand what their users want, functionally test what they want to deliver and even better with some tools.