alanwilliamson

Software Testing - Survey Results

Last week I put up a quick survey asking for some feedback on how you treat your software testing with a particular lean towards open source components.

Let me thank all those that took the time to answer the handful of questions.  Let me also thank TheServerSide for posting a link to the survey.  As promised I would like to share some of the results of the survey from the 455 replies I received in a 48hr period.  Some of the questions allowed for more than one answer to be selected.

  1. Within your organization who is responsible for testing?
    48% answered the original developer was, with 45% stating they had a separate Q&A process
  2. Do you have any formal testing procedures?
    54% answer that they do not, with only 10% claiming they have a strict policy to follow
  3. Do you have a staging environment?
    66% stated they do not
  4. How much faith do you have in open source components?
    55% stated they were confident and used the latest releases, where as 14% said they stay one or two versions behind the latest release
  5. When a new componet is released what do you do?
    4% said they upgraded immediately, where as 60% said they downloaded and played with it.
  6. What problems have you faced with open source libraries?
    27% said they had found show stopping bugs, and 60% said they had found bugs but worked around them.  28% said they had problems with integration to other components and 72% complained of poor documentation.

Then we asked a free answer question; What could open source do better in terms of testing?.  There was a wide range of views expressed here and here is some of the more interesting ones.

  • Rally around an integration testing platform to turn it into a de-facto standard like they did with JUnit.
  • Having a FOSS application test environemnt.
  • Having a good community for testing the latest releases
  • I only use the most popular OSS like Spring, Hibernate and Struts. I wholly trust the testing they already done
  • 100% Code Coverage :)
  • Not much, many open source libraries come with a nice set of automated tests, usually the lack of documentation is causing more problems then the code itself.
  • good test suites
  • I've actually found open source quite good for testing
  • Enhance the quality and usability and proper documentation
  • Publicize the result of unit test coverage and unit test execution with every release
  • Introduce quality levels for quick reference
  • I find the testing to be adequate, the amount of bugs coming through is generally minimal for most libraries
  • Transparency. Expose clearly what testing has been done, and in general what quality processes the software has been through during development
  • you can pretty much count on Apache to vigorously test new releaseses
There you have it.  I will let you draw your own conclusions from this very small, unscientific, quick survey.

 

Recent Cloud posts

Recent JAVA posts

Latest CFML posts


 
Site Links