Test Run Blog

Test Run 1.1 Beta Finally Begins

2008-04-28T06:58:57Z

Welcome the new version of Test Run: 1.1. Behind the scenes the product has undergone a great deal of change in the last twelve months and I am finally pleased enough with what has accumulated to release the first public beta. ## More Information about this release: * Detailed change log ## Finding Help and Reporting Bugs * Report a bug * Find help from the community in our forums ## Frequently Asked Questions **I am not sure what I found is a bug, should I still report it?** We want to hear any thoughts you may have about the application. Perhaps what you have found is not a bug, but if it doesn't work the way you want it to then something could be improved. So please, be sure to inform us if: * an error occurs * if the application appears slow * you receive a javascript error * you have a feature request **Do you have special support for Jira and other bug tracking systems?** Support for Bugzilla is almost complete. However before it is released, we wanted to finalize the developer interfaces between Test Run and the bug system driver with more thorough and public testing. Next on the list to support is Jira because of the overwhelming number of requests we have received for it. **How long will the beta last?** Our hope is to wrap up the beta as soon as possible. **Can I subscribe to Test Run during the beta?** Absolutely!

Preparing for the Test Run 1.1 Beta

2008-04-17T05:22:16Z

Test Run 1.1 has been in development for a very, very long time. I admit. But the pay off will be worth it as there is a lot in this release to look forward to, and virtually all of it is the result of direct feedback from you - my users. So, some time next week or two I hope to launch a beta of the next version of Test Run. I would like to first extend the opportunity to users of the downloadable version of Test Run to try it out and after an initial round of feedback and testing, roll it out to everyone using the online version. If you would like to get a preview of this upcoming version of Test Run, [drop me a line](http://www.runtestrun.com/help/ttx.cgi?style=corp2), and I will happily provide you with a copy. For a complete list of enhancements that have been made in Test Run 1.1 to date, keep reading. #### Test Run 1.1 Enhancements and Bug Fixes * An import feature for Test Link which will allow people to easily migrate to Test Run from the Test Link case tracking tool. * Test Plan printing - many users asked for the ability to print a test plan to make it easy to review plans offline. * Rich Text Editing - several users have asked for me to integrate a WYSIWYG editor into Test Run * Upgraded to latest version of YUI * Updated almost all buttons and dialogs to be more consistently styled * Redesigned the test plan header to be more condensed and generally useful * Refactored primary navigation through out the application to use pull down menus to make easier to move from one place in the app to another. * Added a Manage Products area * Added tag search to "Add Test Cases" dialog * Converted "Add Product" to javascript dialog * Eliminated some PHP errors about undefined indexes * Made product selection sticky * Added ability to track when a test plan was archived - now records the date and displays the archive date in the table listing * Fixed alignment of "Forgot password?" text * Anonymous users now see proper text formatting on a test case * Updated feedback link in footer to point to a generic end point that redirects to the proper location allowing us to more easily point installed customer base to a new location easily. * Status of test cases in detail view now get updated when test is run * Fixed bug where the summary field was not in focus when create/edit test case form is displayed * Fixed bug whereby the scroll bar on the add cases dialog would not disappear reliably when dialog was closed * Finish implementing Test Link import routine * Made it more intuitive on how to archive a test plan * Create test case buttons use old style * Adding a bug that already exists does nothing * Adding a bug that is already resolves, sets bug to proper state * Test Run will now resolve all bugs within an account of the same Bug ID when that bug id is marked as resolved. * Remove "Your Profile" - should just take user to edit profile screen * Edit profile form is kinda borked - fixed * Edit permissions form could use better alignment * Bug ID is linked when it shouldn't be (when a bug url pattern is absent) in report * Create/Edit Test Plan button in old style * Backend logic for resolving all logged bugs with the same ID implemented There are of course many other smaller tweaks, a number of other fixes behind the scenes to trim down the application, improve its overall design, and so forth. But I imagine most people don't really care about that stuff. Remember, if you are interesting in testing out Test Run 1.1 Beta 1, [please let me know](http://www.runtestrun.com/help/ttx.cgi?style=corp2)!

Sneak Peak of Test Run 1.1

2007-11-16T19:40:28Z

I wanted to give a quick development update as we near a release of Test Run 1.1 Beta. I have nearly completed Bugzilla integration with FogBugz, but there are still a lot of kinks to work out in regards to allowing users of our hosted product to access their private instance of Bugzilla. I have also been working on a number of UI tweaks to make Test Run easier to navigate and use. Without going into too much detail yet, here are a few screenshots to whet your appetite:

Test Run 1.1 Development Begins

2007-09-28T18:35:57Z

Summer is over, the traditional slow time of the year in the software industry, and I have been getting a lot of renewed interest in Test Run. Customers have been submitting bug reports and a lot of feature requests, and many of them have in all honesty gone relatively unanswered in the last several months. Well, that is about to change. I feel there is a critical mass of feature requests to begin working on Test Run 1.1. So last night work officially began. I have started to assess a lot of different things about the product in an effort to improve customer support and the experience around finding help. I upgraded the Test Run Blog and Knowledge Base to use Movable Type 4.0, and I have migrated Test Run's forums off of my personal web server onto the production Test Run machines. This has the added benefit of dedicating Test Run's forums exclusively to Test Run. Users will no longer find content related to my other software projects. This is a good thing. As for what features I am planning for this release: * At the top of the list is the need to integrate with popular bug tracking systems. This feature will allow users to submit bug reports directly into Bugzilla and FogBugz from within Test Run. * An import feature for Test Link which will allow people to easily migrate to Test Run from that case tracking tool. * Test Plan printing - many users asked for the ability to print a test plan to make it easy to review plans offline. * Rich Text Editing - several users have asked for me to integrate a WYSIWYG editor into Test Run, and I will see what I can do to accommodate. * Naturally there will be lots of bug fixes as well. I encourage users to submit feature requests to me (as a comment to this post if you wish) and let me know what you would like to see out of the next version of Test Run.

New features: Tag Management and Time Estimates

2007-05-24T06:18:15Z

"I have worked at a lot of companies and have been responsible for reviewing and recommending countless test management tools. Test Run is far and away the best and simplest tool I have ever encountered." God, I can't tell you how wonderful it is to get feedback like that. When it sometimes becomes really hard to create awesome software while continuing to work full time at another job you also love, feedback like that picks you up and keeps you going. And what better way to reward a growing and increasingly involved and engaged user base then with a new release of Test Run? Tonight I am happy to announce the latest version of Test Run which contains features implemented exclusively from customer feedback. Most notably, Test Run 1.09 has the the following new features.]]> * *Time Estimates* - The test case property known as "Difficulty" has been removed and replaced with a far more useful field for collecting a time estimate from the test case creator. Time estimates will allow for a number of cool new project management features down the road, but in the short term the Resource Balancing Report has become much more useful because now an administrator can quickly see more specifically how much work is assigned to each member on their team. * *Better Tag Management* - A number of users mentioned that they have tags that continue to lie about their installation that they have a lot of difficulty purging and cleaning up. Now Test Run allows administrators to delete and manage tags within their account from the convenience of a single console. * *New User Tour* - Now new users of Test Run are greeted by a redesigned welcome page and dashboard designed to help them get up and running using Test Run quickly and more easily. In addition to the above features, numerous bug fixes have been made, and a few small features have been added here and there. Thank you again to all the users who continue to send feedback, without you Test Run would not be nearly as useful.

And now with full text search

2007-04-22T04:31:27Z

So I have been working on a number of features requested by potential customers - oh the lengths I will go to to try and get someone to actually pay for the services! (It really is worth paying for honest!) One customer I have been corresponding with really admired the use of tags within Test Run and how they facilitated their filtering cases. However, they still found tags inadequate for searching test cases and asked if there was a way to offer a full-text search allowing users to find test cases by text found in the summary, detailed description, the notes and more. A quick search on Google revealed that thanks to Lucene the feature is relatively easy to implement. So, a quick download of the PHP Zend Framework and presto: full text search is done. I would have released the feature a while ago, but thanks to some issues with executing PHP from the command line it took a little longer than I anticipated to index all of the thousands and thousands of cases stored in Test Run. But I was finally able to get 100% of the test cases indexed and so search is now available online. It may take a while for me to document the process to enable full text search, so users of the downloadable version will need to be patient while I wrap a few things up. In the meantime, users of the hosted version of Test Run can begin using full text search right away.

Announcing Reorderable Test Cases

2007-04-02T03:11:37Z

We have been hard at work this week making a number of small improvements to Test Run, and one big one. A number of customers have been expressing the desire to have more control over the order in which they execute their test cases. Until now Test Run always ordered test cases in reverse chronological order. What made this challenging for many users was the fact that some testing requires test cases to be executing in a very specific sequence. So to address their concern we implemented the feature to allow test planners the ability to order test cases however they see fit.

When we designed and developed this feature we wanted to make the process as intuitive and as simple as possible, allowing test planners to drag and drop test cases in what ever order they choose. We could tell you more about this feature, but I think it much better to see it in action.

In development: Test case re-ordering

2007-03-29T06:35:12Z

Last night I released Test Run 1.04, a version that contained a minor feature enhancement enabling greater interoperability with popular bug tracking systems. Tonight I started work on Test Run 1.05 which contains a single feature: the ability to re-order test cases within a test plan. This feature request comes after a number of customers expressed concern that their process required tests to be run in a specific sequence. However Test Run's default sort order (the created on date) was insufficient. They wanted more control then that. So in the next version of Test Run, customers will be allowed to specify the order of their test cases within a test plan using a simple drag-and-drop interface. Stay tuned...

New feature: Non-numeric bug ids

2007-03-28T22:34:08Z

Jira, a bug tracking tool created by Atlassian. Jira is unique among other bug tracking systems in that the bug IDs it produces are not exclusively numeric. This made it difficult, if not impossible for those using Jira to integrate with Test Run's bug tracking capabilities. So last night we released a few changes to enabled Test Run users to enter bugs into the system that can be any arbitrary string. So Jira users: rejoice. Test Run now works even better for you.]]> A complete change log for Test Run version 1.04 follows: * Added ability to log non-numerical bug types specifically so that I could support Jira per a customer's request. * modified upgrade_map for schema changes * created new sql upgrade files to support addition of necessary columns * added SCHEMA_VERSION global var, which is now different than the public display version of Test Run * new column added to logged_bugs: bug_id_str * bug_id column in logged_bugs now obsolete * Actual/discrete version of Test Run now properly displayed in the footer * Fixed minor bug associated with building the "edit" link for a test case. * Fixed a number of bugs having to do with failing test cases and the display of the fail case dialog * fail case dialog did not always appear as a Yahoo UI dialog, sometimes was an alert * the log bug form did not always disappear properly when failure of case was canceled. * fixed egregious bug left over from testing where when you failed a test case it always logged a bug with the id of "100" no matter what you entered in the input box

"The only viable on demand solution"

2007-03-25T18:55:35Z

Tagged.com is one of Test Run's earliest adopters. Their team has been using the hosted version of Test Run for over a year now. They provide us with invaluable feedback to make the application better and we sincerely appreciate their business.

"Test Run was the only viable On Demand Test Case Management solution that provides a usable, flexible, comprehensive and reliable framework to manage tests. If you want everyone in your team to get up and running quickly, Test Run is the ideal solution."

]]> ## About Tagged, Inc. Tagged.com was launched in 2004, and is an explosively growing social-networking portal developed specifically for teens aged 13 through 19. Emerging as the premier brand in the lucrative teen demographic, Tagged is rapidly becoming the number one teen site on the web.

A Better Way to QA

2007-03-12T21:59:08Z

The past several years the Internet has been transformed by a slew of new web based tools and applications. A key success factor in all of these applications is adherence to a development methodology in which product teams release fewer features at a time, but release them at a greater frequency. Adherents to this methodology call this process “iteration.” This methodology allows teams to: * respond more quickly to customer feedback * adapt the product based upon actual usage * become more agile in the marketplace However, this methodology requires a discipline that is difficult for many to obtain and maintain. That is because it is a radical departure for most software development companies who have deeply ingrained habits and cultures built up around more rigid and waterfall like approaches to software development. For those that have made the transition to this Agile Development model, what is most intriguing is that while engineering teams and product management have undergone a tremendous process revolution the process of testing software has remained relatively unchanged. And for those companies, I ask you, how agile can your process be when chances are your QA engineers are still testing software the same way they did five years ago and ten years ago?]]> Six Apart’s (now a Test Run customer) migration to an Agile Development methodology: no matter how many advances our project managers and software engineers would make in their processes, our QA team was still relying on Excel to manage their test planning process. Excel is a great tool don’t get me wrong, but in this instance Excel made it difficult: * to gain transparency into what they were testing * to contribute and collaborate in a meaningful and measurable way * to audit QA’s progress from release to release This ultimately resulted in what QA was trying to avoid: more bugs. On numerous occasions we suffered a regression in overall quality because a QA engineer failed to copy a test case they added in one version of a test plan to another version of the test plan located in a different spreadsheet. Before implementing our Agile development process their process and tools worked just fine. However, their process had failed to keep up with the quickened pace of release cycles. In the past we rarely tested multiple releases concurrently; therefore there as rarely a need to manage the merging and consolidation of test case data throughout a release. What also made the transition difficult was a turn over in QA staff. As we shuffled resources we lost a lot of knowledge that Excel made difficult to capture. So as new staff members tried to pick up where others had left off, details were missed and bugs surfaced. It took months for those engineers to develop their own internalized knowledge base about our software and be able to operate it with the efficiency of previous team members. These are some of the challenges that inspired the creation of Test Run. Unlike other test planning tools, Test Run is architected to be iterative, because I believe that test planning is a learning process, just as much as feature development is. In other words, a test case is something that should get better each time your execute it. This is something Test Run helps every QA team achieve in their process, which is something no other test case management tool helps you do. It accomplishes this by: * utilizing a single, but large repository of test cases to draw upon * maintaining a test case execution history for you automatically * allowing users to attach notes and file to a test case that travel with the test case no matter where or when it is executed * using tags as a means of categorization as opposed to complex and rigid database schemas In my many years devoted to optimizing the QA process through useful and meaningful software I have learning one thing over and over: your software development process is only as agile as your least agile stage of development. Therefore, if you really want to capitalize on the benefits of the Agile Development Methodology, it is critical for you to look outside the context of software engineering alone and look at your whole process. Ask yourself:

“If my software engineers are exploring more efficient ways of designing and building applications, shouldn’t my QA engineers be exploring more efficient ways of testing those applications?”

Yes. Of course. A million times, yes! Check out Test Run today!

Zoom zoom zoom - Performance Upgrades

2007-02-20T06:38:35Z

A few customers with very large accounts began experiencing performance problems with Test Run soon after it was officially released. These performance issues were restricted to only a few accounts, however they posed a risk to all customers when and if Test Run ever came under severe load. Therefore we sought to address these problems proactively by taking the following steps. First we have optimized the database to make certain searches and queries faster and more efficient. We also deployed support for memcached throughout the application to eliminate unnecessary database queries. Right now these enhancements have only been made available to users of Test Run Online, but they will soon be made available to users of the downloadable version of Test Run as well.

Test Run is now more secure

2007-02-12T22:28:08Z

In a continuing effort to make Test Run more secure for its users, Test Run Online has been upgraded to use SSL from a trusted and well established certificate authority. Now when you use Test Run your account information will be encrypted and secured.

Test Run 1.0 is here!

2007-02-06T08:52:08Z

After two years of development, Test Run 1.0 is finally here. Thank you to all the beta testers who have helped make this release and product possible. I honestly could not have done it without your ideas, patience, support and dedication. Thank you. Most questions can be answered quickly by reading Test Run's list of Frequently Asked Questions. However, over the next couple of days and weeks we will be updating the blog to tell you more and more about Test Run and its capabilities. In the meantime, please keep reading to receive a brief overview of the some of the immediate changes you will notice when you login to Test Run 1.0 for the first time.]]> support know and we can help you to resolve it ASAP. * One of the biggest changes between the last beta and the full release is the elimination of the "Objectives" field from all test cases. The data users entered into this field has been prepended to the text found in the "Detailed Description" field. This field was removed to further simplify the test case schema and make creating test cases even faster and easier. * While Test Run LLC secures an SSL certificate for its web servers, users wishing to purchase the downloadable version of Test Run may have to wait up to several hours for your payment to be processed manually. Please be patient, this process will be automated and streamlined soon. * All QA was conducted on the standards compliant browser of FireFox 2.0. If you are using another web browser and encounter difficulty using the web application please let support know ASAP. Again, stay tuned to the Test Run Blog for more news and announcements as we slowly build up steam for more announcements, more users and more features that will be introduced over the course of the next several months!

Test Run 1.0 is coming...

2007-02-04T09:57:25Z

It has been a long time since I have posted to this blog, but it is not because this project and product have been abandoned. Quite the contrary. I simply have been in a habit of posting only to announce releases, a habit I hope to break. But not tonight. Tonight I am happy to announce that the long awaited release of Test Run 1.0 is almost here. It is hard to believe that this release has been in development for almost two years. And that in the course of those two years Test Run has been downloaded almost 2500 times. Plus, the online version has had almost 700 sign ups, and over 16,000 test cases have been executed. This release is the culmination of a tremendous amount of input from users provided over those two years. Without this feedback, Test Run would never have evolved to be so useful to so many companies. In addition, this release is the result of a wonderful collaboration with an incredible partner and design firm, Mule Design. Simply put, they are really good at what they do. And I believe it shows in the final product. Long story short, in the coming days, users of Test Run will experience a number of changes. Most notably Test Run will be given a completely new design and interaction model to make test case management even simpler and easier. Sadly for some users, this release will mark the end of the free beta period for Test Run. When Test Run 1.0 is released all existing users will be asked to purchase a subscription to Test Run Online, or a license for the downloadable version of Test Run which can be renewed annually. There is so much to explain, and still many things to do. But I promise, to answer everyone's questions soon. So before I sign off for the evening to head to bed, I will leave you with one last treat, a sneak peak of the new Test Run web site. Stay tuned. We are almost there...