Try it for free

Test Run Blog

We are committed to a clear, fair blog.

"Dreaming about Test Run"

We have been floored by the customer response to the new Test Run. We have been working hard every day to make it better and better and the support we have been receiving makes this process so much fun. One email sent to us caught our eye especially and we just had to share it with the rest of the world:

Test Run is a product that I've been literally dreaming about so much to the point that I almost built something like it myself. I was really stoked to see that someone shared the same vision for a quality test case execution product that is easy to use. Keep up the great work!

Seriously, kudos like this is what keeps us going. Thank you for your support and thank you everyone for choosing Test Run!

A preview of our new web site...

We have been hard at work gearing up for our upcoming launch of Test Run 1.2. An important piece of this relaunch is to take a fresh look at everything and figure out if there is a way we could do it better.

This weekend was time for us to look at our homepage in a new light. We love, love, love the work our partners Mule did when we launched, but we felt we could do a much better job of communicating Test Run's value, as well as help them to choose the plan that is right for them. So we took a swag at the homepage and the pricing page and want to hear what you think.

Homepage

Homepage v2

Pricing Page

Screen shot 2010-04-12 at 11.09.01 AM.png

We drew our inspiration from products we love and products we feel do an excellent job with their homepage, including:

Upcoming Changes to Account Plans

We are very excited about the upcoming enhancements to Test Run, the test planning solution quality assurance professionals love. There are tons of new features, and we have been working hard to dramatically improve the performance, stability and overall quality of the application. Lastly, to better serve the needs of our wide spectrum of customers we are making the following changes to the account plans available to our users and customers:

New Account Level

One of the things numerous users have asked for is a subscription level that was less expensive, and geared towards smaller teams. We are happy to oblige. When Test Run 1.2 is finally released, we will be introducing "Test Run for Small Teams" for the low cost of $20 per month, an account level perfect for QA teams with 3 or fewer team members.

Storage Limits

To account for the costs associated with storing data uploaded by our users, we have come to a point where we need to institute fair and reasonable limits on storage. When Test Run 1.2 is released we will begin monitoring and enforcing1 the following storage limits:

  • Free Trials will be limited to 1MB of storage.
  • Test Run for Small Teams accounts will be limited to 1GB of storage.
  • Test Run Pro accounts will be limited to 10GB of storage.

All existing users will be able to see how much storage their account is utilizing by visiting "Your Subscription" found under the Account tab.

15-day Trials

Over time our ability to reasonably support a perpetual one-user free trial has become unsustainable. Therefore, we have made the difficult decision to discontinue this free perpetual trial once Test Run 1.2 is released.

We recognize however that in order to evaluate our product, every user must have the freedom to go beyond screenshots and screencasts and have the ability to actually use it. So while a free perpetual trial will be no more, we are happy to offer a free 15 day trial. These free trials will be limited in the following ways:

  • 1 user maximum.
  • 1MB storage capacity maximum.

Furthermore, users wishing to trial Test Run with multiple users can upgrade their free trial to any of the available subscription levels. No matter what however, the full 15 day free trial period will be honored and you will not be billed for the subscription level you selected until the end of your trial period. If you are not happy with Test Run during your trial, you can cancel at any time at no charge.

Frequently Asked Questions

  • Paying customers should have zero disruption to their service, and the terms of their existing subscriptions will be honored for as long as they remain active and in good standing.

  • Starting with the release of Test Run 1.2, all users of the free perpetual one user trial account level will be required to upgrade their account to a paid subscription. Accommodations can be made to users wishing to export their data and cancel their accounts if they wish.

  • Users who meet the requirements of Test Run for Small Teams are welcome to downgrade their accounts if they so desire.

  • If you have any questions or concerns about these changes, please don't hesitate to contact us. We are reasonable people and will do our best to accomodate your needs.

1 - To help customers with the transition of these limits we will not begin enforcing storage limits immediately. Instead, customers are encouraged to view their account details and bring their accounts into compliance with the terms of their elected service level on their own.

Follow Test Run on Twitter

Test Run has been undergoing some serious development as of late, which has given us a lot of reasons to communicate with our customers. Of course, some folks may just want to hear the "big news" when it is available - for those of you in that camp, this blog is perfect. But I have also found a number of people are also interested in following along with the details of the development process as well. If you are the kind of person who likes more communication from Test Run (of the 140 character variety) then follow us on Twitter - the perfect place to get small doses of news from the test planning product you know and love!

Working on Test Run 1.2 - what's in store

For the past two years I have been Test Run has been hosted under what might have seemed like a perpetual beta. Over the course of those two years, bugs were fixed here and there, but by and large no major enhancements were made. This left customers wondering:

What is going on with Test Run? And has this product completely stalled out?

A completely fair question under the circumstances. You see, Test Run was started as a side project of mine and, and to be perfectly candid, tended to play second fiddle to my day job. Loyal customers have always understood this, and for that, I will be eternally grateful.

But my loyal customers, and indeed all of my customers deserve much, much more from an application that they depend on. So I have rededicated myself to this labor of love to bring an added level of quality, performance and feature enhancement to the platform.

For the past four weeks I have been working tirelessly on improving Test Run and here are just some of the enhancements I have made:

  • Performance Boost - As a modern web application Test Run relies upon javascript for a great deal of its functionality, therefore I have re-written and optimized all of Test Run's javascript to be slimmer, less error prone, and best of all faster. Navigation is snappier, dialogs open and close more smoothly, AJAX is faster, all Javascript is more lean and faster than ever before.

  • Bug Drivers - Customers have been asking for Jira support for as long as I can remember and I am happy to report that Jira support is finally coming. Now when a test case fails you will be able to create a bug in Jira from directly within Test Run. This initial implementation may have some limitations with regards to reporting, but with customer feedback I hope to resolve those issues quickly.

  • Style Enhancements - All dialogs are now rendered in native Javascript and share a consistent styling, and every form in the application has been redesigned with a consistent styling in order to provide the polish Test Run customers deserve.

  • Improved Support - Sometimes, God forbid, Test Run customers have questions, and when they do, they should have the best experience possible in asking them, and having them answered. Therefore I have ditched the janky support application I have been using in favor of the best-in-class Tender App support system. Now users will have access to a world class knowledge base and ticketing system so that in the event that things go wrong, customers have the most favorable experience possible under the circumstances.

  • Enhanced Registration - New users will have live and inline form validation during registration to create a smooth pathway into a the free trial period.

  • Better Browser Support - While redesigning the forms and porting the entire application to jQuery we paid special attention to ensure consistent styling among all popular browsers, including the most recent versions of Internet Explorer, Chrome, Firefox and Safari.

  • New Pricing Option for Small Teams - Many customers have asked for a less expensive subscription offering for small teams, a request I am happy to oblige. When Test Run 1.2 is officially released, there will be three options available to customers:

    • Free 15-day trial
    • Small Team Subscription, limiting accounts to three users for just $20 per month.
    • Large Team Subscription, with unlimited users for just $50 per month.

We love all of these new enhancements and features and look forward to a steady stream of improvements to come!

Skip to the Chase: When?

Test Run 1.2 is currently in the final stages of development, which we expect to take two to three more weeks. Once complete, customers will be invited to participate in a free and final beta testing period for several weeks. Then, Test Run will officially come out of its two year beta - a milestone I am anxiously looking forward to!

Making use of "Products" in Test Run: Hierarchies anyone?

Customers frequently ask if Test Run could one day in the future support hierarchical test cases. It is a feature that is definitely on the roadmap, but its implementation would be pretty hairy, so I have honestly been putting it off. But it occurred to me that the feature may already exist in a latent form.

If you have used Test Run than you are familiar with a "Product" property associated with test cases. It is a property I added, but I never developed any *features* around because it was clear that some form of test case segmentation was needed, but it was unclear how it might be used. So this feature exists, but does not provide any benefit to users.

It occurs to me however, that this feature could easily be used to segment test cases into *test suites*. Then a slight UI change could easily be introduced to group and display test cases by test suite as opposed to in a strict reverse chronological order.

I would love to hear from users what they think of this idea and whether they currently use the Product field for anything. Let me know!

Follow Test Run on Twitter!

Test Run now has a Twitter account. If you want to stay up with the latest development news, follow us there!

twitter-logo.jpg

A New Manage Test Plans Design

I have been receiving a lot of feedback in the New Year from new customers, perhaps because people out there are looking for ways to cut costs and keep things simple. Their feedback as usual has been invaluable, and most recently led me to revisit the design of the Manage Test Plans screen. The impetus for the change was to create a framework that allowed for me to surface more management functionality on the screen and to offer a more compressed layout that allows admins to take more information in more quickly and efficiently. I am not finished yet, so feedback is still welcome. Here is a before and after of the Manage Test Plans screen:

Before

Active Test Plans: Before

After

Active Test Plans: After

What do you think?

Redesigning Test Run Forms...

I have been looking and admiring a lot of other web applications recently and have paid special attention to form design. Test Run's forms have gone through multiple iterations, but none of them are as simple as I would like them to be. I feel they require too much scrolling and they don't put the right emphasis on the right things. So I am in the process of redesigning Test Run's forms and would love to hear from customers what they think.

Before

Test Run Form (before)

After

Test Run Form (after)

Here is what I have done:

  • I have place summary and description up front because those are arguably the most important fields on the page.
  • I have devoted less space to the text format radio buttons and kept them closer to the textarea box that they control.
  • I have widened the summary and description fields so you can see more of what you are typing.

Performance Tuning and Enhancements

With the slow and steady increase of customer usage of Test Run Online, the product has begun to feel a little sluggish and a number of you, my customers, had expressed their concern. On the up side, I guess that is bound to happen when you have over a hundred thousand test cases, and five thousand test plans - surely a sign of success by some measure. But the success of Test Run should never come at the expense of paying customers, so I have turned my attention towards performance and database optimization.

Having just completed a thorough analysis of Test Run's database architecture I will, over the course of the next several days, be rolling out a number of recommended improvements to the system. Several have already been deployed and the results are enjoyable - and difference is remarkable.

Of course, I am also working on other improvements to make Test Run more usable, and flexible. A number of users have been having difficulty with text formatting, and in working with them I realized that text formatting doesn't always make sense as a global setting. So, I just finished making text formatting, like the use of markdown and a rich text editor, a test case level setting. Now each and every test case can set its own preference for how it should be formatted.

Other improvements are also coming, so stay tuned.

About Byrne Reese

A photograph of Byrne Reese

Byrne Reese is a product manager by day and an engineer by night.

He conceived of Test Run to help project managers like him stay up to date and informed of what his team was working on.

This web site is powered by Movable Type Pro 4.24-en