Our Process for Adding Features to Q

How we identify new features

We identify improvements to make in Q by:

• Listening to our user's suggestions, which we obtain from direct feedback from them, as well as surveys with consumers and occasionally qualitative research.
• Monitoring the type of queries/problems that occur during:
  • Demonstrations.
  • Training.
  • Support.
• Analyzing the type of bugs that occur.
• Reviewing the academic literature.
• Comparisons with competitive products.
• Dogfooding (i.e., using our products to analyze data).

How we prioritize new features

We prioritize these features by trading off:

• The extent to which a new feature will improve a user's productivity. So, a feature that saves an hour will be given a much higher priority than one that saves two mouse-clicks, all else being equal.
• The number of users who will benefit from the feature.
• The cost of building the feature.
• How the feature fits in terms of our long-term objectives. That is, some features we add have little immediate benefit for users, but are requisites for large features that we are planning to release in the future.

Our process for releasing new versions of Q

When we add a new feature to Q, we do the following:

1. Test it against millions of example/benchmarking computations, checking that we have not inadvertently changed any outputs.
2. Use bots to automatically check that we have not broken the way features work (i.e., check that menu items and buttons still works, and that tasks can still be completed in various orders).
3. Use it on internal dogfooding projects.
4. If no problems are identified, we then do an initial beta release, which means that we:
   • Release it to users who have previously made it known to us that they are happy to work with a version that may be buggy.
   • Release it to users who we know are blocked by the absence of the feature.
5. Fix any bugs and re-release the beta (i.e., notify users that it is available to download).
6. Once we have had a version that has no serious bugs identified for a period of, typically, not less than two weeks, but often considerably longer, we release it to most of our users. This is known as the stable release, and it becomes available via Q informing you that there is a new version to download.
7. If we identify bugs in the stable release, we fix them and give the users the option of downloading the fixed version.
8. Once the stable release has been found to have no serious bugs for a period of at least 3 months, but often closer to 6, we release it as a super stable version, which is used by clients that have advised us that there IT departments are particularly slow moving and unwilling to install new versions of software except on rare occasions.