Setting Statistical Assumptions When Setting Up Projects

From Q
Jump to: navigation, search

Contents

Setting defaults in a project

The default statistical assumptions in Q:

  • Are conservative. In particular, the False Discovery Rate Correction is automatically applied on all tables.
  • Focus on using statistical testing as a way of highlighting results that are in some sense exceptions, and displays these results using colors and arrows (e.g., EighteenUp.png)

The defaults can be modified using Edit > Project Options > Customize > Statistical Assumptions. Common modifications are:

  • Changing Show significance to Compare columns (see Ways of Showing Statistical Significance).
  • Changing the Overall significance level from 0.05 to 0.10 (i.e., from the 95% level of confidence to 90%).
  • Set the Minimal sample size for testing to 30.
  • Changing both Multiple comparison correction settings to None.
  • Either:
    • In the Significance levels and appearance grid, copy one of the uppercase Column letters fields (i.e., A,B,C,...,) and paste it over the top of each of the lowercase fields (i.e., a,b,c,...,), which means that Q will only use UPPERCASE letters when showing significance.
    • In the Significance levels and appearance grid, copy one of the uppercase Column letters fields (i.e., A,B,C,...,) and paste it over the top of each of the lowercase fields (i.e., a,b,c,...,), except those for the rows with a Cutoff p-value of 0.005, 0.01 and 0.05. In conjunction with changing the Overall significance level, this means that Q shows lowercase letters for results significance at the 0.10 level and UPPERCASE for 0.10 and higher.

If wanting to replicate results from other programs it will be necessary to adjust more options (see Results Are Different to those from Another Program). However, some caution should be undertaken when modifying the settings, as Q's default statistical tests are, in general, safer than those in more traditional survey programs (in particular, they deal better with weighting and edge cases, such as if testing against a cell containing 0% or 100%).

Applying the defaults to other projects

Modifications that are made to Statistical Assumptions can be saved as Project Templates, and automatically applied to new projects (see Project Templates).

Over-riding Q's statistical testing

Rules can be used to modify how Q performs statistical tests. For example, Significance Testing - Independent Samples Column Means and Proportions Tests can be used to perform tests that assume samples are independent, even when the underlying data are not independent. Considerable caution should be undertaken when performing using rules to over-ride Q's inbuilt statistical tests:

  • The rules are less sophisticated than the in-built tests (i.e., they have fewer checks and balances).
  • Rules are essentially modifications of the core functionality of Q and can have unintended consequences (e.g., if inadvertently applied to the wrong table, or, if using them in ways not anticipated when they were created).

See also

Personal tools
Namespaces

Variants
Actions
Navigation
Categories
Toolbox