Draft Statistical Test Landing Page

From Q
Jump to: navigation, search
Related Online Training modules
Automatic Tests of Statistical Significance
Significance Tests on Grids
Type 1 Error
Population Weights
Non-Proportional Sampling Weights
Planned Tests of Statistical Significance
Column Comparisons
ANOVA with Post Hocs
ANOVA Repeated Measures with Post Hocs
Generally it is best to access online training from within Q by selecting Help : Online Training

How Q Determines Significance Testing

640 x 374px


Two main methods:

  1. Cell comparisons - test a cell against it's complement, see also: Testing the Complement of a Cell
  2. Column comparisons - test columns against one another, see also: Interpreting Column Comparisons and How to Specify Columns to be Compared

Default Statistical Assumptions

These settings can be found under Edit > Project Options > Customize > Statistical Assumptions or modified for individual charts under Edit > Table Options > Statistical Assumptions.


800 x 441px


  1. Show significance: Arrows and font colors will designate significant results in tables
  2. Overall significance level: testing will be done at the 95% confidence level and above
  3. Minimal sample size for testing: you must have at least 2 respondents in your sample to test

  4. Statistical tests for categorical and numeric data:

  5. Proportions: non-parametric tests will be done on categorical data
  6. Means: t-test will be done on numeric data and corrected with Bessel’s correction
  7. Correlations: default is Pearson
  8. Equal variance in tests when sample size is less than: if the sample size is less than 10 variance is assumed equal

  9. Cell comparisons: the complement of the cell with be tested
  10. Multiple comparison correction: False Discovery Rate is by default applied to help reduce for the number of false positives based on the entire table

  11. Weights and significance: Automatically a mix of Taylor Series Linearization and Kish's Effective Sample Size Formula
  12. Date questions: tests compare across all dates rather than previous period




800 x 441px


  1. Significance levels and appearance: Arrows: get longer with increased significance. Colors: Blue = significantly higher. Red = significantly lower. Font: Letters for column comparisons become capitalized after .001 is reached.

  2. Column comparisons: take affect only if Column Comparisons are selected

  3. Multiple Comparison correction: False Discovery Rate is by default applied to help reduce for the number of false positives based on the number of columns within the row & column span
  4. Overlaps: Default is for Q to ignore the sample that overlaps between columns when respondents in columns are not mutually exclusive
  5. ANOVA-Type Test: ANOVA is not run before displaying significance
  6. Show redundant tests: show significance on one cell (the one with the higher value)
  7. Show as groups: Show letters for insignificant columns rather than significant
  8. Recycle column letters: each span begins labeling columns at A
  9. No test symbol: - is shown if a test isn’t performed due to settings
  10. Symbol for non-significant test: nothing is shown if a test comes back insignificant


Quick Links

Popular How Tos

Troubleshooting

Other technical resources

Technical Assumptions of Tests of Statistical Significance contains a general discussion about the use and interpretation of tests of statistical significance. Further reading: Market Research Analysis Software