Category:Tests Of Statistical Significance

From Q
Jump to: navigation, search
Related Online Training modules
Automatic Tests of Statistical Significance
Significance Tests on Grids
Type 1 Error
Population Weights
Non-Proportional Sampling Weights
Planned Tests of Statistical Significance
Column Comparisons
ANOVA with Post Hocs
ANOVA Repeated Measures with Post Hocs
Generally it is best to access online training from within Q by selecting Help : Online Training

Quick Links

Popular How Tos and Frequently Asked Questions

Statistical Significance Testing in Tables

There are two ways that Q can do statistical significance testing in a table:

Income summary arrows and font.png
Income v Gender arrows and font.png

Income v Gender column compare.png

How Q Determines Significance Testing

800 x 117px

Project & table-specific Statistical Assumption settings

Q Rules that might override settings

Question Type and merged values/categories used for sample sizes and percentages

Variable Type (Mean or Proportion testing)

Table structure such as column spans

Select significance test to run based on the data and settings

Calculate p-values

Correct p-values

Identify significance

Testing the cell complement:
Font Colors:
Red = lower
Blue = higher
Arrow length = degree of significance

Testing using Compare columns:

Upper case = 99.9% confidence and higher
Lower case = 95 - 99.9%

More detail can be found here: Overview of Statistical Testing in Q

Default Statistical Assumptions

These settings can be found under Edit > Project Options > Customize > Statistical Assumptions or modified for individual charts under Edit > Table Options > Statistical Assumptions.

800 x 441px

  1. Show significance: Arrows and font colors will designate significant results in tables
  2. Overall significance level: testing will be done at the 95% confidence level and above
  3. Minimal sample size for testing: you must have at least 2 respondents in your sample to test

  4. Statistical tests for categorical and numeric data:

  5. Proportions: non-parametric tests will be done on categorical data
  6. Means: t-test will be done on numeric data and corrected with Bessel’s correction
  7. Correlations: default is Pearson
  8. Equal variance in tests when sample size is less than: if the sample size is less than 10 variance is assumed equal

  9. Cell comparisons: the complement of the cell with be tested
  10. Multiple comparison correction: False Discovery Rate is by default applied to help reduce for the number of false positives based on the entire table

  11. Weights and significance: Automatically a mix of Taylor Series Linearization and Kish's Effective Sample Size Formula
  12. Date questions: tests compare across all dates rather than previous period

800 x 441px

  1. Significance levels and appearance: Arrows: get longer with increased significance. Colors: Blue = significantly higher. Red = significantly lower. Font: Letters for column comparisons become capitalized after .001 is reached.

  2. Column comparisons: take affect only if Column Comparisons are selected

  3. Multiple Comparison correction: False Discovery Rate is by default applied to help reduce for the number of false positives based on the number of columns within the row & column span
  4. Overlaps: Default is for Q to ignore the sample that overlaps between columns when respondents in columns are not mutually exclusive
  5. ANOVA-Type Test: ANOVA is not run before displaying significance
  6. Show redundant tests: show significance on one cell (the one with the higher value)
  7. Show as groups: Show letters for insignificant columns rather than significant
  8. Recycle column letters: each span begins labeling columns at A
  9. No test symbol: - is shown if a test isn’t performed due to settings
  10. Symbol for non-significant test: nothing is shown if a test comes back insignificant


To work out which test has been conducted on the cell of a table:

  1. Check to see if any Rules have been applied. If they have, review their documentation.
  2. Select the cell or both cells if comparing columns and press AlphaButton.png (the same as right clicking and selecting Test Statistical Significance)
Show sig test details.png

Other troubleshooting links:

Other technical resources

Technical Assumptions of Tests of Statistical Significance contains a general discussion about the use and interpretation of tests of statistical significance. Further reading: Market Research Analysis Software


This category has the following 2 subcategories, out of 2 total.

Pages in category ‘Tests Of Statistical Significance’

The following 134 pages are in this category, out of 134 total.