# Category:Tests Of Statistical Significance

Related Online Training modules
Automatic Tests of Statistical Significance
Significance Tests on Grids
Type 1 Error
Population Weights
Non-Proportional Sampling Weights
Planned Tests of Statistical Significance
Column Comparisons
ANOVA with Post Hocs
ANOVA Repeated Measures with Post Hocs
Generally it is best to access online training from within Q by selecting Help : Online Training

## Statistical Significance Testing in Tables

There are two ways that Q can do statistical significance testing in a table:

## How Q Determines Significance Testing

 Project & table-specific Statistical Assumption settings Q Rules that might override settings Question Type and merged values/categories used for sample sizes and percentages Variable Type (Mean or Proportion testing) Table structure such as column spans Select significance test to run based on the data and settings Calculate p-values Correct p-values Identify significance Testing the cell complement: Font Colors: Red = lower Blue = higher Arrows: Arrow length = degree of significance Testing using Compare columns: Upper case = 99.9% confidence and higher Lower case = 95 - 99.9%

More detail can be found here: Overview of Statistical Testing in Q

## Default Statistical Assumptions

These settings can be found under Edit > Project Options > Customize > Statistical Assumptions or modified for individual charts under Edit > Table Options > Statistical Assumptions.

1. Show significance: Arrows and font colors will designate significant results in tables
2. Overall significance level: testing will be done at the 95% confidence level and above
3. Minimal sample size for testing: you must have at least 2 respondents in your sample to test

4. Statistical tests for categorical and numeric data:

5. Proportions: non-parametric tests will be done on categorical data
6. Means: t-test will be done on numeric data and corrected with Bessel’s correction
7. Correlations: default is Pearson
8. Equal variance in tests when sample size is less than: if the sample size is less than 10 variance is assumed equal

9. Cell comparisons: the complement of the cell with be tested
10. Multiple comparison correction: False Discovery Rate is by default applied to help reduce for the number of false positives based on the entire table

11. Weights and significance: Automatically a mix of Taylor Series Linearization and Kish's Effective Sample Size Formula
12. Date questions: tests compare across all dates rather than previous period

1. Significance levels and appearance: Arrows: get longer with increased significance. Colors: Blue = significantly higher. Red = significantly lower. Font: Letters for column comparisons become capitalized after .001 is reached.

2. Column comparisons: take affect only if Column Comparisons are selected

3. Multiple Comparison correction: False Discovery Rate is by default applied to help reduce for the number of false positives based on the number of columns within the row & column span
4. Overlaps: Default is for Q to ignore the sample that overlaps between columns when respondents in columns are not mutually exclusive
5. ANOVA-Type Test: ANOVA is not run before displaying significance
6. Show redundant tests: show significance on one cell (the one with the higher value)
7. Show as groups: Show letters for insignificant columns rather than significant
8. Recycle column letters: each span begins labeling columns at A
9. No test symbol: - is shown if a test isn’t performed due to settings
10. Symbol for non-significant test: nothing is shown if a test comes back insignificant

## Troubleshooting

To work out which test has been conducted on the cell of a table:

1. Check to see if any Rules have been applied. If they have, review their documentation.
2. Select the cell or both cells if comparing columns and press (the same as right clicking and selecting Test Statistical Significance)

## Other technical resources

Technical Assumptions of Tests of Statistical Significance contains a general discussion about the use and interpretation of tests of statistical significance. Further reading: Market Research Analysis Software