# Draft Statistical Test Landing Page

From Q

Related Online Training modules | |
---|---|

Automatic Tests of Statistical Significance | |

Significance Tests on Grids | |

Type 1 Error | |

Population Weights | |

Non-Proportional Sampling Weights | |

Planned Tests of Statistical Significance | |

Column Comparisons | |

ANOVA with Post Hocs | |

ANOVA Repeated Measures with Post Hocs | |

Generally it is best to access online training from within Q by selecting Help : Online Training |

## Contents

## How Q Determines Significance Testing

**Two main methods:**

- Cell comparisons - test a cell against it's complement, see also: Testing the Complement of a Cell
- Column comparisons - test columns against one another, see also: Interpreting Column Comparisons and How to Specify Columns to be Compared

## Default Statistical Assumptions

These settings can be found under **Edit > Project Options > Customize > Statistical Assumptions** or modified for individual charts under **Edit > Table Options > Statistical Assumptions**.

**Show significance**: Arrows and font colors will designate significant results in tables**Overall significance level**: testing will be done at the 95% confidence level and above**Minimal sample size for testing**: you must have at least 2 respondents in your sample to test**Proportions**: non-parametric tests will be done on categorical data**Means**: t-test will be done on numeric data and corrected with Bessel’s correction**Correlations**: default is Pearson**Equal variance in tests when sample size is less than**: if the sample size is less than 10 variance is assumed equal**Cell comparisons**: the complement of the cell with be tested**Weights and significance**: Automatically a mix of Taylor Series Linearization and Kish's Effective Sample Size Formula**Date questions**: tests compare across all dates rather than previous period

**Statistical tests for categorical and numeric data:**

**Multiple comparison correction**: False Discovery Rate is by default applied to help reduce for the number of false positives based on the entire table

**Significance levels and appearance**: Arrows: get longer with increased significance. Colors: Blue = significantly higher. Red = significantly lower. Font: Letters for column comparisons become capitalized after .001 is reached.**Multiple Comparison correction**: False Discovery Rate is by default applied to help reduce for the number of false positives based on the number of columns within the row & column span**Overlaps**: Default is for Q to ignore the sample that overlaps between columns when respondents in columns are not mutually exclusive**ANOVA-Type Test**: ANOVA is not run before displaying significance**Show redundant tests**: show significance on one cell (the one with the higher value)**Show as groups**: Show letters for insignificant columns rather than significant**Recycle column letters**: each span begins labeling columns at A**No test symbol**: - is shown if a test isn’t performed due to settings**Symbol for non-significant test**: nothing is shown if a test comes back insignificant

**Column comparisons**: take affect only if Column Comparisons are selected

## Quick Links

- A general
**overview**on how to perform a statistical test is here: Overview of Statistical Testing in Q - A guide to what settings and
**statistical assumptions**are available in Q is here: Statistical Assumptions - Info on how Q
**displays significance**testing is here: How Q Highlights Results as Being Significant - To learn how to
**interpret significance**testing results start here: Category: Reading Tables and Interpreting Significance Tests - If your significant
**results are different**than expected start here: Results Are Different to those from Another Program- Many times results differ due to:
- Overlapping data -- Statistical Assumptions - Overlaps
- Multiple comparisons correction -- Multiple Comparison Correction

- Many times results differ due to:

## Popular How Tos

- How To Test Between Adjacent Time Periods
- How to Change Significance Levels for Column Comparisons
- How To Compare A Sub-Group Against The Total

## Troubleshooting

- Category: Troubleshooting Significance Tests
- Interpreting "Inconsistent" Statistical Testing Results
- The Statistical Assumptions selected for this table are inappropriate
- Results Are Different to those from Another Program

## Other technical resources

Technical Assumptions of Tests of Statistical Significance contains a general discussion about the use and interpretation of tests of statistical significance. Further reading: Market Research Analysis Software