Ways of Showing Statistical Significance
Where a result is computed as being statistically significant (see Overview of Statistical Testing in Q) it can be presented in a variety of ways.
Tests on tables and charts
The way that tests are shown on tables and charts is determined by the selection in the Show significance drop-down menu at the top-middle of the Outputs Tab: . When set to No, no significance is shown.
By default Arrows and Font Colors are used; this can be changed for a project in Statistical Assumptions, and for all future projects using Project Templates.
On a standard crosstab, arrows represent the results of Testing the Complement of a Cell. For discussions of the interpretations of tests on different types of tables, see Reading Tables and Interpreting Significance Tests.
Arrows point up when a result is significantly "higher" and down if significantly lower. The length of the arrows relates to the degree of statistical significance, as determined by the Corrected p. The specific relationship between the length of arrows and significance is governed by the settings in the Statistical Assumptions table of settings called Significance levels and appearance.
Font Color (and Column and Line Color)
In tables and most charts, significant results are color-coded (by default, blue and red; e.g., ). The testing is conducted in the same way as with Arrows, except that a higher result is by default shown in blue and lower in red.
When using Trend Plots the lines and bars are color-coded to indicate significance if the Font Color option is selected.
Arrows and Font Colors (default)
Both arrows and colors are used. See the two sections above for details.
Letters or other codes are used to indicate significant differences between results in different columns. Alternatively, these can be selected by right-clicking on the table and selecting Statistics - Cells and Column Comparisons and Column Names.
Q uses Corrected p when determining whether to assign letter or not, and when determining which symbols to apply (e.g., UPPERCASE or lowercase).
Settings regarding the symbols to be shown are in the Statistical Assumptions sections on Significance levels and appearance and Column comparisons.
Other ways of showing statistical significance
Where Smart Tables is used, tables are classified as being Significant and Insignificant in the Report Tree, with the p-Values shown in the names of the tables.
Reports from planned tests
Significance tests can be conducted by selecting cells and pressing . See Planned Tests Of Statistical Significance.
Regression and Segments analyses of Experiment and Ranking questions contain statistical tests of the parameters in their outputs.
Cell shading and other tools