Significance Testing - Identify Interesting Tables

This QScript scans through the tables you have selected in your report and creates a new table to help you identify the most interesting results from your selection. Each of your selected tables gets a score between 0 and 5 with higher scores indicating that the table contains a more interesting result, and lower scores indicating that the table does not contain any interesting results. To visualize the results, and get an overview of your tables, use Create > Charts > Visualization > Heatmap and connect it to the summary table that is produced by this QScript.

Example

In this example there are several hundred crosstabs and SUMMARY tables in the project. All of the tables in the report were selected before running the script. The initial table produced by this QScript looks like this:

There is one row for each of the questions featured among the rows of the selected tables, and one column for each of the questions featured among the columns of the selected tables. Each cell contains a score from 0 to 5 indicating the z-Score of the most significant result shown on the table for those two questions. Only a small part of the total table is shown.

In this example, the table shows that there is a high score (5) for the SUMMARY table for Q5. Unaided Awareness and a comparatively smaller score (2.4) for the crosstab between Q5. Unaided Awareness and Occupation.

A heatmap is a great way to visualize this table. After running the script, you can:

1. Select Create > Charts > Visualization > Heatmap.
2. In the Object Inspector for the heatmap, click the Output in 'Pages' menu and choose the most.significant.results output that was produced by the script.
3. Tick Automatic.

The heatmap then shades the cells darker when the pair of questions produces a more interesting result, and shades the cells lighter when the most interesting result is less significant. You can hover your mouse over the cells in the heatmap to pick out which combinations produce the most interesting results, and then go and inspect those tables further to determine which results are of most interest to your research.

Technical details

The 0-5 scores for each table are worked out based on the z-Statistics of the table. For each table, the script identifies the z-Statistic with the largest magnitude (absolute value), and caps this value at 5. The z-Statistics on a table can be checked by selecting Statistics - Cells > z - Statistic.

The z-Statistics are used by Q's statistical testing to indicate the lengths of arrows shown in cells of the table. Where cells are determined to be significant, the largest Z will correspond to the cell with the largest arrow.

The table that is produced by this script is created with an R Output.

The results in this table are static and not linked to the underlying data. They are determined at the time the script runs, and they will not update if the data changes. If you update your project with new data, or change the structure of the variables in your project, then you should run this script again to get an updated reading of the results.

How to apply this QScript

• Start typing the name of the QScript into the Search features and data box in the top right of the Q window.
• Click on the QScript when it appears in the QScripts and Rules section of the search results.

OR

• Select Automate > Browse Online Library.
• Select this QScript from the list.

Customizing the QScript

This QScript is written in JavaScript and can be customized by copying and modifying the JavaScript.

Customizing QScripts in Q4.11 and more recent versions

• Start typing the name of the QScript into the Search features and data box in the top right of the Q window.
• Hover your mouse over the QScript when it appears in the QScripts and Rules section of the search results.
• Press Edit a Copy (bottom-left corner of the preview).
• Modify the JavaScript (see QScripts for more detail on this).
• Either:
  • Run the QScript, by pressing the blue triangle button.
  • Save the QScript and run it at a later time, using Automate > Run QScript (Macro) from File.

Customizing QScripts in older versions

• Copy the JavaScript shown on this page.
• Create a new text file, giving it a file extension of .QScript. See here for more information about how to do this.
• Modify the JavaScript (see QScripts for more detail on this).
• Run the file using Automate > Run QScript (Macro) from File.

JavaScript

includeWeb('QScript R Output Functions');
includeWeb('QScript Selection Functions');
includeWeb('QScript Functions to Generate Outputs');
includeWeb("JavaScript Utilities");
includeWeb("QScript Table Functions");
main();
function main() {
    // On the web just take from what is selected.
    var web_mode = (!!Q.isOnTheWeb && Q.isOnTheWeb());
    // Collect all selected tables and plots
    var selected_items = project.report.selectedItems().filter(function (item) { return item.type == "Table"; });
    // Exclude text tables, plots with 3 dimensions
    selected_items = selected_items.filter(function (item) {
        return item.primary.questionType != "Text";
    });
    
    if (selected_items.length == 0) {
        log("No appropriate tables selected. Select several tables or pages in your Report and try again.");
        return;
    }
    // Get largest absolute z-Statistics
    var values_object = {};
    selected_items.forEach(function (table) {
        var biggest_z;
        var table_output;
        try {
            table_output = table.calculateOutput();
        } catch (e) {
            return;
        }
        
        var stats = getStatisticsFromTable(table, ["z-Statistic"]);
        if (stats == null)
            biggest_z = null
        else {
            var row_indices_without_net = table_output.rowIndices(false);
            var stats_without_net = [];
            row_indices_without_net.forEach(function (index) {
                stats_without_net.push(stats["z-Statistic"][index]);
            });
            var all_values = [];
            stats_without_net.forEach(function (a) {
                all_values = all_values.concat(a);
            })
            all_values = all_values.filter(function (x) { return !isNaN(x); });
            all_values = all_values.map(function (x) { return Math.abs(x); });
            if (all_values.length == 0){
                biggest_z = null;
            } else {
                biggest_z = Math.max.apply(null, all_values);
                if (biggest_z > 5)
                    biggest_z = 5;    
            }
            
        }
    
        var blue = table.primary.name;
        var brown = (table.secondary == "SUMMARY") ? "SUMMARY" : table.secondary.name;
        if (!values_object[blue])
            values_object[blue] = {};
        values_object[blue][brown] = biggest_z;
    });
    // Find unique set of brown questions
    var unique_browns = [];
    var unique_blues = [];
    for (var key1 in values_object) {
        unique_blues.push(key1);
        for (var key2 in values_object[key1]) {
            if (unique_browns.indexOf(key2) == -1)
                unique_browns.push(key2);
        }
    }
    if (unique_blues.length < 2 || unique_browns < 2) {
        log("There are not enough results to tabulate. Select several tables or pages in your Report and try again.");
        return;
    }
    var summary_position = unique_browns.indexOf("SUMMARY");
    if (summary_position > -1 && summary_position != 0) {
        unique_browns.splice(summary_position, 1);
        unique_browns.splice(0,0,"SUMMARY");
    }
    var values_array_expression = 'c('
    for (var key1 in values_object) {
        unique_browns.forEach(function (key2) {
            if (typeof values_object[key1][key2] == 'undefined' || values_object[key1][key2] == null) {
                values_array_expression += 'NA,';
            } else {
                values_array_expression += values_object[key1][key2] + ','; 
            }
        });
    }
    values_array_expression = values_array_expression.substring(0, values_array_expression.length - 1) + ')'
    var item_name = generateUniqueRObjectName('most.significant.results')
    var r_expression = item_name + ' = matrix(' 
                     + values_array_expression 
                     + ', nrow = ' 
                     + unique_blues.length 
                     + ', byrow = TRUE, dimnames = list(c(' 
                     + unique_blues.map(function (str) { return '"' + str + '"';}).join(",") 
                     + '), c(' 
                     + unique_browns.map(function (str) { return '"' + str + '"';}).join(",") 
                     +  ') ));'
    
    
    if (web_mode){
        // Create page and title
        const pageName = "Most Interesting Results"
        const page = project.report.appendPage('TitleOnly');
        page.group.moveAfter(page, null);
        page.name = pageName;
        var titleText = page.subItems[0];
        titleText.text = pageName;
        

        var results_item = page.appendR(r_expression);

        var descriptive_text = page.appendText();
        descriptive_text.top = results_item.top + results_item.height + 5;
        descriptive_text.text = "Each of the input tables has been given a score between 0 and 5, with higher scores indicating that the table contains a more interesting result, and lower scores indicating that the table does not contain any interesting results."

        log("A table showing the most interesting results has been added to the top of your Document.");    
    } else {
        var results_item = project.report.appendR(r_expression);
        project.report.moveAfter(results_item, null);    
    }
    
}

See also

• QScript for more general information about QScripts.
• QScript Examples Library for other examples.
• Online JavaScript Libraries for the libraries of functions that can be used when writing QScripts.
• QScript Reference for information about how QScript can manipulate the different elements of a project.
• JavaScript for information about the JavaScript programming language.
• Table JavaScript and Plot JavaScript for tools for using JavaScript to modify the appearance of tables and charts.