Create New Variables - Contrast Periods

This tool allows you to create a new QuestionVariable Set that compares two date periods. This enables you to build a table (or visualization) that focuses on two date periods of interest and includes a statistical comparison between the two. This is common in tracking studies, for example, when wanting to display the latest results along with a statistical comparison to the last period or the same period last year.

You can choose the period to Compare and the period to compare against (Versus) in the settings for the new QuestionVariable Set (see below). When placed in the Blue drop-downColumns of a table with another QuestionVariable Set in the Brown drop-downRows, the table will display results for the two columns of interest and the statistical testing in the table will reflect the comparison between the Compare period and the Versus period.

Example

This table shows the proportion of survey respondents who said they would consider eating from a set of different fast-food chains, by year:

The settings for statistical testing have been selected to highlight results that are significant compared to the previous period.

Using the Contrast Periods tool allows us to make a table which selects the period of interest, in this case 2017 and compare it to 2016:

.

Here, the test results are the same as the original table. However, if we instead wish to compare 2017 to 2015, a comparison which is not available in the original table except via the Column Comparisons statistic, we can change the Versus period to 2015, result in a table (or visualization) which shows these comparisons:

.

Usage

1. Select a Date question in the Variables and Questions tab.
2. Select Automate > Browse Online Library > Create New Variables > Contrasts.

To change the time periods that are being compared:

1. Select the Contrasts Question in the Variables and Questions tab.
2. Right-click and select Edit R Variable.
3. Change the settings.
4. Click Update R Variable.

1. Select a Date variable under Data Sets.
2. Click the variable-hover (+) button to the right of your variable(s).
3. Select Ready-Made New Variables > Contrasts.

To change the time periods that are being compared, select the Contrasts variable set under Data Sets and then use the settings in the Inputs tab on the right hand side.

Settings

Date variable This is the variable that will be used to identify the periods to use in the contrast. Date aggregation Choose the level of aggregation for the periods you wish to compare. This can be Year, Quarter, Month, 4-Week, 2-Week, or Week. Compare This is the date period of interest to be shown in tables or charts. Versus This is the date period against which the results for Compare should be compared.

Technical details

The date periods available in the Compare and Versus menus are generated at the time you first use this tool. They are evaluated relative to the dates in your data set, and they are extended for a period of three years to allow you to continue to use the same Contrasts as you continue to update your data set with new data.

The Contrasts variable contains two categories, but only one of them is shown. Date values which are not in the Compare or Versus periods are assigned a missing value. The resulting statistical tests are then equivalent to a comparison between Compare and Versus.

How to apply this QScript

• Start typing the name of the QScript into the Search features and data box in the top right of the Q window.
• Click on the QScript when it appears in the QScripts and Rules section of the search results.

OR

• Select Automate > Browse Online Library.
• Select this QScript from the list.

Customizing the QScript

This QScript is written in JavaScript and can be customized by copying and modifying the JavaScript.

Customizing QScripts in Q4.11 and more recent versions

• Start typing the name of the QScript into the Search features and data box in the top right of the Q window.
• Hover your mouse over the QScript when it appears in the QScripts and Rules section of the search results.
• Press Edit a Copy (bottom-left corner of the preview).
• Modify the JavaScript (see QScripts for more detail on this).
• Either:
  • Run the QScript, by pressing the blue triangle button.
  • Save the QScript and run it at a later time, using Automate > Run QScript (Macro) from File.

Customizing QScripts in older versions

• Copy the JavaScript shown on this page.
• Create a new text file, giving it a file extension of .QScript. See here for more information about how to do this.
• Modify the JavaScript (see QScripts for more detail on this).
• Run the file using Automate > Run QScript (Macro) from File.

JavaScript

includeWeb('QScript Selection Functions');
includeWeb('QScript Utility Functions');
includeWeb('QScript Value Attributes Functions');


Array.prototype.max = function() {
  return Math.max.apply(null, this);
};

Array.prototype.min = function() {
  return Math.min.apply(null, this);
};

const MONTH_NAMES = ["January", "February", "March", "April", "May", "June",
  "July", "August", "September", "October", "November", "December"
];

const QUARTER_NAMES = ["Q1", "Q2", "Q3", "Q4"];

function arrayToJSString(array) {
    return "[" + array.map(x => '"' + x + '"').join(", ") + "]";
}


function getMonday(d) {
    d = new Date(d);
    let day = d.getDay();
    let diff = d.getDate() - day + (day == 0 ? -6:1); // adjust when day is sunday
    return new Date(d.setDate(diff));
}

function getFirstDayOfMonth(d) {
    let date = new Date(d);
    let firstDay = new Date(date.getFullYear(), date.getMonth(), 1)
    return firstDay.getTime();
}

function getFirstDayOfQuarter(x) {
    let d = new Date(x);
    let quarter = Math.floor((d.getMonth() / 3));
    let firstDate = new Date(d.getFullYear(), quarter * 3, 1);
    return firstDate.getTime();
}

function getFirstDayOfYear(x) {
    let d = new Date(x);
    let firstDate = new Date(d.getFullYear(), 0, 1);
    return firstDate.getTime();
}

function getDateInNYears(x, n) {
    let d = new Date(x);
    let year = d.getFullYear();
    let month = d.getMonth();
    let day = d.getDate();
    let c = new Date(year + n, month, day);
    return c.getTime();
}

function addMonths(date, months) {
    var d = date.getDate();
    date.setMonth(date.getMonth() + months);
    if (date.getDate() != d) {
      date.setDate(0);
    }
    return date.getTime();
}

function addWeeks(x, n) {
    var d = new Date(x);
    d.setDate(d.getDate() + (n * 7) );
    return d.getTime();
}


function generateDateSequence(start, end, n, period) {
    let new_dates = [start];
    let current_date = start;
    while (current_date < end) {
        current_date = incrementDateBy(current_date, n, period)
        new_dates.push(current_date);
    }
    return new_dates;
}

function generateMultiWeekSequence(max_date, min_date, end, n) {
    let new_dates = [];
    let d = getMonday(max_date); 
    d.setDate(d.getDate() + 7); //Next monday after max_date
    let post_sequence = generateDateSequence(d.getTime(), end, n, "week");  

    // Working backwards in time
    d.setDate(d.getDate() - n * 7);
    while(d.getTime() > min_date) {
        new_dates.push(d.getTime());
        d.setDate(d.getDate() - n * 7);
    }

    new_dates.sort();
    return new_dates.concat(post_sequence);    
}


function incrementDateBy(x, n, period) {
    let date = new Date(x);
    let new_date;
    if (period == "week") {
        new_date = addWeeks(date, n);
    } else if (period == "month") {
        new_date = addMonths(date, n);
    } else if (period == "year") {
        new_date = getDateInNYears(x, n);
    }
    return new_date;

}

function getMonthString(x) {
    let d = new Date(x);
    return MONTH_NAMES[d.getMonth()] + " " + d.getFullYear();
}

function getQuarterString(x) {
    let d = new Date(x);
    return QUARTER_NAMES[Math.floor((d.getMonth() / 3))] + " " + d.getFullYear();
}

function getYearString(x) {
    let d = new Date(x);
    return d.getFullYear().toString();
}

function getWeekString(x, n) {
    let d = new Date(x);
    let month = (d.getMonth() + 1).toString();
    month = month.length == 1 ? "0" + month : month;
    let day =  d.getDate().toString();
    day = day.length == 1 ? "0" + day : day
    let start_label =  d.getFullYear() + "-" + month  + "-" + day;
    return n == 1 ? "Week commencing " + start_label : n + " weeks commencing " + start_label;
}

function createContrastPeriodsVariable() {


    const q_allowed_version = 20.14;
    const is_displayr = inDisplayr();
    if (!is_displayr && Q.fileFormatVersion() < q_allowed_version) {
        log("This feature requires a newer version of Q. Please contact support@q-researchsoftware.com");
        return false;
    }

    const user_selections = getAllUserSelections();
    let selected_questions = user_selections.selected_questions;
    let n_selected_questions = selected_questions.length;


    let selected_data_files = user_selections.selected_data_sets;
    if (selected_data_files.length > 1) {
        log('The selected variables come from more than one Data Set. Select variables from the same dataset and and run this feature again.');
        return false;
    }
    let data_file = selected_data_files[0];

    if (n_selected_questions === 0) {
        if (is_displayr) {
            log('No variables are selected. Select a Date variable under Data Sets and run this feature again.');
            return false;    
        } else {
            data_file = requestOneDataFileFromProject();
            let candidate_questions = data_file.questions;
            candidate_questions = candidate_questions.filter(function (q) {
                return !q.isHidden && q.isValid && q.questionType == "Date";
            });
            let prompt = correctTerminology("Select a Date variable set to use for contrasts:");
                selected_questions = [selectOneQuestion(prompt, candidate_questions, true)];
            n_selected_questions = selected_questions.length;
        }
        
    }

    if (selected_questions.some(v => v.isHidden)) {
        log("Some of the selected variables are hidden. Unhide these variables and then run this option again.");
        return;
    }


    // Compute date defaults for combo boxes and prefil for 3 years
    let date = selected_questions[0];
    if (date.questionType != "Date") {
        log(correctTerminology("The selected variable is not a Date. Select a Date variable and run this option again."));
        return false;
    }

    let num_date = date.uniqueValues;
    num_date = num_date.filter(function(x) { return !isNaN(x); });

    // Identify first monday before or equal to first date


    let min_date = num_date.min();
    let max_date = num_date.max();
    let first_monday = getMonday(min_date).getTime()
    let first_of_month = getFirstDayOfMonth(min_date);
    let three_years_from_max = getDateInNYears(max_date, 3);
    let first_day_of_quarter = getFirstDayOfQuarter(min_date);
    let first_day_of_year = getFirstDayOfYear(min_date);


    let month_sequence = generateDateSequence(first_of_month, three_years_from_max, 1, "month");
    let week_sequence =    generateDateSequence(first_monday, three_years_from_max, 1, "week");
    let two_week_sequence = generateMultiWeekSequence(max_date, min_date, three_years_from_max, 2);
    let four_week_sequence = generateMultiWeekSequence(max_date, min_date, three_years_from_max, 4);
    let quarter_sequence = generateDateSequence(first_day_of_quarter, three_years_from_max, 3, "month");
    let year_sequence = generateDateSequence(first_day_of_year, three_years_from_max, 1, "year");

    let month_labels = month_sequence.map(getMonthString);
    let quarter_labels = quarter_sequence.map(getQuarterString);
    let year_labels = year_sequence.map(getYearString);
    let week_labels = week_sequence.map(x => getWeekString(x, 1));
    let two_week_labels = two_week_sequence.map(x => getWeekString(x, 2));
    let four_week_labels = four_week_sequence.map(x => getWeekString(x, 4));

    function getMostRecentPeriodsInSequence(date_sequence, last_date) {
        let ind = date_sequence.findIndex(x => x > last_date) - 1;
        if (ind > 0) {
            return date_sequence.slice(ind-1, ind + 1);
        } else {
            return date_sequence.slice(ind, ind + 2);
        }
    }

    let week_defaults = getMostRecentPeriodsInSequence(week_sequence, max_date).map(x => getWeekString(x, 1));
    let two_week_defaults = getMostRecentPeriodsInSequence(two_week_sequence, max_date).map(x => getWeekString(x, 2));
    let four_week_defaults = getMostRecentPeriodsInSequence(four_week_sequence, max_date).map(x => getWeekString(x, 4));
    let month_defaults = getMostRecentPeriodsInSequence(month_sequence, max_date).map(getMonthString);
    let quarter_defaults = getMostRecentPeriodsInSequence(quarter_sequence, max_date).map(getQuarterString);
    let year_defaults = getMostRecentPeriodsInSequence(year_sequence, max_date).map(getYearString);

    let ui_code = 

`
form.dropBox({label: "Date variable",
              name: "formDate",
              types: ["Q:Date"]})
let agg_box = form.comboBox({label: "Date aggregation", 
                             name: "formAggregation",
                             alternatives: ["Week", 
                                            "2-Week", 
                                            "4-Week", 
                                            "Month", 
                                            "Quarter", 
                                            "Year"],
                             default_value: "Month"});

let month_labels = ${arrayToJSString(month_labels)};
let quarter_labels = ${arrayToJSString(quarter_labels)};
let year_labels = ${arrayToJSString(year_labels)};
let week_labels = ${arrayToJSString(week_labels)};
let two_week_labels = ${arrayToJSString(two_week_labels)};
let four_week_labels = ${arrayToJSString(four_week_labels)};

let week_defaults = ${arrayToJSString(week_defaults)};
let two_week_defaults = ${arrayToJSString(two_week_defaults)};
let four_week_defaults = ${arrayToJSString(four_week_defaults)};
let month_defaults = ${arrayToJSString(month_defaults)};
let quarter_defaults = ${arrayToJSString(quarter_defaults)};
let year_defaults = ${arrayToJSString(year_defaults)};

let compare_list;
let default_selections;

switch(agg_box.getValue()) {
    case "Week": 
        compare_list = week_labels; 
        default_selections = week_defaults;
        break;
    case "2-Week": 
        compare_list = two_week_labels; 
        default_selections = two_week_defaults;
        break;
    case "4-Week": 
        compare_list = four_week_labels; 
        default_selections = four_week_defaults;
        break;
    case "Month": 
        compare_list = month_labels; 
        default_selections = month_defaults;
        break;
    case "Quarter": 
        compare_list = quarter_labels; 
        default_selections = quarter_defaults;
        break;
    case "Year": 
        compare_list = year_labels;
        default_selections = year_defaults;
}

form.comboBox({label: "Compare", 
               name: "formCompare", 
               alternatives: compare_list,
               default_value: default_selections[1]});
form.comboBox({label: "Versus", 
               name: "formVersus", 
               alternatives: compare_list,
               default_value: default_selections[0]});


`

    let r_code = 
`
library(flipTime)
agg <- tolower(formAggregation)
long.name <- TRUE

if (agg == "quarter") {
    agg <- "nice.quarter"
    long.name <- FALSE
} 
anchor.date = max(formDate) + weeks(1) # Used to ensure matches Displayr date aggregation of weeks

# Convert date variable to periods
date.periods = Period(formDate, 
                      by = agg, 
                      anchor.date = anchor.date, 
                      long.name = long.name, 
                      week_start = 1)

# Identify the two periods of interest as categories
contrasts <- rep(NA, length(formDate))
contrasts[date.periods == formCompare] <- 2
contrasts[date.periods == formVersus] <- 1
if (Sum(date.periods == formCompare) == 0) warning("There are no observations in ", formCompare)
if (Sum(date.periods == formVersus) == 0) warning("There are no observations in ", formVersus)
out = factor(contrasts, 
            levels = 1:2,
            labels = c(formVersus, formCompare))
`

    if (test_mode) {
        let new_item = project.currentPage().appendR(r_code);
        new_item.codeForGuiControls = ui_code;    
    } else {
        let new_q = data_file.newRQuestion(r_code, preventDuplicateQuestionName(data_file, date.name + " contrasts"), 
                                             "contrast"+makeid(), date.variables[0], ui_code, {'formDate': date.guid});
        new_q.variableSetStructure = "Ordinal";
        let data_reduction = new_q.dataReduction;
        let labels = data_reduction.rowLabels; 
        data_reduction.hide(labels[2]); //Will always be the NET
        if (is_displayr)
            project.report.setSelectedRaw([new_q]);
        else {
            let new_table = project.report.appendTable();
            new_table.primary = new_q;
            project.report.setSelectedRaw([new_table]);
        }
    }



}

createContrastPeriodsVariable(test_mode = false);

See also

• QScript for more general information about QScripts.
• QScript Examples Library for other examples.
• Online JavaScript Libraries for the libraries of functions that can be used when writing QScripts.
• QScript Reference for information about how QScript can manipulate the different elements of a project.
• JavaScript for information about the JavaScript programming language.
• Table JavaScript and Plot JavaScript for tools for using JavaScript to modify the appearance of tables and charts.