Choice Modeling - Analyze with Experiment Question (Legacy) - Experiment Question from Sawtooth CHO File

This QScript sets up an Experiment question for a choice-based conjoint experiment where the data for the experiment is contained in a Sawtooth .CHO format file.

Before using this QScript you must:

Convert the .CHO file to an Excel file. See below for instructions.
Import a survey data file in to Q. This file must contain IDs for each individual (or respondent) which match those from the choice data in the CHO file.

The QScript will add the data from the CHO file to your survey data so that you can analyse the results of the experiment together with the other data that has been collected from the individuals. In Q 4.11.11 and later you do not need to have a survey data file, and you can create a Q Project from the CHO file directly by running this QScript in a new Q window.

When you run this script you will be prompted to:

Select the Excel file which contains the CHO data.
(Optional) Select an Excel file which contains the labels for the attributes that were asked about in the conjoint experiment.
Choose whether or not to add alternative-specific constants.
Select the variable which contains the individuals' IDs.

Technical details

This QScript requires that you have two different files:

An Excel file containing the data from a Sawtooth CHO format file. See below for how to convert the CHO file to an Excel file.
A survey data file that the Experiment question will be added to. See below for the minimum requirements of this file. In Q 4.11.11 and later versions this file is no longer required, and the choice data can be added using only the CHO file.

Begin with your survey data file imported into a Q project, and then run this QScript.

Each task shown to the individuals should have the same number of alternatives, and each alternative should contain the same set of attributes.

Converting the CHO File

The CHO file produced by Sawtooth is a plain text file that can be opened in Excel. Follow the steps below to convert the CHO file into a file that Q can read.

Open the .cho file in Excel
In the Text Import Wizard, select Original Data Type > Delimited. Make sure Start at import row is 1 and that My data has headers is NOT ticked.
Click Next.
Select Delimiters > Space and deselect all other options.
Click Finish.
Use File > Save As and save the file as an Excel Workbook (.xlsx format).

The importing options in Excel look like those shown in the picture below.

Once in the correct format, the Excel file should look like the following example.

Survey Data File

The data file in your project needs to contain at least one variable which contains the individuals' IDs.

Attribute Labels

The file containing the labels for the levels should contain one column for each attribute, with the labels of each level for that attribute labeled in order. For example:

If no file is specified then the attribute levels will be labeled with the numbers as they appear in the experimental design. These can be changed later by highlighting the variables for a single attribute in the Variables and Questions tab and editing the Values (...) to update the labels.

Extra Variables

In addition to the experimental design and choices made by the individuals, CHO files can also contain additional variables (e.g. the duration of the survey, or demographic or segmentation variables). These can also be imported along with the experiment, and you will be given the option of importing them when you run the script. CHO files do not contain any metadata (labels, variable type, etc) for these variables, and so they will initially appear as Numeric variables in Q.

None of These Options

Sawtooth experiments can contain a dual-response option, where the individual is asked to choose which alternative they prefer, and subsequently asked whether they would actually purchase that alternative. Q does not support this kind of experiment.

How to apply this QScript

Start typing the name of the QScript into the Search features and data box in the top right of the Q window.
Click on the QScript when it appears in the QScripts and Rules section of the search results.

OR

Select Automate > Browse Online Library.
Select this QScript from the list.

Customizing the QScript

This QScript is written in JavaScript and can be customized by copying and modifying the JavaScript.

Customizing QScripts in Q4.11 and more recent versions

Start typing the name of the QScript into the Search features and data box in the top right of the Q window.
Hover your mouse over the QScript when it appears in the QScripts and Rules section of the search results.
Press Edit a Copy (bottom-left corner of the preview).
Modify the JavaScript (see QScripts for more detail on this).
Either:
- Run the QScript, by pressing the blue triangle button.
- Save the QScript and run it at a later time, using Automate > Run QScript (Macro) from File.

Customizing QScripts in older versions

Copy the JavaScript shown on this page.
Create a new text file, giving it a file extension of .QScript. See here for more information about how to do this.
Modify the JavaScript (see QScripts for more detail on this).
Run the file using Automate > Run QScript (Macro) from File.

JavaScript

includeWeb('QScript Selection Functions');
includeWeb('QScript Utility Functions');
includeWeb('QScript Functions for Setting Up Experiments');
includeWeb('QScript Functions to Generate Outputs');
 
Array.prototype.max = function() {
    return Math.max.apply(null, this);
};
 
Array.prototype.min = function() {
    return Math.min.apply(null, this);
};
 
 
try {
    if (main())
        conditionallyEmptyLog("QScript Finished.");
    else
        log("QScript Cancelled.")
} catch (e) {
    if (e instanceof SetupError)
        log(e.message);
    else
        throw e;
}
 
 
function main() {
 
    var original_data_files = project.dataFiles;

    // You can't start a project with a CHO data file until Q4.11.11 or later
    if (project.dataFiles.length == 0) {
        if (fileFormatVersion() < 9.11)
            throw new SetupError("This QScript requires you to have at least one data file in your project to add your experimental data to.");
    }

    var design_file = project.addDataFileDialog("Select the Excel File Containing the CHO Data", {col_names_included: false});

    // In newer versions of Q we can offer to add the choice data in place in the CHO design file.
    var candidate_data_files = fileFormatVersion() < 9.11 ? original_data_files : project.dataFiles;
    var target_data_file = candidate_data_files.length == 1 ? candidate_data_files[0] : selectOneDataFile("Which data file do you want to add the Conjoint Experiment to?", candidate_data_files);
    var add_variables_to_design = target_data_file.equals(design_file); 

 
    var design_data = scrapeDataFromCHOFile(design_file);
    if (!add_variables_to_design)
        design_file.remove();
    else 
        design_file.questions.forEach(function (q) { q.isHidden = true; });
 
    var attribute_labels = null;
    var level_labels = null;
    var import_labels = askYesNo("Do you want to import attribute labels from an external file?");
    if (import_labels) {
        var label_file = project.addDataFileDialog("Select the Excel File Containing the Attribute Labels", {col_names_included: true});
        attribute_labels = label_file.variables.map(function (v) { return v.label; });
        level_labels = scrapeLabelFile(label_file);
        label_file.remove();
    }
 
    // Do you want to add variables for alternative-specific constants?
    var add_asc = askYesNo("Do you want to include alternative-specific constants?");
 
    var add_extra_vars = true; //askYesNo("Do you want to add data from any Extra Variables included with the experimental design?");
 
    // Check consistency of design paramaters and get numbers of tasks, alternatives, attributes for the design overall.
    var design_specs = getDesignSpecs(design_data);
 
    // Identify the ID variable
    var id_var;
    if (add_variables_to_design)
        id_var = createIDVariable(design_file, design_data);
    else {
        var candidate_id_vars = target_data_file.variables.filter(function (v) { return !v.question.isHidden; });
        id_var = selectOneVariableByName("Select the variable that contains the individuals' ID numbers.", candidate_id_vars, true);    
    }
 
    // Create experiment question
    var experiment_question = createExperimentFromCHODesign(design_data, 
                                                            design_specs, 
                                                            add_asc, 
                                                            id_var, 
                                                            import_labels ? attribute_labels : null, 
                                                            import_labels ? level_labels : null);
    // Generate extra variables
    if (add_extra_vars) {
        var extra_vars = addExtraVariables(design_data, id_var);
    }
 
 
    // Generate outputs
    var new_group = project.report.appendGroup();
    new_group.name = "Conjoint Experiment";
    var new_text = new_group.appendText();
    var new_html = Q.htmlBuilder();
 
    new_html.appendParagraph("The experiment has " + design_specs.numAtts + " attributes, " + design_specs.numTasks + " tasks for each individual, and " + design_specs.numAlts + " alternatives per task.\r\n", { font: 'Times New Roman', size: 12 });
    if (add_extra_vars && extra_vars.length > 0)
        new_html.appendParagraph(extra_vars.length == 1 ? "One extra variable has been included from the CHO file." : (extra_vars.length + "extra variables have been included from the CHO file."), { font: 'Times New Roman', size: 12 });
    new_text.content = new_html;
    var html_title = Q.htmlBuilder();
    html_title.appendParagraph("Experiment");
    new_text.title = html_title;
    var new_table = new_group.appendTable();
    new_table.primary = experiment_question;
    new_table.changeDecimalPlacesBy(3);
    if (add_extra_vars)
        extra_vars.forEach(function (v) {
            var t = new_group.appendTable();
            t.primary = v.question;
        });

    return true;
 
 
}
 
function createIDVariable(data_file, design_data) {
    if (fileFormatVersion() < 9.11)
        throw new SetupError("This functionality is not available in the current version of Q.");
    var ids = design_data.map(function (obj) { return obj.id; });
    var raw_file_length = data_file.variables[0].rawValues.length;
    while (ids.length < raw_file_length)
        ids.push(NaN);
 
    // Create ID variable
    var expression = "[" + ids.join(",") + "]";
    var new_var_name = preventDuplicateVariableName(data_file, "CHO_ID");
    var id_variable = data_file.newJavaScriptVariable(expression, false, new_var_name, "ID", null, { accessAllRows: true } );
    return id_variable;
}
 
 
function addExtraVariables(design_data, id_variable) {
    var extra_var_data = design_data.map(function (obj) { return obj.extraVars; });
    var data_file = id_variable.question.dataFile;
 
    // If the number of extra variables differs between individuals, then there is
    // no guarantee as to the order of the values. I expect that this will never happen
    // but if it does then we will need to look carefully at the example data to
    // determine if it can be set up properly.
    var num_extras = extra_var_data.map(function (a) { return a.length; });
    var max_extras = num_extras.max();
    var min_extras = num_extras.min();
    if (max_extras != min_extras)
        throw new SetupError("The number of extra variables differs between individuals and cannot be correctly interpreted. Please contact Q Support.");
 
 
    extra_var_data = Q.transpose(extra_var_data);
    var extra_variables = [];
    var last_var = null;
 
 
    var ids_expression = "var _ids = [" + design_data.map(function (obj) { return obj.id; }) + "];\r\n";
    extra_var_data.forEach(function (data, ind) {
        var v_name = preventDuplicateVariableName(data_file, "extra_variable_" + (ind+1));
        var v_label = "Extra Variable " + (ind+1);
        var data_expression = "var _data = [" + data.join(",") + "];\r\n";
        var expression = ids_expression + data_expression + "_data[_ids.indexOf(" + id_variable.name + ")];";
        var new_var = data_file.newJavaScriptVariable(expression, false, v_name, v_label, last_var);
        extra_variables.push(new_var);
        last_var = new_var;
    });
    return extra_variables;
}
 
 
function generateAlternativeSpecificConstantVariables(num_tasks, num_alts, has_none_of_these, id_variable, last_var) {
    var alternative_vars = [];
    var data_file = id_variable.question.dataFile;
    for (task = 0; task < num_tasks; task++)
        for (var alt = 1; alt < num_alts + 1; alt++) {
            var expression = "isNaN(" + id_variable.name + ") ? NaN : " + alt;
            var new_var_name = preventDuplicateVariableName(data_file, "alternative_task" + (task+1) + "_alt" + alt);
            var new_var = data_file.newJavaScriptVariable(expression, false, new_var_name, "Alternative", last_var);
            new_var.variableType = "Categorical";
            if (has_none_of_these && alt == num_alts) {
                var value_attributes = new_var.valueAttributes;
                value_attributes.setLabel(num_alts, "None of these");
            }
            alternative_vars.push(new_var);
            last_var = new_var;
        }
    return alternative_vars;
}
 
function createExperimentFromCHODesign(design_data, 
                                       design_specs, 
                                       include_alternative_specific_constants, 
                                       ID_variable, 
                                       attribute_labels, 
                                       level_labels) {
 
    var has_none = design_specs.hasNone
    var num_alts = design_specs.numAlts + has_none;
    var num_tasks = design_specs.numTasks;
    var num_atts = design_specs.numAtts;
 
 
    var data_file = ID_variable.question.dataFile;
    // Define the array of IDs
    // This will be used to match data from the design
    // with data from the survey.
    ids_expression = "var _ids = [" + design_data.map(function (obj) { return obj.id; }) + "];\r\n";
 
    var experiment_vars = [];
    var last_var = null;
 
 
    // 1 Add Choice Variables
    var choice_vars = [];
    for (var j = 0; j < num_tasks; j++) {
        var choice_expression = "var _choices = [" + design_data.map(function (obj) { return obj.choices[j]; }) + "];\r\n";
        var expression = ids_expression 
                         + choice_expression
                         + "_choices[_ids.indexOf(" + ID_variable.name + ")];";
        var v_name = preventDuplicateVariableName(data_file, "choice" + (j+1));
        var new_var = data_file.newJavaScriptVariable(expression, false, v_name, "Choice", last_var);
        new_var.variableType = "Categorical";
        choice_vars.push(new_var);
        last_var = new_var;
    }
 
    experiment_vars = experiment_vars.concat(choice_vars);
 
 
    // 2 Add Alt-specific constants if required
    if (include_alternative_specific_constants) {
        var alt_specific_vars = generateAlternativeSpecificConstantVariables(num_tasks, 
                                                                             num_alts, 
                                                                             has_none, 
                                                                             ID_variable, 
                                                                             last_var);
        last_var = alt_specific_vars[alt_specific_vars.length - 1];
        experiment_vars = experiment_vars.concat(alt_specific_vars);
    }
 
 
    // 3 Add variables for attributes in design
    var cols_per_task = num_alts*num_atts;
    for (att = 0; att < num_atts; att++) {
        var attribute_label = attribute_labels == null ? "Attribute " + (att+1) : attribute_labels[att];
        var attribute_name =  "attribute" + (att+1);
        for (task = 0; task < num_tasks; task ++) {
            for (alt = 0; alt < num_alts; alt++) {
                var index = (task * cols_per_task) + (alt * num_atts) + att;
                var cur_data = design_data.map(function (obj) { return obj.levelData[index]; });
                var data_name = "_task" + (task+1) + "alt" + (alt+1) + ""
                var attribute_data_expression = "var " + data_name + " = [" + cur_data + "];\r\n";
                var expression = ids_expression + attribute_data_expression + data_name + "[_ids.indexOf(" + ID_variable.name + ")];";
                var new_var_name = preventDuplicateVariableName(data_file, attribute_name + "_task" + (task+1) + "_alt" + (alt+1) );
                var new_var = data_file.newJavaScriptVariable(expression, false, new_var_name, attribute_label, last_var, true);
                new_var.variableType = "Categorical";
                var value_attributes = new_var.valueAttributes;
                var unique_values = new_var.uniqueValues;
                var max_val = unique_values.filter(function (x) { return !isNaN(x); }).max();
                if (level_labels != null && level_labels[att].length < max_val)
                    throw new SetupError("The number of levels in " + attribute_label + " is " + level_labels[att].length + " but the design refers to level " + max_val);
                unique_values.forEach(function (x) {
                    if (!isNaN(x))
                        value_attributes.setLabel(x, level_labels == null ? ("Level " + x) : level_labels[att][x-1]);    
                    else 
                        value_attributes.setIsMissingData(NaN, true);
                });
                experiment_vars.push(new_var);
                last_var = new_var;
            }
        }
    }
 
    // Set the question
    var experiment_name = preventDuplicateQuestionName(data_file, "Conjoint Experiment");
    var new_question = data_file.setQuestion(experiment_name, "Experiment", experiment_vars);
    return new_question;
}
 
 
 
function getDesignSpecs(design_data) {
 
 
    // Check that number of alternatives per task is constant
    // In the future we may wish to relax this restriction by
    // being more clever about how we scrape out the design info
    var max_alts = design_data[0].altsPerTask.max();
    var min_alts = design_data[0].altsPerTask.min();
    design_data.forEach(function (obj) {
        if (max_alts < obj.altsPerTask.max())
            max_alts = obj.altsPerTask.max();
        if (min_alts > obj.altsPerTask.min())
            min_alts = obj.altsPerTask.min();
 
        if (max_alts != min_alts)
            throw new SetupError("The number of alternatives differs between tasks.");
    });
 
    // Check that the number of attributes is the same
    // across all individuals
    var attribute_numbers = design_data.map(function (obj) { return obj.numAttributes; });
    if (unique(attribute_numbers).length != 1)
        throw new SetupError("The number of attributes differs between tasks.")
 
 
    // If any individuals have fewer numbers of tasks we can fill out their design
    // info with NaNs
    var num_tasks = design_data.map(function (obj) { return obj.numTasks; });
    var max_num_tasks = num_tasks.max();
    var data_lengths = design_data.map(function (obj) {
        return obj.levelData.length;
    });
    var max_length = data_lengths.max();
    if (data_lengths.some(function (x) { return x < max_length; })) {
        design_data.forEach(function (obj) {
            while (obj.levelData.length < max_length)
                obj.levelData.push(NaN);
        });
    }
 
    // Check "none of these" settings
    // may need to accomodate "Dual-response none" option in future.
    var none_options = design_data.map(function (obj) { return obj.hasNone; });
    var has_none_of_these = none_options.some(function (x) { return x == 1; });
 
    return { numAlts: max_alts, numAtts: attribute_numbers[0], numTasks: max_num_tasks, hasNone: has_none_of_these };
}