Marketing - MaxDiff - Experimental Design
Creates an experimental design for MaxDiff.
Example
In Displayr, the MaxDiff Experimental Design is created by selecting Insert > More > MaxDiff > Experimental Design.
To run the MaxDiff Experimental Design in Q, select Create > Marketing > MaxDiff > Experimental Design.
The following table contains an experimental design whereby respondents were shown 10 questions, each of which contains 4 options, with the experiment designed to assess a total of 8 alternatives (e.g., 8 different products). In the 6th question, for example, the respondent is to be asked which they like most and which they like least of alternatives D, A, H and c.
Options
The inputs used to generate the MaxDiff design are shown below.
Number of alternatives The number of alternatives in the experiment. For example, if you are doing a study investigating preferences for 10 brands, then 10 is the number of alternatives.
Add alternative labels Optional labels for the alternatives specified in the first column of the data editor.
Alternatives per question For example, if you have a study of 10 brands, and in each question you show five brands, asking the respondent to choose the one of the five that they like the most and the one that they like the least, then Alternatives per question = 5.
Questions The number of MaxDiff questions to show to respondents. Sawtooth Software suggests that a rough guideline is:
Number of questions >= 3 * Number of alternatives / Alternatives per question
Versions The number of versions of the experimental design (defaults to 1). Where this is set to 2 or more, subsequent versions (after the first version) are obtained by permuting the columns of the binary design while maintaining pairwise balance.
Repeats The number of times that the algorithm seeks to find a solution. The higher the number, the greater the chance that the best possible solution is found. For most problems, this makes little difference (i.e., a marginally sub-optimal experimental design will tend not to have any meaningful consequence on the conclusions drawn from the analyses).
Random order of alternatives Randomize the order of the alternatives within each question. If not checked, alternatives for each question are shown in the order of the labels (or numeric order if no labels are provided).
Detailed outputs If selected, various technical outputs shown below are provided (otherwise, the table above is shown).
Detailed Outputs
The first output is a table showing the design as binary variables. For more than one version this is permuted for each version, hence only the first version is shown.
$binary.design
Version Question 1 2 3 4 5 6 7 8
[1,] 1 1 1 0 0 1 1 0 1 0
[2,] 1 2 0 0 1 0 1 1 0 1
[3,] 1 3 0 0 1 1 0 1 1 0
[4,] 1 4 0 1 1 1 1 0 0 0
[5,] 1 5 0 1 0 0 1 0 1 1
[6,] 1 6 1 0 1 1 0 0 0 1
[7,] 1 7 1 1 0 0 1 1 0 0
[8,] 1 8 1 0 0 0 0 1 1 1
[9,] 1 9 1 1 1 0 0 0 1 0
[10,] 1 10 0 1 0 1 0 1 0 1
The next output is the design described above except without labels (if supplied).
The frequencies indicate the number of times each alternative appears in the experiment. In this case, each person sees each alternative five times. Where this number varies substantially, it can be useful to increase the Number of repeats. Again, this and pairwise frequencies below are shown for the first version.
$frequencies 1 2 3 4 5 6 7 8 5 5 5 5 5 5 5 5
The pairwise frequencies show the number of times that each alternative appears with each other alternative. Note that the main diagonal is a the same as the frequencies. Where there is substantial variation on the off-diagonal entries, it can be useful to increase the Number of repeats.
$pairwise.frequencies
Alternative
Alternative 1 2 3 4 5 6 7 8
1 5 2 2 2 2 2 3 2
2 2 5 2 2 3 2 2 2
3 2 2 5 3 2 2 2 2
4 2 2 3 5 2 2 2 2
5 2 3 2 2 5 2 2 2
6 2 2 2 2 2 5 2 3
7 3 2 2 2 2 2 5 2
8 2 2 2 2 2 3 2 5
The final output is a Correlation Matrix showing the correlations between the columns in the binary design for the first version. Ideally, the off-diagonal entries should be close to 0, and should not contain any large values (e.g., values more than 0.5 or less than -0.5). Where there are large values, you can reduce the size of these by increasing the Number of Questions. Where there is variation, it can be useful to increase the Number of repeats. Note that with incomplete block designs it is not possible for the correlations of the off-diagonal entries to be 0.
$binary.correlations
Alternative
Alternative 1 2 3 4 5 6 7 8
1 1.0 -0.2 -0.2 -0.2 -0.2 -0.2 0.2 -0.2
2 -0.2 1.0 -0.2 -0.2 0.2 -0.2 -0.2 -0.2
3 -0.2 -0.2 1.0 0.2 -0.2 -0.2 -0.2 -0.2
4 -0.2 -0.2 0.2 1.0 -0.2 -0.2 -0.2 -0.2
5 -0.2 0.2 -0.2 -0.2 1.0 -0.2 -0.2 -0.2
6 -0.2 -0.2 -0.2 -0.2 -0.2 1.0 -0.2 0.2
7 0.2 -0.2 -0.2 -0.2 -0.2 -0.2 1.0 -0.2
8 -0.2 -0.2 -0.2 -0.2 -0.2 0.2 -0.2 1.0
For more than one version, the following additional outputs are shown:
- versions.binary.correlations This shows the correlations of the design across all the respondents. The more versions, the less variation that will exist.
- versions.pairwise.frequencies This has the same interpretation as above, except that the analysis is across versions. The more versions, the less variation among the off-diagonal cells.
The following outputs are on a scale between 0 and 1 where 1 means perfect balance, i.e. each alternative or pair of alternatives is shown the same number of times. 0 means the worst possible balance, i.e. each question shows the same alternatives.
- mean.version.balance The average balance (between 0 and 1) of alternatives of all versions.
- mean.version.pairwise.balance The average pairwise balance of pairs of alternatives of all versions.
- across.version.balance The balance of alternatives for the whole design across all versions.
- across.version.pairwise.balance The pairwise balance of alternatives for the whole design across all versions.
Finally the original labelled design is shown.
Technical Details
Single version designs are created with the optBlock algorithm from the AlgDesign R package which optimizes the D-criterion of the design. Multiple-version designs are created by permuting the columns of the single version binary.design. A collection of 20 * Versions such permutations are computed, from which versions that maintain the overall pairwise balance of the multi-version design are greedily selected.
More information
See How to Create a MaxDiff Experimental Design in Q for a guide on using this feature.
Code
var heading_text = "MaxDiff Experimental Design";
if (!!form.setObjectInspectorTitle)
form.setObjectInspectorTitle(heading_text);
else
form.setHeading(heading_text);
form.numericUpDown({name: "formAlternatives", label: "Number of alternatives", default_value: 8, increment: 1,
maximum: Number.MAX_SAFE_INTEGER, minimum: 1, prompt: "Number of alternatives in the design"});
form.dataEntry({name: "formLabels",
prompt: "Enter alternative labels (optional)",
required: false,
default_value: [["Enter alternative labels below:"]], label: "Add alternative labels",
edit_label: "Edit alternative labels",
large_data_error: "The data entered is too large. You may only enter data with up to 1000 rows and up to 100 columns."});
form.numericUpDown({name: "formAlternativesPerQuestion", label: "Alternatives per question", default_value: 4, increment: 1, maximum: Number.MAX_SAFE_INTEGER, minimum: 1, prompt: "Number of alternatives per question in the design"});
form.numericUpDown({name: "formQuestions", label: "Questions", default_value: 10, increment: 1, maximum: Number.MAX_SAFE_INTEGER, minimum: 1, prompt: "Number of questions in the design"});
form.numericUpDown({name: "formVersions", label: "Versions", default_value: 1, increment: 1, maximum: Number.MAX_SAFE_INTEGER, minimum: 1, prompt: "Number of versions in the design"});
form.numericUpDown({name: "formRepeats", label: "Repeats", default_value: 1, increment: 1, maximum: Number.MAX_SAFE_INTEGER, minimum: 1, prompt: "Number of times to run the algorithm to find a better design"});
form.checkBox({ name: "formRandomAlternatives", label: "Random order of alternatives", default_value: true, prompt: "If not selected, options for each question are given in the order of their labels" });
form.checkBox({ name: "formDetail", label: "Detailed outputs", default_value: false, prompt: "Display detailed information about the design" });
library(flipMaxDiff)
des <- MaxDiffDesign(number.alternatives = formAlternatives,
alternative.labels = formLabels,
number.questions = formQuestions,
alternatives.per.question = formAlternativesPerQuestion,
n.versions = formVersions,
n.repeats = formRepeats,
random.alternative.order = formRandomAlternatives)
options(width = 300)
max.diff.design <- if(formDetail) des else des$labeled.design
See Also
See MaxDiff Software or What is MaxDiff? for an overview of key MaxDiff concepts and resources.