Online Feature Release Notes

The changes to the online features within Q and Displayr prior to August 2019 are listed below.

For online features released after July 2019 and all changes of the Q application please see here.

For details of new online features released since July 2019, see here.

July 2019

• Marketing - Brand Health Table
• Visualization - Venn Diagram now supports the same color options as the other charts.
• Visualization - Bar Chart and Visualization - Column Chart can be shown using Multiple colors within a series.
• Additional ways to specify the color palette using Custom palette (color pickers) and Custom palette (R output). Sequential color palettes can also be used with Values, an optional numeric table to specify the colors used.
• Dimension Reduction - Correspondence Analysis of a Table using a Bubble Chart output now allows the bubbles to be colored according the values in a user-provided table.
• Update of xgboost to version 0.90.0.1 may cause slight changes to Gradient Boosting outputs.

June 2019

• Visualizations can now switch between multiple templates. Templates have also been changed changed so that Brand colors has been replaced by a palette called Named colors.
• Update to Tensorflow package may cause slight changes in Deep Learning outputs
• Trend lines in visualzations can now be computed using moving averages
• Added TURF - TURF Analysis, TURF - Incrementality Plot and TURF - Upset Plot Standard R features.
• TURF, Dimension Reduction - Principal Components Analysis, t-SNE, Multidimensional Scaling (MDS) and Segments - K-Means Cluster Analysis now accept questions as inputs instead of just variables.
• Added Text Analysis - Automatic Categorization - List of Items Standard R feature, along with the related scripts Text Analysis - Save Variable(s) - Categories, Text Analysis - Save Variable(s) - First Category.

May 2019

• In Displayr, visualizations now allow row and columns to be selected using List Box or Combo Box controls.
• Preliminary Project Setup QScripts can now be run in Displayr to produce summary plots and tables, identify flatlining, check for errors in data files, and more.
• Utilities for "None of these" alternatives have been fixed when running the scripts in Choice Modeling - Save Variable(s) - Utilities.

April 2019

• Data labels in stacked Column Chart are now by default vertically centered.
• Several QScripts for creating variables that previously only worked in Q are now available in Displayr under Insert > Utilties > Create New Variables including Numeric Variable(s) from Code/Category Midpoints, Top 2 Category Variable(s) (Top 2 Boxes), and Case-Level Shares.
• Updated to R 3.6.0. This includes a change to the default method for generating from a discrete uniform distribution to fix an existing bug. This may cause small change in R outputs using using imputed data, resampled (weighted) data, and many of the Machine Learning, Choice Modeling and MaxDiff outputs. To reproduce old results add 'RNGversion("3.5.3")' to the beginning of the R code.
• Visualizations now allows the Legend to be set to Show, to force the legend to always be shown even for a single data series.
• Added Text Analysis - Advanced - Map.
• A template can created (in Displayr, Insert > Utilities > Visualization > Template; in Q, Automate > Browse Online Library > Visualization > Template) to control the fonts and colors used in all visualizations. The template also allows Brand colors to be associated with row/column names in the input data.

March 2019

• Choice Modeling - Hierarchical Bayes now supports choice experiments with a dual-response 'none' question (i.e. questionnaires where the 'None' option is asked as a second stage question separate from the main choice task).
• The mice package used for imputation has been updated to version 3.4.0. Some values may change in R outputs where missing data has been imputed.
• The aspect ratio in radar plots are now fixed at 1 to avoid values along a particular axis looking disproportionately large.
• Visualization - Scatterplot now has controls to adjust marker size.
• Visualization - Time Series Graph now has controls to adjust range on the y-axis.
• Most of the Visualizations (Bar, Column, Area, Line, Radar, Time Series) allow the legend to be orientated vertically (default) or horizontally.
• Updated output of Text Analysis - Advanced - Setup Text Analysis to show both the transformed text and word frequency simultaneously. Tokens in the frequency table and transformed text are color coded for easier reading.

February 2019

• Updated Visualization - Pictograph - Single Icon, Conditional Image, and Visualization - Number so that values can be entered as percentages instead of decimals (e.g "50%" instead of 0.5) into textboxes. This affects a number of controls including Input data, Thresholds to control the fill color and/or image, and Scale and Maximum. The Number type can also be set to "Percentage (no sign)" to hide the percentage sign in the output widget.
• Updated Missing Data - Plot by Case. Label formatting has been improved and now has hovertext and data labels to make it easier to identify missing cases.
• Version variables from response data are now accepted by Choice Modeling - Hierarchical Bayes and Choice Modeling - Latent Class Analysis, as an alternative to task variables.
• Added Choice Modeling - Multinomial Logit and Marketing - MaxDiff - Multinomial Logit, which are equivalent to single-class latent class analysis.
• Visualizations using Regression outputs as the Data Source now shows the importance scores (if available) or coefficients of the regression model.
• Handling of variables as the data source in Visualizations has been improved. A grouping variable can be applied to multiple variables, and categorical variables can be treated as binary variables (previously they were always converted to numerical variables for aggregation).
• For Visualization - Heatmap, convert to percentages/proportions computes the Total percentage instead of the Row percentage.

January 2019

• Added Sparklines and Conditional Image to Visualization.
• A non-backwards compatible change has been made to how percentage data is handled in Visualization - Pictograph - Single Icon, Conditional Image, and Visualization - Number. Percentages from table inputs are now stored as decimals. This means that old widgets with Format > DATA LABEL > Number type set to "Number" (which was previously the default) will have changed outputs, i.e 45% will be shown as '0.45' (or '0' with rounding). Set Format > DATA LABEL > Number type to "Automatic" or "Percentages" to show "45%" instead. (The Standard R page for these items has now been modified so the Number type is by default set to "Automatic", and percentages will automatically have the "%" suffix added).

December 2018

• New feature Choice Modeling - Preview Choice Questionnaire to preview experimental designs output from Choice Modeling - Experimental Design as they would appear to respondents in a questionnaire.
• Improved output formatting for Choice Modeling - Experimental Design.
• Alternative-specific choice designs can be modeled in Choice Modeling - Hierarchical Bayes and Choice Modeling - Latent Class Analysis.
• None-of-these alternatives can be modeled without alternative-specific constants in choice models.
• Labeled-alternatives no longer have alternative-specific constants (bug fix)in Choice Modeling - Hierarchical Bayes and Choice Modeling - Latent Class Analysis.

November 2018

• Added new visualization Parallel Coordinates.
• Options to change colors of Correlation - Correlation Matrix.
• Added Number in Bar to Visualization - Number display options.
• Visualization charts now has a control for font units, so users can switch between px (pixels) or pt (points). Note that previously, font sizes were always specified in px, but we have now set the default to pt for consistency with font sizes in text boxes.
• Added warnings to Choice Modeling - Experimental Design for cases where the design may not sufficiently explore the choices.
• Added ability to select the number of responses used to calculate standard errors in Choice Modeling - Experimental Design.
• Added ability to save Choice Modeling utilities with scaling options of minimum zero, mean zero, mean zero and maximum range 100, mean zero and mean range 100, minimum zero and mean range 100 and minimum 0 and maximum range 100.
• Added Choice Modeling - Utilities Plot
• Rtsne package used by Dimension Reduction - t-SNE has been upgraded. This may lead to differences in plots. Note that t-SNE is particularly sensitive to the choice of random seed (which can be amended in the R code).
• Sankey Diagrams can now show percentages and counts in node labels.
• RLH (in-sample and holdout) is reported in the footer and respondent RLH can be saved as a variable for both MaxDiff and choice modeling outputs.
• Respondent parameter histograms in Choice Modeling - Latent Class Analysis and Marketing - MaxDiff - Latent Class Analysis outputs are color-coded by class.
• Categorical attributes can be coded to be numeric in Choice Modeling - Hierarchical Bayes and Choice Modeling - Latent Class Analysis.
• Default hierarchical Bayes model prior settings (hb.sigma.prior.rate and hb.sigma.prior.shape) have been changed to reduce model overfitting.

October 2018

• MaxDiff experimental designs can now be created with labeled alternatives (instead of numbers).
• Stacked Bar and Columns charts can now be used with both positive and negative values
• Updated Visualization - Pictograph - Single Icon and Visualization - Pictograph - Repeated Icon to use the same controls as Visualization - Number. This means that pictographs also allows conditional coloring, and can take input data from tables and variables.
• New visualizations to display a single number in Visualization - Number (Displayr only) as a Number, Oval, Rectangle, Donut or Gauge. Controls also allow conditional coloring of the shape/icon.
• Added option for Area, Bar, Column, Line, Radar and Pyramid charts to have data labels shown in different colors for each series.
• Added option for data labels in Labeled Scatter plots to be automatically shown in the color of data series.
• Added option to data labels in Stacked Area, Stacked Bar, Stacked Column and Pyramid charts to be automatically shown in black or white depending on the series color.
• Visualization can now be applied to most of the non-tabular R outputs allowing customization. R outputs supported include Dimension Reduction (data is most appropriately shown as scatterplots), Regression and Machine Leaning (heatmaps), Choice and MaxDiff (distribution charts), Correlation matrix (heatmap), Confusion matrix (heatmap).
• Added checkbox option to toggle the fitting of alternative-specific constants (coefficients) in Choice Modeling - Hierarchical Bayes and Choice Modeling - Latent Class Analysis.
• Added ability to randomize order of alternatives in Marketing - MaxDiff - Experimental Design.
• Version and Question columns added to Marketing - MaxDiff - Experimental Design, for consistency with multi-version designs.

September 2018

• Updated color palettes in Visualizations. Primary colors has been removed, and Colorblind safe colors, Spectral colors have been added. Heat colors has also been reversed so larger values are shown in red.
• Geolocation of IP addresses. Data - Countries from IP Address(es) (Geocoding) outputs country and continent from IPv4 or IPv6 addresses.
• Marketing - MaxDiff - Compare Models calculates a table of summary metrics allowing easy comparison of different MaxDiff models.
• Marketing - MaxDiff - Ensemble of Models creates an ensemble by averaging the respondent parameters of multiple MaxDiff models. Models must all use the same underlying data.
• Compare Models and Ensemble Models for Choice Modeling. Analogous to MaxDiff as above.
• Zip codes (or postcodes) added to Visualization - Geographic Map for Australia, UK and USA. Note the following:
  • Large maps (USA especially) may be slow to display. Turning off Show missing values improves the speed but may require Background map to provide context.
  • Zip code country is Automatic by default and will attempt to determine if the inputs are zip codes and from which country. Changing this option is helpful to resolve ambiguity.
  • US zip codes do not cover the whole of the land area and are not precisely defined.
  • US zip codes are 5-digit integers, Australian postcodes are 4-digit integers.
  • UK postcodes use only the district prefix e.g. BD14 (one or two letters followed by one or two digits).

August 2018

• Upgrade to R version 3.5.1 (from 3.4.4). This may cause some differences in t-SNE and Multidimensional Scaling.
• All Regression and Machine Learning features now share a common standard interface. The Algorithm control allows easy switching between models.
• Added control to choose which dimensions to plot for Correspondence Analysis of a Table.
• Discriminant functions output for Linear Discriminant Analysis has been changed from an output and is now run from Machine Learning - Diagnostic - Table of Discriminant Function Coefficients. This allows discriminant functions to be shown without recalculating the analysis.
• Amended Linear Discriminant Analysis to convert unordered categorical predictors to binary dummy variables. This is consistent with the approach of other predictive methods such as Regression, Deep Learning and Support Vector Machines. To revert to the previous approach of converting categories to numbers (e.g. red, green, blue -> 1, 2, 3) convert the predictor variable to numeric before using LDA. The approach of converting to numbers is still taken for ordered factors.
• Added new Visualization chart type Pyramid
• Amended Sankey Diagram to allow specifying the node width, node and link colors and label font type and font size.

July 2018

• Additional Detail output for Gradient Boosting from the underlying xgboost package.
• Additional diagnostics for Choice Model Designs are now available: Balances and Overlaps of Design
• Additional options for Sankey Diagrams to input data, customize node and link colors, and node labels.
• The design for Choice Modeling - Hierarchical Bayes may now be supplied from Choice Modeling - Experimental Design.
• Simulated respondent choice data may now be generated for Choice Modeling - Hierarchical Bayes. Prior means and variances may be specified for the simulations. The enables a model to be fitted and a design tested ahead of conducting an actual survey.
• Latent class analysis is now available for choice modeling: Choice Modeling - Latent Class Analysis.
• Additional features for Visualizations
  • Small multiples available for Scatterplots
  • Trendlines can now be constructed using cubic splines, and confidence intervals for LOESS and cubic spline smoothers can be added to the chart.
  • All small multiples charts now supports title, subtitles and footers.
• Alternative-specific designs added to Choice Modeling - Experimental Design. Attributes are specific to each alternative.
  • Random - levels of each attribute are randomly chosen.
  • Federov - designs are optimized to extract maximum information from responses.

June 2018

• Additional features for Visualization - Geographic Map
  • Background ocean color control is now available for leaflet maps.
  • If some regions are not recognized, the map of the known regions is still shown and a warning lists the unrecognized names.
  • For leaflet maps, regions that are not listed in the input data can not be plotted. This allows a map to focus on certain countries or states without showing the other countries/states as NA.
  • The determination of the map type in ambiguous cases (names that could be countries or states) has been improved to use a majority vote method.
• Visualizations can now add trend lines using LOESS (previously called Smooth) and Friedman's super smoother.
• Visualization - Geographic Map now handles Australia SA4 areas. The function flipStandardCharts::AreasInCountry("Australia") can be used to get a list of the areas.
• New feature Choice Modeling - Export Design To Qualtrics allows for questionnaires/surveys to be created from discrete choice experimental designs generated by Choice Modeling - Experimental Design and automatically exported/uploaded to Qualtrics.
• Choice Modeling - Experimental Design now has a partial profiles algorithm which generates designs with a specified number of attributes set constant.
• Marketing - MaxDiff - Latent Class Analysis, Marketing - MaxDiff - Hierarchical Bayes, Marketing - MaxDiff - Varying Coefficients and Choice Modeling - Hierarchical Bayes now have options to deal with missing data.

May 2018

• The mice package used for imputation has been updated to version 3.0.0. Some values may change in R outputs where missing data has been imputed.
• MaxDiff analysis now automatically handles generic labels where the best and worst selections are indices amongst the alternatives show (instead of the usual labels of best and worst selections).
• New controls added to Visualization for row and column manipulations of input data before charting.
• New Visualization options to show multiple series as small multiples. Available for chart types 'Area', 'Bar', 'Column', 'Line', 'Geographic Map', and 'Radar'.
• Added controls to Visualization to adjust values axis range in chart types 'Area', 'Bar', 'Column', 'Line', 'Geographic Map', and 'Radar'.
• Added controls to Visualization to control fonts used in Geographic Map (with plotly package only).

April 2018

• Log-likelihood and BIC statistics added to the outputs of MaxDiff and Choice Modeling.
• MaxDiff Experimental Design now uses a 'pairwise balanced' method to create multiple-version designs where the frequencies of each pair of alternatives occurring together are balanced. The Detailed outputs includes additional diagnostics of balances between and across versions.
• Deep Learning has been added to the machine learning and classifier menus. The model performs classification or regression using a neural network built with keras and tensorflow. Cross-validation is performed to find the optimal number of training epochs.
• Added Choice Modeling - Experimental Design. 3 methods of data entry (individual attributes, number of levels per attribute and spreadsheet). Initially 4 algorithms are available, more to follow. Those algorithms are:
  • Random - randomly chooses each level of each attribute. Ensures the same alternative is not duplicated within a question.
  • Shortcut - ensures frequencies of each level of each attribute are balanced.
  • Complete enumeration - ensures frequencies of each level of each attribute are balanced and balance between pairs of levels across different attributes.
  • Balanced overlap - as per Complete enumeration except allows more overlap whereby the same level may be repeated within a question.
• Added Custom Data Files - Survey Gizmo MaxDiff which makes it possible to analyse Survey Gizmo MaxDiff data in Q using Latent Class Analysis or Hierarchical Bayes.

March 2018

• Visualization now has a checkbox so that users can create a crosstab when when multiple variables from 'Data' are selected as the data source. This provides an alternative to selecting a variable in the 'Groups' dropdown which is not always shown on the form.
• Scatterplot can now handle multiple y-values. The user has a choice whether to treat variables/columns as colors and sizes or as additional y-values.See this document for examples of the various types of scatterplot.
• MaxDiff improvements to footer clarifying that incomplete cases are removed from analysis.
• Warning message for Visualizations with more than 10 variables, suggesting that creating a table first may be more appropriate.
• The randomForest package used by Random Forest has been updated with various bug fixes. Notably "The tie-breaking mechanisms in various places were not implemented correctly. Maximum tree size was computed incorrectly. Splits on categorical variables with >=32 levels were not encoded correctly." The observed impact is that these changes are beneficial to prediction accuracy, although some changes may be expected.
• The random number generation process of the Rcpp package used by t-SNE was corrected, which resulted in changes to t-SNE plots. Note that a random seed parameter seed can be set in the R code, resulting in different outputs.
• Speed improvements for hierarchical Bayes analysis for MaxDiff and Choice Modeling

February 2018

• New visualizations:
  • Distributions - Histogram, Box plot, Bean plot, Violin plot, Density plot
  • Venn diagram
  • Time series graph
  • Streamgraph