Given that Optimizely’s stats engine uses a sequential approach to AB testing to compare the means of two or more independent groups, there is no reason why (as long as traffic permits) analysis of variance (ANOVA) can’t be performed on segmented data, in order to detect statistical differences between demographics, devices, platforms or regional data.For this purpose, this article is going to be both an explorative report on these methods, and a step-by-step of how this analysis can be performed, providing you have analytics integration set-up with your AB vendor.Firstly, ensure your analytics integration is correctly tagged with however your traffic is throttled through your experimentation vendor..This way you’re going to be able to have an initial segment which separates your data by experimental condition (i.e. which variation you are performing analysis on)..Ensuring you have correctly implemented your tagging also means you can track your metrics..This is essential for how data is collected and organised for ANOVA..Data is going to be at browser level (individual data points for each user), but for the purposes of this guide I have showed the data export as un-collapsed, so you can see the groups I am going to be testing..For the purposes of this analysis I will look at device/platform comparisons, for one of my experimentation groups/variations; we will call it AA:When this data is collapsed, you can see individual browsers and performance of a variety of metrics..For the purposes of this guide, I have blanked out the individual browser ID’s..I’m not very good at using Paint, so I blanked them out with Keanu Reeves in Point Break:For this article I am going to use our content items per browser metric..By conducting an ANOVA, the means of this metric will be calculated, so we will have an average content items per browser for each device/platform for our AA condition..This is going to tell us whether a statistically significant difference exists between each device/platform for this experimental condition; which can provide a much more granular statistical analysis of how devices performed for this specific web variation..You’re then going to want to perform a data download from your analytics tool, remove browser ID’s (no need for Keanu now) and any metrics not of interest..For this form of analysis I am going to use R, so import your .csv or .xlsx into your R environment:We are now going to perform a One-Way ANOVA, whereby the same metric (content items per browser) is going to be compared between experimental groups (devices/platforms).. More details