Testing For Normality Clearly Explained

data that has been sampled from normal or Gaussian distribution is one of the main assumptions of parametric statistical tests such as t-tests and anova but how exactly can you decide whether data has been sampled from normal distribution in this video I'll give you an overview of the main ways that are used to determine normality of data so there are two main ways that are commonly used to deduce whether data has been sampled from normal distribution the first is the analysis of graphs such as QQ plots and frequency distributions the other is what are known as normality tests and these include the D'Agostino Pearson omnibus test the Shapiro Wilk test and the kolmogorov-smirnov test but before we go into the tests let's ask ourselves why we do normality tests in the first place let's say we have population of hundred people and we are interested in the height of these people so these 100 people make up our population we can measure their heights and present the data in frequency distribution will discuss frequency distributions in more detail shortly the message here is that the population height data is approximately normally distributed and I'll explain how know this shortly the issue is that we rarely have access to the population data so there is usually no way of knowing this is the case but what people often have access to is sample let's say we randomly selected 10 people from the population this group right here is our sample and again we can measure their height and plot this using frequency distribution now when we talk about testing for normality what we are doing is asking how's the dates have been sampled ie the sample data from distribution the population data that is close to the normal our Gaussian distribution let's quickly recap what our normal ID looks like when plotted on frequency distribution the normal distribution can be seen as bell-shaped curve with the majority of observations being around the mean value which can be seen as the center of the curve so how's our sample come from population that is approximately normal well to answer this we test the data for normality now let's briefly go over the first approach to testing data for normality and that is the analysis of graphs visualizing the data and graphs such as frequency distributions QQ plots and box and whisker plots are powerful means of assessing the distribution of the data and should not be overseen as shown we can easily plot data on frequency distribution let's expand on our previous example of heights but this time there are height data from thousand people here the measure of interest height is categorized into what are known as bins on the x-axis the y-axis represents the relative frequency so the number of times each observation is seen in the data set is counted and the frequency is plotted for example this bin contains all of the people that have height between 64 and 66 inches as mentioned what you're looking for here is that the data is nice bell-shaped curve if so the data is most likely sampled from normal distribution this is quite easily observed for large data sets however this can be tricky to assess when the sample size is much smaller for example if we've randomly sampled the population data that contained thousand people and selected ten values and plotted frequency distribution it's very hard to tell whether this data has been sampled from normal distribution if we increase the samples to twenty values this becomes little easier and again 50 samples hundred samples and 500 samples note all of these samples have been sampled from normal distribution but the characteristics of the normal ideal such as the bell-shaped curve is often hard to see in smaller samples another means of graphically assessing the data for normality is what is known as QQ plots the QQ plot is an abbreviation for quantile quantile plot there is more to QQ plot that goes beyond this video but briefly the x-axis what's the actual sample data whereas the y-axis plots the predicted values assuming the data were sampled from normal distribution usually there is solid line that runs diagonally on the graph and in an ideal normal distribution the values for the and axes will be equal and so the observations will sit on the solid line this is what you should look for in QQ plot that the observations are on or around the line with little deviation but again this is so rarely seen so let's take look at this example notice that there are deviations at either end of the plot this is telltale sign that the data are skewed and here is the frequency distribution of that same data taken together we can see that this data does not represent normal distribution let's now move on to normality tests just like there are statistical tests for hypothesis testing to determine whether there are any significant differences between two group means such as the t-test there are also statistical tests to determine whether data set deviates from the expectations of normal distribution common examples of such normality tests the D'Agostino Pearson omnibus Shapiro Wilk and the kolmogorov-smirnov tests it's important to note that each normality test works slightly differently and so will produce different results when using the same data set and since these tests report p-values there is null and alternative hypothesis behind it specifically the null hypothesis is where the values are sampled from population that follows normal or Gaussian distribution the alternative hypothesis is when the values are not sampled from population that follows normal or Gaussian distribution so how do you interpret p-value from animal t-test commonly an alpha level of 0.05 is applied to statistical hypothesis testing so if the p-value from the malla t-test is greater than 0.05 then we can accept the null hypothesis and reject the alternative hypothesis therefore the sample data is not inconsistent with normal distribution if the p-value from the normality test is less than or equal to 0.05 then we can reject the null hypothesis and accept the alternative hypothesis therefore the sample data is not sampled from normal distribution let's take look at an example of each firstly performed deer Gastineau Pearson normal t-test for this dataset and here is the frequency distribution the p-value for the test was 0.087 so which hypothesis test do we accept and which do we reject the answer is that we accept the null hypothesis and reject the alternative hypothesis so then multi test suggests that data are sampled from population that follows normal distribution let's take look at second example again here is the frequency distribution the p-value from the normality test in this case it was zero point zero one six so because the p-value is less than 0.05 we accept the alternative hypothesis and rejects the null hypothesis this suggests that the data are not sampled from population that follows normal distribution an important note about normality tests is that the sample size can have huge influence on these tests with small sample sizes normality tests have little power to reject the null hypothesis and with large sample sizes normality tests have too much power therefore they can detect even minor deviations from the ideal normal distribution so testing data phenomena should not be restricted to one approach in other words do not perform the malla t-test and used the resulting p-value as simple yes-or-no approach to decide if your data were sampled from normal distribution recommend you perform single normality test don't perform multiple tests as well as this is important to also inspect the QQ plot and frequency distribution of the data graphical analyses are more informative than any p-value returned from normality test then by taking into account all of your findings use your own judgment to determine if your data were sampled from normal distribution did you like this video be sure to give it like leave comment and don't forget to subscribe to be notified when new video is added