The results are very tangible.
Looking at the contingency table and Figure 2, we see that whereas 43% of males were shortlisted, only 57% of females were considered to go through the next stage of the hiring process.
Additionally, the total number of males and females applicants that were shortlisted is presented in Figure 3.
Just by having a glimpse to it, we see that there was approximately an equal proportion between males being rejected and shortlisted.
However, when looking at females, the proportion of those who were rejected was considerably greater than those who were shortlisted.
At this point, our analysis provides some evidence that there might be a male-leaning preference in the shortlisting process.
Nevertheless, at this stage, we can not reject that the above results could have occurred by chance alone.
Let’s carry out a chi-square test to see whether our findings are significant or not.
> The chi-square test consists of analyzing whether there is a relation between two categorical variables (i.
e.
, is applicants’ gender related to applicants being shortlisted?).
With this test, we basically compare the frequencies of our categorical variables that we observe to the ones we might expect to get in those categories by chance.
<Before drawing any conclusions, we have to make sure that we are not violating any assumptions for running the chi-square with the data we have.
We need to ensure that our variables are independent and that we have no expected values below 5.
The assumption of independence is very straightforward, for the chi-square to be meaningful each applicant must contribute to only one cell of the contingency table.
Looking back at our contingency table, we had applicants that were either male or female and that were either rejected or shortlisted, so independence = ✓.
The second assumption refers to our categorical variables presenting values equal to or more than 5.
Basically, we could calculate each of the expected frequencies for the four cells of our contingency table as follows (where row total and column total are abbreviated to RT and CT, respectively)shortlisted_nygender_mf Rejected Shortlisted Male 40 38 Female 152 50model_male_rejected = (RT.
male x CT.
rejected) / 280 = (78 x 192) / 280 = 53,5Fortunately, R can do the rest of the 3 expected counts for us.
chisq[["expected"]] %>% round(2) shortlisted_nygender_mf Rejected Shortlisted Male 53.
49 24.
51 Female 138.
51 63.
49As we can see, the lowest value is 24.
51 so we can have another check mark for our second assumption, chi-square expected values = ✓.
Now, we can check the chi-square result.
Pearson's Chi-squared testdata: gender_mf and shortlisted_nyX-squared = 14.
997, df = 1, p-value = 0.
0001077The result is highly significant at the p < .
001*** meaning that there is an association between applicants’ gender and applicants being shortlisted.
However, the chi-square result cannot tell us which cell is contributing the most to this association, for checking this we need to look at the chi-square standardized residuals.
> chisq$residuals %>% round(2) shortlisted_nygender_mf Rejected Shortlisted Male -1.
84 2.
72 Female 1.
15 -1.
69Without going too much deep into statistics, standardized residuals are z-scores with a mean of 0 and a standard deviation of 1.
By knowing this, we can assess the significance of each cell value as follows:Cell value > +- 1.
96 = p < .
05*Cell value > +- 2.
58 = p < .
01**Cell value > +- 3.
29 = p < .
001***By looking at the chi-square residuals, the only significant cell value is the one Male_Shortlisted (2.
72, p < .
01**).
Therefore, we can be certain that the differences in applicants’ gender being shortlisted are due to the high proportion of males being shortlisted as compared to females.
The last step in our gender analysis is calculating the effect size.
To do so, we are going to calculate the odds ratio of our data.
The odds ratio is a very simple and straightforward metric for expressing the effect size of categorical data.
It is most interpretable in focused comparison (i.
e.
, 2 x 2 contingency tables like the one in our case).
shortlisted_nygender_mf Rejected Shortlisted Male 40 38 Female 152 50To calculate it, we simply first calculate the odds that an applicant was rejected given that he was male, divided by the number of females who were rejected.
odds_gender_rejected = male_rejected (40) / female_rejected (152) = .
26Next, we calculate the odds that an applicant was shortlisted given that he was male, divided by the number of females who were shortlisted.
odds_gender_shortlisted = male_shortlisted (38) / female_shortlisted (50) = .
76The last calculation is simply dividing odds_gender_rejected by odds_gender_shortlisted.
odds_ratio = .
26 / .
76 = .
35You can also calculate the odds ratio as follows using the function odd.
ratio from the package questionr.
> shortl_by_gender_or <- odds.
ratio(shortl_by_gender)> shortl_by_gender_or$OR %>% round(2) [1] 0.
35This result tells us that if an applicant’s gender was male, the odds of he being shortlisted was .
35 times higher than if an applicant’s gender was female.
Another way to say this is that the odds of an applicant being shortlisted was 1/.
35 = 2.
8 times lower if an appliant’s gender was female than if an applicant’s gender was male.
CONCLUSIONNow, we can base our conclusions on our statistical analysis.
We could say:The highly significant result of the chi-square indicates that there was a significant association between applicants’ gender and applicants being shortlisted or not.
The result is significant at p < .
001, meaning that there is less than a 1 in a 1,000 chance that we would find the above association by chance alone.
From the standardized residuals the cell ‘males shortlisted’ was the only significant one contributing to the differences in the chi-square results, hence the differences in applicants being shortlisted.
The odds ratio shows that the odds of applicants being shortlisted were .
35 times higher if they were males than if they were females.
RECOMMENDATIONSBased on the results from our gender analysis, we have evidence that there was a male-preference in the shortlisting process of the company XYZ.
Nevertheless, although the chi-square result is statistically significant, we would have to carry out further analysis before we can argue any kind of gender bias in the hiring process.
Perhaps, male applicants were stronger candidates than females, however, all things being equal we would not expect this pattern.
Additionally, applicants’ individual differences such as their age or personality might be playing a significant role in our findings, as previous research has already demonstrated the importance of these variables in human bias when it comes to recruiting and selecting employees.
REFERENCESEdwards, M.
R.
, & Edwards, K.
Predictive HR analytics: Mastering the HR metric (2019).
Kogan Page Publishers.
R Code developed by me and the datasets are in my github account:https://github.
com/Pablo-A-Baeza/People-Analytics-Recruitment-and-Selection.