Descriptive statistics of scholarship applicants.
Open access peer-reviewed article
This Article is part of THE SPECIAL ISSUE: TECHNOLOGIES AND CREATIVE LEARNING, LED BY DR. SHARON MISTRETTA, JOHNS HOPKINS UNIVERSITY SCHOOL OF EDUCATION, USA
Article metrics overview
380 Article Downloads
View Full Metrics
Article Type: Research Paper
Date of acceptance: May 2023
Date of publication: May 2023
DoI: 10.5772/acrt.20
copyright: ©2023 The Author(s), Licensee IntechOpen, License: CC BY 4.0
The purpose of this study is to show how AI can serve as an assessment tool to detect potential human bias in decision making for students in higher education. Using student application data, we conduct a small study and apply a set of algorithms to perform deep learning analyses and assess human behaviors when identifying scholarship recipients. We conduct an interview with the organization’s leaders using this data to understand their criteria and expectations for identifying scholarship recipients and collectively explore the insights uncovered using these algorithms. Upon comparison to those recipients awarded the scholarships, we identify opportunities for the organization to implement a quantitative framework—a repeatable set of algorithms to help identify potential bias before awarding future scholarship recipients.
clustering
scholarships
equity
AI
bias
unsupervised learning
topic modeling
regression analysis
Author information
With more and more use of the application of AI, comes the potential of bias—for example, in training data, systems, and in people building these solutions. Many examples of bias in AI applications exist and we have discussed a few in the paper. However, what if AI could be a solution to address bias—specifically bias in humans? In this research, we explore how the use of AI can serve this purpose and be positioned and operationalized as an assessment tool in the higher education domain. We will explore student data and develop a framework to demonstrate this concept while predicting additional resource needs to achieve equitable educational outcomes. We do a deeper dive into bias which shows up in many industries from higher education to construction and supply chains. While we focus on higher education in this research, there are several use cases for which AI is being utilized, for example, to build construction projects in effective, more efficient, and innovative ways. This includes boosting workplace safety to enhancing work schedules to continuously monitoring construction facilities to creating better design of buildings. Even with these opportunities for AI, several areas in this industry are prone to bias. AI is also being utilized in supply chain to improve how we all do business—including automating end-to-end management of the flow of materials. While there are promises of what AI can do in supply chain risk management, there exists bias—such as customers relying on third-party inaccurate and biased data.
According to Kometa
According to Pesämaa
In our previous study, we focused on AI-enabled computing, particularly several types of machine learning algorithms that helped uncover insights on scholarship application data [7]. As a result, we applied a sentence transformer, a dimensionality reduction algorithm—Uniform Manifold Approximation and Projection (UMAP), a topic modeling algorithm—BERTopic, and logistic regression to understand the correlation of application responses to the likelihood or unlikelihood of a student being awarded a scholarship [8]. This helped in forming our conclusions on how human bias can impact the student’s chance of securing a scholarship award and enabling us to understand the value of using AI beyond what it’s known for and used for today such as translation of spoken and written language, predictability, reasoning, and perception analysis. Our previous results amplified the need for stakeholders to conduct a deep analysis of the process for reviewing and awarding scholarships ultimately highlighting potential gaps and in forming business process re-engineering techniques prior to integrating future technologies. In other words, we used AI to uncover potential human bias and provided recommendations to manage human bias mitigation prior to implementing future AI solutions to gain operational efficiencies in scholarship decision making.
In this study, we continue to conduct process evaluation for scholarship award decision making utilizing the same sample dataset from our previous study, and we refine prior models to determine if our preliminary findings are now enhanced (or not) when understanding the types of responses that increase or decrease a student’s chance for receiving a scholarship from NABA (National Association of Black Accountants) Inc, a non-profit organization that focuses on the success of black students and professionals. More specifically, NABA provides a platform for students and professionals to get engaged, inspired, and empowered to excel as black business leaders. Upon refinement of our previous algorithms, we conduct a comparative analysis to understand similarities or differences in the algorithms used in our previous and current research.
Ultimately, we aim to provide stakeholders with a model (or set of algorithms) that lay the foundation for what could be future automation work for the scholarship award decision-making process. In order to drive equitable decision making, we also underscore the need to apply an ethical lens to the entire scholarship award process (both from the perspective of people and the technology) to enhance trust in the organization’s ability to distribute scholarships in a transparent, fair, and explainable manner.
We’ve structured this paper as follows: Section 2 defines different types of human bias and introduces several examples of biased automated decision making. Section 3 includes the application of AI algorithms and provides comparative results and analysis. Section 4 describes the different methodologies and highlights our discussion with relevant stakeholders on their expected outcomes as well as aspirations for the current scholarship award process which ultimately informed the conclusions highlighted in Section 5.
There are several types of biases that we all experience and form, which may influence or impact our actions including the decisions we make. Research suggests that our decisions can be influenced based on our own personal preferences, character, beliefs, and experiences [9]. When we unconsciously introduce these biases, especially into sensitive matters such as healthcare decisions, funding decisions, criminal justice decisions, and talent decisions; this can lead to unfair and harmful outcomes for others. We have defined the common types of human biases below. We have discussed the core research presented in this paper in Section 3, where we reflect on some of the biases described below and their potential impact on decision making, specifically for scholarship awards when these types of behaviors are present in one’s evaluation of scholarship applications.
The Dunning–Kruger effect [10] is a cognitive bias that describes the propensity of people who are less competent in a certain area to exaggerate their abilities while people who are more proficient may underestimate them. David Dunning and Justin Kruger first discovered this phenomenon in a series of investigations in the late 1990s. The Dunning–Kruger effect can be quantified by applying the following formula, which connects a person’s self-reported ability to their actual ability:
A cognitive bias known as confirmation bias [11] refers to the propensity for people to seek out, evaluate, and retain information in a way that supports their preexisting views or hypotheses while disregarding or discounting evidence that contradicts those beliefs or hypotheses.
Psychotherapist Charles Lord and his associates conducted a well-known study on confirmation bias in 1979 [12]. Participants in this study were shown a collection of research studies on the efficacy of the death penalty as a deterrent to crime. While some of the studies argued in favor of the death sentence as a powerful deterrent, others disagreed. Even when the data was tenuous or confusing, participants had a tendency to retain and interpret material in a way that reinforced their preconceived notions about the death sentence, the researchers discovered [10].
As we discussed the scholarship application analysis below, we will reflect on a few of these biases to support our recommendations.
Although the focus is on grant-makers, in general, Steele [20], suggests several steps reduce bias in decision making:
Acknowledge bias exists
Develop a bias reduction strategy
Evaluate and re-imagine business processes
Develop a well-defined structure to make decisions
Focus on function rather than presentation
Construct a collective review process
Support diverse perspectives
Analyze and compare decisions to expected outcomes and make refinements as necessary to achieve desired goals.
There are many benefits to leveraging AI in Education. The University of San Diego notes several of these benefits including (1) customized curricula to support an individual student’s learning need, (2) tutoring services, (3) grading papers, (4) course feedback to inform teachers of potential revisions to enhance the quality of coursework, and (5) immediate feedback to students to drive performance improvement.
How might some of the different types of bias described previously influence decision making with scholarship awards? Utilizing a sample of student application data through collaboration with our non-profit organization, we apply several AI algorithms to this data to understand the following:
Does any correlation exist between a student’s choice of words in their scholarship essay and the scholarship award decision?
Are there any insights that may lead to potential unconscious bias based on those students awarded a scholarship?
Do increased insights highlight any additional resource needs beyond financial support to increase a student’s chance of graduating on time from college?
A summary of the scholarship application data is as follows:
The total dataset was representative of 365 applications with 200 of those applications completed. 100 applicants did not complete their applications. Preliminary descriptive statistics of the population are included in Table 1 below:
Academic classification | |
Freshman | 20 |
Sophomore | 32 |
Junior | 64 |
Senior | 52 |
Graduate Student | 36 |
Top 5 states represented by applicants | |
22 | Students from Texas |
21 | Students from Georgia |
21 | Students from New York |
16 | Students from Maryland |
15 | Students from South Carolina |
GPA range of students | |
Below 3.0 GPA | 10 |
3.0 and up to 3.5 GPA | 50 |
Above 3.5 and up to 4.0 | 144 |
We apply the same steps as before replacing UMAP and introducing an alternative dimensionality reduction technique. This includes:
Sentence Transformer: For this research, a pre-trained sentence transformer model, all-mpnet-base-v2 [24], was used for sentence embeddings. In natural language processing, sentence embedding is how sentences are mapped to numerical vectors to allow for further analysis.
Incremental Principal Component Analysis (IPCA) was used for dimensionality reduction. IPCA can be used in place of principal component analysis (PCA) in the case of large datasets that require a significant amount of memory. Dimensionality reduction techniques like this are important when working with text-based data [25].
Dimensionality reduction is used for compressing data, removing noise from data, along with data classification and visualization. In our previous study, we leveraged UMAP—another dimensionality reduction technique which creates a high dimensional visual representation of data then enhances a low dimensional projection or visual of the data that is structurally similar. As with any comparable technique, IPCA is typically competitive when it comes to speed, while UMAP is known for having better scaling performance.
Text-based machine learning is subject to the “curse of dimensionality,” which is especially the case in this research given a large number of features relative to observations in the dataset and the clustering technique used. The curse of dimensionality occurs when the number of dimensions in the data (data points) grows making it difficult to analyze, visualize, and extract patterns and other meaningful insights from this data. As discussed earlier, the IPCA algorithm allows for feature extraction or the creation of a new smaller set of features which still captures useful information from the student essays. IPCA is a versatile algorithm that performs similar to the UMAP algorithm utilized in related research, however, these algorithms operate very differently. While linear transformation-based PCA is one of the most widely used algorithms, UMAP is a newer technique that uses a stochastic or non-linear approach, allowing randomness to aid in optimization. However, with either algorithm, interpretability is still a challenge given the unsupervised nature of this analysis. Critically, the context gleaned from conversations with NABA leadership provides historical context with which results can be interpreted better.
The topic modeling algorithm, BERTopic [8], was applied as it was in the previous analysis, yielding a larger number of different topics as discussed in the results and analysis section. This algorithm is an unsupervised machine learning technique that identifies groups of similar words in a set of data—they ultimately represent a set of shared themes.
Next, we applied a logistic regression model to determine which topics were statistically significant—specifically, the topics most correlated with an increased and decreased chance of getting the scholarship. The logistic regression equation [26] is given as:
𝛽
For the first objective above, we apply the following algorithms to the student’s essay data.
Sentence Transformer—a python framework was used to create the word embeddings—where words with similar meanings will be represented in a similar way. For this research, a pre-trained sentence transformer model, all-money-base-v2, was used for sentence embeddings. In NLP, sentence embedding is how sentences are mapped to numerical vectors to allow for further analysis. Exploring other pre-trained models could expand upon this research.
IPCA was used for dimensionality reduction and can be used in place of PCA in the case of large datasets that require too much memory. In general, PCA is a dimensionality-reduction method used to reduce the dimensionality of large datasets by transforming a large set of variables into smaller ones maintaining as much information as possible. Dimensionality reduction techniques like this are important when working with text-based data. As described in previous research dimensionality reduction is a set of techniques that remove excessive and irrelevant features from machine learning models. The illustrative example in Figure 1 below offers visual verification that IPCA is able to locate a similar projection of the data to PCA [27].
The “curse of dimensionality” affects text-based machine learning, which is particularly true in this study given the size of the dataset in terms of features per observation and the clustering method employed. As mentioned earlier, the IPCA method enables feature extraction or the development of a new, more condensed set of features that nonetheless effectively captures relevant data from student writings. The UMAP algorithm used in related research performs similar to the flexible IPCA technique. These algorithms, however, function in quite different ways. One of the most used techniques is linear transformation-based PCA, although a more recent method called UMAP uses a stochastic or non-linear approach, allowing unpredictability to help with optimization. Yet, interpretability is still a concern with both algorithms.
Let
The mean of
Alternatively, we can compute the eigenvectors and eigenvalues of the revised covariance matrix in each batch using an iterative method known as power iteration. The power iteration approach begins with an eigenvector guess and iteratively updates the eigenvector until convergence.
Let
The eigenvalue of the updated covariance matrix can be computed as the dot product of the eigenvector and the updated covariance matrix:
BERTopic is a leading python package that utilizes transformer models and was used for topic modeling—an unsupervised machine learning NLP technique for determining what topics, which can be thought of as categories, are part of a set of documents (paragraphs) and what topics each document is likely to be a part of.
A logistic regression model was then used to find which topics were statistically significant—specifically, the topics most correlated with an increased and decreased chance of getting the scholarship.
After dropping duplicate essays and preprocessing, the model generated 76 topics. Some examples of the 51 topics include:
For example, Topic 4 (see Figure 2) consists of a string of words including scholarship, nation, work, provide, and community. Topic 5 consists of words including quarantine, matter, and struggle. Topic 58 (see Figure 3) consists of words: work balance, pride, and pre-pandemic.
As shown in Figure 4 below one can see the topics in two-dimensional space with the areas proportional to the number of sentences in each topic. A count was conducted for the number of each topic in each paragraph (applicant essay) and was normalized by the number of sentences in each paragraph.
Next, the logistic regression resulted in the following topics which were statistically significant to scholarship award decisions:
Topics correlated with an increased chance of receiving a scholarship are captured in Figure 5 below:
Topic correlated with a decreased chance of receiving a scholarship are captured in Figure 6 below:
Preliminary topics that are correlated with an increase in an applicant’s chances of getting the scholarship include words such as “instructor” and “conduct” whereas the topic “vaccine” is correlated with reduced chances of getting the scholarship. Upon speaking with several students, some applicants took a more literal interpretation of the essay question—“Discuss a challenge or barrier you have overcome during the Covid-19 pandemic.” and may have been at a disadvantage. However, applicants who interpreted the prompt in a way that allowed them to speak to their past experiences or strengths in overcoming challenges appeared to be at an advantage.
The analysis was expanded beyond the survey data where regression analysis was also conducted on the non-essay data attributes in the student application to generate the most significant coefficients—identifying the variables most correlated with a student receiving the scholarship. This included looking at data attributes including but not limited to, college class level, college major, extracurricular activities, leadership roles, and grade point average. As a result of the analysis, statistically significant variables as shown in Table 2 below, included college majors—with accounting and finance students having positive correlations (with coefficients 1.87 and 3.15, respectively) and the highest chance of receiving a scholarship, whereas business majors had a negative correlation (−1.22). GPA and leadership roles also yielded positive correlations (with coefficients 0.63 and 1.63 respectively) having the highest probability of a student being a scholarship recipient.
Accounting major | 3.541789 |
Finance major | 3.968811 |
Business major | −2.091610 |
Leadership position (University NABA chapter) | 3.341944 |
Topic 13 (let) | 2.817758 |
Topic 15 (vaccine) | −1.447707 |
Topic 25 (empathy) | 2.264021 |
Topic 42 (resort) | 2.062340 |
Topic 45 (soon) | 2.033636 |
Topic 57 (space/self confidence) | −2.739685 |
Topic 58 (work balance) | −2.670800 |
Combining the core elements of the application data—the non-essay data attributes and the essay data, a final logistic regression is conducted on both the encoded data columns from the non-essay data attributes described in the previous paragraph and the topics generated from the preliminary analysis. As a result of the analysis, statistically, significant variables included college majors with accounting and finance students having positive correlations (with coefficients 3.54 and 3.96, respectively) and the highest chance of receiving a scholarship, whereas business majors had a negative correlation (−2.09). Leadership roles also yielded a positive correlation (3.34). Topics 13, 25, 42, and 45 also have positive correlations with scholarship recipients; whereas Topics 57, 58, and 15 have negative correlations, decreasing a student’s likelihood of being awarded the scholarship.
Upon discussion with NABA regarding the student application data in general and the preliminary insights described above, following were noted:
NABA sees AI as a continuous monitoring tool to help unpack potential human bias prior to making award decisions given some of the insights described in Section 5 where a student’s choice of words could potentially increase or decrease their chances of securing a scholarship award.
NABA is generally looking to award scholarships to students that may be overlooked due to lower GPAs, even though these students have demonstrated their ability—having the skills, experiences, and competencies necessary to be successful. In our research, scholarship recipients appeared to have a higher GPA.
NABA wants to see an increase in applications submitted by students attending community college.
NABA wants to see a sufficient amount of coverage across the U.S. when awarding scholarships to students.
NABA sees an ongoing need to refine scoring rubric requirements as appropriate. In this case, the insights from this research can help inform certain areas of the application response that should be considered when refining the rubric.
NABA is looking to award scholarships to students that have degrees beyond accounting and finance, but donor intentions may limit the type of scholarships they can award.
NABA finds that this research informs an opportunity to re-engage with donors to communicate the new go-forward strategy to make certain that their criteria match their ultimate vision for awarding scholarships.
Given NABA’s stated goals, further research into potential bias from GPA may be warranted. A regression analysis was conducted on the current data yielding insignificant results. However, this was most likely due to the skew in the data toward high GPAs (majority 3.5 and above). As NABA works to expand its reach to students of lower GPAs, it can revisit this analysis to determine the significance of GPA on scholarship decisions—and determine if a high correlation should raise concern that the decision committee would overvalue this quantitative measure.
In higher education, AI is continuously being used to make decisions in a variety of ways including college admissions, grading papers, predicting a student’s success, and many other examples, with significant opportunities for new bias, when implemented carelessly and without human intervention. While we focus our research on AI as a solution to bias, it is equally important to address bias from both sides of the equation. We conclude this research by overlaying the insights uncovered with our pilot data with some of the concepts outlined by Alex Stern and Eugene Sidorin on using AI to fight Biases [28].
Data mining & preliminary data insights: providing automated insights prior to making award decisions can help reduce any potential errors and human bias when making final selections on scholarship award recipients.
Deep learning & anomaly detection: providing a deep understanding of those topics that relate the most with a student’s chance of being a scholarship recipient in recent applications and comparing that to the intended outcomes by NABA (noted above) to see if there are differences in expectations can provide on-demand transparency and real-time appropriate decision making.
Process improvement: refining the application scoring rubric, re-imagining the review board as appropriate, and providing training to the review board for selecting scholarship recipients to make certain that they understand guidelines and measure trends in selection year over year can further position NABA as a trusted advisor to current and future donor recipients.
The authors declare no conflict of interest.
Written by
Article Type: Research Paper
Date of acceptance: May 2023
Date of publication: May 2023
DOI: 10.5772/acrt.20
Copyright: The Author(s), Licensee IntechOpen, License: CC BY 4.0
© The Author(s) 2023. Licensee IntechOpen. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Impact of this article
380
Downloads
804
Views
1
Dimensions Citations
Join us today!
Submit your Article