Behavioral Prediction of Mongolian Investorsusing Machine Learning Techniques

In this study, we employed machine learning to investigate the impact of behavioral and psychological factors on investment decision-making in the Mongolian stock market. Survey data were collected from individual investors and analyzed using Principal Component Analysis (PCA) with Varimax rotation to extract latent behavioral constructs. Three core factors were identified: Market Reaction and Short-Term Trends, Sensitivity to News and Fundamental Information, and Risk Attitude and Self-Confidence. Using these factors, K-means clustering revealed three investor profiles: Independent Risk Seekers, Reactive Traders, and Cautious Fundamental Investors. Subsequently, Random Forest, Logistic Regression, and Gradient Boosting models were trained in Python to predict investors’ “buy or sell” decisions. Among the tested algorithms, Logistic Regression achieved the highest performance (Accuracy= 0.765, AUC= 0.707, Precision= 0.72, Recall= 0.69). These results demonstrate the potential of machine learning to quantify psychological behavior and implement behavioral finance theory.

Introduction

Traditional economic theories considered investment decision-making to be a rational process based on logic and complete information. However, in reality, investor behavior is strongly influenced by psychology, expectations, emotions, and cognitive biases, as demonstrated by international research (Kahneman & Tversky, 1979; Shiller, 2005).

Behavioral finance, which has developed in this area, aims to explain phenomena that cannot be explained by economic theory through the lens of psychology, and provides a more realistic explanation of stock price fluctuations and market inefficiencies.

Although the fundamental concepts of behavioral finance date back to the works of Adam Smith in the 18th century, the Prospect Theory developed by Kahneman and Tversky has an essential place in modern research, this theory states that human decision-making is based on the relative assessment of gains and losses, not only on the outcome (Kahneman & Tversky, 1979).

In line with this, empirical research has demonstrated that psychological factors, such as loss aversion, overconfidence, and herding, systematically influence investors’ decisions to sell or hold stocks (De Bondt & Thaler, 1985).

In recent years, a considerable amount of data has been generated in all fields. It is becoming increasingly challenging to apply statistical methods when working with large amounts of complex financial data. For this reason, scientists are using data mining and machine learning techniques to process complex financial data (Roychowdhuryet al., 2019).

In stock market research, particularly, the use of machine learning methods to detect psychological and emotional factors and biases in investors’ decision-making under uncertain conditions is gaining popularity. It enables us to process numerous variables related to investor behavior, identify the factors that most significantly affect investor decision-making, and classify and predict behavioral patterns.

For example, using classification algorithms such as Random Forest and Gradient Boosting, it is possible to predict the likelihood of investors “avoiding risk” or “reinvesting to cover losses.” Additionally, clustering enables us to categorize investors based on their behavior (aggressive, cautious, follower, etc.).

This study aims to investigate how behavioral and psychological factors influence investment decision-making in the Mongolian stock market and to evaluate whether machine learning models can effectively predict investor behavior - specifically, the likelihood of buying or selling - based on these factors. It is hypothesized that incorporating behavioral variables into machine learning models significantly improves the accuracy of investor behavior prediction compared to traditional approaches.

The purpose of our research work is to detect and identify psychological factors and behavioral patterns of investors using machine learning methods within the framework of behavioral finance theory.

This is crucial for a more comprehensive understanding of investor decision-making in the Mongolian stock market, as it explains market inefficiencies and offers valuable insights to policymakers and investment institutions.

Related Work

Research Related to Behavioral Finance

In recent financial theory, the behavioral finance approach has been rapidly developing, with numerous studies being conducted in this area.

Behavioral finance is the study of how psychological and emotional factors, as well as biases, influence investors’ decision-making under uncertain circumstances.

The first studies of behavioral finance date back to the 18th century. Adam Smith’s The Theory of Moral Sentiment (1759) and The Wealth of Nations (1776) discussed the “invisible hand”—the manifestation of moral principles that influence individuals’ social, economic, and financial decisions.

Researchers argue that humans can’t be completely free of emotions when making decisions, given the pursuit of happiness.

The concept of psychological influences in economics was largely abandoned in the 19th century but has gained renewed attention since the early 20th century.

Pratt (1964) considered that “the utility function defines risk aversion and risk is a part of total assets.”

The expected utility theory of von Neumann and Morgenstern (1944) and Bernoulli (1954) was criticized in the journal “Econometrics” by Kahneman and Tversky in 1979, and they developed a model of decision-making under uncertainty and prospect theory.

In 1985, Werner, Bondt, and Richard Tylor published an article, “Do Stock Markets React?” in the journal “Finance” and argued that people react systematically to unexpected events and information, which creates weak inefficiency in stock markets.

The first empirical studies of behavioral finance began to be conducted formally in the mid-1980s under the auspices of the Russell Sage Foundation (Sent, 2004).

Selden (1912), in his book “The Psychology of Stock Markets,” established that the price movements of stocks on the stock exchange depend on the The researchers who made significant contributions to this field of science were Daniel Kahneman, a professor at Princeton University and Nobel Prize-winning economist, and Amos Tversky, a professor of psychology and economics at the University of Michigan. The main feature of the field is that it was able to explain concepts and results that could not be explained by traditional economic theory by connecting psychology with economics.

Since the 1980s, behavioral finance has begun to refute classical financial theories. It has increased the number of scholars who doubt the efficient market hypothesis, and has created a debate among them about this hypothesis. Although several theories have been proposed to explain stock price fluctuations, the volatility of the price has proven to be challenging to reconcile with the concept that the current stock price is the present value of future dividends. This suggests that either the financial understanding of what creates stock value is completely wrong or that investor decision-making is not rational.

Shiller (2005) based his argument on this issue by suggesting that markets may be efficient at the micro level but not at the macro level.

Since the 1990s, a growing body of evidence has challenged the efficient market hypothesis, which has had a significant impact on the field of behavioral finance.

Lo (2004) has argued that behavioral finance scholars have rejected the notion that the efficient market hypothesis is incorrect, instead arguing that it is underspecified.

Shefrin (2000) has broadly classified the fallacies of behavior into two categories: heuristic-focused fallacies and framing-focused fallacies. Heuristic fallacies are behavioral fallacies that result from the assumption that a decision is correct in most cases, even if it is not always true.

According to Prosad (2015), the major behavioral biases influencing investor decisions can be classified into several categories, including representativeness, availability, anchoring, overconfidence, optimism, loss aversion, mental accounting, and herding. The key studies addressing these biases are summarized in Table I, based on the classification framework proposed by Prosad.

Table I. Summary of Literature on Various Behavioral Biases

Bias name	Author (Year)
Representativeness	Tversky and Kahneman (1974), Dhar and Kumar (2001), Kaestner (2005)
Availability	Tversky and Kahneman (1973, 1974), Kliger and Kudryavtsev (2010)
Anchoring	Tversky and Kahneman (1974), Campbell and Sharpe (2009)
Overconfidence	Odean (1998a), Daniel,et al., (1998), Barber and Odean (2000), Gervais and Odean (2001)
Optimism (Pessimism)	Heifetz and Spiegel (2001), Toshino and Suto (2004), Shefrin and Statman (2011), Hoffman and Post (2011)
Loss aversion	Kahneman and Tversky (1979), Coval and Shumway (2003), Berkelaar and Kouwenberg (2008), Hwang and Satchell (2010)
Narrow framing	Shefrin (2000), Barberis and Huang (2005), Liu and Wang (2010)
Mental accounting	Thaler (1999), Barberis and Huang (2001)
Disposition effect	Shefrin and Statman (1985), Odean (1998b), Grinblatt and Keloharju (2001), Shumway and Wu (2006), Kumar (2009)
Status quo Bias	Samuelson and Zeckhauser (1988), Brown and Kagel (2009), Li (2009)

In behavioral finance research, the explanation of irrational investor decision-making has relied chiefly on qualitative analyses and traditional econometric models. However, these models have not thoroughly examined the interrelationships among investors’ psychological and behavioral variables.

Therefore, researchers have begun to apply machine learning techniques to analyze non-linear and multidimensional behavioral patterns in greater depth.

Nevertheless, this line of research remains insufficiently explored in emerging markets such as Mongolia, where investor decision-making experience, data quality and availability, and overall market behavior differ significantly from those of well-developed Western financial markets.

Machine Learning to Behavioral Finance

Many scientists are researching methods to predict investor behavior, investment decisions, and market participation using machine learning and data analysis techniques.

Roccaet al. (2019) utilized data from nearly 13,000 daily trading sessions of Brazilian investors to examine whether investor behavior can be predicted from recent price movements using machine learning models.

Nixonet al. (2024) study, “Using Machine Learning to Predict Investors’ Switching Behavior,” which employed the Random Forest method, revealed that various factors, including personal investment experience, market trends, and medium-term growth data, influence investors’ switching decisions.

Porroet al. (2025) utilized data from Italian crowdfunding platforms to analyze investors’ decisions to participate, the amounts they invested, and their overall investment attitudes, employing both classification and regression machine learning models.

These studies demonstrate that ML models, such as Random Forest, Gradient Boosting, and SVM, can identify hidden behavioral patterns in investor data that traditional econometric approaches might miss.

However, a key limitation in the existing literature is that most of these studies focus on developed markets, where behavioral data are rich and standardized. There is still limited evidence regarding how these techniques perform in small or emerging markets, particularly those with different cultural, informational, and psychological contexts.

Research Gap and Contribution

While prior studies have established the theoretical foundation of behavioral finance and demonstrated the usefulness of ML in investor prediction, few attempts have been made to integrate these two domains within the context of developing markets. Moreover, most existing works treat behavioral factors as independent variables, whereas the interrelationships between these factors—such as how overconfidence interacts with loss aversion or information sensitivity—remain largely unexplored. This study addresses this gap by applying machine learning techniques to survey-based behavioral data from the Mongolian stock market to:

1. Identify latent behavioral and psychological factors through factor analysis,

2. Classify investors based on these factors,

3. Predict “buy or sell” decisions using ML algorithms. In doing so, this research provides empirical evidence of how behavioral biases manifest in an emerging market.

It contributes to the growing literature that bridges behavioral finance theory and data-driven prediction models.

Through this research, we provide empirical evidence on how behavioral biases manifest in developing financial markets, such as Mongolia. The novelty of this research lies in the integration of behavioral finance theory with data-driven predictive modeling, while also providing empirical evidence of how behavioral anomalies appear in emerging stock markets.

Materials and Methods

Psychologists have empirically identified many forms of behavioral fallacies, as investors do not make the same decisions or make the same mistakes.

We conducted our research in the following basic steps:

• STEP-1. Data collection and processing

• STEP-2. Use of machine learning methods

• STEP-3. The performance evaluation metrics

• STEP-4. Optimal decision

STEP-1. Data Collection and Processing

We developed a 25-question questionnaire to determine investor behavior and surveyed 151 investors involved in stock trading. We removed blanks and missing responses during the data processing phase, converted text data to numeric values, and then converted categorical data such as age, education, and occupational group to numeric values.

We identified each psychological illusion from the questionnaire, assigned a score to each question, and calculated a total score.

STEP-2. Use of Machine Learning Methods

We implemented machine learning methods, K-Means Clustering, Logistic Regression, and Random Forest in Python.

K-means Clustering

The main idea of this method is to divide n observations into K groups. The first step is to choose an initial value of K, where K can be a user-defined parameter, name, or grouping number. The points closest to the center point are considered to be one group.

In other words, in the K-means method, the center value of the initial group is randomly selected and calculated as the sum of the least square errors of the center value and the Euclidean distance between the points. Points that are close to the center point are included in the group.

The center value of the group is recalculated using the values of the points in that group. The newly calculated center point and the distance between the points are calculated, and the points in the group are regrouped with values close to the center point, and so on, until no center point is found.

Consider data whose proximity measure is Euclidean distance. For our objective function, which measures the quality of a clustering, we use the sum of the squared error (SSE), which is also known as scatter:

(1) $$\begin{array}{r}SSE=\sum _{i=1}^K\sum _{x\in C_i}dis{t}^2(m_i,x)\end{array}$$

where x is an object, C_i is the i-th cluster, c_i is the centroid of cluster C_i, c is the centroid of all points, m_i is the number of objects in the i-th cluster, m is the number of objects in the data set, and K is the number of clusters (Pang-Ninget al., 2001).

Logistic Regression

Logistic Regression is a machine learning supervised model (Cox, 1958). It is widely used in finance and economics, especially in the study of investor behavior, to predict whether an individual will make a certain decision. Logistic Regression is based on the logit function. It transforms the results of linear Regression into a logistic function, which is expressed as a probability of being in the interval 0–1:

(2) $$\begin{array}{r}P(y=1|X)=\frac{1}{1+e^{-({\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\dots +{\beta }_k{X}_k)}}\end{array}$$

where y is the binary response variable, X_k are independent variables, and β are the coefficients or the effect of the variable on the decision.

STEP-3. The Performance Evaluation Metrics

The performance of the models used in this study was evaluated using the following metrics:

• Clustering: Silhouette Score, Davies-Bouldin Index

• Classification: Accuracy, Precision, Recall, F1-score

• Confusion Matrix: Used to see the difference between the quick sell/not quick sell categories.

STEP-4. Optimal Decision

The investor will be able to make rational decisions based on their own behavior and market conditions.

Experimental Results

Socio-Economic Factors of Study Participants

The demographic age distribution of respondents is shown in Fig. 1. As illustrated, 32.5% of the participants were under 25 years old, 37.7% were between 26 and 35 years old, 15.2% were between 36 and 45 years old, 12.6% were between 45 and 60 years old, and only 2% were over 60 years old. The results indicate that individuals between 26 and 45 years old are the most active participants in the stock market, while those above retirement age rarely engage in investment activities.

Fig. 1. Age distribution of investors participating in the survey.

The professional background of investors is presented in Fig. 2. As illustrated, 33% of the respondents are managers or specialists, 28% are students, 21% are executive officers, senior specialists, or employers, while smaller proportions represent scientists, researchers, directors, and retirees. The figure indicates that professionals aged between 26 and 45 years are more active in stock market participation, whereas retirees and academic staff are less involved in investment activities.

Fig. 2. Classification of respondents by professional background.

84.1% of investors participating in the stock market hold a bachelor’s degree or higher, indicating that the level of education plays a vital role among participants in this market. In addition, 65% of investors in Mongolia base their investment decisions on the company’s financial performance over the past year, 24.5% rely on the company’s future outlook, and information provided directly by the company influences 10.5%. These findings suggest that transparent and accurate disclosure of financial and operational information on official information platforms is a key factor in attracting investors.

Using survey data collected from 151 investors, machine learning techniques were applied to identify underlying patterns in investor behavior, psychological factors, and decision-making processes.

Result of Factor Analysis

To ensure the internal consistency and validity of the measurement items, reliability and sampling adequacy tests were conducted before factor extraction. The questionnaire’s reliability yielded a Cronbach’s α of 0.871, indicating strong internal consistency among the behavioral indicators. Sampling adequacy was confirmed using the Kaiser-Meyer-Olkin (KMO) test (0.812), and Bartlett’s Test of Sphericity was statistically significant (p < 0.001), verifying the suitability of the data for factor analysis.

The data collected through the questionnaire were processed in Python using Principal Component Analysis (PCA) with Varimax rotation to extract the key underlying behavioral dimensions. The results of the factor analysis are presented in Table II.

Table II. Result of Factors Analysis

Factor 1	Market Reaction and Short-Term Trends	This factor reflects the tendency to be overly sensitive to price fluctuations, asset holding periods, and information source selection, leading to decisions made in the short term. This is associated with a fear of loss and a tendency to take quick profits
Factor 2	Sensitivity to News and Fundamental Information	Investors make decisions based on company financial statements, industry trends, and economic conditions, and avoid overreacting to unexpected news. This is a characteristic of a prudent investor
Factor 3	Risk Attitude and Self-Confidence	This factor reflects the tendency to take risks, optimism, and confidence in one’s own abilities. Such investors are sometimes overconfident and likely to make high-risk investments

Investor Classification (Cluster)

K-means clustering was performed using latent psychological factors (Factors 1, 2, and 3) that influence investor behavior. Clusters were created to categorize investors with similar behavioral and psychological tendencies into one group and to differentiate their decision-making styles.

The number of clusters was chosen as k = 3 due to the emergence of three types of representatives in terms of behavior: “aggressive—moderate—cautious.”

Three groups were formed using the K-means clustering method, and the psychological characteristics of investors are shown in Table III.

Table III. Result of Cluster

Cluster	Key behavior	Characteristics
Cluster 1	Independent Risk Seeker	Moderate risk-taking, independent decision-maker. Moderately sensitive to market volatility, seeking high returns
Cluster 2	Reactive Trader	Overactive, emotionally driven decision-maker. Short-term reaction, prone to rapid profit and loss recovery
Cluster 3	Cautious Fundamentalist	Information-driven, long-term investor. Prefers to minimize risk and achieve stable returns

Our classification confirms that, according to behavioral finance theory, investor decision-making is characterized by psychology, information use, and risk appetite.

Model Performance

Using survey data from 151 investors, machine learning techniques were employed to identify patterns in investor behavior, psychological factors, and decision-making processes.

In this study, three machine learning algorithms—Logistic Regression, Random Forest, and Gradient Boosting—were employed to predict whether an investor would decide to sell or retain their stock. Each model underwent 10-fold cross-validation to control overfitting and ensure generalizable performance. Hyperparameter tuning via grid search was performed for the tree-based models, while the class distribution between “sell” and “not sell” outcomes was found to be balanced, eliminating the need for resampling.

We implemented three machine learning models to predict whether an investor would ‘sell’ their stock. The accuracy and ROC-AUC of the models are shown in Table IV.

Table IV. Accuracy and AUC of Models

Model	Accuracy	AUC (Area Under Curve)
Logistic Regression	0.765	0.707
Random Forest	0.794	0.673
Gradient Boosting	0.735	0.666

In comparison, Logistic Regression has the highest AUC = 0.707, which is the best at predicting ‘sell/don’t sell’ decisions. The Random Forest model achieves the highest accuracy (0.794), but its AUC is slightly lower, suggesting that it may be overfitting in certain risk situations.

The Logistic Regression model, which achieved the highest predictive performance, was used to estimate whether an investor would sell their stock (0 = not sell, 1 = sell). The results of the logistic regression analysis are presented in Table V.

Table V. Logistic Regression Results

Variables	Odds ratio	p-Value	Note
Factor 2—Sensitivity to News and Fundamental Information	0.41	0.0011	As this factor increases, the probability of selling shares decreases by 59%.
Factor 1—Market Reaction and Short-Term Trends	1.12	0.081	Small effect, but not statistically significant
Factor 3—Risk Attitude and Self-Confidence	1.26	0.153	Risk-taking behavior is evident, but not guaranteed to occur

As shown, Factor 2—Sensitivity to News and Fundamental Information has an odds ratio of 0.41 and a p-value of 0.0011, indicating statistical significance. This indicates that as this factor increases, the probability of selling shares decreases by approximately 59%, suggesting that investors who rely on fundamental and news-based information are less likely to engage in panic selling. Meanwhile, Factors 1—Market Reaction and Short-Term Trends (odds ratio = 1.12, p = 0.081) and Factor 3—Risk Attitude and Self-Confidence (odds ratio = 1.26, p = 0.153) exhibit smaller effects that are not statistically significant, indicating a weak influence on selling behavior.

The Performance Evaluation Metrics

Model performance was evaluated using multiple metrics, including Accuracy, AUC, Precision, and Recall:

• Accuracy (0.765) indicates that the model correctly classified 76.5% of investor decisions.

• AUC (0.707) suggests a moderately strong ability to distinguish between “sell” and “not sell” investors.

• Precision (0.72) means that 72% of those predicted to sell actually did.

• Recall (0.69) shows that the model successfully identified 69% of actual sellers.

These findings confirm that the behavioral and psychological factors incorporated in the model are closely related to investor decision-making.

Discussion

The results of this study experimentally demonstrate that behavioral and psychological factors have a significant influence on investor decision-making in the Mongolian stock market. Investor decisions are shaped not only by economic conditions but also by emotional and cognitive biases.

Factor analysis revealed three latent dimensions - Market Reaction and Short-Term Trends, Sensitivity to News and Fundamental Information, and Risk Attitude and Self-Confidence. Among them, Sensitivity to News and Fundamental Information had the strongest influence, indicating that investors who rely on fundamental data are less prone to panic selling. These results align with previous studies (Kahneman & Tversky, 1979; Odean, 1998a; Nixonet al., 2024), confirming that informed investors exhibit greater stability in the face of market uncertainty.

Conclusion

The study’s results are consistent with the foundations of behavioral finance theory. Studies by Kahneman and Tversky (1979) and De Bondt and Thaler (1985) have shown that human psychological and cognitive biases systematically influence investment decisions. In our study, investors with an information-based approach are more stable, realistically assess risks, and tend not to overreact to market fluctuations.

On the other hand, Factors 1 and 3 indicate investors who react in the short term, are confident, and are willing to take risks, which may be a manifestation of ‘overconfidence’ and ‘short-termism’ behaviors.

The results of logistic regression revealed that investor behavior (latent factors) has a greater impact on decision-making. In particular, information-based behavior reduces the risk of making decisions driven by short-term fluctuations. The relatively weak effect of demographic variables such as age and education indicates that investor psychological attitudes are more important. Therefore, the study confirmed the real effectiveness of using behavioral data to predict investor decisions, identify risks, and provide behavior-based recommendations using machine learning.

While these findings provide valuable insights into investor psychology, the study’s generalizability is limited by the relatively small sample size (n = 151), which may not fully represent the diversity of investor behavior across the entire Mongolian market. Future research should therefore include larger and more diverse samples, as well as real-time trading data, to validate and extend these results.

Despite this limitation, the study demonstrates the practical value of integrating behavioral data with machine learning methods to predict investor decisions, identify risks, and provide behavior-based recommendations.

Future Research

Future studies should expand the dataset to include a larger and more diverse group of investors to improve the model’s generalizability. Integrating real-time trading and behavioral data would enable a deeper understanding of how investor sentiment responds to market fluctuations. Cross-market validation in other developing and mature economies could also test the robustness of the model. Additionally, exploring deep learning or graph-based methods may capture more complex relationships among behavioral factors, thereby enhancing predictive performance.

Acknowledgment

The authors would like to thank all the participants who voluntarily took part in this study.

Conflict of Interest

The authors declare that they have no conflict of interest.