Research

Our laboratory focuses on the development and theoretical analysis of machine learning methods, with a foundation in mathematical statistics. We are particularly interested in learning from biased data and understanding the mathematical properties of such learning processes. Machine learning enables computers to automatically discover patterns and regularities from large datasets, allowing them to make predictions or classifications on new, unseen data. For instance, in supervised learning, data consists of pairs of features (like images or text) and labels (such as object names or categories). Algorithms are designed to mathematically model the relationship between features and labels, resulting in a trained model (predictor) that can estimate labels for new features. Machine learning is widely applied in areas such as image recognition, natural language processing, and medical diagnosis, and its significance continues to grow.

Modern machine learning, especially with the advent of deep learning, often involves highly complex models. This complexity makes it difficult to predict whether a learning algorithm will perform well just from its design. The effectiveness and properties of machine learning algorithms have been evaluated through experiments on benchmark datasets. However, results observed in such experiments may not generalize to other datasets, even if they appear similar. To gain a deeper and more reliable understanding, we employ mathematical and theoretical analysis, using tools from mathematical statistics to rigorously study the behavior and performance of machine learning algorithms.

A central theme in our research is understanding how machine learning algorithms behave when trained on biased data. But what exactly is data bias? In supervised learning, we typically assume that the data pairs (features and labels) follow a relationship $f(\text{feature}) = \text{label}$, and our goal is to estimate the function $f$. However, in practice, the data may be distorted by various factors, resulting in a different relationship $f'(\text{feature}) = \text{label}$. The discrepancy between the model we wish to learn and the actual data-generating process is referred to as data bias. When standard machine learning algorithms are applied to biased data, they may learn $f'$instead of $f$, leading to undesirable outcomes. Our research aims to develop methods that can recover $f$even when the data is biased.

There are several important machine learning tasks related to data bias that we focus on:

• Fairness
• Transfer Learning
• Out-of-Distribution Generalization

Below, we introduce each of these research areas.

Fairness

Fairness in machine learning addresses the issue that data can be skewed by conscious or unconscious human biases during its generation or collection, which can then be reflected in the predictions of machine learning models, resulting in discriminatory outcomes. Even if our goal is to build fair models, the underlying data may be unfair, making fairness a specific instance of the broader data bias problem.

Real-world examples of such bias have been reported. One notable case is the COMPAS recidivism prediction algorithm investigated by ProPublica. COMPAS was used in the U.S. judicial system to predict the likelihood of defendants reoffending. ProPublica found that African American defendants were about twice as likely as White defendants to be incorrectly labeled as high risk, while White defendants were about twice as likely as African Americans to be incorrectly labeled as low risk. This demonstrates how the algorithm’s predictions were systematically biased against African Americans.

To address fairness, our laboratory conducts both theoretical and empirical research on algorithms that reduce bias and promote fairness in various settings. Examples of our work include:

• Theoretical analysis of algorithms that maximize predictive performance while satisfying fairness constraints (K. Fukuchi et al., 2023)
• Developing algorithms that achieve fairness even when only a small number of labels for sensitive attributes (such as race or gender) are available (小路口, 2023)
• Investigating, both empirically and theoretically, whether it is possible to detect situations where fairness is maliciously misrepresented (K. Fukuchi et al., 2020)
• Designing algorithms that provide theoretical guarantees of fairness at prediction time (K. Fukuchi et al., 2014)
• Studying fairness when sensitive attribute information is only available through prediction models rather than direct labels (K. Fukuchi et al., 2013; K. Fukuchi et al., 2015)

Transfer Learning

Transfer learning is a technique for improving performance on a target task with limited data by leveraging data from a related source task where data is abundant. For example, consider the problem of diagnosing diseases from medical images. Collecting large amounts of medical images is difficult due to the need for specialized equipment and expert annotation. In contrast, general photographs are readily available from sources like social media. Transfer learning aims to combine a large set of general images (the source sample) with a small set of medical images (the target sample) to improve diagnostic accuracy on the medical images. This scenario exemplifies data bias, as the source and target samples differ in their properties.

A transfer learning algorithm is considered successful if incorporating a target sample leads to better performance than using only a source sample. However, transfer learning does not always guarantee improvement; in some cases, it can even degrade performance, a phenomenon known as negative transfer. Understanding, from a mathematical perspective, the conditions under which transfer learning succeeds or fails is a key research goal.

Our laboratory investigates the theoretical foundations of transfer learning, including:

• Providing theoretical guarantees for successful transfer learning under covariate shift, where only the distribution of features differs between source and target (M. Fujikawa et al., 2025; 藤川, 2023; 藤川, 2023)

Out-of-Distribution Generalization

While transfer learning assumes access to at least small target sample, out-of-distribution generalization focuses on improving performance on target data without any target sample, instead relying on prior knowledge about the relationship between source and target domains. The nature of this relationship defines various out-of-distribution generalization tasks. One important topic is spurious correlation, where labels are strongly associated with irrelevant features (such as background), causing simple algorithms to rely on these non-essential cues. When such models are applied to data where the spurious correlation does not hold, their performance can drop dramatically.

Our research in this area includes developing new methods and theoretical analyses to address spurious correlations, such as:

• Developing approaches that mitigate spurious correlations even when the relevant attributes are unknown, leveraging vision-language models (下坂, 2024)

References