Yasutaka Kamei

[Overview]
This research aims to establish an interactive support framework in which developers and machines (AI) collaboratively develop software by sharing intentions through natural language. In the first phase (FY2023 to FY2025), we developed practical methods leveraging large language models, including automated bug fixing as the core component, multilingual vulnerability detection, and fuzzing support, achieving high-accuracy detection and root-cause prediction. Furthermore, we worked on enhancing program generation accuracy through intent specification and visual information, automatically generating log statements for test code, and predicting side effects in repairing deep neural networks (DNNs). Through these efforts, we demonstrated the effectiveness of AI-assisted collaborative software development.

[Details]
This research aims to build an interactive development support framework in which developers can share their requirements and intentions, expressed in natural language, with machines (AI) during software development. Rather than pursuing full automation, the goal is to establish a collaborative framework in which humans and AI learn from and complement each other. This approach seeks not only to support individual developers but also to transform development practices at the team and software ecosystem levels.
During the first phase (FY2023 to FY2025), based on the advice received at the initial Advisory Board Meeting in FY2023 (clarifying “interaction,” ensuring evaluability, and narrowing the scope), the project focused on automated bug fixing, particularly the detection and repair of security vulnerabilities. In addition, we initiated efforts related to AI-assisted program generation and challenges concerning the sustainability of open source software ecosystems. Below, we report three main achievements to date.

1. Research on vulnerability detection and automated repair
This research seeks to work on vulnerability detection and automated repair in real-world development environments. Our efforts have centered on two perspectives: improving vulnerability detection capabilities in multilingual codebases, and supporting the identification and repair of build and execution failures that occur during fuzzing (a testing technique that identifies bugs and vulnerabilities by injecting large amounts of random and unexpected data). To address challenges that conventional techniques based on single-language and static analysis cannot sufficiently handle, we employed and implemented large language models (LLMs) and code-specialized models (PLMs) from a practical standpoint.
Evaluation of multilingual vulnerability detection [LLMSC 2025] Traditional vulnerability detection techniques are generally tailored to specific programming languages, making it difficult to handle real systems that span multiple languages. In this project, we constructed a benchmark covering seven programming languages and systematically compared the detection performance of general-purpose LLMs and code-specialized PLMs such as CodeT5P. The results showed that CodeT5P outperformed LLMs particularly in detecting high-severity vulnerabilities, demonstrating the effectiveness of compact code-specialized models that learn programming knowledge. The complementary strengths of language-agnostic LLMs and specialized PLMs offer promising directions for future work.
Systematization of errors during fuzzing and analysis toward automated repair. Our work on vulnerability classification (TOSEM 2023) manually categorized 829 issue reports encountered by developers when applying fuzzing, organizing them into seven categories and twenty-two patterns. Through evaluations by 109 practitioners, we found that failures in building and executing code represent a primary obstacle in practice. To address this challenge, we developed a method for predicting the causes of fuzzing build errors (TOSEM 2025-nourry). By classifying error causes from build logs using vectorization via LLMs and learning with random forests, our approach achieved an overall F1 score of 0.75 and high accuracy for the three major categories (0.98, 0.97, and 0.73).


2．Program generation through collaboration between developers and AI
As the use of large language models (LLMs) continues to expand in software development, key challenges include how developers can effectively communicate their requirements and intentions to AI, and how to control the outputs of LLMs. In this research, we built a framework that supports program generation by enabling humans and AI to understand each otherʼs knowledge and reasoning processes.
Improving program generation accuracy through human‒AI collaboration [MSR 2024] We investigated how developers can supplement AI with meanings and intentions to improve generation accuracy. Using 758 problems from the competitive programming platform AtCoder, we examined how program generation success rates change when developers incrementally provide information in natural language, such as a high-level task description, input and output specifications, sample outputs, and constraints. The success rates increased to 28 percent, 37 percent, 39 percent, and 35 percent, respectively. These results quantitatively demonstrate how information provided by developers influences the accuracy of AI-based program generation.
Building on this work, we further showed that adding visual information in the form of flowcharts to a multimodal LLM can improve program generation accuracy by up to 10 percent. The effect was particularly pronounced for more difficult problems. This finding empirically highlights the importance of programming with an understanding of the overall structure in the age of AI, suggesting that sharing structural interpretations of code between developers and AI is beneficial for interactive development.
Automatic generation of log statements in test code [TOSEM 2025-shu] Log insertion is a key activity that improves maintainability, debugging efficiency, and observability by embedding log statements (for example, log.info) within programs to monitor and diagnose runtime behavior. While previous studies have mainly focused on log insertion in source code, we argue that test code, which plays a crucial role in software quality assurance, also requires similar attention. Using GPT-3.5-Turbo, we evaluated three subtasks: selecting log levels, identifying insertion points, and generating log messages. The results showed that our approach achieved a 33.97 percent improvement over conventional code-specialized PLMs in the insertion point identification task, and demonstrated significant advantages in BLEU and ROUGE scores.
Constructing a collaborative AI model based on DNN repair prediction [TOSEM 2025-ishimoto] Deep neural network (DNN) repair attempts often risk causing side effects, in which a successful repair inadvertently breaks other correct outputs. To address this issue, we developed lightweight models (logistic regression, random forest, and LightGBM) that predict the occurrence of repair and break behaviors in advance by using intermediate indicators contained in pre-repair outputs (entropy, PCS, LPS, and loss). This allows developers to design their interactions with AI more effectively by identifying which samples should be repaired and which behaviors must not be broken.


[Key Publications]
[LLMSC 2025] J. Yu, H. Shu, M. Fu, D. Wang, C. Tantithamthavorn, Y. Kamei, J. Chen, “A Preliminary Study of Large Language Models for Multilingual Vulnerability Detection, ” In the Proceedings of The 1st International Workshop on Large Language Model Supply Chain Analysis (LLMSC), pp.161-168, 2025.
[TOSEM 2023] O. Nourry, Y. Kashiwa, B. Lin, G. Bavota, M. Lanza, and Y. Kamei, “The Human Side of Fuzzing: Challenges Faced by Developers during Fuzzing Activities,” ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 33, Issue 1, Article No.: 14, Pages 1 ‒ 26, 2023. [CORE A*]
[TOSEM 2025-nourry] O. Nourry, Y. Kashiwa, W. Shang, H. Shu, and Y. Kamei, “My Fuzzers Wonʼt Build: An Empirical Study of Fuzzing Build Failures,” ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 34, Issue 2, Article No.: 29, Pages 1 ‒ 30, 2025. [CORE A*]
[MSR 2024] K. Koyanagi, D. Wang, K. Noguchi, M. Kondo, A. Serebrenik, Y. Kamei, and
N. Ubayashi, “Exploring the Effect of Multiple Natural Languages on Code Suggestion Using GitHub Copilot”, In the Proceedings of International Conference on Mining Software Repositories (MSR), pp.481-486, 2024. [CORE A]
[TOSEM 2025-shu] H. Shu, D. Wang, A. Mastropaolo, G. Bavota, and Y. Kamei, “An Empirical Study on Language Models for Generating Log Statements in Test Code,” ACM Transactions on Software Engineering and Methodology (TOSEM), (to be published). [CORE A*]
[TOSEM 2025-ishimoto] Y. Ishimoto, M. Kondo, L. Ma, N. Ubayashi, and Y. Kamei, “Repairs and Breaks Prediction for Deep Neural Networks” ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 34, Issue 4, Article No.: 94, Pages 1 ‒ 42, 2025. [CORE A*]

Information systems are widely used as social infrastructures for public transportation, power supply, finance, etc. in today’s society. Furthermore, their applications are being expanded and innovated including digital transformation (DX) and the realization of Society 5.0 where IoT and AI are highly utilized. The challenges for industries are to create, transfer, and disseminate new services and business models while maintaining and operating existing legacy systems. Software development is becoming a key skill that is required both highly specialized and advanced software developers and also a wide range of citizens who work in various industries. In addition, a society where people can choose diverse work styles to promote a work-life balance is widely demanded, and hence, the work style reforming of software developers is also a keen issue in our future society.

Dr. Kamei is addressing to reform software development styles to realize diverse work styles leading to well-being of software developers as well as by providing tools to assist and automate software development. To create a new paradigm of software development styles, he will tackle research from three perspectives: individual software developers, development teams, and development communities. The core idea of the research is interaction. He aims to support software development exceeding the limits of existing tools, by making software developers and machines (development support tools) interact and learn each other. He also aims to achieve both improvement of the efficiency of development and realization of well-being of software developers together by supporting communication between developers. This is a research to try to answer the essential research question of information science and engineering: How can we utilize data, knowledge, and information, and then lead to intellectual production activities?

In recent years, he has been active in developing new software development styles to support software development teams and communities. As a result, he recognized the problems of the current development team and community, which inspired this proposal to challenge a new software development style.

Dr. Kamei is a promising young researcher who is leading software engineering research internationally. With the support of the InaRIS Fellowship, he is expected to make further advances and further contributions to information science in the next decade.