Book
Seisei AI Application Hyoka Nyumon (An Introduction to Evaluating Generative AI Applications)
Shinsuke Matsuki
Summary
A Japanese-language introduction to testing and evaluating generative AI applications, by Shinsuke Matsuki. Generative AI applications are becoming indispensable partners across much knowledge work, including research, software development, and creative tasks. Structurally different from software built by stacking deductive logic, they show probabilistic output (the same input does not always produce the same output), complex behavior (the input-output relationship is unclear and partly a black box), and context dependency (output changes based on past interaction and external information). Because of these traits, assuring the quality of a generative AI application requires testing and evaluation methods different from conventional software testing. The book focuses on generative AI applications and systems, introducing approaches for testing and evaluating them.
Target Readers
- Developers, QA engineers, and product managers building generative AI applications
- Companies and individuals using generative AI applications
- QA engineers who want to learn a testing approach different from what they've used before
Tags
Colophon
- Publisher
- Gijutsu-Hyohron
- ISBN
- 978-4-297-15614-5
- Published
- May 2026
- List price
- ¥2,640incl. taxMay differ from the actual selling price on Amazon
Get this book
* The link above is an advertisement via Amazon Associates.Related Books
Prerequisites
- Recommended
Effective Software Testing
A developer's guide
Maurício Aniche
Reason: Once you have test design techniques like boundary analysis and equivalence partitioning, you apply that thinking to a new kind of target: generative AI applications, whose outputs are probabilistic and whose behavior is complex. This book covers evaluation methods specific to generative AI, from building an evaluation perspective model to confusion matrices and RAG-oriented metrics.
- Recommended
Generative AI for Software Development
Building Software Faster and More Effectively
Sergio Pereira
Reason: After learning to use generative AI as a development tool, you face the next question: how do you measure the quality of an application that itself embeds generative AI? This book covers evaluation methods along the development lifecycle, from LLM-as-a-Judge to security evaluation and AI agent evaluation.
- Related
Seisei AI no Anzensei Nyumon (An Introduction to Generative AI Safety)
Koki Wataoka
Reason: After learning the techniques of safety evaluation via benchmarks and red-teaming, you broaden your view to how the act of 'evaluating' concretely gets built into your own generative-AI product. This book on generative-AI application evaluation covers everything from building an evaluation perspective model to confusion matrices, RAG-oriented metrics, security evaluation, and AI agent evaluation along the development lifecycle, connecting the abstract idea of safety evaluation to your own product's quality-assurance process.