Book
生成AIアプリケーション評価入門
松木晋祐
Summary
生成AIを組み込んだアプリケーションは出力が確率的で挙動が複雑なため、従来の ソフトウェアテストの手法だけでは品質を測りにくい。本書は生成AIアプリ特有の 評価観点モデルの構築から、混同行列やRAG向けメトリクスなどの具体的な指標、 LLMを評価者として使うLLM-as-a-Judge、pytestとの統合運用、セキュリティ評価、 AIエージェントの評価まで、開発ライフサイクルに沿って解説する。
Target Readers
- 生成AIプロダクトの品質保証を担当するQAエンジニア
- LLMアプリケーションの評価・テスト手法を学びたいソフトウェアエンジニア
- AIエージェントのセキュリティ・品質評価に関わる開発者
Tags
Colophon
- Publisher
- 技術評論社
- ISBN
- 978-4-297-15614-5
- Published
- May 2026
- List price
- ¥2,640incl. taxMay differ from the actual selling price on Amazon
Get this book
* The link above is an advertisement via Amazon Associates.Related Books
Prerequisites
- Recommended
Effective Software Testing
A developer's guide
Maurício Aniche
Reason: Once you have test design techniques like boundary analysis and equivalence partitioning, you apply that thinking to a new kind of target: generative AI applications, whose outputs are probabilistic and whose behavior is complex. This book covers evaluation methods specific to generative AI, from building an evaluation perspective model to confusion matrices and RAG-oriented metrics.
Sources
- Recommended
Generative AI for Software Development
Sergio Pereira
Reason: After learning to use generative AI as a development tool, you face the next question: how do you measure the quality of an application that itself embeds generative AI? This book covers evaluation methods along the development lifecycle, from LLM-as-a-Judge to security evaluation and AI agent evaluation.
Sources
- Related
生成AIの安全性入門
綿岡晃輝
Reason: After learning the techniques of safety evaluation via benchmarks and red-teaming, you broaden your view to how the act of 'evaluating' concretely gets built into your own generative-AI product. This book on generative-AI application evaluation covers everything from building an evaluation perspective model to confusion matrices, RAG-oriented metrics, security evaluation, and AI agent evaluation along the development lifecycle, connecting the abstract idea of safety evaluation to your own product's quality-assurance process.
Sources