YoVDO

Evaluate generative AI applications

Offered By: Microsoft via Microsoft Learn

Tags

Generative AI Courses Risk Assessment Courses Synthetic Data Courses Azure AI Studio Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
  • Module 1: Learn the fundamental concepts of evaluating generative AI applications.

    By the end of this module, you're able to:

    • Apply best practices for choosing evaluation data
    • Understood the purpose of and types of synthetic data for evaluation
    • Comprehend the scope of the built-in metrics
    • Choose the appropriate metrics based on your AI system use case
    • Understand how to interpret evaluation results
  • Module 2: Learn how to run evaluations and generate synthetic datasets with the Azure AI Evaluation SDK.

    By the end of this module, you're able to:

    • Assess a generative AI app response using performance and quality metrics
    • Assess a generative AI app response using risk and safety metrics
    • Run an evaluation and track the results in Azure AI Studio
    • Create a custom evaluator with Prompty
    • Send queries to an endpoint and run evaluators on the resulting query and response
    • Generate a synthetic dataset using conversation starters

Syllabus

  • Module 1: Module 1: Evaluating generative AI applications
    • Introduction
    • Evaluate generative AI
    • The role of data in evaluations
    • Choose and utilize metrics
    • Custom prompt-based evaluators
    • Interpret evaluation results
    • Common pitfalls in result interpretation
    • Knowledge check
    • Summary
  • Module 2: Module 2: Run evaluations and generate synthetic datasets
    • Introduction
    • Prepare
    • Exercise - Performance and quality metrics
    • Exercise - Risk and safety metrics
    • Exercise - Track evaluation results in Azure AI Studio
    • Exercise - Custom evaluator with Prompty
    • Exercise - Evaluate an endpoint
    • Exercise - Generate a synthetic dataset
    • Knowledge check
    • Summary

Tags

Related Courses

Designing and Executing Information Security Strategies
University of Washington via Coursera Caries Management by Risk Assessment (CAMBRA)
University of California, San Francisco via Coursera Diagnosing the Financial Health of a Business
Macquarie Graduate School of Management via Open2Study Enfermedades transfronterizas de los animales
Miríadax Unethical Decision Making in Organizations
University of Lausanne via Coursera