YoVDO

Evaluate generative AI applications

Offered By: Microsoft via Microsoft Learn

Tags

Generative AI Courses Risk Assessment Courses Synthetic Data Courses Azure AI Studio Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
  • Module 1: Learn the fundamental concepts of evaluating generative AI applications.

    By the end of this module, you're able to:

    • Apply best practices for choosing evaluation data
    • Understood the purpose of and types of synthetic data for evaluation
    • Comprehend the scope of the built-in metrics
    • Choose the appropriate metrics based on your AI system use case
    • Understand how to interpret evaluation results
  • Module 2: Learn how to run evaluations and generate synthetic datasets with the Azure AI Evaluation SDK.

    By the end of this module, you're able to:

    • Assess a generative AI app response using performance and quality metrics
    • Assess a generative AI app response using risk and safety metrics
    • Run an evaluation and track the results in Azure AI Studio
    • Create a custom evaluator with Prompty
    • Send queries to an endpoint and run evaluators on the resulting query and response
    • Generate a synthetic dataset using conversation starters

Syllabus

  • Module 1: Module 1: Evaluating generative AI applications
    • Introduction
    • Evaluate generative AI
    • The role of data in evaluations
    • Choose and utilize metrics
    • Custom prompt-based evaluators
    • Interpret evaluation results
    • Common pitfalls in result interpretation
    • Knowledge check
    • Summary
  • Module 2: Module 2: Run evaluations and generate synthetic datasets
    • Introduction
    • Prepare
    • Exercise - Performance and quality metrics
    • Exercise - Risk and safety metrics
    • Exercise - Track evaluation results in Azure AI Studio
    • Exercise - Custom evaluator with Prompty
    • Exercise - Evaluate an endpoint
    • Exercise - Generate a synthetic dataset
    • Knowledge check
    • Summary

Tags

Related Courses

BLEURT - Learning Robust Metrics for Text Generation
Yannic Kilcher via YouTube 3D Deep Learning for Gaming with Srinath Sridhar and Stanford Artificial Intelligence
Resemble AI via YouTube Deep Learning in Gaming with Idan Beck
Resemble AI via YouTube Preserving Patient Safety as AI Transforms Clinical Care - Curt Langlotz, Stanford University
Alan Turing Institute via YouTube Synthesizing Plausible Privacy-Preserving Location Traces
IEEE via YouTube