AI & LLM Testing Automation Course with Python, DeepEval & CI/CD – Live Training
(Python Foundations | AI & LLM Testing | RAG Evaluation | Playwright UI Automation | DeepEval CI/CD Integration | Capstone Project)
This AI & LLM Testing Automation Course equips QA engineers and software testers with the skills required to test and validate modern AI, Generative AI, and LLM-based applications. The course begins with Python foundations for test automation, followed by in-depth coverage of AI and LLM testing challenges, RAG (Retrieval-Augmented Generation) evaluation, and AI model testing strategies. Learners will understand different types of AI testing, including bias, fairness, and safety along with industry-standard LLM evaluation metrics such as faithfulness, relevancy, robustness, and toxicity. The program provides hands-on training with Playwright for UI automation, DeepEval, and OpenAI APIs, including local LLM testing using Ollama. It also covers AI test automation, CI/CD integration using GitHub Actions, and concludes with real-world case studies and a capstone project focused on end-to-end AI and LLM application testing.
About the Instructor:
|
About the Trainer: Kavya Kavya is an accomplished Quality Engineering and Test Automation Professional with over 7 years of industry experience in software testing, test automation, and quality assurance practices. She has worked extensively on building and maintaining automation frameworks, implementing UI and API test automation, and integrating automated testing into CI/CD pipelines for enterprise applications. She has successfully trained 200+ students and professionals, helping them upskill in automation testing, AI testing, and modern QA methodologies. With a strong focus on AI, LLM, and Generative AI testing, Kavya brings real-world insights into testing intelligent systems, including AI model evaluation, bias and safety testing, and automation strategies. Known for her practical and learner-focused teaching approach, Kavya simplifies complex concepts using hands-on examples, real project scenarios, and industry best practices, enabling learners to confidently apply their skills in real-world testing environments and advance their QA careers. |
Live Sessions Price:
For LIVE sessions – Offer price after discount is 129 USD 109 119 USD Or USD15000 INR 12900 INR 9900 Rupees.
OR
Free Demo On:
Indian Timings: 8th January @ 9 PM – 10 PM (IST)/
U.S Timings: 8th January @ 10:30 AM – 11:30 AM (EST)/
U.K Timings: 8th January @ 3:30 PM – 4:30 PM (BST)
Class Schedule:
For Participants in India: Monday, Tuesday, Thursday & Friday @ 9:00 PM – 10:00 PM (IST)
For Participants in the US: Monday, Tuesday, Thursday & Friday @ 10:30 AM – 11:30 PM (EST)
For Participants in the UK: Monday, Tuesday, Thursday & Friday @ 3:30 PM – 4:30 PM (BST)
What students have to say about Kavya:
|
I joined this course mainly to understand how AI and LLM testing actually works in real projects. The way Python, Playwright, and DeepEval were explained made it very practical. The hands-on sessions helped a lot. – Faria Aziz What I liked most is that this is not just theory. We worked on real AI testing scenarios like bias, hallucination, and evaluation metrics. The capstone project gave me confidence to apply this at work. – Arjun Mehta I was new to AI testing, but the trainer started from the basics and slowly moved to advanced topics. The modules on RAG testing and CI/CD integration were especially useful. – Vibha Good course for QA engineers who want to move into AI and Generative AI testing. The Playwright automation and LLM evaluation frameworks were explained clearly with examples. – Sarah Johnson The sessions on OpenAI APIs and running LLMs locally using Ollama were something I had not seen in other courses. It gave a clear idea of how AI testing is done in enterprise environments – Vikram Patel This course helped me understand the difference between traditional testing and AI/LLM testing. Topics like safety and fairness were covered in a simple and practical way. – Robert Anderson Overall a well-structured and job-relevant course. The combination of Python, AI testing concepts, automation tools, and real-world use cases makes it worth attending. – Sangeetha Roy |
Salient Features:
- 40+ Hours of Live Training along with recorded videos
- Lifetime access to the recorded videos
- Course Completion Certificate
Who can enroll for this course?
- Software Testers and QA Engineers who want to upskill in AI, LLM, and Generative AI testing
- Automation Test Engineers looking to expand into AI test automation using Python and Playwright
- Manual Testers aiming to transition into AI and LLM testing roles
- QA Leads and Test Managers seeking practical knowledge of AI model evaluation, bias, and safety testing
- Developers and SDETs involved in building or testing AI and LLM-based applications
- Professionals working on chatbots, RAG systems, and AI-driven products
- Engineering graduates and freshers interested in starting a career in AI testing and automation
What will I learn by the end of this course?
- Build a strong foundation in Python for AI and test automation, including data handling, logging, and modular code
- Understand AI, Generative AI, LLM, and RAG architectures from a testing and quality perspective
- Identify and handle unique AI/LLM testing challenges such as non-deterministic outputs, hallucinations, bias, and privacy risks
- Apply LLM evaluation metrics like faithfulness, relevancy, robustness, toxicity, and latency
- Automate AI and LLM application testing using Playwright, and DeepEval
- Test OpenAI-powered applications and run LLMs locally using Ollama for private AI testing
- Integrate AI test automation into CI/CD pipelines using GitHub Actions
- Gain hands-on experience through real-world case studies and complete an end-to-end AI testing capstone project
Course syllabus:
Module 0: Introduction to AI and testing LLM Applications (1 hr)
- Introduction to AI & LLM Applications
- What is RAG (Retrieval-Augmented Generation)?
- Why RAG matters in AI applications: search, QA, summarization, decision support
- RAG Architecture Overview
- What is LLM and its application
Module 1: Unique Testing Challenges in AI/LLM (1 hr)
- Differences between traditional software and AI systems testing
- Probabilistic, non-deterministic outputs
- Risks: hallucinations, bias, toxicity, privacy concerns
- Ethical and legal implications
- Regulatory landscape and compliance basics
Module 2: Types of Testing in AI/LLM Applications (2 hrs)
- Functional testing principles in AI contexts
- Bias and fairness testing: detecting and mitigating unwanted bias
- Safety and ethical testing: toxicity, refusal, harmful content prevention
- Performance and scalability testing for AI inference services
- Usability and accessibility testing for AI user interfaces
Module 3: Evaluation Metrics for AI/LLM Outputs (1 hr)
- Understanding evaluation metrics: what to measure and why
- Core metrics: faithfulness, relevancy, completeness
- Fairness, bias scores, stereotype detection
- Safety metrics: toxicity, refusal rate
- Robustness and consistency
- Latency and scalability measures
- Sentiment, readability, coherence, and fluency
- Privacy and compliance metrics
Module 4: Running LLM locally (2 hrs)
- Introduction to Ollama: What Ollama is & why a private AI matters and Key use case
- Setup: System requirements & installation and Basic configuration
- Exploring Models: Pre-trained models & switching between them, Key parameters (temperature, max tokens)
- Using Ollama: Text generation, summarization, Q&A and Prompt tips & exporting outputs
- Exposing via API: Running locally and API calls & script integration
Module 5: Introduction to OpenAI (4 hrs)
- What is OpenAI
- How to generate Open AI key
- Creating a custom evaluation using Open AI
Module 6: Introduction to Python (4+ hrs)
- Introduction to Python and installation setup
- Understanding variables, data types, and operators
- Control flow: conditional statements and loops
- Working with functions and reusable modules
- Lists, tuples, sets, and dictionaries in test data handling
- String manipulation and regular expressions
- File handling and JSON data parsing
- Exception handling, debugging, and logging best practices
- Virtual environments (venv, pip)
- Writing clean, modular Python code
- Mini hands-on tasks for QA automation
Module 7: AI/LLM Testing Frameworks and Tools (10+ hrs along with hands-on)
- Overview of popular testing frameworks
- Playwright for frontend UI automation (4 hrs)
- What is Playwright and why it’s popular
- Setting up playwright
- Core Playwright Concepts – Locators, assertions,..
- DeepEval for LLM evaluation (4 hrs)
- What DeepEval is and where it fits in the AI testing stack
- Key concepts: metrics, test cases, evaluators
- Faithfulness, relevancy, etc
- Core metrics evaluation
- Golden datasets and expected outputs
- LLM as Judge
- Rule based evaluation
Module 8: Hands-On Setup and Test Automation
- Environment setup: js, Python virtualenv, API keys
- Writing and running Playwright UI tests
- Implementing DeepEval Python test scripts
- Automating test suites for continuous integration
Module 9: Integrating AI/LLM Testing into CI/CD Pipelines (5+ hrs)
- Overview of CI/CD concepts for AI apps
- Connecting tests to GitHub Actions
- Automating evaluations on model updates or frontend changes
- Monitoring test results and alerting mechanisms
Module 10: Case Studies and Industry Practices (1 hr)
- Analysis of real AI system failures and lessons learned
- Successful AI testing implementations
- Ethical AI and responsible innovation in testing context
- Future trends: multimodal AI, self-supervised evaluation
Module 11: Capstone Project and Assessment (8+ hrs)
- Learners design and implement a full testing pipeline
- Test automation covering UI and model outputs
- Evaluate a publicly available LLM or AI chatbot system
- Submit report and demo with lessons learned
- Peer review and instructor feedback
How can I enroll for this course?
OR
For any other details, Call me or Whatsapp me on +91- 90529 03733
Live Sessions Price:
For LIVE sessions – Offer price after discount is 129 USD 109 119 USD Or USD15000 INR 12900 INR 9900 Rupees.