Function 7: Plot the Optimal Threshold

Python for Machine Learning and APIs (Course 2)

Welcome to Machine Learning APIs with Python

Course Project: Python Machine Learning & APIs Course. Let's do this! (4:51)
Course Certificate - Instructions
Join Our Private Slack Channel
Video Subtitles (Captions)
Would You Like To Become An Affiliate (And Earn 20% On Your Sales)?

Prerequisites

Getting Help (IMPORTANT)
Prerequisites
Course Software Requirements

🎉Bonus Cheat Sheets!

🔽 Bonus #1: The Ultimate Python Cheat Sheet [Download] (3:11)
🔽 Bonus #2: The VSCode Cheat Sheet [Download] (5:08)

Student Orientation (Part 1): Overview, Business Case & BSPF Framework

🔽 Download the Student Orientation Slides
🥋 Student Orientation Kickstart (1:24)
Why Learn from Me? (The Value of this Training) (1:45)
My First BIG Project. (Story of FAILURE. And then success! The BSPF) (6:13)
The Interview Secret. (How Students Use the BSPF to get Jobs) (2:51)
[BUSINESS CASE] How the BSPF Works (On the Course Project: Email Lead Scoring) (11:15)

Student Orientation (Part 2): Course Workflow, Software & Prerequisites

🔽 Download the Course Workflow Diagram
Course Workflow, Software, & Prerequisites (4:28)
Pre-Course Knowledge Check
👉 Get Python for Data Science Automation (DS4B 101-P): Prerequisite Course

Course Project Setup

🔽 Course Project Files [File Download] (1:00)
Step 1: VSCode Setup (Building on DS4B 101-P) (3:59)
Step 2: VSCode Extensions: Python, Files, Themes & Icons (5:11)
Step 3: Install Anaconda (0:39)
***Windows Users*** Do this! (Before Using Conda via VSCode Terminal)
Option 1: How Jonathan Regenstein got his python environment up in 5 minutes
Option 2: (20 Minute Experimental Installation): Minimal Python Environment (6:31)
Option 3: (Fully Reproducible Installation) Step 4: Environment Setup with Conda (6:05)
Option 3: (Fully Reproducible Installation) Step 5: Add Python Packages from Requirements.txt (4:16)
Python Environment Wrapup (2:03)

Module 0: Jumpstart with ML & APIs

🔽 Setup: Jumpstart (2:26)

Jumpstart Part 1: Intro To Machine Learning With Pycaret

Reading Data (SQL Database Connection) (4:05)
Examining Email Subscribers & Transactions Tables (5:26)
Simplified Data Preparation (5:36)
Pycaret: Classifier Setup (5:28)
Pycaret: Make an XGBoost Model (4:02)
Pycaret: Lead Scoring (Prediction) (5:16)
Pycaret: Save & Load ML Models (3:40)
Conclusions (Pycaret) (3:52)

Jumpstart Part 2: Intro To APIs with Fast API

API Setup (4:58)
The Main Endpoint (6:31)
The Predict Endpoint Part 1 (6:01)
The Predict Endpoint Part 2 (3:57)
Conclusions (FastAPI) (3:15)

✅ Code Checkpoint: Module 0 - Jumpstart

🔽 Code Checkpoint [File Download]

🥳 UNANNOUNCED BONUS: Jumpstart Part 3: Streamlit Jumpstart

🔽 Streamlit Introduction [Download the Streamlit Starter Files] (1:07)
📚 Streamlit Documentation (2:29)
How to Run Your Streamlit App (3:49)
Customizing the Theme Appearance (1:09)
st.write(): Write arguments to the app (2:14)
st.file_uploader(): Uploading Data (2:51)
st.write(): Displaying Data (2:16)
📈 Chart 1: Sales Over Time (5:26)
🥧 Chart 2: Customer Segment by Region (3:10)
📊 Chart 3: Product Analysis (4:32)
User Input: Form Feedback (2:36)
Add a Footer to Your App (1:27)
Conclusions: Streamlit is Awesome! (0:30)
🔽 Code Checkpoint [File Download]

Module 1: Business Understanding

Welcome to Module 1: Business Understanding (1:14)
Setup, Part 1: Frameworks (1:20)
🔽 Setup, Part 2: File Download (1:21)

1.1 Initial Cost Analysis: Quick Rough-Cut Costs

Python Analysis Packages (2:32)
View the Business as a Machine (5:41)
Understanding the Drivers (4:03)
Initial Cost Analysis, Part 1 | Lost Customers per Month (5:58)
Initial Cost Analysis, Part 2 | Lost Revenue per Month (3:18)

1.2 Cost Table: Modeling Time-Varying Costs 📈

Understanding Growth Rates (3:33)
Cost Table: No Growth Scenario, Part 1 (5:10)
Cost Table: No Growth Scenario, Part 2 (3:58)
Cost Table: Compounding Growth Scenario, Part 1 (5:07)
Cost Table: Compounding Growth Scenario, Part 2 (4:39)
FIX 1: Lost Customers Calculation
🆚 Understanding Cost: Comparing Growth vs No-Growth Scenarios (2:38)
🤔Is Reducing the Problem Worth It? (2:49)

1.3 "Parameter-izing" Your Cost Analysis (with Functions)

The Cost Table Function (Part 1): Key Parameters (6:16)
The Cost Table Function (Part 2): No Growth Scenario (7:31)
📈The Cost Table Function (Part ): With Growth Scenario (2:39)
The Cost Summary Function (3:42)

1.4 Cost Simulation

Are the Objectives Being Met? (The Importance of Simulation) (3:06)
Parameter Grid: Cartesian Product with Janitor expand_grid() (6:05)
⚙️Run the Simulation (List Comprehension) (8:00)
Function: Simulate Unsubscriber Costs, Part 1 (4:09)
Function: Simulate Unsubscriber Costs, Part 2 (3:06)
Function: Simulation Unsubscriber Costs, Part 3 (2:32)

1.5 Heat Map: Cost Simulation Visualization

🔥Plotly Heat Map (6:02)
Function: Plot Simulation (4:01)
Business Conclusions: Reviewing the Simulation Results (2:45)

1.6 Email Lead Scoring Package: The Cost Calculations Module

Email Lead Scoring Package: Introduction (0:35)
Setting Up the Email Lead Scoring Package (2:45)
Adding the Cost Table Function (4:50)
Fix 1: Correcting Lost Customers Calculation
👍 Importing functions into __init__.py (5:11)
Adding the Total Cost Function (3:17)
👍Pandas Flavor: Method Chaining (4:09)
Adding the Simulation Function (5:18)
Adding the Plot Simulation (Heat Map) Function (5:19)

✅ Code Checkpoint: Module 1 - Business Understanding

🔽 FIX: Lost Customer Calculation [File Download] (4:43)

Module 2 - Data Understanding & KPIs

🔽 Setup [File Download] (1:32)
Where Does Data Understanding Fit? (BSPF Workflow) (0:55)

2.1 Data Understanding - Collecting Data & Discovering What It Means

Data Understanding: Python Libraries & ELS Module Import (3:04)
SQL Database Connection (4:41)
The Products Table (4:25)
The Subscribers Table (7:07)
The Tags Table (4:15)
The Transactions Table (5:05)
The Website Table (4:29)
Close the Connection (2:00)

2.2 Combining & Analyzing The Data

Combining The Data (Our Data Strategy) (1:42)
Adding the Target (Made a Purchase) (3:40)
Analysis: Who is Purchasing? (2:16)
Geographic Analysis Part 1 (5:45)
Geographic Analysis Part 2 (2:49)
Top N Countries (3:42)
Using Quantile To Understand Differences In Group (1:45)
Tag Analysis Part 1 (2:23)
Tag Analysis Part 2 (4:58)
Tag Analysis Part 3 (2:57)

🍬 2.3 Sweetviz & KPIs

Sweetviz Part 1 (4:09)
Sweetviz Part 2 (6:05)
Developing KPIs (4:39)
Conclusions (1:37)

2.4 Data Import Functions (Add to the email_lead_scoring module)

Database Connection: db_read_els_data() (5:41)
Subscribers: db_read_els_data() (2:20)
Tags: db_read_els_data() (2:41)
Target: db_read_els_data() (3:26)
Table Names: db_read_els_table_names() (2:19)
Raw Tables: db_read_raw_els_table() (3:44)
database.py (6:25)
Test Your Module Out (3:44)

2.5 Exploratory Functions (Add to the email_lead_scoring module)

Exploratory Functions (1:55)
Sales By Category (Part 1) (5:02)
Sales By Category (Part 2) (4:42)
Sales By Numeric (9:04)
exploratory.py (7:58)

✅ Code Checkpoint: Module 2 - Data Understanding & KPIs

🔽 Code Checkpoint [File Download]

⛰️ CHALLENGE #1 - Who Has Purchased Specific Products?

🔽 Challenge: Analyze Products With Regex (Regular Expressions)? (11:33)
Solution: Analyze Products With Regex (12:40)
Checking My Work With The ELS Exploratory Functions (1:54)

Module 3 - Data Processing Pipelines

🔽 Setup [File Download] (1:45)
Where Does Creating A Data Processing Pipeline Fit? (BSPF Workflow) (1:04)

3.1 Feature Exploration

Libraries (1:54)
Feature Exploration: Categorical (3:15)
Feature Exploration: Ordinal (2:07)
Feature Exploration: Interaction (2:53)

3.2 Feature Engineering & Data Preprocessing

Date Features (Time On Email List) (4:41)
Text Features (Email Provider) (2:47)
Rate Features (Activity Per Time) (2:57)
One-To-Many Features (Tags) (5:18)
Merging One-To-Many Features (Tags) (5:01)
Fill Missing One-To-Many Features Using Regex (Tags) (6:26)
Reducing High Cardinality Features (Country Code) (6:45)

🍬 3.3 Sweetviz Report (Round 2) - Analyzing Engineered Features

Making Our 2nd SweetViz Report (4:40)
Business Insights from SweetViz Round 2 (7:14)
Data Preprocessing Conclusions (1:22)

3.4 Data Processing Pipeline Functions

Libraries Needed (1:13)
Create The Data Processing Pipeline Function (2:11)
Add Date Features (1:08)
Add Email Features (1:52)
Add Activity Features (2:36)
Fill NA Regex (0:51)
Adjust High Cardinality Features (1:14)
Improving The Data Pipeline Function (3:25)
Try Out Our ELS Package (5:42)

✅ Code Checkpoint: Module 3 - Data Processing Pipelines

🔽 Code Checkpoint [File Download]

Module 4 - Machine Learning with Pycaret (Deep Dive)

🔽 Setup [File Download] (3:54)
Course Workflow (1:15)
BSPF (1:10)

4.1 Preparing For Pycaret Classifier Setup (Don't mess this up, garbage in = garbage out)

Load the ML Libraries (1:58)
Pycaret Classification Setup (1:59)
Removing Unnecessary Columns (3:24)
Identifying Feature Types (1:22)
Numeric Feature Selection (3:13)
Categorical Feature Selection (1:12)
Ordinal Feature Selection (2:06)

4.2 Pycaret Classifier Setup: clf.setup()

Classifier Setup Part 1 (5:25)
Classifier Setup Part 2 (6:42)
Classifier Setup Part 3 (4:11)
Running the Classifier Setup (3:32)
Examining the Classifier Object: clf.get_config() (7:52)

4.3 Machine Learning With Pycaret's Compare Models

💪 Train 19 Models with Compare Models (Part 1): clf.compare_models() (5:19)
💪 Train 19 Models with Compare Models (Part 2): Examining Best Models (3:53)
Finalizing the Best Models (2:53)

4.4 Plotting the Model Performance (Model Selection)

Plotting Model Performance Part 1 (5:05)
Plotting Model Performance Part 2 (5:45)

4.5 Individual Models: Making and Hyperparameter Tuning

Making Individual Models (XGBoost): clf.create_model() (1:30)
Hyperparameter Tuning Models: clf.tune_model() (3:34)

4.6 Explainable AI: Model Interpretability

Interpretable Machine Learning: clf.interpret_model() (4:18)
SHAP Correlation Plot (Summary): Get Top Features (3:05)
SHAP PDP (Partial Dependence Plot): Analyze Specific Features (1:58)
SHAP Force Plot: Examine Specific Observations (5:28)

4.7 Ensembles (Blending), Calibrating, and Finalizing Classification Models

Ensembles & Blending Models: clf.blend_models() (1:30)
Calibrating a Classification Model: clf.calibrate_model() (2:48)
Finalizing a Classification Model: clf.finalize_model() (1:18)

4.8 Predictions & Lead Scoring (And Model Storage)

Making Predictions: clf.predict_model() (1:53)
Lead Scoring (2:48)
Saving & Loading Models (1:19)
Pycaret Conclusions (1:01)

4.9 Lead Scoring Function (Adding Into Your Email Lead Scoring Module)

Lead Scoring Function Setup (1:12)
Model Scoring Function (5:07)
ELS Module (3:01)
Docstring (1:32)
UPDATE: Model Lead Scoring Fix (8:44)

✅ Code Checkpoint: Module 4 - Machine Learning with Pycaret

🔽 Code Checkpoint [File Download]

Module 5: Advanced Machine Learning with Scikit Learn & H2O

🔽 Advanced ML with Scikit Learn & H2O Setup [File Download] (3:03)
Course Workflow Roadmap (1:17)
BSPF Roadmap (0:50)

🧠 5.1 Scikit Learn Pipelines

Scikit Learn Pipelines: Libraries & Common Functions (3:39)
Load a Pycaret Model (2:31)
What is a Pycaret Model? (4:44)
Data Setup: X and y (5:27)
Train / Test Split (3:16)
Creating A Scikit Learn Pipeline, Part 1: Column Transformers (3:51)
FIX: Remainder Passthrough
Creating A Scikit Learn Pipeline, Part 2: Pipelines (1:52)
Creating A Scikit Learn Pipeline, Part 3: Fit & Predict (2:05)
Creating A Scikit Learn Pipeline, Part 4: Metrics (7:49)
Hyperparameter Tuning in Scikit Learn, Part 1: GridsearchCV() (6:01)
Hyperparameter Tuning in Scikit Learn, Part 2: Fitting, Best Model Parameters (1:59)
Hyperparameter Tuning in Scikit Learn, Part 3: Metrics (4:08)
Pycaret vs Scikit Learn (3:38)
Joblib for Saving / Loading Scikit Learn Models (1:41)
Conclusions (1:44)

🌊 5.2 H2O Automated Machine Learning

H2O Overview & Setup (2:25)
H2O Initialization: h2o.init() (3:20)
H2OFrame Part 1: h2o.H2OFrame() and h2o.describe() (2:00)
H2OFrame Part 2: Setting the Target Column as ENUM with h2o.asfactor() (1:53)
Specifying Predictor (x) and Target (y) Columns (3:41)
H2O AutoML Part 1: H2OAutoML() (4:08)
H2O AutoML Part 2: Training the AutoML (4:21)
Saving & Loading H2O Models (4:21)
Predictions & Lead Scoring with H2O (3:11)
Conclusions (1:46)

✅ Code Checkpoint - Module 5: Advanced Machine Learning

🔽 Code Checkpoint [File Download]

Module 6: MLFlow

🔽 MLFlow Setup [File Download] (1:35)
Course Workflow Roadmap (1:14)

6.1 MLFlow + Pycaret: Interfacing with MLFlow Experiments API

Getting Started (0:58)
Pycaret's MLFlow Integration (3:44)
MLFlow User Interface (UI) (8:05)
MLFlow Tracking API (1:19)
Experiments Part 1 (Listing Experiments) (2:14)
Experiments Part 2 (Programmatic Use & Attributes) (2:07)
Searching Experiments & Runs (7:15)
Time Saver: Pycaret clf.get_logs() (1:35)
Working With Runs (3:17)
Predictions Part 1: Default Predictions (2:15)
Problem: Need Lead Scores (2 Solutions) (4:37)
Conclusions (2:35)

6.2 MLFlow + H2O (And Scikit Learn): Using the Tracking API

H2O + MLFlow Integration (1:32)
H2O Setup (Recap from Module 5) (2:42)
MLFlow Tracking Setup (6:49)
MLFlow UI (0:41)
Starting an MLFlow Run (0:58)
H2O AutoML (Recap from Module 5) (2:36)
Logging Models (H2O & Scikit Learn) (2:48)
Logging Metrics (2:02)
Setting Tags (2:50)
Model URI Location (1:08)
Storing Custom Artifacts Part 1: Getting the Full Leaderboard (Extra Cols) (2:28)
Storing Custom Artifacts Part 2: Logging the Leaderboard as a CSV File (5:46)
Ending an MLFlow Run (0:46)
Making Predictions from MLFlow H2O Models (2:16)
Conclusions (1:06)

6.3 Creating Custom MLFlow Functions (for Email Lead Scoring Module)

Adding MLFlow Functionality to our Email Lead Scoring (ELS) Module (6:42)
Making the Get Best Run Function (5:38)
Predict with the MLFlow Model (6:50)
Making the MLFlow Lead Scoring Function (7:36)
Let's Add our Functions to the ELS Module (7:14)
Testing the MLFlow Lead Scoring Workflow (3:41)

✅ Code Checkpoint - Module 6: MLFlow

🔽 Code Checkpoint [File Download]

Challenge 2: Make a Scikit Learn + MLFlow Tracked Lead Scoring Model

🔽 Challenge: Make a Scikit Learn + MLFlow Tracked Lead Scoring Model [File Download] (12:23)
Solution: Lead Scoring Model with Scikit Learn + MLFlow (11:53)

Module 7: Return On Investment (ROI)

🔭 7-Minute ROI Roadmap: What is ROI AND the 4 Elements of Lead Scoring ROI (7:32)
🔽 ROI Setup [File Download] (2:30)
🗺️ Course Roadmap Update (1:49)
🔽 Fix 1: Correcting the Lost Customers Calculation cost_calculation.py file (Revisited) (1:40)

7.1 Cost vs Savings Tradeoff

Introduction to Cost vs Savings Tradeoff (5:32)
Lead Targeting Strategy (6:28)
Threshold Selection (4:20)
Aggregating the Results (3:21)
Confusion Matrix Calculations (Part 1): Try-Except (6:53)
Confusion Matrix Calculations (Part 2): Summaries (2:24)
Preliminary Expected Value Calcs (Part 1): Savings Cold that are Not Targeted (3:55)
Preliminary Expected Value Calcs (Part 2): Cost Missed Sales that are Not Targeted (2:03)
Preliminary Expected Value Calcs (Part 3): Cost Hot Leads Targeted that Unsubscribe (2:18)
Preliminary Expected Value Calcs (Part 4): Savings Made Purchases (1:52)
Expected Value (Part 1): 4 Important Lead Strategy Values (4:51)
Expected Value (Part 2): Summary Output for Management (3:08)
Conclusions: ROI Part 1 (1:46)

7.2 Threshold Optimization & Profit Maximization

Introduction to Threshold Optimization (2:20)
Function 1: Making the Lead Strategy Function (from the Scored Leads) (7:08)
Function 1: Lead Strategy (Functional Workflow) (2:54)
Function 2: Aggregating the Lead Strategy Results (2:13)
Function 2: Aggregating the Lead Strategy (Functional Workflow) (2:06)
Function 3: Expected Value Calculation (8:21)
Function 3: Expected Value Calculation (Functional Workflow) (5:38)
Function 4: Create the Threshold Table (Part 1: Setting up the function args) (5:36)
Function 4: Create the Threshold Table (Part 2: List Comprehension as a Loop) (9:33)
Function 4: Create the Threshold Table (Part 3: Highlighting Max Values) (3:53)
Function 4: Create the Threshold Table (Functional Workflow) (3:33)
Function 5: Select the Best Threshold (Part 1: The 2 Criteria) (3:46)
Function 5: Select the Best Threshold (Part 2: The 1st Filter Mask) (5:10)
Function 5: Select the Best Threshold (Part 3: The 2nd Filter Mask) (6:17)
Function 5: Select the Best Threshold (Functional Workflow) (2:18)
Function 6: Get Expected Value (3:56)
Function 6: Get Expected Value (Functional Workflow) (1:56)
Function 7: Plot the Optimal Threshold (7:10)
Function 7: Plot the Optimal Threshold (Functional Workflow) (1:40)
Function 8: Optimal Strategy (Part 1: Function Setup) (3:25)
Function 8: Optimal Strategy (Part 2: Thresh Table) (3:49)
Function 8: Optimal Strategy (Part 3: Optimal Thresh & Expected Value) (3:22)
Function 8: Optimal Strategy (Part 4: Thresh Plot & Final Lead Strategy) (3:46)
Function 8: Optimal Strategy (Functional Workflow) (3:33)
Conclusions: ROI Part 2 (3:13)

7.3 Creating Lead Strategy Automation Functions (for Email Lead Scoring Module)

Creating the Lead Scoring Python Module (5:11)
Documentation (6:01)
Testing the Lead Strategy Optimization Function (4:18)

✅ Code Checkpoint - Module 7: ROI

🔽 Code Checkpoint [File Download]

💨 Module 8: FastAPI (Deep-Dive)

🔽 FastAPI Setup [File Download] (4:00)
🗺️ Course Roadmap Update (1:28)

8.1 Lead Scoring API with FastAPI

Welcome to Part 1: Creating a FastAPI to Lead Score (3:25)
Introduce Your API: Adding a Home Screen at Your API's Webpage Root (5:16)
Run the API: Fire up the API and inspect it's Documentation (2:49)
Endpoint 1: GET /get_email_subscribers - Used to get data from your API (5:12)
Testing Our API Endpoint 1: GET /get_email_subscribers (4:54)
Endpoint 2: POST /data - Used to demonstrate how to send and retrieve data from an API (8:51)
Endpoint 3: POST / predict - Used to demonstrate how to run an ML model and return predictions (8:04)
Endpoint 4 (Part 1): POST /calculate_lead_strategy (5:16)
Endpoint 4 (Part 2): POST /calculate_lead_strategy (3:43)
Endpoint 4 (Part 3): POST /calculate_lead_strategy (3:24)
Endpoint 4 (Part 4): POST /calculate_lead_strategy (2:48)
Endpoint 4 (Part 5): POST /calculate_lead_strategy (4:09)
Endpoint 4 (Part 6): POST /calculate_lead_strategy (1:55)

8.2 API Security (API Keys)

FastAPI API Key Security Template: App File (5:13)
Testing the Security Template App (2:33)
Integrate the Security Template into Your Lead Scoring API (5:20)
Testing the Security of Your Lead Scoring API (6:29)

✅ Code Checkpoint - Module 8: FastAPI

🔽 Code Checkpoint - Module 8 Fast API [File Download]

Module 9: Streamlit App

🔽 Welcome to Module 9: Setup [File Download] (2:56)
Course Workflow and BSPF (1:33)
Transitioning to the Streamlit Jumpstart (1:14)

9.1 Streamlit Jumpstart

Streamlit Introduction (1:07)
📚 Streamlit Documentation (2:29)
How to Run Your Streamlit App (3:49)
Customizing the Theme Appearance (1:09)
st.write(): Write arguments to the app (2:14)
st.file_uploader(): Uploading Data (2:51)
st.write(): Displaying Data (2:16)
Chart 1: Sales Over Time (5:26)
Chart 2: Customer Segment by Region (3:10)
Chart 3: Product Analysis (4:32)
User Input: Form Feedback (2:36)
Add a Footer to Your App (1:27)
Conclusions: Streamlit is Awesome! (0:30)

9.2 Lead Scoring Application (Front End)

Transitioning from the Streamlit Jumpstart (0:38)
Setting Up for the Lead Scoring App (7:00)
Upload and Cache Large Data (3:24)
Display the Leads Data (3:16)
User Inputs: Used to Perform the Lead Scoring Analysis (7:26)
Connecting to the API and Making Requests (6:44)
Collect the JSON Response and Get the Analysis Results (5:01)
Display the Analysis Results: Lead Strategy Summary (Expected Savings), Threshold Plot, and Sample Strategy (4:55)
Download the Lead Strategy Results as a CSV File (3:33)
Tidy Our Workspace Up (0:24)

9.3 App Security: Adding Login Authentication

The Streamlit Security Template (5:22)
Adding Authentication to Our Lead Scoring App (7:31)
Testing the Secure App and API (3:18)
Conclusions: You've Accomplished A Lot! (1:41)

✅ Code Checkpoint - Module 9: Streamlit App

🔽 Code Checkpoint - Module 9 Streamlit App

🎉 Congratulations! You Did It! Get Your Course Certificate!

This is a MASSIVE achievement! Congrats. (Watch this to get your Cert!) (1:11)

Function 7: Plot the Optimal Threshold

Lesson content locked

If you're already enrolled, you'll need to login.

Enroll in Course to Unlock