Machine Learning Using Python has been designed to meet the needs of students undergoing a one-semester Machine Learning course offered by universities for Data Science, Computer Science, Statistics, and Commerce students. It provides comprehensive coverage of all key concepts in a style that is simple, thorough, accurate and engaging.

Salient Features

• Simple and user-friendly explanations, supported by numerous programming examples. • Comprehensive coverage of both supervised and unsupervised learning techniques, with suitable
real-life examples.
• Explores various ways machine learning techniques can be used to analyze datasets and derive
valuable insights.
• Discusses four engaging case studies related to regression and classification problems. • A variety of exercises at the end of each chapter, including MCQs, Short Answer Questions, and
Programming Problems.

Chapter 1 Getting Started with Machine Learning 1
1.1 Introduction 1
1.2 Supervised Learning 1
1.3 Unsupervised Learning 3
1.3.1 Clustering 3
1.3.2 Dimensionality Reduction 4
1.3.3 Association Rule Mining 5
1.4 Design of a Machine Learning System 5
1.5 Tools for Machine Learning 5
1.6 Summary 6
Try it Yourself 7
Chapter 2 Linear Regression Model 11
2.1 Introduction 11
2.2 Implementation in Python 12
2.3 Multiple Linear Regression 34
2.4 Polynomial Regression 37
2.5 Hyperparameters 45
2.6 Advantages and Disadvantages 45
2.7 Regularized Regression 46
2.8 Lasso Regularization 46
2.9 Ridge Regularization 47
2.10 Elastic Net Regression 47
2.11 Summary 48
Try it Yourself 49
Chapter 3 Case Study: Predicting Medical Expenses 53
3.1 Problem Definition 53
3.2 Loading the Packages 54
3.3 Loading the Data 54
3.4 Exploring the Data 55
3.5 Exploring the Relationship Among the Features 57

vi Machine Learning Using Python

3.6 Visualizing the Relationship Among Features 57
3.7 Building the Regression Model 59
3.8 Improving the Performance of the Regression Model 62
3.9 Complete Program 65
3.10 Takeaways 67
Chapter 4 Classification of Logistic Regression Method 69
4.1 Introduction 69
4.2 Implementation in Python 69
4.3 Goodness of Fit and Accuracy of Logistic Regression Model 87
4.4 Summary 89
Try it Yourself 90
Chapter 5 Case Study: Predicting the Variety of a Flower 95
5.1 Problem Definition 95
5.2 Exploring the Data 95
5.3 Building a Classification Model for Predicting the Variety of a Flower 99
5.4 Using the Classification Model for Prediction 100
5.5 Complete Program 101
5.6 Conclusion 102
Chapter 6 KNN Classification and Regression Methods 103
6.1 Introduction 103
6.2 Implementation in Python 104
6.3 Hyperparameters 105
6.4 Advantages 105
6.5 Disadvantages 105
6.6 Summary 130
Try it Yourself 130
Chapter 7 SVM Classification & Regression Methods 135
7.1 Introduction 135
7.2 SVM Kernel Trick 137
7.3 Implementation in Python 139
7.4 Hyperparameters 140
7.5 Advantages 140
7.6 Disadvantages 140
7.7 Summary 151
Try it Yourself 151

Contents vii
Chapter 8 Decision Tree Classification & Regression Methods 155
8.1 Introduction 155
8.2 Implementation in Python 156
8.3 Hyperparameters 156
8.4 Advantages 157
8.5 Disadvantages 157
8.6 Selection of a Root Node in a Regression Tree 165
8.7 Selection of Next Level Nodes in a Regression Tree 167
8.8 Selection of Nodes in a Classification Tree 168
8.9 Summary 177
Try it Yourself 177
Chapter 9 Ensemble Classification & Regression Models 181
9.1 Introduction 181
9.2 Bagging Method 181
9.3 Boosting Method 182
9.4 Random Forest Method 183
9.5 Implementation in Python 184
9.6 Hyperparameters 184
9.7 Advantages 184
9.8 Disadvantages 185
9.9 Extremely Randomized Trees 196
9.10 Adaptive Boosting (AdaBoost) Method 197
9.11 Gradient Boosting Method (GBM) 202
9.12 Rotation Forest Method 204
9.13 Summary 212
Try it Yourself 213
Chapter 10 Case Study: Identifying Potential Customers 217
10.1 Problem Description 217
10.2 Exploring the Data 218
10.3 Preprocessing the Data 221
10.4 Exploratory Data Analysis and Data Visualization 223
10.4.1 Univariate Analysis of Categorical Variables 223
10.4.2 Univariate Analysis of Continuous Variables 226
10.4.3 Bivariate Analysis of Continuous Variables 230
10.5 Label Encoding of Categorical Variables 232

viii Machine Learning Using Python

10.6 Building a Random Forest Classification Model 233
10.7 Takeaways 237
Chapter 11 Artificial Neural Networks 239
11.1 Introduction 239
11.2 Layers and Neurons 239
11.3 Feed Forward Network 240
11.4 Important Terminology 241
11.5 ANN-Based Classification and Regression Models 242
11.6 Advantages of ANN 243
11.7 Disadvantages of ANN 243
11.8 Applications of Artificial Neural Networks 243
11.9 Summary 251
Try it Yourself 252
Chapter 12 Evaluating Regression & Classification Models 255
12.1 Introduction 255
12.2 Overfitting and Underfitting 255
12.3 Cross Validation 256
12.4 Evaluation Metrics 257
12.4.1 Evaluation Metrics for Regression Models 257
12.4.2 Evaluation Metrics for Classification Models 258
12.5 Model Selection 260
12.6 Summary 272
Try it Yourself 272
Chapter 13 Case Study: Loan Default Prediction 275
13.1 Introduction 275
13.2 Dataset 275
13.3 Loading the Python Packages 276
13.4 Loading the Data 276
13.5 Preparing the Predicted Variable 277
13.6 Feature Selection 278
13.6.1 Feature Selection by Limiting the Feature Space 278
13.6.2 Feature Elimination Based on Significant Missing Values 278
13.6.3 Feature Elimination Based on Intuitiveness 279
13.6.4 Feature Elimination Based on the Correlation 280
13.6.5 Feature Selection Based on Exploratory Analysis 280
13.7 Label Encoding of Categorical Data 284

Contents ix
13.8 Sampling the Data 284
13.9 Comparing the Models 285
13.9.1 Train-test Split 285
13.9.2 Choosing Test Options and Evaluation Metrics 285
13.9.3 Comparing the Models 285
13.10 Model Tuning and Grid Search 287
13.11 Finalizing the Model 287
13.12 Finding Feature Importance 288
13.13 Conclusion 289
Chapter 14 Clustering Models: K-Means Method 291
14.1 Introduction 291
14.2 K-Means Clustering Method 292
14.3 Hierarchical Clustering Method 304
14.4 Density-Based Clustering Model 307
14.5 Collaborative Filtering 309
14.6 Summary 310
Try it Yourself 311
Chapter 15 Dimensionality Reduction Methods 315
15.1 Introduction 315
15.2 Principal Component Analysis (PCA) 316
15.2.1 Eigen Decomposition Approach 316
15.2.2 Singular Value Decomposition (SVD) Approach 317
15.3 Kernel Principal Component Analysis 317
15.4 Summary 333
Try it Yourself 334
Chapter 16 Association Rule Mining 337
16.1 Introduction 337
16.2 Important Terminology 337
16.3 A Simple ARM Algorithm 338
16.4 Apriori Algorithm 340
16.5 A Larger Apriori Example 342
16.6 Closed and Maximal Itemsets 346
16.7 Frequent Pattern – Growth (FP-Growth) Algorithm 350
16.8 ECLAT Method 360
16.9 Summary 361
Try it Yourself 362

DR. C MUTHU is currently Head, Department of Data Science, Loyola College, Chennai, Tamil Nadu.
An experienced computer professional of over 38 years. Dr. C. Muthu has been teaching Python,
Machine Learning for 9 years. A prolific writer, his books include Programming with Java, Visual C#.
Net and Basic.Net.
MR. M C PRAKASH is currently providing consultancy services for Data Science projects at Shalom Infotech. He is an alumnus of elite institutions such as CEG and BIM. An IT Professional with 7 years of work experience in well known MNCs such as IBM and Cognizant, he is also a passionate researcher who has published five research papers in Analytics domain.