Machine Learning Maverick – My Path of Learning

In this article, I will show how to use a Machine Learning Model to predict whether the patient has heart disease or not based on the medical records available for free on the internet.

It’s been a while since my last post. I was learning, and thinking about the form of the next post. Should it be a gradually introduced theory or a practical example showing the big picture? I’ve decided to show the big picture with an example source code. All the necessary theories will be added along the way in my upcoming posts. Otherwise, I would wait for the right moment again and not create another post.

Components required

Development environment

For the whole machine learning process, I will be using Jupyter Notebook. For me, it was hard to switch from rich-in features IDE like IntelliJ (for Java development) to a simple yet powerful IDE which is Jupyter Notebook, and now everything is in the web browser. After a while, I got used to it and now I know this is a widespread environment for Machine Learning and/or Data Science.

• All the code is written in Python programming language.
• All the commands are run from the terminal window, your favorite one.

Source code

First things first, all the source code can be found on GitHub – https://github.com/machinelearning-maverick/bp-working-with-data

Tools

I’m using the below tools:

• Anaconda – data science on your workstation
• Conda – language-agnostic, multi-platform package management ecosystem
• Jupyter Notebook – web services for interactive computing across all programming languages
• pandas – Python data analysis library
• NumPy – the fundamental package for scientific computing with Python
• Matplotlib – visualization with Python
• scikit-learn – free software machine learning library for Python

Creating a project

Create a new folder anywhere you want, using the terminal in the newly created folder run the below commands in the terminal. NOTE: I assume that Anaconda is properly installed.

conda create --prefix ./env

conda activate ./env

conda install jupyter pandas numpy matplotlib scikit-learn

jupyter-notebook

After running the last command jupyter-notebook we will get a web page with the environment for Machine Learning and/or Data Science. In Jupyter Notebook we need to create a new Notebook with Python.

The above steps are necessary once. After creating the project, whenever we want to work on this project, we will be running only two commands in the project folder using the terminal:

conda activate ./env
jupyter-notebook

Machine Learning Workflow

Having in mind the Machine Learning Workflow briefly described in my previous post. I will be using it as a guidelines for any Machine Learning projects.

1. Defining the problem
   • Predict whether a person has heart disease or not
2. The data
   • Medical records are available for free – Kaggle: Your Machine Learning and Data Science Community
3. Assessment / Evaluation
   • NOT covered
4. Characteristics / Features
   • NOT covered
5. Modeling
   • Classification Model (Supervised ML) using RandomForestClassifier from scikit-learn
6. Examination / Experiments
   • NOT covered

Working with the data

I will go through all the above steps described by the Machine Learning Workflow. I will be working in Jupyter Notebook using all the tools listed above and writing code in Python programming language.

Now let’s dive into the code. Code clarifications are in the form of Python comments, after the # sign.

We have to run the below code sequentially, one after the other. Inside each code block, we need to press SHIFT+ENTER.

Importing the tools

# Importing the tools
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

Loading the data

# Get the data with medical records from Kaggle.com
# Load the .data file into a DataFrame
heart_disease_ch = pd.read_csv('data/heart-disease_kaggle')

Table with loaded patient characteristics – age, sex, cp, etc., look at Data Characteristics for more info.

	age	sex	cp	trestbps	chol	fbs	restecg	thalach	exang	oldpeak	slope	ca	thal	target
0	52	1	0	125	212	0	1	168	0	1.0	2	2	3	0
1	53	1	0	140	203	1	0	155	1	3.1	0	0	3	0
2	70	1	0	145	174	0	1	125	1	2.6	0	0	3	0
3	61	1	0	148	203	0	1	161	0	0.0	2	1	3	0
4	62	0	0	138	294	1	1	106	0	1.9	1	3	2	0

The above table shows a few rows of the loaded heart disease data. All the necessary info about the data can be found here https://www.kaggle.com/datasets/johnsmith88/heart-disease-dataset

Split the data into features and labels

Why do we split the data for features and labels? We need to separate the desired result – whether the patient has heart disease (value 1) or no heart disease (value 0) – from patient characteristics – age, sex, cp, etc., look at Data Characteristics for more info – based on which our algorithm will predict the probability of a patient developing heart disease.

# X - training input samples, features
X = heart_disease.drop("target", axis=1)

# y - training input labels, the desired result, the target value
y = heart_disease["target"]

We will use feature columns – X variable – to predict – y variable – labels, the desired result, and the target column. In our case, feature columns are “age, sex, cp, etc.” – look at Data Characteristics for more info – and the labels, target column indicates whether the patient has heart disease (value 1) or no heart disease (value 0).

	age	sex	cp	trestbps	chol	fbs	restecg	thalach	exang	oldpeak	slope	ca	thal
0	52	1	0	125	212	0	1	168	0	1.0	2	2	3
1	53	1	0	140	203	1	0	155	1	3.1	0	0	3
2	70	1	0	145	174	0	1	125	1	2.6	0	0	3
3	61	1	0	148	203	0	1	161	0	0.0	2	1	3
4	62	0	0	138	294	1	1	106	0	1.9	1	3	2

The above table shows a few rows of the splitted data, the “X” variable – the “feature” columns – without the target column. The target column will be predicted by the Machine Learning Model.

	target
0	0
1	0
2	0
3	0
0	0

The above table shows a few rows of the splitted data, the “y” variable – the “label” column, the desired result – without the “feature” columns.

Split the data into training and test sets

Why do we split the data for training and test sets? In Machine Learning, one of the most fundamental principles is never to evaluate or test models on data that it is learned from, which is why we split it into training and test sets. It’s like looking at the final exam before we’ve looked at the practice exam, not what we want to be doing! If the professor accidentally leaked the final exam, everyone would be getting perfect marks and no one would be learning anything.

# Import 'train_test_split()' function
# "Split arrays or matrices into random train and test subsets."
from sklearn.model_selection import train_test_split

# Split the data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

Why do the variables have names like X, y, X_train, y_train, etc.? This is the common naming convention. It can be found in many places in the functions and tools documentation, e.g.: sklearn.model_selection.train_test_split — scikit-learn 1.4.1 documentation

Using Machine Learning Model/Algorithm/Estimator

# Setup random seed - to have the same results, me and you
np.random.seed(42)

RandomForestClassifier – Supervised ML

Why RandomForestClassifier Machine Learning Model?

This model is capable of learning patterns in data and then classifying weather a sample aka a row is one thing or another, e.g.: classify whether a patient has or not a heart disease.

# Import the RandomForestClassifier estimator class
from sklearn.ensemble import RandomForestClassifier

# Instantiate RandomForestClassifier to create a Machine Learning Model
model = RandomForestClassifier()

Training the ML Model

First of all, we need to train our ML Model for the Training Set and test it on the Test Set. We want to train the ML Model on the training data so it’s going to learn the patterns, the relationship between the X variables, features, and the y variable, labels – indicates whether the patient has heart disease (value 1) or no heart disease (value 0). The ML Model aka algorithm or estimator is a black box where the Machine Learning magic goes.

# 'fit()' - Build a forest of trees from the training set (X, y).
model.fit(X_train, y_train)

Predicting values with ML Model

# 'predict()' - Predict class for X.
y_preds = model.predict(X_test)

Now what we’ve done is we’ve made a predictions on the Test Data Set to predict whether a patient has or not a heart disease.

array([1, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0])

The above 0’s and 1’s are the predicted values made by the Machine Learning Model for each patient in a Test Data Set. The ML Model was trained/fitted on the Training Data Set.

Data characteristics

Attribute Information:

1. age
2. sex
3. chest pain type (4 values)
4. resting blood pressure
5. serum cholestoral in mg/dl
6. fasting blood sugar > 120 mg/dl
7. resting electrocardiographic results (values 0,1,2)
8. maximum heart rate achieved
9. exercise induced angina
10. oldpeak = ST depression induced by exercise relative to rest
11. the slope of the peak exercise ST segment
12. number of major vessels (0-3) colored by flourosopy
13. thal: 0 = normal; 1 = fixed defect; 2 = reversable defect
    — The names and social security numbers of the patients were recently removed from the database, replaced with dummy values.

Final remarks, the Feynman Technique – mentioned in How did my journey with AI & ML start? – works very well. Before I started writing this post I had a blurry vision about the things that I learned, and I will write in this post. While writing I had to research a lot, and finally, I fully understood the whole concept of using the Machine Learning Models!

NOTE: In this article, I’m just barely scratching the surface. This topic needs more reading and research on your own. I’m still at the beginning of my learning process of AI & ML!