BADMINTON STROKE CLASSIFICATION¶
AN END TO END MACHINE LEARNING PROJECT¶
This project is on classifying the type of badminton stroke player played using the data collected from a small wrist device. This device gives us accelerometer and gyroscope data and is attached to the player's wrist. This project was part of Data Analytics course we did at IISc. And it involved everything from data collection, cleaning up till the final ML model evaluation.
This jupyter notebook explains "what we did?" and "how we did it?" (the code).
Defining the problem¶
To start off, we first need to define and properly formulate the problem we want to solve. Since, this was a course project we could have chosen any problem. We finalized to work on classifying the kind of stroke a player is playing based on the sensor's data we get from her smartwatch. This was an extension of Human Activity Recognition (HAR) project, which using similar data, classifies the activities as running, walking, etc. In our case, we decided to have the following badminton strokes as classes:
- Backhand Overarm (bo)
- Backhand Underarm (bu)
- Forehand Overarm (fo)
- Forehand Underarm (fu)
- Forehand Smash (fs)
The 2 letter in parenthesis are abbreviations of the strokes, which we'll be using throughout this notebook.
Data¶
As per our knowledge, there was no publicly available data for this problem. There have been some research papers, but we couldn't get our hands on the data. So we decided to collect it ourselves (Why not? Can give it a try).
Setup for collecting data¶
We got hold on a device which can give us accelerometer and gyroscope readings over a period of time at a descent frequency. The device was powered by a small cell and connected via bluetooth low energy (BLE) to a rasperry pi. So we collected our data on rasperry pi in csv formats.
Collecting data¶
Thanks to our 5 volunteers, we collected the data for the 5 mentioned strokes by making them repeatedly play the same shot. Although this might be very different than a real game, but we tried to make them play as they are playing a real game, (i.e. starting from center position, going towards the shuttle cock and returning back to resting position). The device had it's own limitation (coverage, power, etc) which made it challenging. Going through all of this we finally got our data.
Visualizing the data¶
# Some imports
import pandas as pd
pd.set_option('display.max_columns', None)
import numpy as np
import itertools
import matplotlib
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.signal import find_peaks #For finding peaks (we'll see later)
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
import pickle
The data is saved in the data folder. We have 25 different csv files, for 5 players, 5 classes each. The naming convention is playerid_strokename. e.g. p2_bu.csv file contains data of player id 2 for "backhand underarm".
The columns in each file are: timestamp, accelerometer x-axis (ax), y-axis (ay), z-axis (az), gyroscope x-axis (gx), y-axis (gy), z-axis (gz). We also added magnitudes of accelerometer and gyroscope.
# Columns in data files
cols = ['ax', 'ay', 'az', 'gx', 'gy', 'gz'] # Accelerometer and gyroscope in 3 axis
addedCols = ['acc_mag', 'gyro_mag'] # Magnitude of acc and gyro
# Data reading
data = {}
persons = ["p1", "p2", "p3", "p4", "p5"]
shots = ["bo", "bu", "fo", "fs", "fu"]
for person in persons:
for shot in shots:
data[person + "_" + shot] = pd.read_csv("data/" + str(person) + "_" + str(shot) + ".csv")
data[person + "_" + shot]['acc_mag'] = np.linalg.norm(data[person + "_" + shot][cols[:3]].values,axis=1)
data[person + "_" + shot]['gyro_mag'] = np.linalg.norm(data[person + "_" + shot][cols[3:]].values,axis=1)
Let's plot one of the csv, say p5_bo.
plotName = "p5_bo"
# Set figure width as per the number of readings
plt.figure(figsize=(len(data[plotName])/10, 8), dpi=150)
# Plot for each accelerometer column
for col in cols[0:3]:
plt.plot(data[plotName][col], label=col)
plt.xlim(xmin=0)
plt.xticks(np.arange(0, len(data[plotName])+1, 5.0))
plt.title(plotName + "_acc")
plt.legend()
plt.show()
Data cleaning¶
The peaks in the above plot are the shots, and in between we have some rest times. The shots are not at uniform intervals, sometimes the rest period is very long (as in between 160 and 200 in above plot). So we need to extract out shots from this continous plot and remove all these players' resting periods.
The approach we took was to take a window around the peaks in the signal. Many design decisions were to be made, like, should the window by dynamic or fixed? If fixed, window size? peaks in which signal, accelerometer or gyroscope, x or y or z? one signal or many signals? how to consider some spike as peak? any thresholding? any noise reduction algorithm?
We did some EDA by plotting all the data and chose a window size of 13. This was our choice by our observations. And we chose the ay-signal to consider for extracting peaks and window, with different threshold values for each class of stroke.
# Size of the window, it is common for all shots
windowSize = 13
# shots = ["bo", "bu", "fo", "fs", "fu"] for reference
# Name of the sensor to be used for thresholding and the threshold value, the order is as per the shots list order
sensorToThreshold = ['ay', 'ay', 'ay', 'ay', 'ay']
threshold = [1.25, 1.25, 1.5, 1.5, 1.5]
# DATA PLOTTING - This will save all the plots in plots folder
for p_name in persons:
for s_name in shots:
s_index = shots.index(s_name) # Shot index of that shot
plotName = p_name + "_" + s_name
# Set figure width as per the number of readings
plt.figure(figsize=(len(data[plotName])/10, 5), dpi=100)
# Plot peaks with threashold of that shot and give the name of sensor used for thresholding
peaks, _ = find_peaks(data[plotName][sensorToThreshold[s_index]], height=threshold[s_index])
#plt.plot(peaks, data[plotName][sensorToThreshold[s_index]][peaks], "x", c='red')
# Plot for each accelerometer column
for col in cols[0:3]:
plt.plot(data[plotName][col], label=col)
# drawing windows around each peak
for peak in peaks:
plt.axvspan(int(peak - windowSize/2), int(peak + windowSize/2), alpha =.1, facecolor='g',edgecolor='black')
plt.xlim(xmin=0)
plt.xticks(np.arange(0, len(data[plotName])+1, 5.0))
plt.title(plotName + "_acc")
plt.legend()
plt.savefig("plots/" + plotName + "_acc")
#plt.show()
plt.close();
# Plot for each gyrometer column (not drawing peaks in these)
# Set figure width as per the number of readings
plt.figure(figsize=(len(data[plotName])/10, 5), dpi=100)
for col in cols[3:]:
plt.plot(data[plotName][col], label=col)
# drawing windows around each peak
for peak in peaks:
plt.axvspan(int(peak - windowSize/2), int(peak + windowSize/2), alpha =.1, facecolor='g',edgecolor='black')
plt.xlim(xmin=0)
plt.xticks(np.arange(0, len(data[plotName])+1, 5.0))
plt.title(plotName + "_gyro")
plt.legend()
plt.savefig("plots/" + plotName + "_gyro")
#plt.show()
plt.close();
# Magnitude - Acc
# Set figure width as per the number of readings
plt.figure(figsize=(len(data[plotName])/10, 5), dpi=100)
plt.plot(data[plotName][addedCols[0]], label="Magnitude Accelerometer")
# drawing windows around each peak
for peak in peaks:
plt.axvspan(int(peak - windowSize/2), int(peak + windowSize/2), alpha =.1, facecolor='g',edgecolor='black')
plt.xlim(xmin=0)
plt.xticks(np.arange(0, len(data[plotName])+1, 5.0))
plt.title(plotName + "_mag_acc")
plt.legend()
plt.savefig("plots/" + plotName + "_mag_acc")
#plt.show()
plt.close();
# Magnitude - Gyro
# Set figure width as per the number of readings
plt.figure(figsize=(len(data[plotName])/10, 5), dpi=100)
plt.plot(data[plotName][addedCols[1]], label="Magnitude Gyroscope")
# drawing windows around each peak
for peak in peaks:
plt.axvspan(int(peak - windowSize/2), int(peak + windowSize/2), alpha =.1, facecolor='g',edgecolor='black')
plt.xlim(xmin=0)
plt.xticks(np.arange(0, len(data[plotName])+1, 5.0))
plt.title(plotName + "_mag_gyro")
plt.legend()
plt.savefig("plots/" + plotName + "_mag_gyro")
#plt.show()
plt.close();
Now this is what our extracted shots look like.
Accelerometer:
Gyroscope:
There are some overlaps, either because of shots being very close to each other or because of double peaks showing up in some shots. And some shots are missed because of low threshold value. But this is what we found to give reasonable balance between extracting good shots and not the noisy ones.
We'll save the start and end frame for each of these extracted shots in an X_y dataframe, along with the true labels. Later on, we'll augment this dataframe with hand engineered features. We are also saving person id although it's not used in classification.
# Final data frame with features described in doc along with shot name
X_y = pd.DataFrame(columns=['StartFrame', 'EndFrame', 'PersonID', 'ShotName'])
# Creating final data frame and adding end and begin frame of window
for p_name in persons:
for s_name in shots:
s_index = shots.index(s_name) # Shot index of that shot
plotName = p_name + "_" + s_name
# Find peaks for window
#TODO: we can look at peak_features(second returned value) for more data features
timeSeries = data[plotName][sensorToThreshold[s_index]]
peaks, _ = find_peaks(timeSeries, height=threshold[s_index])
for peak in peaks:
if(peak < windowSize/2 or peak > len(timeSeries)-windowSize/2):
#print(peak)
continue
d = {'StartFrame': int(peak - windowSize/2),
'EndFrame': int(peak + windowSize/2),
'PersonID': p_name,
'ShotName': s_name}
#print(d)
X_y = X_y.append(d, ignore_index=True)
X_y
We have extracted the shots out of the continous data we had. Total data: 708 (Not much, but let's see). Now let's see how much imbalance is there in our data. We'll handle that (if any). And then move on to adding features.
sns.set_style('whitegrid')
plt.figure(figsize=(16,8))
plt.title('Data provided by each user', fontsize=20)
sns.countplot(x='PersonID', hue='ShotName', data = X_y)
plt.savefig("plots/data_count")
plt.show()
shotFullName = {
'fo': "Overhead Forehand",
'bo': "Overhead Backhand",
'fs': "Forehand Smash",
'fu': "Underarm Forehand Stroke",
'bu': "Underarm Backhand Stroke",
}
plt.title('No of Datapoints per Stroke', fontsize=15)
sns.countplot([shotFullName[i] for i in X_y.ShotName]).set_xticklabels([shotFullName[i] for i in shots],
rotation=30, horizontalalignment='right')
plt.tight_layout()
plt.savefig("plots/stroke_count")
plt.show()
There is a lot of difference in the amount of data we collected from each person (We didn't realized it during collection). But there isn't much imbalace between the classes (there is, but not much), so we'll work with this only.
Adding Features for classical ML models¶
We will be using this X_y dataframe to store all our features for all the samples. And then use it for training classical machine learning models.
After some searching we found following features that we can add to such a time series data.
- Range
- Minimum
- Maximum
- Average
- Absolute Average
- Kurtosis f
- Kurtosis p
- Skewness statistic and p value
- Entropy
- Standard Deviation
- Angle betweenness
- Inter quartile range
- maxmin relative position ( Max position - min position , to see if max comes before min or vice-versa)
# List of features we'll add
features = []
# Some helper functions
# Add feature which depends only on one sensor, like range
def add_feature(fname, sensor):
v = [fname(data[str(row['PersonID']) + "_" + str(row['ShotName'])][int(row['StartFrame']):int(row['EndFrame'])],
sensor)
for index, row in X_y.iterrows()]
X_y[fname.__name__ + str(sensor)] = v
if(fname.__name__ + str(sensor) not in features):
features.append(fname.__name__ + str(sensor))
print("Added feature " + fname.__name__ + str(sensor) + " for " + str(len(v)) + " rows.")
# Add feature which depends on more than one sensors, like magnitude
def add_feature_mult_sensor(fname, sensors):
v = [fname(data[str(row['PersonID']) + "_" + str(row['ShotName'])][int(row['StartFrame']):int(row['EndFrame'])],
sensors)
for index, row in X_y.iterrows()]
name = "_".join(sensors)
X_y[fname.__name__ + name] = v
if(fname.__name__ + name not in features):
features.append(fname.__name__ + name)
print("Added feature " + fname.__name__ + name + " for " + str(len(v)) + " rows.")
# Range
def range_(df, sensor):
return np.max(df[sensor]) - np.min(df[sensor])
for sensor in cols + addedCols:
add_feature(range_, sensor)
# Minimum
def min_(df, sensor):
return np.min(df[sensor])
for sensor in cols + addedCols:
add_feature(min_, sensor)
# Maximum
def max_(df, sensor):
return np.max(df[sensor])
for sensor in cols + addedCols:
add_feature(max_, sensor)
# Average
def avg_(df, sensor):
return np.mean(df[sensor])
for sensor in cols + addedCols:
add_feature(avg_, sensor)
# Absolute Average
def absavg_(df, sensor):
return np.mean(np.absolute(df[sensor]))
for sensor in cols + addedCols:
add_feature(absavg_, sensor)
def kurtosis_f_(df , sensor):
from scipy.stats import kurtosis
val = kurtosis(df[sensor],fisher = True)
return val
for sensor in cols + addedCols:
add_feature(kurtosis_f_, sensor)
def kurtosis_p_(df , sensor):
from scipy.stats import kurtosis
val = kurtosis(df[sensor],fisher = False)
return val
for sensor in cols + addedCols:
add_feature(kurtosis_p_, sensor)
#skewness
def skewness_statistic_(df, sensor):
if(len(df) == 0):
print(df)
from scipy.stats import skewtest
statistic, pvalue = skewtest(df[sensor], nan_policy='propagate')
return statistic
for sensor in cols + addedCols:
add_feature(skewness_statistic_, sensor)
def skewness_pvalue_(df, sensor):
from scipy.stats import skewtest
statistic, pvalue = skewtest(df[sensor])
return pvalue
for sensor in cols + addedCols:
add_feature(skewness_pvalue_, sensor)
#entropy
def entropy_(df, sensor):
from scipy.stats import entropy
ent = entropy(df[sensor])
return ent
for sensor in addedCols:
add_feature(entropy_, sensor)
# Standard Deviation
def std_(df, sensor):
return np.std(df[sensor])
for sensor in cols + addedCols:
add_feature(std_, sensor)
#angle between two vectors
def anglebetween_(df, sensors):
v1 = sensors[0]
v2 = sensors[1]
v1_u = df[v1] / np.linalg.norm(df[v1])
v2_u = df[v2] / np.linalg.norm(df[v2])
return np.arccos(np.clip(np.dot(v1_u, v2_u), -1.0, 1.0))
add_feature_mult_sensor(anglebetween_, ["ax", "ay"])
add_feature_mult_sensor(anglebetween_, ["ay", "az"])
add_feature_mult_sensor(anglebetween_, ["ax", "az"])
add_feature_mult_sensor(anglebetween_, ["gx", "gy"])
add_feature_mult_sensor(anglebetween_, ["gy", "gz"])
add_feature_mult_sensor(anglebetween_, ["gx", "gz"])
#inter quartile range
def iqr_(df, sensor):
from scipy import stats
return stats.iqr(df[sensor])
for sensor in cols + addedCols:
add_feature(iqr_, sensor)
# Max position - min position (relative difference)
def maxmin_relative_pos_(df, sensor):
return np.argmax(np.array(df[sensor])) - np.argmin(np.array(df[sensor]))
for sensor in cols + addedCols:
add_feature(maxmin_relative_pos_, sensor)
Saving the processed data¶
Our final X_y, with all the hand-engineered features, is ready. Now we can do some "Machine Learning" on it. But before that, it's better to save this X_y, so that we need not do any preprocessing and we can do all ML model testing on a different notebook. (For case of this article, we'll continue in same notebook).
X_y
# Total number of features
len(features)
# Save all the features in a txt file for later use.
with open('data/features.txt', 'w') as f:
for feature in features:
f.write("%s\n" %feature)
# Save X_y as csv file for using in (classical) ML models
X_y.to_csv('data/X_y.csv', index=False)
Classical ML models¶
Now it's time for some machine learning model stuff! We'll load this saved X_y. Then apply some classical machine learning models and asses their performance by plotting the confusion matrix. For paramter tuning in each of the model, we used grid searching cross-validation.
# Read Features
with open('data/features.txt') as f:
features = f.read().strip().split("\n")
f.close()
# Load data
X_y = pd.read_csv('data/X_y.csv')
X_y = X_y.dropna()
shot_labels = X_y.ShotName.unique()
# Train Test split Randomly:
from sklearn.model_selection import train_test_split
train, test = train_test_split(X_y, test_size=0.2, random_state=42)
X_train = train[features].values
Y_train = train["ShotName"].values
X_test = test[features].values
Y_test = test["ShotName"].values
# Helper function for plotting confusion matrix
from sklearn.metrics import confusion_matrix
def plot_confusion_matrix(cm, shots,
model_name,
normalize=False,
cmap=plt.cm.Wistia):
tick_marks = np.arange(len(shots))
if normalize:
cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
plt.imshow(cm, interpolation='nearest', cmap=cmap)
plt.yticks(tick_marks, shots)
plt.title("Confusion matrix - " + model_name)
plt.colorbar()
plt.xticks(tick_marks, shots, rotation='vertical')
fmt = '.2f' if normalize else 'd'
thresh = cm.max() / 2.
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
plt.text(j, i, format(cm[i, j], fmt),
horizontalalignment="center",
color="black")
plt.tight_layout()
plt.ylabel('True Shot')
plt.xlabel('Predicted Shot')
plt.savefig("plots/" + "Confusion matrix - " + model_name)
Logistic Regression¶
from sklearn import linear_model
from sklearn import metrics
from sklearn.model_selection import GridSearchCV
# Save the hyperparameters ie C value and loss-function type:
parameters = {'C':[0.01,0.1,1,10,20,30], 'penalty':['l2','l1']}
log_reg_clf = linear_model.LogisticRegression()
log_reg_model = GridSearchCV(log_reg_clf, param_grid=parameters, cv=3,verbose=1, n_jobs=8)
log_reg_model.fit(X_train,Y_train)
y_pred = log_reg_model.predict(X_test)
# y_prob = log_reg_model.predict_proba(X_test)
# print(y_prob)
accuracy = metrics.accuracy_score(y_true=Y_test,y_pred=y_pred)
# Accuracy of our stroke detectiony
print('Accuracy of strokes detection: {}\n\n'.format(accuracy))
# confusion matrix
cm = metrics.confusion_matrix(Y_test, y_pred)
# plot confusion matrix
plt.figure(figsize=(8,8))
plt.grid(b=False)
plot_confusion_matrix(cm, model_name="Logistic Regression",
shots=shot_labels, normalize=True)
plt.show()
# get classification report
print("Classifiction Report for this model")
classification_report = metrics.classification_report(Y_test, y_pred)
print(classification_report)
K- Nearest Neighbours¶
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=4)
knn.fit(X_train, Y_train)
y_pred = knn.predict(X_test)
accuracy = metrics.accuracy_score(y_true=Y_test,y_pred=y_pred)
# Accuracy of our stroke detectiony
print('Accuracy of strokes detection: {}\n\n'.format(accuracy))
# confusion matrix
cm = metrics.confusion_matrix(Y_test, y_pred)
# plot confusion matrix
plt.figure(figsize=(8,8))
plt.grid(b=False)
plot_confusion_matrix(cm, model_name='KNeighborsClassifier',
shots=shot_labels, normalize=True, )
plt.show()
# get classification report
print("Classifiction Report for this model")
classification_report = metrics.classification_report(Y_test, y_pred)
print(classification_report)
Linear SVC¶
from sklearn.svm import LinearSVC
parameters = {'C':[0.125, 0.5, 1, 2, 8, 16]}
lr_svc_reg_clf = LinearSVC(tol=0.00005)
lr_svc_reg_model = GridSearchCV(lr_svc_reg_clf, param_grid=parameters, n_jobs=8, verbose=1)
lr_svc_reg_model.fit(X_train,Y_train)
y_pred = lr_svc_reg_model.predict(X_test)
accuracy = metrics.accuracy_score(y_true=Y_test,y_pred=y_pred)
# Accuracy of our stroke detectiony
print('Accuracy of strokes detection: {}\n\n'.format(accuracy))
# confusion matrix
cm = metrics.confusion_matrix(Y_test, y_pred)
# plot confusion matrix
plt.figure(figsize=(8,8))
plt.grid(b=False)
plot_confusion_matrix(cm, model_name='LinearSVC',
shots=shot_labels, normalize=True)
plt.show()
# get classification report
print("Classifiction Report for this model")
classification_report = metrics.classification_report(Y_test, y_pred)
print(classification_report)
SVC with RBF kernel¶
from sklearn.svm import SVC
parameters = {'C':[2,8,16],\
'gamma': [ 0.0078125, 0.125, 2]}
rbf_svm_clf = SVC(kernel='rbf')
rbf_svm_model = GridSearchCV(rbf_svm_clf,param_grid=parameters,n_jobs=8)
rbf_svm_model.fit(X_train,Y_train )
y_pred = rbf_svm_model.predict(X_test)
accuracy = metrics.accuracy_score(y_true=Y_test,y_pred=y_pred)
# Accuracy of our stroke detectiony
print('Accuracy of strokes detection: {}\n\n'.format(accuracy))
# confusion matrix
cm = metrics.confusion_matrix(Y_test, y_pred)
# plot confusion matrix
plt.figure(figsize=(8,8))
plt.grid(b=False)
plot_confusion_matrix(cm, model_name='SVC', shots=shot_labels, normalize=True)
plt.show()
# get classification report
print("Classifiction Report for this model")
classification_report = metrics.classification_report(Y_test, y_pred)
print(classification_report)
from sklearn.tree import DecisionTreeClassifier
parameters = {'max_depth':np.arange(3,20,2)}
decision_trees_clf = DecisionTreeClassifier()
decision_trees = GridSearchCV(decision_trees_clf, param_grid=parameters, n_jobs=8)
decision_trees.fit(X_train,Y_train )
y_pred = decision_trees.predict(X_test)
accuracy = metrics.accuracy_score(y_true=Y_test,y_pred=y_pred)
# Accuracy of our stroke detection
print('Accuracy of strokes detection: {}\n\n'.format(accuracy))
# confusion matrix
cm = metrics.confusion_matrix(Y_test, y_pred)
# plot confusion matrix
plt.figure(figsize=(8,8))
plt.grid(b=False)
plot_confusion_matrix(cm, model_name='Decision Tree',
shots=shot_labels, normalize=True)
plt.show()
# get classification report
print("Classifiction Report for this model")
classification_report = metrics.classification_report(Y_test, y_pred)
print(classification_report)
Random Forest¶
from sklearn.ensemble import RandomForestClassifier
params = {'n_estimators': np.arange(10,120,20), 'max_depth':np.arange(3,15,2)}
rfclassifier_clf = RandomForestClassifier()
rfclassifier = GridSearchCV(rfclassifier_clf, param_grid=params, n_jobs=8)
rfclassifier.fit(X_train,Y_train )
y_pred = rfclassifier.predict(X_test)
accuracy = metrics.accuracy_score(y_true=Y_test,y_pred=y_pred)
# Accuracy of our stroke detection
print('Accuracy of strokes detection: {}\n\n'.format(accuracy))
# confusion matrix
cm = metrics.confusion_matrix(Y_test, y_pred)
# plot confusion matrix
plt.figure(figsize=(8,8))
plt.grid(b=False)
plot_confusion_matrix(cm, model_name='Random Forest',
shots=shot_labels, normalize=True)
plt.show()
# get classification report
print("Classifiction Report for this model")
classification_report = metrics.classification_report(Y_test, y_pred)
print(classification_report)
Gradient Boosting¶
from sklearn.ensemble import GradientBoostingClassifier
param_grid = {'max_depth': np.arange(1,30,4), \
'n_estimators':np.arange(1,300,15)}
gbdt_clf = GradientBoostingClassifier()
gbdt_model = GridSearchCV(gbdt_clf, param_grid=param_grid, n_jobs=8)
gbdt_model.fit(X_train,Y_train )
y_pred = gbdt_model.predict(X_test)
accuracy = metrics.accuracy_score(y_true=Y_test,y_pred=y_pred)
# Accuracy of our stroke detectiony
print('Accuracy of strokes detection: {}\n\n'.format(accuracy))
# confusion matrix
cm = metrics.confusion_matrix(Y_test, y_pred)
# plot confusion matrix
plt.figure(figsize=(8,8))
plt.grid(b=False)
plot_confusion_matrix(cm, model_name='GradientBoostingClassifier',
shots=shot_labels, normalize=True)
plt.show()
# get classification report
print("Classifiction Report for this model")
classification_report = metrics.classification_report(Y_test, y_pred)
print(classification_report)
Deep Learning Models¶
Applying some deep learning models. The obvious choice will be to apply 1D CNN or RNNs for such a task. We tried only LSTMs.
# Importing tensorflow
np.random.seed(42)
import tensorflow as tf
tf.random.set_seed(42)
from sklearn.preprocessing import StandardScaler
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import Dropout
from keras.layers.convolutional import Conv1D
from keras.layers.convolutional import MaxPooling1D
from keras.models import Sequential
from keras.layers import LSTM
from keras.layers.core import Dense, Dropout
# ShotNames are the class labels
# It is a 5 class classification
ShotNames = {
'bo': [1, 0, 0, 0, 0],
'bu': [0, 1, 0, 0, 0],
'fo': [0, 0, 1, 0, 0],
'fs': [0, 0, 0, 1, 0],
'fu': [0, 0, 0, 0, 1],
}
X = []
y = []
for index, row in X_y.iterrows():
df = data[row["PersonID"] + "_" + row["ShotName"]][row["StartFrame"]:row["EndFrame"]][cols]
X.append(df.to_numpy())
y.append(row["ShotName"])
X = np.array(X)
# One Hot Encoding
y = np.array([ShotNames[i] for i in y])
n_classes = len(ShotNames)
timesteps = len(X[0]) # Window size
input_dim = len(X[0][0]) # num of sensors = 6
# Initializing parameters
epochs = 100
batch_size = 16
n_hidden = 32
# Loading the train and test data
from sklearn.model_selection import train_test_split
X_train, X_test, Y_train , Y_test = train_test_split(X, y, test_size=0.1)
LSTM¶
# Initiliazing the sequential model
model = Sequential()
# Configuring the parameters
model.add(LSTM(n_hidden, input_shape=(timesteps, input_dim)))
# Adding a dropout layer
model.add(Dropout(0.5))
# Adding a dense output layer with sigmoid activation
model.add(Dense(n_classes, activation='sigmoid'))
model.summary()
# Compiling the model
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# Training the model
model.fit(X_train, Y_train,
batch_size=batch_size,
validation_data=(X_test, Y_test),
epochs=epochs)
Data being pretty less for deep learning models, so we didn't try deeper networks.
TODO¶
- Data Augmentation
- Try using 1D-CNN
- Attention models
Final Notes¶
This project taught us a lot about doing a machine learning project. Specially the challenges that arise in data collection and it's preprocessing. Overall this is just a small project we did for learning. It can be extended in several ways, first of all by getting more data, so that models like LSTM can be employed. For classical models, we can add more features, do PCA, etc.
Overall a great learning experience. Please feel free to play with it and suggestions are always welcome.