You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1.6 MiB
1.6 MiB
<html>
<head>
</head>
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
</html>
K-Means Clustering¶
Let's work through an example of unsupervised learning - clustering customer data.
Goal:¶
When working with unsupervised learning methods, its usually important to lay out a general goal. In our case, let's attempt to find reasonable clusters of customers for marketing segmentation and study. What we end up doing with those clusters would depend heavily on the domain itself, in this case, marketing.
The Data¶
LINK: https://archive.ics.uci.edu/ml/datasets/bank+marketing
This dataset is public available for research. The details are described in [Moro et al., 2011].
[Moro et al., 2011] S. Moro, R. Laureano and P. Cortez. Using Data Mining for Bank Direct Marketing: An Application of the CRISP-DM Methodology.
In P. Novais et al. (Eds.), Proceedings of the European Simulation and Modelling Conference - ESM'2011, pp. 117-121, Guimarães, Portugal, October, 2011. EUROSIS.
Available at: [pdf] http://hdl.handle.net/1822/14838
[bib] http://www3.dsi.uminho.pt/pcortez/bib/2011-esm-1.txt
For more information, read [Moro et al., 2011].
# bank client data:
1 - age (numeric)
2 - job : type of job (categorical: 'admin.','blue-collar','entrepreneur','housemaid','management','retired','self-employed','services','student','technician','unemployed','unknown')
3 - marital : marital status (categorical: 'divorced','married','single','unknown'; note: 'divorced' means divorced or widowed)
4 - education (categorical: 'basic.4y','basic.6y','basic.9y','high.school','illiterate','professional.course','university.degree','unknown')
5 - default: has credit in default? (categorical: 'no','yes','unknown')
6 - housing: has housing loan? (categorical: 'no','yes','unknown')
7 - loan: has personal loan? (categorical: 'no','yes','unknown')
# related with the last contact of the current campaign:
8 - contact: contact communication type (categorical: 'cellular','telephone')
9 - month: last contact month of year (categorical: 'jan', 'feb', 'mar', ..., 'nov', 'dec')
10 - day_of_week: last contact day of the week (categorical: 'mon','tue','wed','thu','fri')
11 - duration: last contact duration, in seconds (numeric). Important note: this attribute highly affects the output target (e.g., if duration=0 then y='no'). Yet, the duration is not known before a call is performed. Also, after the end of the call y is obviously known. Thus, this input should only be included for benchmark purposes and should be discarded if the intention is to have a realistic predictive model.
# other attributes:
12 - campaign: number of contacts performed during this campaign and for this client (numeric, includes last contact)
13 - pdays: number of days that passed by after the client was last contacted from a previous campaign (numeric; 999 means client was not previously contacted)
14 - previous: number of contacts performed before this campaign and for this client (numeric)
15 - poutcome: outcome of the previous marketing campaign (categorical: 'failure','nonexistent','success')
# social and economic context attributes
16 - emp.var.rate: employment variation rate - quarterly indicator (numeric)
17 - cons.price.idx: consumer price index - monthly indicator (numeric)
18 - cons.conf.idx: consumer confidence index - monthly indicator (numeric)
19 - euribor3m: euribor 3 month rate - daily indicator (numeric)
20 - nr.employed: number of employees - quarterly indicator (numeric)
21 - y - has the client subscribed a term deposit? (binary: 'yes','no')
Imports¶
In [166]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
Exploratory Data Analysis¶
In [167]:
df = pd.read_csv("../DATA/bank-full.csv")
In [168]:
df.head()
Out[168]:
In [169]:
df.columns
Out[169]:
In [170]:
df.info()
Continuous Feature Analysis¶
In [171]:
plt.figure(figsize=(12,6),dpi=200)
sns.histplot(data=df,x='age')
Out[171]:
In [172]:
plt.figure(figsize=(12,6),dpi=200)
sns.histplot(data=df,x='age',hue='loan')
Out[172]:
In [174]:
plt.figure(figsize=(12,6),dpi=200)
sns.histplot(data=df,x='pdays')
Out[174]:
In [175]:
plt.figure(figsize=(12,6),dpi=200)
sns.histplot(data=df[df['pdays']!=999],x='pdays')
Out[175]:
Contact duration - contact with customer made, how long did call last?
In [176]:
plt.figure(figsize=(12,6),dpi=200)
sns.histplot(data=df,x='duration',hue='contact')
plt.xlim(0,2000)
Out[176]:
- 15 - previous: number of contacts performed before this campaign and for this client (numeric)
- 16 - poutcome: outcome of the previous marketing campaign (categorical: "unknown","other","failure","success"
In [177]:
plt.figure(figsize=(12,6),dpi=200)
sns.countplot(data=df,x='previous',hue='contact')
Out[177]:
In [178]:
sns.countplot(data=df,x='contact')
Out[178]:
In [179]:
# df['previous'].value_counts()
df['previous'].value_counts().sum()-36954
# 36954 vs. 8257
Out[179]:
Categorical Features¶
In [180]:
df.head()
Out[180]:
In [181]:
plt.figure(figsize=(12,6),dpi=200)
# https://stackoverflow.com/questions/46623583/seaborn-countplot-order-categories-by-count
sns.countplot(data=df,x='job',order=df['job'].value_counts().index)
plt.xticks(rotation=90);
In [182]:
plt.figure(figsize=(12,6),dpi=200)
# https://stackoverflow.com/questions/46623583/seaborn-countplot-order-categories-by-count
sns.countplot(data=df,x='education',order=df['education'].value_counts().index)
plt.xticks(rotation=90);
In [183]:
plt.figure(figsize=(12,6),dpi=200)
# https://stackoverflow.com/questions/46623583/seaborn-countplot-order-categories-by-count
sns.countplot(data=df,x='education',order=df['education'].value_counts().index,hue='default')
plt.xticks(rotation=90);
In [184]:
sns.countplot(data=df,x='default')
Out[184]:
In [185]:
# THIS TAKES A LONG TIME!
sns.pairplot(df)
Out[185]:
Clustering¶
Data Preparation¶
UNSUPERVISED LEARNING REMINDER: NO NEED TO TRAIN TEST SPLIT!! NO LABEL TO "TEST" AGAINST!
We do however need to transform categorical features into numeric ones where it makes sense to do so, as well as scaling the data due to distance being a key factor in clustering.
In [186]:
df.head()
Out[186]:
In [192]:
X = pd.get_dummies(df)
In [193]:
X
Out[193]:
In [194]:
from sklearn.preprocessing import StandardScaler
In [195]:
scaler = StandardScaler()
In [196]:
scaled_X = scaler.fit_transform(X)
Creating and Fitting a KMeans Model¶
Note of our method choices here:
fit(X[, y, sample_weight])
- Compute k-means clustering.
fit_predict(X[, y, sample_weight])
- Compute cluster centers and predict cluster index for each sample.
fit_transform(X[, y, sample_weight])
- Compute clustering and transform X to cluster-distance space.
predict(X[, sample_weight])
- Predict the closest cluster each sample in X belongs to.
In [197]:
from sklearn.cluster import KMeans
In [198]:
model = KMeans(n_clusters=2)
In [199]:
# Make sure to watch video to understand this line and fit() vs transform()
cluster_labels = model.fit_predict(scaled_X)
In [200]:
# IMPORTANT NOTE: YOUR 0s and 1s may be opposite of ours,
# makes sense, the number values are not significant!
cluster_labels
Out[200]:
In [201]:
len(scaled_X)
Out[201]:
In [202]:
len(cluster_labels)
Out[202]:
In [203]:
X['Cluster'] = cluster_labels
In [204]:
sns.heatmap(X.corr())
Out[204]:
In [205]:
X.corr()['Cluster']
Out[205]:
In [206]:
plt.figure(figsize=(12,6),dpi=200)
X.corr()['Cluster'].iloc[:-1].sort_values().plot(kind='bar')
Out[206]:
Choosing K Value¶
In [207]:
ssd = []
for k in range(2,10):
model = KMeans(n_clusters=k)
model.fit(scaled_X)
#Sum of squared distances of samples to their closest cluster center.
ssd.append(model.inertia_)
In [212]:
plt.plot(range(2,10),ssd,'o--')
plt.xlabel("K Value")
plt.ylabel(" Sum of Squared Distances")
Out[212]:
Analyzing SSE Reduction
In [213]:
ssd
Out[213]:
In [217]:
# Change in SSD from previous K value!
pd.Series(ssd).diff()
Out[217]:
In [230]:
pd.Series(ssd).diff().plot(kind='bar')
Out[230]: