Now that we have a model, how can we use it? The most typical example of using a collaborative filtering model is to recommend items to people they have not rated yet. That concept works well in domains such as music and movie recommendations where collaborative filtering models are often applied. However, we are going to use it for a different purpose. One concern of marketing managers is the Share of wallet they get from a customer. The definition of this (from https://en.wikipedia.org/wiki/Share_of_wallet) is the percentage ('share') of a customer's expenses ('of wallet') for a product that goes to the firm selling the product. It basically measures the value of a customer on the percentage of the potential spend they could have with us. As an example, we may have customers who visit our shop regularly and spend a considerable amount. But are they buying all of their goods from us? Maybe they buy their fresh food elsewhere, that is, they purchase their meat, fruit, vegetables, and so on, at other stores. We can use collaborative filtering to find customers where the collaborative filtering model predicts that they purchase certain products in our store, but in fact they do not. Remember that collaborative filtering works on the basis of making recommendations based on what other similar customers do. So, if customer A does not purchase meat, fruit, vegetables, and so on, at our store when other similar customers do, then we could try and entice them to spend more at our stores by sending them offers for these products.
We will look for customer-product department codes where the prediction is greater than 4, but the actual value is less than 2. These customers should be purchasing these goods from us (according to the model), so by sending them vouchers for items in these departments, we can capture a greater amount of their spending.
A collaborative filtering model should work well for this type of analysis. The basis of this algorithm is to find the recommend products based on the activity of similar customers, so it already adjusts for the scale of spend. For example, if the prediction for a customer is that their spend on fresh fruit and vegetables should be 5, that is based on the comparison with other similar customers. Here is the evaluation code, which is also in Chapter8/kerarecommend.R. The first part of the code generates the predictions and links it back. We output a few metrics, which look impressive, but note that they are run on all the data, including the data that the model was trained on, so these metrics are overly optimistic. We make one adjustment to the predictions – some of these values are greater than 5 or less than 1, so we change them back to valid values. This produces a very small improvement on our metrics:
##### model use-case, find products that customers 'should' be purchasing ######
df$preds<-predict(model,list(df$cust_id2,df$prod_id2))
# remove index variables, do not need them anymore
df$cust_id2 <- NULL
df$prod_id2 <- NULL
mse<-mean((df$rating-df$preds)^2)
rmse<-sqrt(mse)
mae<-mean(abs(df$rating-df$preds))
print (sprintf("DL Collaborative filtering model: MSE=%1.3f, RMSE=%1.3f, MAE=%1.3f",mse,rmse,mae))
[1] "DL Collaborative filtering model: MSE=0.478, RMSE=0.691, MAE=0.501"
df <- df[order(-df$preds),]
head(df)
prod_id cust_id rating preds
10017 D00003 CUST0000283274 5 5.519783
4490 D00002 CUST0000283274 5 5.476133
9060 D00002 CUST0000084449 5 5.452055
6536 D00002 CUST0000848462 5 5.447111
10294 D00003 CUST0000578851 5 5.446453
7318 D00002 CUST0000578851 5 5.442836
df[df$preds>5,]$preds <- 5
df[df$preds<1,]$preds <- 1
mse<-mean((df$rating-df$preds)^2)
rmse<-sqrt(mse)
mae<-mean(abs(df$rating-df$preds))
print (sprintf("DL Collaborative filtering model (adjusted): MSE=%1.3f, RMSE=%1.3f, MAE=%1.3f",mse,rmse,mae))
[1] "DL Collaborative filtering model (adjusted): MSE=0.476, RMSE=0.690, MAE=0.493"
Now, we can look at the customer-product department codes that have the biggest difference between predicted ratings and actual ratings:
df$diff <- df$preds - df$rating
df <- df[order(-df$diff),]
head(df,20)
prod_id cust_id rating preds diff
3259 D00001 CUST0000375633 1 5.000000 4.000000
12325 D00003 CUST0000038166 1 4.306837 3.306837
14859 D00004 CUST0000817991 1 4.025836 3.025836
15279 D00004 CUST0000620867 1 4.016025 3.016025
22039 D00008 CUST0000588390 1 3.989520 2.989520
3370 D00001 CUST0000530875 1 3.969685 2.969685
22470 D00008 CUST0000209037 1 3.927513 2.927513
22777 D00008 CUST0000873432 1 3.905162 2.905162
13905 D00004 CUST0000456347 1 3.877517 2.877517
18123 D00005 CUST0000026547 1 3.853488 2.853488
24208 D00008 CUST0000732836 1 3.810606 2.810606
22723 D00008 CUST0000872856 1 3.746022 2.746022
22696 D00008 CUST0000549120 1 3.718482 2.718482
15463 D00004 CUST0000035935 1 3.714494 2.714494
24090 D00008 CUST0000643072 1 3.679629 2.679629
21167 D00006 CUST0000454947 1 3.651651 2.651651
23769 D00008 CUST0000314496 1 3.649187 2.649187
14294 D00004 CUST0000127124 1 3.625893 2.625893
22534 D00008 CUST0000556279 1 3.578591 2.578591
22201 D00008 CUST0000453430 1 3.576008 2.576008
This gives us a list of customers and the products we should send them offers for. For example, for the second row, the actual rating is 1 and the predicted rating is 4.306837. This customer is not purchasing the items for this product code and our model predicts he should be purchasing these items.
We can also look at cases where the actual rating is much higher than the predicted value. These are customers who are over-spending in that department compared to other similar customers:
df <- df[order(df$diff),]
head(df,20)
prod_id cust_id rating preds diff
21307 D00006 CUST0000555858 5 1.318784 -3.681216
15353 D00004 CUST0000640069 5 1.324661 -3.675339
21114 D00006 CUST0000397007 5 1.729860 -3.270140
23097 D00008 CUST0000388652 5 1.771072 -3.228928
21551 D00006 CUST0000084985 5 1.804969 -3.195031
21649 D00007 CUST0000083736 5 1.979534 -3.020466
23231 D00008 CUST0000917696 5 2.036216 -2.963784
21606 D00007 CUST0000899988 5 2.050258 -2.949742
21134 D00006 CUST0000373894 5 2.071380 -2.928620
14224 D00004 CUST0000541731 5 2.081161 -2.918839
15191 D00004 CUST0000106540 5 2.162569 -2.837431
13976 D00004 CUST0000952727 5 2.174777 -2.825223
21851 D00008 CUST0000077294 5 2.202812 -2.797188
16545 D00004 CUST0000945695 5 2.209504 -2.790496
23941 D00008 CUST0000109728 5 2.224301 -2.775699
24031 D00008 CUST0000701483 5 2.239778 -2.760222
21300 D00006 CUST0000752292 5 2.240073 -2.759927
21467 D00006 CUST0000754753 5 2.240705 -2.759295
15821 D00004 CUST0000006239 5 2.264089 -2.735911
15534 D00004 CUST0000586590 5 2.272885 -2.727115
What can we do with these recommendations? Our model assigns a score of 1-5 based on a customers' spend in each product department, so if a customer has a high actual rating compared to the predicted value in general, they are over-spending in these departments compared to similar customers. These people are probably not spending in other departments, so they should be targeted as part of a cross-sell campaign; that is, they should be sent offers for products in other departments to tempt them to purchase there.