3. Model-Based Collaborative Filtering

From a practical point of view, preprocessing is essential for efficiency. However, one could implement the neighborhood method without a preprocessing phase, albeit with larger latencies at query time.

Parameter-tuning methods, such as hold-out and cross-validation, are discussed in Chapter  7

It is also possible to use more sophisticated ways of removing bias for better performance. For example, the bias B _ij , which is specific to user i and item j, can be computed using the approach discussed in section  3.7.1. This bias is subtracted from observed entries and all missing entries are initialized to 0s during pre-processing. After computing the predictions, the biases B _ij are added back to the predicted values during postprocessing.

A detailed description of the method used for performing this estimation in various scenarios is discussed in section  3.6.5.3.

The row space of a matrix is defined by all possible linear combinations of the rows of the matrix. The column space of a matrix is defined by all possible linear combinations of the columns of the matrix.

In SVD [ 568], the basis vectors are also referred to as singular vectors, which, by definition, must be mutually orthonormal.

Refer to Chapter  6 for a discussion of the bias-variance trade-off.

A more precise update should be \(\overline{u_{i}} \Leftarrow \overline{u_{i}} +\alpha (e_{ij}\overline{v_{j}} -\lambda \overline{u_{i}}/n_{i}^{user})\) and \(\overline{v_{j}} \Leftarrow \overline{v_{j}} +\alpha (e_{ij}\overline{u_{i}} -\lambda \overline{v_{j}}/n_{j}^{item})\). Here, n _i ^user represents the number of observed ratings for user i and \(n_{j}^{item}\) represents the number of observed ratings for item j. Here, the regularization terms for various user/item factors are divided equally among the corresponding observed entries for various users/items. In practice, the (simpler) heuristic update rules discussed in the chapter are often used. We have chosen to use these (simpler) rules throughout this chapter to be consistent with the research literature on recommender systems. With proper parameter tuning, \(\lambda\) will automatically adjust to a smaller value in the case of the simpler update rules.

The inner-product of two column-vectors \(\overline{x}\) and \(\overline{y}\) is given by the scalar \(\overline{x}^{T}\overline{y}\), whereas the outer-product is given by the rank-1 matrix \(\overline{x}\,\overline{y}^{T}\). Furthermore, \(\overline{x}\) and \(\overline{y}\) need not be of the same size in order to compute an outer-product.

In many cases, this approach can outperform SVD + +, especially when the number of observed ratings is small.

For matrices, which are not mean-centered, the global mean can be subtracted during preprocessing and then added back at prediction time.

We use a slightly different notation than the original paper [ 309], although the approach described here is equivalent. This presentation simplifies the notation by introducing fewer variables and viewing bias variables as constraints on the factorization process.

The literature often describes these updates in vectorized form. These updates may be applied to the rows of U, V, and Y as follows:

These effects are best understood in terms of the bias-variance trade-off in machine learning [ 22]. Setting the unspecified values to 0 increases bias, but it reduces variance. When a large number of entries are unspecified, and the prior probability of a missing entry to be 0 is very high, the variance effects can dominate.

Refer to Chapter  6 for a discussion of the bias-variance trade-off in collaborative filtering.

Note that we use upper-case variable K to represent the size of the neighborhood that defines Q _j ( i). This is a deviation from section  2.​6.​2 of Chapter  2 We use lower-case variable k to represent the dimensionality of the factor matrices. The values of k and K are generally different.