Блог

Я пишу наивную программу завершения матрицы. Любопытно, что RMSE на обучающем наборе уменьшается во время обучения, но RMSE на тестовом наборе сначала резко возрастает, а затем медленно снижается. Я не знаю, нормально ли это.

Вот текущий журнал:

 Iter    0: cost=122720.2005, TRN_rmse=0.0991, TST_rmse=0.1787
Iter    1: cost=52211.5079, TRN_rmse=0.0640, TST_rmse=0.2769
Iter    2: cost=51622.2058, TRN_rmse=0.0636, TST_rmse=0.2906
Iter    3: cost=51548.6679, TRN_rmse=0.0636, TST_rmse=0.2929
Iter    4: cost=51479.9532, TRN_rmse=0.0635, TST_rmse=0.2935
Iter    5: cost=51401.8634, TRN_rmse=0.0635, TST_rmse=0.2936
Iter    6: cost=51308.3281, TRN_rmse=0.0634, TST_rmse=0.2938
Iter    7: cost=51190.9587, TRN_rmse=0.0634, TST_rmse=0.2939
Iter    8: cost=51037.6073, TRN_rmse=0.0633, TST_rmse=0.2940
Iter    9: cost=50831.0238, TRN_rmse=0.0631, TST_rmse=0.2942
Iter   10: cost=50547.6240, TRN_rmse=0.0630, TST_rmse=0.2944
Iter   11: cost=50156.6966, TRN_rmse=0.0627, TST_rmse=0.2947
Iter   12: cost=49620.2475, TRN_rmse=0.0623, TST_rmse=0.2950
Iter   13: cost=48897.6851, TRN_rmse=0.0618, TST_rmse=0.2954
Iter   14: cost=47972.2207, TRN_rmse=0.0612, TST_rmse=0.2957
Iter   15: cost=46877.6057, TRN_rmse=0.0604, TST_rmse=0.2961
Iter   16: cost=45671.0552, TRN_rmse=0.0595, TST_rmse=0.2963
Iter   17: cost=44423.6615, TRN_rmse=0.0585, TST_rmse=0.2964
Iter   18: cost=43188.0642, TRN_rmse=0.0576, TST_rmse=0.2961
Iter   19: cost=41988.5550, TRN_rmse=0.0567, TST_rmse=0.2955
Iter   20: cost=40845.6367, TRN_rmse=0.0558, TST_rmse=0.2946
Iter   30: cost=33144.0653, TRN_rmse=0.0493, TST_rmse=0.2847
Iter   40: cost=29772.7998, TRN_rmse=0.0460, TST_rmse=0.2789
Iter   50: cost=27966.7217, TRN_rmse=0.0442, TST_rmse=0.2754
Iter   60: cost=26816.7560, TRN_rmse=0.0429, TST_rmse=0.2730
Iter   70: cost=26037.9318, TRN_rmse=0.0420, TST_rmse=0.2714
Iter   80: cost=25465.6860, TRN_rmse=0.0413, TST_rmse=0.2703
Iter   90: cost=25026.3467, TRN_rmse=0.0408, TST_rmse=0.2695
Iter  100: cost=24683.8015, TRN_rmse=0.0404, TST_rmse=0.2688
 

И вот мой код:

 def matrix_completion(OMEGA, Y, m, n, rank, lambd, max_iter, eps=1e-8, test_mask=None, test_Y=None):
    """
    Standard matrix completion.
    :param OMEGA: 0-1 indicator matrix to mask the observed training entries
    :param Y: known entries
    :param m: number of rows of Y
    :param n: number of columns of Y
    :param rank: rank of U and V
    :param lambd: regularization parameter
    :param max_iter: max number of iterations
    :param eps: term added to the denominator to improve numerical stability (default: 1e-8)
    :param test_mask: mask of test matrix
    :param test_Y: test matrix
    :return: recovered matrix X
    """
    # initialize U and V
    U = np.random.rand(m, rank)
    V = np.random.rand(n, rank)

    # training cycle
    old_cost = 1e10     # cost of last cycle, initialize to a huge number
    for i in range(max_iter):
        # alternating least square amp; multiplicative update rule
        U = U * ((lambd * np.matmul(OMEGA * Y, V)) /
                 (lambd * np.matmul(OMEGA * np.matmul(U, V.T), V)   U   eps))
        V = V * ((lambd * np.matmul((OMEGA * Y).T, U)) /
                 (lambd * np.matmul((OMEGA * np.matmul(U, V.T)).T, U)   V   eps))
        # compute the cost
        cost = (lambd / 2) * np.sum(np.square(OMEGA * (np.matmul(U, V.T) - Y)))   
               0.5 * np.sum(np.square(U))   0.5 * np.sum(np.square(V))
        if i % 10 == 0 or i < 20:
            trn_rmse = np.sqrt(np.sum(np.square(OMEGA * (Y - np.matmul(U, V.T)))) / np.sum(OMEGA))
            if test_mask is not None and test_Y is not None:
                tst_rmse = np.sqrt(np.sum(np.square(test_mask * (test_Y - np.matmul(U, V.T)))) / np.sum(test_mask))
            else:
                tst_rmse = 0
            print("Iter M: cost=%.4lf, TRN_rmse=%.4lf, TST_rmse=%.4lf" % (i, cost, trn_rmse, tst_rmse))
        # stopping condition: improvement < 0.01%
        if abs(cost - old_cost) / old_cost < 1e-4:
            print("Early stopping.")
            break
        old_cost = cost
    # get recovered matrix X
    X = np.matmul(U, V.transpose())

    return X
 

Подробнее:

1. Моя целевая функция (пожалуйста, запустите фрагмент кода, чтобы просмотреть уравнение)

 <a href="https://www.codecogs.com/eqnedit.php?latex=inlineamp;space;min_{Uamp;space;geamp;space;0,amp;space;Vamp;space;geamp;space;0}amp;space;frac{lambda}{2}amp;space;Vertamp;space;Omegaamp;space;circamp;space;(Uamp;space;V^Tamp;space;-amp;space;Y)amp;space;Vert_F^2amp;space;amp;plus;amp;space;frac{1}{2}amp;space;Vertamp;space;Uamp;space;Vert_F^2amp;space;amp;plus;amp;space;frac{1}{2}amp;space;Vertamp;space;Vamp;space;Vert_F^2" target="_blank"><img src="https://latex.codecogs.com/gif.latex?inlineamp;space;min_{Uamp;space;geamp;space;0,amp;space;Vamp;space;geamp;space;0}amp;space;frac{lambda}{2}amp;space;Vertamp;space;Omegaamp;space;circamp;space;(Uamp;space;V^Tamp;space;-amp;space;Y)amp;space;Vert_F^2amp;space;amp;plus;amp;space;frac{1}{2}amp;space;Vertamp;space;Uamp;space;Vert_F^2amp;space;amp;plus;amp;space;frac{1}{2}amp;space;Vertamp;space;Vamp;space;Vert_F^2" title="min_{U ge 0, V ge 0} frac{lambda}{2} Vert Omega circ (U V^T - Y) Vert_F^2   frac{1}{2} Vert U Vert_F^2   frac{1}{2} Vert V Vert_F^2" /></a> 

2. Мои правила обновления следующие (пожалуйста, запустите фрагмент кода, чтобы просмотреть уравнение)::

 <a href="https://www.codecogs.com/eqnedit.php?latex=inlineamp;space;Uamp;space;leftarrowamp;space;Uamp;space;circamp;space;frac{lambdaamp;space;(Omegaamp;space;circamp;space;Y)amp;space;V}{lambdaamp;space;(Omegaamp;space;circamp;space;UV^T)amp;space;Vamp;space;amp;plus;amp;space;U}amp;space;\amp;space;Vamp;space;leftarrowamp;space;Vamp;space;circamp;space;frac{lambdaamp;space;(Omegaamp;space;circamp;space;Y)^Tamp;space;U}{lambdaamp;space;(Omegaamp;space;circamp;space;UV^T)^Tamp;space;Uamp;space;amp;plus;amp;space;V}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?inlineamp;space;Uamp;space;leftarrowamp;space;Uamp;space;circamp;space;frac{lambdaamp;space;(Omegaamp;space;circamp;space;Y)amp;space;V}{lambdaamp;space;(Omegaamp;space;circamp;space;UV^T)amp;space;Vamp;space;amp;plus;amp;space;U}amp;space;\amp;space;Vamp;space;leftarrowamp;space;Vamp;space;circamp;space;frac{lambdaamp;space;(Omegaamp;space;circamp;space;Y)^Tamp;space;U}{lambdaamp;space;(Omegaamp;space;circamp;space;UV^T)^Tamp;space;Uamp;space;amp;plus;amp;space;V}" title="U leftarrow U circ frac{lambda (Omega circ Y) V}{lambda (Omega circ UV^T) V   U} \ V leftarrow V circ frac{lambda (Omega circ Y)^T U}{lambda (Omega circ UV^T)^T U   V}" /></a> 

3. гиперпараметры: ранг = 100, лямбд = 1. На самом деле, для других рангов и лямбд тенденция та же.