Блог

Я безуспешно пытаюсь научить свою нейронную сеть играть в довольно простую игру. Проблема в том, что функция differential_evolution() из scipy работает недостаточно долго: я установил maxiter=1000 , но функция работает только для 41 итерации. вот код:

 def fitness_func(x, *args):
    #print('fitness func started')
    arch, width, height = args
    net = genome_to_nn(x, arch)
    my_game = Game_2(height, width)
    count = 0
    move = -1
    while count < 100:
        count  = 1
        field = my_game.get_np_field()
        decision_tensor = net(field)
        move = int(tf.math.argmax( decision_tensor , axis =1))
        if move != 2:           
            my_game.make_a_move(move)       
        if count % 2:
            my_game.make_random()        
        my_game.next_iter()
    
    result = 1/(150   my_game.score)
    return result


if __name__ == '__main__':
    field_width = 5
    field_height = 10
    inp_size = field_width*(field_height-1)   1    
    
    model = keras.Sequential(
        [
            layers.Dense(10, input_dim = inp_size),
            layers.Dense(10, input_dim = 10 ,activation='sigmoid'),
            layers.Dense(3, input_dim =10, activation='softmax')
        ]
    )

    args = (model, field_width, field_height)
    bounds = np.asarray([(-10,10) for i in range(len(nn_to_genome(model)))])

    print('start evolution')
    res = differential_evolution(fitness_func, bounds= bounds, args=args, maxiter=50, workers=70, disp=True) 

    print('DE finished')
    
    fitted_model = genome_to_nn(res.x, model)
    
    print(res)
 

И тогда я получаю следующий результат:

 start evolution
/opt/anaconda3/envs/myenv2/lib/python3.7/site-packages/scipy/optimize/_differentialevolution.py:494: UserWarning: differential_evolution: the 'workers' keyword has overridden updating='immediate' to updating='deferred'
  " updating='deferred'", UserWarning)
differential_evolution step 1: f(x)= 0.00606061
differential_evolution step 2: f(x)= 0.00606061
differential_evolution step 3: f(x)= 0.00606061
differential_evolution step 4: f(x)= 0.00606061
differential_evolution step 5: f(x)= 0.00606061
differential_evolution step 6: f(x)= 0.00606061
differential_evolution step 7: f(x)= 0.00606061
differential_evolution step 8: f(x)= 0.00606061
differential_evolution step 9: f(x)= 0.00606061
differential_evolution step 10: f(x)= 0.00606061
differential_evolution step 11: f(x)= 0.00606061
differential_evolution step 12: f(x)= 0.00606061
differential_evolution step 13: f(x)= 0.00606061
differential_evolution step 14: f(x)= 0.00606061
differential_evolution step 15: f(x)= 0.00606061
differential_evolution step 16: f(x)= 0.00606061
differential_evolution step 17: f(x)= 0.00606061
differential_evolution step 18: f(x)= 0.00606061
differential_evolution step 19: f(x)= 0.00606061
differential_evolution step 20: f(x)= 0.00606061
differential_evolution step 21: f(x)= 0.00606061
differential_evolution step 22: f(x)= 0.00606061
differential_evolution step 23: f(x)= 0.00606061
differential_evolution step 24: f(x)= 0.00606061
differential_evolution step 25: f(x)= 0.00606061
differential_evolution step 26: f(x)= 0.00606061
differential_evolution step 27: f(x)= 0.00606061
differential_evolution step 28: f(x)= 0.00606061
differential_evolution step 29: f(x)= 0.00606061
differential_evolution step 30: f(x)= 0.00606061
differential_evolution step 31: f(x)= 0.00606061
differential_evolution step 32: f(x)= 0.00606061
differential_evolution step 33: f(x)= 0.00606061
differential_evolution step 34: f(x)= 0.00606061
differential_evolution step 35: f(x)= 0.00606061
differential_evolution step 36: f(x)= 0.00606061
differential_evolution step 37: f(x)= 0.00606061
differential_evolution step 38: f(x)= 0.00606061
differential_evolution step 39: f(x)= 0.00606061
differential_evolution step 40: f(x)= 0.00606061
differential_evolution step 41: f(x)= 0.00606061
DE finished
     fun: 0.006060606060606061
 message: 'Optimization terminated successfully.'
    nfev: 399735
     nit: 41
 success: True
       x: array([-6.59142662, -6.6655827 ,  7.01109519, -6.61588426, -8.99447424,
         ...
         just a lot of unnecessary numbers here
         ...
       -4.19133698,  1.62013289,  5.72924953, -0.29303238, -2.17649926,
        1.91011116,  9.8819633 , -9.58588766,  6.05450803])
WARNING:tensorflow:From /opt/anaconda3/envs/myenv2/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:tensorflow:Assets written to: model_trained/assets
 

P.S. Не имеет значения, равен ли maxiter 50 из 5000 (больше 41), функция по-прежнему выполняется для 41 итерации

Если я установлю maxiter=30 , я получу следующее:

 message: 'Maximum number of iterations has been exceeded.'
    nfev: 298425
     nit: 30
 success: False
 

UPD: я изменил возвращаемое значение функции пригодности на

 result = - my_game.score
 

Итак, теперь возвращаемое значение находится в диапазоне [-100, 100] (раньше было в [1/250; 1/50]), и это работает! Но я до сих пор понятия не имею, почему он не работает со старой версией function . В официальной документации ничего не говорится о каких-либо ограничениях на возвращаемое значение (за исключением того, что это должно быть число)