Блог

Я запускаю, как мне кажется, довольно маленький CNN на nVidia Jetson Nano с Jetpack 4.4. nVidia утверждает, что Nano может запускать ResNet-50 со скоростью 36 кадров в секунду, поэтому я ожидал, что моя гораздо меньшая сеть будет работать со скоростью 30 fps с легкостью.

На самом деле, каждый прямой проход занимает 160-180 мс, поэтому я набираю в лучшем случае 5-6 кадров в секунду.

Мой CNN:

 Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
lambda (Lambda)              (None, 210, 848, 3)       0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 210, 282, 3)       0
_________________________________________________________________
conv2d (Conv2D)              (None, 102, 138, 16)      2368
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 51, 69, 16)        0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 24, 33, 32)        12832
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 12, 16, 32)        0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 4, 6, 64)          51264
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 2, 3, 64)          0
_________________________________________________________________
flatten (Flatten)            (None, 384)               0
_________________________________________________________________
dropout (Dropout)            (None, 384)               0
_________________________________________________________________
dense (Dense)                (None, 64)                24640
_________________________________________________________________
dropout_1 (Dropout)          (None, 64)                0
_________________________________________________________________
elu (ELU)                    (None, 64)                0
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 65
=================================================================
Total params: 91,169
Trainable params: 91,169
Non-trainable params: 0
_________________________________________________________________
 

Код:

 import numpy as np
import cv2
import time
import tensorflow as tf
from tensorflow import keras

model_name = 'v9_small_FC_epoch_3'
loaded_model = keras.models.load_model('/home/jetson/notebooks/trained_models/'   model_name   '.h5')
loaded_model.summary()
frame = cv2.imread('/home/jetson/notebooks/frame1.jpg')    
test_data = np.expand_dims(frame, axis=0)

for i in range(10):
    start = time.time()
    predictions = loaded_model.predict(test_data)
    print(predictions[0][0])
    end = time.time()
    print("Inference took {}s".format(end-start))
 

Результат:

 4.7763316333293915
Inference took 10.111131191253662s
4.7763316333293915
Inference took 0.1822071075439453s
4.7763316333293915
Inference took 0.17330455780029297s
4.7763316333293915
Inference took 0.18085694313049316s
4.7763316333293915
Inference took 0.16646790504455566s
4.7763316333293915
Inference took 0.1703803539276123s
4.7763316333293915
Inference took 0.1788337230682373s
4.7763316333293915
Inference took 0.17131853103637695s
4.7763316333293915
Inference took 0.1660606861114502s
4.7763316333293915
Inference took 0.18377089500427246s