Hello everyone! My name is Dmitry, I am a computer vision developer in the MTS AI team . It so happened historically that in my work, as a rule, I use a bunch of outdated versions of TensorFlow 1 and Keras. It's time to move on, but before moving completely to TensorFlow 2, I decided to compare the most popular deep learning frameworks today: TensorFlow and PyTorch . This article will be useful to all Data Scientists who want to learn a little more about the features and performance of TensorFlow and PyTorch.
TensorFlow and Keras
TensorFlow Google 2015 . Google . : , ( ), . TensorFlow .
TensorFlow 2.x 2019, . :
import tensorflow as tf
tf.compat.v1.disable_v2_behavior()
Keras , TensorFlow, Theano CNTK. Keras Google, . Keras TensorFlow, 1.4.0, TensorFlow 2. : tf.keras
. Keras , Theano CNTK .
TensorFlow API: TensorFlow Lite IoT; TensorFlow.js JavaScript ; TensorFlow Extended , REST API, .
PyTorch
PyTorch Facebook 2016 . PyTorch Torch – Matlab- Lua, 2017 . PyTorch .
PyTorch : numpy arrays, .
. , , , ( tf.Graph()
TensorFlow 1.x). , :
# TensorFlow 1.15.0
import numpy as np
import tensorflow as tf
#
inputs = tf.placeholder(tf.float32, [1, 224, 224, 3])
#
x = tf.layers.conv2d(inputs, 64, (3, 3), strides=2, padding='same', use_bias=False)
x = tf.layers.batch_normalization(x)
output_tensor = tf.nn.relu(x)
# tf.Session()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
outputs = sess.run([output_tensor], feed_dict={inputs: np.random.randint(0, 256, (1, 224, 224, 3)).astype(np.float32)})
:
# TensorFlow 2.5.0
import numpy as np
import tensorflow as tf
#
inputs = np.random.randint(0, 256, (1, 224, 224, 3)).astype(np.float32)
#
x = tf.keras.layers.Conv2D(64, (3, 3), strides=2, padding='same', use_bias=False)(inputs)
x = tf.keras.layers.BatchNorimalization()(x)
outputs = tf.keras.layers.Relu()(x)
, :
;
;
, .
TensorFlow PyTorch
TensorFlow PyTorch , . , TensorFlow 1. TensorFlow 2.., 2017 , .
, , ( GPU), ( GPU CPU), .
:
forward propagation – , backward propagation – . forward propagation, .
, , . , : ( ) .
:
GPU: Nvidia GeForce RTX 2060;
CPU: Intel(R) Core(TM) i7-9750H.
/, . , :
import numpy as np
image_data = np.random.randint(0, 256, (steps_number, batch_size, height, width, channels))
labels_data = np.random.randint(0, 2, (steps_number, batch_size, 1000))
1. ( GPU)
, 100 . 1. , . 1 , ( ).
1. ( ).
PyTorch TensorFlow. : PyTorch 1.9.0 30% , TensorFlow 2.5.0, 45% , TensorFlow 1.15.0. , vgg16 efficientnet TensorFlow 2.5.0 , PyTorch 1.9.0. TensorFlow 2.5.0 TensorFlow 1.15.0, . PyTorch 1.9.0 , TensorFlow 2.5.0.
2. ( GPU)
, 2, , , 2.
2. GPU ( ).
GPU TensorFlow 1.15.0 PyTorch 1.9.0 : , PyTorch 1.9.0 5% ResNet VGG, TensorFlow 1.15.0 5% MobileNet EfficientNet. TensorFlow 2.5.0 30% . , , : , TensorFlow 2.5.0 , TensorFlow 1.15.0.
3. ( CPU)
3 CPU. 3 , CPU.
3. CPU ( ).
: PyTorch 1.9.0 , , . CPU: TensorFlow 2.5.0 5% , TensorFlow 1.15.0 40% , PyTorch 1.9.0.
, CPU TensorFlow 2.5.0 .
4.
python- nvidia-smi. , / . , TensorFlow : , . , , , TensorFlow . , ResNet18 (128, 128, 3) 1 4679 , ResNet18 (320, 320, 3) 1 3911 . PyTorch : 1329 1355 .
4 4 PyTorch , , .
4. PyTorch 1.9.0 ( ).
, :
, TensorFlow 1 , TensorFlow 2 GPU;
tf.keras
GPU TensorFlow 2 , TensorFlow, ó , , TensorFlow 2 ;
PyTorch GPU, CPU, , , onnxruntime. , onnxruntime CPU.
, :