Why not all the layer weights equal
:
import tensorflow as tf import numpy as np model = tf.keras.Sequential([ tf.keras.layers.Conv2D(4, kernel_size=3, padding="same", activation="selu", input_shape=(28,28,1) ), tf.keras.layers.MaxPool2D(pool_size=2), tf.keras.layers.Conv2D(8, kernel_size=3, padding="same"), tf.keras.layers.Conv2D(16, kernel_size=3, padding="same"), tf.keras.layers.MaxPool2D(pool_size=2), tf.keras.layers.Flatten(), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(10) ]) model.summary() a = [] b = [] for layer in model.layers: weights = layer.get_weights() a.append(weights) b.append(weights) for index, i in enumerate(a): a_weights = a[index] b_weights = b[index] print("index [ {} ]: a_weights == b_weights: {}".format( index, np.array_equal(a_weights, b_weights) ))
Here is the output:
Model: "sequential_10" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_15 (Conv2D) (None, 28, 28, 4) 40 max_pooling2d_15 (MaxPoolin (None, 14, 14, 4) 0 g2D) conv2d_16 (Conv2D) (None, 14, 14, 8) 296 conv2d_17 (Conv2D) (None, 14, 14, 16) 1168 max_pooling2d_16 (MaxPoolin (None, 7, 7, 16) 0 g2D) flatten_10 (Flatten) (None, 784) 0 dense_20 (Dense) (None, 128) 100480 dense_21 (Dense) (None, 10) 1290 ================================================================= Total params: 103,274 Trainable params: 103,274 Non-trainable params: 0 _________________________________________________________________ index [ 0 ]: a_weights == b_weights: False index [ 1 ]: a_weights == b_weights: True index [ 2 ]: a_weights == b_weights: False index [ 3 ]: a_weights == b_weights: False index [ 4 ]: a_weights == b_weights: True index [ 5 ]: a_weights == b_weights: True index [ 6 ]: a_weights == b_weights: False index [ 7 ]: a_weights == b_weights: False
The a_weights == b_weights
are not all the “True
“.
What’s the problem?
Advertisement
Answer
Notice that the only time a_weights == b_weights
is True
, is when you are referencing a layer, which does not have any weights. np.array_equal
is returning False
because you are actually comparing lists of arrays and not the arrays themselves. Each trainable layer of yours has a kernel weight tensor and bias tensor. So actually your code should look like this:
import tensorflow as tf import numpy as np model = tf.keras.Sequential([ tf.keras.layers.Conv2D(4, kernel_size=3, padding="same", activation="selu", input_shape=(28,28,1) ), tf.keras.layers.MaxPool2D(pool_size=2), tf.keras.layers.Conv2D(8, kernel_size=3, padding="same"), tf.keras.layers.Conv2D(16, kernel_size=3, padding="same"), tf.keras.layers.MaxPool2D(pool_size=2), tf.keras.layers.Flatten(), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(10) ]) model.summary() a = [] b = [] for layer in model.layers: weights = layer.get_weights() a.append(weights) b.append(weights) for index, i in enumerate(a): a_weights = a[index] b_weights = b[index] if a_weights: print("index [ {} ]: kernel: a_weights == b_weights: {}".format( index, np.array_equal(a_weights[0], b_weights[0]) )) print("index [ {} ]: bias: a_weights == b_weights: {}".format( index, np.array_equal(a_weights[1], b_weights[1]) )) else: print("index [ {} ]: no layer weights: a_weights == b_weights: {}".format( index, np.array_equal(a_weights, b_weights) ))
... index [ 0 ]: kernel: a_weights == b_weights: True index [ 0 ]: bias: a_weights == b_weights: True index [ 1 ]: no layer weights: a_weights == b_weights: True index [ 2 ]: kernel: a_weights == b_weights: True index [ 2 ]: bias: a_weights == b_weights: True index [ 3 ]: kernel: a_weights == b_weights: True index [ 3 ]: bias: a_weights == b_weights: True index [ 4 ]: no layer weights: a_weights == b_weights: True index [ 5 ]: no layer weights: a_weights == b_weights: True index [ 6 ]: kernel: a_weights == b_weights: True index [ 6 ]: bias: a_weights == b_weights: True index [ 7 ]: kernel: a_weights == b_weights: True index [ 7 ]: bias: a_weights == b_weights: True
You could also use a_weights == b_weights
directly.