I am trying to use the VGG16 network for multiple input images.
Training this model using a simple CNN with 2 inputs gave me an acc. of about 50 %, which is why I wanted to try it using an established model like VGG16.
Here is what I have tried out:
JavaScript
x
37
37
1
# imports
2
from keras.applications.vgg16 import VGG16
3
from keras.models import Model
4
from keras.layers import Conv2D, MaxPooling2D, Activation, Dropout, Flatten, Dense
5
6
def def_model():
7
model = VGG16(include_top=False, input_shape=(224, 224, 3))
8
# mark loaded layers as not trainable
9
for layer in model.layers:
10
layer.trainable = False
11
# return last pooling layer
12
pool_layer = model.layers[-1].output
13
return pool_layer
14
15
m1 = def_model()
16
m2 = def_model()
17
m3 = def_model()
18
19
# add classifier layers
20
merge = concatenate([m1, m2, m3])
21
22
# optinal_conv = Conv2D(64, (3, 3), activation='relu', padding='same')(merge)
23
# optinal_pool = MaxPooling2D(pool_size=(2, 2))(optinal_conv)
24
# flatten = Flatten()(optinal_pool)
25
26
flatten = Flatten()(merge)
27
dense1 = Dense(512, activation='relu')(flatten)
28
dense2 = Dropout(0.5)(dense1)
29
output = Dense(1, activation='sigmoid')(dense2)
30
31
32
inshape1 = Input(shape=(224, 224, 3))
33
inshape2 = Input(shape=(224, 224, 3))
34
inshape3 = Input(shape=(224, 224, 3))
35
model = Model(inputs=[inshape1, inshape2, inshape3], outputs=output)
36
37
- I get this error while calling the
Model
function.
JavaScript
1
2
1
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_21:0", shape=(?, 224, 224, 3), dtype=float32) at layer "input_21". The following previous layers were accessed without issue: []`
2
I understand that the graph is a disconnect, but I could not find out where.
Here are the compile
and fit
functions.
JavaScript
1
5
1
# compile model
2
model.compile(optimizer="Adam", loss='binary_crossentropy', metrics=['accuracy'])
3
model.fit([train1, train2, train3], train,
4
validation_data=([test1, test2, test3], ytest))
5
- I have commented on some lines:
optinal_conv
andoptinal_pool
. What could be the effect to applyConv2D
andMaxPooling2D
after theconcatenate
function?
Advertisement
Answer
I recommend seeing this answer Multi-input Multi-output Model with Keras Functional API. Here is one way you can achieve this:
JavaScript
1
22
22
1
# 3 inputs
2
input0 = tf.keras.Input(shape=(224, 224, 3), name="img0")
3
input1 = tf.keras.Input(shape=(224, 224, 3), name="img1")
4
input2 = tf.keras.Input(shape=(224, 224, 3), name="img2")
5
concate_input = tf.keras.layers.Concatenate()([input0, input1, input2])
6
# get 3 feature maps with same size (224, 224)
7
# pretrained models needs that
8
input = tf.keras.layers.Conv2D(3, (3, 3),
9
padding='same', activation="relu")(concate_input)
10
11
# pass that to imagenet model
12
vg = tf.keras.applications.VGG16(weights=None,
13
include_top = False,
14
input_tensor = input)
15
16
# do whatever
17
gap = tf.keras.layers.GlobalAveragePooling2D()(vg.output)
18
den = tf.keras.layers.Dense(1, activation='sigmoid')(gap)
19
20
# build the complete model
21
model = tf.keras.Model(inputs=[input0, input1, input2], outputs=den)
22