I am trying to run a Keras sequential model but can’t get the right shape for the model to train on.
I reshaped x and y to:
JavaScript
x
3
1
x = x.reshape(len(x), 500)2
y = y.reshape(len(y), 500)3
Currently, both the input shape and output shape are:
JavaScript
1
3
1
(9766, 500)2
(9766, 500)3
The dataset consists of 9766 inputs and 9766 outputs respectively. Each input is a single array of 500 values and each output is also an array of 500 values.
So here is one single input array:
JavaScript
1
85
85
1
[0.99479668 0.99477965 0.99484778 0.99489887 0.99483926 0.994515652
0.99458378 0.99457526 0.99453268 0.99468597 0.99466042 0.994498623
0.99453268 0.99454971 0.99463487 0.99461784 0.99451565 0.994634874
0.99467745 0.99502661 0.99480519 0.99493294 0.99493294 0.995222485
0.99526506 0.99528209 0.99527358 0.99515435 0.99529913 0.994881846
0.99508623 0.99512881 0.99522248 0.99497552 0.9954439 0.995546097
0.99581861 0.99573345 0.9957079 0.99626144 0.99626144 0.995929328
0.99558867 0.99541835 0.99524803 0.99586119 0.99601448 0.995886749
0.99584416 0.99559719 0.995495 0.99520545 0.99552055 0.9951032610
0.9951799 0.99560571 0.99561422 0.99541835 0.99586119 0.99575911
0.9957079 0.99583564 0.9959208 0.99578454 0.99604854 0.9961251912
0.99609112 0.99630402 0.9961337 0.99672983 0.99655099 0.9964317613
0.99643176 0.99648286 0.99649138 0.99645731 0.99670428 0.9965424714
0.99647435 0.99607409 0.99589525 0.99600596 0.99596338 0.9962103515
0.99633809 0.99632106 0.99583564 0.99581009 0.99574196 0.995971916
0.99557164 0.99567383 0.99572493 0.9958697 0.99568235 0.995920817
0.99598893 0.99620183 0.99611667 0.99620183 0.9959719 0.995707918
0.99612519 0.99558867 0.99569938 0.99518842 0.99553758 0.9955205519
0.99576751 0.99577603 0.99583564 0.99602299 0.99630402 0.9963721520
0.99701937 0.99701086 0.99731744 0.99700234 0.99696828 0.9966872521
0.99703641 0.99725782 0.99684054 0.99605706 0.99608261 0.9958186122
0.9958697 0.99583564 0.99566532 0.99585267 0.99566532 0.9960400323
0.99540984 0.99473707 0.995231 0.99441346 0.9942261 0.9939791424
0.99367256 0.99409836 0.99415797 0.99420907 0.99398765 0.9935618525
0.99382585 0.99428571 0.9945412 0.99444752 0.99436236 0.9940472626
0.9938003 0.99424313 0.99483074 0.99474558 0.99457526 0.9945752627
0.99465191 0.99466042 0.99467745 0.99448158 0.99454971 0.9947966828
0.994703 0.99455823 0.99472855 0.99507771 0.99529913 0.9951543529
0.99525655 0.99621886 0.99586119 0.99576751 0.9962359 0.9961422230
0.99723228 0.99685757 0.99680647 0.99689163 0.99644028 0.9970193731
0.99675538 0.99637215 0.99614222 0.99628699 0.9964488 0.9964147332
0.99652544 0.99652544 0.99664467 0.99698531 0.99712157 0.9970364133
0.99799872 0.99859485 0.99876517 0.99950607 0.99902065 0.9989184634
0.99804982 0.99839898 0.99857782 0.99850117 0.99891846 0.9991228435
0.99919097 0.99919949 0.99896956 0.99896104 0.99877369 0.9989865936
0.99918246 0.99890994 0.9990462 0.99895252 0.99885033 0.9987140737
0.99871407 0.99871407 0.99864594 0.99854375 0.9983564 0.998569338
0.99870556 0.99868001 0.9987822 0.99877369 0.99900362 0.9988247839
0.99896956 0.99885885 0.99880775 0.99890994 0.99906323 0.9990802640
0.9990462 0.99921652 0.99920801 0.99936129 0.99937833 0.9994379441
0.99935278 0.99943794 0.99967639 0.99956568 0.99960826 0.9996252942
0.99942942 0.99940387 0.9992591 0.99908878 0.99912284 0.9991398843
0.99905472 0.99914839 0.99913136 0.99933575 0.99935278 0.9992931744
0.99931871 0.99905472 0.99965084 0.99995742 1. 0.9996252945
0.999472 0.99939536 0.99932723 0.99929317 0.99931871 0.9993187146
0.99950607 0.99953162 0.99942942 0.99919097 0.99902917 0.9991398847
0.99915691 0.9990462 0.9990973 0.99923355 0.99940387 0.9995486548
0.99958271 0.99940387 0.99943794 0.99928465 0.9990973 0.9990547249
0.99915691 0.99921652 0.99913988 0.99913136 0.99912284 0.999259150
0.99916542 0.99917394 0.99918246 0.99906323 0.99905472 0.9990717551
0.99901214 0.9990462 0.99913988 0.9990462 0.9990462 0.9988077552
0.99890994 0.99868852 0.99868852 0.99889291 0.99896956 0.9988673653
0.99932723 0.99943794 0.99932723 0.99931871 0.99931871 0.9992165254
0.99874814 0.99871407 0.99915691 0.99969342 0.99962529 0.9991654255
0.99902917 0.99887588 0.99919097 0.99943794 0.99847562 0.998833356
0.99905472 0.99913988 0.99931871 0.99936129 0.99893549 0.9986970457
0.99842453 0.99868001 0.99868852 0.9987822 0.9987311 0.9987140758
0.99860336 0.99826272 0.99805834 0.99785395 0.99792208 0.9980498259
0.99797317 0.99797317 0.99778582 0.99749627 0.99751331 0.9975814360
0.99732595 0.99741111 0.99699383 0.99733447 0.99728337 0.9968660861
0.99714712 0.9973515 0.99753885 0.99753034 0.99762402 0.9977432462
0.99781989 0.99765808 0.99739408 0.9974026 0.99723228 0.9973770563
0.99728337 0.99728337 0.99736002 0.99726634 0.99732595 0.9972152464
0.99728337 0.99701937 0.99715563 0.99715563 0.99744518 0.9975303465
0.99747073 0.99765808 0.9978284 0.99726634 0.99724931 0.9977687966
0.99746221 0.9976666 0.9976666 0.99744518 0.99734298 0.9983308567
0.99866298 0.99800724 0.99714712 0.99648286 0.99588674 0.9959804168
0.99563125 0.99595486 0.99626144 0.99601448 0.99456674 0.994754169
0.99499255 0.99483926 0.9950181 0.99497552 0.99484778 0.9942431370
0.99416649 0.99416649 0.9942772 0.99288908 0.99266766 0.9929316671
0.99248031 0.99312753 0.99269321 0.99307643 0.99286353 0.9931956672
0.99346817 0.99337449 0.99322972 0.99302534 0.99322121 0.9930764373
0.99295721 0.99344262 0.99262508 0.99259953 0.99246327 0.9925484474
0.99265063 0.99288908 0.99288908 0.9930594 0.9933234 0.9934000475
0.99320417 0.99331488 0.99319566 0.99335746 0.99322121 0.9927187676
0.99271024 0.99270172 0.99259102 0.99308495 0.99331488 0.993008377
0.99285501 0.99289759 0.99276134 0.99259102 0.99266766 0.9922163178
0.99216521 0.99225889 0.99227592 0.99196934 0.99162018 0.9914754179
0.99134767 0.99159463 0.99152651 0.99166276 0.99169683 0.9916883180
0.99175644 0.99178199 0.99161167 0.99165425 0.99170534 0.991577681
0.9915776 0.99144135 0.99169683 0.99170534 0.99144986 0.9917053482
0.99187567 0.99192676 0.99183308 0.99177347 0.99173941 0.9917649683
0.99170534 0.9917905 0.99178199 0.99144986 0.99147541 0.9914243184
0.99149244 0.99139877]85
And here is one output array:
JavaScript
1
85
85
1
[0. 0. 0. 0. 0. 0.2
0. 0. 0. 0. 0. 0.994498623
0. 0. 0. 0. 0. 0.4
0. 0. 0. 0. 0. 0.5
0. 0. 0. 0. 0. 0.6
0. 0. 0. 0. 0. 0.7
0. 0. 0. 0. 0. 0.8
0. 0. 0. 0. 0. 0.9
0. 0. 0. 0. 0. 0.10
0. 0. 0. 0. 0. 0.11
0. 0. 0. 0. 0. 0.12
0. 0. 0. 0. 0. 0.13
0. 0. 0. 0. 0. 0.14
0. 0. 0. 0. 0. 0.15
0. 0. 0. 0. 0. 0.16
0. 0. 0. 0. 0. 0.17
0. 0. 0. 0. 0. 0.18
0. 0. 0. 0. 0. 0.19
0. 0. 0. 0. 0. 0.20
0. 0. 0.99731744 0. 0. 0.21
0. 0. 0. 0. 0. 0.22
0. 0. 0. 0. 0. 0.23
0. 0. 0. 0. 0. 0.24
0. 0. 0. 0. 0. 0.9935618525
0. 0. 0. 0. 0. 0.26
0. 0. 0. 0. 0. 0.27
0. 0. 0. 0. 0. 0.28
0. 0. 0. 0. 0. 0.29
0. 0. 0. 0. 0. 0.30
0. 0. 0. 0. 0. 0.31
0. 0. 0. 0. 0. 0.32
0. 0. 0. 0. 0. 0.33
0. 0. 0. 0. 0. 0.34
0. 0. 0. 0. 0. 0.35
0. 0. 0. 0. 0. 0.36
0. 0. 0. 0. 0. 0.37
0. 0. 0. 0. 0. 0.38
0. 0. 0. 0. 0. 0.39
0. 0. 0. 0. 0. 0.40
0. 0. 0. 0. 0. 0.41
0. 0. 0. 0. 0. 0.42
0. 0. 0. 0. 0. 0.43
0. 0. 0. 0. 0. 0.44
0. 0. 0. 0. 1. 0.45
0. 0. 0. 0. 0. 0.46
0. 0. 0. 0. 0. 0.47
0. 0. 0. 0. 0. 0.48
0. 0. 0. 0. 0. 0.49
0. 0. 0. 0. 0. 0.50
0. 0. 0. 0. 0. 0.51
0. 0. 0. 0. 0. 0.52
0. 0. 0. 0. 0. 0.53
0. 0. 0. 0. 0. 0.54
0. 0. 0. 0. 0. 0.55
0. 0. 0. 0. 0. 0.56
0. 0. 0. 0. 0. 0.57
0. 0. 0. 0. 0. 0.58
0. 0. 0. 0. 0. 0.59
0. 0. 0. 0. 0. 0.60
0. 0. 0. 0. 0. 0.9968660861
0. 0. 0. 0. 0. 0.62
0. 0. 0. 0. 0. 0.63
0. 0. 0. 0. 0. 0.64
0. 0. 0. 0. 0. 0.65
0. 0. 0. 0. 0. 0.66
0. 0. 0. 0. 0. 0.67
0.99866298 0. 0. 0. 0. 0.68
0. 0. 0. 0. 0. 0.69
0. 0. 0. 0. 0. 0.70
0. 0. 0. 0. 0. 0.71
0. 0. 0. 0. 0. 0.72
0. 0. 0. 0. 0. 0.73
0. 0. 0. 0. 0. 0.74
0. 0. 0. 0. 0. 0.75
0. 0. 0. 0. 0. 0.76
0. 0. 0. 0. 0. 0.77
0. 0. 0. 0. 0. 0.78
0. 0. 0. 0. 0. 0.79
0.99134767 0. 0. 0. 0. 0.80
0. 0. 0. 0. 0. 0.81
0. 0. 0. 0. 0. 0.82
0. 0. 0. 0. 0. 0.83
0. 0. 0. 0. 0. 0.84
0. 0. ]85
And this is the model I am trying to train the data on (most likely with a bad architecture):
JavaScript
1
15
15
1
model = Sequential()2
model.add(LSTM(128, input_shape=(x.shape[1:]), return_sequences=True))3
model.add(Dropout(0.2))4
5
model.add(LSTM(128, input_shape=(x.shape[1:])))6
model.add(Dropout(0.1))7
8
model.add(LSTM(32, input_shape=(x.shape[1:]) ,activation = 'relu'))9
model.add(Dropout(0.2))10
11
model.add (Dense(1 ,activation = 'sigmoid'))12
opt = tf.keras.optimizers.Adam(lr=0.001, decay=1e-3)13
model.compile(loss='mse',optimizer=opt, metrics=['accuracy'])14
model.fit(x,y,epochs=20,validation_split=0.20)15
How I would like the model to train is to see the input and produce an array of 500 values like the output array shown above. But no matter what shape I try, I get an error like the following:
JavaScript
1
2
1
ValueError: Input 0 of layer "lstm" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 500)2
What shape is the proper shape here and what am I doing wrong with the model architecture?
UPDATE 1:
I also tried reshaping x and y to:
JavaScript
1
3
1
(9766, 1, 500)2
(9766, 1, 500)3
still no luck.
Advertisement
Answer
LSTM layer expects input shape as [batch, timesteps, feature]. So, with the shape (9766, 1, 500), you have one timestep with 500 features. If you have 500 timesteps, your shape should be like (9766, 500, 1).
Here is an example architecture:
JavaScript
1
19
19
1
x = tf.random.uniform((9766,500,1))2
y = tf.random.uniform((9766,500,1))3
4
5
model = tf.keras.models.Sequential()6
model.add(tf.keras.layers.InputLayer(input_shape=(x.shape[1:])))7
model.add(tf.keras.layers.LSTM(128, return_sequences=True, activation='relu'))8
model.add(tf.keras.layers.Dropout(0.2))9
10
model.add(tf.keras.layers.LSTM(128, activation='relu', return_sequences=True))11
model.add(tf.keras.layers.Dropout(0.1))12
13
model.add(tf.keras.layers.LSTM(32, activation = 'relu', return_sequences=True))14
model.add(tf.keras.layers.Dropout(0.2))15
16
model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(1 ,activation = 'sigmoid'))) # You can also remove timedistributed wrapper if you get better result. I supposed you need to have your output values between 0.0 and 1.017
model.compile(loss='mse',optimizer=tf.keras.optimizers.Adam(lr=0.001, decay=1e-3), metrics=['accuracy']) # Be careful about your chosen metric. 18
model.summary()19
If you check the model summary, you see input and output shape are the same as you expected:
JavaScript
1
24
24
1
_________________________________________________________________2
Layer (type) Output Shape Param # 3
=================================================================4
lstm_15 (LSTM) (None, 500, 128) 66560 5
6
dropout_13 (Dropout) (None, 500, 128) 0 7
8
lstm_16 (LSTM) (None, 500, 128) 131584 9
10
dropout_14 (Dropout) (None, 500, 128) 0 11
12
lstm_17 (LSTM) (None, 500, 32) 20608 13
14
dropout_15 (Dropout) (None, 500, 32) 0 15
16
time_distributed_1 (TimeDis (None, 500, 1) 33 17
tributed) 18
19
=================================================================20
Total params: 218,78521
Trainable params: 218,78522
Non-trainable params: 023
_________________________________________________________________24