I’m working in a project that isolate vocal parts from an audio. I’m using the DSD100 dataset, but for doing tests I’m using the DSD100subset dataset from I only use the mixtures and the vocals. I’m basing this work on this article
First I process the audios to extract a spectrogram and put it on a list, with all the audios forming four lists (trainMixed, trainVocals, testMixed, testVocals). Like this:
def to_spec(wav, n_fft=1024, hop_length=256): return librosa.stft(wav, n_fft=n_fft, hop_length=hop_length) def prepareData(filename, sr=22050, hop_length=256, n_fft=1024): audio_wav = librosa.load(filename, sr=sr, mono=True, duration=30)[0] audio_spec=to_spec(audio_wav, n_fft=n_fft, hop_length=hop_length) audio_spec_mag = np.abs(audio_spec) maxVal = np.max(audio_spec_mag) return audio_spec_mag, maxVal # FOR EVERY LIST (trainMixed, trainVocals, testMixed, testVocals) trainMixed = [] trainMixedNum = 0 for (root, dirs, files) in walk('./Dev-subset-mix/Dev/'): for d in dirs: filenameMix = './Dev-subset-mix/Dev/'+d+'/mixture.wav' spec_mag, maxVal = prepareData(filenameMix, n_fft=1024, hop_length=256) trainMixed.append(spec_mag/maxVal)
Next i build the model:
import keras from keras.models import Sequential from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D from keras.optimizers import SGD from keras.layers.advanced_activations import LeakyReLU model = Sequential() model.add(Conv2D(16, (3,3), padding='same', input_shape=(513, 25, 1))) model.add(LeakyReLU()) model.add(Conv2D(16, (3,3), padding='same')) model.add(LeakyReLU()) model.add(MaxPooling2D(pool_size=(3,3))) model.add(Dropout(0.25)) model.add(Conv2D(16, (3,3), padding='same')) model.add(LeakyReLU()) model.add(Conv2D(16, (3,3), padding='same')) model.add(LeakyReLU()) model.add(MaxPooling2D(pool_size=(3,3))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(64)) model.add(LeakyReLU()) model.add(Dropout(0.5)) model.add(Dense(1, activation='sigmoid')) sgd = SGD(lr=0.001, decay=1e-6, momentum=0.9, nesterov=True) model.compile(loss=keras.losses.binary_crossentropy, optimizer=sgd, metrics=['accuracy'])
And run the model:
model.fit(trainMixed, trainVocals,epochs=10, validation_data=(testMixed, testVocals))
But I’m getting this result:
ValueError: in user code: /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function * return step_function(self, iterator) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:796 step_function ** outputs = model.distribute_strategy.run(run_step, args=(data,)) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1211 run return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica return self._call_for_each_replica(fn, args, kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica return fn(*args, **kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:789 run_step ** outputs = model.train_step(data) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:747 train_step y_pred = self(x, training=True) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:976 __call__ self.name) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py:158 assert_input_compatibility ' input tensors. Inputs received: ' + str(inputs)) ValueError: Layer sequential_1 expects 1 inputs, but it received 2 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, 2584) dtype=float32>, <tf.Tensor 'IteratorGetNext:1' shape=(None, 2584) dtype=float32>]
I am new to this topic, thanks for the help provided in advance.
Advertisement
Answer
It’s probably an issue with specifying input data to Keras’ fit()
function. I would recommend using a tf.data.Dataset
as input to fit()
like so:
import tensorflow as tf train_data = tf.data.Dataset.from_tensor_slices((trainMixed, trainVocals)) valid_data = tf.data.Dataset.from_tensor_slices((testMixed, testVocals)) model.fit(train_data, epochs=10, validation_data=valid_data)
You can then also use functions like shuffle()
and batch()
on the TF datasets.
EDIT: It also seems like your input shapes are incorrect. The input_shape
you specified for the first conv layer is (513, 25, 1)
, so the input should be a batch tensor of shape (batch_size, 513, 25, 1)
, whereas you’re inputting the shape (batch_size, 2584)
. So you’ll need to reshape and probably cut your inputs to the specified shape, or specify a new shape.