Hi, I am wonder that how to initialize the state of the encoder and decoder. I tried both encoder_states and [a1, b1] to initialize the state of the decoder_lstm2 but got bad result. I can get good results when I use only decoder_lstm1 with one or several encoders which without initialize the state. Could you please help me? Here is my code.
encoder_inputs = Input(shape=(max_video_length, 4096), dtype='float32')
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_outputs2)
encoder_states = [state_h, state_c]
decoder_inputs = Input(shape=(None, len(char_list)), name="decoder_inputs")
decoder_lstm1 = LSTM(latent_dim, return_sequences=True, return_state=True, name="decoder_lstm1")
decoder_lstm2 = LSTM(latent_dim, return_sequences=True, return_state=True, name="decoder_lstm2")
decoder_dense = Dense(len(char_list), activation='softmax')
decoder_outputs, a1, b1 = decoder_lstm1(decoder_inputs, initial_state=encoder_states)
decoder_outputs, a2, b2 = decoder_lstm2(decoder_outputs, initial_state=encoder_states)
decoder_outputs = decoder_dense(decoder_outputs)
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
Hi, I am wonder that how to initialize the state of the encoder and decoder. I tried both encoder_states and [a1, b1] to initialize the state of the decoder_lstm2 but got bad result. I can get good results when I use only decoder_lstm1 with one or several encoders which without initialize the state. Could you please help me? Here is my code.
encoder_inputs = Input(shape=(max_video_length, 4096), dtype='float32')
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_outputs2)
encoder_states = [state_h, state_c]
decoder_inputs = Input(shape=(None, len(char_list)), name="decoder_inputs")
decoder_lstm1 = LSTM(latent_dim, return_sequences=True, return_state=True, name="decoder_lstm1")
decoder_lstm2 = LSTM(latent_dim, return_sequences=True, return_state=True, name="decoder_lstm2")
decoder_dense = Dense(len(char_list), activation='softmax')
decoder_outputs, a1, b1 = decoder_lstm1(decoder_inputs, initial_state=encoder_states)
decoder_outputs, a2, b2 = decoder_lstm2(decoder_outputs, initial_state=encoder_states)
decoder_outputs = decoder_dense(decoder_outputs)
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)