Implementing Keras image captioning example

I want to implement the image captioning example that https://keras.io/getting-started/sequential-model-guide/#examples has , for experimentation.

Instead of using their mentioned convnet, I decided to use resnet50 that i found at https://github.com/fchollet/deep-learning-models .

After some modifications , my code looks like:

vocab_size = 10000
max_caption_len = 16

model = ResNet50(weights='imagenet')
model.layers.pop()
output = model.get_layer('avg_pool').output

output = model.get_layer('avg_pool').output
output = Flatten()(output)
output = RepeatVector(max_caption_len)(output) # your newlayer Dense(...)
image_model = Model(model.input, output)

language_model = Sequential()

model = Sequential()
# let's encode this vector sequence into a single vector
# which will be used to compute a probability
# distribution over what the next word in the caption should be!