Sequence Processing with Recurrent Layers

From One Stream to Several

A single-input model receives one stream of data. A multi-input model receives several streams and gives each stream its own path through the network before combining the resulting representations. The Keras functional API is designed for this graph-shaped model structure.

Tracing the Model Graph

The important shift is to stop treating the model as one straight sequence of layers. Instead, inspect it as several branches that eventually meet. For a two-input model, trace the data in this order: each input node, its branch encoder, the merge operation, and the final output.

Constructing the Two-Branch Model

The construction has four main stages. First, create an input node for each data source. Second, send each input through its own encoder branch. Third, combine the branch outputs with a Keras merge operation. Finally, define the model with both input tensors and the answer tensor as its output.

question = Input(name="question")
text = Input(name="text")

question_representation = question_encoder(question)
text_representation = text_encoder(text)

combined = concatenate([question_representation, text_representation])
answer = output_layer(combined)

model = Model(
    inputs=[question, text],
    outputs=answer
)

This example is schematic: question_encoder, text_encoder, and output_layer stand for the branch and output components you choose. The important graph structure is explicit. The two named input tensors are passed to Model, and the answer tensor is declared as the output.

Choosing the Merge Operation

The branches must meet through a merge operation. Keras provides operations such as keras.layers.concatenate and keras.layers.add. The choice determines how the branch representations are brought together, so the merge line deserves separate inspection whenever the model behaves unexpectedly.

Supplying Training Data

Training data must preserve the model's input structure. The source describes two supported forms. A list supplies arrays in the same order as the model's inputs. A dictionary uses input names, associating the text array with text and the question array with question. Dictionary-based training therefore requires named inputs.

model.fit(
    [question_array, text_array],
    answers
)

model.fit(
    {
        "question": question_array,
        "text": text_array
    },
    answers
)

These two training calls express the same structural requirement in different ways. The list relies on the model's input order. The dictionary relies on the names question and text. If the order or names do not match the model inputs, the data path no longer matches the intended graph.

Debugging by Boundary

A multi-input model can fail at several distinct boundaries. Treat it as a trace: data enters an input node, passes through its branch, reaches the merge operation, and contributes to the output. The most useful debugging question is where the actual path first differs from the intended path.

• Treating a multi-input graph as one straight sequence of layers.

  The model has several paths that remain separate until the merge operation.

  Fix: Trace each input node through its own branch before inspecting the merge and output.

• Supplying list-based training data in the wrong input order.

  List-based training maps arrays according to the model's input order.

  Fix: Compare the list order with the order of the model's inputs before training.

• Using dictionary keys that do not identify the named inputs.

  Dictionary-based training depends on input names.

  Fix: Check that the model inputs are named and that the dictionary keys match those names.

• Skipping inspection of the merge line.

  Concatenate and add are distinct merge choices.

  Fix: Inspect whether the graph uses the intended Keras merge operation.

1. Check which array or named value is reaching each input node.
2. Follow each branch independently and verify that it is receiving the intended source.
3. Inspect the merge operation and confirm whether concatenate or add was intended.
4. Confirm that the model output is built from the merged representation.
5. If using a list, verify input order. If using a dictionary, verify input names.

Practice the Trace

Key Takeaways

1. A multi-input Keras model gives each data source its own branch before combining branch representations.
2. The main graph stages are input nodes, branch encoders, a merge operation, and an output.
3. keras.layers.concatenate and keras.layers.add are distinct choices for the merge stage.
4. List-based training data follows model input order, while dictionary-based training data uses named inputs.
5. The most practical debugging route is to find where the actual data path first differs from the intended graph.