What're the inputs of encoder and decoder onnx of nllb?
My encoder runs well I think, but the decoder keep giving me the same output no matter what I give to it.
Model I used:
encoder_model.onnx
decoder_model.onnx
The code of decoder run session is :
outputs = decoder_session.run(
['logits', 'present.0.decoder.key', 'present.0.decoder.value'], # logits
{
'encoder_attention_mask':attention_mask.astype(np.int64),
'input_ids': decoder_input_ids.astype(np.int64), # Decoder
'encoder_hidden_states': encoder_hidden_states, # Encoder
}
)
anyone could help? Thank you!
Hi there - you should use decoder_model_merged.onnx for this.
Hi there - you should use decoder_model_merged.onnx for this.
Hi, thanks for your reply. I have tested these days, but I haven't success yet.
I didn't get the structure of it,
The decoder_model_merged.onnx has plenty of inputs,
past_key_values.**** are the previous steps of output from decoder? If we initialize it, what should I put in?
And the number of past_key_values are fixed or is it flexible by the length of sentence?
Additionally, I tried to run this model, it shows the error that I can't solve：
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from onnx_official/decoder_model_merged_fp16_fixed.onnx failed:D:\a_work\1\s\onnxruntime\core\graph\graph.cc:1437 onnxruntime::Graph::InitializeStateFromModelFileGraphProto This is an invalid model. Subgraph output (
logits) is an outer scope value being returned directly. Please update the model to add an Identity node between the outer scope value and the subgraph output.
Could you please give me some hints for these? If you could show a example of how to run this onnx model would be perfect, appreciate it.