Update README.md
#5
by
reach-vb
HF staff
- opened
README.md
CHANGED
@@ -424,6 +424,8 @@ Once a valid PyTorch version is installed, SDPA is activated by default. It can
|
|
424 |
+ model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True, attn_implementation="sdpa")
|
425 |
```
|
426 |
|
|
|
|
|
427 |
#### Torch compile
|
428 |
|
429 |
Coming soon...
|
|
|
424 |
+ model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True, attn_implementation="sdpa")
|
425 |
```
|
426 |
|
427 |
+
For more information about how to use the SDPA refer to the [Transformers SDPA documentation](https://huggingface.co/docs/transformers/en/perf_infer_gpu_one#pytorch-scaled-dot-product-attention).
|
428 |
+
|
429 |
#### Torch compile
|
430 |
|
431 |
Coming soon...
|