AWS Sagemaker Inference Endpoint Error : unsupported operand type(s) for *: 'NoneType' and 'Tensor'
I am trying to setup AWS Sagemaker Inference Endpoints but getting the following error for both Real Time and Asynchronous AWS Sagemaker Endpoints.
unsupported operand type(s) for *: 'NoneType' and 'Tensor'
Following Code has been used for deploying the whisper-large-v3-turbo from Hugging Face into AWS Sagemaker.
huggingface_model = HuggingFaceModel(
role=role, # IAM role with SageMaker permissions
transformers_version="4.37.0", # Specify the transformers version
pytorch_version="2.1.0", # Specify the PyTorch version
py_version="py310", # Specify the Python version
env=hub
)
audio_serializer = DataSerializer(content_type='audio/x-audio')
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.g4dn.xlarge",
async_inference_config=AsyncInferenceConfig(
output_path="s3://sagemaker-output-directory
/output", # S3 bucket for output
max_concurrent_invocations_per_instance=4, # Max concurrent invocations
),
endpoint_name='whisper-async-endpoint',
serializer=audio_serializer
)
You might want to try using Python 3.10. I switched my Python version from 3.9 to 3.10 and reset my environment. It should work, hopefully!
@SockingPanda : I am referring Python Version 3.10 in the HuggingFaceModel's "py_version" attribute and still getting the error.