bigscience/bloom · Batch Processing / Parallelism

Hello!

I have an issue while using BLOOM via the Inference API with batch processing / parallelism as the provided Python code gives the following error.

---------------------------------------------------------------------------
IncompleteReadError                       Traceback (most recent call last)
[/usr/local/lib/python3.8/dist-packages/websockets/legacy/protocol.py](https://localhost:8080/#) in transfer_data(self)
    967             while True:
--> 968                 message = await self.read_message()
    969 

14 frames
IncompleteReadError: 0 bytes read on a total of 2 expected bytes

The above exception was the direct cause of the following exception:

ConnectionClosedError                     Traceback (most recent call last)
[/usr/local/lib/python3.8/dist-packages/websockets/legacy/protocol.py](https://localhost:8080/#) in ensure_open(self)
    942 
    943         if self.state is State.CLOSED:
--> 944             raise self.connection_closed_exc()
    945 
    946         if self.state is State.CLOSING:

ConnectionClosedError: no close frame received or sent

Your help is appreciated. If this is not the right place to send this issue, I will appreciate guiding me to where I should send it. Thanks!