Leaderboard is down

#149
by fourpartswater - opened

Runtime error
on'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/app/app.py", line 153, in
original_df = get_leaderboard_df()
File "/home/user/app/app.py", line 90, in get_leaderboard_df
apply_metadata(all_data) # Populate model type based on known hardcoded values in metadata.py
File "/home/user/app/src/auto_leaderboard/get_model_metadata.py", line 56, in apply_metadata
get_model_infos_from_hub(leaderboard_data)
File "/home/user/app/src/auto_leaderboard/get_model_metadata.py", line 17, in get_model_infos_from_hub
model_info = api.model_info(model_name)
File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 120, in inner_fn
return fn(*args, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/hf
***.py", line 1623, in model_info
r = requests.get(path, headers=headers, timeout=timeout, params=params)
File "/home/user/.local/lib/python3.10/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/Gryphe/MythoBoros-13b (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0806dc6f80>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
Container logs:

===== Application Startup at 2023-07-31 17:21:24 =====

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Pulling evaluation requests and results.
Cloning https://huggingface.co/datasets/open-llm-leaderboard/requests into local empty directory.
Cloning https://huggingface.co/datasets/open-llm-leaderboard/results into local empty directory.
Pulling evaluation results for the leaderboard.
eval-results/EleutherAI/gpt-j-6b
['results_2023-07-18T11:28:34.858547.json', 'results_2023-07-19T10:47:17.854530.json']
results_2023-07-19T10:47:17.854530.json

eval-results/Fredithefish/ScarletPajama-3B-HF
['results_2023-07-18T10:40:07.998848.json', 'results_2023-07-18T10:59:29.744691.json', 'results_2023-07-18T11:22:36.276384.json']
results_2023-07-18T11:22:36.276384.json

eval-results/NousResearch/Nous-Hermes-Llama2-13b
['results_2023-07-24T14:44:05.322938.json', 'results_2023-07-25T11:02:46.466402.json', 'results_2023-07-26T14:55:06.636628.json']
results_2023-07-26T14:55:06.636628.json

eval-results/NousResearch/Redmond-Puffin-13B
['results_2023-07-24T14:36:07.179231.json', 'results_2023-07-25T10:58:43.573402.json']
results_2023-07-25T10:58:43.573402.json

eval-results/Open-Orca/OpenOrca-Preview1-13B
['results_1982e56_d120381.json', 'results_2023-07-19T18:45:02.691094.json', 'results_2023-07-25T10:13:37.099822.json']
results_2023-07-25T10:13:37.099822.json

eval-results/OpenAssistant/stablelm-7b-sft-v7-epoch-3
['results_2023-07-19T17:06:42.731727.json', 'results_2023-07-19T17:07:54.588127.json']
results_2023-07-19T17:07:54.588127.json

eval-results/OpenBuddyEA/openbuddy-llama-30b-v7.1-bf16
['results_2023-07-25T17:39:33.477068.json', 'results_2023-07-25T17:52:33.029329.json']
results_2023-07-25T17:52:33.029329.json

eval-results/ehartford/dolphin-llama-13b
['results_2023-07-24T12:30:40.142317.json', 'results_2023-07-24T16:19:11.269492.json']
results_2023-07-24T16:19:11.269492.json

eval-results/gpt2
['results_2023-07-25T09:06:01.910820.json', 'results_2023-07-27T10:20:14.115679.json']
results_2023-07-27T10:20:14.115679.json

eval-results/huggingface/llama-30b
['results_2023-07-19T07:25:43.005787.json', 'results_2023-07-19T13:27:01.742279.json']
results_2023-07-19T13:27:01.742279.json

eval-results/kfkas/Llama-2-ko-7b-Chat
['results_2023-07-27T10:54:54.901743.json', 'results_2023-07-27T16:15:02.960730.json']
results_2023-07-27T16:15:02.960730.json

eval-results/klosax/pythia-70m-deduped-step44k-92bt
['results_2023-07-24T09:05:01.376273.json', 'results_2023-07-24T09:26:02.759648.json', 'results_2023-07-24T09:43:50.721558.json']
results_2023-07-24T09:43:50.721558.json

eval-results/lilloukas/GPlatty-30B
['results_2023-07-19T13:09:17.218494.json', 'results_2023-07-19T22:25:28.445280.json']
results_2023-07-19T22:25:28.445280.json

eval-results/openlm-research/open_llama_13b
['results_2023-07-20T10:02:18.035133.json', 'results_2023-07-24T14:23:47.053275.json']
results_2023-07-24T14:23:47.053275.json

eval-results/openlm-research/open_llama_3b
['results_2023-07-18T11:22:49.433588.json', 'results_2023-07-19T10:43:17.176281.json']
results_2023-07-19T10:43:17.176281.json

eval-results/openlm-research/open_llama_7b
['results_2023-07-18T12:27:20.581564.json', 'results_2023-07-19T10:52:35.127282.json']
results_2023-07-19T10:52:35.127282.json

eval-results/togethercomputer/RedPajama-INCITE-7B-Base
['results_2023-07-18T12:24:47.590202.json', 'results_2023-07-19T10:56:03.209346.json']
results_2023-07-19T10:56:03.209346.json

eval-results/wenge-research/yayi-7b-llama2
['results_2023-07-24T11:30:18.986210.json', 'results_2023-07-27T10:53:28.461116.json']
results_2023-07-27T10:53:28.461116.json

Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/home/user/.local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/home/user/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._validate_conn(conn)
File "/home/user/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
conn.connect()
File "/home/user/.local/lib/python3.10/site-packages/urllib3/connection.py", line 363, in connect
self.sock = conn = self._new_conn()
File "/home/user/.local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f0806dc6f80>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/home/user/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/home/user/.local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/Gryphe/MythoBoros-13b (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0806dc6f80>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/app/app.py", line 153, in
original_df = get_leaderboard_df()
File "/home/user/app/app.py", line 90, in get_leaderboard_df
apply_metadata(all_data) # Populate model type based on known hardcoded values in metadata.py
File "/home/user/app/src/auto_leaderboard/get_model_metadata.py", line 56, in apply_metadata
get_model_infos_from_hub(leaderboard_data)
File "/home/user/app/src/auto_leaderboard/get_model_metadata.py", line 17, in get_model_infos_from_hub
model_info = api.model_info(model_name)
File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 120, in inner_fn
return fn(*args, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/hf
***.py", line 1623, in model_info
r = requests.get(path, headers=headers, timeout=timeout, params=params)
File "/home/user/.local/lib/python3.10/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/Gryphe/MythoBoros-13b (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0806dc6f80>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

you can refer to clone leaderboard while HF team fix this

Open LLM Leaderboard org

Hi, the leaderboard is up and running again!
This error can sometimes happen when there is a network problem.
Can you please tell me where these logs were displayed?

clefourrier changed discussion status to closed

Sign up or log in to comment