-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Core] ray.exceptions.GetTimeoutError: Get timed out: some object(s) not ready. #47183
Comments
can you provide a full stacktrace when you see
|
|
@hxue3 this line... |
This is an error related to the timeout of the actor. As the model size increases, the time required to download the model from Hugging Face and to load it into vLLM also increases. You can avoid the import ray
from ray.data import DataContext
# ray init
runtime_env = {"env_vars": {"HF_TOKEN": "__YOUR_HF_TOKEN__"}}
ray.init(runtime_env=runtime_env)
# data context
ctx = DataContext.get_current()
ctx.wait_for_min_actors_s = 60 * 10 * tensor_parallel_size The |
@hxue3 lmk if this was fixed! |
Im not the original poster but this worked for me as i was loading model from s3. it was taking more than 10mins for 70b+models. And increasing the timeout fixed the issue. |
What happened + What you expected to happen
I am trying to load a quantized large model with vLLM. It is able to start the model loading, but it sometimes will stop loading the model and returns the error message
ray.exceptions.GetTimeoutError: Get timed out: some object(s) not ready.
Versions / Dependencies
ray: 2.34.0
python: 3.10
OS: ubuntu 22
Reproduction script
Issue Severity
High: It blocks me from completing my task.
The text was updated successfully, but these errors were encountered: