It seems the 11B-Vision-Instruct model is limited to 8k context length,
when deployed on via the serverless API.When will it be, or is it
possible to use the full 128k length with the serverless API? See the
inference_config.json file: { "inference_engine": "vllm", "vllm_kwargs":
{ "limit-mm-per-pro...
@ThasmikaGokal, are there any updates on deploying Meta models with
Serverless API if your subscription is with a Cloud Service Provider?
We're running into the same problem as described here in this thread. We
cannot deploy Meta models due to the fact that our subscription is
managed by a CSP.Manag...
@ThasmikaGokal Is there any news about the CSP restrictions? We are also
running into the problem that a deployment on our CSP subscription is
not possible, but would like to use the models.Thank you!
Latest Comments