-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nuget for Linux #273
Comments
@luisquintanilla Could you share your platform details (are you using windows)? Were you able to install the nuget package successfully? Are you downloading the package from nuget.org? |
@baijumeswani I am on Windows. However, I'm using devcontainers. Here is my configuration: https://github.com/luisquintanilla/ort-genai-phi2/blob/main/.devcontainer.json System Info:
Let me know if you need anything else. I also have an NVIDIA RTX 3080. However, I also get this issue on the CPU. So to keep things simple, let's assume CPU only. |
We have not published the nuget binaries for linux yet. This will be supported in future releases but is not supported right now. |
I got the same error on windows, with nuget packages.
|
I also encountered the same error when loading phi-3, but it occurred during cuda and was normal on CPU |
For CUDA, make sure your CUDA binaries are discoverable by adding them to your PATH. |
at Microsoft.ML.OnnxRuntimeGenAI.NativeMethods.OgaCreateModel(Byte[] configPath, IntPtr& model) i don't have nvidia, just want to run on CPU |
I got the same error experimenting on a Surface Studio Laptop 2 on Windows, but discovered I could get past this error by ensuring I had both references:
|
Still running into issues on Windows with version 0.3.0-rc2. Here's the error I'm getting.
I'm using the Phi-3 model. https://huggingface.co/microsoft/Phi-3-mini-4k-instruct With FP16 CUDA My code is still the same. The only thing that changed was the model I'm using. |
That would be perfect, especially for the Cuda version. |
Just to make sure - currently there's no way of running apps using |
@L-Sypniewski Work is being done to support NuGet scenarios on Linux. I'm not sure if it can make it in 0.4.0, though. |
0.4.0-rc1 NuGet package is out with Linux-x64 support. Feel free to check it out: https://www.nuget.org/packages/Microsoft.ML.OnnxRuntimeGenAI/0.4.0-rc1 I'll close this for now. |
I'm seeing dependencies on packages I can't resolve via nuget.org |
Please add the Ort-Nightly index as a source. You can find more instructions here: https://onnxruntime.ai/docs/genai/howto/install.html#pre-requisites |
When trying out a samples that uses Phi in .NET, I get the following error.
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'onnxruntime-genai' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libonnxruntime-genai: cannot open shared object file: No such file or directory
Full Stack Trace:
Repo: https://github.com/luisquintanilla/ort-genai-phi2
The text was updated successfully, but these errors were encountered: