Hello there! I'm not part of the dev team but I may have the workaround for you.
(I actually tested your code on Win10 and it actually runs with 0 issues.)

The problem

Native Windows GPU support is no longer a thing. If you read the docs they show you which versions are compatible. As you may see cuda 11.8 isn't even on the page...

WARNING

I can see that you try to use LSTM. This version of Tensorflow.NET can save this layer, but can't load it back (I couldn't find a version where this wasn't a problem).
Please check whether it's possible with your environment.

My solution

Just downgrade the version of your NuGet packages until native GPU support comes back. Right now this seems the fastest way to do it.
This you can train and run a model in this environment on GPU:

But saving it could be problematic as I've mentioned.

Workaround

IF the save method actually supported save_formats which would be compatible with the Tensorflow python library then you could just simply save the model and use WSL2 for training. (This would require a little python scripting).
But right now this is not possible.