Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support pty on windows with ConPty api #109

Closed
wants to merge 13 commits into from
Closed

Support pty on windows with ConPty api #109

wants to merge 13 commits into from

Conversation

ghost
Copy link

@ghost ghost commented Nov 26, 2020

This is an attempt to support pseudo-terminal on windows using ConPty API (should resolve issue #95)

NOTICE:

This pull requests introduces major api change for Open (*os.File -> Pty), but I think it's easy to migrate from the original api since you can convert Pty to os.File on unix platforms easily with simple type cast, but for windows platform, you probably need to handle it differently since the ConPty api requires user provided data streams for io.

Sample wrapper for windows tty: https://github.com/arhat-dev/go-pkg/blob/master/exechelper/tty_windows.go

Known issues:

  • exit code 0xC0000142 on first command execution (help wanted)
  • content in a tty window not rendered correctly after serval interaction, but can be resolved after serval Resize

Design decisions:

  • Currently using go:linkname for os/exec.Cmd compatibility
    • need to test with multiple go versions in CI to ensure go stdlib compatibility or
    • need to implement new api for windows platform? (discouraged)

Credits goes to https://github.com/ActiveState/termtest, they provided a relatively good example for ConPty in go.

@mattcanty
Copy link

Interested in seeing this happen. Let me know if I can help in any way...

README.md Show resolved Hide resolved
@creack
Copy link
Owner

creack commented Apr 5, 2021

@jeffreystoke I apologize, I still have this in my todo list, will get to it when I can.

@mattcanty This is quite a large change and I haven't find time yet to look into it. Need to test it, make sure it doesn't break existing codebases, doesn't force a large dependency tree, etc.

Ideally, we'd need some automated testing... Checking for various exec modes, daemons, foreground, etc, and ensure that it works in all OS/Arch while staying backward compatible

Copy link
Owner

@creack creack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry again for the delay, I still didn't have time to test it, but here are some comments from reviewing the code

.gitignore Outdated Show resolved Hide resolved
run_windows.s Outdated Show resolved Hide resolved
pty_windows.go Show resolved Hide resolved
doc.go Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
run_windows.go Show resolved Hide resolved
run_windows.go Show resolved Hide resolved
run_windows.go Show resolved Hide resolved
run_windows.go Outdated Show resolved Hide resolved
util_windows.go Outdated Show resolved Hide resolved
@creack
Copy link
Owner

creack commented Apr 5, 2021

I managed to try on windows, however, I couldn't get anything to work. Would you have a sample code that would demonstrate how to use this on windows?

All my tests ended up with an error from pty.Start:

CreatePseudoConsole: The operation completed successfully.

A bit strange to have "success" to be an error, but it might be because I do something wrong.

I tried using the lib directly, and I also tried the lib your linked to (arhat.dev/pkg/exechelper), but without any luck.

@JustinTimperio
Copy link

Hi @creack and @jeffreystoke,

Glad to see some action here. It seems this is one of many libraries looking to make use of this new tool. You may be interested in some work done by @marcomorain in his repo go-conpty. I too have been searching for a Windows PTY solution but have ended up arriving at similar issues.

@ghost
Copy link
Author

ghost commented Apr 6, 2021

@creack #109 (comment)

Did you run as system admin? If not, can you try again with admin privileges to see whether it's a security issue?

As noted before, I'm not a windows expert, you may have to seek help from others, good luck :)

@blasten
Copy link

blasten commented Apr 28, 2021

Thanks for working on this. I tried the following on Windows, but got an error:

if err := pty.InheritSize(os.Stdin, ptmx); err != nil {
  log.Printf("error resizing pty: %s", err)
}

produced "error resizing pty: The handle is invalid."

Copy link
Owner

@creack creack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update. I'll try to test it later today.

run_windows.go Show resolved Hide resolved
winsize_unix.go Outdated Show resolved Hide resolved
winsize_windows.go Outdated Show resolved Hide resolved
doc.go Outdated Show resolved Hide resolved
pty_windows.go Outdated Show resolved Hide resolved
@creack
Copy link
Owner

creack commented May 31, 2021

Would you mind adding an example for windows in the readme? Once merge, this PR will be gone and the links provided in the description forgotten.

@creack
Copy link
Owner

creack commented May 31, 2021

nit: if you could push commits, it would make review easier. Keeping everything in one commit with force push is neat and tidy but makes it difficult to see what changed and requires a full re-review.
We can squash at the end while merging, resulting in a single commit.

@ghost
Copy link
Author

ghost commented May 31, 2021

Would you mind adding an example for windows in the readme? Once merge, this PR will be gone and the links provided in the description forgotten.

The usage is the same with other platforms', so I don't think it's necessary to create a special windows example, but we can note requirements for windows platform, wdyt?

Requirements: https://docs.microsoft.com/en-us/windows/console/createpseudoconsole#requirements

@ghost
Copy link
Author

ghost commented May 31, 2021

nit: if you could push commits, it would make review easier. Keeping everything in one commit with force push is neat and tidy but makes it difficult to see what changed and requires a full re-review.
We can squash at the end while merging, resulting in a single commit.

Ok, I will do that for following changes.

winsize.go Show resolved Hide resolved
winsize.go Outdated Show resolved Hide resolved
doc.go Show resolved Hide resolved
@ghost
Copy link
Author

ghost commented May 31, 2021

@blasten The handle invalid error means the provided os.Stdin is not a ConPty handle, we have to add support for old windows console api to make it working, I guess that would happen in a new pull request?

@creack
Copy link
Owner

creack commented Jun 21, 2021

Sorry, didn't get time to dig into this yet. Something came to my attention though, no context, no detail but saw this:

leokhoa@4fa3b1e

As the lib would now panic if the DLL loading fails, we'll need to figure out want's going on, and whether fix the issue or make sure importing the lib doesn't break unrelated applications if a dll is missing.

@ghost
Copy link
Author

ghost commented Aug 29, 2021

@creack No idea why that would panic, but never mind, I have just pushed the latest implementation using LazyDLL and LazyProc, it will never panic on dll loading/proc finding failures as noted in my commit message.

@ghost ghost requested a review from creack August 29, 2021 10:13
@ghost
Copy link
Author

ghost commented Aug 30, 2021

fyi, go1.17 disabled //go:linkname support for private struct member, so this pull request will probably not be merged

ref: golang/go#46777 (comment)

@ghost
Copy link
Author

ghost commented Aug 30, 2021

Made it working for go1.17 according to the issue referenced above (also works with go1.16, probably will work for earlier go toolchain)

@photostorm
Copy link

photostorm commented Aug 30, 2021

Is there anything that can be done with "The operation completed successfully." being reported as error when calling CreatePseudoConsole or Resize?

@photostorm
Copy link

photostorm commented Aug 30, 2021

@jeffreystoke I made an attempt to fix the issue I was talking about and seeing if I could fix render bug. I do not work with Windows a lot. I could just been luck when I was testing my code, but it seem to fixed the issue. My copy of your fork: https://github.com/photostorm/pty

@ghost
Copy link
Author

ghost commented Aug 31, 2021

@photostorm me neither.

re: your fix

Good point, I completely misused the COORD, would you like to merge it on your own or something else?

@photostorm
Copy link

pty-patch.txt

I created a patch file for you.

Jeffrey Stoke and others added 12 commits October 4, 2021 13:19
Signed-off-by: Jeffrey Stoke <me@arhat.dev>
For backward compatibility

Signed-off-by: Jeffrey Stoke <me@arhat.dev>
Remove module dependency golang.org/x/sys/windows

Signed-off-by: Jeffrey Stoke <me@arhat.dev>
All platforms share the same GetSize implementation

Signed-off-by: Jeffrey Stoke <me@arhat.dev>
Keep it constant with other platforms

Signed-off-by: Jeffrey Stoke <me@arhat.dev>
Signed-off-by: Jeffrey Stoke <me@arhat.dev>
Signed-off-by: Jeffrey Stoke <me@arhat.dev>
Add missing SmallRect

Signed-off-by: Jeffrey Stoke <me@arhat.dev>
winsize: w to ws
Signed-off-by: Jeffrey Stoke <me@arhat.dev>
Always call LazyProc.Find() before calling LazyProc.Call() to avoid
panic due to compulsory mustFind() call in Call()

Signed-off-by: Jeffrey Stoke <me@arhat.dev>
Signed-off-by: Jeffrey Stoke <me@arhat.dev>
* COORD passed as uintptr for pty creation and resizing

  This should fix unexpected error when creating and resize terminal

* Check return value of syscalls to determine error state instead of using err directly

  This should resolve "The operation completed successfully." problem.

photostorm contributed these fixes, I just removed some redundant variable declarations and made this commit.

Signed-off-by: Jeffrey Stoke <me@arhat.dev>

Co-authored-by: photostorm <3271896+photostorm@users.noreply.github.com>
Co-authored-by: Jeffrey Stoke <me@arhat.dev>
@bin101
Copy link

bin101 commented Oct 27, 2021

Is something still missing here so that this can be merged into main?

@jkzilla
Copy link

jkzilla commented Dec 10, 2021

@creack are you available to take a glance to see if this matches and fixes the requested changes?

@pete-woods
Copy link

Apologies for a non-helpful 👍 comment, but I've been eagerly watching this PR since it was opened, and it would be a great xmas present to see it merged.

@creack
Copy link
Owner

creack commented Mar 27, 2022

I still haven't had the time to dig into this, I am sorry.

If someone could come up with an easy way to test it, it would be quite helpful as it is not clear to me in which scenario it is supposed to be working.

@pete-woods
Copy link

What we're hoping to use this for is a Go based SSH server where the terminal is provided by this package. At present the server works great on UNIX platforms, using this package, but has to fall back to plain stdout for Windows.

@pete-woods
Copy link

We're using this high level SSH server wrapper for the low level Go SSH package: https://github.com/gliderlabs/ssh

@creack
Copy link
Owner

creack commented Mar 27, 2022

Just tried the pty example from that repository, it doesn't even compile, which is quite bad as even if it is not supported, it shouldn't break compilation.
I'll push a fix for that asap and will keep digging. Still not sure what/how to test though. Once I run the pty example, how do I check that this PR works as expected? What am I supposed to see before/after?

@pete-woods
Copy link

pete-woods commented Mar 27, 2022

You should have a working terminal, honestly that's it. Like pressing "up", ctrl-C, stuff like that should work. Previously you literally couldn't make a pty with your package at all on Windows, at the level of the code (intentionally) not compiling there.

@creack
Copy link
Owner

creack commented Mar 27, 2022

I just cloned the branch, it doesn't compile (go1.18)
Getting:

# github.com/creack/pty
.\cmd_windows.go:24:3: //go:linkname must refer to declared function or variable

@pete-woods
Copy link

Having never done Windows (or indeed any C/C++) bindings in Go, I couldn't offer better suggestions than you'd find on Google. Does this need to be compiled on a Windows machine?

At any rate, it seems like Windows would have to be added to the CI pipeline for this project in order to keep it building on Windows at very least.

@drakkan
Copy link

drakkan commented Mar 28, 2022

Hi,

I also tried this patch a few months ago. It compiled back then but had several issues with window resizing and other things I don't rember well. I added this feature externally to the library using conpty, unfortunately the code was part of a proprietary project and I cannot share it. I suggest to try conpty to the ones interested in this feature

@lonnywong
Copy link

Can't build with go1.18.1:

../go/pkg/mod/github.com/donorp/pty@v1.1.12-0.20211004111936-294eccab62ed/cmd_windows.go:24:3: //go:linkname must refer to declared function or variable

@photostorm
Copy link

photostorm commented Jul 27, 2022

I have opened a new pull request for ConPTY if everyone would like to try it.

#155

@donorp donorp closed this by deleting the head repository Jan 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.