Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows issues #29

Closed
yiufung opened this issue Jul 6, 2018 · 27 comments
Closed

Windows issues #29

yiufung opened this issue Jul 6, 2018 · 27 comments
Labels
discussion help wanted Extra attention is needed

Comments

@yiufung
Copy link

yiufung commented Jul 6, 2018

Configuration

  (require 'magit-todos)
  (magit-todos-mode)

Symptom

No TODOs section shown on magit status.

Debug trials

Check mode, magit-todos-mode enabled. No errors on *Backtrace*/*Compile-Log*/*Messages*, but a special buffer *magit-todos--ag-scan-async* gives /usr/bin/nice: 'ag': No such file or directory. executable-find ag returns "c:/msys64/mingw64/bin/ag.exe".

System info

- OS: windows-nt
- Emacs: 25.3.1
- Spacemacs: 0.300.0
- Spacemacs branch: develop (rev. dc97c3c4c)
- Graphic display: t
- Distribution: spacemacs
- Editing style: emacs
- Completion: ivy
- Layers:
```elisp
(javascript
 (auto-completion :variables spacemacs-default-company-backends
                  '((company-dabbrev-code company-gtags company-etags company-keywords company-files company-ob-ipython))
                  auto-completion-return-key-behavior 'complete auto-completion-tab-key-behavior 'cycle auto-completion-complete-with-key-sequence "jk" auto-completion-enable-snippets-in-popup t auto-completion-enable-help-tooltip t auto-completion-enable-sort-by-usage t)
 autohotkey chinese colors csv emacs-lisp finance command-log spacemacs-purpose git haskell html ibuffer
 (ivy :variables ivy-enable-advanced-buffer-information t)
 latex
 (markdown :variables markdown-live-preview-engine 'vmd markdown-command "marked")
 chrome
 (org :variables org-enable-github-support t)
 (python :variables python-shell-completion-native-enable 'nil)
 ranger search-engine shell-scripts sql syntax-checking windows-scripts xkcd yaml ess syntax-checking
 (shell :variables shell-default-height 30 shell-enable-smart-eshell t shell-default-term-shell "c:/Program Files/Git/bin/bash.exe"))
- System configuration features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS MODULES

Any suggestions to hunt down where goes wrong?

@alphapapa
Copy link
Owner

I guess it's because of using Windows, the command nice -n5 ag ... doesn't work. Maybe something to do with paths or the executable being ag.exe instead of ag. Try the other scanners, and try disabling magit-todos-nice.

@alphapapa
Copy link
Owner

Please let me know if using the git grep scanner works for you. I'm considering removing support for ag.

@alphapapa
Copy link
Owner

I've just pushed a commit which rewrites some of the scanner-related code and removes ag support. Please let me know if the supported scanners (rg, git grep, and find | grep) don't work for you. Thanks.

@alphapapa alphapapa changed the title magit-todos--ag-scan-async cannot find ag Windows issues Jul 8, 2018
@alphapapa alphapapa added the WAITING Waiting for response from someone label Jul 17, 2018
@yiufung
Copy link
Author

yiufung commented Jul 18, 2018

Tried with rg with nice enabled, and see this error message.

Error parsing regex near '^(*+)[[' at character offset 2: Missing expression for repetition operator.
(Hint: Try the --fixed-strings flag to search for a literal string.)

So at least it finds rg and executes it. Maybe it's the path issue (again). Is it possible to print the actual command to *magit-todos* buffer so I can experiment with it?

Thanks and sorry for the late reply.

@alphapapa
Copy link
Owner

alphapapa commented Jul 18, 2018 via email

@yiufung
Copy link
Author

yiufung commented Jul 19, 2018

By *magit-todos* buffer, I mean the error buffer created with *magit-todos* prefix. So, for rg, it's *magit-todos--scan-with-rg*.

So I add a (message (concat command " " (s-join " " args))) in magit-todos--async-start-process, and found that the command run is

nice -n5 rg --no-heading ^(\*+)[[:blank:]]+(\?\?\?|DONT|F(?:AIL|IXME)|H(?:ACK|OLD)|KLUDGE|NEXT|OKAY|PROG|T(?:HEM|ODO)|XXXX?)[[:space:]]+(.+)|(^|[[:blank:]]+)(\?\?\?|DONT|F(?:AIL|IXME)|H(?:ACK|OLD)|KLUDGE|NEXT|OKAY|PROG|T(?:HEM|ODO)|XXXX?):(?:[[:blank:]]+(.+))? ./

Then I paste the command to cmd and run it and indeed it fails. And the reason seems to be that it doesn't quote " around the regular expression, so it's treating the | as pipe operator.. I added manually and at cmd it runs correctly.

nice -n5 rg --no-heading "^(\*+)[[:blank:]]+(\?\?\?|DONT|F(?:AIL|IXME)|H(?:ACK|OLD)|KLUDGE|NEXT|OKAY|PROG|T(?:HEM|ODO)|XXXX?)[[:space:]]+(.+)|(^|[[:blank:]]+)(\?\?\?|DONT|F(?:AIL|IXME)|H(?:ACK|OLD)|KLUDGE|NEXT|OKAY|PROG|T(?:HEM|ODO)|XXXX?):(?:[[:blank:]]+(.+))?" ./

So I tried to s-wrap the search-regexp under magit-todos-defscanner macro with ". Now the command printed seems to be the one I want, but magit-todos still didn't show up. *magit-todos--scan-with-rg* buffer is created, but empty.

My elisp ability stops me from here. Guess I will take some time to learn edebug before I can dig into it again. Anyway would like to see if this helps, or happens to anyone under Windows

@alphapapa
Copy link
Owner

And the reason seems to be that it doesn't quote " around the regular expression, so it's treating the | as pipe operator.

That shouldn't be the case, because we're not running it as a shell command, but with start-process.

I added manually and at cmd it runs correctly.

That's because, when you paste the command into a shell manually, you do need to quote it, because you're running it as a shell command.

Have you tried disabling magit-todos-nice? It's worth a try. Is nice even present on Windows?

Which Windows are you using? What environment are you running it in? Cygwin? WSL?

Anyway would like to see if this helps, or happens to anyone under Windows

I'm guessing that there are some Windows users among those who have downloaded the package, but I can't say for sure.

@yiufung
Copy link
Author

yiufung commented Jul 20, 2018

That shouldn't be the case, because we're not running it as a shell command, but with start-process.

Ah, a big misconception here.

Have you tried disabling magit-todos-nice? It's worth a try. Is nice even present on Windows?

Yes, I tried disable nice with (setq magit-todos-nice nil), no luck. nice is not present on Windows, I think the nice it uses is the one bundled with Git, which is in my PATH.

Which Windows are you using? What environment are you running it in? Cygwin? WSL?

Native Windows 10, and native-build GNU Emacs.

I'm guessing that there are some Windows users among those who have downloaded the package, but I can't say for sure.

I suppose so, maybe it's on my end.


I tried again with below steps:

  1. Start emacs with -q option
  2. Add MELPA, package-refresh-contents and package-install magit-todos.
  3. Open magit-status. The error message is same as before:

Error parsing regex near '^(*+)[[' at character offset 2: Missing expression for repetition operator. (Hint: Try the --fixed-strings flag to search for a literal string.)

@alphapapa
Copy link
Owner

Open magit-status. The error message is same as before:

Where are you seeing that error message?

Native Windows 10, and native-build GNU Emacs.

And where does your rg binary come from?

Also, please try the other scanners and let me know if they work.

@yiufung
Copy link
Author

yiufung commented Jul 20, 2018

Where are you seeing that error message?

As mentioned before, *magit-todos--scan-with-rg*.

And where does your rg binary come from?

Official release

Also, please try the other scanners and let me know if they work.

No, tried them all, no one works

@alphapapa alphapapa added help wanted Extra attention is needed and removed WAITING Waiting for response from someone labels Jul 31, 2018
@alphapapa
Copy link
Owner

alphapapa commented Jul 31, 2018

I'm sorry, I don't have a Windows 10 system to test on, so I don't know if there's much I can do about it myself. If you're able to debug it and find a solution, I'll be happy to fix it. You might also try using Cygwin Emacs, although that may not perform as well as the native build.

I see that there are two different kinds of official releases of rg for Windows. I don't know what the difference is, but you might try the other one. And if nothing else, you might consider reporting a bug on the rg tracker. I can't say for certain that it is a bug in rg, but if it's handling regexps differently on Windows (which seems unlikely to me, but not impossible), that would seem like a bug.

@Kungsgeten
Copy link
Contributor

I've tried on my Windows 7 machine and have some issues:

Using rg doesn't work. I tried to debug but I'm not really sure how to debug macro generated functions (haven't used them much myself in my elisp coding). Here's the log when using toggle-debug-on-error (doesn't matter if I use magit-todos-nice or not):

Debugger entered--Lisp error: (wrong-type-argument consp nil)
  edebug-after(0 28 ("HOLD" "TODO" "NEXT" "THEM" "PROG" "OKAY" "DONT" "FAIL" "KLUDGE" "HACK" "TEMP" "FIXME" "XXX" "XXXX" "???"))
  (cons 'or (edebug-after 0 28 keywords))
  (list 'group (cons 'or (edebug-after 0 28 keywords)))
  (cons (list 'group (cons 'or (edebug-after 0 28 keywords))) '((1+ space) (group (1+ not-newline))))
  (cons '(1+ blank) (cons (list 'group (cons 'or (edebug-after 0 28 keywords))) '((1+ space) (group (1+ not-newline)))))
  (cons '(group (1+ "*")) (cons '(1+ blank) (cons (list 'group (cons 'or (edebug-after 0 28 keywords))) '((1+ space) (group (1+ not-newline))))))
  (cons 'bol (cons '(group (1+ "*")) (cons '(1+ blank) (cons (list 'group (cons 'or (edebug-after 0 28 keywords))) '((1+ space) (group (1+ not-newline)))))))
  (cons 'seq (cons 'bol (cons '(group (1+ "*")) (cons '(1+ blank) (cons (list 'group (cons 'or (edebug-after 0 28 keywords))) '((1+ space) (group (1+ not-newline))))))))
  (backquote-list* 'seq 'bol '(group (1+ "*")) '(1+ blank) (list 'group (cons 'or (edebug-after 0 28 keywords))) '((1+ space) (group (1+ not-newline))))
  (list 'or (backquote-list* 'seq 'bol '(group (1+ "*")) '(1+ blank) (list 'group (cons 'or (edebug-after 0 28 keywords))) '((1+ space) (group (1+ not-newline)))) (backquote-list* 'seq '(or bol (1+ blank)) (list 'group (cons 'or (edebug-after 0 29 keywords))) (list 'regexp (edebug-after 0 30 magit-todos-keyword-suffix)) '((optional (1+ blank) (group (1+ not-newline))))))
  `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))
  (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline))))))
  (rxt-elisp-to-pcre (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))))
  (let* ((process-connection-type 'pipe) (directory (f-relative directory default-directory)) (extra-args (when magit-todos-rg-extra-args (--map (s-split (rx (1+ space)) it 'omit-nulls) magit-todos-rg-extra-args))) (keywords magit-todos-keywords-list) (search-regexp (rxt-elisp-to-pcre (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))))) (results-regexp (or nil (rx-to-string `(seq bol (group-n 8 (1+ (not (any ":")))) ":" (group-n 2 (1+ digit)) ":" (or (seq (group-n 1 (1+ "*")) (1+ blank) (group-n 4 (or ,@(edebug-after 0 32 keywords))) (1+ blank) (group-n 5 (1+ not-newline))) (seq (optional (1+ not-newline)) (group-n 4 (or ,@(edebug-after 0 33 keywords))) (optional (group-n 6 (regexp ,(edebug-after 0 34 magit-todos-keyword-suffix)))) (optional (1+ blank)) (optional (group-n 5 (1+ not-newline))))))))) (command (-flatten (-non-nil (list (when magit-todos-nice (list "nice" "-n5")) (list "rg" "--no-heading" (when depth (list "--maxdepth" (1+ depth))) (when magit-todos-ignore-case "--ignore-case") (when magit-todos-exclude-globs (--map (list "--glob" (concat "!" it)) magit-todos-exclude-globs)) extra-args search-regexp directory)))))) (cl-loop for elt in-ref command when (numberp elt) do (setf elt (number-to-string elt))) (magit-todos--async-start-process "magit-todos--scan-with-rg" :command command :finish-func (apply-partially (function magit-todos--scan-callback) magit-status-buffer results-regexp)))
  (catch '--cl-block-magit-todos--scan-with-rg-- (let* ((process-connection-type 'pipe) (directory (f-relative directory default-directory)) (extra-args (when magit-todos-rg-extra-args (--map (s-split (rx (1+ space)) it 'omit-nulls) magit-todos-rg-extra-args))) (keywords magit-todos-keywords-list) (search-regexp (rxt-elisp-to-pcre (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))))) (results-regexp (or nil (rx-to-string `(seq bol (group-n 8 (1+ (not (any ":")))) ":" (group-n 2 (1+ digit)) ":" (or (seq (group-n 1 (1+ "*")) (1+ blank) (group-n 4 (or ,@(edebug-after 0 32 keywords))) (1+ blank) (group-n 5 (1+ not-newline))) (seq (optional (1+ not-newline)) (group-n 4 (or ,@(edebug-after 0 33 keywords))) (optional (group-n 6 (regexp ,(edebug-after 0 34 magit-todos-keyword-suffix)))) (optional (1+ blank)) (optional (group-n 5 (1+ not-newline))))))))) (command (-flatten (-non-nil (list (when magit-todos-nice (list "nice" "-n5")) (list "rg" "--no-heading" (when depth (list "--maxdepth" (1+ depth))) (when magit-todos-ignore-case "--ignore-case") (when magit-todos-exclude-globs (--map (list "--glob" (concat "!" it)) magit-todos-exclude-globs)) extra-args search-regexp directory)))))) (cl-loop for elt in-ref command when (numberp elt) do (setf elt (number-to-string elt))) (magit-todos--async-start-process "magit-todos--scan-with-rg" :command command :finish-func (apply-partially (function magit-todos--scan-callback) magit-status-buffer results-regexp))))
  (cl--block-wrapper (catch '--cl-block-magit-todos--scan-with-rg-- (let* ((process-connection-type 'pipe) (directory (f-relative directory default-directory)) (extra-args (when magit-todos-rg-extra-args (--map (s-split (rx (1+ space)) it 'omit-nulls) magit-todos-rg-extra-args))) (keywords magit-todos-keywords-list) (search-regexp (rxt-elisp-to-pcre (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))))) (results-regexp (or nil (rx-to-string `(seq bol (group-n 8 (1+ (not (any ":")))) ":" (group-n 2 (1+ digit)) ":" (or (seq (group-n 1 (1+ "*")) (1+ blank) (group-n 4 (or ,@(edebug-after 0 32 keywords))) (1+ blank) (group-n 5 (1+ not-newline))) (seq (optional (1+ not-newline)) (group-n 4 (or ,@(edebug-after 0 33 keywords))) (optional (group-n 6 (regexp ,(edebug-after 0 34 magit-todos-keyword-suffix)))) (optional (1+ blank)) (optional (group-n 5 (1+ not-newline))))))))) (command (-flatten (-non-nil (list (when magit-todos-nice (list "nice" "-n5")) (list "rg" "--no-heading" (when depth (list "--maxdepth" (1+ depth))) (when magit-todos-ignore-case "--ignore-case") (when magit-todos-exclude-globs (--map (list "--glob" (concat "!" it)) magit-todos-exclude-globs)) extra-args search-regexp directory)))))) (cl-loop for elt in-ref command when (numberp elt) do (setf elt (number-to-string elt))) (magit-todos--async-start-process "magit-todos--scan-with-rg" :command command :finish-func (apply-partially (function magit-todos--scan-callback) magit-status-buffer results-regexp)))))
  (cl-block magit-todos--scan-with-rg (let* ((process-connection-type 'pipe) (directory (f-relative directory default-directory)) (extra-args (when magit-todos-rg-extra-args (--map (s-split (rx (1+ space)) it 'omit-nulls) magit-todos-rg-extra-args))) (keywords magit-todos-keywords-list) (search-regexp (rxt-elisp-to-pcre (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))))) (results-regexp (or nil (rx-to-string `(seq bol (group-n 8 (1+ (not (any ":")))) ":" (group-n 2 (1+ digit)) ":" (or (seq (group-n 1 (1+ "*")) (1+ blank) (group-n 4 (or ,@(edebug-after 0 32 keywords))) (1+ blank) (group-n 5 (1+ not-newline))) (seq (optional (1+ not-newline)) (group-n 4 (or ,@(edebug-after 0 33 keywords))) (optional (group-n 6 (regexp ,(edebug-after 0 34 magit-todos-keyword-suffix)))) (optional (1+ blank)) (optional (group-n 5 (1+ not-newline))))))))) (command (-flatten (-non-nil (list (when magit-todos-nice (list "nice" "-n5")) (list "rg" "--no-heading" (when depth (list "--maxdepth" (1+ depth))) (when magit-todos-ignore-case "--ignore-case") (when magit-todos-exclude-globs (--map (list "--glob" (concat "!" it)) magit-todos-exclude-globs)) extra-args search-regexp directory)))))) (cl-loop for elt in-ref command when (numberp elt) do (setf elt (number-to-string elt))) (magit-todos--async-start-process "magit-todos--scan-with-rg" :command command :finish-func (apply-partially (function magit-todos--scan-callback) magit-status-buffer results-regexp))))
  (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '(:magit-status-buffer :directory :depth :allow-other-keys)) (setq --cl-keys-- (cdr (cdr --cl-keys--)))) ((car (cdr (memq ':allow-other-keys --cl-rest--))) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:magit-status-buffer :directory :depth)" (car --cl-keys--)))))) (cl-block magit-todos--scan-with-rg (let* ((process-connection-type 'pipe) (directory (f-relative directory default-directory)) (extra-args (when magit-todos-rg-extra-args (--map (s-split (rx (1+ space)) it 'omit-nulls) magit-todos-rg-extra-args))) (keywords magit-todos-keywords-list) (search-regexp (rxt-elisp-to-pcre (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))))) (results-regexp (or nil (rx-to-string `(seq bol (group-n 8 (1+ (not (any ":")))) ":" (group-n 2 (1+ digit)) ":" (or (seq (group-n 1 (1+ "*")) (1+ blank) (group-n 4 (or ,@(edebug-after 0 32 keywords))) (1+ blank) (group-n 5 (1+ not-newline))) (seq (optional (1+ not-newline)) (group-n 4 (or ,@(edebug-after 0 33 keywords))) (optional (group-n 6 (regexp ,(edebug-after 0 34 magit-todos-keyword-suffix)))) (optional (1+ blank)) (optional (group-n 5 (1+ not-newline))))))))) (command (-flatten (-non-nil (list (when magit-todos-nice (list "nice" "-n5")) (list "rg" "--no-heading" (when depth (list "--maxdepth" (1+ depth))) (when magit-todos-ignore-case "--ignore-case") (when magit-todos-exclude-globs (--map (list "--glob" (concat "!" it)) magit-todos-exclude-globs)) extra-args search-regexp directory)))))) (cl-loop for elt in-ref command when (numberp elt) do (setf elt (number-to-string elt))) (magit-todos--async-start-process "magit-todos--scan-with-rg" :command command :finish-func (apply-partially (function magit-todos--scan-callback) magit-status-buffer results-regexp)))))
  (let* ((magit-status-buffer (car (cdr (plist-member --cl-rest-- ':magit-status-buffer)))) (directory (car (cdr (plist-member --cl-rest-- ':directory)))) (depth (car (cdr (plist-member --cl-rest-- ':depth))))) (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '(:magit-status-buffer :directory :depth :allow-other-keys)) (setq --cl-keys-- (cdr (cdr --cl-keys--)))) ((car (cdr (memq ':allow-other-keys --cl-rest--))) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:magit-status-buffer :directory :depth)" (car --cl-keys--)))))) (cl-block magit-todos--scan-with-rg (let* ((process-connection-type 'pipe) (directory (f-relative directory default-directory)) (extra-args (when magit-todos-rg-extra-args (--map (s-split (rx (1+ space)) it 'omit-nulls) magit-todos-rg-extra-args))) (keywords magit-todos-keywords-list) (search-regexp (rxt-elisp-to-pcre (rx-to-string `(or (seq bol (group (1+ "*")) (1+ blank) (group (or ,@(edebug-after 0 28 keywords))) (1+ space) (group (1+ not-newline))) (seq (or bol (1+ blank)) (group (or ,@(edebug-after 0 29 keywords))) (regexp ,(edebug-after 0 30 magit-todos-keyword-suffix)) (optional (1+ blank) (group (1+ not-newline)))))))) (results-regexp (or nil (rx-to-string `(seq bol (group-n 8 (1+ (not (any ":")))) ":" (group-n 2 (1+ digit)) ":" (or (seq (group-n 1 (1+ "*")) (1+ blank) (group-n 4 (or ,@(edebug-after 0 32 keywords))) (1+ blank) (group-n 5 (1+ not-newline))) (seq (optional (1+ not-newline)) (group-n 4 (or ,@(edebug-after 0 33 keywords))) (optional (group-n 6 (regexp ,(edebug-after 0 34 magit-todos-keyword-suffix)))) (optional (1+ blank)) (optional (group-n 5 (1+ not-newline))))))))) (command (-flatten (-non-nil (list (when magit-todos-nice (list "nice" "-n5")) (list "rg" "--no-heading" (when depth (list "--maxdepth" (1+ depth))) (when magit-todos-ignore-case "--ignore-case") (when magit-todos-exclude-globs (--map (list "--glob" (concat "!" it)) magit-todos-exclude-globs)) extra-args search-regexp directory)))))) (cl-loop for elt in-ref command when (numberp elt) do (setf elt (number-to-string elt))) (magit-todos--async-start-process "magit-todos--scan-with-rg" :command command :finish-func (apply-partially (function magit-todos--scan-callback) magit-status-buffer results-regexp))))))
  magit-todos--scan-with-rg(:magit-status-buffer #<buffer magit-todos-list: bidtrainer> :directory "c:/Dropbox/git/bidtrainer/" :depth nil)
  funcall(magit-todos--scan-with-rg :magit-status-buffer #<buffer magit-todos-list: bidtrainer> :directory "c:/Dropbox/git/bidtrainer/" :depth nil)
  (setq magit-todos-active-scan (funcall magit-todos-scanner :magit-status-buffer (current-buffer) :directory default-directory :depth magit-todos-depth))
  (closure ((pcase-1 closure (t) nil (magit-todos--insert-items (current-buffer) magit-todos-item-cache)) t) nil (setq magit-todos-updating t) (setq magit-todos-active-scan (funcall magit-todos-scanner :magit-status-buffer (current-buffer) :directory default-directory :depth magit-todos-depth)))()
  funcall((closure ((pcase-1 closure (t) nil (magit-todos--insert-items (current-buffer) magit-todos-item-cache)) t) nil (setq magit-todos-updating t) (setq magit-todos-active-scan (funcall magit-todos-scanner :magit-status-buffer (current-buffer) :directory default-directory :depth magit-todos-depth))))
  (cond ((eq magit-todos-update 't) (funcall pcase-0)) ((null magit-todos-update) (if magit-todos-updating (funcall pcase-0) (magit-todos--insert-items (current-buffer) magit-todos-item-cache))) ((or (not (integerp magit-todos-update)) (not (or magit-todos-updating (>= (float-time (time-subtract (current-time) magit-todos-last-update-time)) magit-todos-update) (null magit-todos-last-update-time)))) (funcall pcase-1)) (t (funcall pcase-0)))
  (let* ((pcase-1 (lambda nil (magit-todos--insert-items (current-buffer) magit-todos-item-cache))) (pcase-0 (lambda nil (setq magit-todos-updating t) (setq magit-todos-active-scan (funcall magit-todos-scanner :magit-status-buffer (current-buffer) :directory default-directory :depth magit-todos-depth))))) (cond ((eq magit-todos-update 't) (funcall pcase-0)) ((null magit-todos-update) (if magit-todos-updating (funcall pcase-0) (magit-todos--insert-items (current-buffer) magit-todos-item-cache))) ((or (not (integerp magit-todos-update)) (not (or magit-todos-updating (>= (float-time (time-subtract (current-time) magit-todos-last-update-time)) magit-todos-update) (null magit-todos-last-update-time)))) (funcall pcase-1)) (t (funcall pcase-0))))
  (pcase magit-todos-update ((or 't (and 'nil (guard magit-todos-updating)) (and (pred integerp) (guard (or magit-todos-updating (>= (float-time (time-subtract (current-time) magit-todos-last-update-time)) magit-todos-update) (null magit-todos-last-update-time))))) (setq magit-todos-updating t) (setq magit-todos-active-scan (funcall magit-todos-scanner :magit-status-buffer (current-buffer) :directory default-directory :depth magit-todos-depth))) (_ (magit-todos--insert-items (current-buffer) magit-todos-item-cache)))
  magit-todos--insert-todos()
  (let ((magit-insert-section--parent magit-root-section)) (magit-insert-status-headers) (magit-todos--insert-todos))
  magit-todos-list-refresh-buffer()
  apply(magit-todos-list-refresh-buffer nil)
  magit-refresh-buffer()
  (save-current-buffer (set-buffer buffer) (run-hooks 'magit-mode-setup-hook) (magit-refresh-buffer))
  (with-current-buffer buffer (run-hooks 'magit-mode-setup-hook) (magit-refresh-buffer))
  (let* ((value (and locked (magit-buffer-lock-value mode args))) (buffer (magit-mode-get-buffer mode nil nil value)) (section (and buffer (magit-current-section))) (created (not buffer))) (unless buffer (setq buffer (magit-with-toplevel (magit-generate-new-buffer mode value)))) (with-current-buffer buffer (setq magit-previous-section section) (setq magit-refresh-args args) (funcall mode) (when created (magit-status-goto-initial-section) (run-hooks 'magit-create-buffer-hook))) (magit-display-buffer buffer) (with-current-buffer buffer (run-hooks 'magit-mode-setup-hook) (magit-refresh-buffer)))
  magit-mode-setup-internal(magit-todos-list-mode nil)
  magit-mode-setup(magit-todos-list-mode)
  (let ((default-directory directory)) (magit-mode-setup (function magit-todos-list-mode)))
  magit-todos-list-internal("c:/Dropbox/git/bidtrainer/src/cljs/bidtrainer/")
  (let ((magit--refresh-cache (list (cons 0 0)))) (setq directory (if directory (file-name-as-directory (expand-file-name directory)) default-directory)) (magit-todos-list-internal directory))
  (condition-case nil (let ((magit--refresh-cache (list (cons 0 0)))) (setq directory (if directory (file-name-as-directory (expand-file-name directory)) default-directory)) (magit-todos-list-internal directory)) ('magit-outside-git-repo (cl-letf (((symbol-function 'magit-toplevel) (lambda (&rest _) default-directory))) (call-interactively (function magit-todos-list)))))
  magit-todos-list(nil)
  funcall-interactively(magit-todos-list nil)
  call-interactively(magit-todos-list record nil)
  command-execute(magit-todos-list record)
  #f(compiled-function (cmd) #<bytecode 0x1834199>)("magit-todos-list")

I removed a lot of info regarding ivy-M-x above.

I also tried with (setq magit-todos-scanner 'magit-todos--scan-with-git-grep) which works if magit-todos-nice is nil. I have some kind of nice installed (from GNU coreutils), but am not familiar with the program. Enabling nice doesn't show any hits. git-grep can not be auto-detected because of the help file showing up in the browser (as discussed in #43).

Using (setq magit-todos-scanner 'magit-todos--scan-with-find|grep) generates no debug info, but doesn't show any hits.

@alphapapa
Copy link
Owner

Hi Erik,

Thanks for the report and backtrace, but the edebug stuff makes it hard to follow. Could you please provide a backtrace without edebug on the functions? Also, please try using it in a Magit status buffer, as there might be a bug specific to the magit-todos-list command.

@Kungsgeten
Copy link
Contributor

Hi again! Sorry, when I restarted Emacs I get no errors, but using magit-todos--scan-with-rg still doesn't show any hits. Is there any good way to debug that function, that you know of? Perhaps I can check where things go wrong.

@alphapapa
Copy link
Owner

If there aren't any errors, I'm guessing there are no hits in that repo. Try running something like rg "TODO:" in a shell in the repo directory. You could also try the other scanners. And you might need to check the option magit-todos-keyword-suffix.

@Kungsgeten
Copy link
Contributor

Running rg "TODO:" returns the correct hits in a shell. I'm also using the package deadgrep in Emacs, which works. The only scanner which works for me is magit-todos--scan-with-git-grep. I tried changing magit-todos-keyword-suffix to "", but that didn't change anything regarding the results. So I find it interesting that git-grep works, but not ripgrep.

@alphapapa
Copy link
Owner

That is interesting. To debug it further would probably require using edebug to stop the callback function and inspect the contents of the rg process buffer.

@Kungsgeten
Copy link
Contributor

Kungsgeten commented Dec 14, 2018

@alphapapa The problem seems to be in the function magit-todos--line-item and is probably related to how the regex is constructed. I tried adding (message (buffer-substring (point) (line-end-position)) to this function and got the following output:

src\cljs\bidtrainer\events.cljs: ;; TODO: Better failure info?

The passed regexp argument is

\(?:^\(?8:[^:]+\):\(?2:[[:digit:]]+\):\(?:\(?1:\*+\)[[:blank:]]+\(?4:\(?:\?\?\?\|DONT\|F\(?:AIL\|IXME\)\|H\(?:ACK\|OLD\)\|KLUDGE\|NEXT\|OKAY\|PROG\|T\(?:EMP\|HEM\|ODO\)\|XXXX?\)\)[[:blank:]]+\(?5:.+\)\|\(?:.+\)?\(?4:\(?:\?\?\?\|DONT\|F\(?:AIL\|IXME\)\|H\(?:ACK\|OLD\)\|KLUDGE\|NEXT\|OKAY\|PROG\|T\(?:EMP\|HEM\|ODO\)\|XXXX?\)\)\(?6:\(?:([^)]+)\)?:\)?\(?:[[:blank:]]+\)?\(?5:.+\)?\)\)

Now (re-search-forward regexp (line-end-position) t) evaluates to nil.

When I run rg --no-heading "TODO:" in Windows I get src\cljs\bidtrainer\events.cljs:67: ;; TODO: Better failure info?.

@alphapapa
Copy link
Owner

alphapapa commented Dec 15, 2018

It appears that the line number is missing from rg output. Here's what I get for the same item:

events.cljs:1: ;; TODO: Better failure info?

I'm using rg 0.8.1 on Linux.

@alphapapa
Copy link
Owner

alphapapa commented Dec 15, 2018

BTW, you may find this helpful for debugging regexps. It shows how I converted the regexp you pasted to an rx form (after using query-replace to convert single-backslashes to double-backslashes:

;; TODO: Better failure info?

(setq argh-re "\\(?:^\\(?8:[^:]+\\):\\(?2:[[:digit:]]+\\):\\(?:\\(?1:\\*+\\)[[:blank:]]+\\(?4:\\(?:\\?\\?\\?\\|DONT\\|F\\(?:AIL\\|IXME\\)\\|H\\(?:ACK\\|OLD\\)\\|KLUDGE\\|NEXT\\|OKAY\\|PROG\\|T\\(?:EMP\\|HEM\\|ODO\\)\\|XXXX?\\)\\)[[:blank:]]+\\(?5:.+\\)\\|\\(?:.+\\)?\\(?4:\\(?:\\?\\?\\?\\|DONT\\|F\\(?:AIL\\|IXME\\)\\|H\\(?:ACK\\|OLD\\)\\|KLUDGE\\|NEXT\\|OKAY\\|PROG\\|T\\(?:EMP\\|HEM\\|ODO\\)\\|XXXX?\\)\\)\\(?6:\\(?:([^)]+)\\)?:\\)?\\(?:[[:blank:]]+\\)?\\(?5:.+\\)?\\)\\)")

(rxt-elisp-to-rx argh-re)

;;=>
;; (seq bol
;;      (submatch-n 8 (+ (not (any 58)))) ":"
;;      (submatch-n 2 (+ digit)) ":"
;;      (or (seq (submatch-n 1 (+ "*")) (+ blank)
;;               (submatch-n 4 (or "???" "DONT"
;;                                 (seq "F" (or "AIL" "IXME"))
;;                                 (seq "H" (or "ACK" "OLD"))
;;                                 "KLUDGE" "NEXT" "OKAY" "PROG"
;;                                 (seq "T" (or "EMP" "HEM" "ODO"))
;;                                 (seq "XXX" (32 "X"))))
;;               (+ blank)
;;               (submatch-n 5 (+ nonl)))
;;          (seq (32 (+ nonl))
;;               (submatch-n 4 (or "???" "DONT"
;;                                 (seq "F" (or "AIL" "IXME"))
;;                                 (seq "H" (or "ACK" "OLD"))
;;                                 "KLUDGE" "NEXT" "OKAY" "PROG"
;;                                 (seq "T" (or "EMP" "HEM" "ODO"))
;;                                 (seq "XXX" (32 "X"))))
;;               (32 (submatch-n 6 (seq (32 (seq "(" (+ (not (any 41))) ")")) ":")))
;;               (32 (+ blank))
;;               (32 (submatch-n 5 (+ nonl))))))

@fadecaps
Copy link

fadecaps commented Dec 20, 2018

I've been trying to debug this problem as well on Win7. Here's what my user-config looks like (using spacemacs):

  (require 'magit-todos)
  (magit-todos-mode)
  (setq magit-todos-scanner 'magit-todos--scan-with-git-grep)
  (setq magit-todos-git-grep-extra-args (list "-n"))
  (setq magit-todos-nice nil)
  (use-package magit-todos
    :ensure t
    :commands (magit-todos-mode)
    :config
    (setq magit-todos-recursive t
          magit-todos-depth 100)
    (custom-set-variables
     '(magit-todos-keywords (list "TODO(fadecaps)"))
     '(magit-todos-ignore-file-suffixes '("todo"))))

I haven't been able to get the todos to show up at all. It does seem that git-grep and rg both require some arguments to display line number (-n on git-grep) or to show filenames for each line versus as a heading (--no-heading on rg). I'm passing these as lists since it showed an error when using a bare string. I've been trying to get edebug to show error message, but I'm afraid I don't quite understand how to get errors to show up properly using it.

@alphapapa
Copy link
Owner

alphapapa commented Dec 21, 2018

@fadecaps

   (custom-set-variables
     '(magit-todos-keywords (list "TODO(fadecaps)"))

I don't think you can set the keywords list to a keyword with a suffix like that unless you also disable the suffix option.

Other than that, you'll need to provide information such as Git version, ripgrep version, etc.

Finally, generally, you should not be setting any options with setq. magit-todos uses lots of customization setters to ensure values are set to valid values and cascaded to other dependent settings, and using setq skips all of that. To be safe, never use setq on any variable defined with defcustom.

@fadecaps
Copy link

@fadecaps

   (custom-set-variables
     '(magit-todos-keywords (list "TODO(fadecaps)"))

I don't think you can set the keywords list to a keyword with a suffix like that unless you also disable the suffix option.

Ahh sorry I didn't realize. Copied the use-package from another thread because I was getting an error about not overwriting "jT" shortcut

Other than that, you'll need to provide information such as Git version, ripgrep version, etc.

Git: 2.14.1.windows.1
ripgrep: 0.10.0 (rev 8a7db1a918)
Emacs: 25.3.1
Spacemacs: 0.200.13.x

Finally, generally, you should not be setting any options with setq. magit-todos uses lots of customization setters to ensure values are set to valid values and cascaded to other dependent settings, and using setq skips all of that. To be safe, never use setq on any variable defined with defcustom.

Ok, how should I be setting the correct scanner and scanner parameters? This was my only lead on why the todos were not showing up which is why I had added them in there.

Thanks for the help!

@alphapapa
Copy link
Owner

alphapapa commented Dec 21, 2018

Ahh sorry I didn't realize. Copied the use-package from another thread because I was getting an error about not overwriting "jT" shortcut

That's not necessarily an error, it just indicates a conflict in an existing keymap, depending on your config.

Ok, how should I be setting the correct scanner and scanner parameters?

They should be set automatically, depending on what tools are installed on your system. If that doesn't work automatically, it's a bug.

If you need to set them, use the customization UI: M-x customize-group RET magit-todos RET.

@fadecaps
Copy link

@alphapapa
I have gotten the todos to show up! I used customize-group to set these settings:

 '(magit-todos-git-grep-extra-args (quote ("-n")))
 '(magit-todos-nice nil)
 '(magit-todos-scanner (quote magit-todos--scan-with-git-grep))

The todos show up with these settings after magit-todos-mode is run.

Having just issues making it enabled on program start. If the customize-group setting for Magit Todos Mode is set to on and saved for future sessions, the below expression as created in my .spacemacs file causes it to fail loading in Spacemacs:

'(magit-todos-mode t nil (magit-todos))

I didn't understand this description in the customize-group page about it:

See the 'magit-todos-mode' command
for a description of this minor mode.
Setting this variable directly does not take effect;
either customize it (see the info node 'Easy Customization')
or call the function 'magit-todos-mode'

Also, I wasn't able to get it to work with rg on my machine by trying similar settings with --no-heading in extra-args.

In any case, glad to having things working with git-grep! Thanks for the help @alphapapa

@alphapapa
Copy link
Owner

Glad you were able to get it working.

I'm afraid I can't help you with Spacemacs. Maybe if you file an issue on their repo they can help you.

The rg scanner already uses --no-heading, so you don't need to add it yourself.

@alphapapa
Copy link
Owner

For future reference:

If you are here because you are having a problem on Windows, and you can't find a solution or workaround here, please file a new issue that clearly describes the specific problem you're having, and include all relevant information, i.e. relevant software versions, etc. Please do not reuse this issue to report yet another Windows-related problem.

Since there is not a clearly defined problem here to be solved, I'm closing this issue. If you already reported a problem here that remains unsolved, please file a separate report about that specific problem so it can be addressed. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants