Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Accessibility support for Jupyter notebooks in VSCode #90408

Open
mltony opened this issue Feb 11, 2020 · 34 comments
Open

Feature request: Accessibility support for Jupyter notebooks in VSCode #90408

mltony opened this issue Feb 11, 2020 · 34 comments
Assignees
Labels
accessibility Keyboard, mouse, ARIA, vision, screen readers (non-specific) issues feature-request Request for new features or functionality notebook
Milestone

Comments

@mltony
Copy link
Contributor

mltony commented Feb 11, 2020

Hello, I would like to request accessibility support to be added to jupyter notebook in VSCode. Jupyter web application has very poor accessibility, and it seems like jupyter team is not actively working on its accessibility:
jupyterlab/jupyterlab#4878

It would be great if VSCode jupyter functionality can be made accessible to screenreader users. There are requests on the blind mailing lists (e.g. on program-l@freelists.org) every now and than about ways to use jupyter with a screenreader, and given the fact that jupyter team doesn't seem to be working actively on their accessibility, if VSCode jupyter mode can be made accessible, it will be very welcome among blind developers.

I have tried it with recent NVDA (2019.3) and it seems like pressing Control+Enter works correctly to evaluate expression in the current cell, but I couldn't find a good way for the cursor to jump to the cell output, even though visually the output is right there. Also, when you press up and down, the cursor seems to be jumping to the next-previous cells once it reaches the end of current cell. This is confusing for screenreader users, since there is no indication of that. It would be great if either some indication of another cell being activated is announced, or better if the cursor can stay within current cell and extra shortcuts to jump to previous/next cells could be added. With these two changes I think the basic functionality of jupyter would be accessible to screenreader users.

Thanks!

@vscodebot
Copy link

vscodebot bot commented Feb 11, 2020

(Experimental duplicate detection)
Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

@rebornix
Copy link
Member

rebornix commented Feb 11, 2020

@mltony thanks for your feedback. May I ask if you are currently using the Python extension which provides the notebook experience in VS Code now? For now, you can file issues against them if you are using that heavily https://github.com/microsoft/vscode-python and IMHO the cursor movements related issues are not that hard to fix.

At this same time, we are also looking into how to provide a decent notebook experience in VS Code core. We will make sure that we have a good accessibility story for notebooks. Our accessibility champion @isidorn asked me last week that if I need any help/advice on accessibility in jupyter notebook, I'm sure we can use his expertise in this area.

Thanks again for chiming in and providing concrete examples for accessibility issues you currently have with Jupyter Notebook!

@rebornix rebornix added accessibility Keyboard, mouse, ARIA, vision, screen readers (non-specific) issues notebook labels Feb 11, 2020
@isidorn
Copy link
Contributor

isidorn commented Feb 11, 2020

Agree with what @rebornix pointed out.
We definetely want to make the Jupiter notebook expereince in VS Code core accessible.
@mltony However since we are still working on that, we can let you know when it is in Insiders so you also try it out if you want and provide feedback. We also plan to test it for accessibility, but direct feedback is always best imho.
As far as I know the Jupiter support should be in Insiders later in February or in our March milestone. But @rebornix can correct me if I am wrong.

For reference here's the Notebook branch which is currently work in progress https://github.com/microsoft/vscode/tree/rebornix/notebook

@rebornix assignign this for now for February, and if we do not find time we can post pone for next milestone.

@isidorn
Copy link
Contributor

isidorn commented Mar 5, 2020

@rebornix let me know when notebooks are in a good state to try out and provide feedback. For now unassigning myself from this issue and I would be happy to try this out and provide feedback, so just ping me.

@isidorn isidorn removed their assignment Mar 5, 2020
@rebornix rebornix modified the milestones: March 2020, April 2020 Mar 31, 2020
@rebornix rebornix modified the milestones: April 2020, May 2020 Apr 24, 2020
@rebornix rebornix modified the milestones: May 2020, June 2020 Jun 1, 2020
@rebornix rebornix modified the milestones: June 2020, Backlog Jun 29, 2020
@venkateshpotluri
Copy link

venkateshpotluri commented Oct 6, 2020

Hi, I wanted to know if IPython notebook interface is screen reader accessible in VSCode. I tried loading a notebook but find the screen reader experience confusing an inaccessible. Is there documentation that I can look at, on using notebooks inside VSCode with a screen reader? Thanks!

@rebornix
Copy link
Member

rebornix commented Oct 8, 2020

@venkateshpotluri thanks for offering help! Currently the new/native notebook support in VS Code is not fully accessible so I would love to sit together with @isidorn to go through all details and figure out what are missing before asking you to try. Will keep you posted.

@rebornix rebornix modified the milestones: Backlog, October 2020 Oct 8, 2020
@jooyoungseo
Copy link

Hello, I am a screen-reader user. As far as I tested with the insider version (Commit: f5d3ba4), Jupyter Notebook in VSCode is not fully accessible.

Unlike Google Colab, the accessible mode is not activated for Jupyter Notebook in VScode.

You should be able to switch on the accessible mode within Jupyter editor by pressing Alt+F1, followed by Ctrl+E. Then, the default Jupyter editor should be made accessible via Monaco editor. However, it does not work in VSCode.

In sum, accessible Monaco editor for Jupyter Notebook is not enabled so screen readers say "blank" repeatedly in editor area.

Hope this helps.

@isidorn
Copy link
Contributor

isidorn commented Oct 20, 2020

@jooyoungseo thanks a lot for providing feedback. We plan to organise one accessibility call in the next couple of weeks where the focus will be Notebook accessibility in VS Code. Let us know if you would like to join
As for Alt+F1 not working, it might be already captured via this issue #108850 which I pushed a fix for. So you can try it out tomorrow.

fyi @sana-ajani

@jooyoungseo
Copy link

Thanks for the update @isidorn, but the latest commit (6c8c600) does not seem to have resolved the original issue.

Inside a Jupyter Notebook cell, I pressed Alt+1, followed by Ctrl+E, and I heard the following message:

Now changing the setting accessibilitySupport to ’on’.

However, the Notebook cell still remained inaccessible, meaning that screen readers (e.g., JAWS and NVDA) do not read anything in the editor.

I don't think I will be able to attend the accessibility meeting this time, but thanks for the invitation!

@isidorn
Copy link
Contributor

isidorn commented Oct 21, 2020

@jooyoungseo I just tried it out and this works fine for me both on Mac and Win.
Here are my steps:

  1. I have "editor.accessibilitySupport": "on",
  2. I install GitHub Issues Notebook extnesion
  3. I clone the vscode repository and open .vscode/notebooks/mywork.github-issues and everything is read out nicely both by NVDA and VoiceOver

@rebornix is this week on vacation, but once he gets back next week I plan to do a detailed test pass over notebooks and try to help him to improve the notebook accessibility experience.

@jooyoungseo
Copy link

jooyoungseo commented Oct 21, 2020

Still, neither NVDA nor JAWS reads cell content in document mode (AKA, forms mode), which means you are not heard back when moving your system caret inside the cell editor. Although they read the content in browse mode (AKA, virtual cursor).

I have tested with several *.ipynb files.

@rebornix rebornix added this to the On Deck milestone Oct 26, 2020
@rebornix rebornix added the feature-request Request for new features or functionality label Nov 3, 2020
@Hypokeimenon
Copy link

Hi, I'm blind and I'm tring to use vscode with jupyter extension. I can use almost everything, but I'm also having the problem to read the output. I can't know whether the cell is running or not, as well as the result. I press ctrl+shift + u, but it's not accessible.

@isidorn
Copy link
Contributor

isidorn commented Oct 10, 2022

@Hypokeimenon thanks for letting us know.
@minsa110 should be able to help, but I know she is on vacation the next two weeks. Maybe @rebornix has some advice.

@jooyoungseo
Copy link

jooyoungseo commented Oct 16, 2022

@isidorn -- Four suggestions to make the current Jupyter Notebook support more accessible for screen reader users:

  1. Need an accessibility feature to toggle lock focus to the current notebook cell

In cell edit mode, it is very confusing and embarrassing for screen readers to move their focus to the next and previous notebook cells when not intended. Similar to Toggle Tab Move Focus (Ctrl+M), it would be more accessible and less confusing for screen reader users to decide whether they want to move between cells via arrows or not explicitly.

  1. Need an audio cue to indicate when users going to the previous and next cells.

Currently, when cell edit mode focus crosses the cell border line to either the previous or next cells with up/down arrows, only text information is announced to indicate the current cell location. This is definitely better than nothing; however, when users quickly press up/down arrows to review their cell content, this kind of text alert is not as obvious as we want. I suggest providing an audio cue that users can toggle to indicate the border line of each notebook cell. This way would allow screen reader users to easily identify the cell ending/starting point by sound.

  1. Need a clearer landmark and label for notebook output area.

Currently, the notebook cell output area is marked up as role = "list". This means all the notebook cell output is included in a list container. List is not a good way to indicate this important result area. I recommend using other effective landmark region semantic with a proper aria label so that screen reader users can more easily identify this area. also this landmark area needs to have a document role to auto-activate browse mode for screen readers.

  1. Need to fix Jupyter Variables view tab table semantics.

I have noticed that the Jupyter Variables table semantics are broken against accessibility. For example, "Name", "Type", "Size", and "Value" need to be marked up with <th> tag. since they are column headers. Currently, they are not properly associated with table data cells.

image

@westurner
Copy link

westurner commented Oct 16, 2022 via email

@jooyoungseo
Copy link

@westurner, I don't see the any settings, called "auto-scroll when the current cell output is clipped by the viewport"

The following two settings do not work for disabling the auto-scroll in cell edit mode

{
    "interactiveWindow.alwaysScrollOnNewCell": false,
    "output.smartScroll.enabled": false
}

@jooyoungseo
Copy link

@minsa110 -- What do you think about the four suggestions that I shared above? I feel like the first point is especially critical and needs to be improved as the first priority.

@minsa110
Copy link

Hi @jooyoungseo, thank you for the feedback and for the ping!

For microsoft/vscode-jupyter#1 and microsoft/vscode-jupyter#2, can you try turning this off in the settings and let us know if that addresses those issues?
"notebook.navigation.allowNavigateToSurroundingCells": false

For microsoft/vscode-jupyter#3, I've routed the issue to the Jupyter team here: #168814.

For microsoft/vscode-jupyter#4, we have a few issues filed here for Variables Viewer accessibility: https://github.com/microsoft/vscode-jupyter/issues?q=is%3Aopen+is%3Aissue+label%3Aaccessibility+variable+viewer. To set expectations, these issues will likely not be addressed immediately because our variable viewer was implemented through SlickGrid, which is not accessible. However, we are in discussions to build a new accessible variable viewer as a core VS Code functionality, and use that one.

@joseluizn
Copy link

@minsa110 Seems like the other issue was closed.

I have been teaching python and this semester will teach some core pandas and related methods for a class with a visually impaired student who uses screen reader.

We tried using VSCode but we had ran into the output issue, when prompted to read NVDA would start reading the whole notebook content (including code), and there was no way to access the cell output.

Would love to know whether there is any way I can help with this issue, since #168814 seems to have been closed due to missing information.

@jooyoungseo
Copy link

jooyoungseo commented Jan 12, 2023

@joseluizn -- Until they come up with a better solution, I recommend you and your student use Quarto and its VSCode extension.

quarto preview command (Ctrl+Shift+K) will reveal an accessible Jupyter Notebook output in HTML format in side-by-side web view. quarto preview <file.ipynb> will open the result in an external browser when executed in terminal outside VSCode. If the student uses Quarto via VSCode extension, they may want to configure the following settings for better accessibility:

settings.json:

{
  "quarto.render.previewReveal": true,
  "quarto.render.renderOnSave": true,
  "quarto.render.previewType": "external"
}

@joseluizn
Copy link

@jooyoungseo Thank you very much for this suggestion, someone had mentioned to me I should check quarto accessibility and I planned to look into it on the following days. Your information has moved it up and already offer the solution, I will check the material compatibility and needed changes to work well with quarto.

@Hypokeimenon
Copy link

For students that probably don't need a robust solution, Google Colab is a good way to study. It'll be better VSCode for real and complex projects with bigdata and others. I'm a blind data scientist and I think colab is the best tool to study. VSCode will be amasing to work with when becoming accessible, but not yet. This week I'm testing a new version of databriks, and it seems accessible - solved the issues about output and others that are very simmilar to VSCode. I'm waiting a faster answer from Microsoft about this. This issue is taking a long time to resolve.

@joseluizn
Copy link

For students that probably don't need a robust solution, Google Colab is a good way to study. It'll be better VSCode for real and complex projects with bigdata and others. I'm a blind data scientist and I think colab is the best tool to study. VSCode will be amasing to work with when becoming accessible, but not yet. This week I'm testing a new version of databriks, and it seems accessible - solved the issues about output and others that are very simmilar to VSCode. I'm waiting a faster answer from Microsoft about this. This issue is taking a long time to resolve.

Thank you very much for your input.

I did try Colab earlier today with NVDA and was unable to access the output of cells. How can I access that?

It looks like you are Brazilian (or Portuguese speaker) from your profile. Would you mind providing any way for me to reach out and exchange some information about the tools you use currently? This would be very helpful if it is not an issue.

@Hypokeimenon
Copy link

Hypokeimenon commented Jan 12, 2023

You can call me on linkedin https://www.linkedin.com/in/pietro-ayres-sanchini-b8b00999/. It'll be a pleasure to help you with this issue. Colab is accessible. Below the cells you can read the output, but you need to press insert + space to get out of the input focus. I'm talking about NVDA, but it's the same way to jaws, excepting you'll need other shortcut to get out of input focus. Im' talking about this here, because I believe it is useful as a consumer experience for the changes that must happen in VSCode

@joseluizn
Copy link

Within my limited testing and knowledge about accessibility, I feel like having indicators of cell running starting and finishing might also be an important aspect for accessibility within the scope of this issue.

The following puglin I found for colab may be an useful reference for this: https://github.com/hassaku/colab-a11y-utils

@Hypokeimenon thank you, I've asked to connect with you on Linkedin.

@jooyoungseo
Copy link

Just want to highlight that "accessibility" is part of experience and it can vary person by person. FYI, I am also a blind individual teaching data science at university and I have found using VSCode in conjunction with Quarto more effective than Google Colab. However, again each person has different experience.

@joseluizn -- VSCode has audio cue settings to indicate the completion of Notebook code execution.

settings.json:

{
  "audioCues.notebookCellCompleted": "on",
  "audioCues.notebookCellFailed": "on"
}

@joseluizn
Copy link

Just want to highlight that "accessibility" is part of experience and it can vary person by person. FYI, I am also a blind individual teaching data science at university and I have found using VSCode in conjunction with Quarto more effective than Google Colab. However, again each person has different experience.

@joseluizn -- VSCode has audio cue settings to indicate the completion of Notebook code execution.

settings.json:

{
  "audioCues.notebookCellCompleted": "on",
  "audioCues.notebookCellFailed": "on"
}

Thank you for the input, I am trying quarto now. It provides an extra knowledge barrier as I myself had never used it, but it seems promising for multiple uses.

A few questions. Do you work with .ipynb notebook files or quarto .qmd files?

I ask because I'm testing an existing notebook file, and changed the code in a cell. After executing and rendering preview is there any method to go towards the relevant cell result or would it need to go through the whole of the rendered html file?

How would you proceed to the output of the cell you are currently working

@jamesowers-roo
Copy link

Hello! I'm interested to know whether being able to access the output of a cell after execution is sorted? I can't find any docs on how to navigate to that with JAWS after the cell has finished executing.

@brunoprietog
Copy link

Has anyone been able to fix the output reading? I can't do it either.

@isidorn Is there any plan to work on this? If there is no way to read the output of the cells, it is impossible to use the notebooks.

Thanks

@brunoprietog
Copy link

Before the new native design, notebooks worked much better and everything was readable. This has not been accessible for many years since you made the change

@isidorn
Copy link
Contributor

isidorn commented Jun 5, 2023

@brunoprietog thanks for the ping. I defer here to @minsa110

@fcnjd
Copy link

fcnjd commented Apr 5, 2024

Same question from my side: Has anyone got the notebook output working? I am a blind student getting into machine learning, and came across this issue via a research of exactly this problem. @minsa110 Could you please give us a progress update on this?

@CyrilleB79
Copy link

I am using (and contributing to) NVDA. I am also about to attend a course using Jupyter notebooks.

@minsa110 could you please give an update on this request, especially regarding accessing and reading smoothly the output result? Thanks.

@diegov12
Copy link

diegov12 commented Oct 4, 2024

Any updates? like at all? I need this to work with a new python math course, and don't want to run the document through quarto every time I make a change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accessibility Keyboard, mouse, ARIA, vision, screen readers (non-specific) issues feature-request Request for new features or functionality notebook
Projects
None yet
Development

No branches or pull requests