Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring in localization into the API #158213

Closed
TylerLeonhardt opened this issue Aug 15, 2022 · 12 comments · Fixed by #163344
Closed

Bring in localization into the API #158213

TylerLeonhardt opened this issue Aug 15, 2022 · 12 comments · Fixed by #163344
Assignees
Labels
api api-finalization feature-request Request for new features or functionality insiders-released Patch has been released in VS Code Insiders l10n-platform Localization platform issues (not wrong translations) on-testplan
Milestone

Comments

@TylerLeonhardt
Copy link
Member

TylerLeonhardt commented Aug 15, 2022

vscode-nls and it's build time companion vscode-nls-dev has existed for quite a while now... and in its original design it was very filepath driven. We felt this pain when bundling extensions became the norm and introduced a webpack loader and bundler that baked filepaths into source code...

That mitigation of rewriting JS code worked... but this all became more complicated when talking about the web because the web has no "nice" way of loading strings synchronously. And even if there was such a nice thing, we wouldn't want synchronous loading from the internet... so instead we are moving in a direction where Core is responsible for loading the bundle of strings for an extension, asynchronously, before an extension is loaded and then those strings are provided to the extension via an API.

declare module 'vscode' {
	export namespace env {
		export function nls(message: string, ...args: string[]): string;
		export function nls(options: { message: string; comment: string[] }, ...args: string[]): string;
		export interface nls {
			contents: { [key: string ]: string };
			uri: Uri;
		}
	}
}

In addition to this, we will have an additional field in the package.json: nlsBundleLocation which is at the root next to main and browser. This is useful for:

  • being explicit in an extension that supports localized strings
  • allows VS Code to not have to "guess" where the localized strings are in the extension (this is especially important in web where we don't have a filesystem that we can crawl)

Important definitions:

  • message: the message parameter is the English string. This is used for a few things:
    • It's the string used when VS Code is in the default (English) language
    • It's the fallback string when a bundle doesn't contain a translation for a particular string
    • It's the key in the bundle (which is essentially just a big dictionary) who's value will be the translated version of itself in whatever language is chosen
  • options: This follows the same format we have when you wanna add a comment in a package.nls.json. The comment array will be an array of comments that will be included in the exported XLF files (the files sent to translators). The message is the same as defined above.
  • i18n.contents: This will contain the key-value pairs of the strings that VS Code loaded. This is useful if you want to pass a subprocess, a web worker, or a webview the contents of the bundle so that that environment can have all the translations that are currently being used
  • i18n.uri: This will contain the Uri to the bundle that was loaded. This is useful if you want to pass a subprocess, a web worker, or a webview the Uri of the bundle so that that environment can load the translations that are currently being used.

Along side of this work, we will ship a very small helper package called @vscode/nls or something like that that can be given the contents or uri value and load the bundle and handle string interpolation. This library will really only be used for subprocesses/workers/webviews that can't depend on the vscode API directly.

@vscode/nls

Current API proposal:

export function config(config: { bundleUri: string; } | { bundleContents: string | i18nJsonFormat; }): void;

export function nls(message: string, ...args: string[]): string;
export function nls(options: { message: string; comment: string[] }, ...args: string[]): string;

@vscode/nls-dev

This is a CLI & library for managing the usages of vscode.env.nls. It has two main usecases:

  • Export: export the English strings out of the code and put them in a familiar JSON or XLF format
  • Import: imports XLF files and turns them into the respective package.nls.*.json and bundle.nls.*.json files

It currently has three commands:

  • Export
  cli.js export [args] <pattern..>    Export strings from source files

Positionals:
  pattern  Glob pattern of ts files to include  [array] [required] [default: []]

Options:
      --version  Show version number                                   [boolean]
      --help     Show help                                             [boolean]
  -o, --outDir   Output directory                        [string] [default: "."]
  • Generate XLF
  cli.js generate-xlf [args]          Generate an XLF file from a JSON i18n file

Options:
      --version             Show version number                        [boolean]
      --help                Show help                                  [boolean]
  -p, --packageNlsJsonPath  JSON i18n file to generate XLF from
                                                             [string] [required]
  -b, --i18nBundleJsonPath  JSON i18n file to generate XLF from
                                                             [string] [required]
  -o, --outFile             Output file                      [string] [required]
  • Import XLF
  cli.js import-xlf [args] <xlfPath>  Import an XLF file into a JSON i18n file

Positionals:
  xlfPath  Glob pattern of ts files to include               [string] [required]

Options:
      --version  Show version number                                   [boolean]
      --help     Show help                                             [boolean]
  -o, --outDir   Output directory that will contain the i18n.<locale>.json file
                 and the package.nls.json file           [string] [default: "."]
@TylerLeonhardt TylerLeonhardt added feature-request Request for new features or functionality api l10n-platform Localization platform issues (not wrong translations) api-proposal labels Aug 15, 2022
@TylerLeonhardt TylerLeonhardt added this to the August 2022 milestone Aug 15, 2022
@TylerLeonhardt TylerLeonhardt self-assigned this Aug 15, 2022
@wandyezj
Copy link

wandyezj commented Aug 16, 2022

Is it possible to see which language is currently being used via the API?

Does anything need to be changed in an extension based on Left to Right languages (e.g. English) verses Right to Left languages (e.g. Arabic)? The main case I tend to see is layout, for example if there is a custom rendered view.

@TylerLeonhardt
Copy link
Member Author

@wandyezj

Is it possible to see which language is currently being used via the API?

vscode.env.language

Does anything need to be changed in an extension based on Left to Right languages (e.g. English) verses Right to Left languages (e.g. Arabic)? The main case I tend to see is layout, for example if there is a custom rendered view.

VS Code doesn't support RTL: #11770 and I suspect that idea of LTR vs RTL would come in as an API separate to this proposal.

@wandyezj
Copy link

wandyezj commented Aug 16, 2022

I see:

vscode.env.language

Represents the preferred user-language, like de-CH, fr, or en-US.

  • Will the vscode.end.language sync up with language translation the vscode.env.i18n API returns?
  • How does this work if the language is changed? Does the entire editor and the extension reload?
  • Is there a way to account for formatted strings like loads ${value} or Hello ${name}, would you like ${item}? (Including the interpolated piece matters for translation, simply concatenating doesn't translate well).

With vscode.env.language it should be simple enough for a developer to create their own function to map the provided language code to a direction if the feature is needed for the extension.

@TylerLeonhardt
Copy link
Member Author

TylerLeonhardt commented Aug 16, 2022

Will the vscode.end.language sync up with language translation the vscode.env.i18n API returns?

Yes. vscode.env.i18n() will return the string in whatever vscode.env.language is set to so long as there was a translation available for that string in that language. And fallback to English if not.

Does the entire editor and the extension reload?

Yes. This is what happens today and will continue to happen.

Is there a way to account for formatted strings like loads ${value} or Hello ${name}, would you like ${item}?

Not at this time. We are aligning to what vscode-nls already does but in theory something like that could be added in the future.

It should be simple enough for a developer to create their own function to map the provided language code to a direction if the feature is needed for the extension.

Yes but I think there's enough value here:

  • String interpolation - the {0}
  • Automatic loading of your strings before your extension activates (so that any top level vscode.env.i18n calls still work) - this is especially important in the web where we must load strings via an async fetch call
  • Tooling around exporting strings out of your source code into a familiar format (keeping the same XLF format that vscode-nls uses) but additional formats could be added

@TylerLeonhardt
Copy link
Member Author

Updates from UX sync:

  • nls isn't better than i18n... maybe l10n?
  • use a t function on i18n instead of this half function/half object thing
  • either take in (key, ...args) or ({ key: ..., args: ..., comments: ...})

@TylerLeonhardt
Copy link
Member Author

TylerLeonhardt commented Sep 8, 2022

Ok I have been playing with this a lot today… and here’s what I’d like to run:

  • vscode.l10n - The API for translating strings in your extension’s code
  • @vscode/l10n-dev - The tooling used for extracting l10n strings from vscode extensions and working with XLF files
  • @vscode/l10n - The library used for loading the translations into subprocesses of your extension
  • package.l10n.json - The file used for translating static contributions in your extension’s package.json

vscode.l10n

This contains the following definition:

declare module 'vscode' {
	export namespace l10n {
		export function t(message: string, ...args: string[]): string;
		export function t(options: { message: string; args: string[], comment: string[] }): string;
		contents: { [key: string ]: string };
		uri: Uri;
	}
}

The new namespace keeps the i18n.t(...) that we will see all over code short and sweet.

@vscode/l10n-dev

Tooling used for extracting l10n strings from vscode extensions and working with XLF files. Here's what the CLI looks like:

❯node ./dist/cli.js export --help
cli.js export [args] <path..>

Export strings from source files. Supports glob patterns.

Positionals:
  path  TypeScript files to extract strings from. Supports folders and glob
        patterns.                               [array] [required] [default: []]

Options:
      --version  Show version number                                   [boolean]
      --help     Show help                                             [boolean]
  -o, --outDir   Output directory                        [string] [default: "."]
❯node ./dist/cli.js generate-xlf --help
cli.js generate-xlf [args] <path..>

Generate an XLF file from a collection of `*.l10n.json` files. Supports glob
patterns.

Positionals:
  path  L10N JSON files to generate an XLF from. Supports folders and glob
        patterns.                               [array] [required] [default: []]

Options:
      --version   Show version number                                  [boolean]
      --help      Show help                                            [boolean]
  -l, --language  The source language that will be written to the XLF file.
                                                        [string] [default: "en"]
  -o, --outFile   Output file                                [string] [required]
❯node ./dist/cli.js import-xlf --help  
cli.js import-xlf [args] <path..>

Import an XLF file into a JSON l10n file

Positionals:
  path  XLF files to turn into `*.l10n.<language>.json` files. Supports folders
        and glob patterns.                      [array] [required] [default: []]

Options:
      --version  Show version number                                   [boolean]
      --help     Show help                                             [boolean]
  -o, --outDir   Output directory that will contain the l10n.<language>.json
                 files                                   [string] [default: "."]

There is also a programatic way to interact with the tooling via JS.

@vscode/l10n

Library used for loading the translations into subprocesses of your extension. It has an API of:

  • the same t function
  • a config function that looks like this that will receive:
function config(config: { uri: string; } | { contents: string | l10nJsonFormat }): void

interface l10nJsonFormat {
	[key: string]: MessageInfo;
}

package.l10n.json

> NOTE: for backwards compatibility, package.nls.json is also supported.

This file is used for translating static contributions in your extension's package.json.
deferred

Niceness

The api being vscode.l10n.t() pairs nicely with @vscode/l10n because you either do this in your extension code:

import { l10n } from 'vscode';

or in a subprocess you do

import * as l10n from '@vscode/l10n';

and the API becomes the same:

l10n.t('Hello world {0}', blah);

@TylerLeonhardt
Copy link
Member Author

TylerLeonhardt commented Sep 28, 2022

Moving to Oct and marking finalization. This has 2 TPIs associated with it:
#161870
https://github.com/microsoft/vscode-internalbacklog/issues/3167

@jrieken
Copy link
Member

jrieken commented Oct 4, 2022

api-notes

  • document when/why i10n.uri is undefined
  • align bundle and uri value
  • document sub-logic for t, eg {0} syntax
  • for the t function support format2
  • document for params, namespace

@TylerLeonhardt
Copy link
Member Author

for the t function support format2

This isn't so simple. What does the change to the API proposal look like to support this?

export function t(message: string, ...args: any[]): string;

export function t(message: string, args: Record<string, any>): string;

Some scenarios:

l10n.t('asdf {0}', 'world');
l10n.t('asdf {world}', { world: 'world' });
l10n.t('asdf {0}', { world: 'world' });
l10n.t('asdf {0}', ['asdf', 'world']);

I think that: l10n.t('asdf {0}', 'world'); looks nicer than l10n.t('asdf {0}', ['world']); but I do want to support the object syntax

@jrieken
Copy link
Member

jrieken commented Oct 5, 2022

I think that: l10n.t('asdf {0}', 'world'); looks nicer than l10n.t('asdf {0}', ['world']); but

I agree with that. Could we be more strict and spec the var-args to not be any but string or number? That would allow you to disambiguate them and IMO it won't hurt any user of the var-args variant

@alexr00
Copy link
Member

alexr00 commented Oct 11, 2022

I adopted this in GHPRI, ran into an issue, and discussed with @TylerLeonhardt. Posting here for API discussion:

const pr: PullRequestModel | undefined = await this.manager.getMatchingPullRequestMetadataFromGitHub(branch.upstream?.remote, branch.upstream?.name);
		if (pr && (pr.model.state !== GithubItemStateEnum.Open)) {
			const mergedMessage = vscode.l10n.t('The pull request for {0} has been merged. Do you want to create a new branch?', branch.name);

I ended up with the following compile error:

Overload 1 of 3, '(message: string, ...args: (string | number)[]): string', gave the following error.
      Argument of type 'string | undefined' is not assignable to parameter of type 'string | number'.
    Overload 2 of 3, '(message: string, args: Record<string, any>): string', gave the following error.
      Argument of type 'string | undefined' is not assignable to parameter of type 'Record<string, any>'.
        Type 'undefined' is not assignable to type 'Record<string, any>'.

I assumed this was just a breaking API change so updated my loc API usage to this:

const mergedMessage = vscode.l10n.t('The pull request for {0} has been merged. Do you want to create a new branch?', [branch.name]);

@TylerLeonhardt saw this and commented that it's not the usual pattern for templates with a variable number of arguments.

@jrieken
Copy link
Member

jrieken commented Oct 11, 2022

  • Either extension the export function t(message: string, ...args: Array<string | number>): string; overload to accept undefined or don't allow undefined to surface in messages
  • support comment: string | string[]

@VSCodeTriageBot VSCodeTriageBot added the unreleased Patch has not yet been released in VS Code Insiders label Oct 11, 2022
@VSCodeTriageBot VSCodeTriageBot added insiders-released Patch has been released in VS Code Insiders and removed unreleased Patch has not yet been released in VS Code Insiders labels Oct 12, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Nov 25, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
api api-finalization feature-request Request for new features or functionality insiders-released Patch has been released in VS Code Insiders l10n-platform Localization platform issues (not wrong translations) on-testplan
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants