i18n library

[oruburos]

Nature of issue?
New feature request / Request For Comments

New feature details:
As a new feature request, the Internationalization of the web editor and in particular, its localization in Latin american Spanish (see #1434) depends in choosing a i18n library.
Currently, there is no library included in the web editor, and while p5 website is using one, maybe it is better to move to another library given the current stack.
I propose the following libraries.

Possible choices
Extending the discussion about two possible libraries mentioned in #595 the possible choices are:

• i18njs (Same as in p5 website)
• React-i18next
• React-intl (Same as in MERN example)
• React-intl-Redux

Considerations

• Work with React
• Display "live" versions of the components so that changes to them are immediately visible
• First case to test Internationalization/Location is Latinamerican Spanish(see Web Editor in Spanish / Consistency with Languages Selection in p5 website #1434)
• Depending in the amount of labels, async loading of labels would be nice
• Easy to add new languages
• Facilitating compliance with Unicode
• Minimizing the use of concatenated strings
• Avoiding hard-coded text
• Designing for independence from cultural conventions (e. g., date and time displays),

[oruburos]

Two more possible choices

• [Globalize] (https://github.com/globalizejs/globalize)
• [Polyglot] (https://airbnb.io/polyglot.js/)

Couple of links for comparison:

(Metrics and community size) (https://npmcompare.com/compare/i18n-react,react-i18next,react-intl,react-intl-redux)
(Features) (https://medium.com/@jamuhl/i18n-frameworks-the-unfair-showdown-8d436cd6f470)

Disclaimer

I'm tempted to use react-i18next, what do you think?

[andrewn]

This list of considerations is really good! I have a few things to think about too:

Works in node.js

Let's keep in mind we might also want to translate things on the server. For example, we send out account emails so it'd be nice if these could be translated in future. This isn't a dealbreaker as we might find another way of translating these. But would be good to have in the list of the pros/cons.

Async loading
Async loading of at least the language would be really useful. If I only view the spanish translation, we shouldn't ever load the other languages to reduce the amount of data being sent to the client.

Number formatting
Number formatting is something to add to the final bullet point e.g. in Spain, one thousand is written "1.000" and in the US, it's "1,000". My feeling is that the library we pick should support this kind of number/date/time formatting so we don't have to find a solution later on.

Pluralisation
The library should support pluralisation in the translation strings. e.g. "1 items / 5 items". This is usually done with a special format in the translation string.

Bundle size, Browser support, Intl API

It's important that we understand how large the library's bundle size. This is an important feature, so we can consider up the extra page weight against features the library provides.

Which web browsers does the library support? It's important that it works for

Many web browsers now have a built-in Internationalisation API so a library that uses this if it's available might be smaller?

Unicode / string concatenation

Facilitating compliance with Unicode
Minimizing the use of concatenated strings

Can you explain a little more about what these two mean? I'm not sure I understand. Thanks :-D

[andrewn]

It'd be great to see a list of pros/cons for a few of the most promising libraries to help us reach a decision! The "i18n frameworks — the unfair showdown" article you linked to is from 2016 and I think the libraries have all had updates in the meantime.

For example, I used react-intl a couple of years ago and the main issues at the time seem to have been addressed:

Owned by Yahoo so wasn't being updated

It seems to now be under active development again.

Accessing the translator function within the component required using a Higher Order Component

This resulted in quite convoluted code and was difficult to use. But now I see they have a useIntl() hook so this is no longer an issue.

[oruburos]

PROS

React-i18next

• Dynamic, we can choose how to store the translations and install just the module.
• Async loading. Users just download what they need, and we use the current english translation
  as the fallback
• Smaller size.

React-Intl

• Uses the Intl standard when is possible.
• More popularity/ battle tested

CONS

React-i18next

• No relative dates ( Yesterday = Date.now()-1, Tomorrow = Date.now() +1), in case we need we need to use moment.js
  (which is heavy, up to 200kbs according to bundlephobia, or look for similar solutions)
• Not standard use of the new Intl Object

React-Intl

• Async Loading of multiple namespaces is not straightforward
• Use of polyfills can give high variance in module size
• Bigger Size

Consideration
Looking the spreadsheet data about the visitor browsers, safari is the second most used browser to access the project.
According to Mozilla in https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl , the support for it would require to use polyfills.
That’s why in the next part I’m focusing on react-i18next and the i18next ecosystem https://www.i18next.com/ .

Node.js
Using this plugin cover the node js case
https://github.com/i18next/i18next-http-middleware

Browse detection
https://github.com/i18next/i18next-browser-languageDetector

Async loading
Async loading of at least the language would be really useful. If I only view the spanish translation, we shouldn't ever load the other languages to reduce the amount of data being sent to the client.
Moreover, we can split the translation in different “namespaces”, in that way we don’t need to load the whole IDE translation at once, we can first load the translation for the main namespace and load some the remaining menu translations after the first screen is properly displayed.

Translation files
Part of the discussion with @andrewn was to decide between where and how to store the translations:
It seems like the sensible approach is to put them in a static directory in the server given that:

• We don’t want to bloat the client with all the translations.
• Maybe we don’t want to store that in mongodb because it will add friction in the process of engage users to translate the editor.
• we want them to be easily accessible and exposed from the github page without previous configuration (Enticing translation efforts)

Approach
At this moment my approach is:
The translations will live in the server side, in a static directory that would be exposed in the router paths.

Once there, we will be using one of the several backends available according if we stick to this solution, use the filesystem, store the data in mongo or even use a service provider as locize (https://locize.com/?lng=en.) to retrieve the translation. These backends can provide async loading capabilities, in my current development branch I am using
https://github.com/i18next/i18next-http-backend

Size

Library	Size ( minified/ minified+gzipped )
react-i18next	17.2kb /5.7km
i18next	39.5kb /11.3kb
i18next-http-backend	5.4kb /2kb
i18next-browser-languagedetector	6.2kb/ 2.1kb

Important things

• Relative dates, is important to use them or not?
• The current development needs an extra folder in the server to store the translations, with change in the routing to access that static folder, is that appropriate?

[andrewn]

Thanks for doing such thorough research into the options. It seems like react-18next has a lot of documentation and different helper libraries that we can use.

Relative dates, is important to use them or not?

Looking into it a bit more, it seems that react-i18next doesn't do date formatting at all? They defer to a 3rd party library. We use date-fns in the Editor so would be good to figure out how that would work.

The current development needs an extra folder in the server to store the translations, with change in the routing to access that static folder, is that appropriate?

This is absolutely fine. Having the translations in the git repo as JSON makes it much easier for others to contribute.