5.9 Translating the documentation

The mailing list translations@lilynet.net is dedicated to LilyPond web site and documentation translation; on this list, you will get support from the Translations Meister and experienced translators, and we regularly discuss translation issues common to all languages. All people interested in LilyPond translations are invited to subscribe to this list regardless of the amount of their contribution, by sending an email to translations-request@lilynet.net with subject subscribe and an empty message body. Unless mentioned explicitly, or except if a translations coordinator contacts you privately, you should send questions, remarks and patches to the list translations@lilynet.net. Please note that traffic is high on the English-speaking list lilypond-user@gnu.org, so it may take some time before your request or contribution is handled.

5.9.1 Getting started with documentation translation

First, get the sources of branch translation from the Git repository, see Starting with Git.

Translation requirements

Working on LilyPond documentation translations requires the following pieces of software, in order to make use of dedicated helper tools:

• Python 3.5 or higher,
• GNU Make,
• Gettext,
• Git.

It is not required to build LilyPond and the documentation to translate the documentation. However, if you have enough time and motivation and a suitable system, it can be very useful to build at least the documentation so that you can check the output yourself and more quickly; if you are interested, see Compiling.

Before undertaking any large translation work, contributors are encouraged to contact the Translation Meister.

Which documentation can be translated

The makefiles and scripts infrastructure currently supports translation of the following documentation:

• the web site, the Learning Manual, the Notation Reference and Application Usage – Texinfo source, PDF and HTML output; Info output might be added if there is enough demand for it;
• the Changes document.

Support for translating the following pieces of documentation should be added soon, by decreasing order of priority:

• automatically generated documentation: markup commands, predefined music functions;
• the Snippets List;
• the Internals Reference.

Starting translation in a new language

At top of the source directory, do

./autogen.sh

or (if you want to install your self-compiled LilyPond locally)

./autogen.sh --prefix=$HOME

If you want to compile LilyPond – which is almost required to build the documentation, but is not required to do translation only – fix all dependencies and rerun ./configure (with the same options as for autogen.sh).

Then cd into ‘Documentation/’ and run

make ISOLANG=MY-LANGUAGE new-lang

where MY-LANGUAGE is the ISO 639 language code.

Finally, add a language definition for your language in ‘python/langdefs.py’, ‘Documentation/lilypond-texi2html-lang.init’ and ‘Documentation/web/server/lilypond.org.htaccess’. Add this language definition and the corresponding section in ‘Documentation/lilypond-texi2html.init’ and ‘scripts/build/create-weblinks-itexi.py’.

5.9.2 Documentation translation details

Please follow all the instructions with care to ensure quality work.

All files should be encoded in UTF-8.

Files to be translated

Translation of ‘Documentation/foo/bar’ should be ‘Documentation/LANG/foo/bar’. Unmentioned files should not be translated.

Priorities:

• 1. delivery,
• 2. 3. 4. 5. 6. later,
• 7. optional.

Files of priority 1 should be submitted along all files generated by starting a new language in the same commit and thus a unique patch, and the translation of files marked with priority 2 should be committed to Git at the same time and thus sent in a single patch. Files marked with priority 3 or more may be submitted individually. Word counts (excluding LilyPond snippets) are given for each file. For knowing how to commit your work to Git, then make patches of your new translations as well as corrections and updates, see Basic Git procedures.

-1- Web site
752   web.texi
6073  web/introduction.itexi
1174  web/download.itexi
1139  macros.itexi
9     po/lilypond-doc.pot (translate to po/MY_LANGUAGE.po)
0     search-box.ihtml
---   lilypond-texi2html.init (section TRANSLATIONS)
9147  total

-2- Tutorial
1279  web/manuals.itexi
124   learning.tely
2536  learning/tutorial.itely
4517  learning/common-notation.itely
8456  total

-3- Fundamental Concepts, starting of Usage and Community
11576 learning/fundamental.itely -- Fundamental concepts
135   usage.tely
6877  usage/running.itely
2097  usage/updating.itely
2359  web/community.itexi
23044 total

-4- Rest of Learning manual and Suggestions on writing LilyPond files
16582 learning/tweaks.itely -- Tweaking output
1236  learning/templates.itely -- Templates
2793  usage/suggestions.itely -- Suggestions on writing LilyPond files
20611 total

-5- Notation reference
361   notation.tely
91    notation/notation.itely -- Musical notation
5513  notation/pitches.itely
7098  notation/rhythms.itely
1907  notation/expressive.itely
1310  notation/repeats.itely
3001  notation/simultaneous.itely
3017  notation/staff.itely
1700  notation/editorial.itely
3572  notation/text.itely
81    notation/specialist.itely -- Specialist notation
5174  notation/vocal.itely
2177  notation/chords.itely
702   notation/piano.itely
1000  notation/percussion.itely
826   notation/guitar.itely
66    notation/strings.itely
242   notation/bagpipes.itely
5468  notation/ancient.itely
13941 notation/input.itely -- Input syntax
2164  notation/non-music.itely -- Non-musical notation
11050 notation/spacing.itely -- Spacing issues
17137 notation/changing-defaults.itely -- Changing defaults
5187  notation/programming-interface.itely -- Interfaces for programmers
3450  notation/notation-appendices.itely -- Notation manual tables
252   notation/cheatsheet.itely -- Cheat sheet
96487 total

-6- Rest of Application Usage
4212  usage/lilypond-book.itely -- LilyPond-book
1122  usage/converters.itely -- Converting from other formats
5334  total

-7- Appendices whose translation is optional
382   essay/literature.itely
1222  learning/scheme-tutorial.itely (should be revised first)
1604  total

In addition, not listed above, Snippets’ titles and descriptions should be translated; they are a part of the Notation Reference and therefore their priority is 5.

Translating the Web site and other Texinfo documentation

Every piece of text should be translated in the source file, except Texinfo comments, text in @lilypond blocks and a few cases mentioned below.

Node names are translated, but the original node name in English should be kept as the argument of @translationof put after the section title; that is, every piece in the original file like

@node Foo bar
@section_command Bar baz

should be translated as

@node translation of Foo bar
@section_command translation of Bar baz
@translationof Foo bar

The argument of @rglos commands and the first argument of @rglosnamed commands must not be translated, as it is the node name of an entry in Music Glossary.

Every time you translate a node name in a cross-reference, i.e. the argument of commands @ref, @rprogram, @rlearning, @rlsr, @ruser or the first argument of their *named variants, you should make sure the target node is defined in the correct source file; if you do not intend to translate the target node right now, you should at least write the node definition (that is, the @node @section_commmand @translationof trio mentioned above) in the expected source file and define all its parent nodes; for each node you have defined this way but have not translated, insert a line that contains @untranslated. That is, you should end up for each untranslated node with something like

@node translation of Foo bar
@section_command translation of Bar baz
@translationof Foo bar

@untranslated

Please think of the fact that it may not make sense translating everything in some Texinfo files, and you may take distance from the original text; for instance, in the translation of the web site section Community, you may take this into account depending on what you know the community in your language is willing to support, which is possible only if you personally assume this support, or there exists a public forum or mailing list listed in Community for LilyPond in your language:

• (web)Bug reports section ‘Bug reports’ in General Information: this page should be translated only if you know that every bug report sent on your language’s mailing list or forum will be handled by someone who will translate it to English and send it on bug-lilypond or add an issue in the tracker, then translate back the reply from developers.
• (contributor)Help us section ‘Help us’ in Contributor’s Guide: this page should be translated very freely, and possibly not at all: ask help for contributing to LilyPond for tasks that LilyPond community in your language is able and going to handle.

In any case, please mark in your work the sections which do not result from the direct translation of a piece of English translation, using comments i.e. lines starting with ‘@c’.

Finally, press in Emacs <C-c C-u C-a> to update or generate menus. This process should be made easier in the future, when the helper script texi-langutils.py and the makefile target are updated.

Some pieces of text manipulated by build scripts that appear in the output are translated in a ‘.po’ file – just like LilyPond output messages – in ‘Documentation/po’. The Gettext domain is named lilypond-doc, and unlike lilypond domain it is not managed through the Free Translation Project.

Take care of using typographic rules for your language, especially in ‘macros.itexi’.

If you wonder whether a word, phrase or larger piece of text should be translated, whether it is an argument of a Texinfo command or a small piece sandwiched between two Texinfo commands, try to track whether and where it appears in PDF and/or HTML output as visible text. This piece of advice is especially useful for translating ‘macros.itexi’.

Please keep verbatim copies of music snippets (in @lilypond blocs). However, some music snippets containing text that shows in the rendered music, and sometimes translating this text really helps the user to understand the documentation; in this case, and only in this case, you may as an exception translate text in the music snippet, and then you must add a line immediately before the @lilypond block, starting with

@c KEEP LY

Otherwise the music snippet would be reset to the same content as the English version at next make snippet-update run – see Updating documentation translation.

When you encounter

@lilypondfile[<number of fragment options>,texidoc]{filename.ly}

in the source, open ‘Documentation/snippets/filename.ly’, translate the texidoc header field it contains, enclose it with texidocMY-LANGUAGE = " and ", and write it into ‘Documentation/MY-LANGUAGE/texidocs/filename.texidoc’. Additionally, you may translate the snippet’s title in doctitle header field, in case doctitle is a fragment option used in @lilypondfile; you can do this exactly the same way as texidoc. For instance, ‘Documentation/MY-LANGUAGE/texidocs/filename.texidoc’ may contain

doctitlees = "Spanish title baz"
texidoces = "
Spanish translation blah
"

@example blocks need not be verbatim copies, e.g. variable names, file names and comments should be translated.

Finally, please carefully apply every rule exposed in Texinfo introduction and usage policy, and Documentation policy. If one of these rules conflicts with a rule specific to your language, please ask the Translation meister on translations@lilynet.net list and/or the Documentation Editors on lilypond-devel@gnu.org list.

Adding a Texinfo manual

In order to start translating a new manual, simply copy the English files within your language directory and translate them.

For example, if you want to translate the first chapter of the Learning Manual:

cp Documentation/learning.tely Documentation/LANG/learning.tely
cp Documentation/learning/tutorial.itely Documentation/LANG/tutorial.itely

5.9.3 Documentation translation maintenance

Several tools have been developed to make translations maintenance easier. These helper scripts make use of the power of Git, the version control system used for LilyPond development.

You should use them whenever you would like to update the translation in your language, which you may do at the frequency that fits your and your cotranslators’ respective available times. In the case your translation is up-do-date (which you can discover in the first subsection below), it is enough to check its state every one or two weeks. If you feel overwhelmed by the quantity of documentation to be updated, see Maintaining without updating translations.

Check state of translation

First pull from Git – see Pulling and rebasing, but DO NOT rebase unless you are sure to master the translation state checking and updating system – then cd into ‘Documentation/’ (or at top of the source tree, replace make with make -C Documentation) and run

make ISOLANG=MY_LANGUAGE check-translation

This presents a diff of the original files since the most recent revision of the translation. To check a single file, cd into ‘Documentation/’ and run

make CHECKED_FILES=MY_LANGUAGE/manual/foo.itely check-translation

In case this file has been renamed since you last updated the translation, you should specify both old and new file names, e.g. CHECKED_FILES=MY_LANGUAGE/{manual,user}/foo.itely.

To see only which files need to be updated, do

make ISOLANG=MY_LANGUAGE check-translation | grep 'diff --git'

To avoid printing terminal colors control characters, which is often desirable when you redirect output to a file, run

make ISOLANG=MY_LANGUAGE NO_COLOR=1 check-translation

You can see the diffs generated by the commands above as changes that you should make in your language to the existing translation, in order to make your translation up to date.

Global state of the translation is recorded in ‘Documentation/translations.itexi’, which is used to generate Translations status page. To update that page, do from ‘Documentation/’

make translation-status

This will also leave ‘out/translations-status.txt’, which contains up-to-dateness percentages for each translated file, and update word counts of documentation files in this Guide.

See also

Maintaining without updating translations.

Updating documentation translation

Instead of running check-translation, you may want to run update-translation, which will run your favorite text editor to update files. First, make sure environment variable EDITOR is set to a text editor command, then run from ‘Documentation/’

make ISOLANG=MY_LANGUAGE update-translation

or to update a single file

make CHECKED_FILES=MY_LANGUAGE/manual/foo.itely update-translation

For each file to be updated, update-translation will open your text editor with this file and a diff of the file in English; if the diff cannot be generated or is bigger than the file in English itself, the full file in English will be opened instead.

‘.po’ message catalogs in ‘Documentation/po/’ may be updated by issuing from ‘Documentation/’ or ‘Documentation/po/’

make po-update

Updating music snippets can quickly become cumbersome, as most snippets should be identical in all languages. Fortunately, there is a script that can do this odd job for you (run from ‘Documentation/’):

make ISOLANG=MY_LANGUAGE snippet-update

This script overwrites music snippets in ‘MY_LANGUAGE/foo/every.itely’ with music snippets from ‘foo/every.itely’. It ignores skeleton files, and keeps intact music snippets preceded with a line starting with @c KEEP LY; it reports an error for each ‘.itely’ that has not the same music snippet count in both languages. Always use this script with a lot of care, i.e. run it on a clean Git working tree, and check the changes it made with git diff before committing; if you don’t do so, some @lilypond snippets might be broken or make no sense in their context.

See also

Maintaining without updating translations, Adding and editing snippets.

Updating translation committishes

At the beginning of each translated file except PO files, there is a committish which represents the revision of the sources which you have used to translate this file from the file in English.

When you have pulled and updated a translation, it is very important to update this committish in the files you have completely updated (and only these); to do this, first commit possible changes to any documentation in English which you are sure to have done in your translation as well, then replace in the up-to-date translated files the old committish by the committish of latest commit, which can be obtained by doing

git rev-list HEAD |head -1

Most of the changes in the LSR snippets included in the documentation concern the syntax, not the description inside texidoc="". This implies that quite often you will have to update only the committish of the matching .texidoc file. This can be a tedious work if there are many snippets to be marked as up do date. You can use the following command to update the committishes at once:

cd Documentation/LANG/texidocs
sed -i -r 's/[0-9a-z]{40}/NEW-COMMITTISH/' *.texidoc

See also

LSR work.

5.9.4 Translations management policies

These policies show the general intent of how the translations should be managed, they aim at helping translators, developers and coordinators work efficiently.

Maintaining without updating translations

Keeping translations up to date under heavy changes in the documentation in English may be almost impossible, especially as during the former Grand Documentation Project (GDP) or the Grand Organization Project (GOP) when a lot of contributors brings changes. In addition, translators may be — and that is a very good thing — involved in these projects too.

it is possible — and even recommended — to perform some maintenance that keeps translated documentation usable and eases future translation updating. The rationale below the tasks list motivates this plan.

The following tasks are listed in decreasing priority order.

1. Update macros.itexi. For each obsolete macro definition, if it is possible to update macro usage in documentation with an automatic text or regexp substitution, do it and delete the macro definition from ‘macros.itexi’; otherwise, mark this macro definition as obsolete with a comment, and keep it in ‘macros.itexi’ until the documentation translation has been updated and no longer uses this macro.
2. Update ‘*.tely’ files completely with make check-translation – you may want to redirect output to a file because of overwhelming output, or call check-translation.py on individual files, see Check state of translation.
3. In ‘.itelys’, match sections and .itely file names with those from English docs, which possibly involves moving nodes contents in block between files, without updating contents itself. In other words, the game is catching where has gone each section. In Learning manual, and in Notation Reference sections which have been revised in GDP, there may be completely new sections: in this case, copy @node and @section-command from English docs, and add the marker for untranslated status @untranslated on a single line. Note that it is not possible to exactly match subsections or subsubsections of documentation in English, when contents has been deeply revised; in this case, keep obsolete (sub)subsections in the translation, marking them with a line @c obsolete just before the node.

   Emacs with Texinfo mode makes this step easier:

   • without Emacs AucTeX installed, <C-c C-s> shows structure of current Texinfo file in a new buffer *Occur*; to show structure of two files simultaneously, first split Emacs window in 4 tiles (with <C-x 1> and <C-x 2>), press <C-c C-s> to show structure of one file (e.g. the translated file), copy *Occur* contents into *Scratch*, then press <C-c C-s> for the other file.

     If you happen to have installed AucTeX, you can either call the macro by doing <M-x texinfo-show-structure> or create a key binding in your ‘~/.emacs’, by adding the four following lines:

     (add-hook 'Texinfo-mode-hook '(lambda () (define-key Texinfo-mode-map "\C-cs" 'texinfo-show-structure)))

     and then obtain the structure in the *Occur* buffer with <C-c s>.

   • Do not bother updating @menus when all menu entries are in the same file, just do <C-c C-u C-a> (“update all menus”) when you have updated all the rest of the file.
   • Moving to next or previous node using incremental search: press <C-s> and type node (or <C-s @node> if the text contains the word ‘node’) then press <C-s> to move to next node or <C-r> to move to previous node. Similar operation can be used to move to the next/previous section. Note that every cursor move exits incremental search, and hitting <C-s> twice starts incremental search with the text entered in previous incremental search.
   • Moving a whole node (or even a sequence of nodes): jump to beginning of the node (quit incremental search by pressing an arrow), press <C-SPACE>, press <C-s node> and repeat <C-s> until you have selected enough text, cut it with <C-w> or <C-x>, jump to the right place (moving between nodes with the previous hint is often useful) and paste with <C-y> or <C-v>.
4. Update sections finished in the English documentation; check sections status at

   https://lilypondwiki.tuxfamily.org/index.php?title=Documentation_coordination.

5. Update documentation PO. It is recommended not to update strings which come from documentation that is currently deeply revised in English, to avoid doing the work more than once.
6. Fix broken cross-references by running (from ‘Documentation/’)

   make ISOLANG=YOUR-LANGUAGE fix-xrefs

   This step requires a successful documentation build (with make doc). Some cross-references are broken because they point to a node that exists in the documentation in English, which has not been added to the translation; in this case, do not fix the cross-reference but keep it "broken", so that the resulting HTML link will point to an existing page of documentation in English.

Rationale

You may wonder if it would not be better to leave translations as-is until you can really start updating translations. There are several reasons to do these maintenance tasks right now.

• This will have to be done sooner or later anyway, before updating translation of documentation contents, and this can already be done without needing to be redone later, as sections of documentation in English are mostly revised once. However, note that not all documentation sectioning has been revised in one go, so all this maintenance plan has to be repeated whenever a big reorganization is made.
• This just makes translated documentation take advantage of the new organization, which is better than the old one.
• Moving and renaming sections to match sectioning of documentation in English simplify future updating work: it allows updating the translation by side-by-side comparison, without bothering whether cross-reference names already exist in the translation.
• Each maintenance task except ‘Updating PO files’ can be done by the same person for all languages, which saves overall time spent by translators to achieve this task: the node names and section titles are in English, so you can do. It is important to take advantage of this now, as it will be more complicated (but still possible) to do step 3 in all languages when documentation is compiled with texi2html and node names are directly translated in source files.

Managing documentation translation with Git

This policy explains how to manage Git branches and commit translations to Git.

• Translation work is made on translation branch. This branch is merged on master once a week, approximately. Then, master branch is merged on translation, where the check-translation script (see Check state of translation) shows changes in English docs which should be translated, and the cycle starts again.
• Translations may be merged directly to master only if they do not break compilation of LilyPond and its documentation. Those changes could be pushed to translation too, or alternatively translators could wait until they come from master the next time it is merged on translation. Similarly, changes matching stable/X.Y are preferably made on X.Ytranslation.
• translation Git branch may be merged into master branch only if LilyPond (make all) and documentation (make doc) compile successfully.
• make and make doc are usually successful in master Git branch because those tests should have already succeeded when merging the contained commits. master branch may be merged into translation when significant changes had been made in documentation in English in master branch.
• General maintenance may be done by anybody who knows what he does in documentation in all languages, without informing translators first. General maintenance include simple text substitutions (e.g. automated by sed), compilation fixes, updating Texinfo or lilypond-book commands, updating macros, updating ly code, fixing cross-references, and operations described in Maintaining without updating translations.

5.9.5 Technical background

A number of Python scripts handle a part of the documentation translation process. All scripts used to maintain the translations are located in ‘scripts/auxiliar/’.

• ‘check_translation.py’ – show diff to update a translation,
• ‘texi-langutils.py’ – quickly and dirtily parse Texinfo files to make message catalogs and Texinfo skeleton files,
• ‘update-snippets.py’ – synchronize ly snippets with those from English docs,
• ‘translations-status.py’ – update translations status pages and word counts in the file you are reading,
• ‘tely-gettext.py’ – gettext node names, section titles and references in the sources; WARNING only use this script once for each file, when support for "makeinfo –html" has been dropped.

Other scripts are used in the build process, in ‘scripts/build/’:

• ‘mass-link.py’ – link or symlink files between English documentation and documentation in other languages.

Python modules used by scripts in ‘scripts/auxiliar/’ or ‘scripts/build/’ (but not by installed Python scripts) are located in ‘python/auxiliar/’:

• ‘manuals_definitions.py’ – define manual names and name of cross-reference Texinfo macros,
• ‘buildlib.py’ – common functions (read piped output of a shell command, use Git),
• ‘postprocess_html.py’ (module imported by ‘www_post.py’) – add footer and tweak links in HTML pages.

And finally

• ‘python/langdefs.py’ – language definitions module

This document was generated by gub on July 13, 2020 using texi2html 1.82.