Version 115 of msgcat

Updated 2013-05-30 12:04:10 by oehhar

Purpose: information relating to the message catalog interface.


Documentation can be found at http://www.purl.org/tcl/home/man/tcl8.5/TclCmd/msgcat.htm


The msgcat package (standard with Tcl since 8.1) addresses the problem of hard coded application text.

Instead of

 label .x -text Files

if you write

 label .x -text [mc Files]

then you can create catalog entries for various languages for the term "Files". Also, mc will apply formatting to the string, so

 puts [mc "Directory contains %d files" $nfiles]

will allow a translator to work with the string "Directory contains %d files".

The above example assumes you have once invoked

 package require msgcat ;# and done 
 namespace import msgcat::* ;# (or msgcat::mc).

Provided you have .msg files containing

 ::msgcat::mcset de Files Dateien
 ::msgcat::mcset fr Files Fichiers

(notice that the rootname part has the naming convention that it shall be the contained language - for e.g. locale "en_us_funky", msgcat searches the contents of first en_us_funky.msg, then en_us.msg, then en.msg) then either automagically (when env(LANG) is accordingly set) or by

 ::msgcat::mclocale fr

.x will have the text "Fichiers".

If you want to package your messages into the application file, you can as well do this with mcset. See Multilingual menu and A little stopwatch for examples.

Note that msgcat 1.3 (comes with Tcl 8.4) also initializes the locale from ::env(LC_ALL) or ::env(LC_MESSAGES).

LES: I don't have ::env(LC_ALL) and ::env(LC_MESSAGES) on Slackware 11.


Arjen Markus This came up in the newsgroup:

The utility frink is able to adjust the source code so that the new code uses the msgcat package.


The 8.3.4 release of the PPC binary is broken in that it lacks msgcat and bgerror implementations. Melissa Schrumpf explains the situation with her customary clarity

    ... bgerror and msgcat are contained in tcl/library/msgcat/msgcat.tcl.

    Do this:

    Copy "Simply Tk (PPC)" to "Simply Tk (PPC).edit"

    Run ResEdit (or Resourcer, or whathaveyou).  Open a "Simply Tk
    (PPC).edit" in ResEdit.  Open the "TEXT" resources.  Open resource ID
       "package require msgcat"

    This will be right at the top.

    Next, open tcl/library/msgcat/msgcat.tcl in a text editor.  Copy from
    the beginning of the line:

       package provide msgcat 1.1.1  




    through to the end of the file.

    Return to ResEdit.  Delete the line "package require msgcat" and,
    in its place, paste the code you copied from msgcat.tcl.
    Close all resources and save.

    You now have a fixed stand-alone Tk shell.      

Anton Kovalenko has written, "Also, there is a small but useful script, which converts non-ascii characters from current system encoding to \uXXXX sequences. I think it would be useful to anyone who works often with tcl message catalogs. http://kovalenko.webzone.ru/unicodize.tcl (A/AK: chat.ru is down, so my page migrated) "

LES: "Forbidden. You don't have permission to access /unicodize.tcl on this server."


A/AK: there are some things about msgcat and shortcut keys, that are important for GUI application writers. EKB And this seems to have been continued into ampersand magic.


KHM Any Idea where to download this tool?

LV KHM - msgcat is a part of Tcl itself. Or are you talking about some other tool?


GNU gettext has support (since V 0.13 [L1 ]) for Tcl msgcat and conversion tools from and to PO format. http://ftp.gnu.org/gnu/gettext/


Simple Tk example of switching languages, complements of dgp:

 package require Tk

 bgerror 123

 msgcat::mclocale "fr"
 msgcat::mcload [file join $::tk_library msgs]
 bgerror 123
 
 msgcat::mclocale "en_gb"
 msgcat::mcload [file join $::tk_library msgs]
 bgerror 123

Now, how can I determine what languages are available for mclocale?


escargo 8 Dec 2003 - What if you want to do some form of parameter substitution in the strings that you might use with msgcat? In some systems I have developed, messages were somewhat macro-like; you might pass a number or a name (of a file, for example) that should be part of the message. Is there anything in the msgcat that does the substitution internally, or do I have to add my own layer on top of it?

RS: Easy, for instance with format:

 msgcat::mcset fr "Directory contains %d files" "Il y a %d fichiers"
 ...
 puts [format [mc "Directory contains %d files"] $nfiles]

DGP Even easier. There's a format already built-in to mc:

 puts [mc "Directory contains %d files" $nfiles]

EKB I moved this example up to the top of the page and combined it with the intro text so it's easier to find.

AM Also very useful: the %1$s type format codes - with this you can interchange variables if needed.

  % mc "this is a %s test for %s" first anything
  this is a first test for anything
 
  % mc {this is a %2$s test for %1$s} first anything
  this is a anything test for first

Note the {} around the text to prevent the $ to be interpreted and looking for the variable $s.


RS 2007-01-08: Here's a simple self-contained example for testing:

 #!/usr/bin/env tclsh
 package require msgcat

 msgcat::mcset de yes Ja
 msgcat::mcset de no  Nein
 msgcat::mcset fr yes Oui
 msgcat::mcset fr no  Non

 proc localbool value {
    if {$value} {msgcat::mc yes} else {msgcat::mc no}
 }
 puts [localbool 0]/[localbool 1]
That's all. Save that to a file, and run it with different LANG settings:
 $ /Tcl/msgcat.tcl
 Nein/Ja
 $ LANG=fr /Tcl/msgcat.tcl
 Non/Oui
 $ LANG=en /Tcl/msgcat.tcl
 no/yes


  Dynamic mode - change locale on run-time (TIP399)

HaO 2012-07-17: TIP 399 contains a patch to use msgcat in an environment which dynamically changes languages.

A file msgcat-1.5.0.tm is available in the linked feature request #3511941 . This file may be put to the tcl lib to be tested. In my experience, it is not sufficient, to have the file within the own project. It must be copied to the tcl installation beside the present file msgcat-1.4.4.tm.

  • At this place, there were an alternate implementation and some use instructions. These are superseeded by the upper information.
  • This feature is also included in the download files of #3544988 . There is also a tclapp tap-file available as download.

HaO 2012-08-27: Two things happened:

  • AK has asked how other packages may ask for information about a locale change (see below Dynamic namespace change callback).
  • TIP 399 was voted yes. DGP voted present with the following comment:
        What I would prefer is to vote YES for the "turn the whole
        package over to Harald Oehlmann to do as he pleases".

        The proposal seems fine, to address the problem posed.
        What gives me pause is that the TIP seems to demonstrate that
        the original design of msgcat just isn't any good.  At some
        point we'd be better off not slapping in more bells and whistles
        and alternatives and just creating a new (version of a?) package
        that learns the lessons and gets it right this time.

Complete solution

I came to the conclusion, that dynamic local change is wanted but the proposed solution only solves parts of the issue. Here is an extract of my message on the core-list:


Unfortunately, the solution within the tip is half-baken and I ask for some time to design a better solution and rewrite a tip.

I am sorry for that but I am convinced, this is the best way to go. I might be wrong, but at least I will try.

A good solution for TIP 399 must automatically reload locale files on a locale change.

Sketch of my current view of a solution

There is a new command to set per caller namespace config data (namespace equal package, as each package should have one namespace and one set of locale files):

msgcat::mcpackageconfig ?-namespace namespace? setting ?value?

Settings per caller namespace/package:

  • message file folder (also auto-set by msgcat::mcload) (to be able to load missing locale files on a locale change)
  • callback function before reload of the message file due to a locale change (change to a jet unloaded locale)
  • callback function after a locale change
  • flag, if locale files should be reloaded if needed

All of them are get and setable. The function returns with an error, if the namespace does not exist.

The advantages of this:

  • the concept of the caller/package namespace already exists within msgcat. This is continued.
  • the caller namespace may be autodetected or specified
  • the settings might automatically be cleared if the namespace is removed and the corresponding function may be automatically disabled
  • IMHO very transparent

There are the following additional configuration settings:

  • currently loaded locales (sum of all mcpreferences, readonly)
  • Flag, if unneeded locale data is cleared on locale change

  Dynamic namespace change callback (TIP399 related)

HaO 2012-08-22: AK asked by private mail:

Do we have a <<LocaleChanged>> virtual event like we have <<ThemeChanged>>, which is used to inform all widgets when 'mclocale' was switched at runtime ?

HaO answered:

No, we dont have this currently.

As it is a tcl-only package, we do not have the bind command.

Thus we could register a command which is called when the locale changes:

msgcat::mcconfig -command ?lcmd?
  • lcmd : list of cmd ?namespace?
  • cmd: Command to call when locale changes. Specify empty string to unregister.
  • namespace: the callers module namespace which is only used as a register id to store the command. If not specified, the callers current namespace is used. There is one registered command possible per namespace value.

When the command fails, bgerror is called. An eventual locale change command (ex: msgcat::mclocale en_us), which executes the commands does not return with an error if a locale change callback fails.

Example:

package provide mymodule 1.0
namespace eval mymodule {
    # Register a locale change callback
    msgcat::mcconfig -command [list myLocaleChanged]
    ...
    # Unregister the locale change callback
    msgcat::mcconfig -command ""
}

AK 2012-08-22 by E-Mail:

> As it is a tcl-only package, we do not have the bind command.

True. Nor the 'event generate'.

I had thought about auto-detecting a Tk environment to restrict when it broadcasts a change, however your idea with a configurable callback is better, from an architectural point of view.

Hm. ... Hadn't thought about multiple callbacks. Guess because I was thinking in terms of events, where I can bind multiple things as I see fit.

This restriction to one-per-namespace, and providing it through a list ... I do not like that too much. It is workable, true. Still don't like it. This shoehorning multiple callbacks through a single config option feels wrong.

For multiple callbacks I would use dedicated (un)register commands, and tokens. I.e.

msgcat::register <cmdprefix>

returns a token, which is then the argument to

msgcat::unregister <token>

to identify the callbacks and remove them individually.

See http://docs.activestate.com/activetcl/8.5/tcllib/uev/uevent.html for an example of how I did that for a tcl event package.

WHD 2012-08-22 by E-Mail:

See also the hook module in Tcllib.

HaO 2012-08-27: Discussion continues in the upper point Dynamic mode - change locale on run-time. There I write, why I prefer namespace instead of tolkens.


MG 2012-08-22: I use something similar to this in his app:

proc setLocale {new_locale} {
  variable locales;
  variable current_locale;

  set split [split $new_locale "_"]
  set preflist [list]
  for {set i 0} {$i < [llength $split]} {incr i} {
    lappend preflist [join [lrange $split 0 end-$i] "_"]
  }

  set current_locale ""
  foreach x $preflist {
    if { [info exists locales($x)] } {
         set current_locale $x
         break;
       }
  }

  if { ![info exists current_locale] || $current_locale eq "" } {
       set current_locale "en_gb"
     }

  # Set our best available
  ::msgcat::mclocale $current_locale

  # Update display to make sure we're using it
  if { $skin ne "" } {
       ::skin::${skin}::locale
     }
  setUpMenu
  setAppTitle

  return;

};# setLocale

The $locales array records exactly which locales I've loaded some form of translation for, since msgcat doesn't seem to expose that information. The code makes sure that there actually is a translation for the requested locale, and picks a less specific one if necessary in much the same way msgcat does. It falls back on en_gb as a default. Like msgcat, all the locale names are stored entirely in lowercase (en_gb not en_GB). The code after the last comment calls various procs which alter existing GUI elements to use the new locale, but could easily be replaced with something like

proc rchildren {widget} {
  event generate $widget <<LocaleChanged>>
  foreach x [winfo children $widget] {
    rchildren $x
  }
}
rchildren .

so you could bind to <<LocaleChanged>> for the assorted widgets instead.

HaO 2012-08-27: Thank you, Mike, for the contribution. The language change is user level and should still work as you do it, but this is slightly off-topic. Andreas is asking, how non-user packages which store translated messages somehow (like the tcllib tooltip package) may by their action be informed about a locale change.

  New TIP#412: Dynamic Locale Changing for msgcat with On-Demand File Load

HaO 2012-10-15: The new TIP and draft implementation is the current step in the dynamic locale msgcat feature. It is ready for public discussion:

Anybody is invited to contribute!


  Add notion of MC file locale to simplify mc files (TIP404)

HaO 2012-10-15: This feature is now included in TCL8.6b3 (docs) and is backported to TCL8.5 to be included in TCL8.5.13.

It is now recommended to use:

msgcat::mcflset original translation

instead of

msgcat::mcset locale original translation

within message files which are loaded by msgcat::mcload.


  Use tags instead of text

HaO 2012-07-17: Given the code with a localised error message:

if {[catch {open $File r} Err]} {
    puts stderr [mc "File error accessing '%s': %s" $File $Err]
}

and the German translation:

--de.msg--
msgcat::mcset de "File error accessing '%s': %s" "Zugriffsfehler Datei '%s': %s"
--eof--

I use tags instead of a default translation. The default (english) text is contained in the root translation:

--de.msg--
msgcat::mcset de errFile "Zugriffsfehler Datei '%s': %s"
--eof--
--ROOT.msg--
msgcat::mcset {} errFile "File error accessing '%s': %s"
--eof--
if {[catch {open $File r} Err]} {
    puts stderr [mc errFile $File $Err]
}

Why that ?

  • If I change the default text, I do not have to change the translation keys
  • It feels more systematic to me to separate text and code
  • If a text is the same in english but different in german, I may not make any difference
  • If I forget a translation, I get only a stupid tag. Nothing for lazy guys...

In this case, it is helpful to redefine 'mcunknown' to also return the given parameters:

proc ::msgcat::mcunknown {locale args} {
        return $args
}

Default mcunknown:

% msgcat::mc tag1 p1 p2
tag1

Custom mcunknown:

% msgcat::mc tag1 p1 p2
tag1 p1 p2


  Exchange package msgcat by a custom version

HaO 2012-07-24: Tk loads msgcat on startup. The clock command also loads msgcat. An alternate msgcat version must also be in the default path, as tk or the clock package may be loaded before any script is able to extend the auto_path.

Thus the new msgcat.tcl is renamed to msgcat-1.x.x.tm (1.x.x: version greater 1.4.5 which is the version in tcl 8.5.12) and saved to:

  • <tcl install folder>/lib/lib8/8.5

If the program is wrapped to an executable starkit (not .kit), one may save it to:

  • <programname>.vfs/lib/lib8/8.5

If an application is wrapped using ActiveState TclApp, I did not find any way to wrap anything to the upper path. But I made a custom tap file containing a pckIndex.tcl and a msgcat.tcl and this worked (at least for a console application).

As an alternative, an already loaded msgcat package may be reloaded by:

if {1 != [catch {package present msgcat} ver] && [package vsatisfies $ver 1.6-] } {
    package forget msgcat
    namespace delete ::msgcat
    package require msgcat 1.6-
    # if tk is used reload the tk msgcat setup
    msgcat::mcload [file join $::tk_library msgs]
}


  Using msgcat with TclOO

Twylite 2013-03-01: msgcat searches for messages in the current namespace, then parent namespaces up to the global namespace. To look up a message in a different (non-parent) namespace you must use namespace eval or an equivalent.

TclOO class names are distinct from the unique namespace dynamically allocated to the class on construction. The class name may or may not correspond to the name of a namespace. To support file-based message catalogs (*.msg) we would need to (i) have the messages in a namespace; and (ii) have a static pre-determined name for the namespace.

Twylite 2013-03-12: Working towards a TIP, as requested (see below). Following feedback from DKF I think that it is conceptually reasonable to search for messages in an object's inheritance hierarchy, and to avoid using namespace search at all when dealing with objects. To keep the interface clean there should be new msgcat::oo::* procs to indicate that we are working with a message in the inheritance hierarchy, rather than turning msgcat::mc into a DWIM.

The solution below is a minimal embodiment of the interface, and does not yet support message inheritance. Use oo::define $cls mcset ... to define a message on a class, or msgcat::oo::mcset $cls ... to do the same in a .msg file. Use mymc ... in a method declared on a class to retrieve a message defined on that class. mymc should support inheritance and polymorphism in future, but for now you can use msgcat::oo::mc $otherclass ... to explicitly refer to a message in a superclass.

For the original solution see the page History. Description of the original solution: The solution I propose is to put messages for a class ::$ns::$cls into the namespace ::msgs::$ns::$cls, and to provide helpers oo::define $cls mcset and ::msgcat::classmc to respectively set and retrieve catalog messages. As an alternative to ::msgcat::classmc a class method mc is also provided.

# See http://wiki.tcl.tk/21595 for oo::DefWhat
proc ::oo::DefWhat {} { uplevel 3 [list namespace which [lindex [info level -2] 1]] }

namespace eval ::msgcat::oo {}

proc ::msgcat::oo::mcset {cls locale src {dest {}}} {
  namespace eval $cls [list ::msgcat::mcset $locale $src $dest] 
}

proc ::msgcat::oo::mc {cls src args} {
  namespace eval $cls [list ::msgcat::mc $src {*}$args] 
}

proc ::oo::define::mcset {args} {
  tailcall ::msgcat::oo::mcset [::oo::DefWhat] {*}$args
}

proc oo::Helpers::mymc {args} {
  tailcall ::msgcat::oo::mc [uplevel 1 { self class }] {*}$args
}

Using it:

package require compat

oo::class create ::Alpha 

oo::define Alpha mcset en FooMsg "this is the foo msg"
namespace eval ::Alpha {
  ::msgcat::mcset fr FooMsg "this is the french msg"
}

oo::define ::Alpha method testMsg1 {} {
  mymc FooMsg
}

oo::define ::Alpha method testMsg2 {} {
  msgcat::oo::mc [self class] FooMsg
}

set a [Alpha new]


oo::class create ::Beta {
  superclass ::Alpha
  mcset en BarMsg "this is the bar msg"
  method testMsg3 {} {
    mymc BarMsg
  }
}
set b [Beta new]

oo::class create ::Gamma {
  method testMsg4 {} {
    mymc FooMsg ;# no inheritance yet
  }
  method testMsg5 {} {
    msgcat::oo::mc ::Alpha FooMsg
  }
}
set g [Gamma new] 

::msgcat::mclocale en
$a testMsg1
$a testMsg2
$b testMsg1
$b testMsg2
$b testMsg3
$g testMsg4
$g testMsg5

::msgcat::mclocale fr
$a testMsg1
$a testMsg2
$b testMsg1
$b testMsg2
$b testMsg3
$g testMsg4
$g testMsg5

HaO Looks reasonable to me. Could you prepare a TIP?


  What to do when I am unsatified by the default language under windows

HaO 2013-05-10: It is relatively hard to debug language guessing, as the developpers depend on computers in any language which is seldomly the case. Thus I ask your help, if you think, the language guessing is not correct.

From Windows Vista on, the language system of windows passed from a numerical identifier to IETF language tags.

Start wish and type:

% msgcat::mclocale
de_de

On Windows Vista and later, you should mostly get a language and a country. If you only get the language, please check the following:

Verify, if one of the following environment variables are set: 'LC_ALL', 'LC_MESSAGES' or 'LANG'. This might be the fact due to an installed Cygwin:

% msgcat::mclocale
de
% parray env
...
LANG=de

In this case start wish without those environment variables set, for example by the following batch file:

set LANG=
c:\tcl\bin\wish86.exe

Now, the language settings are taken from the registry. If you still get bad results, please execute the following commands in a wish and post them with the correct language choice:

package require msgcat
msgcat::mclocale
package require registry
registry get {HKEY_CURRENT_USER\Control Panel\Desktop} PreferredUILanguages
registry get {HKEY_CURRENT_USER\Control Panel\International} LocaleName
registry get {HKEY_CURRENT_USER\Control Panel\International} locale

In the upcoming msgcat 1.5.2, those keys are checked in this order. The first is the IETF language tag and exists only, if language packs are installed. The second is the IETF language tag for default file language. The thiered is the numeric language ID as used up to Windows XP

Here is a result table:

mclocalePreferredUILanguagesLocaleNamelocaleSystem description and comments
de_devalue does not existde-DE00000407German Windows Vista 32 bit SP3
el_grel-GRel-GR00000408English Windows 7 Ultimate 64 bit, SP 1, with Greek UI language enabled