68 Locales ① — a program’s regional settings

68.1 Why it exists — conventions differ by country

When a program shows something to a person, there is not one “correct” way to write it.

Writing this out by hand in every program multiplies the code by the number of countries. So Unix and C took another road — make the conventions data, keep them outside the program, and let the program choose only which set to work with. That bundle of data is a locale.

68.2 A locale is process-global state

The nature of a locale in one line: one locale per process.

/* A locale is process-global state — what it starts as, and what changes it. */
#include <locale.h>
#include <stdio.h>
#include <stdlib.h>

/* setlocale returns "the name of the locale now in force".
   Pass NULL as the second argument to ask without changing anything. */
static void show_current(const char *when)
{
    printf("%s LC_ALL=%s\n", when, setlocale(LC_ALL, NULL));
}

int main(void)
{
    /* (1) A program always starts in the "C" locale (standard §7.11.1.1p4) */
    show_current("at startup:         ");

    /* (2) "" means "the locale the environment says".
           It reads LC_ALL, then LC_xxx, then LANG. */
    const char *applied = setlocale(LC_ALL, "");
    if (applied)
        printf("setlocale(LC_ALL,\"\"):  %s\n", applied);
    else
        puts("setlocale(LC_ALL,\"\") failed — the environment has no locale");

    /* (3) How many bytes one character may take depends on the locale.
           MB_CUR_MAX looks like a constant but reads *the current locale*. */
    printf("MB_CUR_MAX:             %zu\n", (size_t)MB_CUR_MAX);

    /* (4) If the requested locale is not on this machine, setlocale returns
           null. So always look at the return value — it fails quietly. */
    static const char *candidates[] = {
        "C", "C.UTF-8", "en_US.UTF-8", "ko_KR.UTF-8", "ko_KR.EUC-KR",
        "de_DE.UTF-8", "tr_TR.UTF-8", "ja_JP.UTF-8", "no_SUCH.locale",
    };

    puts("\nAvailable on this machine?");
    char saved[128];
    snprintf(saved, sizeof saved, "%s", setlocale(LC_ALL, NULL));

    for (size_t i = 0; i < sizeof candidates / sizeof *candidates; i++) {
        const char *got = setlocale(LC_ALL, candidates[i]);
        if (got) printf("  %-15s yes  MB_CUR_MAX=%zu\n",
                        candidates[i], (size_t)MB_CUR_MAX);
        else     printf("  %-15s no\n", candidates[i]);
    }

    /* (5) Put it back — hand the name string in again. But the pointer
           setlocale returned may be overwritten by the next call, so copy it. */
    setlocale(LC_ALL, saved);
    show_current("\nrestored:           ");

    /* (6) Categories can be set separately. This is the idiom in practice —
           what people see follows the environment, numbers a machine reads
           stay in "C". */
    setlocale(LC_ALL, "");
    setlocale(LC_NUMERIC, "C");
    printf("mixed — LC_CTYPE=%s, LC_NUMERIC=%s\n",
           setlocale(LC_CTYPE, NULL), setlocale(LC_NUMERIC, NULL));
    /* Asking LC_ALL lists every category when they are not all the same */
    printf("asking LC_ALL gives: %.60s...\n", setlocale(LC_ALL, NULL));
    return 0;
}

at startup:          LC_ALL=C
setlocale(LC_ALL,""):  C.UTF-8
MB_CUR_MAX:             6

Available on this machine?
  C               yes  MB_CUR_MAX=1
  C.UTF-8         yes  MB_CUR_MAX=6
  en_US.UTF-8     yes  MB_CUR_MAX=6
  ko_KR.UTF-8     yes  MB_CUR_MAX=6
  ko_KR.EUC-KR    yes  MB_CUR_MAX=2
  de_DE.UTF-8     yes  MB_CUR_MAX=6
  tr_TR.UTF-8     yes  MB_CUR_MAX=6
  ja_JP.UTF-8     yes  MB_CUR_MAX=6
  no_SUCH.locale  no

restored:            LC_ALL=C.UTF-8
mixed — LC_CTYPE=C.UTF-8, LC_NUMERIC=C
asking LC_ALL gives: LC_CTYPE=C.UTF-8;LC_NUMERIC=C;LC_TIME=C.UTF-8;LC_COLLATE=C.U...

The first line of the demonstration shows the standard’s rule in the flesh. Before anything happens, LC_ALL is C. And one setlocale(LC_ALL, "") moves it to whatever the environment says.

Two things follow from its being global.

First, it can change without you calling anything. If a library you linked calls setlocale, your printf("%f") is affected from that moment. GUI toolkits commonly do.

Second, it is a race between threads. C23 states this explicitly — a call to setlocale may introduce a data race with other setlocale calls or with functions affected by the locale (§7.11.1.1p5). If you want different locales in different threads, standard C has no road; POSIX’s uselocale family is needed (chapter 69).

68.3 The six categories — what governs what

A locale is not one lump; it divides into categories. The standard fixes six, and it also enumerates what each one affects.

You need not memorise the table, but one line is worth keeping: LC_NUMERIC governs printf. Nearly every case of this device corrupting data in practice comes from that line (chapter 69).

68.4 The exact contract of setlocale

One function, simple in shape, dense in contract.

char *setlocale(int category, const char *locale);

The return value splits two ways. On success it returns a string holding the name of the locale that was set (or is in force); on failure it returns null.

if (!setlocale(LC_ALL, "")) {
    fprintf(stderr, "warning: could not apply the locale; continuing in C.\n");
}

The returned pointer has a rule too. That string points into static storage that a later setlocale call may overwrite. If you intend to restore it later, copy it, as the demonstration does.

setlocale(LC_ALL, "");        /* follow the environment for everything, then */
setlocale(LC_NUMERIC, "C");   /* put numbers back into "C" */

68.5 The grammar of a locale name

ko_KR.UTF-8. Take the name apart and there are four international standards inside it.

The whole grammar can be written like this.

language[_TERRITORY][.codeset][@modifier]

ko_KR.UTF-8      Korean, Republic of Korea, UTF-8
de_DE@euro       German, Germany, the euro variant
sr_RS@latin      Serbian written in the Latin script
C   or  POSIX    the minimal locale the standard fixes

What matters is that the language and the territory are codes borrowed from other standards. ko is the code ISO 639-1 gave Korean and KR is the one ISO 3166-1 gave the Republic of Korea. A locale name is not something C invented; it is an assembly of code systems that already existed.

68.5.1 Codeset names and normalisation

.UTF-8, .utf8, .UTF8 — all three name the same thing. glibc compares names ignoring case and -. So even when locale -a prints ko_KR.utf8, a program may ask for "ko_KR.UTF-8".

The codeset names themselves come from yet another registry. UTF-8, EUC-KR and ISO-8859-1 are registered in IANA’s character-set registry, and behind them stand the ISO/IEC 8859 family or Unicode (ISO/IEC 10646).

The codeset part often matters more than the rest of the name. ko_KR.UTF-8 and ko_KR.EUC-KR share a language and a territory but differ in how many bytes a character takes. In the demonstration their MB_CUR_MAX values split, 6 against 2.

68.5.2 Names in other worlds — BCP 47 and Windows

The same “Korean (Republic of Korea)” is named differently by each system.

BCP 47 is the internet standard — the tag in HTML’s lang="ko-KR" and in HTTP’s Accept-Language. It uses a hyphen instead of an underscore and has no codeset part, because the web handles encoding separately. When the script must be stated, an ISO 15924 code goes in the middle, as in sr-Latn-RS.

A program that spans both worlds must translate names. To pass a browser’s ko-KR to setlocale it has to become ko_KR.UTF-8, which means holding a mapping table somewhere. Mistakes are frequent at that seam.

68.6 The precedence of the environment variables

We said setlocale(LC_ALL, "") uses “the locale the environment says”. Exactly what environment — POSIX fixes the precedence.

So LANG=ko_KR.UTF-8 LC_NUMERIC=C ./program runs mostly with Korean conventions but with numbers in C conventions. The scripts that launch server programs pin LC_ALL=C for the same reason — to insulate logs and parsing from the environment.

$ locale                 # what the environment is setting right now
$ locale -a              # the locales installed on this machine
$ locale -k LC_NUMERIC   # the values of one category in detail

68.7 Where locale data lives

A locale is not inside the program. It has to be installed on the machine.

Knowing this structure lets you solve the “missing locale” problem yourself. If a container has no ko_KR.UTF-8, put the definition file in and build it with localedef.

localedef -i ko_KR -f UTF-8 <path>/ko_KR.UTF-8
localedef -i de_DE -f UTF-8 <path>/de_DE.UTF-8
export LOCPATH=<path>

We have seen what a locale is and how one is chosen. The next chapter is what it changes and how — numbers, money, time and sorting, one at a time.