65 Strings and memory — <string.h>

65.1 The truth about strncpy

#include <stdio.h>
#include <string.h>

static void dump(const char *label, const char *buf, size_t n)
{
    printf("%-14s", label);
    for (size_t i = 0; i < n; i++) {
        unsigned char c = (unsigned char)buf[i];
        if (c == 0)       printf(" \\0");
        else if (c >= 32) printf("  %c", c);
        else              printf(" %02x", c);
    }
    printf("\n");
}

/* Put it behind a function boundary so the compiler cannot see the source's
   length. (Otherwise gcc catches it with -Wstringop-truncation.) */
static void copy_into(char *dst, size_t cap, const char *src)
{
    strncpy(dst, src, cap);
}

int main(void)
{
    /* (1) it fills every spare place with 0 — expensive on a large buffer */
    char pad[10];
    memset(pad, 'X', sizeof pad);
    copy_into(pad, sizeof pad, "abc");
    dump("short source:", pad, sizeof pad);

    /* (2) if it fits exactly or overflows, no NUL is attached — it stops being a string */
    char tight[4];
    memset(tight, 'X', sizeof tight);
    copy_into(tight, sizeof tight, "abcd");
    dump("exact fit:", tight, sizeof tight);
    printf("               there is no NUL — printing this with %%s is outside the contract\n");

    /* (3) to use it safely, close the last place by hand */
    char safe[4];
    copy_into(safe, sizeof safe - 1, "abcd");
    safe[sizeof safe - 1] = '\0';
    printf("closed by hand : [%s]\n", safe);

    /* (4) to know whether it was truncated, the length must be measured separately after all */
    const char *src = "abcd";
    printf("truncated?     : %s\n", strlen(src) >= sizeof safe ? "truncated" : "whole");
    return 0;
}

short source:   a  b  c \0 \0 \0 \0 \0 \0 \0
exact fit:      a  b  c  d
               there is no NUL — printing this with %s is outside the contract
closed by hand : [abc]
truncated?     : truncated

Two things come out.

First, it fills all the spare places with zeros. Put 3 characters into a 10-byte buffer and it writes zeros over the remaining seven. The bigger the buffer, the bigger the waste.

Second, when it fits exactly or overflows it does not attach a NUL. Put abcd into a 4-byte buffer and there is no room for the NUL, so what remains is a byte array, not a string. Print that with %s or pass it to strlen and it reads outside the buffer — the typical route of “I used the safe function and it blew up.”

gcc really does catch this mistake. Here is the diagnosis received when first writing the example.

error: ‘strncpy’ output truncated before terminating nul copying 4 bytes
       from a string of the same length [-Werror=stringop-truncation]

But as seen in chapter 62, the compiler catches only what it can see. If the source’s length is settled during execution, this warning does not appear.

char dst[32];
strncpy(dst, src, sizeof dst);      /* there may be no NUL */
printf("%s\n", dst);                 /* it reads outside the buffer */

strncpy(dst, src, sizeof dst - 1);
dst[sizeof dst - 1] = '\0';          /* close it by hand */
if (strlen(src) >= sizeof dst) { /* truncated — handle it */ }

65.2 Then what is used

Within the standard, the most practical tool for safely joining strings is in fact in <stdio.h>.

int need = snprintf(dst, sizeof dst, "%s", src);
if (need < 0 || (size_t)need >= sizeof dst) { /* truncated */ }

There is the criticism that it is slow (the cost of interpreting the format), but it is the only standard function that keeps the boundary while letting you know about truncation.

strncat’s argument is a particular trap — the second argument is not the destination’s size but the number of bytes that may additionally be written. strncat(dst, src, sizeof dst) is almost always wrong, and sizeof dst - strlen(dst) - 1 is right.

65.3 Overlapping regions — memcpy and memmove

#include <stdio.h>
#include <string.h>

/* memcpy does not allow overlapping regions. memmove does. */
static void show(const char *label, const char *s)
{
    printf("%-22s [%s]\n", label, s);
}

int main(void)
{
    char buf[32];

    /* a copy that does not overlap — the place where memcpy fits */
    strcpy(buf, "abcdefgh");
    char other[32];
    memcpy(other, buf, strlen(buf) + 1);
    show("no overlap (memcpy):", other);

    /* shifting by one — source and destination overlap, so memmove is used */
    strcpy(buf, "abcdefgh");
    memmove(buf + 1, buf, 7);       /* shift one place back */
    buf[8] = '\0';
    show("shift by 1 (memmove):", buf);

    /* pulling forward overlaps as well */
    strcpy(buf, "abcdefgh");
    memmove(buf, buf + 2, 6 + 1);   /* pull two places forward */
    show("pull by 2 (memmove):", buf);

    /* strtok destroys the original — and hides state inside the function */
    char line[] = "name,age,city";
    printf("before strtok : [%s]\n", line);
    for (char *t = strtok(line, ","); t; t = strtok(NULL, ","))
        printf("  token: [%s]\n", t);
    printf("after strtok  : [%s]  <- the original has been cut\n", line);
    return 0;
}

no overlap (memcpy):   [abcdefgh]
shift by 1 (memmove):  [aabcdefg]
pull by 2 (memmove):   [cdefgh]
before strtok : [name,age,city]
  token: [name]
  token: [age]
  token: [city]
after strtok  : [name]  <- the original has been cut

memcpy’s contract includes “the two regions must not overlap”. Calling it with them overlapping is undefined behaviour, and in an optimised implementation values really do get scrambled — because there is no guarantee that bytes are moved in order (several bytes may be moved at once with SIMD, or moved from the back).

If they may overlap, it is memmove. Contrary to the impression its name gives, it does not mean “moving” but copying that is safe even when overlapping.

65.4 strtok — it destroys the original and hides state

The last part of the example. strtok has two sins.

First, it destroys the original. It makes tokens by overwriting the separators with NUL. So it cannot be used on a read-only string (a string literal) — using it there is outside the contract — and if the original is needed it must be copied first.

Second, it hides state inside the function. That is why NULL is passed from the second call onward. That state is singular, so if another function calls strtok in the middle of cutting tokens the two wreck each other’s traversal. In a program running along several strands it gets worse.

There are three alternatives. Use an edition in which the caller holds the state, such as strtok_r (POSIX) or strtok_s (annex K); cut it yourself with strcspn and strchr; or use a tool that does not touch the original, like Part XII’s view-based splitting.

65.5 The traps of the remaining functions

The two starred entries are especially dangerous in practice.

Erasing a secret with memset — the memset(key, 0, len) that erases after use may, if key is not read afterwards, be seen by the compiler as a “useless write” and deleted (chapter 13′s optimisation story). C11 put memset_s in annex K for this, and each platform has a function such as explicit_bzero or SecureZeroMemory.

Comparing structs with memcmp — because of the padding seen in chapter 46. Even for two structs holding the same values, if the padding bytes differ memcmp answers “different”. The members must be compared one by one.

We have passed the strings. The next chapter is the drawer of odds and ends and a treasury of accidents — <stdlib.h>.