Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The tipping point for me would be `snprintf()` and related functions. I've found it generally more useful/memorable/readable than strncpy()/strlcpy() and other updates to the dangerous and deprecated strcpy() just for copying strings safely, never mind it's other formatting abilities.

If Curl already has its own "decent and functional replacement" for `snprintf()` that's used extensively throughout the codebase, or if they just don't need that functionality (I haven't checked) then I guess that's not an issue. But that would be the big selling point as far as I'm concerned.



Note that strncpy() is not intended for safety. The purpose of strncpy() is to write to fixed size data structures such as part of the filesystem where you don't want to store NUL termination on strings.

Like 1980s Internet protocol features the rationale for weird things in C is more often "That's how Unix works" than "This is actually a clever safety feature".


> write to fixed size data structures such as part of the filesystem where you don't want to store NUL termination on strings

... AND where you want to pad the remaining space with zero bytes, so that you don't leak uninitialized memory onto the disk, or network.

The null byte padding behavior of strncpy makes it clear what the intended use was.

Also, the way C initializes character arrays from literals has strncpy-like behavior, because the entire aggregate is initialized, so the extra bytes are all zero:

   char a[4] = "a";      // like strncpy(a, "a", 4);
   char b[4] = "abcd";   // like strncpy(a, "abcd", 4);
the compiler could literally emit strncpy calls to do these initializations, so we might say that strncpy is a primitive that is directly relevant for run-time support for a C declaration feature.


Is the original intent of strncpy() germane to the GPs comment? Explicitly stating the max length to copy is an effective tool for avoiding buffer overruns, regardless of whether the designers imagined that important use case.


But the thing it does (fill out a fixed sized buffer without caring about NUL-termination) is not at all what you'd want from a safety feature.

If you look at this function assuming it's a safety feature, that's a huge surprise, and indeed if you were skimming you might miss what it does because (in the context of "it's a safety feature") this is an insane choice. "Why would you do that?". Well, because it's not a safety feature.

The perf cost isn't what you'd expect from a safety feature either. Suppose we have a 1MB buffer, and we strncpy "THIS" into it using n = 1024. That's just four bytes right? Nope. strncpy() will write "THIS" and then 1020 zero bytes.


Except strncpy is broken for C strings, because it doesn't guarantee nul termination. So if you forget to force a termination on every use, you get buffer overruns.

Not only that, but (because of its actual purpose) it also fills the buffer with nuls, which is a complete waste of resources.

So yes, the original intent of strncpy() germane to the GPs comment, because it makes strncpy actively dangerous and complete shit when working with C strings.


The issue is that strncpy() isn't a str___() function, despite its name. It's a 0x00-padded memcpy.


The output side of strncpy() might not be a str___() function, but AFAICS the input side of strncpy() is clearly a str___() function, since it stops reading (but not writing) at the first NUL byte.


But it's not an str* function, it's an strn* function. And most (though not all, that would be too easy) work on fixed-size (hence the n) nul-padded strings.


No, it isn't a string function of any kind. "A string is a contiguous sequence of characters terminated by and including the first null character." § 7.1.1.

Calling a bespoke byte-sequence data structure a "string" is inaccurate. Treating strncpy() as a string function is erroneous and can easily lead to memory corruption.


[flagged]


> Have you considered giving reading comprehension a try?

Don't do this.


Then don’t demand it by wilfully misunderstanding comments in order to “well actually” them.


Do we have a source for what the intended purpose is? I think you speak well to the effective purpose, but I'm not sure if it was that clear when it was introduced.


> I think you speak well to the effective purpose, but I'm not sure if it was that clear when it was introduced.

It was completely clear, and can easily be inferred from its specified behaviour.

It's just completely useless nowadays, because its purpose is essentially obsolete, because the data type it works with is almost never used anymore.

strncpy works with fixed-size nul-padded fields as you'd find in e.g. mainframe-type software. That is why it:

- fills the destination buffer with NULs if the source is shorter

- does not nul-terminate if the source is the same size or longer than the destination

strncpy is essentially equivalent to zero-ing a buffer of size `n` then copying the first `n` bytes of src (up to the first nul) in the target


That's your understanding now. Fine, but not evidence of what someone else thought about this some decades ago.

(I don't use strncpy and don't defend its functionality, just want to know what was intended when it was introduced.)


I found an actual quote of a source which verifies the intention for fixed-length fields: https://softwareengineering.stackexchange.com/a/438090


On early Unix systems, the directory structure was a simple 16-byte record; 14 bytes for the name, and 2 bytes for the inode. [1] strncpy() was used to simply record the file name into this structure.

[1] "UNIX Implementation" by Ken Thompson, _The Bell System Technical Journal_, July-August 1978, Vol 57, No 6, Part 2, pg 1942.


Many protocols still relevant today make use of that structure, it still has widespread use.


For the average protocol you're going to init the entire message then set into it, you don't need to fully zero fields.

This is mostly relevant to write into existing memory or memory-mapped records.


Sometimes yes, but you must also not null-terminate the strings in those cases.

And on embedded devices you are often memory constrained so you might reuse an existing structure.



Doesn't say anything about the intention as understood when it was introduced


Get strlcpy and strlcat from OpenBSD.


Any version that still relies on separate parameters is unsafe, no matter what.

Some typo on the buffer limits and the same hazards as always.

Only fix is hardware memory tagging.


C needs to bite the bullet and adopt slices as first-class types, so that they can be optimized on ABI level.


How would you optimize slices the ABI level? Supporting them in function calling conventions should be easy. But figuring out storage representations I suppose would be a huge can of worms. There are too many ways of encoding slices depending on the use case. It only starts with the choice of a length field type (8, 16, 32, 64 bit. signed or unsigned)? There are also other representations thinkable, like sentinel values (NUL terminator) or more implicit storage of the size. Supporting them all in the compiler is not possible in practice.


"Slice" is, by now, a fairly established term in PL design which implies a tuple of (start, end) or (start, length), so it specifically excludes prefixed length, null termination etc - because experience has shown that slices are the only sane choice.

What I meant by optimization is not treating them same as other structs, but e.g. guaranteeing pass-by-register like other primitive types, spelled out explicitly in the ABI. The choice of length field type would be size_t, obviously.


Like every other systems programming language with a slice like feature, including those that predate C.


Please show, don't tell. Your sibling explained that they meant only function call optimization. And I would say it's debatable that this is an "optimization" since slices would be a new concept that is distinct from structs. I agree though that the obvious naive choice would be to pass them in the same way that structs of { ptr, len } are passed in the ABI.


Infosec people have been showing the C folks for decades, showing alone isn't enough, when people refuse to change their habits.


Multics?


I don’t think Multics did hardware memory tagging.

Systems which do/did include Burroughs Large Systems (now Unisys ClearPath MCP), IBM System/38 and AS/400 and IBM i (the RISC versions of which used PowerPC AS Tagged Memory Extensions), ARM MTE, SPARC ADI, and CHERI/ARM Morello.


Multics didn't need it, because PL/I does bounds checking by default, it has proper string and array data structures.


then why not just use memmove() instead of strncpy() if no-NUL is the goal? not to mention memmove() is overlap-safe.


> then why not just use memmove() instead of strncpy() if no-NUL is the goal?

no-nul is not the goal of strncpy, it's the effect of strncpy.

strncpy is designed to work on fixed-size, nul-padded strings. That's why it fills the destination buffer with nuls if the source is too short, and it doesn't guarantee nul-termination (if the source is exactly the size of the destination).


If anyone reading this and feels uncertain on how this is done, DON'T do this:

    snprintf(dest, sizeof dest, source);   // BAD code do not repeat
that looks great at first sight, just another size-checked way of copying strings, but remember that the third argument to `snprintf()` [1] is of course a `printf()`-style formatting string. So if that `source` argument contains any percent symbols, there's gonna be a party in your computer and both Undefined and Behavior are going to show up. You don't want that.

Instead, if you want to use `snprintf()` for this, remember to do:

    snprintf(dest, sizeof dest, "%s", source);

[1]: https://linux.die.net/man/3/snprintf


Thanks for mentioning this. I didn't include it in my comment because I originally thought it was too obvious, but considering it now, it probably was worth making explicit.


I blame Stack Overflow for conditioning me into seeing more of the possible ways things could break, when it comes to C code. :)


`-Wformat-security` [0] to the rescue!

[0]: https://fedoraproject.org/wiki/Format-Security-FAQ


Curl uses its own implementations, curl_msnprintf and curl_mvsnprintf.


The printf() family is much slower than direct string operations. snprintf() is not a good substitution when string copying is frequent.


I tend to stay away from the entire C style strings approach in general whenever I write C.


I always use antirez's SDS string Library. The fact that they are compatible C strings with the only price to pay being a call to sdsfree() instead of free I like a lot. Simple and yet super useful. Check it out.


Strlcpy and strlcat from OpenBSD works.


IMO you should store the size of your strings. If you know the sizes already then you can just memcpy/memmove.


Only for those that never do mistakes with parameter passing.


> But that would be the big selling point as far as I'm concerned.

I mean, it's a trivially replicated function, you can just copy it over from an existing codebase. So it's not exactly a major feature.


strlcpy() doesn't fix the problem of getting the length parameter wrong. The right solution is this:

    char *stecpy(char *d, const char *s, const char *e)
    {
      if (e) e--;
      while (d < e && *s)
        *d++ = *s++;
      if (d)
        *d = '\0';
      return d;
    }

    main() {
      char buf[64];
      char *ptr, *end = buf+sizeof(buf) ;
    
      ptr = stecpy(buf, "hello", end);
      ptr = stecpy(ptr, " world", end);
    }
As discussed here https://twitter.com/hyc_symas/status/1382298601641152513

The point is that the end of the buffer is invariant, there's no reason to screw around recalculating the length of remaining space after each copy into the buffer. This also fixes the nonsense of strcpy/strcat returning the same dst pointer that was passed in. By returning the pointer to where copying ended, you don't need a separate strcat function any more, nor do you have the Shlemiel The Painter problem with strcat.


> The tipping point for me would be `snprintf()` and related functions

Functions can always be implemented with additional header files.

They aren't actual extensions to the language.


But snprintf can be used in C90. You just detect if it's available and use it, as if it were any other platform-specific function.


Hm, why do people choose to do that instead of using the fallback case unconditionally? Expectation that platform-provided snprintf is faster?


The platform-provided snprintf is smaller; it takes up zero bytes in your program.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: