Here's one more tip: did you ever notice that "ls" displays multiple columns, but "ls | cat" prints only one filename per line? Or how "ps -f" truncates long lines instead of wrapping, while "ps -f | cat" lets the long lines live?
You can do it too, and if you're serious about writing Unix-style filter programs, you will someday need to. How do you know which format to write? Call "isatty(STDOUT_FILENO)" in C or C++, "sys.stdout.isatty()" in Python, etc. This returns true if stdout is a terminal, in which case you can provide pretty output for humans and machine-readable output for programs, automatically.
IMO, this is an anti-pattern. It's violates the principle of least surprise. (How come I see X when I run the command, but I can't grep for X in its output? How come it works when I run it from my interactive shell, but it's broken when I run it from a script? And things like that.)
Interesting. Doesn't git (a pretty new tool) violate this? I think running log from the terminal pauses per page of output but if you pipe it to something it pipes all the content.
I think it's just piping it to another program. It's still giving you the same output, but sending you to a program meant for humans to be able to read text in a console, instead of just printing it all out.
That said, I'm not entirely sure which git pipes to.
I think it depends what sort of things you use it for. I often use it to switch on or off ANSI colourization, which doesn't really violate the principle of least surprise.
When used sparingly and thoughtfully, I've never personally had an issue with it.
You may not have issue with the sort of things you use it for, but others might.
For example, I run shells in Emacs and have had to tweak loads of shell scripts written by colleagues to fix their poorly-implemented colourisation. It's useful to know when a test has failed; it's not so useful to have the whole terminal set to white text on a pale pink background.
One day I couldn't SSH into our servers from Emacs. It turned out somebody had edited .bashrc for the admin user to make the bash prompt blue. Emacs' TRAMP process was looking for a prompt ending in "$" or "#", not "$\[\033[0m\]", so it didn't realise the connections were successful.
There are two ways of handling this: we can blame the source of the bug (the person adding the colours incorrectly, or the assumption-loaded TRAMP regex), but there will always be more bugs in situations we'd never think of. Alternatively, we can avoid being 'too clever', and instead aim for consistency and least surprise.
As much as I love Emacs, that sounds like it's a bug to assume that one's prompt will follow a convention (ends-in-$). The convention is useful and good, but it seems strange to blame someone for breaking your tool's expectations when they had made a valid prompt.
Are you suggesting that colored prompts violate the rules of consistency and least surprise?
(Actually, if you are suggesting that, I'm not going to disagree. But I am going to say that if so, those rules don't apply in the case of colored prompts, because colored prompts are useful.)
I suppose I'm suggesting that, aside from personal scripts, we shouldn't assume too much about who our users are and what they're trying to do. The principle of least power tells us to use the dumbest format that will work, eg. plain text.
Anything we add on top of that, eg. ANSI colour codes, will be useful to some but harmful to others. The tricky part is working out which of those categories the current user is in.
So is your proposed solution not to have colored prompts? (I vehemently disagree.) Or not to put them in a .bashrc? (I still disagree, but only strongly.) Or something else?
To be precise, what you're suggesting is that we have prompts which are allowed to be colored except for the $/# at the end, because you can't color those without following them by escape characters. And that prompts must have a $/# at the end.
Good point, and it makes me even more convinced that all rules of thumb have important exceptions. In fact, I've also used tools that use ANSI colorization, and disable that when not talking to a tty, and wished that they wouldn't because I was piping them to "less -R" :) (At least that's a graceful failure mode, and it's pretty clear what probably happened even if one doesn't exactly understand the details.)
I've always thought it would be nice to have a utility like cat that I could pipe commands to, which would trick them into thinking all their streams were attached to a tty, so you could do "uses_colours | ttycat | less -R".
I'm sure it's possible, but you'd have to acquire a new pty and decide what termios settings you want. It's a nontrivial hack, I think.
I'm actually kind of surprised it's not in moreutils[1].
> I've always thought it would be nice to have a utility like cat that I could pipe commands to, which would trick them into thinking all their streams were attached to a tty,
Daniel J. Bernstein wrote a "pty" package back around 1991 that did this. Version 4 of the package was published in 1992 to comp.sources.unix (volume 25 issues 127 to 135). It's still locatable on the World Wide Web.
Bernstein later updated this, around 1999, with a "ptyget" package that was more modular and that had the session management commands moved out of the toolset to elsewhere. The command from that package to do exactly what you describe is "ptybandage". There is also "ptyrun". Paul Jarc still publishes a fixed version of ptyget (that attempts to deal with the operating-system-specific pseudo-terminal device ioctls in the original) at http://code.dogmap.org./ptyget/ .
It’s not possible in the way you think, because the two programs don’t know about each other. All the pipe character does is put the stdout into the stdin of the other program.
So this feature must actually be present in the shell and maybe it is. I’m no expert but maybe zsh already offers something like this?
I agree, especially for the behaviour from the parent:
"ps -f" truncates long lines instead of wrapping, while "ps -f | cat" lets the long lines live
How people usually discover what these commands do is by running them interactively, and if that results in some output being hidden vs being run noninteractively, then they have little reason to believe that it could yield more output than what they're used to seeing. I think a certain number of "ps" users don't know it can display full paths and commands, if they've only ever used it interactively.
Yep, and the result is I have needless sprinklings of "www" in my shell scripts cause of habit. Technically I don't need it the moment I pipe into grep, but oh well ;) Anyway I personally dislike the SysV-like 'I need to add stuff like -aef' to get all that on-the-screen/into-grep vs the BSD-like it knows about TTY but I can convince it otherwise into stuf like 'less -S' - personal taste I guess.
Actually I just found this in the 'ps' manual, it looks like the output width is actually undefined!
"If ps can not determine display width, as when output is redirected (piped) into a file or another command, the output width is undefined (it may be 80, unlimited, determined by the TERM variable, and so on)."
Ideed. I've seen people over and over stumbling over this weird behaviour.
It may have some merits, but as a general advice this is definitely an anti-pattern.
Another example is "curl", where "curl URL >outfile" is chatty on stderr, while "curl URL" is quiet on stderr. That's very annoying for scripting, you easily forget to set "-s" in your scripts due to that behaviour.
However, it's not on trunk yet because it's hard to find good defaults. An automatic pager makes sense for some commands, but not all -- and in a meritocratic development model this kind of thing can cause an endless discussion... I suspect we'll eventually merge the feature in a disabled by default state and allow users to enable it on a per-command basis.
That's an interesting case. Should it not just output to stdout and you have to pipe it to less? If you wanted it to always pipe to less you could just set up an alias for it.
I've just realised that the best solution to this (which I've never seen) is for the shell or terminal emulator to capture long output into a pager for you once it exceeds a certain length.
The git diff and log commands are primarily intended for human consumption. I'm not the one you were relying to, but I think it makes sense that git defaults to make that consumption easier, even if it isn't strictly "Unix". (You can also disable this behavior and get a more Unix-y interface by default via gitconfig or on a case by case basis with --no-pager).
For sure, and in fact, I very happy with the current behaviour. I guess the pager is a case where it doesn't matter much if it's included or not since I guess it doesn't get in the way if you pipe to something else. Does it consume extra resources?
git log | wc
vs
git log --no-pager | wc
I'm sure it's neither here nor there in practice. More of a hypothetical question.
I despise it when commands do this - mysql -e results are formatted differently depending on whether the output is directed to the terminal or to a file.
Or, execute "/bin/[ -t 1" (or "test -t 1", or "[[ -t 1 ]]", or ...). This is handy in shellscripts (obviously), but also in languages like Go, which lack a builtin way to test whether stdout is a TTY. e.g.:
As I recall, the original ls didn't have that feature.
Examining the characteristics of the output stream and changing behavior is another "rule" that is not mentioned often. Another example is buffering the output to a large block if sending to a pipe, but making it line-buffered if going to a terminal.
You can do it too, and if you're serious about writing Unix-style filter programs, you will someday need to. How do you know which format to write? Call "isatty(STDOUT_FILENO)" in C or C++, "sys.stdout.isatty()" in Python, etc. This returns true if stdout is a terminal, in which case you can provide pretty output for humans and machine-readable output for programs, automatically.