I’ve tried a few of these, but most of them seem to be following the trend of folding other shell functionality into one tool. Searching for contents (find + grep -H, or ripgrep), filtering (grep), sorting (ls does it natively, or you can use sort, sort -h for sorting human readable sizes), the list goes on and on.
I guess this is a mini lament that many of these tools are moving away from the Unix philosophy of do one thing well, and make it easy to chain.
And a last very small lament that BeOS didn’t succeed, and their filesystem-as-a-database approach didn’t become more standard.
You can still chain ripgrep. I specifically designed it so that you can chain it just like you would a normal grep.
It does indeed also include other functionality that might traditionally be left to other tools (like filtering files). But this is nothing that GNU grep wasn't already doing itself anyway.
IMO, it's better to view the Unix philosophy as a means to an end and not an end to itself. And IMO, it's important to weigh the benefits of coupling to the user experience.
>view the Unix philosophy as a means to an end and not an end to itself
it won't be a means to an end any more if you don't preserve it, so not breaking that aspect of it has to be one of your ends. if you use it to take ls to a new place but that place is not within the ecosystem, it will be an evolutionary dead end, or worse, the first meteor in the meteor storm that ends all life.
current/traditional unix may not be the be-all/end-all, but replacing it/changing it requires viewing it comprehensively and changing all the tools at once or having a plan to. A good example of this is Plan9
I don't know what you're trying to say and I don't see how it's in conflict with anything I've said.
>not an end to itself
it is an end to itself. the reason it's a means to an end is because that was its end goal. in being a means to an end, it is an end (its end) unto itself, opposite to what you said, imho
I still can't parse what you're saying. The Unix philosophy is a means to an end, where the ultimate end is improved user experience. The means is de-coupling and composition. But there are other means to improving the user experience.
> in being a means to an end, it is an end (its end) unto itself
This either makes zero sense or is vacuously true and clearly not in conflict with what I'm saying.
I think ripgrep specifically is counted in the comment you reply to as a tool that _does_ do one thing well, and that one should use it (or grep) in combination with an ls, instead of giving ls filtering abilities.
I suppose. But I wanted to point out that ripgrep couples functionality, specifically in contradiction to the Unix philosophy. And actually, many command, including "traditional" tooling, so as well.
The point is that many pay lip service to the Unix philosophy as if it were an end. But it isn't.
> You can still chain ripgrep. I specifically designed it so that you can chain it just like you would a normal grep.
Headings on when isatty and off when piping the output put me off when I first tried ripgrep. I don't expect the tools to change their output format on me.
Luckily, you made this behavior configurable, so I'm a happy convert now.
> I don't expect the tools to change their output format on me.
You probably do! If you've ever used `ls`, then it does exactly this.
If you mean the ANSI color stuff, yes - I do expect these to disappear :)
I meant the "shape" of the output. It just doesn't follow the principle of least surprise.
edit: you probably meant the columns. I forgot about that, I haven't parsed ls(1) output in ages ;)
Yes. The columns. The point is that commands have been changing their output format, not just their colors, based on tty for ages. So the criticism you lodge against ripgrep also applies to some of the most core commands you probably use daily.
I would be quite surprised if you didn't rely on this without even knowing it. Even a simple `ls | wc -l` relies on it.
I say this because it's tiring to see folks lament about this feature in ripgrep as if it's something new that ripgrep does. It's not. It's a well established idiom among Unix command line tools.
Isn’t “don’t parse ls” like the third commandment of Unix?
You've never done `ls | wc -l`?
I've always assumed that ls doesn't change it's output when piped; I've always done ls -1|wc -l. I guess I can save on a few keystrokes now.
They don't do one thing well since it's all text, not structured data, which makes chained analysis a challenge, which leads to the desire for integration
ls is tabular data, and you can format it (ls -1, ls -l, ls -w, plus sorting, field formatting, and more), and you can cut/parse/format in a standard way. Every field sans the filename is fixed length, can be handled with awk/cut/sed according your daily mood and requirements, etc. etc.
So, ls can be chained very nicely, which I do every day, even without thinking.
You don't need to have a "structured data with fields" to parse it. You just need to think it like a tabular data with line/column numbers (ls -l, etc.) or just line numbers (ls -1).
So, as long as ls does one thing well, it's alright.
Ah, some of the "enhanced" ls tools can't distinguish between pipe and a terminal, and always print color/format escape codes to pipe too, doubling the fun of using them. So, thanks, I'll stick with my standard ls. That one works.
> You don't need to have a "structured data with fields" to parse it.
You do if you want to have nice things like being able to format your output without having to worry about breaking the dumb tools down the pipe, which can't sort the numbers they don't see:
- 2.1K (this isn't the same as the second) - 2.1K - 2.1M
Also, why do I need to count columns like a cave man in 'sort -k 5' instead of doing the obvious "sort by size"?
> print color/format escape codes to pipe too
A problem that would disappear with... structured data!
> Ah, some of the "enhanced" ls tools
so use the other "some" that can?
> which can't sort the numbers they don't see
Then you sort at the point you can see the numbers and discard them later.
> Also, why do I need to count columns like a cave man in 'sort -k 5' instead of doing the obvious "sort by size"
awk can sort the columns for you. Plus, ls can already sort by size. Try "ls -lS " for biggest file first, or "ls -lSr" for smallest file first. Add "-h" to make human readable.
> A problem that would disappear with... structured data!
No. A problem that would disappear with "a small if block which asks which environment I'm in". If you're in a shell "-t" test in sh/bash will tell you that. If you're coding a tool, there are standard ways to do that (via termcap IIRC). Standard UNIX tools are doing this for decades now.
IOW, structured data is not a cure for laziness...
> so use the other "some" that can?
Yes, because their authors are not that lazy.
> Then you sort at the point you can see the numbers and discard them later
This sort of human overhead is only needed to compensate for the deficiencies of the data structures
> ls can already sort by size
That's the benefit of integration you're arguing against with your deficient piping suggestions
> IOW, structured data is not a cure for laziness...
It is precisely what good design is for - it reduces the need for various dumb workarounds that bad design requires, which means you can be more lazy and avoid said workaround
> Yes, because their authors are not that lazy.
This just ignores the argument, which was "some better new tools don't do that" isn't relevant when some better new tools also do that
vanilla ls has never been particularly chainable - https://mywiki.wooledge.org/ParsingLs
A lot of this post hinges on the fact that newlines in filenames were legal, and that people wrote shell without handling quoting correctly. While quoting (as well as ls altering filenames) is still an issue, find -print0, read -d '', and similar are no longer neccessary. Newlines are now forbidden in filenames: https://blog.toast.cafe/posix2024-xcu
> Newlines are now forbidden in filenames
No. To quote that article
> A bunch of C functions are now encouraged to report EILSEQ if the last component of a pathname to a file they are to create contains a newline
This, yes, makes newlines in filenames effectively illegal on operating systems strictly conforming to the new POSIX standard. However, older systems will not be enforcing this and any operating system which exposes a syscall interface that does not require libc (such as Linux) is also not required to emit any errors. The only time even in the future that you should NOT worry about handling the newline case is on filesystems where it's is expressly forbidden, such as NTFS.
Most utilities that create files are encouraged to error on newline filenames, which makes this effective illegality stronger. The post also discusses the future of this encouragement, which is turning it into a requirement.
> However, older systems will not be enforcing this
Eventually, newlines in filenames will go the way of /usr/xpg4/bin/sh.
I'd like to note that up until this point, there hasn't (and isn't) been a fully POSIX compliant way to do many shell operations on newline containing filenames. They are already effectively unsupported, and the standard that adds support also discourages them from being created and used. The best way to handle them up until this point has been to not use sh(1).
Linux isn't POSIX compliant, and as far as I know has no plans to ban newlines in filenames, or even add an option to disable newlines.
In past, there have been Linux-based operating systems that have been certified as Single Unix Specification compliant, and part of said specification is POSIX. I would imagine GNU and Busybox and Musl will be willing to implement the changes proposed by POSIX 2024, which inevitably leads down the road of newlines being banned.
Howw would that work? Checking strings passed to open and rejecting them? Would we then have undeletable files, as we can't refer to their filenames?
I know Linux allows newlines in filenames, but every time I hear it I want to drink.
If you like that philosophy check out nushell. They go pretty hard core on that and they can because of structured output
I agree with this.
If they want something that is easy to use in a non-scriptable way, maybe they should replicate Norton Commander instead.