#] #] ********************* #] "$d_SysMaint"'Linux/tac opposite of cat notes.txt' # www.BillHowell.ca 15Oct2023 initial # view in text editor, using constant-width font (eg courier), tabWidth = 3 15Oct2023 Wow! - baeldung's parallel sort! $ time nl megafile | sort -S 200G -nr --parallel=23 --batch=1021 | cut -f 2- >> /dev/null but normally - sud tac, sed not so good #48************************************************48 #24************************24 # Table of Contents, generate with : # $ grep "^#]" "$d_SysMaint"'Linux/tac opposite of cat notes.txt' | sed "s/^#\]/ /" # #24************************24 # Setup, ToDos, #08********08 #] ??Oct2023 #08********08 #] ??Oct2023 #08********08 #] 15Oct2023 tac - opposite of cat +-----+ https://stackoverflow.com/questions/742466/how-can-i-reverse-the-order-of-lines-in-a-file How can I reverse the order of lines in a file? Asked 14 years, 6 months ago Modified 8 months ago Viewed 461k times +--+ at the end of your command put: | tac tac does exactly what you're asking for, it "Write each FILE to standard output, last line first." tac is the opposite of cat :-). edited Nov 11, 2018 at 21:53 Dror answered Aug 8, 2018 at 10:24 Yakir GIladi Edry +-----+ https://www.baeldung.com/linux/reverse-order-of-file-lines How to Reverse the Order of Lines in a File in Linux Written by: baeldung 2. Tac One of the core philosophies of Linux is “everything is a file”. So it’s no surprise that most Linux distributions come pre-equipped to help us. cat is one of the most popular and well-known ways to output the contents of a file. On the other hand, tac is much a lesser-known command. Its name comes from cat spelled backward, and tac functions as a reverse cat. Both commands belong to the coreutils package, which comes preinstalled in almost all Linux distributions. 3. nl/sort/cut Commands Like cat and tac, these commands come with the coreutils package, which comes preinstalled in all of the common distributions. The sort, nl, and cut commands need to be chained together to reverse a file. Let’s build up the chain a piece at a time, so we can understand how each command contributes. 3.1. nl To be able to order the file in reverse order, we need an index for each row. So, we use the nl command to put line numbers at the beginning of each line: $ nl /tmp/test 1 line_one 2 line_two 3 line_three 3.2. sort Now we want to sort these indexed lines into reverse order, for which we should use sort: $ nl /tmp/test | sort -nr 3 line_three 2 line_two 1 line_one By default, sort orders lines lexically and arranges them from smallest to largest. We’re using a couple of parameters to change that here: -n – numerical sort -r – reverse order We could also add some other parameters for extra performance: –parallel – number of sorts to run at the same time –batch – max number of inputs to process at once -S – max memory sort can use The optimal values for these parameters will depend on our system hardware and operating system limits. 3.3. cut As we added a numeric index at the start of the process, we need to remove it to get our lines back to their original form: $ nl /tmp/test | sort -nr | cut -f 2- line_three line_two line_one The -f 2- parameter tells cut to print characters which appear after the second whitespace. Here, this means just after the line number generated by the nl command. 4. sed 4.2. sed Script to Reverse a File To reverse the given text file with sed: $ sed '1!G;h;$!d' /tmp/test line_three line_two line_one sed scripts are hard to understand, so let’s unpack this one. 4.3. Subcommands of sed The one-liner above applies three sed commands to every line. The commands, separated by semicolons, are: 1!G – G command appends what is in the hold space to the pattern space, 1! makes sure this command will ignore the first line h – copies the pattern space to the hold space $!d – delete the line, $! makes sure this command will ignore the last line For more, check out our in-depth guide on sed and its different spaces. 5. Comparison Between Methods Let’s compare the methods we’ve learned. 5.1. Daily Performance For testing, we’ll use a file with 100,000 lines and size of 6.6 megabytes: $ wc -l test 100000 test $ du -h test 6,6M test Let’s reverse this file with tac while keeping track of time: $ time tac test ... ... ... real 0m0,571s user 0m0,004s sys 0m0,086s tac completes the task in about half a second. Now, let’s do the same test with nl sort and cut: $ time nl test | sort -nr | cut -f 2- ... ... ... real 0m1,063s user 0m0,122s sys 0m0,236s This command takes little more than a second to complete. Now, let’s run it on multiple cores, with 1 GB memory and batches of 1021: $ time nl test | sort -nr -S 1G --parallel=7 --batch=1021 | cut -f 2- ... ... ... real 0m0,882s user 0m0,130s sys 0m0,194s Now it takes less than a second. This is not a big improvement. Now, let’s time the sed command: $ time sed '1!G;h;$!d' test ... ... ... real 0m54,336s user 0m53,802s sys 0m0,104s This is by far the worst result. We wouldn’t use this for speed. However, the point of using sed is its advanced text processing capabilities. As the results show, tac is the clear winner in terms of performance for daily tasks. So, is there any benefit of the sort approach? 5.2. Big Data Performance sort can use multiple cores at once, and the real power of sort comes in to play when we deal with huge files on powerful workstations. Let’s work on the following file: $ du -h megafile 54G megafile $ wc -l megafile 1000000000 megafile This file has 1,000,000,000 lines and is 54 gigabytes. $ time tac megafile >> /dev/null real 13m5.686s user 0m59.028s sys 0m47.556s tac completes the task in 13 minutes 5 seconds. Let’s try sort with 23 cores and 200 gigabytes of RAM: $ time nl megafile | sort -S 200G -nr --parallel=23 --batch=1021 | cut -f 2- >> /dev/null real 6m34.510s user 9m47.677s sys 3m1.545s tac clearly uses fewer resources overall, even though it takes longer. However, if we need to get things done as quickly as possible on a huge dataset, sort is much better. And, the gap in performance will grow as the files get larger. # enddoc