Throughout your career as a Software Engineer, working with the Shell (Command Line) is a daily routine and an essential skill. You will encounter numerous tasks that can be time-consuming, and eventually, you may discover that a specific tool could have saved you hours. Allow me to save you some trouble by providing a list of excellent tools that can transform the way you approach certain tasks.

#1 C (B)at

Yes, it's not a typo, bat is the evolution of what cat is. bat makes you able to stdout the content of a text file with keeping the formatting and text highlighting. for example, here is the difference between using bat on a Python file vs using cat

It doesn't end here; bat offers numerous useful features that you can utilize. For instance, let's examine this file to determine whether I used spaces or tabs!

by using the -A flag, I was able to see things like spaces, tabs, new lines ..etc. and As you can see, I used spaces inside the function and tabs inside the if statement (never do that, please!!)

bat is by default connected to git. so if you have git setup in your repo, it will read and display the changes automatically, for example

Project homepage

#2 Grep, your search tool

grep is a very powerful utility available in UNIX and UNIX-like operating systems. grep stands for g/re/p (global/regular expression search / and print) It is used to search text or search the given file for lines containing a match to the given strings or patterns.

Basic use

grep 'pattern' filename

This will search for 'pattern' inside the file called 'filename' and print out any lines that match.

Case Insensitive Search

grep -i 'pattern' filename

This command will match lines that contain 'Pattern', 'PATTERN', 'pattern', or any mix of cases.

Recursively Search Directories

grep -R 'pattern' directory_path

grep can be used to recursively search directories with the -r or -R option (recursive).

Search Multiple Patterns

grep -e 'pattern1' -e 'pattern2' filename

This command will print out lines that contain either 'pattern1' or 'pattern2' in the file 'filename'.

Line Number information

grep -n 'pattern' filename

With -n option, grep shows the line numbers of the lines in the output.

Count of lines with a certain pattern

grep -c 'pattern' filename

With -c option, grep will return the count of lines that match the pattern.

you can learn more about grep by exploring the options in grep --help or from the Grep Manual

#3 Crontab

Crontab is your friend when you need to schedule various jobs on your server or machine, which will be managed and executed by the cron daemon. You can use it to schedule updates, and cleanups, or even execute your scripts. Allow me to guide you through the main functionalities.

Listing your jobs

crontab -l

this will view the current jobs registered by the current user

Creating a new Cron

crontab -e

This will open an editor for a crontab file for you to write your corn job, you will need to use this format to describe the new corn job.

*     *     *   *    *        command to be executed
-     -     -   -    -
|     |     |   |    |
|     |     |   |    +----- day of week (0 - 6) (Sunday=0)
|     |     |   +------- month (1 - 12)
|     |     +--------- day of month (1 - 31)
|     +----------- hour (0 - 23)
+------------- min (0 - 59)

for example

0 0 * * 1 python3 example.py

This will execute the example.py at 12 AM every Monday

Deleting all the current registered cron jobs

crontab -r

This will remove all the current cron jobs registered for the current user. if you need access to all the system-wide cron jobs, you will need superuser privileges

Crontab Manual

#4 Xargs

Basic Usage

xargs is a powerful tool that reads and utilizes the output from one command, allowing you to apply various commands on the output with a range of options. For example:

ls | xargs

Here, xargs will get the output of ls and will print it in one line separated by spaces. As there was no command provided to xargs, it will just do a print. Let's try to apply ls -lh to every output we get from ls

ls | xargs ls -lh

Combine with other tools

Mostly, you will get the most of xargs when you use it combined with other tools, for example, combine it with find

find . -type f -mtime +30 | xargs rm -f

Here, you use find to filter every file that is over 30 days old, and pass it to xargs to apply rm -f on it. you can use it combined with crontab to make it a scheduled job (it's a perfect example of using these powerful tools instead of writing a specific program for it)

Use placeholders for more complex commands

Sometimes you need to automate commands that might need more than one argument, here you can use the power of placeholder. Let me give an example

ls -1 *.txt | xargs -I {} mv {} old/

Here, we define a placeholder using -I {} and use it to specify the exact location that we want our argument to be placed in.

Parallelize it

Sometimes you would want to parallelize the job executed on the arguments supported, to do that you will need to use -P , here is an example

find . -type f -name '*.txt' | xargs -n 1 -P 4 grep -i 'pattern'

Here, we use find to get all the text files, pass it to xargs to parallelize grep on each file using up to 4 parallel greps at once!

Xargs Manual

I come across so many handy tools every day that really make a difference in my daily work. I've shared a bunch of my favorites that I think will help you out too. If you've got any cool tools you use in your day-to-day tasks, feel free to share them in the comments section!

Catch you in the next one!