Linux commands and shell scripting

Introduction to Linux Shell and Shell Scripting

If you are using any major operating system you are indirectly interacting to shell. If you are running Ubuntu, Linux Mint or any other Linux distribution, you are interacting to shell every time you use terminal. In this article I will discuss about linux shells and shell scripting so before understanding shell scripting we have to get familiar with following terminologies –

• Kernel
• Shell
• Terminal

What is Kernel

Attention reader! Don’t stop learning now. Get hold of all the important CS Theory concepts for SDE interviews with the CS Theory Course at a student-friendly price and become industry ready.

The kernel is a computer program that is the core of a computer’s operating system, with complete control over everything in the system. It manages following resources of the Linux system –

• File management
• Process management
• I/O management
• Memory management
• Device management etc.
  It is often mistaken that Linus Torvalds has developed Linux OS, but actually he is only responsible for development of Linux kernel.
  Complete Linux system = Kernel + GNU system utilities and libraries + other management scripts + installation scripts.

  What is Shell

  A shell is special user program which provide an interface to user to use operating system services. Shell accept human readable commands from user and convert them into something which kernel can understand. It is a command language interpreter that execute commands read from input devices such as keyboards or from files. The shell gets started when the user logs in or start the terminal.

  

  Command Line Shell

  Shell can be accessed by user using a command line interface. A special program called Terminal in linux/macOS or Command Prompt in Windows OS is provided to type in the human readable commands such as “cat”, “ls” etc. and then it is being execute. The result is then displayed on the terminal to the user. A terminal in Ubuntu 16.4 system looks like this –

  

  linux command line

  It will list all the files in current working directory in long listing format.
  Working with command line shell is bit difficult for the beginners because it’s hard to memorize so many commands. It is very powerful, it allows user to store commands in a file and execute them together. This way any repetitive task can be easily automated. These files are usually called batch files in Windows and Shell Scripts in Linux/macOS systems.

  Graphical Shells

  Graphical shells provide means for manipulating programs based on graphical user interface (GUI), by allowing for operations such as opening, closing, moving and resizing windows, as well as switching focus between windows. Window OS or Ubuntu OS can be considered as good example which provide GUI to user for interacting with program. User do not need to type in command for every actions.A typical GUI in Ubuntu system –

  

  There are several shells are available for Linux systems like –

  • BASH (Bourne Again SHell) – It is most widely used shell in Linux systems. It is used as default login shell in Linux systems and in macOS. It can also be installed on Windows OS.
  • CSH (C SHell) – The C shell’s syntax and usage are very similar to the C programming language.
  • KSH (Korn SHell) – The Korn Shell also was the base for the POSIX Shell standard specifications etc.

  Each shell does the same job but understand different commands and provide different built in functions.

  Shell Scripting

  Usually shells are interactive that mean, they accept command as input from users and execute them. However some time we want to execute a bunch of commands routinely, so we have type in all commands each time in terminal.
  As shell can also take commands as input from file we can write these commands in a file and can execute them in shell to avoid this repetitive work. These files are called Shell Scripts or Shell Programs. Shell scripts are similar to the batch file in MS-DOS. Each shell script is saved with .sh file extension eg. myscript.sh

  A shell script have syntax just like any other programming language. If you have any prior experience with any programming language like Python, C/C++ etc. it would be very easy to get started with it.
  A shell script comprises following elements –

  • Shell Keywords – if, else, break etc.
  • Shell commands – cd, ls, echo, pwd, touch etc.
  • Functions
  • Control flow – if..then..else, case and shell loops etc.

  Why do we need shell scripts

  There are many reasons to write shell scripts –

  • To avoid repetitive work and automation
  • System admins use shell scripting for routine backups
  • System monitoring
  • Adding new functionality to the shell etc.

  Advantages of shell scripts

  • The command and syntax are exactly the same as those directly entered in command line, so programmer do not need to switch to entirely different syntax
  • Writing shell scripts are much quicker
  • Quick start
  • Interactive debugging etc.

  Disadvantages of shell scripts

  • Prone to costly errors, a single mistake can change the command which might be harmful
  • Slow execution speed
  • Design flaws within the language syntax or implementation
  • Not well suited for large and complex task
  • Provide minimal data structure unlike other scripting languages. etc

  Simple demo of shell scripting using Bash Shell

  If you work on terminal, something you traverse deep down in directories. Then for coming few directories up in path we have to execute command like this as shown below to get to the “python” directory –
  

  Источник

  Adam the Automator

  Your One and Only Linux Shell Scripting Tutorial

  

  Shanky

  Read more posts by this author.

  If you’re on a Linux machine and need to automate Linux shell commands, this Linux shell scripting tutorial is for you. In this tutorial, you’ll learn everything you need to turn Linux shell scripting into your bread-and-butter with automation.

  Table of Contents

  Prerequisites

  To follow along with this tutorial, it is necessary to have a remote SSH host. This tutorial uses an Ubuntu 18.04.5 LTS machine with sudo/administration rights.

  What is a Shell?

  Before jumping into shell scripting, it’s necessary to understand what shell is. A shell is a command language interpreter that executes commands read from the standard input device such as a keyboard or a file to execute programs, create files, etc.

  Whenever a user opens up the terminal (a console to a shell), the shell issues a command prompt where you’ll type your input. That input is further executed as you hit the Enter key, displaying the output on the terminal.

  You can see below that the kernel communicates between the hardware and software throughout the shell script workflow.

  There are majorly four kinds of Unix shell as listed below:

  • Bourne Shell: The Bourne shell (sh) is a shell command-line interpreter for computer operating systems.
  • C Shell: C Shell is a Unix shell with an overall language style more like C.
  • Korn Shell: Korn Shell (aka ksh) is the default shell on many UNIX systems.
  • Bourne-Again Shell: Bourne Again Shell, also known as Bash, is Bourne Shell’s successor with additional functionalities, such as command-line completion, basic debugging, etc. Bash is the default shell environment for most GNU/Linux systems.

  Perhaps you want to view all the shells available on your Ubuntu machine. To do so, launch your terminal and run the cat command below.

  Below, the cat command prints out the

  /etc/shells file’s content.

  In the output below, you can see that multiple shells are available. Later, you’ll see how to indicate which shell to use when running a script.

  Creating a Shell Script

  Understanding what a shell is wouldn’t be complete unless you experience a shell script in action. So let’s work on creating your first shell script. Perhaps you want to start with a basic script to print messages on the console.

  1. SSH into your Ubuntu VM using your favorite SSH client or open a terminal if you’re already on a Linux machine.

  2. Find the running shell by running the echo «$SHELL» command.

  In the below image, you will notice the output says /bin/Bash, which indicates Bash is running.

  3. Change into the

  /opt directory first and create a shell script file named my_first_shell_script.sh by running the following commands.

  Due to security concerns, running or installing your software or programs inside the opt directory is recommended.

  4. Now, copy and paste the code below to the my_first_shell_script.sh file and save it.

  In the script below, you can see the shebang ( #! ) written in the first line of the script. The shebang is a character sequence that tells the kernel which shell to use to execute the script; in this case, it’s Bash ( /bin/bash ).

  Configuring File Permission to Run a Script as an Executable

  After creating the script file, by default, you only have read and write access to it. A shell will only run scripts you have executable permission for. So, before running the script you created, you must first configure the script as executable. Otherwise, you’ll get a “Permission denied” error message when you run the script.

  The chmod command below lets you add the execute ( x ) permissions to all users ( a ) running the script ( my_first_shell_script.sh ).

  After saving your script, refer to any of the methods below to run your script:

  • bash script.sh or /bin/bash script.sh – Indicates the shell to use and executes the script from the working directory.
  • bash /$HOME/script.sh – Indicates the shell to use and execute the script by specifying the script’s full path.
  • ./script.sh – Executing the script from the working directory

  For this example, run the command below to execute your script, assuming that your script is in the working directory.

  Below, you can see that the script printed the words Hello World in the console.

  Working with User-Defined Variables

  Now you know how to create and run a script, let’s look into declaring variables. In every programming language, variables play an important role. In Linux shell scripting, you’ll run into more than one type of variable, but let’s cover user-defined variables for this example.

  Perhaps you don’t want to keep writing the same string or value over and over in a script. Declaring user-defined variable stores a value in memory to substitute to a string or another value.

  You can see below the basic syntax of declaring a user-defined variable. Notice that there are no spaces before and after the equal ( = ) operator; otherwise, you’ll get an error. Why? Because the shell will interpret the variable_name as a command, not a variable.

  Below, you can see the error message saying that the variable_name command is not found.

  Let’s see how to substitute string outputs with variables.

  Copy and paste the script below to a script file, configure the script as executable like you previously did, and run the script.

  The code below declares the year and sitename variables to hold different types of values. It then prints a message where strings are substituted by year and sitename variables.

  Working with Environment Variables

  Unlike user-defined variables, you can set environment variables system-wide. Environment variables are then made available to all shells. For instance, an application that creates logs could access your USERNAME and your $HOME directory.

  Like other variables, you can set your own environment variable, but predefined environment variables are already set in the Unix shell. Check out the list of predefined environment variables with their corresponding values by running the env command.

  In the image below, you’ll notice that all the variables are defined in variable=values format.

  Let’s see how environment variables work in a script.

  Create a script file named variables.sh in the

  /opt directory, copy and paste the code below to the variable.sh file. Configure the script as executable and run it.

  Below, you can see that you’re substituting a string with the $PWD environment variable, pointing to the present working directory.

  You can see below that the script executes from the

  Setting Your Own Environment Variable

  You can also set your own environment variables. By default, your own environment variables are only accessible in the current shell session. Still, you can also make environment variables accessible outside the current shell session with the instructions below.

  Setting Environment Variables for the Current Shell

  By default, environment variables are only accessible within the same shell session you declare them in. To create a session-specific environment variable, use the export command, as shown below.

  The command below is created an environment variable called VARNAME with a value of my value .

  Setting Environment Variables with .bashrc

  When you just use the export command inside a script as above, the variable will not be available to any other script. To make an environment variable available globally, declare the variable in the .bashrc file in your $HOME directory.

  The .bashrc file contains various shell configuration values for each session created. These configuration items include setting up or enabling: coloring, completion, shell history, command aliases, and more.

  To create global environment variables, open the .bashrc file in a text editor and write the variable below.

  Below, run the source command to execute the content of the .bashrc file to apply the changes you made to the file. Then, run the printenv command piped with the grep command to print the variable name and its value.

  Setting Environment Variables with /etc/environment

  You can also set system-wide environment variables with the /etc/environment file. The /etc/environment file is specifically meant for system-wide settings like the one you’re about to set for your own.

  1. To modify the /env/environment file, open the

  /etc/environment file in your preferred text editor. The example below uses the nano text editor.

  2. In the /env/environment file, set the variable’s value, as shown below, save it, and that’s it! All users and processes can now access the variable ( VARNAME ) anytime.

  3. Execute the /etc/environment file and notice that the VARNAME variable is available and will be across all future sessions.

  Below, you can see a comparison table to understand better the scope of setting your own environment variables.

  Method	Scope
  The export command	The environment variable is only accessible for the current user in the current shell and processes
  The .bashrc file	The environment variable is permanently accessible for the current user in all future Base sessions and processes
  The /etc/environment file	The environment variable is permanently accessible for all users in all Bash sessions and processes

  Understanding Special Variables

  Special variables are just like other variables, except that you can’t assign values to them, as they are reserved for specific functions. Perhaps you’re trying to substitute strings based on a specific order. In that case, use special positional variables to set the strings’ position.

  Let’s test the special positional variables in a script. Copy and paste the script below to a script file named variable.sh, configure it as executable, and run it.

  The code below declares a variable with a string value. It then prints out the string value, followed by two other strings based on the declared positional variable.

  Check out few other special variables below and what purpose they serve.

  • $! – Provides the PID of the most recently executed command in the background.
  • $$ – Provides ****the PID of the Bash shell itself.
  • $1 , $2 , $3 ,… – Provides the number of positional parameters passed in the script.
  • $0 – Provides the name of the shell script.
  • $n – Corresponds to the arguments with which a script was invoked.
  • $? – Provides the ****exit status of the most recently executed command and check whether your Bash script is completed successfully or not.
  • $_ – Provides ****the absolute file name of the shell or Bash script, which is being executed as specified in the argument list.

  Getting and Setting Exit Codes in a Shell Script

  All processes in Linux exit with a specific code known as an exit code. Exit codes are integers that indicate the status of a process, typically if it was successful or failed somehow. In Bash, you can override these default exit codes and set your own.

  Within a Bash script, you can both read exit codes and set custom ones as well. Let’s first read an exit code. To do that, read the value of the special variable $? by echoing out its value with the echo command.

  In the following example, the cat command tries to get the contents of a file ( myfile.txt ) that doesn’t exist. The echo command then prints out the exit status of the cat command.

  Below, you can see the returned error message saying that the myfile.txt file doesn’t exist, along with the exit code (1).

  Now, let’s see how to use the exit codes to tell whether the command ran successfully or not with a condition.

  The code below tries to create a directory named mydir but with an invalid command. Since the command is invalid, the $? will not have a success exit code of 0. You can see that the if statement checks for an exit code and performs some action depending on the value within the $? special variable.

  You can see below that the script failed and returned an error message saying the “Failed creating mydir directory.”

  Below are the other exit codes to specify when getting or changing an exit status:

  • 1 – Catches all the general errors
  • 2 – Misuse of shell built-ins
  • 126 – Command invoked cannot execute
  • 128 – Invalid argument passed
  • 130 – Script terminated by Control-C

  Creating a Shell Function to Run Series of Commands

  Now you know how to identify script errors with the exit status. Let’s create a function in your script without worrying you’ll run into errors.

  Perhaps you realized that you’re duplicating commands over and over in a script. In that case, a shell function can help save time and increase your script efficiency. A function is a block of code that only executes when called from any parts of your shell script.

  Below is the basic code block of a shell function. Notice that you first need to declare the function name and then write the series of commands in the curly braces.

  Now, let’s see a shell function in action.

  Create a shell script called Shell_function_demo.sh in opt directory, then copy and paste the script below to the Shell_function_demo.sh file. Configure the script as executable and run it.

  The script below calls a function that will print first and last name strings based on the substituted value by the positional parameters ( $1 and $2 ).

  After running the script, you’ll notice that the function printed two different echo statements with the substituted positional parameters.

  Viewing File Names and Size via a For Loop

  Like the function in the previous section, a for loop is also a code block that performs specific tasks but is iterated over a set of items. For instance, the for loop will run until it reaches the last file in a directory.

  Perhaps you need to check the size of each file in the working directory. If so, a short script with a for loop will suffice.

  Let’s check out below an example of viewing each file’s size in the working directory. To do so, create a script file looping_over_file.sh, then copy and paste the below code to the looping_over_file.sh file. Configure the script as executable and run it.

  The code below scans each file in the current directory and prints out its name and size in bytes until all files are scanned.

  You can see below that each file size is listed, followed by the file name.

  Requiring User Input in a Running Shell Script

  So far, you’ve learned scripts that are focused on commands and variables. But with advanced-level projects, you would often require users to input data at runtime. The user input data could be anything, such as username and password in a program asking to fill in security information, for example.

  Let’s cover how to require user input while the shell script is running.

  Create a new script file under the

  /opt directory and name it user_input.sh, then copy and paste the code below to the user_input.sh file. Now configure the script as executable and run it.

  The script below allows the user to input a username and password, then print a confirmation message.

  You can see below the input for the username is visible, while the input for the password is not.

  Running Code While a Condition is True

  Unlike the for loop, a while loop runs code while a certain condition is true. Perhaps you’re writing code to print a set of incrementing numbers. In that case, a while loop would be ideal for executing the code.

  Let’s check out an example below and learn how while loop works.

  Create a script file first with your preferred text editor, and name it while_loop.sh. Save the code below to the script file, and run it to see how the while loop iterates, printing incrementing numbers.

  The code below declares a variable ( a ) with an initial value of 100 , then incrementing by two while the current value is less than 110 .

  After running the script, you can see below the numbers are printed and incremented by two. Notice that the printed numbers only reached 108 since the condition set to the while loop increases the value while less than 110 ( [$a -lt 110] ).

  Printing Colored Output from a Shell Script

  By now, you’ve learned a handful of scripts. But this time, let’s cover how to change the color of a script’s printed output.

  Perhaps you need specific output such as username, password prompt, or errors to look different to recognize them quickly. Let’s check out how to change an output’s color with a shell script.

  Copy and paste the code below to script file named colored_output.sh in the

  /opt directory. Be sure the file is configured as executable, and run it to see how you get colored outputs.

  Notice in the script below that the string values start in \\e[0;00m format to set the color and end with the \\e[0m format to set the color back to default.

  After running the script, you can see below different colored outputs.

  If the colors the tutorial used aren’t to your liking, you can always choose between different colors, as you can see below.

  Colors are represented by color codes as follows:

  Escaping Characters

  If you noticed in the previous scripts, string values were enclosed in double quotes ( » » ), single ( ‘ ‘ ) quotes, and left quotes (). These quotes play an important role while writing a shell script, but if incorrectly placed, the script will not work as intended.

  Perhaps you want to include double quotation marks in your string output. Escaping a character with the backslash symbol in a string will do the trick. The contents inside quotes are considered a string except for the \\ or ! symbols and variables.

  Let’s check out how to include double quotes in a string output.

  Create a script file named doublequotes.sh in the

  /opt directory, then copy and paste the script below to the doublequotes.sh file. Configure the script as executable and run it.

  The code below prints string values with the basic double-quoted string escaped double quotation marks and substituted value from a variable in the script below.

  You can see below that there are double quotes included in the second and third output.

  Printing Special Characters in a String with Single Quotes

  When you insert special characters in a string in double-quotes, Bash interprets those special characters and returns their value. But, what if you need to include special characters or quotes inside of a string? Use single-quotes.

  Perhaps you need to include escape characters or variables in the string, declare them inside single quotes.

  Let’s check how string values in single quotes work. Save the code below to a script file named single_quotes.sh file and run it.

  Anything within the single quote is considered a string in the code below.

  After running the script, you will notice that the $owner variable and escape character ( \ ) are printed out as strings this time.

  Running Command Inside a String with Left Quotes

  Now that you know how to print out string values in double quotes and single quotes. Let’s look at how left quotes work. Unlike double and single quotes, left quotes focus only on declaring a command within a string.

  Perhaps, you need to print out today’s date included in a string using the date command. In that case, using the left quotes is the ideal option to print the date regardless of which quotes enclose the string value. Let’s go over how to write the left quotes inside a string value.

  Below, the code declares two variables with strings enclosed in single and double quotes where the date command is included in the script below.

  You can see below that both string variables printed the same output.

  Conclusion

  You have now acquired rock-solid knowledge from this Linux shell scripting tutorial, from shell scripting basics to executing loops. You’ve also learned how shell scripts automate many tasks, which help save time and make developers’ lives much easier.

  Now take your shell scripting skills up a notch, and start your own project by combining each script you created in this tutorial.

  More from Adam The Automator & Friends

  

  With an ever-increasing visitor base searching for solutions to their problems, we can help you connect with and reach the right kind of audience and generate targeted sales leads.

  

  We’ve put together a list of the resources we, at ATA, can wholeheartedly recommend!

  

  Why not write on a platform with an existing audience and share your knowledge with the world?

  Источник

  Читайте также:  Что такое консоль сервера windows