Linux input from file

Unix / Linux — Shell Input/Output Redirections

In this chapter, we will discuss in detail about the Shell input/output redirections. Most Unix system commands take input from your terminal and send the resulting output back to your terminal. A command normally reads its input from the standard input, which happens to be your terminal by default. Similarly, a command normally writes its output to standard output, which is again your terminal by default.

Output Redirection

The output from a command normally intended for standard output can be easily diverted to a file instead. This capability is known as output redirection.

If the notation > file is appended to any command that normally writes its output to standard output, the output of that command will be written to file instead of your terminal.

Check the following who command which redirects the complete output of the command in the users file.

Notice that no output appears at the terminal. This is because the output has been redirected from the default standard output device (the terminal) into the specified file. You can check the users file for the complete content −

If a command has its output redirected to a file and the file already contains some data, that data will be lost. Consider the following example −

You can use >> operator to append the output in an existing file as follows −

Input Redirection

Upon execution, you will receive the following result −

The following script runs a session with the vi text editor and saves the input in the file test.txt.

If you run this script with vim acting as vi, then you will likely see output like the following −

After running the script, you should see the following added to the file test.txt −

Discard the output

Sometimes you will need to execute a command, but you don’t want the output displayed on the screen. In such cases, you can discard the output by redirecting it to the file /dev/null −

Here command is the name of the command you want to execute. The file /dev/null is a special file that automatically discards all its input.

To discard both output of a command and its error output, use standard redirection to redirect STDERR to STDOUT −

Here 2 represents STDERR and 1 represents STDOUT. You can display a message on to STDERR by redirecting STDOUT into STDERR as follows −

Redirection Commands

Following is a complete list of commands which you can use for redirection −

Output of pgm is redirected to file

Output of pgm is appended to file

Output from stream with descriptor n redirected to file

Output from stream with descriptor n appended to file

Merges output from stream n with stream m

Источник

How to write the output into the file in Linux

How do I save terminal output to a file?

A command can receive input from a file and send output to a file.

Writing the output into the file

The syntax is
command > filename
For example, send output of the ls command to file named foo.txt
$ ls > foo.txt
View foo.txt using the cat command:
$ cat foo.txt
Please note that when you type ‘ls > foo.txt’, shell redirects the output of the ls command to a file named foo.txt, replacing the existing contents of the file. In other words, the contents of the file will be overwritten.

Appending the output or data to the file

The syntax is
command >> filename
For example the following will append data:

• No ads and tracking
• In-depth guides for developers and sysadmins at Opensourceflare✨
• Join my Patreon to support independent content creators and start reading latest guides:
  • How to set up Redis sentinel cluster on Ubuntu or Debian Linux
  • How To Set Up SSH Keys With YubiKey as two-factor authentication (U2F/FIDO2)
  • How to set up Mariadb Galera cluster on Ubuntu or Debian Linux
  • A podman tutorial for beginners – part I (run Linux containers without Docker and in daemonless mode)
  • How to protect Linux against rogue USB devices using USBGuard

Join Patreon ➔

Verify it:
cat /tmp/data.txt

How to save the output of a command to a file in bash using tee command

The tee command read from standard input and write to standard output and files. The syntax is as follows for writing data into the file:
command | tee file.txt
Want to append data? Try
command | tee -a output.txt

Examples

Display output of the date command on screen and save to the file named /tmp/output.txt. If the output.txt already exists, it gets overwritten:
$ date | tee /tmp/output.txt
$ cat /tmp/output.txt
Same as above but append to the given files, do not overwrite file:
$ pwd | tee -a /tmp/test.txt
$ echo «Today is $(date)» | tee -a /tmp/test.txt
$ hostnamectl | tee -a /tmp/test.txt
$ cat /tmp/test.txt

The above commands will append the output to the end of the file, just like the shell >> operator as explained earlier.

I/O redirection summary for bash and POSIX shell

Sr.No.	Command & Description
1

Shell operator	Description	Overwrite existing file?
command > output.txt	Save terminal output (standard output) to a file named output.txt	Yes
command >> output.txt	Append terminal output (standard output) to a file named output.txt	No
command	Takes standard input from output.txt file	N/A
command 0	Takes standard input from output.txt file	N/A
command 1> output.txt	Puts standard output to output.txt file	Yes
command 1>> output.txt	Appends standard output to output.txt	No
command 2> output.txt	Puts standard error to output.txt	Yes
command 2>> output.txt	Appends standard error to output.txt file	No
command &> output.txt	Puts both standard error and output to output.txt	Yes
command > output.txt 2>&1	<POSIX> Puts both standard error and output to file named output.txt	Yes
command &>> output.txt	Appends both standard error and output to file named output.txt	No
command >> output.txt 2>&1	<POSIX> Appends both standard error and output to file called output.txt	No
command | tee output.txt	Puts standard output to output.txt while displaying output on screen	Yes
command | tee -a output.txt	Appends standard output to output.txt while displaying output on screen	No
command |& tee output.txt	Puts both standard output and error to output.txt while displaying output on terminal	Yes
command 2>&1 | tee output.txt	<POSIX> Puts both standard output and error to file named output.txt while displaying output on terminal	Yes
command |& tee -a output.txt	Append both standard output and error to file called output.txt while displaying output on terminal	No
command 2>&1 | tee -a output.txt	<POSIX> Append both standard output and error to file named output.txt while displaying output on terminal	No

Conclusion

You learned how to write the output to the file in Linux or Unix-like system when using bash or POSIX shell. We have:

1. /dev/stdin (standard input) — File descriptor 0 is duplicated.
2. /dev/stdout (standard output) — File descriptor 1 is duplicated.
3. /dev/stderr (standard error) — File descriptor 2 is duplicated.

See I/O redirection documentation for more information. We can read bash man page as follows using the man command:
man bash

🐧 Get the latest tutorials on Linux, Open Source & DevOps via

Источник

Linux/UNIX: Bash Read a File Line By Line

Syntax: Read file line by line on a Bash Unix & Linux shell:

Tutorial details
Difficulty level	Easy
Root privileges	No
Requirements	Bash/ksh/zsh/tcsh
Est. reading time	10m

1. The syntax is as follows for bash, ksh, zsh, and all other shells to read a file line by line
2. while read -r line; do COMMAND; done
3. The -r option passed to read command prevents backslash escapes from being interpreted.
4. Add IFS= option before read command to prevent leading/trailing whitespace from being trimmed —
5. while IFS= read -r line; do COMMAND_on $line; done

How to Read a File Line By Line in Bash

Here is more human readable syntax for you:

The input file ( $input ) is the name of the file you need use by the read command. The read command reads the file line by line, assigning each line to the $line bash shell variable. Once all lines are read from the file the bash while loop will stop. The internal field separator (IFS) is set to the empty string to preserve whitespace issues. This is a fail-safe feature.

• No ads and tracking
• In-depth guides for developers and sysadmins at Opensourceflare✨
• Join my Patreon to support independent content creators and start reading latest guides:
  • How to set up Redis sentinel cluster on Ubuntu or Debian Linux
  • How To Set Up SSH Keys With YubiKey as two-factor authentication (U2F/FIDO2)
  • How to set up Mariadb Galera cluster on Ubuntu or Debian Linux
  • A podman tutorial for beginners – part I (run Linux containers without Docker and in daemonless mode)
  • How to protect Linux against rogue USB devices using USBGuard

Join Patreon ➔

How to use command/process substitution to read a file line by line

Process or command substitution means nothing more but to run a shell command and store its output to a variable or pass it to another command. The syntax is:

Using a here strings

How to file descriptor with read command

The syntax is simple:

Here is a sample script:

Examples that shows how to read file line by line

Here are some examples:

The same example using bash shell:

You can also read field wise:

Fig.01: Bash shell scripting- read file line by line demo outputs

Bash Scripting: Read text file line-by-line to create pdf files

My input file is as follows (faq.txt):

Read from bash shell variable

Let us say you want a list of all installed php packages on a Debian or Ubuntu Linux, enter:

You can now read from $list and install the package:

Conclusion

This page explained how to read file line by line in bash shell script.

🐧 Get the latest tutorials on Linux, Open Source & DevOps via

Category	List of Unix and Linux commands
Documentation	help • mandb • man • pinfo
Disk space analyzers	df • duf • ncdu • pydf
File Management	cat • cp • less • mkdir • more • tree
Firewall	Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04
Linux Desktop Apps	Skype • Spotify • VLC 3
Modern utilities	bat • exa
Network Utilities	NetHogs • dig • host • ip • nmap
OpenVPN	CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04
Package Manager	apk • apt
Processes Management	bg • chroot • cron • disown • fg • glances • gtop • jobs • killall • kill • pidof • pstree • pwdx • time • vtop
Searching	ag • grep • whereis • which
Shell builtins	compgen • echo • printf
Text processing	cut • rev
User Information	groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w
WireGuard VPN	Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04

Comments on this entry are closed.

The bash example is correct for all shells. The ksh example is not correct; it will strip leading and trailing whitespace.

To demonstrate, use:

on a file with leading and/or trailing whitespace on one or more .

The BASH password file parsing example does not work for my centos 7 box: all the colons are stripped from each line and the whole line without colons is stored in f1
my output reads:
Username: root/root/bin/bash, Home directory: , Shell:

Reading a file line by line with a single loop is fine in 90% of the cases.

But if things get more complicated then you may be better off opening the file as a file descriptor.

Situations such as:

* the read of the file is only a minor aspect and not the main task, but you want to avoid opening an closing the file multiple times. For example preserving where you have read up to or avoiding file closure (network, named pipes, co-processing)

* the file being read requires different methods of parsing, that is better done in simpler but different loops. Reading emails (header and body) is an example of this, but so is reading files with separate operational and data blocks.

* you are reading from multiple files at the same time, and switching between different files, in a pseudo-random way. That is you never quite know whch file you will need to read from next.

I have a scenario like this. I have a file which has servername, port and ipaddress
svr1
port1
ip1
svr2
port2
ip2
…
…

I tried couple of ways reading these records and assigning them a field and print the output to a file like below but nothing is helpful

servername port ipaddress
svr1 port1 ip1
scr2 port2 ip2
and so on

can someone please help me.

I figured it out. May be not professional but it is working for me.

if the file is 100% predictable in each host,ip,port coming in groups of 3 you can use “paste”

cat list.txt | paste – – –

HOW DO YOU SPLIT INPUT INTO FIELDS FOR PROCESSING?

I have a question concerning how you’d actually process individual fields of input.
For example, let’s say you’re doing the usual

while read LINE; do
process the statements
done firstbyteinq and looking at firstbyteinq with a read, but 1). the cut doesn’t seem to be doing it just to the record being read, it does it to ALL the records all at one time; 2). the read of the firstbyteinq then advances the other file one record.

It seems to me there ought to be some way to “declare” the positions from a to b as fields and THEN use the while read, but I’m not sure how to do that.
I’ll try anything. Thanks VERY much in advance.

princess anu, there’s no need to use arithmetic; just read the file in groups of three lines:

A correction; you need to check that the read succeeded; checking the first should be enough:

I have scenario :

One file which contain values like:
ABS
SVS
SGS
SGS

another file is template which has to use values from 1st file (One value at a time, line by line )

ARGS=ABS >> save in one file

New file:
ARGS=SVS >> save in another file

Can you please help me to achive this

u can use paste command if the fields are same …..and then output the records to the other file and print another.

rm 3 4 5
first=1
seco=2
cat 1 | while read l
do
echo “$l =” >> 3
done
paste 3 2 | while read line
do
echo “$line” >> 4
done
awk ‘’ 4 > 5

now u can move the records line by line to another file

echo “$line” # Chris F.A. Johnson Jul 3, 2013 @ 11:40

Some shells shells will output -n; that is what POSIX requires.

My while/read is having an issue with the data not being printed for the last line read.
The info is read and summed but is not being printed to the report.
The script is reading a file with tab delimited fields, sorting on a field then reading each line while assigning the fields to var names.
If statements determine whether the numbers are summed or if a new category is found with the previous categories data being written to the report. All categories except the last are complete in the report. The last category is left out.
I added trouble shooting echo statements to check the workflow and they showed that the last category is being read and summed but it doesnt get printed.
It’s as if the read gets to the last line, does the work and then doesnt know what to do with the output.
The sorted data looks something like:

Code with troubleshooting echos and most comments deleted:

The read name is different than the last line (new category)

cat | sort -k3 | while read -r Count IP Name

sort -k3 | while read -r Count IP Name

It is also possible to store the elements in a bash array using the -a option

The following example reads some comma separated values and print them in reverse order:
input.txt

Hi guys. I’m just getting started with scripting and I am needing some help reading a file into the script. This shell script below works and does what it needs to do but I have to manually input the file name for “read list”. How do I modify this script to automatically read the file. Inside the file is just a list of aix server names.

#!/usr/bin/sh
echo ” Please input the master list you know to use”
read list
for i in `cat $list`
do
ssh $i hostname
ssh $i df
done

Hi all,
I need to write a script to look for user in /etc/passwd file. The program takes user input from command line and check to see if the user that the person entered exists. If user doesn’t exist, the script sends error message. I use grep to print the user which exist. But not sure what command to use and look and if the user doesn’t exist, send error message?
Thanks,

Hi stone,
me trying to get the list of server have /mysqlshare file system in it but i am unable to get it..
for now trying get only 2 server details from txt file ie(text.txt) which have
server1_name
server2_name
when i try this script its only reading 1st server details and getting exit from script

Can someone help me out..

can anyone help me to include the line number while read line by line ,

Источник