Linux bash replace string

How to Replace a String in a File in Bash

Sales.txt

Date Amount Area

01/01/2020 60000 Dhaka
10/02/2020 76000 Rajshahi
21/03/2020 54000 Khulna
15/04/2020 78000 Chandpur
17/05/2020 45000 Bogra
02/06/2020 67000 Comilla

Replace String in a File with the `sed` Command

The basic syntax of the `sed` command for replacing the particular string in a file is given below.

Syntax

Every part of the above syntax is explained below.

‘-i’ option is used to modify the content of the original file with the replacement string if the search string exists in the file.

‘s’ indicates the substitute command.

‘search_string’ contains the string value that will be searched in the file for replacement.

‘replace_string’ contains the string value that will be used to replace the content of the file that matches the ‘search_string’ value.

‘filename’ contains the filename where the search and replace will be applied.

Example 1: Replace File with the ‘sed’ Command

In the following script, the search-and-replace text will be taken from the user. If the search string exists in ‘Sales.txt’, then it will be replaced by the replacement string. Here, a case-sensitive search will be performed.

# Assign the filename
filename = «Sales.txt»

# Take the search string
read -p «Enter the search string: » search

# Take the replace string
read -p «Enter the replace string: » replace

if [ [ $search ! = «» && $replace ! = «» ] ] ; then
sed -i «s/ $search / $replace /» $filename
fi

Output

Example 2: Replace File with the ‘sed’ Command with ‘g’ and ‘i’ Flag

The following script will work like the previous example, but the search string will be searched globally for the ‘g’ flag, and the case-insensitive search will be done for the ‘i’ flag.

# Take the search string
read -p «Enter the search string: » search

# Take the replace string
read -p «Enter the replace string: » replace

if [ [ $search ! = «» && $replace ! = «» ] ] ; then
sed -i «s/ $search / $replace /gi» $1
fi

Output

Example 3: Replace File with ‘sed’ Command and Matching Digit Pattern

The following script will search for all numerical content in a file and will replace the content by adding the ‘$’ symbol at the beginning of the numbers.

# Check the command line argument value exists or not
if [ $1 ! = «» ] ; then
# Search all string containing digits and add $
sed -i ‘s/\b4\<5\>\b/$&/g’ $1
fi

Output

Replace String in a File with `awk` Command

The ‘awk’ command is another way to replace the string in a file, but this command cannot update the original file directly like the ‘sed’ command.

Example 4: Replace File with ‘awk’ Command

The following script will store the updated content in the temp.txt file that will be renamed by the original file.

# Check the command line argument value exists or not
if [ $1 ! = «» ] ; then
# Search all string based on date
awk ‘1′ $1 > temp.txt && mv temp.txt $1
fi

Output

Conclusion

This article showed you how to use bash scripts to replace particular strings in a file. The task to replace a string in a file should become easier for you after practicing the above examples.

About the author

Fahmida Yesmin

I am a trainer of web programming courses. I like to write article or tutorial on various IT topics. I have a YouTube channel where many types of tutorials based on Ubuntu, Windows, Word, Excel, WordPress, Magento, Laravel etc. are published: Tutorials4u Help.

Источник

Bash Shell: Replace a String With Another String In All Files Using sed and Perl -pie Options

/foo directory has 100s of text file and I would like to find out xyz string and replace with abc. I need to to use sed or any other tool to replace all occurrence of the word under Linux and Unix-like operating systems. How do we replace words in files?

How to use sed command to replace a string with another string

The syntax is as follows:
sed -i ‘s/ old-word / new-word /g’ *.txt
GNU sed command can edit files in place (makes backup if extension supplied) using the -i option. If you are using an old UNIX sed command version try the following syntax:
sed ‘s/ old / new /g’ input.txt > output.txt
The — option also know as replace in place. We can use old sed syntax along with bash for loop as follows:

A Note About Bash Escape Character

A non-quoted backslash \ is the Bash escape character. It preserves the literal value of the next character that follows, with the exception of newline. If a \newline pair appears, and the backslash itself is not quoted, the \newline is treated as a line continuation (that is, it is removed from the input stream and effectively ignored). This is useful when you would like to deal with UNIX paths. In this example, the sed command is used to replace UNIX path “/nfs/apache/logs/rawlogs/access.log” with “__DOMAIN_LOG_FILE__”:

The $_r1 is escaped using bash find and replace parameter substitution syntax to replace each occurrence of / with \/.

Replace one substring for another string in shell

Say you need to find ‘/foo/bar’ and replace with ‘/home/bar’. Here is what we do in sed:
sed ‘s+/foo/bar+/home/bar+g’ input
sed ‘s+/foo/bar+/home/bar+g’ input > output
sed -i’.Bak’ ‘s+/foo/bar+/home/bar+g’ input
We used the + instead of / as delimiter to avoid errors. See “how to use sed to find and replace text in files in Linux / Unix shell” for more info.

Replace strings in files with bash string manipulation operators only

No need to use the sed or Perl. We can replace the first occurrence of a pattern/string with a given string. The syntax is for bash version 4.x+:

For instance, replace a string ‘Unix’ with ‘Linux’:

Here is another bash string manipulation example that replaces all occurrences of ‘Linux’ with ‘Unix’:

Replacing a string in a file in Bash variable when using sed

Often our shell scripts have variables, and we need to pass those variables to sed. For instance:

How to replace an array of values using sed

Let us use the cat command to see our current template file that contains AWS RDS info config for PHP app:
cat config_template.txt
Herer is what I have:

My task is to replace ‘ __NAME__ ‘ and other place holder values using bash script:

How to replace a string in a File in Perl

The perl -pi -e syntax to find and replace a string is as follows:
perl -pi -e ‘s/ old-word / new-word /g’ input.file
perl -pi -e ‘s/old/new/g’ *.conf
Where,

• -p : act like sed
• -i : edit file in place
• -e : One line Perl programe

When Perl executed, it will look all files matching *.conf . Then it will run the replacement pattern ‘ s/old/new/g ‘. In other words, we used Perl regular expression to eplaces all occurrences of old with new .

Summing up

In this tutorial, we learned how to use sed, bash, and Perl to find and replace strings in a single or bunch of files. See man pages for more info:

• No ads and tracking
• In-depth guides for developers and sysadmins at Opensourceflare✨
• Join my Patreon to support independent content creators and start reading latest guides:
  • How to set up Redis sentinel cluster on Ubuntu or Debian Linux
  • How To Set Up SSH Keys With YubiKey as two-factor authentication (U2F/FIDO2)
  • How to set up Mariadb Galera cluster on Ubuntu or Debian Linux
  • A podman tutorial for beginners – part I (run Linux containers without Docker and in daemonless mode)
  • How to protect Linux against rogue USB devices using USBGuard

Join Patreon ➔

🐧 Get the latest tutorials on Linux, Open Source & DevOps via

Category	List of Unix and Linux commands
Documentation	help • mandb • man • pinfo
Disk space analyzers	df • duf • ncdu • pydf
File Management	cat • cp • less • mkdir • more • tree
Firewall	Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04
Linux Desktop Apps	Skype • Spotify • VLC 3
Modern utilities	bat • exa
Network Utilities	NetHogs • dig • host • ip • nmap
OpenVPN	CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04
Package Manager	apk • apt
Processes Management	bg • chroot • cron • disown • fg • glances • gtop • jobs • killall • kill • pidof • pstree • pwdx • time • vtop
Searching	ag • grep • whereis • which
Shell builtins	compgen • echo • printf
Text processing	cut • rev
User Information	groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w
WireGuard VPN	Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04

Comments on this entry are closed.

Glancing at this I see a typo:
sed “s/$OLD/$NEW/g” “$f” > $TFILE && mv TFILE “$f”

Should be:
sed “s/$OLD/$NEW/g” “$f” > $TFILE && mv $TFILE “$f”

Also, you didn’t seem to mention that sed can auto-create backup files using syntax like this:
sed -i.bak ‘s/old-word/new-word/g’ filename.txt

That will edit filename.txt and copy the original to filename.txt.bak

thanks for the heads up!

I don’t know normal sed has -i.bak option but perl provides the backup of the original file.

perl -p -i.bak -e ‘s/AAA/aaa/’ file_name

Just a small remark:
perl -pie ‘s/old-word/new-word/g’ input
modifies the input file, so no need for ‘ > output’ like with sed.

Thanks for the great tip. Heres how I did it using a pipe to provide file paths:

find . -name ‘*.xaction’ | xargs sed -i ‘s/test-data/live-data/g’

Thank you, this was useful!

Recursively replace a word in multiple files in multiple directories?
I couldn’t find a recursive option in sed, so …

grep -ilr ‘old-word’ * | xargs -i@ sed -i ‘s/old-word/new-word/g’ @

Which will find each file with the old-word in it, complete with path, then pipe through xargs and run sed.
Does the trick.

@Matthew, that worked perfectly for recursive find and replace. Thanks.

I had to do about 5 things to make this work in OSX Snow Leopard. NOTE THAT COPYING DIRECTLY MIGHT NOT WORK. This is because the single quote characters could turn into curly quotes after posted. What I ended up with is

grep -ilr ‘old-word’ * | xargs -I@ sed -i ” ‘s/old-word/new-word/g’ @

1. Curly single quotes had to become straight single quotes.
2. Lowercase -i doesn’t appear in xargs in this OSX version, had to use uppercase -I (and spent a long time trying to figure out that @ wasn’t the problem because it didn’t have some special meaning).
3. OSX didn’t like the missing backup suffix, it interpreted this to mean that replacement pattern was the suffix and that @ was the command. Now that I think of it, how does any OS get that the pattern isn’t the suffix? Maybe the those were accent marks that were changed into curly single quotes?

Ok, only three things, but it felt like an eternity trying to figure it out. Actually, it was probably over an hour.

Well… thanks for your post. Your hour made it like a 5 min. work to me. Seems like ubuntu has the same issue. I just had to replace the curly apostrophes 😉

Hello,
I’m a linux newbie.
I’m trying to change a string which is a windows path file (e.g. C:\Documents and Settings\path) into a unix path-file (e.g. /home/path). The file containing the string I want to change is a textual file (a bibtex file: file.bib). This files containes the addresses of other files, and they appear as: “C\:\\Documents and Setting\\path\\”). I’m trying with the “sed” command but apparently unix do not like the “\\”. Someone knows the solution?

This wont work when I try to use it for replacing a work having “/” in it.
I use a script to change password for other machine. But once it a while the problem comes like
New password is “dkCHQ4/uTfSIU” old paswd “14eMXYEwUD7Cc”

sed ‘s/14eMXYEwUD7Cc/dkCHQ4/uTfSIU’ /etc/passwd > $newpassfile
it throws error.

You can change the / from the sed option to any char you want. Try this

sed ‘s|14eMXYEwUD7Cc|dkCHQ4/uTfSIU’ /etc/passwd > $newpassfile

I think it’s better to understand then escape all the ‘/’s in the string like Mattheu suggested.

@Suvankar, you need to escape backslashes with forward slashes.

sed ’s/14eMXYEwUD7Cc/dkCHQ4\/uTfSIU’ /etc/passwd > $newpassfile

hi,
i need to remove the SOP in the code. My code contains a call to another function immediately after SOP call. please suggest how can i find the first occurance of ‘;’ in the line and remove the text till first occurance of ‘;’

System.out.println(“06”); String sDebitTxnmemo= cm.getString(“********”);

replace OldString NewString — *.txt

****** BE CAREFUL LOL…. YOU WILL DELETE THE ENTIRE FILE IF IT IS A .CC FILE WITH “sed s/”old”/”new”/g name.cc > name.cc

hi
i want search for a word in a file and if found ,i want to add a word at the begning of that line where the word is found….. is there any command for that?

Hi Nahas,
something like this will do the job

cat the.file|perl -pne ‘s/^(thewordtofind)/TheWordToAdd$1/’

I have a template file, and would like to replace some words in that file with words taken from another file like:
File1
Hi $1 I miss being with you at $2
File2
Johan London
So the output will be a new file contains
Hi Johan I miss being with you at London.

You can read data into variable using cut, sed, awk, and any other shell builtin:

Hi All,
We had a situation to find and replace a word in huge number of files. But if we create a new file with the sed command, it may create space issue. So we are asked to remove the old file once this replacement happens and retain only the modified file.
You may make use of the following script:
> vi change.sh

> nohup sh change.sh (run this in background)
> tail -f Change.log (To track changes)

Thank you for these useful tips. Same question as Suvankar with a twist:
A=/path1
B=/path2
sed -i ‘s/$A/$B/g’ f.txt
throws an error.
I tried escaping with A=\/path1 to no avail.
Any further tip for such a case? Thanks
PS Also, what is the difference between using single and double quotes above? Thanks again.

Should be as follows:

You need to escape / so that sed will not confuse with s/ command. See the difference between single and double quote here:

a] Enclosing characters in single quotes preserves the literal value of each character within the quotes
b] Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of ‘$’, ‘`’, ‘\’, and, when history expansion is enabled, ‘!’. The characters ‘$’ and ‘`’ retain their special meaning within double quotes.

That works perfectly (even with spaces in the path). Thank you.
So the single quotes in A=’\/path1′ prevent interpreting the escape character until it is used within double quotes, right?

Now my script has matured a bit. Here is where I am at. This attempts to delete files in a tree if they are absent from another directory tree.

Any clue how to soft-code the paths? I have run out of ideas to try.
Thanks again.

I have a problem could anyone tell me a way to open the access.log file which does not have permission to read or write. I need to read that file for monitoring purpose. I cannot use chmod command to change the permission as it says permission denied. I came to know that there is a shell command to open it. It will be great help if anyone give any idea in perl or php to read that information.

thanks in advance

Shameel,
Use sudo chmod. If you don’t have super user access, then you might not be able to open this file.

I could not understand the following line can u explain its functionality.

if [ -f $f -a -r $f ]; then //explain only this one whole script are below

#!/bin/bash
OLD=”xyz”
NEW=”abc”
DPATH=”/home/you/foo/*.txt”
BPATH=”/home/you/bakup/foo”
TFILE=”/tmp/out.tmp.$$”
[ ! -d $BPATH ] && mkdir -p $BPATH || :
for f in $DPATH
do
if [ -f $f -a -r $f ]; then
/bin/cp -f $f $BPATH
sed “s/$OLD/$NEW/g” “$f” > $TFILE && mv $TFILE “$f”
else
echo “Error: Cannot read $f”
fi
done

I have a dictionary file:
sme,replace
me,place

And on the processing file, i have:
123456,sme,89
678901,me,78

Need to replace values in processing file with corresponding values from dictionary file. e.g. sme -> replace

Would appreciate on this.

Hi All,
I am facing an issue wherein i need to replace a word with another one. But my case is a bit different here. Only a part of the word is constant which I know. The other half is variable ( too many possibilities ). I need to change the complete word with a new word. For example
The possible old words may be
abc123ad, abc123bc, abc123ak and so on….
i need to replace any of the above in my file with a new word.

Please help :(.
Thanks in advance.

Thanks,
I guess i have found it…..
sed ‘s/abc123[aa-zz]*/new_word/g’ filename

If any of the files fed through xargs are binary, both sed and perl will change the timestamps of the files that all have the string, binary files or text files, but no changes are made (in at least the one file *I* was looking at at the top of the file directory hierarchy)

So filter out binary files.

Hi All,
I want to replace a string in a file and redirect the output to a new file,but this is slightly tricky.The input file contains data as A|B|C|D|E|F|G|H|I|K , now i want to replace alphabet “F” to “:” and redirect the output file to a file.The final output must look like this:A|B|C|D|E|:|G|H|I|K. Please help it’s urgent

It means i create a database using mysql and create a website using joomla so when i click a link from the web i can directly retrieve the data from mysql server.

I need help immediately. I need to monitor multiple web servers access.log file. I have completed the coding for single server completely but the problem we are facing is let us imagine two servers 192.47.155.200 and 192.47.155.36. The server 192.47.155.200 has database now how to fetch the information from 192.27.155.36 being in that server and insert in that database. I want to create a link between two servers that it does not ask for password everytime fetch the data by itself as i am running cron job. I have access to both the server as i am the admin of it. I am doing coding in php.

Ok , sorry for this stupid question but someone can tell me how i can “delete” “/home/” for example i use

echo $HOME
/home/user_name

i need to delete “/home/” to get “username” without “whoami” or something like that

Источник