Search and replace is such a common task that it should be a tool that’s in every command line script author’s toolbox.
There are probably endless solutions to the problem.
I’ve put together my standard methods for tackling the problem, with Perl. You can also use Python, of course, as well as other tools like sed.

Use Models

In most of the following discussion, I’m just replacing ‘foo’ with ‘bar’.
However ‘foo’ can be ANY regular expression.

The dummy situation I’m going to set up is this.
I’ve got a file named example.txt. It looks like this:

foo foo fiddle foo 
and baz too, as well as foo

I realize that I want to replace ‘foo’ with ‘bar’.

I’d like to be able to have a script that runs in a way that I can either pipe the contents of the file into the script, or just give a file name to the script, like this:

  • cat example.txt | perl <something>
  • perl <something> example.txt

Now, I may or may not want to have the script modify the file in place.
If not, then the second example above would just print the modified contents.
I also may want to make a backup of example.txt first.
We can all of those things, as I’ll show below.

There are lots of ways to implement this in Perl.

On the Command Line

The following are all roughly equivalent:

  • cat example.txt | perl -pe 's/foo/bar/g;'
  • cat example.txt | perl -ne 's/foo/bar/g;print'
  • cat example.txt | perl -e 'while (<>){s/foo/bar/g;print;}'
  • cat example.txt | perl -e 'while (<>){$_ =~ s/foo/bar/g;print;}'
  • perl -pe 's/foo/bar/g;' example.txt
  • perl -ne 's/foo/bar/g;print' example.txt
  • perl -e 'while (<>){s/foo/bar/g;print;}' example.txt
  • perl -e 'while (<>){$_ =~ s/foo/bar/g;print $_;}' example.txt

flags used

  • -e : execute the code on the command line
  • -n : wrap everything in while (<>){ [code here] }
  • -p : wrap everything in while (<>){ [code here]; print; }

As a Script

Of course, you don’t have to use the ‘-e’ flag.
You can stick the script in a file.

The following scripts are equivalent:

#!/usr/bin/perl 
while (<>) {
  $_ =~ s/foo/bar/;
  print $_;
}

#!/usr/bin/perl 
while (<>) {
  s/foo/bar/;
  print;
}

#!/usr/bin/perl -n
s/foo/bar/;
print;

#!/usr/bin/perl -p
s/foo/bar/;

Changing files in place

Perl also allows changing the files in place, with the addition of the ‘-i’ flag, usually used as ‘-i.bak’ to make backup copies of the original:

perl -pi.bak -e 's/foo/bar/g;' example.txt