Search and Replace Across Multiple Files (Perl)
I suggest that testers learn at least one scripting language. The first one that I learned was perl. Yesterday, I was reminded why a little bit of perl can save a lot of time (and eliminate the possibility of human error). Due to some changes in our corporate email system, we needed to change several hundred datafiles from using one email address to a different email address. While there are many to do this (editor macros, shell programming using tools like sed, etc.) I decided some perl executed from the command line would be appropriate.
I started with:
perl -p -i -w -e 's/oldemail\@meesqa.com/newemail\@meesqa.com/g;' *.data
For those unfamiliar with perl, this line can be explained pretty quickly:
perl — executes the perl interpreter
-p — assume loop like structure. Essentially execute all input to the line as if this were inside a while loop.
-i — edit files "in place" – Change the current copy of the files. It is possible to create a backup copy. I didn’t because all of the data files were checked into source code control.
-w — enable warnings
-e (following be some perl code) — The one line program to execute
‘s/searchvalue/replacevalue/g’ — This is the generic form of the program. The g at the end means to apply this expression globally — replacing all instances of searchvalue with replacevalue. In the code above, I needed to escape the @ character so it would be interpreted correctly.
*.data — Finally, what input to pass into the program, in this case all files that match the pattern *.data
After running this in one directory and examining the output, I realized that we had not been consistent with case for the email address domain. While I had fixed all of the instances of @meesqa.com, I missed @MEESQA.COM (or any other permutation of upper and lower case. Easy enough. I made the following minor change:
perl -p -i -w -e 's/oldemail\@meesqa.com/newemail\@meesqa.com/gi;' *.data
The i added after the /g tells perl to ignore case. Rerunning and all of the email addresses in that directory had been changed appropriately.
That fixed one directory of files. However, the rest of the files were scattered over a deep hierarchy of directories. Since I was doing this all on unix, I executed the following line:
find . -name *.data | xargs perl -p -i -w -e 's/oldemail\@meesqa.com/newemail\@meesqa.com/gi;'
The find . -name *.data creates a list of all of the *.data files in the current directory and subdirectories. This is piped to the next command. The xargs command allows this piped list of filenames to be passed in as an argument to the command following xargs — in this case my perl program. Running this was fairly quick. I verified the changes and checked into source while be reminded that a little scripting can go a long way.
1 Comment
Thanks! this was a great help.. especially because i have just started using/learning perl.