Sed: Append file to end of line

Sed can let you replace part of a line with new content from another file. This is how I do this in three steps. You can see the code run a codingground

By using a file to insert the replacement text the content can contain special characters that would normally require a lot of extra work to escape. 

#!/bin/bash

# Replace the right portion of a line with the content of a file.
# 1) Replace the right portion of a line with a marker word
# 2) Append after the marker with the content from a file
# 3) Remove the marker and new line to pull the next line up.

cat > outfile <<!EOF
hello there tom
        How are you today?
    Where is the dog show?
!EOF

cat outfile

cat > testfile <<!EOF
flowers doing?
!EOF


sed -i outfile -e 's/you.*/REPLACE_MARKER/g'
printf "\n\n\n"
cat outfile

sed -i outfile -e "/REPLACE_MARKER/r testfile"
printf "\n\n\n"
cat outfile

sed -i outfile -e "/REPLACE_MARKER/{:a;N;s/REPLACE_MARKER\n//}"
printf "\n\n\n"
cat outfile

Truncate a file with sed

Log files grow over time but you don't want to fill your disk. Often a log file is just for simple debug and does not need to be rotated or kept in /var/log/, sometimes a simple debug log just goes well in /dev/shm/. In this case it is important to keep it short and trim off the top of the file from time to time.

I don't know why but this problem seems very difficult for a lot of people and they end up writing long complex multi line blocks of code to truncate a file and just keep the end of it. It is super simple with sed.

# only keep the last 100 lines of the log file
sed -i /dev/shm/my_debug.log -e :a -e '$q;N;100,$D;ba'

That's what I sed!

The power of the Linux CLI comes from the ability to connect thousands of programs together and make new tools. The most commonly used connector is the pipe. | It lets you send the output "STDOUT" from one program to the input "STDIN" of another. The sed program is great for taking input and changing it. The name sed is short for Stream EDitor, the stream is the data that flows in to and out of the program. sed can also edit files in place.

I use a command like this to edit the XML configuration files of Tomcat servers to point the Java program to new MySQL servers.

sed -i /usr/share/tomcat6/conf/server.xml.new 
-e '/Context docBase="someblock"/,/<\/Context>/{s/url="jdbc.*mydb/url="jdbc:mysql:\/\/'$DB_HOST':3306\/mydb/}'