SED

Anchors

Anchors Description
^ Start of string, or start of line in
multi-line pattern
\A Start of string
$ End of string, or end of line in
multi-line pattern
\Z End of string
\b Word boundary
\B Not word boundary

Character Classes

Character Description Example
\d One digit from 0 to 9
\w Word character:
ASCII letter, digit or underscore
\w-\w\w\w = A-b_1
\s Any whitespace:
space, tab, newline, carriage return,
vertical tab
\D Not a digit \D\D\D = ABC
\W Not a word character \W\W\W = *-+=)
\S Not a whitespace \S\S\S = yoyo

Quantifiers

Quantifiers Description
* 0 or more
+ 1 or more
? 0 or 1
{3} Exactly 3
{3,} 3 or more
{3,5} 3, 4 or 5

Group and Ranges

Group and Ranges Description
. Any character except new line
(a|b)
a or b
(...)
Group
(?...) Non-capturing group
[abc] Range (a or b or c)
[^abc] Not (a or b or c)
[a-z] Lower case letter from a to z
[A-Z] Upper case letter from a to z
[0-9] Digits

Match One of Several Alternatives

To remove leading and trailing whitespace from a string.

Items to match a single character

Metacharacter Matches
. Any one character
[....] Any character listed in brackets
[^...] Any character not listed in brackets
\char Escape character, used when you want
to search for a special character such as
$ "\$"

Items to match a position

Metacharacter Matches
^
Start of a line
$ End of a line
\< Start of a word
\>
End of a word

Quantifiers

Metacharacter Matches
? 0 or 1
* 0 or more
+ 1 or more
{N} Exactly N times
{N,} Match al least N times
{min,max} Match between min and max

. (any single character)

  • This particular wildcard will match any single character.
  • ‘r.d’ would match red, rod, rzd and so on..

[…] (character class)

  • Two basic ways to use character classes:
    • to specify a range
    • to specify a list of characters
  • [a-f]
    [aeiou]
  • An important point is that a character class will match only one character.

[^…] (negation)

  • Allows a user to search for anything but a specific character or set of characters
  • ..[^24680]
  • This will look for any three-character pattern that does not end in an even number.

\ (escape)

  • When placed before another metacharacter, it signifies to treat that character as the literal symbol instead of its special meaning.
  • ‘.’ match any single character
  • ‘\.’ would only match the actual “period”

^ (start of a line)

  • ‘^red’
  • This pattern would match all lines that begin with “red”, not just the ones that contain the word “red”.

M to N occurrences {m,n}
Match 1, 2 or 3 digits.

				
					$ grep '^[0-9]\{1,3\}$' number
1
12
123
				
			

Exact N occurrence {N}
Match exactly 5 digits.

				
					$ grep '^[0-9]\{5\}$' number
12345
				
			

N or more occurrences {N,}
Match 5 or more digits.

				
					$ grep '[0-9]\{5,\}' number
12345
123456
1234567
12345678
123456789
				
			

Basic sed Commands

Substitution

[address]s/pattern/replacement/flags file

[line-address (particular location or range in a file)]s/pattern/replacement/number (replaces only nth match of the pattern)

where the flags that modify the substitution are:

  • n : a number (1 to 512) indicating that a replacement should be made for only the nth occurrence of the pattern.
  • g : Make changes globally on all occurrences in the pattern space.
    Normally only the first occurrence is replaced.
  • The substitute command is applied to the lines matching the address. If no address is specified, it is applied to all lines that match the pattern, a regular expression.
  • If a regular expression is supplied as an address, and no pattern is specified, the substitute command matches what is matched by the address. This can be useful when the substitute command is one of multiple commands applied at the same address.
Replace ALL instances in the second line.
				
					sed '2s/unix/linux/g'
				
			
Replace the 2nd occurrence on each line.
				
					sed 's/unix/linux/g2'
				
			
Replace the 1st occurrence in the first line.
				
					sed '1s/unix/linux/'
				
			
Replace the LAST occurrence in the first line.
				
					sed '1s/\(.*\)unix/linux/'
				
			
Replace the 1st occurrence in the last line.
				
					sed '$s/unix/linux/'
				
			
Replace the 1st occurrence from line 1 to 5 (address range substitution)
				
					sed '1,5s/unix/liunx/'
				
			

Delete

[address]/pattern/d file

It takes an address and deletes the contents of the pattern space if the line matches the address.

The important thing to remember is: if the line matches the address, the entire line is deleted, not just the portion of the line that is matched. (To delete a portion of a line, use the substitute command and specify an empty replacement.)

Delete the 2nd line.

				
					sed '2d'
				
			

Delete a range of lines.
Delete lines other than the specified range.

				
					sed '2,4d'
sed '2,4!d'
				
			

Deleting Lines based on Pattern Match

Using a pattern to delete the line if matched.
				
					sed '/test/d'
				
			
Delete matching lines and 2 lines after the matching line.
				
					sed '/OS/,+2d'
				
			
Delete a range of lines using pattern.
				
					sed '/second/ ,/fourth/d'
				
			
Delete lines containing multiple keywords.
				
					sed '/green\|violet/d'
				
			

Delete lines but create a backup of the original file.

				
					sed -i.bak '/green\|violet/d'
sed '1d' file > newfile
				
			
Delete blank lines.
Delete lines with one or more tabs.
				
					sed '/^$/d'
sed '/^\t*$/d'
				
			

Delete lines starting with a specific character or pattern.
Delete lines ending with a specific character or pattern.

				
					sed '/^#/d'
sed '/amos$/d'
				
			

Delete lines that are in upper case.

				
					sed '/^[A-Z]*$/d'
				
			

Delete last line only if it contains the pattern.

				
					sed '${/debian/d;}'
				
			

Append, Insert, and Change

append [line-address]a\text
insert [line-address]i\text
change [line-address]c\text

The append command places it after the current line.
The insert command places the supplied text before the current line in the pattern space.
The change command replaces the contents of the pattern space with the supplied text.

The append and insert commands can be applied only to a single line address, not a range of lines. The change command, however, can address a range of lines. In this case, it replaces all addressed lines with a single copy of the text.

Append after the 2nd line.

				
					sed '2a\Appended Text'
				
			
Insert before the 2nd line.
				
					sed '2i\Inserted Text'
				
			
Modifying the 3rd line.
				
					sed '3c\Line Modified'
				
			

Trim Leading and Trailing Whitespace

To remove leading and trailing whitespace from a string.

Whitespaces:

				
					\t  # tab
\n  # new line
._. # space
\r  # carriage return
\s  # matches any of the specific whitespaces above
\S  # Any Non-whitespace
				
			

Match Leading whitespace:

				
					\A\s+
^\s+
				
			

Match Trailing whitespace:

				
					\s+\Z
\s+$
				
			

The regular expressions just shown contain three parts each:

  1. the shorthand character class to match any whitespace character (‹\s›)
  2. a quantifier to repeat the class one or more times (‹+›)
  3. and an anchor to assert position at the beginning or end of the string.
    • ‹\A› and ‹^› match at the beginning
    • ‹\Z› and ‹$› match at the end

Examples

Remove leading whitespaces
				
					sed -r 's/^\s+//'
				
			
				
					sed 's/^[ \t]+//'
				
			
Remove trailing whitespaces
				
					sed -r 's/\s+$//'
				
			
				
					sed 's/[ \t]+$//'
				
			
Remove all whitespaces
				
					sed -r 's/\s+//g'
				
			
Remove both leading and trailing whitespaces
				
					sed -r 's/^\s+//;s/\s+$//'
				
			
				
					sed 's/^[ \t]*//;s/[ \t]*$//'
				
			
Replace multiple whitespaces with single whitespace
				
					sed -r 's/\s+/ /'
				
			
				
					sed 's/[ ]\+/ /g'
				
			
Replace new line
				
					sed -z 's/\n//g'
				
			

Leave a Reply

Related Post