keepcalculator

06 Numbers and Powers

Sed Delete Lines: How to Remove Lines with sed

Card No. 06·80514 · Updated · Standard formula, shown below

To delete lines with sed, use the d command: sed 'Nd' file deletes line N, and sed '/pattern/d' file deletes every line matching a pattern. This article explains the syntax, common use cases, and pitfalls so you can remove lines from files quickly and safely.

What Is sed?

sed (stream editor) is a Unix utility that processes text line by line. It reads a file (or standard input), applies a script of commands, and writes the result to standard output (or a file if you redirect it). The d command tells sed to delete the current line, meaning it is not printed to the output. Because sed works on a stream, it can handle files of any size without loading the whole file into memory.

sed is available on Linux, macOS, and Windows Subsystem for Linux (WSL). The commands shown here use the GNU version of sed (default on Linux) and the BSD version (default on macOS) with only minor differences, which we will note where relevant.

Basic Deletion Syntax

The general form is:

sed 'ADDRESSd' file

where ADDRESS can be a line number, a range, a regular expression, or a combination. The d command deletes the line that matches the address. Here are the most common forms:

  • sed '5d' file – deletes the 5th line.
  • sed '1,5d' file – deletes lines 1 through 5.
  • sed '/error/d' file – deletes every line containing “error”.
  • sed '/^#/d' file – deletes every line that starts with a # (often used to remove comments).
  • sed '$d' file – deletes the last line (the $ symbol represents the last line).

You can combine addresses with a comma for a range, and you can use regular expressions as endpoints. For example, sed '/start/,/end/d' deletes from the line containing “start” through the line containing “end”.

Deleting Lines by Pattern

Pattern-based deletion is the most common use case. The pattern is a regular expression (regex) that sed matches against each line. If the pattern matches, the line is deleted. For example, to remove all lines with “DEBUG” in a log file:

sed '/DEBUG/d' app.log

You can also delete lines that do not match a pattern using the ! operator. For instance, sed '/keep/d' file deletes lines that contain “keep”, but to delete lines that do not contain “keep”, you would write sed '/keep/!d' file. This is equivalent to printing only lines that match, but it is useful when you want to keep the deletion logic in one command.

Regular expressions are powerful but can be tricky. For example, to delete lines that begin with a space or tab, use sed '/^[ \t]/d' file. To delete lines that are empty (or contain only whitespace), use sed '/^[[:space:]]*$/d' file. Note that the [[:space:]] character class is more portable than \s across different sed versions.

Deleting Lines by Number and Range

Line numbers are straightforward: sed '10d' file deletes the 10th line. You can also delete a range: sed '10,20d' file deletes lines 10 through 20. The range can use $ to mean the last line, so sed '10,$d' file deletes from line 10 to the end of the file.

You can also use a step value with the GNU sed syntax: sed '1~2d' file deletes every other line starting from line 1 (i.e., odd lines). This is a GNU extension and may not work on BSD sed (macOS). For a portable way to delete every other line, you can use sed 'n;d' file (which prints the first line, then deletes the second, and repeats), but that is a different approach.

When deleting a range, sed keeps track of the range as it processes lines. For example, sed '/start/,/end/d' file deletes from the first line containing “start” through the next line containing “end”. If “end” never appears, it will delete to the end of the file. This behavior can be surprising, so test with a small file first.

Modifying the File In-Place

By default, sed writes to standard output, so to change the file itself you must redirect the output or use the -i option. The -i option edits the file in place. On GNU sed (Linux), you can simply write:

sed -i '5d' file.txt

On BSD sed (macOS), you must provide a backup extension (even if empty) right after -i:

sed -i '' '5d' file.txt

The empty string means no backup file is created. If you want a backup, provide a suffix like .bak (on both versions, though the syntax differs slightly). For example, on Linux: sed -i.bak '5d' file.txt creates file.txt.bak before modifying the original.

Always test without -i first to see the output. A common mistake is to run sed -i on a file and accidentally delete more lines than intended because the pattern was too broad. Make a backup or use -i.bak until you are confident.

Common Mistakes and Pitfalls

  • Forgetting to redirect output: Without -i or a redirect, sed prints to the terminal and leaves the file unchanged.
  • Using the wrong pattern delimiter: If your pattern contains a slash (/), you need to escape it or use a different delimiter. For example, to delete lines containing “/usr/local”, write sed '\/usr\/local/d' file or use a different delimiter like sed '|/usr/local|d' file (GNU sed allows any character as delimiter).
  • Deleting more than intended with ranges: If the start pattern appears multiple times, the range will start and stop repeatedly. For example, sed '/start/,/end/d' on a file with “start” on line 1, “end” on line 3, “start” on line 5, and no “end” after that will delete lines 1-3 and then lines 5 to the end (because the second range never closes).
  • Not escaping special characters in patterns: In basic regular expressions (BRE), characters like ., *, [, and ^ have special meaning. If you want to match a literal dot, you must escape it: sed '/\./d' file deletes lines containing a period.
  • Using extended regex without -E: Some patterns like + or ? are only available in extended regular expressions, which you enable with -E (or -r in some versions). For example, sed -E '/[0-9]+/d' deletes lines with one or more digits.

Alternatives to sed for Deleting Lines

sed is not the only tool for deleting lines. Depending on your needs, you might use:

  • awk: awk 'NR!=5' file prints all lines except line 5. For pattern deletion: awk '!/error/' file. awk is more readable for complex conditions.
  • grep: To delete lines that match a pattern, you can use grep -v 'pattern' file. This is simpler for pure pattern removal, but it does not handle line numbers or ranges as easily.
  • ed: The line editor ed can delete lines with 5d and then write the file with w. It is interactive but useful in scripts.
  • Perl: perl -ni -e 'print unless /pattern/' file is a one-liner that modifies the file in place and is more powerful for complex regexes.

For most simple deletions, sed is the fastest and most portable. For complex logic, awk or Perl may be easier to maintain.

Frequently Asked Questions

How do I delete multiple lines that are not consecutive?

You can specify multiple addresses separated by semicolons: sed '1d;3d;5d' file deletes lines 1, 3, and 5. For patterns, use multiple expressions: sed '/foo/d;/bar/d' file deletes lines containing “foo” or “bar”.

How do I delete the last line of a file?

Use sed '$d' file. The $ symbol matches the last line. This works in both GNU and BSD sed.

How do I delete lines between two patterns, but only the first occurrence?

Use a range with a counter or use awk. A simple sed range will delete all occurrences. To delete only the first block, you can use a command like sed '0,/end/d' file which deletes from line 1 to the first “end”, but this is not a true “start” pattern. A more robust approach is to use awk: awk '/start/{flag=1} !flag; /end/{flag=0}' file but that prints lines until “start”. For deletion, you need to adjust the logic.

Why does sed ‘s/foo/bar/’ not delete lines?

The s command substitutes text, not delete lines. To delete lines, you must use the d command. If you want to delete lines that match a pattern, use sed '/pattern/d'.

Can I delete lines in multiple files at once?

Yes, you can pass multiple files to sed, but without -i the output is concatenated. With -i, each file is modified in place. For example, sed -i '/error/d' file1.txt file2.txt deletes lines containing “error” from both files.

How do I delete blank lines?

Use sed '/^$/d' file to delete lines that are completely empty. To also delete lines with only whitespace, use sed '/^[[:space:]]*$/d' file.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →