To remove empty lines from a file or command output in Bash, you can use grep, sed, awk, or a simple while loop. The most common and fastest method is grep -v '^$' file, which prints every line that is not empty.
What Counts as an Empty Line?
Before choosing a command, decide what you mean by “empty.” A truly empty line has zero characters and only a newline at the end. However, many files contain lines that look empty but actually have spaces or tabs. Those are not empty—they contain whitespace. The commands below treat these two cases differently:
- Zero-length line:
^$in regex matches a line with nothing between start and end. - Whitespace-only line:
^[[:space:]]*$matches lines that have only spaces/tabs (or nothing).
Using grep to Remove Empty Lines
The grep command is the simplest tool for this task. Use the -v flag to invert the match, meaning it prints lines that do not match the pattern.
grep -v '^$' file.txt
This prints the file without empty lines to standard output. To save the result back to the same file, you need a temporary file:
grep -v '^$' file.txt > tmp && mv tmp file.txt
If you also want to remove lines that contain only spaces or tabs, use:
grep -v '^[[:space:]]*$' file.txt
Using sed to Delete Empty Lines
sed (stream editor) can delete lines that match a pattern. The command /^$/d deletes every empty line.
sed '/^$/d' file.txt
To edit the file in-place (on GNU sed, which is standard on Linux), add the -i flag:
sed -i '/^$/d' file.txt
On macOS (BSD sed), use sed -i '' '/^$/d' file.txt.
To also remove whitespace-only lines, use sed -i '/^[[:space:]]*$/d' file.txt.
Using awk to Filter Non-Empty Lines
awk is a text-processing tool that can filter lines based on the number of fields. An empty line has zero fields, so the condition NF > 0 selects only lines with at least one field (i.e., non-empty).
awk 'NF > 0' file.txt
This automatically treats lines with only spaces as empty because NF counts fields separated by whitespace. To write back to the file, use a temporary file or sponge from moreutils:
awk 'NF > 0' file.txt > tmp && mv tmp file.txt
Or use sponge if installed: awk 'NF > 0' file.txt | sponge file.txt.
Using a While Loop (Portable but Slow)
For small files or when you need to process each line individually, a while read loop works in any POSIX shell:
while IFS= read -r line; do
[ -n "$line" ] && printf '%s\n' "$line"
done < file.txt > tmp && mv tmp file.txt
The -n test checks if the string is non-empty. This method is slower than grep or sed for large files but gives you full control if you need to modify lines while filtering.
Removing Trailing Blank Lines Only
Sometimes you only want to remove blank lines at the end of a file (e.g., trailing newlines). sed can do this with a range address:
sed -i -e :a -e '/^\n*$/{$d;N;ba' -e '}' file.txt
This is a classic idiom. A simpler approach is to use perl:
perl -0pi -e 's/\n+\z/\n/' file.txt
This removes all trailing newlines and adds exactly one back. It does not affect blank lines in the middle of the file.
Common Mistakes and Pitfalls
- Using
^\s*$with grep: GNU grep supports\sas a shorthand for[[:space:]], but it is not POSIX. Use[[:space:]]for portability. - Overwriting the input file: Redirecting output to the same file you are reading (
grep ... file > file) truncates the file before reading, resulting in an empty file. Always use a temporary file orsponge. - Windows line endings: If your file has CRLF (
\r\n), a line may appear empty but contain a carriage return. Usedos2unixfirst or adjust the pattern:grep -v '^\r$'.
Frequently Asked Questions
How do I remove empty lines from a variable in Bash?
Use grep with a here-string: cleaned=$(grep -v '^$' <<< "$var"). This works for small strings. For large text, write the variable to a file first.
Does sed -i work on macOS?
Yes, but the syntax differs. macOS uses BSD sed, which requires an empty argument after -i: sed -i '' '/^$/d' file. On Linux, sed -i works without the extra argument.
What is the fastest way to remove empty lines from a huge file?
grep -v '^$' is typically the fastest because it is optimized in C and does not buffer the entire file. sed is slightly slower but still fast. Avoid shell loops for files larger than a few megabytes.
How do I remove empty lines and also trim leading/trailing spaces?
Use awk '{gsub(/^[ \t]+|[ \t]+$/, ""); if (NF) print}' file. This strips spaces and tabs from each line, then prints only non-empty lines.
Can I remove empty lines from command output without a file?
Yes, pipe the output directly: command | grep -v '^$'. This works for any command that produces text on stdout.
Why does my file still have blank lines after using grep -v '^$'?
Your blank lines likely contain spaces or tabs. Use grep -v '^[[:space:]]*$' instead. Also check for CRLF line endings as mentioned above.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.
More cards in this drawer
- Compressed Zipped Folder Access Denied? Fix It
- IPTV TV Boxes: How to Choose the Right Streaming Device
- Custom Races in Assetto Corsa: Server Setup Guide
- Dueling Grounds Events Schedule: Full Guide
- Nokia E71 Auto Lock: Step-by-Step Setup
- Evaluating Co-Packaged Optics Switches for AI Data Centers