Convert DOS to Unix: Line Endings Explained

To convert a DOS/Windows text file to Unix/Linux format, you need to change its line endings from CRLF (carriage return + line feed) to LF (line feed only). The simplest way is to run the command dos2unix filename.txt on a Unix-like system, or use sed -i 's/\r$//' filename.txt if dos2unix isn’t installed.

Background: Why DOS and Unix Differ

The difference goes back to the early days of computing. Unix systems adopted the line feed character (LF, ASCII 10, written as \n) to mark the end of a line. MS-DOS, and later Windows, kept the older convention of using both a carriage return (CR, ASCII 13, \r) and a line feed. The carriage return moves the cursor to the start of the line, and the line feed moves it down one line. This is why files created on Windows often display with a stray ^M at the end of each line when opened in a Unix editor like vim or less.

Today, most modern editors and tools handle both formats transparently, but the issue still surfaces when you run shell scripts, compile code, or process data with tools that expect strict Unix line endings. A script with CRLF endings can fail with errors like command not found because the shell sees the CR as part of the command name.

How Line Endings Work

Let’s look at the actual bytes. A simple text file with the word “hello” followed by a newline is stored as:

• Unix/Linux: hello\n (68 65 6C 6C 6F 0A in hex)
• Windows/DOS: hello\r\n (68 65 6C 6C 6F 0D 0A in hex)
• Old Mac (pre-OS X): hello\r (68 65 6C 6C 6F 0D in hex) – this is rare now but you might encounter it in very old files.

So converting DOS to Unix simply means removing the \r (0x0D) byte before each \n (0x0A). The reverse conversion (Unix to DOS) adds the \r back.

How to Convert: Command-Line Methods

Here are the most common ways to do the conversion on a Unix-like system (Linux, macOS, WSL):

1. Using dos2unix (recommended): If installed, it’s the simplest and safest. It handles multiple files, preserves encoding, and can even convert in-place.
   dos2unix file.txt (converts in place)
   dos2unix -n file.txt file_unix.txt (converts to a new file)
2. Using sed: Works everywhere without extra tools.
   sed -i 's/\r$//' file.txt (in-place, GNU sed)
   On macOS (BSD sed), use: sed -i '' 's/\r$//' file.txt
3. Using tr: Deletes all carriage returns, not just those before a newline.
   tr -d '\r' < file.txt > file_unix.txt
4. Using perl: Similar to sed, good for one-liners.
   perl -pi -e 's/\r\n?/\n/g' file.txt

For Windows users, you can use PowerShell: (Get-Content file.txt) -join "`n" | Set-Content -NoNewline file_unix.txt (this reads lines, joins them with LF, and writes without an extra newline).

Choosing the Right Method

Which method you pick depends on your situation:

• One-off file: dos2unix is easiest if you have it.
• Many files: dos2unix *.txt or find . -name "*.txt" -exec dos2unix {} \;
• No admin rights: Use sed or tr – they’re almost always present.
• Preserve the original: Use dos2unix -n or tr with output redirection.
• Binary files: Never run these on binary files – they will corrupt them. Only text files.

If you’re in a Git repository, you might not need to convert at all. Git can handle line endings automatically with core.autocrlf or a .gitattributes file. For example, setting *.txt text eol=lf in .gitattributes forces LF in the working tree.

Common Problems and Pitfalls

Here are issues you might run into:

• Script errors: A shell script with CRLF endings will fail with ./script.sh: bad interpreter: /bin/bash^M: No such file or directory. The ^M is the CR character. Convert the script first.
• Accidental removal of CR in binary files: Running tr -d '\r' on a binary file will remove every 0x0D byte, which can break the file. Always check the file type first with file.
• Mixed line endings: Some files have a mix of LF and CRLF. dos2unix handles this by converting all CRLF to LF, leaving existing LF alone. tr -d '\r' would remove all CRs, which might be fine, but if a CR is part of the data (rare), it would be lost.
• Empty files: Converting an empty file is a no-op, so no problem.
• Editing in Windows: If you edit a Unix file in Notepad, it will save as CRLF. Use an editor like VS Code, Notepad++, or Sublime Text that respects the original line endings.

Comparing with Other Conversions

Besides DOS to Unix, you might need the reverse (Unix to DOS) or conversion to old Mac format. Here’s a quick comparison:

• Unix to DOS: unix2dos file.txt or sed -i 's/$/\r/' file.txt
• DOS to old Mac: Replace CRLF with CR only: sed -i 's/\r\n/\r/g' file.txt (rarely needed).
• Old Mac to Unix: Replace CR with LF: tr '\r' '\n' < file.txt > file_unix.txt

Most modern tools (like Git, editors, and compilers) handle LF and CRLF transparently, but when you’re in a Unix environment and a tool expects LF, conversion is necessary.

Safety and Best Practices

Before converting, always back up important files. The conversion is usually lossless (you’re only removing a byte), but if the file contains intentional CR characters in the middle of lines (very rare), you could lose data. Use dos2unix which is designed to be safe, or use sed to only remove CR at the end of lines (s/\r$//) rather than all CRs.

Check the file type first: file file.txt will show “ASCII text, with CRLF line terminators” if it’s DOS. After conversion, verify with file again – it should say “ASCII text”. You can also use cat -v file.txt to see ^M characters before conversion.

If you’re working with a version control system, let the system handle line endings rather than converting manually. For example, in Git, set core.autocrlf=true on Windows to automatically convert to LF on commit and to CRLF on checkout. This avoids the need to convert files at all.

Frequently Asked Questions

How do I convert a file from DOS to Unix on Windows?

You can use PowerShell: (Get-Content file.txt) -join "`n" | Set-Content -NoNewline file_unix.txt. Or install the dos2unix tool via chocolatey or use WSL (Windows Subsystem for Linux) to run the Linux commands.

What does ^M mean in a file?

^M is how the carriage return character (CR, ASCII 13) is displayed when you run cat -v or open a DOS file in a Unix editor. It appears at the end of each line and is the extra byte that needs to be removed.

Can I convert a file without installing anything?

Yes, on most Unix-like systems, sed and tr are preinstalled. Use sed -i 's/\r$//' file.txt or tr -d '\r' < file.txt > newfile.txt. No additional software needed.

Will converting a file change its encoding?

No, converting line endings only changes the line terminator bytes. The actual text encoding (UTF-8, ASCII, etc.) remains unchanged. However, if the file is UTF-16, the line endings are different (2-byte sequences), and these methods won’t work directly – you’d need a tool that understands UTF-16.

What is the difference between CRLF and LF?

CRLF is two characters: carriage return (CR,
) and line feed (LF,
). LF is just the line feed. Unix uses LF, Windows uses CRLF. The CR moves the cursor to the beginning of the line, and LF moves it down one line. In modern systems, only LF is needed to indicate a new line.

How do I convert all files in a directory recursively?

Use find . -type f -exec dos2unix {} \; or find . -type f -exec sed -i 's/\r$//' {} \;. Be careful to exclude binary files – you can add a condition like -name "*.txt" or use file to check.

Related

• Do Smartphones Reduce Brain Power? Facts
• Qnap Data Recovery: What Works and What Doesn’t
• Samsung OEM Lock: What It Means & How to Fix It