Use Shell Script Variable in awk

Yes, you can use a shell script variable inside an awk program, and the cleanest, safest way is with the -v option: awk -v var="$shell_var" '...'. This passes the variable as an awk variable before the program runs, so you can reference it by name inside the script without worrying about quoting or escaping issues.

Why the -v Option Is the Best Choice

The -v option assigns a value to an awk variable before the first input record is read. It is part of the POSIX awk standard, so it works in all modern awk implementations (gawk, mawk, nawk, BusyBox awk, and the original Bell Labs awk). The syntax is simple:

awk -v var="$shell_var" 'BEGIN { print var }'

Because the shell expands $shell_var before awk even starts, the value is passed as a command-line argument. The -v option handles the quoting internally, so you don’t need to worry about spaces, special characters, or newlines in the value. This is the recommended approach in most scripting guides, including the GNU awk manual.

Other Ways to Pass Variables

There are three other common methods, each with its own trade-offs:

Command-line assignment: awk '...' var="$shell_var" input.txt. This assigns the variable after the program is compiled but before the first record is processed. It works the same as -v for most cases, but it is not available inside a BEGIN block because the assignment happens after BEGIN runs. It also has a subtle difference: if the variable name is the same as a built-in (like FS or NR), the command-line assignment overrides it, while -v would cause an error.
Environment variables: export var="$shell_var"; awk 'BEGIN { print ENVIRON["var"] }'. This works because awk has an ENVIRON array that contains all environment variables. It is useful when you need to pass many variables or when the value contains characters that are tricky to quote, but it is slower and less direct.
Embedding directly in the program: awk '{ print "'"$shell_var"'" }'. This is the worst approach because it breaks if the variable contains spaces, quotes, or backslashes. It also makes the code hard to read and maintain. Avoid it unless you are writing a one-liner for a known, simple value.

How to Choose the Right Method

For almost every real-world script, use -v. It is the most portable, the most readable, and the least error-prone. Use command-line assignment only if you need to override a built-in variable like FS and you are not using a BEGIN block. Use ENVIRON when you want to pass a large set of variables or when you want to avoid any possible quoting issues entirely. The embedded approach is a last resort for quick one-liners where you are certain the value is a simple word or number.

Common Pitfalls and How to Avoid Them

The most frequent mistake is forgetting that the shell expands variables inside double quotes but not inside single quotes. If you write awk '{ print $shell_var }', awk will interpret $shell_var as a field reference, not as a shell variable. The shell variable is not expanded because the single quotes prevent it. Always use double quotes around the shell variable when passing it with -v:

awk -v var="$shell_var" '...'

Another pitfall is passing a variable that contains a newline. With -v, the newline is preserved and awk will print it as part of the variable. With the embedded approach, a newline would break the awk program. Also, be careful with backslashes: in the shell, "$shell_var" does not interpret backslashes, so they are passed literally. That is usually what you want.

Real-World Examples

Let’s look at a few practical scenarios.

Filtering a log file by a date:

date="2024-03-15"
awk -v d="$date" '$0 ~ d' app.log

This prints all lines that contain the date string. The variable d is used in a regular expression, and the -v option ensures it is treated as a literal string.

Calculating a sum with a multiplier:

rate=2.5
awk -v r="$rate" '{ sum += $1 * r } END { print sum }' numbers.txt

Here the shell variable rate is passed as r and used in arithmetic.

Using an environment variable:

export USER_ID=42
awk 'BEGIN { print ENVIRON["USER_ID"] }'

This prints 42 without needing -v. It is useful when the variable is already exported in the shell.

Performance and Portability

There is no meaningful performance difference between -v and command-line assignment; both are effectively free. The ENVIRON array is slightly slower because it requires a hash lookup, but for typical scripts the difference is negligible. Portability is the real concern: -v is POSIX, so it works everywhere. Command-line assignment is also POSIX, but the behavior with BEGIN blocks can vary between implementations. The embedded approach is not portable at all and should be avoided in any script that might be run on a different system.

Frequently Asked Questions

Can I use a shell variable inside an awk BEGIN block? Yes, but only if you pass it with -v or use ENVIRON. Command-line assignment happens after BEGIN, so it is not available there. For example, awk -v var="$x" 'BEGIN { print var }' works.

What if my shell variable contains spaces? With -v, spaces are preserved because the shell passes the value as a single argument. For example, name="John Doe"; awk -v n="$name" 'BEGIN { print n }' prints John Doe correctly.

How do I pass multiple shell variables to awk? Use multiple -v options: awk -v a="$var1" -v b="$var2" '...'. There is no limit to the number of variables you can pass.

Is there a difference between awk -v var="$x" and awk '...' var="$x"? Yes, mainly in the BEGIN block. The -v assignment is available in BEGIN; the command-line assignment is not. Also, if the variable name is a built-in like FS, the command-line assignment changes it, while -v would cause a fatal error.

Can I use a shell variable as a field separator? Yes, you can pass it with -v and then assign it to FS in a BEGIN block, or use command-line assignment directly: awk -F "$sep" '...' works too, but if the separator contains special regex characters, you need to escape them.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →