AWK - Recompiling $0 where the record is just spaces results in unexpected behavior

AWK's default "FS" variable is a single space " "
A common trick people use to remove unnecessary whitespace in a record is to force a recompilation of the record using:

{ $1 = $1 }

Which indeed trims whitespace.

However, if you have a line which is comprised ENTIRELY of whitespace AWK will trim it down to nothing (Which is odd as you'd expect a single ' ' to be left) and for some reason AWK will now report that a field now exists, when there's literally nothing there.

Input File:

    hello
    charles
           <--- This is a series of spaces
one
two
 three

AWK Script:

#!/usr/bin/awk -f

{ $1 = $1; print }

Output:

hello
charles

one
two
three

Now this looks correct, until you get AWK to report on it's field count using the "NF" variable:

New AWK Script:

#!/usr/bin/awk -f

{ $1 = $1; print NF }

Output:

1
1
1
1
1
1

Where the heck is this field coming from? When I run the output through the cat utility to check for line endings it reports nothing there:

dev@pop-os:~/Scripts/awk$ ./space_test.awk spaces.txt | cat -e
hello$
charles$
$
one$
two$
three$

As you can see, nothing is there.

What's extra odd is that AWK also reports that the length of this record is zero:

Modified AWK Script:

#!/usr/bin/awk -f

{ $1 = $1; print length($0) }

Output:

5
7
0
3
3
5

What is going on here?

When you assign to a field number, that field and all the fields before it are created. This will increase NF if it's more than the original number of fields.

It doesn't matter that the fields are empty, they're still counted.

$ awk '{print NF; $2 = ""; print NF}' <<< ""
0
2

> AWK's default "FS" variable is a single space " "

Yes, but this prompt GNU AWK to consider one-or-more whitespace characters to be field separator, consider that if you have file.txt with fields delimitied by tripled spaces

1   2   3
4   5   6
7   8   9

then

awk '{print NF,$3}' file.txt

gives output

3 3
3 6
3 9

therefore $1=$1 does not only trim leading and trailing whitespace characters but also will change all whitespace characters with space and then squeeze multiple spaces together, consider that for example if you have TAB-separated file file.tsv like

1	2	3
4	5	6
7	8	9

then

awk '{$1=$1;print}' file.tsv

gives output

1 2 3
4 5 6
7 8 9

Observe that there is not TAB character in output.

as a consequence of

> When you assign to a field number, that field and all the fields before it are created...

as mentioned by @Bramar, one of the less-discussed ways to use awk is leveraging those empty fields and repeat stuff with ease :

> seq 17 | mawk '(OFS = ((__=$_) + 9) % 10)^!(NF += __)' FS='^$'

10
211
3222
43333
544444
6555555
76666666
877777777
9888888888
109999999999
1100000000000
12111111111111
132222222222222
1433333333333333
15444444444444444
165555555555555555
1766666666666666666

or just repeat emojis :

> jot 17 | mawk '(OFS = "\360\237\244\241")^!(NF += $!_)' FS='^$'

1🤡
2🤡🤡
3🤡🤡🤡
4🤡🤡🤡🤡
5🤡🤡🤡🤡🤡
6🤡🤡🤡🤡🤡🤡
7🤡🤡🤡🤡🤡🤡🤡