Which efficient & portable shell statement on GNU/Linux can zero-pad piped bytes to word boundary?

I need to pad NUL bytes at the end of a byte stream exceeding available storage & memory, so output length is divisible by N. Context of the function I am implementing:

#!/bin/sh
generate_arbitrary_length | paddingN | work_with_padded

Working code for N=8192:

padding8192(){ dd status=none bs=8192 conv=sync ; }

But reducing copy block size is orders of magnitude slower for small N, this did not finish:

padding4(){ dd status=none bs=4 conv=sync ; }

I can express the counting & padding using wc and dd, after duplicating the input stream:

padding4(){ { { tee /dev/fd/3 >&2 ; } 3>&1 | wc -c | { read -r isize ; pad=$(( 4 - isize % 4)) ; [ 0 -lt $pad ] && dd status=none if=/dev/zero bs=$pad count=1 >&2 ; } } 2>&1 ; }

Much faster already. But very difficult to read - who could even tell why padding ends up at EOF?

Any better approach?

Though I only need to keep as much state as needed to store byte count modulo word size, I cannot think of a simple yet performant implementation using shell builtins. Dependencies should remain minimal: using GNU coreutils/cpio/tar, no compiler/perl/features that would differ between busybox/dash/bash. I have not come up with an awk solution as I failed to make it perform well (G/s) on binary input not evenly NL/NUL-separated into lines.

Since you mention there's a compiler available, here's a tiny, portable C program. It does not get any faster and memory-economic. It's even readable for most people in the programming community. If not, you can always sprinkle /* Comments! */.

#!/bin/sh
#
# pad.sh - pad input, reading in large blocks from stdin, writing stdout.

# padding $1:padchar $2:alignment $3:blocksize
padding () {
aout="./a$$.out"
cc -x c -o "$aout" - <<EOF
#include <stdio.h>

int main (void) {
  size_t align = $2, nwritten = 0, nread;
  char buffer[$3];
  while ((nread = fread (buffer, 1, sizeof buffer, stdin)) > 0)
    nwritten += fwrite (buffer, 1, nread, stdout);
  if ((nwritten % align) != 0)
    for (align -= nwritten % align; align != 0; --align)
      putchar ($1);
  return 0;
}
EOF
"$aout" && rm "$aout"
}

printf '%s' 123456789 | padding 0       4 16384                  | od -c
printf '%s' abcdefghi | padding "'\n'" 16 BUFSIZ                 | od -c
printf '%s' PAGE_SIZE | padding 65     32 "$(getconf PAGE_SIZE)" | od -c

In action:

$ ./pad.sh
0000000    1   2   3   4   5   6   7   8   9  
$ ./pad.sh
0000000    1   2   3   4   5   6   7   8   9  \0  \0  \0
0000014
0000000    a   b   c   d   e   f   g   h   i  \n  \n  \n  \n  \n  \n  \n
0000020
0000000    P   A   G   E   _   S   I   Z   E   A   A   A   A   A   A   A
0000020    A   A   A   A   A   A   A   A   A   A   A   A   A   A   A   A
0000040
  
$ ./pad.sh
0000000    1   2   3   4   5   6   7   8   9  \0  \0  \0
0000014
0000000    a   b   c   d   e   f   g   h   i  \n  \n  \n  \n  \n  \n  \n
0000020
0000000    P   A   G   E   _   S   I   Z   E   A   A   A   A   A   A   A
0000020    A   A   A   A   A   A   A   A   A   A   A   A   A   A   A   A
0000040
  
$ ./pad.sh
0000000    1   2   3   4   5   6   7   8   9  \0  \0  \0
0000014
0000000    a   b   c   d   e   f   g   h   i  \n  \n  \n  \n  \n  \n  \n
0000020
0000000    P   A   G   E   _   S   I   Z   E   A   A   A   A   A   A   A
0000020    A   A   A   A   A   A   A   A   A   A   A   A   A   A   A   A
0000040

0000014
0000000    a   b   c   d   e   f   g   h   i  \n  \n  \n  \n  \n  \n  \n
0000020
0000000    P   A   G   E   _   S   I   Z   E   A   A   A   A   A   A   A
0000020    A   A   A   A   A   A   A   A   A   A   A   A   A   A   A   A
0000040

If you are concerned about the non-POSIXly compiler option -x c you can easily write the C program to pad.c and compile it from there. Advanced error handling for fwrite, fread and putchar left to the reader.

Note how the here-document avoids main having to parse arguments. You can even pass strings like PAGE_SIZE if your stdio makes them available by default.

I just realized that compiling C like this is not much different from a nifty awk script -- awk also compiles an internal program and then executes it. What's better than compiling to the machine's CPU and running the executable?

The POSIX thing to do would be to use a temporary file.

padding() (
   tmpf=$(mktemp) &&
   trap 'rm "$tmpf"' EXIT &&
   tee "$tmpf" &&
   isize=$(wc -c <"$tmpf") &&
   pad=$(( $1 - isize % $1 )) &&
   if [ "$pad" -ne 0 ]; then
       dd status=none if=/dev/zero bs="$pad" count=1
   fi
)