How to remove trailing commas from array in JSON

I am trying make a bash script to get user input, format that input into a JSON, then post that JSON to a Firebase Realtime Database using the REST API. After getting the input, I use printf statements to format the input into a JSON. The issue is that I have a trailing comma after the last entry in an array.

Here is the printf statement to format the JSON

json=$(printf '{"itemName":"%s","images":[%s],"type":[%s],"link":"/"}' "$itemName" "$(printf '"%s",' "${imgArray[@]%,}")" "$(printf '"%s",' "${typeArray[@]}")")

Here is a sample output

{"itemName":"itemname","images":["url1","url2","url3",],"type":["all","hats",],"link":"/"}

I understand the training comma comes from the %s, statement in the printf.

Is there a way to modify the printf statement to not use a comma at the last entry in the array? Or is there a way to use commands like sed to remove that trailing comma?

Assumptions:

• the 1st element in both arrays have index 0

Sample inputs:

itemName='itemname'
imgArray=(url1 url2 url3)
typeArray=(all hats)

NOTE: While there may be a way to do this with a json tool (eg, jq), I'll focus on some general bash capabilities ...

The general approach is to print the 1st element by itself, then preface the 2nd-nth elements with a comma.

Breaking the current printf calls into two parts:

# these:

$(printf '"%s",' "${imgArray[@]%,}")
$(printf '"%s",' "${typeArray[@]}")

# becomes these:

$(printf '"%s"' "${imgArray[0]}" ; printf ',"%s"' "${imgArray[@]:1}")
$(printf '"%s"' "${typeArray[0]}"; printf ',"%s"' "${typeArray[@]:1}")

This (still) requires OP's code to make 3 subprocess calls.

One idea to eliminate the subprocesses consists of using the (bash) printf builtin to populate some variables (imgs, types, json):

printf -v imgs ',"%s"' "${imgArray[@]:1}"            # preface 2nd-nth elements with a comma
imgs="\"${imgArray[0]}\"${imgs}"                     # add 1st element

printf -v types ',"%s"' "${typeArray[@]:1}"          # preface 2nd-nth elements with a comma
types="\"${typeArray[0]}\"${types}"                  # add 1st element

printf -v json '{"itemName":"%s","images":[%s],"type":[%s],"link":"/"}' "$itemName" "$imgs" "$types"

This generates:

$ typeset -p json
declare -- json="{\"itemName\":\"itemname\",\"images\":[\"url1\",\"url2\",\"url3\"],\"type\":[\"all\",\"hats\"],\"link\":\"/\"}"

$ echo "$json"
{"itemName":"itemname","images":["url1","url2","url3"],"type":["all","hats"],"link":"/"}

Install jq and use it to generate JSON files. Bash and sed are not the right tools to manipulate JSON.

In your case, building the JSON cannot be done in a single step. Let's start with the list of images:

for i in "${ITEMS[@]}"; do
  echo "$i"
done |
jq -sRc '. | rtrimstr("\n") | split("\n")'

The for command above lists the items of the $ITEMS Bash array variable one per line. The output is piped to jq that processes it as follows:

• the command line option -s tells it to "slurp" all the input data into an array instead of processing each line individually;
• the option -R means "raw input"; combined with s it tells jq to read the whole input in a big string and apply the program only once to it;
• the option -c tells it to produce compact output (the entire JSON on a single line, without extra spaces).

The short options can be combined into a single word. -sRc is the same as -s -R -c.

The jq program:

.                  # `.` always means "the current item"; this means the input string here
|                  # pipe the value produced by the previous filter to the next filter
rtrimstr("\n")     # trim "\n" on the right side of the current item
|                  # pass (the trimmed string) to the next item
split("\n")        # split the input string using newlines as the separator

The jq program basically does the opposite of the for loop but, very important, it processes raw data and produces a JSON (an array of strings).

Assuming the array is initialized as:

ITEMS=(foo bar "baz boo")

... the script above produces:

["foo","bar","baz boo"]

This allows us to write a larger command line that does what you need:

jq -n \
  --arg name "$itemName" \
  --argjson images "$(for img in "${imgArray[@]}"; do echo $img; done | jq -sRc '. | rtrimstr("\n") | split("\n")')" \
  --argjson types "$(for type in "${typeArray[@]}"; do echo $type; done | jq -sRc '. | rtrimstr("\n") | split("\n")')" \
  '{ itemName: $name, images: $images, type: $types, link: "/" }'

The command line options:

• -n tells jq to not read anything from stdin;
• --arg defines a variable (name) whose value is a raw string, the next argument); Bash computes that argument by expanding "$itemName";
• --argjson defines a variable (images) whose value is a JSON; jq parses this JSON before starting the execution of the program; "$(...)" produces the evaluation of the command between parentheses and the result is a single word; the command is the one described above, that produces an array encoded as JSON from a Bash array variable;
• { itemName: $name, images: $images, type: $types, link: "/" } is the jq program; it produces an object having the keys itemName, image, type and link, whose values are produced from the values of the jq variables set using --arg and --argjson; the value of link is the string literal /.

All in all, this big command line that invokes jq three times does what you need and you don't have to worry about trailing commas or other JSON details. All you need to care is about wrapping everything in quotes properly.

Check it online.

Update

Because the code that renders imgArray into a JSON that encodes an array of strings runs in a sub-shell, we can change the IFS and use a single echo instead of a for loop to get the same outcome faster:

IFS=$'\n'; echo "${imgArray[*]}"

The complete script becomes:

jq -n \
  --arg name "$itemName" \
  --argjson images "$(IFS=$'\n'; echo "${imgArray[*]}" | jq -sRc '. | rtrimstr("\n") | split("\n")')" \
  --argjson types "$(IFS=$'\n'; echo "${typeArray[*]}" | jq -sRc '. | rtrimstr("\n") | split("\n")')" \
  '{ itemName: $name, images: $images, type: $types, link: "/" }'

It has the same output but it's shorter and it should run faster. Check it oneline.