How to read lines into an array in Bash

1st issue

As others have said, echo "$array" only prints the first element of the array; I suggest

printf '%s\n' "${array[@]}"

to print each element on a line.

2nd issue

More importantly, the loop is partially incorrect. Consider this file:

pear yellow
   apple \\ red   
orange brown    

^{Look closely: Lines 2 and 3 contain trailing spaces.}

That's what the array gets (using underscores as delimiters to better see exactly what each element contains):

$ printf '_%s_\n' "${array[@]}"
_pear yellow_
_apple \ red_
_orange brown_

1. All the leading and trailing spaces were mangled. Address that with an empty IFS.
2. \\ became \. The read command interprets backslash sequences, add the -r flag to disable that.

Solution

while IFS= read -r line; do
    array+=( "$line" )
done < file

or, better, as suggested by r~~, use mapfile -t array < file (mapfile and readarray are synonyms). It is a single command and thus more efficient.

Caveat

Finally, mind that shell loops are very slow. A test on a 10 MB file:

$ wc < /var/log/kern.log.0
  117555  1389245 10000045

~$ time { mapfile -t a < /var/log/kern.log.0; }
real    0m0.060s
user    0m0.052s
sys     0m0.008s

$ time { while IFS= read -r line; do b+=("$line"); done < /var/log/kern.log.0; }
real    0m0.720s
user    0m0.652s
sys     0m0.068s

And that is just reading the file into an array and doing nothing with it.

If attempting to process text, use actual text-processing tools (Awk, Sed, Jq, Csvkit, etc. depending on the task).

Further reading

• BashFAQ/001: How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
• Unix SE: Why is using a shell loop to process text considered bad practice?
• Unix SE: Why is while IFS= read used so often, instead of IFS=; while read?

posted about 3 years ago

CC BY-SA 4.0

Quasímodo‭

391 reputation 4 5 50 3

Your code is correct. You have declared your variable as an array, and you are successfully appending to it.

To display all of the elements of your variable, try echo "${my_array[@]}". (Another answer suggests declare -p for this, but declare -p will possibly give you more information than you wanted. Also, be aware that declare -p isn't intended to substitute for your declare -a; it's an additional command to show the status of your variable.)

To display the current number of elements in your variable, try echo "${#my_array[@]}".

To display the third element in the array, try echo "${my_array[2]}" (arrays in Bash are zero-indexed).

echo $my_array will only display the first element of the array, for what I presume are historical reasons, but that's the documented behavior. I would avoid using this ‘feature’ with array variables; it only leads to confusion.

Note also that in Bash versions >= 4, the readarray builtin will do all of this work for you:

readarray -t my_array < my_file.txt

You don't even need to declare my_array beforehand if you use this builtin. Bash 4 was released in 2009, so unless you're working on a pretty old or stripped-down system, you are probably safe using that.

posted over 3 years ago

CC BY-SA 4.0

3y ago

r~~‭

2785 reputation 3 60 274 33

Your code adds all three elements to the list. You can see this if you add a line inside the while loop:

 echo "$line"

However, if you give the command echo $my_array, it will only show you the first element. (I'm not yet sure why, because I don't do a lot of shell scripting. I'll edit this answer once I find out. I've got a feeling it's supposed to work this way).

To show the entire array, you need declare -p my_array, or write another loop that lists all the elements.

Source used: https://www.tutorialkart.com/bash-shell-scripting/bash-array/

posted over 3 years ago

CC BY-SA 4.0

FractionalRadix‭

409 reputation 7 8 52 22