Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
Comments on How to read lines into an array in Bash
Parent
How to read lines into an array in Bash
I wish to fill an array with strings, using Bash scripting. The strings are in a file, one per line. Here is what I've tried:
declare -a my_array=()
while read line; do
my_array+=( "$line" )
done < my_file.txt
But it seems to only add the first line as tested with echo "$my_array"
.
Post
1st issue
As others have said, echo "$array"
only prints the first element of the array; I suggest
printf '%s\n' "${array[@]}"
to print each element on a line.
2nd issue
More importantly, the loop is partially incorrect. Consider this file:
pear yellow
apple \\ red
orange brown
Look closely: Lines 2 and 3 contain trailing spaces.
That's what the array gets (using underscores as delimiters to better see exactly what each element contains):
$ printf '_%s_\n' "${array[@]}"
_pear yellow_
_apple \ red_
_orange brown_
- All the leading and trailing spaces were mangled. Address that with an empty
IFS
. -
\\
became\
. Theread
command interprets backslash sequences, add the-r
flag to disable that.
Solution
while IFS= read -r line; do
array+=( "$line" )
done < file
or, better, as suggested by r~~, use mapfile -t array < file
(mapfile
and readarray
are synonyms). It is a single command and thus more efficient.
Caveat
Finally, mind that shell loops are very slow. A test on a 10 MB file:
$ wc < /var/log/kern.log.0
117555 1389245 10000045
~$ time { mapfile -t a < /var/log/kern.log.0; }
real 0m0.060s
user 0m0.052s
sys 0m0.008s
$ time { while IFS= read -r line; do b+=("$line"); done < /var/log/kern.log.0; }
real 0m0.720s
user 0m0.652s
sys 0m0.068s
And that is just reading the file into an array and doing nothing with it.
If attempting to process text, use actual text-processing tools (Awk, Sed, Jq, Csvkit, etc. depending on the task).
Further reading
- BashFAQ/001: How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
- Unix SE: Why is using a shell loop to process text considered bad practice?
- Unix SE: Why is
while IFS= read
used so often, instead ofIFS=; while read
?
0 comment threads