Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Comments on How to read lines into an array in Bash

Parent

How to read lines into an array in Bash

+6
−0

I wish to fill an array with strings, using Bash scripting. The strings are in a file, one per line. Here is what I've tried:

declare -a my_array=()

while read line; do
    my_array+=( "$line" )
done < my_file.txt

But it seems to only add the first line as tested with echo "$my_array".

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+8
−0

1st issue

As others have said, echo "$array" only prints the first element of the array; I suggest

printf '%s\n' "${array[@]}"

to print each element on a line.

2nd issue

More importantly, the loop is partially incorrect. Consider this file:

pear yellow
   apple \\ red   
orange brown    

Look closely: Lines 2 and 3 contain trailing spaces.

That's what the array gets (using underscores as delimiters to better see exactly what each element contains):

$ printf '_%s_\n' "${array[@]}"
_pear yellow_
_apple \ red_
_orange brown_
  1. All the leading and trailing spaces were mangled. Address that with an empty IFS.
  2. \\ became \. The read command interprets backslash sequences, add the -r flag to disable that.

Solution

while IFS= read -r line; do
    array+=( "$line" )
done < file

or, better, as suggested by r~~, use mapfile -t array < file (mapfile and readarray are synonyms). It is a single command and thus more efficient.

Caveat

Finally, mind that shell loops are very slow. A test on a 10 MB file:

$ wc < /var/log/kern.log.0
  117555  1389245 10000045

~$ time { mapfile -t a < /var/log/kern.log.0; }
real    0m0.060s
user    0m0.052s
sys     0m0.008s

$ time { while IFS= read -r line; do b+=("$line"); done < /var/log/kern.log.0; }
real    0m0.720s
user    0m0.652s
sys     0m0.068s

And that is just reading the file into an array and doing nothing with it.

If attempting to process text, use actual text-processing tools (Awk, Sed, Jq, Csvkit, etc. depending on the task).

Further reading

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.

1 comment thread

Another caveat with the loop is trailing lines without newline. read will exit because it hit EOF, so... (2 comments)
Another caveat with the loop is trailing lines without newline. read will exit because it hit EOF, so...
Someone‭ wrote over 3 years ago

Another caveat with the loop is trailing lines without newline. read will exit because it hit EOF, so you have to check if $line is empty after the loop. A simple demonstrator: echo -n $'a\nb' | (while read -r line; do echo $line; done; echo "After: $line")

Quasímodo‭ wrote over 3 years ago

Someone‭ Well noted. Though, also to note, is that text-files are required to always have end in a newline character in Unix systems.