Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Post History

83%
+8 −0
Q&A How to read lines into an array in Bash

1st issue As others have said, echo "$array" only prints the first element of the array; I suggest printf '%s\n' "${array[@]}" to print each element on a line. 2nd issue More importantly, th...

posted 2y ago by Quasímodo‭

Answer
#1: Initial revision by user avatar Quasímodo‭ · 2021-08-18T11:48:38Z (over 2 years ago)
### 1st issue

As others have said, `echo "$array"` only prints the first element of the array; I suggest

    printf '%s\n' "${array[@]}"

to print each element on a line.

### 2nd issue

More importantly, **the loop is partially incorrect**. Consider this file:

```
pear yellow
   apple \\ red   
orange brown    
```

<sup>Look closely: Lines 2 and 3 contain trailing spaces.</sup>

That's what the array gets (using underscores as delimiters to better see exactly what each element contains):

```
$ printf '_%s_\n' "${array[@]}"
_pear yellow_
_apple \ red_
_orange brown_
```

1. All the leading and trailing spaces were mangled. Address that with an empty [`IFS`][1].
2. `\\` became `\`. The [`read` command interprets backslash sequences][2], add the `-r` flag to disable that.

### Solution

```
while IFS= read -r line; do
    array+=( "$line" )
done < file
```

or, better, as suggested by r~~, use `mapfile -t array < file` (`mapfile` and `readarray` are synonyms). It is a single command and thus more efficient.

### Caveat

Finally, mind that shell loops are very slow. A test on a 10 MB file:

```
$ wc < /var/log/kern.log.0
  117555  1389245 10000045

~$ time { mapfile -t a < /var/log/kern.log.0; }
real    0m0.060s
user    0m0.052s
sys     0m0.008s

$ time { while IFS= read -r line; do b+=("$line"); done < /var/log/kern.log.0; }
real    0m0.720s
user    0m0.652s
sys     0m0.068s
```

And that is just reading the file into an array and doing nothing with it.

If attempting to process text, use actual text-processing tools (Awk, Sed, Jq, 
Csvkit, etc. depending on the task).

### Further reading

- [BashFAQ/001: How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?][4]
- [Unix SE: Why is using a shell loop to process text considered bad practice?][3]
- [Unix SE: Why is `while IFS= read` used so often, instead of `IFS=; while read`?][5]

[1]: https://www.gnu.org/software/bash/manual/html_node/Word-Splitting.html
[2]: https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html
[3]: https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice
[4]: http://mywiki.wooledge.org/BashFAQ/001
[5]: https://unix.stackexchange.com/questions/18886/why-is-while-ifs-read-used-so-often-instead-of-ifs-while-read