Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Post History

84%
+9 −0
Q&A What are field separators in operating-programming languages (such as Bash)?

In Bash, IFS is an internal variable and it stands for "Internal Field Separator" <- according to this link, it "determines how Bash recognizes fields, or word boundaries, when it interprets cha...

posted 3y ago by hkotsubo‭  ·  edited 3y ago by hkotsubo‭

Answer
#2: Post edited by user avatar hkotsubo‭ · 2021-03-11T13:14:09Z (about 3 years ago)
  • In Bash, `IFS` is an internal variable and it stands for ["Internal Field Separator"](https://tldp.org/LDP/abs/html/internalvariables.html#IFSREF) <- according to this link, it "_determines how Bash recognizes fields, or word boundaries, when it interprets character strings_".
  • Its default value is a "[whitespace](https://tldp.org/LDP/abs/html/special-chars.html#WHITESPACEREF)" (space, tab, and newline), but you can change it to whatever you need.
  • To give an example, using the default value, the following commands:
  • ```bash
  • text="a:b c-d e/f"
  • for word in $text; do echo "Word: $word"; done
  • ```
  • would ouput:
  • ```none
  • Word: a:b
  • Word: c-d
  • Word: e/f
  • ```
  • Note that the spaces were used to split the string into fields/"words", so each iteration of the `for` loop gets one part of the split results.
  • But if we change `IFS`:
  • ```bash
  • IFS=':-/'
  • text="a:b c-d e/f"
  • for word in $text; do echo "Word: $word"; done
  • ```
  • Now the output is:
  • ```none
  • Word: a
  • Word: b c
  • Word: d e
  • Word: f
  • ```
  • By setting `IFS=':-/'`, I'm saying that `:`, `-` and `/` should be the characters used to determine a field/"word" boundaries, thus the result is quite different (note that the spaces were "ignored", so "b c" and "d e" are considered two fields/"words").
  • If we change to `IFS=':'`, only the `:` character will be considered, and the result would be only 2 fields: "a" and "b c-d e/f".
  • ---
  • `IFS` is used by other commands, such as `read`:
  • ```
  • IFS=':'
  • echo "abc:def" | (read x y; echo "x=$x y=$y")
  • # output is "x=abc y=def"
  • ```
  • And it also affects the output of [the special variable `$*`](https://bash.cyberciti.biz/guide/$*) (which contains all the command line arguments of a script), when printed inside double quotes. Suppose I have this simple script:
  • ```bash
  • #!/bin/bash
  • echo "Args: $*"
  • ```
  • If I run this script: `script.sh a b c`, the output will be `Args: a b c`.
  • But if I change it to:
  • ```
  • #!/bin/bash
  • IFS=':'
  • echo "Args: $*"
  • ```
  • The first character of `IFS` will be used in the output, and displayed between the fields, so the output will be: `Args: a:b:c`.
  • ---
  • One detail regarding whitespace versus non-whitespace characters: if `IFS` contains whitespace, a sequence of one or more whitespaces is considered to be a single separator, but a sequence of one or more non-whitespaces isn't. Example:
  • ```
  • # text with 4 spaces before "c", and a trailing space in the end
  • text='a::b c '
  • # IFS is just a space
  • IFS=' '
  • for word in $text; do echo "Word: [$word]"; done
  • ```
  • In this case, `IFS` is just a space, but a sequence of one or more spaces is considered to be a single separator, so the output is:
  • ```none
  • Word: [a::b]
  • Word: [c]
  • ```
  • If we set `IFS=':'`, now the field separator is a non-whitespace, so a sequence of one or more is not considered a single separator, and the output would change to:
  • ```none
  • Word: [a]
  • Word: []
  • Word: [b c ]
  • ```
  • The second field is an empty string, because each `:` is another separator, and `::` is considered "an empty string between two `:`".
  • ---
  • But anyway, the field separator can be whatever you need, not limited to the ones defined in the question.
  • In Bash, `IFS` is an internal variable and it stands for ["Internal Field Separator"](https://tldp.org/LDP/abs/html/internalvariables.html#IFSREF) <- according to this link, it "_determines how Bash recognizes fields, or word boundaries, when it interprets character strings_".
  • Its default value is a "[whitespace](https://tldp.org/LDP/abs/html/special-chars.html#WHITESPACEREF)" (space, tab, and newline), but you can change it to whatever you need.
  • To give an example, using the default value, the following commands:
  • ```bash
  • text="a:b c-d e/f"
  • for word in $text; do echo "Word: $word"; done
  • ```
  • would ouput:
  • ```none
  • Word: a:b
  • Word: c-d
  • Word: e/f
  • ```
  • Note that the spaces were used to split the string into fields/"words", so each iteration of the `for` loop gets one part of the split results.
  • But if we change `IFS`:
  • ```bash
  • IFS=':-/'
  • text="a:b c-d e/f"
  • for word in $text; do echo "Word: $word"; done
  • ```
  • Now the output is:
  • ```none
  • Word: a
  • Word: b c
  • Word: d e
  • Word: f
  • ```
  • By setting `IFS=':-/'`, I'm saying that `:`, `-` and `/` should be the characters used to determine a field/"word" boundaries, thus the result is quite different (note that the spaces were "ignored", so `b c` and `d e` are considered two fields/"words").
  • If we change to `IFS=':'`, only the `:` character will be considered, and the result would be only 2 fields: `a` and `b c-d e/f`.
  • ---
  • `IFS` is used by other commands, such as `read`:
  • ```
  • IFS=':'
  • echo "abc:def" | (read x y; echo "x=$x y=$y")
  • # output is "x=abc y=def"
  • ```
  • And it also affects the output of [the special variable `$*`](https://bash.cyberciti.biz/guide/$*) (which contains all the command line arguments of a script), when printed inside double quotes. Suppose I have this simple script:
  • ```bash
  • #!/bin/bash
  • echo "Args: $*"
  • ```
  • If I run this script: `script.sh a b c`, the output will be `Args: a b c`.
  • But if I change it to:
  • ```
  • #!/bin/bash
  • IFS=':'
  • echo "Args: $*"
  • ```
  • The first character of `IFS` will be used in the output, and displayed between the fields, so the output will be: `Args: a:b:c`.
  • ---
  • One detail regarding whitespace versus non-whitespace characters: if `IFS` contains whitespace, a sequence of one or more whitespaces is considered to be a single separator, but a sequence of one or more non-whitespaces isn't. Example:
  • ```
  • # text with 4 spaces before "c", and a trailing space in the end
  • text='a::b c '
  • # IFS is just a space
  • IFS=' '
  • for word in $text; do echo "Word: [$word]"; done
  • ```
  • In this case, `IFS` is just a space, but a sequence of one or more spaces is considered to be a single separator, so the output is:
  • ```none
  • Word: [a::b]
  • Word: [c]
  • ```
  • If we set `IFS=':'`, now the field separator is a non-whitespace, so a sequence of one or more is not considered a single separator, and the output would change to:
  • ```none
  • Word: [a]
  • Word: []
  • Word: [b c ]
  • ```
  • The second field is an empty string, because each `:` is another separator, and `::` is considered "an empty string between two `:`".
  • ---
  • But anyway, the field separator can be whatever you need, not limited to the ones defined in the question.
#1: Initial revision by user avatar hkotsubo‭ · 2021-03-11T13:11:50Z (about 3 years ago)
In Bash, `IFS` is an internal variable and it stands for ["Internal Field Separator"](https://tldp.org/LDP/abs/html/internalvariables.html#IFSREF) <- according to this link, it "_determines how Bash recognizes fields, or word boundaries, when it interprets character strings_".

Its default value is a "[whitespace](https://tldp.org/LDP/abs/html/special-chars.html#WHITESPACEREF)" (space, tab, and newline), but you can change it to whatever you need.

To give an example, using the default value, the following commands:

```bash
text="a:b c-d e/f"
for word in $text; do echo "Word: $word"; done
```

would ouput:

```none
Word: a:b
Word: c-d
Word: e/f
```

Note that the spaces were used to split the string into fields/"words", so each iteration of the `for` loop gets one part of the split results.

But if we change `IFS`:

```bash
IFS=':-/'
text="a:b c-d e/f"
for word in $text; do echo "Word: $word"; done
```

Now the output is:

```none
Word: a
Word: b c
Word: d e
Word: f
```

By setting `IFS=':-/'`, I'm saying that `:`, `-` and `/` should be the characters used to determine a field/"word" boundaries, thus the result is quite different (note that the spaces were "ignored", so "b c" and "d e" are considered two fields/"words").

If we change to `IFS=':'`, only the `:` character will be considered, and the result would be only 2 fields: "a" and "b c-d e/f".

---
`IFS` is used by other commands, such as `read`:

```
IFS=':'
echo "abc:def" | (read x y; echo "x=$x y=$y")
# output is "x=abc y=def"
```

And it also affects the output of [the special variable `$*`](https://bash.cyberciti.biz/guide/$*) (which contains all the command line arguments of a script), when printed inside double quotes. Suppose I have this simple script:

```bash
#!/bin/bash
echo "Args: $*"
```

If I run this script: `script.sh a b c`, the output will be `Args: a b c`.  
But if I change it to:

```
#!/bin/bash
IFS=':'
echo "Args: $*"
```

The first character of `IFS` will be used in the output, and displayed between the fields, so the output will be: `Args: a:b:c`.


---

One detail regarding whitespace versus non-whitespace characters: if `IFS` contains whitespace, a sequence of one or more whitespaces is considered to be a single separator, but a sequence of one or more non-whitespaces isn't. Example:

```
# text with 4 spaces before "c", and a trailing space in the end
text='a::b    c '
# IFS is just a space
IFS=' '
for word in $text; do echo "Word: [$word]"; done
```

In this case, `IFS` is just a space, but a sequence of one or more spaces is considered to be a single separator, so the output is:

```none
Word: [a::b]
Word: [c]
```

If we set `IFS=':'`, now the field separator is a non-whitespace, so a sequence of one or more is not considered a single separator, and the output would change to:

```none
Word: [a]
Word: []
Word: [b    c ]
```

The second field is an empty string, because each `:` is another separator, and `::` is considered "an empty string between two `:`".

---
But anyway, the field separator can be whatever you need, not limited to the ones defined in the question.