Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
What are field separators in operating-programming languages (such as Bash)?
The concept of field separator has some private cases in different operating system shells and their utilities (IFS on Bourne shell and derivates, RS on AWK and perhaps more) but I am having trouble understanding what is the purpose of the concept in general or which problem it was first aimed to solve.
I have personally understood that a "field separator" is any indentation character and line break character such as the following but I am not sure that's correct:
-
Indentation characters:
- Whitespace character
- Tabulation character
- Other, less standard, indentation characters
-
Line break characters:
- (non-carriage-return) Line Feed (LF)
- Carriage Return Line Feed (CRLF)
- Other, less standard, line break characters
1 answer
In Bash, IFS is an internal variable and it stands for "Internal Field Separator" <- according to this link, it "determines how Bash recognizes fields, or word boundaries, when it interprets character strings".
Its default value is a "whitespace" (space, tab, and newline), but you can change it to whatever you need.
To give an example, using the default value, the following commands:
text="a:b c-d e/f"
for word in $text; do echo "Word: $word"; done
would ouput:
Word: a:b
Word: c-d
Word: e/f
Note that the spaces were used to split the string into fields/"words", so each iteration of the for loop gets one part of the split results.
But if we change IFS:
IFS=':-/'
text="a:b c-d e/f"
for word in $text; do echo "Word: $word"; done
Now the output is:
Word: a
Word: b c
Word: d e
Word: f
By setting IFS=':-/', I'm saying that :, - and / should be the characters used to determine a field/"word" boundaries, thus the result is quite different (note that the spaces were "ignored", so b c and d e are considered two fields/"words").
If we change to IFS=':', only the : character will be considered, and the result would be only 2 fields: a and b c-d e/f.
IFS is used by other commands, such as read:
IFS=':'
echo "abc:def" | (read x y; echo "x=$x y=$y")
# output is "x=abc y=def"
And it also affects the output of the special variable $* (which contains all the command line arguments of a script), when printed inside double quotes. Suppose I have this simple script:
#!/bin/bash
echo "Args: $*"
If I run this script: script.sh a b c, the output will be Args: a b c.
But if I change it to:
#!/bin/bash
IFS=':'
echo "Args: $*"
The first character of IFS will be used in the output, and displayed between the fields, so the output will be: Args: a:b:c.
One detail regarding whitespace versus non-whitespace characters: if IFS contains whitespace, a sequence of one or more whitespaces is considered to be a single separator, but a sequence of one or more non-whitespaces isn't. Example:
# text with 4 spaces before "c", and a trailing space in the end
text='a::b c '
# IFS is just a space
IFS=' '
for word in $text; do echo "Word: [$word]"; done
In this case, IFS is just a space, but a sequence of one or more spaces is considered to be a single separator, so the output is:
Word: [a::b]
Word: [c]
If we set IFS=':', now the field separator is a non-whitespace, so a sequence of one or more is not considered a single separator, and the output would change to:
Word: [a]
Word: []
Word: [b c ]
The second field is an empty string, because each : is another separator, and :: is considered "an empty string between two :".
But anyway, the field separator can be whatever you need, not limited to the ones defined in the question.
1 comment thread
1 comment thread