Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
How to delete contents of a specific field, if it matches a pattern and there is nothing else in the field
How do I delete contents of a specific field, if it matches a pattern, and there is nothing else in the field? I have a several GB tsv file, and I am interested in a specific field (72). If it contains hyphens, and only hyphens, then I want the hyphens deleted, leaving a blank field. I am using Ubuntu 20.04, with GNU awk v5. I've tried something like this:
awk 'BEGIN{FS=OFS="\t"}{gsub(/-/,"",$72)}1' file.tsv
But that also deletes the hyphens if there are other characters in the field too, which I do not want. E.g.
blah------
becomes
blah
but I want to leave it unchanged; but change
---------------
to nothing.
3 answers
You are accessing this answer with a direct link, so it's being shown above all other answers regardless of its score. You can return to the normal view.
You're matching the regex pattern of /-/, so it just matches every individual hyphen, regardless of where. You want to match the entire entry if it's only hyphens, or /^-+$/.
^ – Beginning of line
-+ – One or more hyphens
$ – End of line
Putting that in gives
awk 'BEGIN{FS=OFS="\t"}{gsub(/^-+$/,"",$72)}1' file.tsv
which does what you want.
1 comment thread
The following users marked this post as Works for me:
| User | Comment | Date |
|---|---|---|
| LVx0 | (no comment) | Apr 11, 2022 at 07:56 |
The awk gsub function takes as its first argument a regular expression indicating the substring to be replaced, and replaces a matching substring with the value of the second argument, which is the replacement string.
Since your regular expression, /-/, is not anchored, it will match anywhere in the value that is being matched against. That's why it matches the hyphens that are contained as a part of the value. Here's an example run to illustrate this:
$ (printf '%s\n' 'blah---blah' '---blah---' 'blah------' '---------------') \
> | awk '{gsub(/-/,"",$1)}1'
blahblah
blah
blah
$
(note the empty line)
To get the behavior you are after, you need to anchor the regular expression at both the beginning and the end of the string, and provide a regular expression that matches against that whole string. Since you want to do a replacement only of strings that consist solely of hyphens, the regular expression should instead be /^-+$/ where the ^ anchors at the beginning, the -+ specifies an unbounded but non-zero number of - characters (that's the +), and the $ anchors at the end.
Example:
$ (printf '%s\n' 'blah---blah' '---blah---' 'blah------' '---------------') \
> | awk '{gsub(/^-+$/,"",$1)}1'
blah---blah
---blah---
blah------
$
As you can see, the contents of a field that consists of something other than only hyphens is now preserved, but the all-hyphens field contents is replaced by the empty string.
1 comment thread
The following users marked this post as Works for me:
| User | Comment | Date |
|---|---|---|
| LVx0 | (no comment) | May 5, 2022 at 11:25 |
For the regular expression to only match a full field of hyphens, you have, as others already have explained, to put the ^ anchor at the begin of the regular expression and the $ anchor at the end and make it clear that you are interested in a sequence of hyphens by adding a + after the hyphen.
There is, however, one additional aspect that may be of interest for you: Instead of using gsub for both the matching and the replacement you could separate these two aspects in the following way:
awk 'BEGIN{FS=OFS="\t"}$72~/^-+$/{$72=""}1' file.tsv
The expression $72~/^-+$/ will match those lines where field $72 consists of only hyphens, and the action {$72=""} clears field $72. This approach gives you the freedom to choose some different action on those lines than only deleting the field content. This can be of interest if, for example, you do not only want to clear the field, but also get rid of the surrounding field separators or the like.
0 comment threads
0 comment threads