How to delete contents of a specific field, if it matches a pattern and there is nothing else in the field

The awk gsub function takes as its first argument a regular expression indicating the substring to be replaced, and replaces a matching substring with the value of the second argument, which is the replacement string.

Since your regular expression, /-/, is not anchored, it will match anywhere in the value that is being matched against. That's why it matches the hyphens that are contained as a part of the value. Here's an example run to illustrate this:

$ (printf '%s\n' 'blah---blah' '---blah---' 'blah------' '---------------') \
> | awk '{gsub(/-/,"",$1)}1'
blahblah
blah
blah

$

(note the empty line)

To get the behavior you are after, you need to anchor the regular expression at both the beginning and the end of the string, and provide a regular expression that matches against that whole string. Since you want to do a replacement only of strings that consist solely of hyphens, the regular expression should instead be /^-+$/ where the ^ anchors at the beginning, the -+ specifies an unbounded but non-zero number of - characters (that's the +), and the $ anchors at the end.

Example:

$ (printf '%s\n' 'blah---blah' '---blah---' 'blah------' '---------------') \
> | awk '{gsub(/^-+$/,"",$1)}1'
blah---blah
---blah---
blah------

$

As you can see, the contents of a field that consists of something other than only hyphens is now preserved, but the all-hyphens field contents is replaced by the empty string.

posted over 2 years ago

CC BY-SA 4.0

Canina‭

1489 reputation 2 23 150 31

You're matching the regex pattern of /-/, so it just matches every individual hyphen, regardless of where. You want to match the entire entry if it's only hyphens, or /^-+$/.

^ – Beginning of line
-+ – One or more hyphens
$ – End of line

Putting that in gives

awk 'BEGIN{FS=OFS="\t"}{gsub(/^-+$/,"",$72)}1' file.tsv

which does what you want.

posted over 2 years ago

CC BY-SA 4.0

Ullallulloo‭

214 reputation 4 5 28 2

For the regular expression to only match a full field of hyphens, you have, as others already have explained, to put the ^ anchor at the begin of the regular expression and the $ anchor at the end and make it clear that you are interested in a sequence of hyphens by adding a + after the hyphen.

There is, however, one additional aspect that may be of interest for you: Instead of using gsub for both the matching and the replacement you could separate these two aspects in the following way:

awk 'BEGIN{FS=OFS="\t"}$72~/^-+$/{$72=""}1' file.tsv

The expression $72~/^-+$/ will match those lines where field $72 consists of only hyphens, and the action {$72=""} clears field $72. This approach gives you the freedom to choose some different action on those lines than only deleting the field content. This can be of interest if, for example, you do not only want to clear the field, but also get rid of the surrounding field separators or the like.

posted over 2 years ago

CC BY-SA 4.0

Dirk Herrmann‭

1387 reputation 1 31 141 48