Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
Unable to use conditional Must term in C# NEST client for Elastic Search
I'm trying to write a Elastic Search query using the C# NEST client, but right now I'm stuck on an issue with a conditional query.
The query takes in two optional values, meaning they are both allowed to be omitted, but can also be used together.
I want to be able to search for documents and filter either on:
- Matching both main and secondary ID
- Matching only main ID
- Matching only secondary ID
- No need to match either
My understanding was that you could make use of conditionless queries in ES/Nest, but it does not seems to apply.
Using syntax like this:
m.Term(t => t.Field(f => f.MainId).Value(mainId));
results in a query containing null values, which fails.
So my current attempt is this query, but it only works if all values are provided, and fails to query if some values are missing.
var query = new SearchRequestDescriptor<EventWithPayload>()
.Index("MY-INDEX")
.Query(q => q
.Bool(b => b
.Filter(filter => filter
.Bool(fb => fb
.Must(m =>
{
if (mainId is not null)
m.Term(t => t.Field(f => f.MainId).Value(mainId));
},
m =>
{
if (secondId is not null)
m.Term(t => t.Field(f => f.SecondId).Value(secondId));
},
m =>
{
m.Terms(ts => ts.Field(f => f.EventName).Term(new TermsQueryField(terms)));
}
)
)
)
.MustNot(
mn => mn.Term(t => t.Field(f => f.ThirdId).Value("NOT_FOUND"))
)
)
);
If both the main and secondary IDs are provided, the query works as expected, but not if one of the values are omitted.
I would like the object null to not be serialised at all, but instead I receive an empty object {}.
Resulting JSON query with both values:
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"term": {
"mainId": {
"value": "ABC123"
}
}
},
{
"term": {
"secondId": {
"value": "DEF456"
}
}
},
{
"terms": {
"eventName": [
"A",
"B",
"C"
]
}
}
]
}
},
"must_not": {
"term": {
"thirdId": {
"value": "NOT_FOUND"
}
}
}
}
}
}
JSON query when secondary ID is missing:
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"term": {
"mainId": {
"value": "ABC123"
}
}
},
{},
{
"terms": {
"eventName": [
"A",
"B",
"C"
]
}
}
]
}
},
"must_not": {
"term": {
"thirdId": {
"value": "NOT_FOUND"
}
}
}
}
}
}
Any help here would be appreciated, or if there is another, better way to solve the issue I'm facing.
1 answer
Hello, a a late update, mainly to help others facing the problem in the future.
it turns out that the latest version of NEST could not handle conditionLess values, and it also fails if you iterate over possible values, since that would return a empty object - which Elastic won't accept.
So instead my solution to the problem was to create separate lists for my different filter conditions, populate with possible values and insert the finished list in the query creation.
not in any way pretty code, but it does work, without duplicating JSON attributes all over the place, so its something.
Code example:
public async Task<IEnumerable<DataClass>> GetData(string firstName, string? lastName, string? MiddleName, List<string> eventsFilter, List<string> allowedIds,
CancellationToken cancellation = default)
{
var events = new ReadOnlyCollection<FieldValue>(eventsFilter.ConvertAll(x => (FieldValue)x));
var must = new List<Action<QueryDescriptor<DataClass>>>();
var mustNot = new List<Action<QueryDescriptor<DataClass>>>();
var should = new List<Action<QueryDescriptor<DataClass>>>();
must.Add(q => q.Term(t => t.Field(f => f.FirstName).Value(firstName)));
must.Add(q => q.Terms(ts => ts.Field(f => f.EventName).Term(new TermsQueryField(events))));
mustNot.Add(q => q.Term(t => t.Field(f => f.EventPayload!.Fraud).Value(0)));
mustNot.Add(q => q.Term(t => t.Field(f => f.EventPayload!.IncorrectTransit).Value(0)));
if (lastName is not null)
{
must.Add(q => q.Term(t => t.Field(f => f.lastName).Value(lastName)));
mustNot.Add(q => q.Exists(e => e.Field(f => f.MiddleName)));
}
if (MiddleName is not null)
{
must.Add(q => q.Term(t => t.Field(f => f.MiddleName).Value(MiddleName)));
mustNot.Add(q => q.Term(t => t.Field(f => f.MyId).Value("NOT_FOUND")));
}
should.AddRange(allowedIds.Select(id => (Action<QueryDescriptor<DataClass>>)(q => q.Term(t => t.Field(f => f.MyId).Value(id)))));
var query2 = new SearchRequestDescriptor<DeviceEvent>()
.Index("MY-INDEX*")
.Sort(s => s.Field(f => f.Timestamp, new FieldSort
{
Order = SortOrder.Desc
}))
.Query(q => q
.Bool(b => b
.Filter(filter => filter.Bool(fb => fb
.Must(must.ToArray())
.Should(should.ToArray())
.MustNot(mustNot.ToArray())))));
var result = await _client.SearchAsync(query2, cancellation);
return result.Documents;
}
0 comment threads
0 comment threads