What should healthcheck of an Web API application actually check?
I have to add health checks to a rather new application (Web API, not a microservice) and I and a colleague are not agreeing about what other systems I should include in the check. This application is the beginning of the writing of a legacy one (will work side by side for a long transition period).
The following systems to be included are considered:
- the relational database this application is querying
- an external API that the application is querying once a day (sync some identities)
- the legacy application
My opinion is that only the database check should be currently included because it it fails nothing works. For the other systems, if they fail I do not really care, unless the API fails when I need to sync the data.
I am interested in a guideline to understand how dependencies are considered when building the health check functionality for an API.
Additional details about the usage: currently none of these services will be deployed as microservices (yet) and a failed health-check (actually multiple in a row, as configured in a monitoring system) will trigger an e-mail notification. Naturally, I would like to minimize the false positive failed health checks.
The debate is whether to include dependencies such as the external API in the check since I only use it once a day. My opinion is that my system should not care whether if it fails, as long as I can sync my data when I need it, but I have never designed any serious monitoring for an app and I might be wrong with this one.