There is no single, off-the-shelf solution or offering in Azure, or for that matter any cloud provider, to the monitoring challenges presented by microservices. Interestingly enough, there are not too many open source tools available that can work with .NET-based microservices.
We are utilizing Microsoft Azure Cloud and cloud services for building our microservices, so it is useful to look for the monitoring capability it comes with. If you are looking to manage approximately a couple of hundred microservices, you can utilize a custom monitoring solution (mostly interweaving PowerShell scripts) based on a Microsoft Azure-based solution.
We will be primarily focusing on the following logging and monitoring solutions:
- Microsoft Azure Diagnostics: This helps in collecting and analyzing resources through resource and activity logs.
- Application Insights: This helps in collecting all of the telemetry data about our microservices and analyzing them. This is a framework-based approach for monitoring.
- Log Analytics: Log Analytics analyzes and displays data and provides scalable querying capability over collected logs.
Let's look at these solutions from a different perspective. This perspective will help us visualize our Azure-based microservice monitoring solution. A microservice is composed of the following:
- Infrastructure layer: A virtual machine or an application container (for example, Docker container)
- Application stack layer: Constitutes the operating system, .NET CLR, and the microservice application code
Each of these layer components can be monitored as follows:
- Virtual machine: Using Azure Diagnostics Logs
- Docker containers: Using container logs and Application Insights or a third-party container monitoring solution, such as cAdvisor, Prometheus, or Sensu
- Windows operating system: Using Azure Diagnostics Logs and Activity Logs
- A microservice application: Using Application Insights
- Data visualization and metric monitoring: Using Log Analytics or third-party solutions, such as Splunk or ELK stack
Various Azure services come with an activity ID in their log entries. This activity ID is a unique GUID assigned for each request, which can be utilized as a correlation ID during log analysis.