Menus

Cloud Case Study

Explore our case study to understand how our tech experts offer cloud-related solutions to our clients.

Centralized log management using AWS services

Log management is about more than just collecting log files. It includes centralizing logs, parsing events into individual data points, storing them properly for future use, and analyzing the results.

This provides a more detailed look into system performance, as well as the ability to immediately detect problems or anomalies. Ultimately, it provides you with the insights needed to improve your operations.

Importance of Log Collection:

When you have hundreds or thousands of services running on multiple servers in multiple locations, manually searching through log files is a time taking and a burden to remember the log path.

A centralized log management system will aggregate and store logs, allowing you to access, search, and manage event logs from a single location. In the Market, we have many tools available to do the same kind of process, as our complete Infra is running on AWS we have tried to leverage all the cloud features.

We have designed Log Management by using all the AWS newly launched features which is more cost-effective and easy to manage.

AWS services we have used in our design:

  1. S3
  2. SSM

Before starting, let’s be ready with the below pre-requisite

Create an EC2 role and attach the below policies:

  1. EC2 role for SSM.
  2. S3 Bucket policy to a particular bucket as per the best practices.
  3. Cloud watch role.

The below application needs to be installed in servers:

  1. AWS CLI
    For moving the backups from server to S3.
  2. 7-Zip
    To compress the log files before moving to the S3 bucket, we can create a zip file using any zip utility software. In our solution, we have used utility 7-Zip.
    While installing select default Installation path: C:\Program Files\
  3. Create one S3 bucket
    — keeping the backup script, Input file and Storing the log.

NOTE: We can install the below packages in the server using the SSM Run command if SSM is already in place or manually login into the server.

Let’s get started with our implementation phase:

First, we will prepare one CSV input file, where we are going to mention the server details, log path, retention period, backup location, the script use this details and grep the log files.

Log Management Implementation

We can implement this Log management using SSM for the below two ways, the first solution is to execute the log backup manually from the AWS SSM console and the second solution is to schedule the backup job like a cron service.

1. AWS-Systems Manager Run Command from AWS-SSM console:

We can configure the inputs (command document & command parameters) and select the target to execute the task. If the task has run once means we can go with AWS-SSM console.

2. AWS Cloud watch Event Scheduler with SSM Agent:

AWS managed CRON service to back up the logs directly to S3 bucket, choose Cron expression and specify a Cron expression that defines when the task is to be triggered.

Now, we will deep dive into our first solution using AWS-System Manager console and move the logs to centralized location i.e S3(created earlier)

Select AWS-Run Remote Script from Command document,
Using the AWS-RunRemoteScript document we are going to execute the scripts to SSM managed servers.

Enquire Now

Ready to unlock the full potential of your business with our cutting-edge cloud solutions?

Take the first step towards success and enquire now to explore how our tailored solutions can revolutionize your operations and drive growth.