AWS Lambda Heartbeat Writer – Deployment Guide

Overview

The purpose of this Lambda function is to continuously generate lightweight “heartbeat” logs and store them in one or more Amazon S3 buckets.
This ensures that monitoring systems such as ArcSight SmartConnectors always detect recent activity in the bucket, allowing teams to distinguish between:

• Connector issues (no data reaching ArcSight), and
  
  

• Source system inactivity (product temporarily not generating logs).
  
  

The Lambda periodically writes a small JSON object to each configured S3 target, simulating a regular log entry. These files are then read and forwarded by existing log collectors.

Architecture Summary

Component

Description

AWS Lambda Function

Generates and uploads heartbeat files at regular intervals (for example, every 5 minutes).

Amazon S3 Bucket(s)

Destination for the heartbeat log files. Each target consists of a bucket and an optional folder/prefix.

Amazon EventBridge (CloudWatch Events)

Triggers the Lambda on a recurring schedule.

IAM Role

Grants the Lambda permission to write to S3 and to publish logs to CloudWatch.

Prerequisites

Before starting:

• You must have access to the AWS Management Console with permission to create Lambda functions, IAM roles, and EventBridge rules.
  
  

• One or more S3 buckets where logs are stored and monitored.
  
  

• A copy of the Lambda source code file (provided separately).
  
  

Step 1 – Create the IAM Role

1. Open the IAM service in the AWS Console.
   
   

2. Choose Roles → Create role.
   
   

3. Under Trusted entity type, select AWS service, then choose Lambda as the use case.
   
   

4. Click Next.
   
   

5. Attach the following managed policy:
   
   

   • AWSLambdaBasicExecutionRole (provides CloudWatch Logs permissions)
     
     

6. Choose Next again.
   
   

7. On the Add permissions page, select Create inline policy, and define a policy that allows the function to write to your target buckets, for example:
   
   

   • Service: S3
     
     

   • Action: PutObject
     
     

   • Resources: each of your buckets, using the pattern arn:aws:s3:::bucket-name/*
     
     

8. Review and create the role (e.g., name it lambda-heartbeat-writer-role).
   
   If your S3 buckets enforce KMS encryption, also add KMS permissions (Encrypt, GenerateDataKey*) for the relevant key.
   
   

Step 2 – Create the Lambda Function

1. In the AWS Console, go to Lambda → Create function.
   
   

2. Choose Author from scratch.
   
   

3. Function name: s3-heartbeat-writer (or similar).
   
   

4. Runtime: Python 3.11.
   
   

5. Execution role: select Use an existing role and choose the IAM role created earlier.
   
   

6. Click Create function.
   
   

7. Under the Code tab, upload the provided Python file (from your separate source file).
   
   

8. Click Deploy once uploaded.
   
   

Step 3 – Configure Environment Variables

1. In the Lambda configuration view, open Environment variables → Edit.
   
   

2. Add the following key-value pair:
   
   

   • Key: TARGETS
     
     

Value:

[{"bucket":"scapitalvc","prefix":"Avanan-logs/"}]

This format supports multiple targets. To expand in the future, simply append more entries, for example:

[

  {"bucket":"scapitalvc","prefix":"Avanan-logs/"},

  {"bucket":"client2-logs","prefix":"Office365/"},

  {"bucket":"backup-logs","prefix":"Avanan-backup/"}

]

3. 
   

4. (Optional) You can also define other environment variables such as:
   
   

   • PRODUCT_NAME – a friendly identifier for your log source.
     
     

   • TTL_SECONDS – how long the object should be considered valid (for internal tracking).
     
     

   • STORAGE_CLASS – e.g., STANDARD or STANDARD_IA.
     
     

Click Save after adding the variables.

Step 4 – Add a Schedule Trigger

1. In the function’s Triggers tab, click Add trigger.
   
   

2. Choose EventBridge (CloudWatch Events).
   
   

3. Create a new rule:
   
   

   • Rule name: heartbeat-every-5m
     
     

   • Schedule expression: cron(0/5 * * * ? *)
     (runs every 5 minutes)
     
     

4. Click Add.
   
   

EventBridge now invokes the Lambda automatically on the chosen interval.

Step 5 – Verify the Setup

1. From the Lambda console, click Test and run the function once manually.
   
   

2. Confirm a successful return value (ok: true).
   
   

3. In the S3 bucket, navigate to your configured prefix (e.g., Avanan-logs/heartbeat/...) and verify that a new JSON file was created.
   
   

4. Confirm that your monitoring connector ingests and forwards this heartbeat log as expected.
   
   

Step 6 – (Recommended) Configure S3 Lifecycle Management

To avoid long-term storage buildup:

1. In the S3 bucket → Management → Lifecycle rules, create a new rule.
   
   

2. Filter by prefix Avanan-logs/heartbeat/.
   
   

3. Set the objects to expire after 7–30 days depending on your retention policy.
   
   

Step 7 – Monitoring and Maintenance

• Lambda logs are visible in CloudWatch Logs, under /aws/lambda/<function-name>.
  
  

• Set up a simple CloudWatch Alarm to notify if the function starts failing or if it stops producing new objects.
  
  

• Updating the heartbeat targets requires only editing the TARGETS environment variable—no redeployment needed.
  
  

Summary

This setup ensures continuous visibility into your log pipeline’s health:

Component

Purpose

Lambda function

Periodically writes a small “I’m alive” heartbeat to S3.

S3 bucket(s)

Receives heartbeats in the same path as other logs, ensuring the connector always sees recent data.

EventBridge rule

Automates the schedule (e.g., every 5 minutes).

IAM role

Provides secure, least-privilege access for the Lambda.

With this in place, your SOC or monitoring platform can clearly differentiate between connector downtime and normal log silence from the product source—eliminating unnecessary alerts.