Video summary

Onboard data from AWS CloudTrail using CloudWatch

Main summary

Key takeaways

Technology

Summary: Retrieve AWS CloudTrail data via CloudWatch and ingest into Splunk

This tutorial (Soft Mania community, second in a series) demonstrates how to collect AWS CloudTrail logs using the CloudWatch Logs integration path, then forward them into Splunk using the Splunk Add-on for AWS.

What’s new vs the previous method

  • Previous session: collected CloudTrail logs using SQS and S3.
  • This session: collects CloudTrail logs through the CloudTrail → CloudWatch Logs → Splunk pipeline.
  • Key motivation: using CloudWatch enables near real-time log availability for analysis/notifications, whereas S3-based collection can involve a 5–10 minute delay.

Step-by-step technical flow

1) Create an IAM user for Splunk ingestion

  • In AWS IAM, create a new user (example name: “Dheeraj AWS”).
  • Attach permissions:
    • Administrator Access (for simplicity in the demo).
  • Generate credentials:
    • Create an access key and secret access key (used later to configure Splunk’s AWS add-on).

2) Install/configure the Splunk Add-on for AWS (Heavy Forwarder)

  • On the Splunk heavy forwarder, open Application Management.
  • Search for and open Splunk Add-on for AWS.
  • Configure AWS credentials:
    • Access Key ID
    • Secret Access Key
    • Region: choose global (per the instructor’s note).
  • Finish the add-on configuration so Splunk can authenticate to AWS.

3) Configure CloudTrail to send logs to CloudWatch Logs

  • In AWS CloudTrail, create a new trail.
  • Set up the destination:
    • Create a new S3 bucket for CloudTrail storage.
  • Event selection:
    • Enables management events only (data events not emphasized).
    • Mentions enabling read/write access options.
  • Enable CloudWatch integration:
    • In the trail settings, scroll to CloudWatch Logs
    • Turn on CloudWatch Logs integration
    • Choose an existing IAM role (policy is set automatically)
  • Verify CloudWatch log creation:
    • In CloudWatch Logs → Log management, confirm the log stream exists
    • Note: log folders/streams may take some time to appear.

4) Configure Splunk input for CloudWatch Logs

  • In Splunk, open the Splunk Add-on for AWS application.
  • Create new input data:
    • Select CloudWatch
    • Configure region (example: Northern Virginia / US East (N. Virginia))
    • Use the default source type (CloudWatch)
    • Keep default index values (logs go to the main index)
  • Optional advanced settings:
    • A polling period controls how often the script runs
  • Expected delay:
    • Input may take ~5–10 minutes to show data in Splunk (as stated).

The tutorial indicates the ingestion latency can be noticeable before data appears in Splunk.

Q&A points captured in subtitles

  • Why integrate CloudWatch with CloudTrail?
    • Because it supports near real-time ingestion for analysis and notifications, while S3-based collection can introduce delay.
  • A follow-up suggests that in real scenarios you wouldn’t manually configure CloudWatch/CloudTrail beyond setting credentials/keys (the subtitle wording is unclear).

Main speakers / sources (from the subtitles)

  • Dheeraj (main instructor demonstrating the setup)
  • Soft Mania community (the tutorial series source)

Original video