A batch processing data pipeline, using AWS resources (S3, EMR, Redshift, EC2, IAM), provisioned via Terraform, and orchestrated from locally hosted Airflow containers. The end product is a Superset ...

Database Secrets Management: The script retrieves secrets such as passwords and keys stored in AWS Secrets Manager. It obtains the secrets for database, and SMTP configurations. These secrets are then ...