Amazon Web Services Data-Engineer-Associate Exam With Confidence Using Practice Dumps

Option B is most cost-effective because it combines a serverless ETL service for overnight processing with a serverless warehouse for partner SQL access during business hours. The material describes AWS Glue as a “fully managed, serverless, scalable ETL service” that can extract, transform, and load data with minimal operational overhead, which fits the requirement for multiple transformations and data cleaning.

For the SQL access layer, the study material highlights the cost advantage of Amazon Redshift Serverless: it automatically provisions and scales capacity and you pay only for the compute capacity provisioned, with no compute costs when no workloads are running. That directly supports “multiple business partners” querying during business hours while avoiding the always-on cost of a provisioned cluster.

Options C and D use a provisioned Redshift cluster, which is less cost-effective because capacity must be paid for even when partners are not querying. Option A adds EMR cluster management cost and operational overhead compared to using Glue for ETL.

For near real-time processing of new files in S3, event-driven ingestion is optimal. S3 Event Notifications can trigger AWS Lambda to immediately load data into Amazon Redshift, eliminating latency associated with batch scheduling.

“Event-based triggers using S3 notifications and Lambda functions are effective for near real-time ingestion pipelines into Amazon Redshift.”

– Ace the AWS Certified Data Engineer - Associate Certification - version 2 - apple.pdf

Option C (Glue bookmarks) is best for batch jobs, and zero-ETL applies to Aurora to Redshift, not S3.

Amazon DynamoDB is optimized for Query operations, which retrieve items efficiently using the partition key and optionally the sort key. Queries consume significantly fewer RCUs than Scan operations because only matching partitions are read.

By performing a Query using the partition key in the key condition expression, the AWS Glue job reads only the required items directly from DynamoDB. This minimizes the number of read capacity units consumed and avoids unnecessary reads.

Using a Scan operation, even when followed by filtering in a DynamicFrame, consumes RCUs for the entire table or index and is highly inefficient. Using only the sort key in a query is invalid without the partition key and would not meet DynamoDB query requirements. The Glue DynamoDB connector filter option still performs a scan under the hood and does not minimize RCUs.

The queried data can be directly loaded into a DynamicFrame, satisfying the existing processing logic requirement without additional transformations.

Therefore, Option D provides the most efficient and cost-effective solution.