AWS Certified Data Engineer — Associate (DEA-C01)

The default answer for real-time streaming ingestion with custom consumers and replay. Know shard capacity (1 MB/s or 1,000 records/s in, 2 MB/s out per shard), provisioned vs on-demand mode, retention (24 hours default, up to 365 days), enhanced fan-out for multiple consumers, KPL/KCL, and the causes of ProvisionedThroughputExceeded (hot partition keys, too few shards). Expect 4-6 questions involving Kinesis, often as "which ingestion service" comparisons against Firehose and MSK.

Renamed Amazon Data Firehose in 2024 — the exam may use either name. It is the zero-administration delivery service: near-real-time (buffered, not real-time), fully managed scaling, delivery to S3, Redshift, OpenSearch, Splunk, and HTTP endpoints. Know buffering hints (size and interval), inline Lambda transformation, record format conversion to Parquet/ORC, and dynamic partitioning for S3 prefixes. Firehose vs Data Streams is one of the most common discriminator questions on the exam: Firehose when you just need data delivered, Data Streams when you need custom consumers, sub-second latency, or replay. Expect 3-5 questions.

Managed Apache Kafka. The exam tests it as the answer when a question specifies Kafka compatibility, existing Kafka producers/consumers, or very high throughput with topic-based semantics. Know MSK provisioned vs MSK Serverless, MSK Connect for source/sink connectors, and IAM access control vs SASL/SCRAM vs mTLS authentication. Expect 1-2 questions, usually as a comparison with Kinesis.

Formerly Kinesis Data Analytics. The answer when a question needs stateful stream processing — windowed aggregations, streaming joins, anomaly detection on in-flight data — with results written back to another stream or sink. Know the one-line distinction: Firehose delivers, Flink computes. Usually one question.

Database Migration Service is the exam's answer for replicating relational data into the analytics stack. Know the three task types (full load, full load plus CDC, CDC only), S3 as a target for landing data in a lake, and that ongoing replication (CDC) is how you keep a warehouse or lake in sync with an OLTP source. DMS Schema Conversion and the standalone AWS SCT handle heterogeneous engine conversions. Expect 2-3 questions.

The no-code answer for ingesting SaaS data — Salesforce, SAP, Google Analytics, ServiceNow — into S3 or Redshift. Know that flows run on a schedule, in response to events, or on demand, and that field mapping, filtering, and validation happen in the flow itself. Usually one question where the giveaway word is a SaaS product name.

Learn the one-line distinctions: DataSync for automated online transfer from NFS/SMB/HDFS/on-premises object storage into S3, EFS, or FSx; Transfer Family for managed SFTP/FTPS/FTP endpoints backed by S3 (partners push files to you); Snow Family for offline bulk transfer when bandwidth makes online transfer impractical. Each is typically a single discriminator question or a distractor.

Glue is the single most tested service on the DEA-C01 — the serverless Spark ETL engine plus the Data Catalog (covered in Chapter 9). Know job types (Spark, Spark streaming, Python shell), worker types and DPUs, job bookmarks for incremental processing, Glue Studio visual authoring, connections for JDBC sources, Flex execution for cost savings, and common failure modes (out-of-memory means more or larger workers, or fixing skew). Counting the catalog, crawlers, and DataBrew, expect 8-12 questions that involve Glue somewhere.

Visual, no-code data preparation with 250+ built-in transformations. The exam's answer whenever a question says analysts (not engineers) need to clean or normalise data without writing code. Know recipes (reusable transformation steps), profile jobs (statistics and anomaly detection), and data quality rules. Expect 1-3 questions — DataBrew also reappears in Domain 3 for profiling and quality.

Managed big-data frameworks (Spark, Hive, Presto/Trino, HBase, Flink) when you need cluster-level control that Glue doesn't give you, or when migrating an existing Hadoop/Spark estate. Know cluster composition (primary, core, task nodes), transient vs long-running clusters, Spot Instances for task nodes only (core nodes hold HDFS), instance fleets vs instance groups, managed scaling, EMRFS for S3, and the serverless options (EMR Serverless, EMR on EKS). Glue vs EMR is a classic discriminator: Glue for serverless ETL simplicity, EMR for framework flexibility, custom configuration, or cost at very large scale. Expect 2-4 questions.

Lambda is the glue code of every pipeline on this exam: event-driven transformation on S3 puts, Firehose record transformation, Kinesis/DynamoDB Streams/SQS consumers via event source mappings, and lightweight scheduled jobs. Know the hard limits that decide questions — 15-minute timeout, memory up to 10 GB (CPU scales with it), 512 MB to 10 GB ephemeral /tmp storage, EFS mounts for larger shared state — plus concurrency controls and batching behaviour. If a transformation is big or long-running, the answer is Glue or EMR, not Lambda. Expect 3-5 questions.

Format questions appear all over the exam. Memorise the profile of each: Parquet and ORC are columnar, splittable, compressed — the default answer for analytics on S3 because Athena and Redshift Spectrum scan (and bill) fewer bytes; Avro is row-based with embedded schema, good for record-oriented streaming and schema evolution; CSV/JSON are human-readable but slow and expensive to scan. Know compression trade-offs: Snappy is fast with moderate ratio (default in Parquet), gzip compresses more but is not splittable as a standalone stream, bzip2 is splittable. "Convert to Parquet and partition" is the single most repeated correct answer on this exam.

The serverless orchestration answer. Know Standard vs Express workflows (long-running with exactly-once semantics vs high-volume short-duration), state types (Task, Choice, Parallel, Map, Wait), error handling with Retry and Catch, direct SDK integrations with Glue, EMR, Athena, and Redshift Data API, and the Distributed Map state for massively parallel processing of S3 objects. Step Functions vs MWAA is a reliable question: Step Functions for serverless AWS-native workflows, MWAA when the team already lives in Airflow. Expect 2-4 questions.

Managed Workflows for Apache Airflow. The answer when a question mentions existing Airflow DAGs, Python-defined workflows, or open-source orchestration portability. Know that DAGs, plugins, and requirements files live in an S3 bucket, environments come in small/medium/large classes with autoscaling workers, and that MWAA runs inside your VPC. Common troubleshooting question: a DAG doesn't appear — check the S3 path and requirements.txt dependency errors in CloudWatch. Expect 1-3 questions.

The event router and scheduler of the pipeline world. Know rules with event patterns (react to S3 object created, Glue job state change, Redshift events), scheduled rules with cron and rate expressions, and EventBridge Scheduler for one-off and recurring invocations. EventBridge is how "event-driven ingestion" and "scheduled ingestion" get implemented in most correct answers. Expect 2-3 questions.

Glue's built-in orchestration: workflows chain crawlers and jobs with shared run properties, and triggers start them on a schedule, on demand, or when preceding jobs complete (conditional triggers). The exam likes Glue workflows as the "least additional infrastructure" answer when everything in the pipeline is already Glue. Expect 1-2 questions.

In DEA-C01 these appear as pipeline plumbing: SNS for alerting on job failure (EventBridge rule to SNS topic to email) and fan-out, SQS for buffering and decoupling ingestion stages, dead-letter queues for failed events, and FIFO queues when ordering matters. Know visibility timeout and long polling at a definition level. Expect 1-2 questions.

Pipelines are deployed as code on this exam. Know the roles: CloudFormation is the declarative template engine underneath everything; CDK defines infrastructure in TypeScript/Python and synthesises to CloudFormation; SAM is the shorthand for serverless stacks (Lambda, Step Functions, DynamoDB) with sam build and sam deploy. Expect 1-2 questions, usually "how do we make this deployment repeatable across environments".

Know the developer-tools suite at a component level: CodeCommit (Git hosting — closed to new AWS customers since 2024 but still named in the exam guide), CodeBuild (run tests and package artifacts), CodeDeploy (deploy), CodePipeline (orchestrate all of it). Be comfortable with basic Git operations — clone, branch, commit, merge — at the vocabulary level. Usually one question about promoting pipeline code through environments.

The exam assumes working SQL: joins, aggregation with GROUP BY, window functions for rolling averages, CTEs, and pivoting. Know Redshift stored procedures for encapsulating transformation logic inside the warehouse, and the basics of query optimization — filter early, select only needed columns, use partition/sort keys so the engine can prune. You won't write SQL on the exam, but several questions describe a query and ask why it is slow or expensive. Expect 2-3 questions.

Redshift is the second most tested service after Glue. Know the architecture (leader node compiles queries, compute nodes execute; RA3 nodes separate compute from managed storage), Redshift Serverless (pay per RPU-second — the "least management" answer), distribution styles (KEY for join columns, ALL for small dimension tables, EVEN as fallback, AUTO default), sort keys for range-restricted scans, COPY for parallel loads from S3 (many files, roughly equal size), UNLOAD to write back to S3, VACUUM and ANALYZE maintenance, and workload management with concurrency scaling. Expect 6-9 questions across the exam.

The "query data where it lives" features are heavily tested. Spectrum queries S3 data through external tables in the Glue Data Catalog without loading it; federated queries reach into RDS/Aurora PostgreSQL and MySQL live; materialized views precompute expensive joins and aggregations (with incremental refresh, and auto-refresh for streaming ingestion from Kinesis/MSK); data sharing gives other Redshift warehouses live read access without copying — the exam's answer for cross-team and cross-account warehouse access. Expect 3-4 questions.

S3 is the substrate of every architecture on this exam. Know storage classes cold (Standard, Intelligent-Tiering for unknown access patterns, Standard-IA and One Zone-IA, the three Glacier classes), request performance (3,500 writes and 5,500 reads per second per prefix — parallelise across prefixes), Hive-style partition layout (key=value folders that Athena and Glue understand), multipart upload for large objects, and event notifications to Lambda/SQS/EventBridge for event-driven pipelines. Expect 4-6 questions, mostly embedded in cost and performance scenarios.

Lake Formation layers governance over S3 and the Glue Data Catalog: register S3 locations, then grant database, table, column, row, and cell-level permissions that Athena, Redshift Spectrum, EMR, and Glue all enforce. Know LF-tags (tag-based access control that scales better than named-resource grants) and cross-account sharing. Whenever a question asks for fine-grained (column or row) access control on a data lake, or one place to manage permissions for many analytics services, Lake Formation is the answer. Expect 3-5 questions across Domains 2 and 4.

DynamoDB appears as the low-latency key-value store in pipelines and as a change-capture source. Know partition and sort keys, hot-partition symptoms and remedies (better key distribution, write sharding), provisioned vs on-demand capacity, DynamoDB Streams feeding Lambda for CDC, TTL for automatic expiry, and export to S3 for analytics without consuming read capacity. If a question needs SQL analytics on DynamoDB data, the pattern is export (or stream) to S3 and query with Athena. Expect 2-4 questions.

Relational engines appear as pipeline sources and as the wrong answer for analytics at scale. Know read replicas for offloading extract queries, Multi-AZ for availability, row-level locking behaviour as a contrast with Redshift's table-level model, and Aurora zero-ETL integration with Redshift — the managed alternative to building a DMS/CDC pipeline for near-real-time warehouse sync. Expect 1-3 questions.

Learn one line each, mostly to eliminate distractors: DocumentDB for MongoDB-compatible JSON documents; Neptune for graph data (relationships, recommendation, fraud rings — the answer when the exam guide's "graph data structures" shows up as a use case); Keyspaces for Apache Cassandra workloads; MemoryDB for a durable Redis-compatible in-memory store. Expect 1-2 questions total.

The Data Catalog is the metadata backbone: one Hive-metastore-compatible catalog per Region that Athena, Redshift Spectrum, EMR, and Glue ETL all read. Know crawlers (infer schemas and partitions from S3, JDBC, and DynamoDB sources, on demand or scheduled), classifiers (built-in and custom for unusual formats), how new S3 partitions get registered (re-run the crawler, MSCK REPAIR TABLE in Athena, or partition projection), partition indexes for faster query planning, and the Glue Schema Registry for enforcing Avro/JSON/Protobuf schemas on streams. Expect 3-5 questions.

Lifecycle questions are near-guaranteed. Know transition actions (move objects to colder classes after N days) and expiration actions (delete after N days, including noncurrent versions in versioned buckets), the Glacier retrieval classes (Instant Retrieval for millisecond access, Flexible Retrieval in minutes to hours, Deep Archive at 12+ hours and lowest cost), Intelligent-Tiering when access patterns are unknown, and the minimum-storage-duration charges that make transitioning tiny short-lived objects a wrong answer. Replication (same-Region and cross-Region) covers the resiliency bullet. Expect 2-4 questions.

The recurring load/unload pattern: COPY from S3 into Redshift for hot analytics, UNLOAD back to S3 (as Parquet) when data cools, then query it with Spectrum or Athena — keeping the cluster small and the history cheap. Loading best practice is many similar-sized files (ideally a multiple of the cluster's slice count), compressed, via a single COPY. DynamoDB TTL handles record-level expiry; AWS Backup centralises backup plans and retention across services. Expect 1-2 questions.

Know star and snowflake schemas (facts and dimensions; star denormalises dimensions for fewer joins), and how they map to Redshift physical design — distribute the fact table and its biggest dimension on the join key, DISTSTYLE ALL for small dimensions, sort keys on the common filter columns. For DynamoDB, model around access patterns (single-table design, composite keys), not around normal forms. Partitioning, compression, and columnar formats are the optimisation levers for lake tables. Expect 2-3 questions.

Know the tools by name: AWS SCT and DMS Schema Conversion convert schemas between database engines during migration; the Glue Schema Registry manages schema versions and compatibility modes (backward, forward, full) for streaming data — Avro being the format built for evolution; crawlers can update the catalog when lake schemas drift. SageMaker ML Lineage Tracking is the exam guide's named example for data lineage. Usually one question.

Repeatable operations means code calling APIs. Know the Redshift Data API — run SQL against provisioned or serverless Redshift over HTTPS from Lambda and Step Functions with no persistent connection or driver, results retrieved asynchronously. The same pattern exists for Athena (StartQueryExecution) and Glue (StartJobRun). EventBridge schedules and event rules kick these off; Lambda stitches them together. Expect 1-3 questions, often "trigger this SQL after that job with the least infrastructure".

Managed batch computing on ECS/EC2/Fargate: job definitions, job queues, and compute environments. The answer when a workload is containerised batch processing that isn't Spark-shaped and outlives Lambda's 15 minutes. Learn the one-line distinction against Glue (Spark ETL), EMR (big-data frameworks), and Lambda (short event-driven functions). Usually one question or a distractor.

Serverless SQL over S3, billed per terabyte scanned — which is why every Athena cost/performance question is answered by "scan less": columnar formats, compression, partitioning, and partition projection (compute partitions from a template instead of the catalog — also fixes the "new partitions don't show up" problem). Know CTAS for materialising transformed tables, views, workgroups for per-team query limits and cost controls, federated queries to non-S3 sources through Lambda connectors, and Athena notebooks running Apache Spark for exploratory work. Expect 5-7 questions.

The visualization answer. Know SPICE (in-memory engine — import data for fast dashboards and to stop hammering the source) vs direct query (always current, source pays the cost), scheduled SPICE refreshes, row-level security for per-user filtering, and embedded dashboards. If a question wants business users to explore data visually with no infrastructure, it's QuickSight. Expect 1-2 questions.

Beyond DataBrew and Athena, know the exam guide's named exploration tools at recognition level: SageMaker Data Wrangler for visual data preparation feeding ML workflows, Jupyter notebooks (on SageMaker or EMR) for hands-on exploration, and Glue interactive sessions for on-demand Spark development without a cluster. These mostly show up as answer options rather than question subjects.

The default monitoring answer for every pipeline service. Know metrics, alarms (alarm to SNS is the standard alerting pattern), log groups and retention settings, CloudWatch Logs Insights for querying logs with its purpose-built query language, metric filters to turn log patterns into metrics, and dashboards. Glue, EMR, Lambda, Redshift, and MWAA all emit to CloudWatch — questions often hinge on knowing that the evidence for a failure lives in a specific service's CloudWatch logs. Expect 3-4 questions.

Know where the evidence is: Glue job runs emit metrics and continuous logs to CloudWatch, and the Spark UI exposes stages, skew, and shuffle spills; EMR writes step and application logs to CloudWatch and S3. Common exam scenarios and fixes — Glue OOM errors (more or larger workers, fix skewed keys, enable pushdown), jobs reprocessing old data (enable job bookmarks), slow queries on many small files (compact them), and long-running EMR steps (managed scaling, right instance types). Expect 2-3 questions.

The exam guide names OpenSearch as a log-analysis destination: ship application or pipeline logs (often via Firehose) and search or dashboard them in near real time. Learn it as the "operational search and log analytics at scale" option versus Logs Insights (ad hoc queries on CloudWatch) and Athena (SQL on logs archived in S3). Usually one question.

Two named tools. Glue Data Quality evaluates rules written in DQDL (Data Quality Definition Language) — completeness, uniqueness, freshness — against catalog tables or inside ETL jobs, with recommended rules generated from the data, and can fail a pipeline or just report. DataBrew profile jobs produce column statistics, distributions, and anomaly signals, and DataBrew supports its own data quality rules and validation reports. Know the four validation dimensions from the exam guide: completeness, consistency, accuracy, integrity. Expect 2-3 questions.

Roles everywhere, users nowhere: Glue jobs, Lambda functions, Redshift COPY, and crawlers all act through IAM roles, and cross-service questions usually come down to which role is missing which permission (a Redshift COPY failing to read S3 means the cluster's associated role lacks s3:GetObject). Know policy structure, identity-based vs resource-based policies, AWS managed vs customer managed policies, evaluation logic (explicit deny wins), and least privilege as the default tiebreaker between answer options. Expect 3-4 questions.

Database credentials for Glue connections, MWAA connections, and Lambda must never be hardcoded. Secrets Manager stores and automatically rotates credentials (built-in rotation for RDS, Redshift, and DocumentDB) — it is the answer whenever "rotate" appears. Parameter Store holds configuration and SecureStrings without built-in rotation, with a free standard tier. Expect 1-2 questions.

Know the plumbing that keeps pipeline traffic private: security groups (stateful, instance/ENI level — Glue JDBC connections need a self-referencing rule), gateway VPC endpoints for S3 and DynamoDB (free, route-table based), interface endpoints via PrivateLink for most other services, and Redshift enhanced VPC routing to force COPY/UNLOAD traffic through the VPC. IP allowlisting for partner data sources means security group rules or firewall entries. Expect 1-3 questions.

The centre of gravity for authorization questions. Lake Formation replaces bucket-policy sprawl with grant/revoke permissions on catalog resources — database, table, column, row, and cell level — enforced consistently by Athena, Redshift Spectrum, EMR, Glue, and QuickSight. Know named-resource grants vs LF-tag-based access control (tags scale; the exam favours them for large estates), data filters for row/cell security, and cross-account sharing of catalog resources. Expect 2-4 questions.

Inside Redshift, authorization is SQL: GRANT and REVOKE on schemas, tables, and columns; role-based access control with CREATE ROLE for bundling privileges; row-level security policies; and IAM-based temporary credentials for sign-in instead of passwords. For DynamoDB, IAM condition keys give fine-grained (item and attribute level) access. Least privilege phrasing decides most of these questions. Expect 1-2 questions.

KMS keys sit under encryption in every data service. Know AWS managed keys vs customer managed keys (use customer managed when you need key policies, rotation control, or cross-account access), envelope encryption, and that cross-account use requires both a key policy grant on the key and IAM permissions on the caller. Client-side encryption means the data is encrypted before AWS sees it; server-side means the service encrypts on write. Expect 2-3 questions.

Per-service encryption facts that decide questions: S3 encrypts everything at rest by default (SSE-S3), with SSE-KMS for auditable key control; Redshift clusters can be KMS-encrypted (set at creation); EMR uses security configurations for at-rest (EMRFS/local disk) and in-transit encryption; Glue uses security configurations for job bookmarks, CloudWatch logs, and S3 targets. In transit is TLS everywhere. Expect 1-2 questions.

Distinguish the techniques: masking hides values while keeping the column usable (Redshift dynamic data masking applies masking policies at query time by user role); anonymization removes the link to a person irreversibly; hashing with salts prevents rainbow-table reversal of identifiers; and Lake Formation data filters simply deny access to sensitive columns and cells. DataBrew includes PII redaction and hashing transformations for masking at preparation time. Expect 1-2 questions.

CloudTrail answers "who called what API when": management events by default, data events (S3 object-level, Lambda invocations) opt-in, trails delivering to S3 for retention. CloudTrail Lake is the exam's named answer for centralized, SQL-queryable audit logging without building your own Athena pipeline — immutable event data stores queried directly. Expect 1-2 questions.

Match the tool to the log location: CloudWatch Logs Insights for logs already in CloudWatch, Athena for logs archived in S3 (CloudTrail, ALB, VPC Flow Logs — a very common pattern question), OpenSearch for interactive search at scale, and EMR when log volumes are genuinely big-data sized. Usually one question.

Macie is the managed answer for finding PII: it continuously discovers sensitive data in S3 using managed data identifiers (names, credentials, card numbers) plus custom identifiers, produces findings you can route via EventBridge, and pairs with Lake Formation to lock down what it finds. If the question says "discover" or "identify" PII in S3, it's Macie; if it says "mask" or "restrict access to" columns, it's back to Chapter 18. Expect 1-2 questions.

AWS Config records resource configuration changes and evaluates rules (an unencrypted bucket or a public Redshift cluster triggers noncompliance) — the answer for "track configuration changes in the account". Data sovereignty questions are about keeping data in allowed Regions: restrict replication destinations, and deny disallowed Regions with IAM or SCP condition keys. Redshift data sharing (Chapter 6) reappears here as the governed way to share warehouse data without copying. Usually one question.