You are building a new application that you need to collect data from in a scalable way. Data arrives continuously from the application throughout the day, and you expect to generate approximately 150 GB of JSON data per day by the end of the year. Your requirements are: Decoupling producer from consumer Space and cost-efficient storage of the raw ingested data, which is to be stored indefinitely Near real-time SQL query Maintain at least 2 years of historical data, which will be queried with SQ Which pipeline should you use to meet these requirements?

• A. Write a tool to poll the API and write data to Cloud Storage as gzipped JSON files.
• B. Create an application that writes to a Cloud SQL database to store the dat
• C. Create an application that publishes events to Cloud Pub/Sub, and create a Cloud Dataflow pipeline that transforms the JSON event payloads to Avro, writing the data to Cloud Storage and BigQuery.
• D. Create an application that publishes events to Cloud Pub/Sub, and create Spark jobs on Cloud Dataproc to convert the JSON data to Avro format, stored on HDFS on Persistent Disk.
• E. Create an application that provides an AP
• F. Set up periodic exports of the database to write to Cloud Storage and load into BigQuery.

You are planning to migrate your current on-premises Apache Hadoop deployment to the cloud. You need to ensure that the deployment is as fault-tolerant and cost-effective as possible for long-running batch jobs. You want to use a managed service. What should you do?

• A. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://
• B. Change references in scripts from hdfs:// to gs://
• C. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://
• D. Use an SSD persistent disk and 50% preemptible worker
• E. Install Hadoop and Spark on a 10-node Compute Engine instance group with preemptible instances.Store data in HDF
• F. Change references in scripts from hdfs:// to gs://
• G. Deploy a Cloud Dataproc cluste
• H. Install the Cloud Storage connector, and store the data in Cloud Storag
• I. Install Hadoop and Spark on a 10-node Compute Engine instance group with standard instance
• J. Use a standard persistent disk and 50% preemptible worker
• K. Deploy a Cloud Dataproc cluste

You are developing a software application using Google's Dataflow SDK, and want to use conditional, for loops and other complex programming structures to create a branching pipeline. Which component will be used for the data processing operation?

• A. Transform
• B. Sink API
• C. PCollection
• D. Pipeline

You receive data files in CSV format monthly from a third party. You need to cleanse this data, but every third month the schema of the files changes. Your requirements for implementing these transformations include:
Executing the transformations on a schedule
Enabling non-developer analysts to modify transformations
Providing a graphical tool for designing transformations
What should you do?

• A. Merge the transformed tables together with a SQL query
• B. Load each month's CSV data into BigQuery, and write a SQL query to transform the data to a standard scheme
• C. Use Apache Spark on Cloud Dataproc to infer the schema of the CSV file before creating a Dataframe.Then implement the transformations in Spark SQL before writing the data out to Cloud Storage and loading into BigQuery
• D. Help the analysts write a Cloud Dataflow pipeline in Python to perform the transformatio
• E. The Python code should be stored in a revision control system and modified as the incoming data's schema changes
• F. Use Cloud Dataprep to build and maintain the transformation recipes, and execute them on a scheduled basis