Question 1

As a Gen AI engineer, you are preparing to evaluate your language model both before and after deployment. Which of the following approaches best describes the key focus of online evaluation compared to offline evaluation?

Options :

A : Offline evaluation focuses on analyzing real user behavior data collected from live deployments, while online evaluation involves curating benchmark data and using task-specific metrics.

B : Offline evaluation is concerned with collecting user feedback through live interactions, whereas online evaluation relies on predefined benchmark datasets and task-specific metrics.

C : Online evaluation requires extensive preparation of benchmark datasets before deployment, while offline evaluation is centered on real-time monitoring of user interactions with the model.

D : Online evaluation involves deploying the app and assessing the model based on real user interactions and feedback, whereas offline evaluation is conducted using curated benchmark data and task-specific metrics.

Answer: D

Question 2

Which of the following parameters can affect the determinism and creativity of an LLM?

Options :

A : All of these options

B : "Top-k" value

C : "Top-p" value

D : "Temperature" value

Answer: A

Question 3

As a Gen AI engineer, you’re tasked with monitoring your AI system to ensure it performs effectively and cost-efficiently. Which of the following scenarios best represents an effective monitoring strategy for diagnosing potential issues in your AI model?

Options :

A : Monitoring the system-s user interface for aesthetic design improvements and user experience enhancements

B : Regularly reviewing input data, prompt queries and responses to identify any inconsistencies or anomalies.

C : Focusing solely on analyzing performance/cost details without reviewing mid-training checkpoints or human feedback.

D : Evaluating the model’s performance only during final deployment phases to ensure it meets all business requirements

Answer: B

Question 4

As a Gen AI engineer developing an intelligent system that integrates various external tools and APIs, which statement best describes the role of agents?

Options :

A : Agents only handle coding tasks and do not interact with other types of tools.

B : Agents rely solely on search tools, like web browsing and search engines, for all tasks.

C : Agents use pre-configured tools without considering the context of the task.

D : "Agents' reasoning skills decide which tools to use and when/how to use them.

Answer: D

Question 5

As a Gen AI engineer working on batch inference with MLflow, which of the following methods are correctly used for performing batch inference in single-node and multi-node environments? Choose two correct answers.

Options :

A : Implement mlflow.pyfunc.spark_udf(spark, model_uri=...) and result_df = prod_df.withColumn("result", model_udf("input")) for multi-node batch inference.

B : Apply mlflow.pyfunc.spark_udf(spark, model_uri=...) and .predict() for single-node batch inference.

C : Utilize mlflow.pyfunc.load_model() and result_df = prod_df.withColumn("result", model_udf("input")) for multi-node batch processing.

D : Use mlflow.pyfunc.load_model() and .predict() for single-node batch inference.

Answer: A,D

Higher Test Marks with Free Online Databricks-Certified-Generative-AI-Engineer-Associate Exam Practice

Buying Options: