Question 1

As a Gen AI engineer working on improving a search engine’s ranking algorithm, you want to evaluate the effectiveness of your new ranking model. Which metric would you use to assess how well the model ranks the most relevant documents higher, taking into account both the position of the documents and their relevance?

Options :

A : Normalized Discounted Cumulative Gain (NDCG)

B : Mean Reciprocal Rank (MRR)

C : Precision at k

D : F1 Score

Answer: A

Question 2

A Generative Al Engineer has already trained an LLM on Databricks and it is now ready to be deployed. Which of the following steps correctly outlines the easiest process for deploying a model on Databricks?

Options :

A :

Log the model as a pickle object, upload the object to Unity Catalog Volume, register it to Unity Catalog using MLflow, and start a serving endpoint

B :

Log the model using MLflow during training, directly register the model to Unity Catalog using the MLflow API, and start a serving endpoint

C :

Save the model along with its dependencies in a local directory, build the Docker image, and run the Docker container

D : Wrap the LLMs prediction function into a Flask application and serve using Gunicorn

Answer: B

Question 3

As a GenAI engineer, how would you describe the role of MLOps in managing machine learning systems?

Options :

A : MLOps is exclusively concerned with developing and testing machine learning models before they are deployed into production.

B : MLOps is limited to managing code repositories and version control for machine learning projects.

C : MLOps primarily focuses on automating data collection processes and managing databases for business intelligence.

D : MLOps is automation for managing data, code and models to improve performance, stability and long-term efficiency of ML systems.

Answer: D

Question 4

What is a key challenge when implementing reranking in a Retrieval-Augmented Generation (RAG) pipeline for a Gen AI application?

Options :

A : Reranking simplifies the RAG pipeline by reducing the overall complexity and streamlining the process.

B : Reranking requires minimal computational resources, making it an efficient process for large datasets.

C : Reranking eliminates the need for any further processing or adjustments to the retrieved documents before generating a response.

D : Reranking involves multiple calls to the LLM, which can significantly increase both the cost and latency of the RAG chain.

Answer: D

Question 5

As a Gen AI engineer setting up LLM-as-a-Judge for evaluating your model outputs, which of the following best practices should you follow to ensure effective evaluation?

Options :

A : Employ a wide variety of examples, including only examples of good outputs, to guide the LLM’s evaluation process.

B : Use a broad rubric scale, such as 1-10, to capture nuanced differences in the quality of outputs.

C : Use a high-token LLM to provide more context and detail in the evaluation, improving the quality of feedback from the model.

D : Avoid using an additive scale, as it can complicate the evaluation process and introduce inconsistencies.

Answer: C

Higher Test Marks with Free Online Databricks-Certified-Generative-AI-Engineer-Associate Exam Practice

Buying Options: