Question 1

As a Gen AI engineer, you are preparing to evaluate your language model both before and after deployment. Which of the following approaches best describes the key focus of online evaluation compared to offline evaluation?

Options :

A : Offline evaluation focuses on analyzing real user behavior data collected from live deployments, while online evaluation involves curating benchmark data and using task-specific metrics.

B : Offline evaluation is concerned with collecting user feedback through live interactions, whereas online evaluation relies on predefined benchmark datasets and task-specific metrics.

C : Online evaluation requires extensive preparation of benchmark datasets before deployment, while offline evaluation is centered on real-time monitoring of user interactions with the model.

D : Online evaluation involves deploying the app and assessing the model based on real user interactions and feedback, whereas offline evaluation is conducted using curated benchmark data and task-specific metrics.

Answer: D

Question 2

Which of the following features is offered by Databricks Model Serving for AI models?

Options :

A : It does not offer any monitoring or payload logging features for external models like OpenAI or Anthropic.

B : It requires separate tools and interfaces for managing foundation models and external models.

C : It only supports models built with MLflow, limiting the integration of external models.

D : It provides a unified UI, API, and SDK for managing custom models, foundation models, and external models, ensuring seamless governance, monitoring, and logging.

Answer: D

Question 3

How can a Generative AI Engineer handle complex documents (with mixed text and images) for data extraction and chunking for a RAG based LLM application?

Options :

A : Using python libraries like PyPDF

B : Using Multi-modal models like OpenAI-s GPT-4o and beyond

C : All of these options

D : Using layout model from Hugging Face

Answer: C

Question 4

As a GenAI engineer, how would you describe the role of MLOps in managing machine learning systems?

Options :

A : MLOps is exclusively concerned with developing and testing machine learning models before they are deployed into production.

B : MLOps is limited to managing code repositories and version control for machine learning projects.

C : MLOps primarily focuses on automating data collection processes and managing databases for business intelligence.

D : MLOps is automation for managing data, code and models to improve performance, stability and long-term efficiency of ML systems.

Answer: D

Question 5

A Generative Al Engineer is responsible for developing a chatbot to enable their companys internal HelpDesk Call Center team to more quickly find related tickets and provide resolution. While creating the GenAI application work breakdown tasks for this project, they realize they need to start planning which data sources (either Unity Catalog volume or Delta table) they could choose for this application. They have collected several candidate data sources for consideration: call_rep_history: a Delta table with primary keys representative_id, call_id. This table is maintained to calculate representatives call resolution from fields call_duration and call start_time. transcript Volume: a Unity Catalog Volume of all recordings as a *.wav files, but also a text transcript as *.txt files. call_cust_history: a Delta table with primary keys customer_id, cal1_id. This table is maintained to calculate how much internal customers use the HelpDesk to make sure that the charge back model is consistent with actual service use. call_detail: a Delta table that includes a snapshot of all call details updated hourly. It includes root_cause and resolution fields, but those fields may be empty for calls that are still active. maintenance_schedule “ a Delta table that includes a listing of both HelpDesk application outages as well as planned upcoming maintenance downtimes. They need sources that could add context to best identify ticket root cause and resolution. Which TWO sources do that? (Choose two.)

Options :

A : call_cust_history

B : maintenance_schedule

C : call_rep_history

D : call_detail

E : transcript Volume

Answer: D,E

Higher Test Marks with Free Online Databricks-Certified-Generative-AI-Engineer-Associate Exam Practice

Buying Options: