Databricks-Generative-AI-Engineer-Associate Questions Bank

Problem Context : The engineer needs a cost-effective deployment strategy for an LLM application with relatively low request volume.

Explanation of Options :

Option A : Switching to external models may not provide the required control or integration necessary for specific application needs.

Option B : Using a pay-per-token model is cost-effective, especially for applications with variable or low request volumes, as it aligns costs directly with usage.

Option C : Changing to a model with fewer parameters could reduce costs, but might also impact the performance and capabilities of the application.

Option D : Manually throttling requests is a less efficient and potentially error-prone strategy for managing costs.

Option B is ideal, offering flexibility and cost control, aligning expenses directly with the application ' s usage patterns.

The Generative AI Engineer needs to follow a methodical pipeline to build and deploy a Retrieval-Augmented Generation (RAG) application. The steps outlined in option B accurately reflect this process:

Ingest documents from a source : This is the first step, where the engineer collects documents (e.g., technical regulations) that will be used for retrieval when the application answers user questions.

Index the documents and save to Vector Search : Once the documents are ingested, they need to be embedded using a technique like embeddings (e.g., with a pre-trained model like BERT) and stored in a vector database (such as Pinecone or FAISS). This enables fast retrieval based on user queries.

User submits queries against an LLM : Users interact with the application by submitting their queries. These queries will be passed to the LLM.

LLM retrieves relevant documents : The LLM works with the vector store to retrieve the most relevant documents based on their vector representations.

LLM generates a response : Using the retrieved documents, the LLM generates a response that is tailored to the user ' s question.

Evaluate model : After generating responses, the system must be evaluated to ensure the retrieved documents are relevant and the generated response is accurate. Metrics such as accuracy, relevance, and user satisfaction can be used for evaluation.

Deploy it using Model Serving : Once the RAG pipeline is ready and evaluated, it is deployed using a model-serving platform such as Databricks Model Serving. This enables real-time inference and response generation for users.

By following these steps, the Generative AI Engineer ensures that the RAG application is both efficient and effective for the task of answering technical regulation questions.

The task at hand is to improve the LLM’s ability to generate poem-like article summaries with the desired tone and style. Using a neutralizer to normalize the tone and style of the underlying documents (option B) will not help improve the LLM’s ability to generate the desired poetic style. Here’s why:

Neutralizing Underlying Documents : A neutralizer aims to reduce or standardize the tone of input data. However, this contradicts the goal, which is to generate text with a specific tone and style (like haikus). Neutralizing the source documents will strip away the richness of the content, making it harder for the LLM to generate creative, stylistic outputs like poems.

Why Other Options Improve Results :

A (Explicit Instructions in the Prompt) : Directly instructing the LLM to generate text in a specific tone and style helps align the output with the desired format (e.g., haikus). This is a common and effective technique in prompt engineering.

C (Few-shot Examples) : Providing examples of the desired output format helps the LLM understand the expected tone and structure, making it easier to generate similar outputs.

D (Fine-tuning the LLM) : Fine-tuning the model on a dataset that contains examples of the desired tone and style is a powerful way to improve the model’s ability to generate outputs that match the target format.

Therefore, using a neutralizer (option B) is not an effective method for achieving the goal of generating stylized poetic summaries.

According to Generative AI engineering standards for Retrieval-Augmented Generation (RAG), chunking strategy is a critical optimization variable. Setting the chunk size to match the model ' s maximum context window (e.g., 4k or 8k tokens) is a poor practice and should be discarded. Large chunks introduce significant " noise " into the LLM ' s context, as only a small portion of a massive chunk usually contains the answer to a specific query. This leads to the " lost in the middle " phenomenon where LLMs struggle to extract relevant information from bloated contexts. Furthermore, large chunks reduce the precision of the vector search. Standard best practices involve using smaller, semantically meaningful chunks (typically 256–512 tokens) with overlap to maintain context. In contrast, metadata filtering (B) is essential for narrowing searches to specific car years, fine-tuning embeddings (C) improves retrieval accuracy for domain-specific technical terms, and few-shot examples (D) guide the LLM ' s output format and tone.