Helping Hand Questions for AIF-C01

The correct answer is C, as conversion rate (how many users upgrade to premium) and retention rate (how many stay subscribed) are primary indicators of a generative AI system’s business impact and ROI. According to AWS documentation, evaluating AI performance should go beyond technical accuracy to include business metrics that show real-world value. For a premium service, higher conversion implies successful personalization that attracts free users, while strong retention reflects sustained engagement and user satisfaction. Metrics like ARPU or query reduction are secondary indicators. AWS emphasizes outcome-based measurement to ensure AI initiatives contribute measurable ROI tied to customer adoption, loyalty, and profitability. Therefore, tracking conversion and retention gives the clearest long-term measure of financial success from generative AI.

Referenced AWS AI/ML Documents and Study Guides:

AWS Machine Learning Specialty Guide – ML Business Metrics

AWS AI Adoption Framework – Value Realization and ROI Tracking

The correct answer is C because Amazon SageMaker JumpStart provides administrative controls that allow organizations to manage and restrict access to foundation models within the AWS environment.

According to the official AWS documentation:

" Amazon SageMaker JumpStart provides model access management capabilities that enable administrators to control which foundation models are visible and usable by end users. Using AWS Identity and Access Management (IAM) policies, you can restrict access to specific models or completely disable model discovery in JumpStart. "

This allows companies to enforce governance over which FMs their users can see and interact with, satisfying the requirement to control employee access to publicly available foundation models.

Explanation of other options:

A. AWS Cost Explorer is used to analyze billing and usage data but does not control access to services or models. It is helpful for budgeting and visibility, not access control.

B. AWS Artifact provides access to compliance reports and certifications, not tools for controlling user access to ML models.

D. Amazon OpenSearch Service is used for search and analytics on structured and unstructured data. It does not provide access control mechanisms for foundation models.

Referenced AWS AI/ML Documents and Study Guides:

Amazon SageMaker JumpStart Documentation – Model Access Management

AWS IAM Documentation – Restricting Access to SageMaker Resources

AWS Machine Learning Specialty Certification Guide – Security and Governance Section

The verified answer is B. Data leakage. The model exposes sensitive company information because that information was inadvertently included in training data. AWS guidance warns that using sensitive data to train models can cause unintended disclosure of data from model output, including data leakage during inference. This is exactly the scenario: sensitive information entered the training process and later appeared in model output after deployment.

AWS security reference guidance for generative AI model customization identifies sensitive information disclosure as a threat where a model trained on datasets containing PII leaks sensitive information during inference. Although the question says company information rather than PII specifically, the same security principle applies: protected data included in training or continued pre-training can be memorized or reproduced by the model later.

Jailbreaking is incorrect because jailbreaking refers to attempts to bypass model safety controls through adversarial prompts. The problem here is not that a user bypassed safety controls; the sensitive information was already embedded through training data.

Contextual grounding is incorrect because grounding is a technique for improving answer accuracy by connecting model responses to trusted context, such as RAG sources. It is not a security risk.

Prompt injection is incorrect because prompt injection involves malicious input that attempts to override or manipulate model behavior at inference time. The question does not describe a malicious prompt. It describes accidental exposure from training data.

Therefore, the correct security risk is data leakage.

Transitioning to a new large language model (LLM) on Amazon Bedrock typically involves minimal changes when the new model is pre-trained and available as a foundation model. Since the company is moving from one pre-trained LLM to another, the primary task is to ensure compatibility between the new model ' s input requirements and the existing application. Adjusting the prompt template is often necessary because different LLMs may have varying prompt formats, tokenization methods, or response behaviors, even for similar tasks like document extraction.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

" When switching between foundation models in Amazon Bedrock, you may need to adjust the prompt template to align with the new model’s expected input format and optimize its performance for your use case. Prompt engineering is critical to ensure the model understands the task and generates accurate outputs. "

(Source: AWS Bedrock User Guide, Prompt Engineering for Foundation Models)

Detailed Explanation:

Option A: Create a new labeled dataset.Creating a new labeled dataset is unnecessary when transitioning to a new pre-trained LLM, as pre-trained models are already trained on large datasets. This option would only be relevant if the company were training a custom model from scratch, which is not the case here.

Option B: Perform feature engineering.Feature engineering is typically associated with traditional machine learning models, not pre-trained LLMs. LLMs process raw text inputs, and transitioning to a new LLM does not require restructuring input features. This option is incorrect.

Option C: Adjust the prompt template.This is the correct approach. Different LLMs may interpret prompts differently due to variations in training data, tokenization, or model architecture. Adjusting the prompt template ensures the new LLM understands the task (e.g., document extraction) and produces the desired output format. AWS documentation emphasizes prompt engineering as a key step when adopting a new foundation model.

Option D: Fine-tune the LLM.Fine-tuning is not required for transitioning to a new pre-trained LLM unless the company needs to customize the model for a highly specific task. Since the question does not indicate a need for customization beyond document extraction (a common LLM capability), fine-tuning is unnecessary.