AWS Certified AI Practitioner AIF-C01 Updated Exam

Hallucinations in large language models (LLMs) occur when the model generates outputs that are factually incorrect, irrelevant, or not grounded in the input data. To mitigate hallucinations, adjusting the model's inference parameters, particularly the temperature, is a well-documented approach in AWS AI Practitioner resources. The temperature parameter controls the randomness of the model's output. A lower temperature makes the model more deterministic, reducing the likelihood of generating creative but incorrect responses, which are often the cause of hallucinations.

Exact Extract from AWS AI Documents:

From the AWS documentation on Amazon Bedrock and LLMs:

"The temperature parameter controls the randomness of the generated text. Higher values (e.g., 0.8 or above) increase creativity but may lead to less coherent or factually incorrect outputs, while lower values (e.g., 0.2 or 0.3) make the output more focused and deterministic, reducing the likelihood of hallucinations."

(Source: AWS Bedrock User Guide, Inference Parameters for Text Generation)

Detailed Explanation:

Option A: Set up Agents for Amazon Bedrock to supervise the model training.Agents for Amazon Bedrock are used to automate tasks and integrate LLMs with external tools, not to supervise model training or directly address hallucinations. This option is incorrect as it does not align with the purpose of Agents in Bedrock.

Option B: Use data pre-processing and remove any data that causes hallucinations.While data pre-processing can improve model performance, identifying and removing specific data that causes hallucinations is impractical because hallucinations are often a result of the model's generative process rather than specific problematic data points. This approach is not directly supported by AWS documentation for addressing hallucinations.

Option C: Decrease the temperature inference parameter for the model.This is the correct approach. Lowering the temperature reduces the randomness in the model's output, making it more likely to stick to factual and contextually relevant responses. AWS documentation explicitly mentions adjusting inference parameters like temperature to control output quality and mitigate issues like hallucinations.

Option D: Use a foundation model (FM) that is trained to not hallucinate.No foundation model is explicitly trained to "not hallucinate," as hallucinations are an inherent challenge in LLMs. While some models may be fine-tuned for specific tasks to reduce hallucinations, this is not a standard feature of foundation models available on Amazon Bedrock.

AWS CloudTrail: Logs all API calls to Amazon Bedrock.

Amazon S3 Intelligent-Tiering: Optimizes storage costs for long-term retention with automatic tiering.

According to Amazon Bedrock Logging Documentation:

“CloudTrail records API activity and events, and logs can be stored in S3. For cost optimization, use S3 Intelligent-Tiering to retain logs long-term.”

Comprehensive and Detailed Explanation From Exact Extract:

The best practice for mitigating bias in AI/ML models, according to AWS and responsible AI frameworks, is to ensure that the training data is representative and diverse. If a model demonstrates bias (such as favoring a particular demographic), the recommended, cost-effective approach is to collect additional data from underrepresented groups and retrain (fine-tune) the model with the improved dataset.

A. Include more diverse training data. Fine-tune the model again by using the new data:

“The most effective method to reduce model bias is to curate and include diverse, representative training data, then retrain or fine-tune the model.”

(Reference: AWS Responsible AI, SageMaker Clarify Bias Mitigation)

B (RAG) is unrelated to model fairness or bias mitigation; it’s for grounding LLMs with external knowledge.

C (AWS Trusted Advisor) is for AWS resource optimization/security—not for ML model bias detection or mitigation.

D (Pre-train a new LLM) would be extremely costly and is unnecessary; fine-tuning with better data is much more efficient.

Creating a prompt template that teaches the LLM to detect attack patterns is the most effective way to reduce the risk of the model being manipulated through prompt engineering.

Prompt Templates for Security:

A well-designed prompt template can guide the LLM to recognize and respond appropriately to potential manipulation attempts.

This strategy helps prevent the model from performing undesirable actions or exposing sensitive information by embedding security awareness directly into the prompts.

Why Option A is Correct:

Teaches Model Security Awareness: Equips the LLM to handle potentially harmful inputs by recognizing suspicious patterns.

Reduces Manipulation Risk: Helps mitigate risks associated with prompt engineering attacks by proactively preparing the LLM.

Why Other Options are Incorrect:

B. Increase the temperature parameter: This increases randomness in responses, potentially making the LLM more unpredictable and less secure.

C. Avoid LLMs not listed in SageMaker: Does not directly address the risk of prompt manipulation.

D. Decrease the number of input tokens: Does not mitigate risks related to prompt manipulation.