AI-300 Questions and Answers

GitHub Actions is the appropriate automation tool because the deployment process originates from pull requests and requires CI/CD automation, traceability, and an approval gate. Azure Machine Learning integrates directly with GitHub Actions for automated machine learning workflows, including infrastructure and asset deployment. Workflows can be triggered by repository events such as pull requests, merges, or pushes, providing an auditable record of who initiated the change, which commit was deployed, and whether the workflow succeeded.

Critically, GitHub environments can enforce required reviewers before a deployment job executes. Microsoft documents that when an environment requires approval, a job cannot access the environment ' s protected secrets until one of the required reviewers approves the deployment. This directly satisfies the requirement that deployment must require approval before running.

Azure Machine Learning pipelines orchestrate ML workflow steps such as preprocessing, training, evaluation, and registration, but they are not primarily the repository-level pull-request approval mechanism. MLflow provides experiment tracking and model lifecycle functionality, not CI orchestration. Azure Monitor provides telemetry and alerting rather than source-driven deployment automation.

Therefore, GitHub Actions provides the required combination of pull-request integration, automation, auditability, and protected deployment approval.

Study Guide Reference: Design and implement an MLOps infrastructure — CI/CD, GitHub Actions, pull-request automation, deployment approvals, protected environments, and Azure Machine Learning integration.

===============

Git commits on the main branch provide the immutable, ordered history that auditors need: every change is timestamped, attributed to a specific developer, and reversible via git revert — this is the audit trail and rollback mechanism required by the first requirement. Git branches allow developers to create and test multiple prompt variants in complete isolation from the production prompt on the main branch, addressing the second requirement. A developer on an experiment branch can run full evaluations without touching the production prompt. When a variant is approved, it is merged via pull request. You cannot use branches alone for audit history because branches can be deleted, and you cannot use main-branch commits alone for parallel variant comparison without disrupting the history. Git ' s branch-and-merge model provides both capabilities simultaneously.

Microsoft Learn Reference Topic: Version control for AI prompts – Git branch strategies for prompt management in Microsoft Foundry

A managed online endpoint can host multiple named deployments simultaneously, and you control what percentage of incoming traffic each deployment receives. With traffic splitting, you route a small percentage of live traffic to the new deployment while the majority continues to the proven existing model. You then monitor error rates, latency, and output quality for both deployments in real time under genuine production load. If the new model underperforms, you instantly route traffic back — no downtime, no user disruption. If it outperforms, you gradually increase its traffic share to 100%. Updating the registry version (option B) does not affect running deployments. A staging endpoint (option C) does not validate under real production load. An evaluation script (option D) is a pre-deployment step. Traffic splitting is the blue/green and canary deployment pattern that Microsoft recommends for safe production rollouts.

Microsoft Learn Reference Topic: Perform safe rollout of new model deployments using traffic splitting – Azure Machine Learning

Interactive experimentation: Azure Machine Learning compute instance

Scalable training jobs: Azure Machine Learning compute cluster

For interactive experimentation , use an Azure Machine Learning compute instance . Microsoft defines a compute instance as a fully managed, cloud-based development workstation optimized for machine learning development. It integrates directly with Jupyter, JupyterLab, and other development tools in Azure Machine Learning studio, making it the appropriate target for notebook-driven experimentation, iterative coding, debugging, and development. A compute instance is a single-node environment and can be stopped or configured with idle shutdown to control costs.

For scalable training jobs , use an Azure Machine Learning compute cluster . Compute clusters are managed training targets that can scale from a minimum to a configured maximum number of nodes based on job demand. Microsoft specifically recommends them for larger datasets, distributed training, and workloads requiring elastic compute capacity. Setting the minimum node count to 0 allows the cluster to deallocate all worker nodes when no jobs are running, which is an important cost-optimization mechanism.

Azure Batch, Databricks, AKS, and standalone Azure VMs can support specialized workloads, but they do not match the native Azure Machine Learning development-versus-elastic-training pattern as directly as compute instance and compute cluster.

Study Guide Reference: Design and implement an MLOps infrastructure — Azure Machine Learning compute targets, compute instances, compute clusters, autoscaling, distributed training, and cost optimization.

The first action should be to evaluate the model output . Fabrikam already has evaluation data containing manually defined expected responses, and the technical requirements explicitly state that the organization must use its existing evaluation datasets based on real data with input-output pairs . Microsoft Foundry supports evaluating a model against an existing curated dataset to establish measurable baseline performance before optimization. Evaluation results can identify weaknesses in accuracy, relevance, groundedness, safety, or other quality dimensions and provide objective evidence for subsequent changes.

This sequencing is critical because Fabrikam also requires that advanced fine-tuning be applied only when prompt engineering is insufficient . Therefore, immediately fine-tuning the model would bypass the required baseline evaluation and optimization process. Fine-tuning should be driven by demonstrated performance gaps, not applied automatically.

Deploying directly to production is also inappropriate because leadership requires low operational risk and gradual rollout. Production traffic should not be used as the first evaluation mechanism when curated evaluation data already exists. Synthetic interaction data is useful when evaluation coverage is insufficient, but Microsoft specifically positions synthetic generation as an option when an adequate test dataset is unavailable. Fabrikam already has suitable evaluation data.

Study Guide Reference: Optimize generative AI systems and model performance — evaluate model performance, benchmark against curated datasets, iterate on prompts, and apply advanced fine-tuning when justified.

Load the model when the service starts: init()

Use the model to score new data: run()

Azure Machine Learning scoring scripts for online endpoints use two required entry-point functions: init() and run() . Microsoft explicitly states that the scoring script specified for an online deployment must contain both functions.

The init() function is invoked when the inference container is initialized or started, typically immediately after a deployment is created or updated. It is intended for one-time initialization tasks such as locating the registered model through AZUREML_MODEL_DIR, deserializing the model, and storing it in memory. Loading the model once during initialization avoids repeatedly loading it for every inference request, which reduces latency and processing overhead.

The run() function is called each time the endpoint receives an inference request. It accepts the incoming request data, transforms or parses the input as required, invokes the loaded model ' s prediction logic, and returns the scoring result. Microsoft describes run() as the function that performs the actual scoring or prediction for each endpoint invocation.

main(), score(), and predict() may exist inside application code or model libraries, but they are not the required Azure Machine Learning scoring-script entry points.

Microsoft ' s RAG optimization guidance identifies two high-impact, low-cost improvements for retrieval quality that do not require changing the embedding model. First, tuning chunk size and overlap (option A): chunk size determines how much context each retrieved piece contains — too large and irrelevant content dilutes the signal; too small and answers may be split across chunks. Adjusting these parameters requires only re-indexing the documents with zero additional compute cost. Second, implementing a re-ranker (option B): a re-ranker is a cross-encoder model that takes the top-N retrieved chunks and re-scores them based on their specific relevance to the query, significantly improving precision by filtering out contextually irrelevant chunks. Re-rankers add modest compute cost but are far cheaper than changing the embedding model, which would require re-embedding the entire knowledge base. Increasing token limits (option C) and optimizing embedding vector length (option D) do not address retrieval accuracy without an embedding model change.

Microsoft Learn Reference Topic: Optimize RAG pipelines – Chunk size tuning and re-ranking in Azure AI Search and Azure Machine Learning

Two distinct RAG problems require two distinct solutions. When retrieved results frequently include duplicated content from the same document, the cause is overlapping chunks receiving similarly high similarity scores. The solution is Maximum Marginal Relevance reranking, which diversifies the result set by penalizing results that are semantically similar to already-selected results, reducing redundancy. When retrieved chunks sometimes span unrelated policy sections, the cause is fixed-size character chunking that splits content without regard for semantic boundaries. The solution is semantic chunking — splitting on natural sentence or section boundaries — ensuring each chunk is semantically coherent and does not straddle unrelated content. Both problems must be addressed together: reranking alone solves duplication but not relevance; semantic chunking alone solves relevance but not duplication.

Microsoft Learn Reference Topic: Optimize RAG retrieval in Azure AI Search – Chunking strategies and MMR reranking

Azure Machine Learning ' s data drift monitor solves the first requirement: it continuously compares the statistical distribution of live inference input features against the baseline training data distribution, flagging when significant drift occurs. For the second requirement of triggering appropriate actions when thresholds are exceeded, Azure Monitor alert rules are configured on the drift metrics. When the drift coefficient exceeds a defined threshold, Azure Monitor fires an alert that can invoke Logic Apps, send emails, trigger an Azure ML retraining pipeline, or post to Teams. This two-layer approach — ML-specific drift detection backed by Azure Monitor alerting — is the Microsoft-recommended pattern for production model monitoring. The data drift monitor handles detection, while Azure Monitor handles the operational response, keeping the two concerns cleanly separated.

Microsoft Learn Reference Topic: Monitor model data drift – Azure Machine Learning model monitoring and Azure Monitor integration

Data Zone Standard deployment routes requests to multiple datacenters within a defined geographic zone such as the US data zone, providing lower latency by using zone-level load balancing while maintaining data within a broad compliance boundary. This makes it suitable for enterprise internal applications where multiple business units need low-latency access without strict single-region data residency requirements. Developer deployment (option A) is a low-quota, unguaranteed tier designed for prototype testing, not production workloads serving multiple business units. Data Zone Batch (option B) is designed for high-volume asynchronous batch processing, not interactive low-latency real-time requests. Global Batch (option D) is optimized for scheduled bulk inference across all global regions but adds latency and is inappropriate for synchronous interactive use cases.

Microsoft Learn Reference Topic: Foundation model deployment types in Microsoft Foundry – Data Zone Standard for low-latency internal use

MLflow is the industry-standard open-source platform for experiment tracking, and Azure Machine Learning has first-class native integration with it. When you use MLflow within an Azure ML job, parameters, metrics, and artifacts are automatically logged to the run history of the AML workspace, making every run reproducible and comparable. Option A (AML job output logs) only captures console output and lacks structured parameter and metric logging. Option C (Application Insights logs) is designed for application-level telemetry, not ML experiment metadata. Option D (Azure Monitor alerts) is a reactive notification tool, not a tracking system. MLflow ' s autologging capability means that for common frameworks such as scikit-learn, XGBoost, and PyTorch, parameters and metrics are captured without a single line of custom code, directly answering Fabrikam ' s requirement for consistent experiment tracking.

Microsoft Learn Reference Topic: Track ML experiments with MLflow – Azure Machine Learning MLflow integration

In Azure Machine Learning, a workspace is the top-level resource that acts as a centralized hub for every ML activity. It stores experiments, pipelines, datasets, models, environments, and compute targets all in one place. Before you can register assets or deploy endpoints, the workspace itself must exist and be shared across the team. Fabrikam ' s core challenge is inconsistent experiment tracking, ad-hoc versioning, and no standardized deployment path — all symptoms of the absence of a single, governed workspace. Option A (registering assets) is a subsequent action only possible after the workspace exists. Option C (online endpoint) is a deployment concern, not an asset-management foundation. Option D (Foundry project) is appropriate for generative AI workloads, not for the traditional ML pipeline standardization described here. The workspace is the prerequisite that makes every other governance action possible.

Microsoft Learn Reference Topic: Azure Machine Learning workspaces – Microsoft Learn: Manage Azure Machine Learning workspaces

Domain specialization: Supervised fine-tuning

Poor response accuracy of a RAG-based solution: Apply prompt engineering

For domain specialization , Supervised Fine-Tuning (SFT) is the correct first strategy. Microsoft describes SFT as the foundational fine-tuning technique for training a model from labeled input-output pairs , and specifically identifies domain specialization as one of its principal use cases. Microsoft also recommends starting with SFT for most customization projects because it supports task specialization, instruction following, style, and domain-specific behavior. Fabrikam already possesses evaluation data containing input-output pairs, which aligns directly with the SFT data model.

For poor RAG response accuracy , the first action is prompt engineering . The case explicitly requires advanced fine-tuning only when prompt engineering is insufficient. In a RAG system, prompt engineering determines how the model interprets retrieved context, constrains answers to grounding information, handles missing evidence, and formats responses. Microsoft notes that inadequate RAG prompting can produce false or incomplete answers even when retrieval returns appropriate content.

DPO is primarily appropriate for alignment using preferred versus non-preferred responses, while RFT targets complex reward-based reasoning optimization. Neither is the initial technique for domain specialization in this requirement.

Study Guide Reference: Optimize generative AI systems and model performance — prompt engineering, RAG optimization, supervised fine-tuning, preference optimization, and model customization strategy.

Managed online endpoints with traffic splitting are the best fit because Azure Machine Learning supports multiple deployments behind a single online endpoint and allows production traffic to be distributed between them. This enables a blue-green or progressive rollout strategy : deploy the new model version alongside the current version, validate it independently, send a small percentage of production traffic to it, monitor behavior, and then gradually increase traffic. If problems occur, traffic can be redirected immediately to the previous deployment, providing a practical rollback mechanism.

This approach also aligns with Fabrikam’s requirement to favor managed services and automation because the team has limited Azure DevOps experience. Managed online endpoints handle much of the serving, scaling, security, and monitoring infrastructure automatically and support both traffic splitting and mirrored traffic for safer validation.

Option B can also implement blue-green deployment, but AKS introduces substantially more infrastructure and operational management than Fabrikam requires. Option A would require custom deployment, routing, scaling, and rollback logic. Option D is intended for asynchronous batch inference rather than staged real-time production serving.

Study Guide Reference: Implement machine learning model lifecycle and operations — managed online endpoints, multiple deployments, traffic splitting, blue-green deployment, staged rollout, monitoring, and rollback.