This is an old revision of the document!
Phase 1 – Application Setup and Deployment
This phase established the foundation for the AI Platform by deploying three independent web applications into Azure App Service using GitHub Actions. Each app represents a distinct layer of the AI orchestration pipeline (Agent Gateway → MCP Server → RAG API).
Objectives
Application Architecture Overview
| Component | Purpose | Typical Endpoint | Description |
|---|---|---|---|
| Agent Gateway | Entry point / orchestrator | /chat, /api/messages, /healthz | Receives user messages and orchestrates downstream services (MCP + RAG). |
| MCP Server | Service layer exposing tools | /invoke, /healthz | Exposes tool-like APIs (vision, safety, sentiment, OpenAI chat, etc.). |
| RAG Web App | Retrieval-Augmented Generation API | /api/answer, /healthz | Handles document search, retrieval, and LLM synthesis. |
Each app runs independently and communicates securely over HTTPS. The architecture is designed for horizontal scalability and composability.
App Service Creation (Azure Portal)
For each app (RAG, MCP, Agent-Gateway):
Click Create → Review → Deploy.
Result — Three independent web apps ready for deployment.
Repository Structure
/apps
├─ /agent-gateway
│ ├─ src/
│ ├─ package.json
│ ├─ index.js
│ └─ orchestrator.js
├─ /mcp-server
│ ├─ src/
│ ├─ package.json
│ └─ http.js
└─ /rag-app
├─ src/
├─ package.json
└─ rag.py / server.js
.github/
└─ workflows/
├─ deploy-mcp.yml
├─ deploy-agent-gateway.yml
└─ deploy-rag.yml
Each app contains its own build and runtime configuration.
The .github/workflows folder defines independent CI/CD pipelines.
CI/CD Setup Using GitHub Actions
Each web app uses a lightweight ZipDeploy workflow for Azure App Service:
name: Deploy · MCP Server (ZipDeploy)
on:
push:
paths:
- 'apps/mcp-server/**'
workflow_dispatch:
jobs:
zip-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Create zip
run: |
cd apps/mcp-server
zip -r ../../mcp-server.zip . -x “.git/*” “.github/*”
- name: Deploy to Azure Web App
uses: azure/webapps-deploy@v3
with:
app-name: mcp-server-app
publish-profile: $secrets.azure_publish_profile_mcp
package: mcp-server.zip
Each app has its own publish-profile secret (AZURE_PUBLISH_PROFILE_MCP, …_RAG, …_GATEWAY) stored in GitHub → Settings → Secrets and Variables → Actions.
Result — pushing to the respective folder triggers automatic deployment.
Environment Configuration
Before first deployment, each web app’s configuration was set under:
Azure Portal → App Service → Configuration → Application Settings
Example (Agent-Gateway):
PORT=8080
BOT_APP_ID=<Bot registration ID>
BOT_APP_PASSWORD=<Bot registration secret>
KEY_VAULT_URL=https://<your-keyvault>.vault.azure.net
AOAI_ENDPOINT=https://<your-aoai-resource>.openai.azure.com
AOAI_API_VERSION=2024-12-01-preview
Result — All apps share a consistent configuration model with environment variables mirrored locally via .env.example.
Local Testing
# From app folder
npm install
npm run build
npm start
# Health check
curl http://localhost:8080/healthz
Ensured each service worked independently before deployment.
Post-Deployment Verification
After each successful GitHub Action run:
curl -I https://mcp-server-app.azurewebsites.net/healthz
curl -I https://agent-gateway-app.azurewebsites.net/healthz
curl -I https://rag-app.azurewebsites.net/healthz
Expected response:
HTTP/1.1 200 OK
Content-Type: application/json
Result — All three web apps responding successfully.
Phase 2 – Backend Configuration & Environment Setup
In this phase you’ll wire up configuration for all three apps (Agent Gateway, MCP Server, RAG Web App), move secrets to Key Vault, and establish clean service-to-service contracts. Everything below uses generic names so you can reuse it anywhere.
1) Objectives
2) Prerequisites
3) Environment Variable Model (What each app needs)
Agent Gateway (entry point / orchestrator)
MCP Server (tool backend)
RAG Web App
Note: If your code reads secrets at runtime from Key Vault (SDK), you need KEY_VAULT_URL and Managed Identity.
If you prefer App Settings Key Vault references, you don’t need runtime SDK calls (see Section 5).
4) Configure App Settings (Azure Portal)
For each app:
Use the exact URLs your services expose. Do not include /chat or /healthz in base URLs; those are endpoint paths used by your frontend or health checks.
5) Move Secrets to Key Vault
A) Enable Managed Identity on each App
B) Grant the App access to Key Vault
C) Create secrets in Key Vault
D) Reference secrets from App Settings (no code change)
In each app → Configuration → Application settings, set values like:
AOAI_API_KEY = @Microsoft.KeyVault(SecretUri=https://ai-platform-kv.vault.azure.net/secrets/aoai-api-key/<version>)
VISION_KEY = @Microsoft.KeyVault(SecretUri=https://ai-platform-kv.vault.azure.net/secrets/vision-key/<version>)
TA_KEY = @Microsoft.KeyVault(SecretUri=https://ai-platform-kv.vault.azure.net/secrets/text-analytics-key/<version>)
SAFETY_KEY = @Microsoft.KeyVault(SecretUri=https://ai-platform-kv.vault.azure.net/secrets/content-safety-key/<version>)
SEARCH_API_KEY = @Microsoft.KeyVault(SecretUri=https://ai-platform-kv.vault.azure.net/secrets/ai-search-api-key/<version>)
Tip: Prefer Key Vault references when possible (fewer moving parts). Use runtime SDK only when you must fetch secrets dynamically.
6) CORS (Agent Gateway)
7) Health Endpoints & Readiness
Ensure each app exposes a JSON health endpoint:
In the Portal: App → Monitoring → Health check → set the path to /healthz and Enable.
This lets App Service remove unhealthy instances from rotation.
8) Wire Up Service Contracts
Agent Gateway → MCP
Agent Gateway → RAG
Keep these contracts versioned in code (v1 query shape) so you can evolve independently.
9) Smoke Tests (CLI)
Replace hostnames as appropriate.
# Health
curl -s -w “\n%{http_code}\n” https://agent-gateway-app.azurewebsites.net/healthz
curl -s -w “\n%{http_code}\n” https://mcp-server-app.azurewebsites.net/healthz
curl -s -w “\n%{http_code}\n” https://rag-app.azurewebsites.net/healthz
# Agent Gateway chat (no attachments)
curl -s -X POST https://agent-gateway-app.azurewebsites.net/chat \
-H “content-type: application/json” \
-d '{“message”:“hello from curl”,“locale”:“en”}' | jq .
# Direct RAG query (POST shape)
curl -s -X POST https://rag-app.azurewebsites.net/api/answer \
-H “content-type: application/json” \
-d '{“query”:“what is our refund policy?”,“topK”:3,“temperature”:0.2}' | jq .
Expected:
Phase 3 – Security Hardening (Network & Identity)
This phase locks down the perimeter, enforces private-only service-to-service traffic, and moves all sensitive access to identity-based auth. The guidance below uses generic names so you can reuse it anywhere.
1) Objectives
2) Prerequisites
Keep VNet Integration subnets and Private Endpoint subnets separate.
3) Private Endpoints (Inbound Isolation)
Goal: mcp-server-app and rag-app are reachable only via private IPs from your VNet.
Portal steps (per private app):
Validate:
4) Private DNS Zone (Name Resolution)
Goal: *.azurewebsites.net for your apps resolves to private IP inside VNet.
Portal steps:
Validate from Agent Gateway console (SSH / Kudu):
nslookup mcp-server-app.azurewebsites.net
curl -I https://mcp-server-app.azurewebsites.net/healthz
nslookup rag-app.azurewebsites.net
curl -I https://rag-app.azurewebsites.net/healthz
Expect resolution to 10.x addresses and HTTP 200/OK (or your expected status).
5) Access Restrictions (Deny Public, Allow VNet)
Goal: Only allowed subnets (e.g., Agent Gateway’s VNet Integration subnet) can call private apps.
Portal steps (for mcp-server-app and rag-app):
Validate:
6) VNet Integration (Outbound Control)
Goal: Ensure outbound from apps traverses your VNet → you can apply NSGs/UDRs, and reach private services.
Portal steps (each app):
NSG recommendations (attach to apps-integration-snet):
If using Private Endpoints for Key Vault/Cognitive/Search/Storage, allow to their private IPs instead.
7) Agent Gateway Ingress Strategy
Goal: Keep agent-gateway-app public, but limit who can hit it.
Options (choose one):
Portal steps:
8) TLS, SCM & Legacy Protocols
9) Managed Identity + Key Vault (Identity, Not Keys)
Goal: Apps authenticate to resources using system-assigned managed identities and fetch secrets via Key Vault.
Portal steps (each app):
If your code uses the SDK to fetch secrets dynamically, also set KEY_VAULT_URL and ensure outbound to KV is allowed.
10) CORS (Browser Clients)
Goal: Limit cross-origin browser calls to approved frontends.
Portal steps (Agent Gateway):
11) Validation Checklist
From Agent Gateway container (SSH):
# Private name resolution & reachability
nslookup mcp-server-app.azurewebsites.net
nslookup rag-app.azurewebsites.net
curl -s -w “\n%{http_code}\n” https://mcp-server-app.azurewebsites.net/healthz
curl -s -w “\n%{http_code}\n” https://rag-app.azurewebsites.net/healthz
From your laptop (public):
# Should be allowed only to Agent Gateway
curl -I https://agent-gateway-app.azurewebsites.net/healthz
# Direct hits to private apps should fail
curl -I https://mcp-server-app.azurewebsites.net/healthz # expect 403/timeout
curl -I https://rag-app.azurewebsites.net/healthz # expect 403/timeout
Key Vault access (app logs):
Phase 4 – Monitoring, Logging & Alerts
This phase focuses on making the AI Platform production-ready through observability. You’ll connect all three web apps (Agent Gateway, MCP Server, RAG App) to Application Insights, enable structured logging, and add alerting and dashboards for proactive monitoring.
1) Objectives
2) Application Insights Integration
A) Enable per-app Insights
Repeat for:
B) Verify connection
After enabling:
curl -I https://agent-gateway-app.azurewebsites.net/healthz
→ In the Insights blade, you should soon see requests appearing under “Requests”.
3) Enable OpenTelemetry (if using custom logging)
For more control and cross-service correlation, configure Azure Monitor OpenTelemetry Exporter.
Example (Node.js):
import { AzureMonitorTraceExporter } from “@azure/monitor-opentelemetry-exporter”;
import { NodeSDK } from “@opentelemetry/sdk-node”;
import { getNodeAutoInstrumentations } from “@opentelemetry/auto-instrumentations-node”;
const exporter = new AzureMonitorTraceExporter({
connectionString: process.env.APPLICATIONINSIGHTS_CONNECTION_STRING,
});
const sdk = new NodeSDK({
traceExporter: exporter,
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
Result: Distributed traces now flow between Agent Gateway → MCP → RAG automatically.
4) Logging Strategy
A) Use structured logging
Replace raw console.log() with structured JSON:
console.info(JSON.stringify({
level: “info”,
service: “agent-gateway”,
message: “Chat request processed”,
latency_ms: 120,
routed: “rag”,
}));
B) Application Insights automatic collection
C) Enable diagnostic logs to storage
Portal → App Service → Monitoring → Diagnostic settings → + Add diagnostic setting
This ensures long-term retention beyond Application Insights default (90 days).
5) Custom Metrics
Emit custom metrics (latency, errors, token usage) to Insights:
import appInsights from “applicationinsights”;
const client = appInsights.defaultClient;
client.trackMetric({ name: “chatLatencyMs”, value: latency });
client.trackMetric({ name: “ragHitCount”, value: hits.length });
client.trackException({ exception: err });
Or, if using OpenTelemetry:
const meter = sdk.getMeterProvider().getMeter(“agent-gateway”);
const counter = meter.createCounter(“rag_requests”);
counter.add(1, { route: “chat”, success: true });
6) Alerts
Portal Steps:
You’ll now receive alerts for downtime or performance degradation.
7) Dashboards
Portal → Dashboard → + New → Add Tiles → Application Insights:
Recommended widgets:
Save as AI Platform – Observability Dashboard.
8) Availability Tests
Detects public outages even if the Gateway is otherwise healthy behind a load balancer.
9) Defender for App Service (Optional)
Enable Microsoft Defender for App Service:
Integrates with Security Center for ongoing posture management.
Phase 5 – Frontend Creation & Integration
This phase delivers a lightweight, test-friendly frontend that can exercise the full pipeline (Agent Gateway ↔ MCP ↔ RAG), provides quick health visibility, and is safe to ship publicly. It assumes Phases 1–4 are complete.
1) Objectives
2) Frontend Repo Layout
/frontend
├─ src/
│ ├─ App.tsx # UI (health cards + chat)
│ ├─ main.tsx # React root
│ └─ index.css
├─ index.html
├─ package.json
├─ tsconfig.json
├─ vite.config.ts
└─ .env.local.example # VITE_* endpoints (no secrets)
If you prefer plain JS, the same structure works—only types differ.
3) Environment Variables (Client-side)
Create /frontend/.env.local (not committed) using public base URLs only—never secrets.
# Points to public entry point (Front Door or direct App Service)
VITE_GATEWAY_BASE=https://agent-gateway-app.azurewebsites.net
# Optional (diagnostics-only; keep disabled in prod):
# VITE_MCP_BASE=https://mcp-server-app.azurewebsites.net
# VITE_RAG_BASE=https://rag-app.azurewebsites.net
Why only the gateway? MCP and RAG are private in Phase 3. Keep direct calls disabled outside of troubleshooting.
4) CORS & Networking
CORS_ALLOWED_ORIGINS=https://your-frontend.domain
5) Minimal UI Capabilities
A) Health Cards
B) Unified Chat
C) Error Handling & Timeouts
Fetch wrapper:
async function fetchJson(url: string, init?: RequestInit, timeoutMs = 30000) {
const ctl = new AbortController();
const t = setTimeout1)); }
}
throw err;
}
D) No Secrets in Browser
6) Production Build & Local Dev
# local dev
cd frontend
npm install
npm run dev # default: http://localhost:5173
# production build
npm run build # outputs dist/
npm run preview
Smoke tests:
7) Deployment Options
Option A — Azure Static Web Apps (Recommended)
Portal:
GitHub Action (generated) will resemble:
name: Build & Deploy · Frontend
on:
push:
paths: ['frontend/**']
workflow_dispatch:
jobs:
build_and_deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20 }
- run: |
cd frontend
npm ci
npm run build
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: frontend/dist
- name: Deploy to Static Web Apps
uses: Azure/static-web-apps-deploy@v1
with:
azure_static_web_apps_api_token: $secrets.azure_static_web_apps_api_token
action: upload
app_location: frontend
output_location: dist
Add your SWA domain (and custom domain) to CORS_ALLOWED_ORIGINS.
Option B — Static Website on Azure Storage
Portal:
8) Frontend Telemetry (Optional)
If you want page load/error metrics:
Example (App Insights JS):
import { ApplicationInsights } from '@microsoft/applicationinsights-web';
const ai = new ApplicationInsights({ config: {
connectionString: import.meta.env.VITE_APPINSIGHTS_CONNECTION_STRING,
enableAutoRouteTracking: true,
disableExceptionTracking: false,
samplingPercentage: 50,
}});
ai.loadAppInsights();
// Usage
ai.trackEvent({ name: 'chat_submit' });
ai.trackMetric({ name: 'ui_latency_ms', average: 120 });
Add VITE_APPINSIGHTS_CONNECTION_STRING only if you want client telemetry; otherwise omit.
