Standard Operating Procedure for generating AI Health and AI Monitoring Performance reports from the Affirm Power BI tenant.

1. Purpose
2. Scope
3. Definitions & Glossary
4. Prerequisites
5. Procedure — Access the Affirm Power BI Tenant
6. Procedure — Generate AI Monitoring Performance Report
7. Procedure — Generate AI Health Report
8. Affirm Support Details

—-

This Standard Operating Procedure (SOP) defines the end-to-end steps for generating, validating, and delivering Power BI reports for Affirm clients from the Affirm Power BI tenant. It covers two report templates published under the Synergist Technologies, LLC workspace:

• AI Health Report — long-range health snapshots, trend analysis, and normalized scoring for AI assets.
• AI Monitoring Performance Report — performance evaluation of AI assets across configurable evaluators and methods.

This document is intended to standardize the report generation process so that any qualified operator can produce consistent, accurate, audit-ready outputs without ad-hoc decisions.

• Applies to all Power BI report generation performed against the Affirm tenant for internal or external Affirm clients.
• Covers ad-hoc report generation triggered by client request, scheduled delivery cycles, or internal QA review.

Before starting this procedure, confirm ALL of the following are in place. If any item is missing, stop and contact the Tenant Administrator.

• Client / organization name (e.g., CDW Internal, CDW Corporation, Integration).
• Reporting period (start and end date).
• AI Asset(s) in scope (or “All”).
• Evaluator(s) and Method(s) in scope (or “All”).
• For AI Health Report only: Trending Days window and whether normalized scoring is required.

This section is common to both reports. Complete steps 6.1 through 6.4 before moving to Section 7 or Section 8.

1. Open a supported browser and navigate to the Affirm Power BI workspace URL:
   Affirm Power BI Report Link
2. Sign in with your corporate credentials if prompted. Complete MFA if required.

<note important>Important: If the link opens but reports are not visible, you are still in your default tenant. Continue to step 6.2 to switch tenants.</note>

1. Click your profile picture in the top-right corner of Power BI.
2. In the profile flyout, click Switch tenant.

1. In the Switch tenant dialog, select Synergist Technologies, LLC from the dropdown.
2. Click Switch. The Power BI Home page reloads in the Affirm tenant context.

On the Home page you should see the following two reports listed under the Recommended section and in the Recent list:

• AI Health Report - CDW
• AI Monitoring Performance Report

<note>Both reports use the same filter pattern.
AI Health Report and AI Monitoring Performance Report share a near-identical filter pane: Organization Name, AI Asset Name, Evaluator Name, Method Name, and a Date / Date Range filter. Mastering the filter pane once applies to both reports. The differences are summarized in Section 7 (performance) and Section 8 (health).</note>

Use this report to evaluate the performance of one or more AI assets across selected evaluators and methods, over a specific reporting period.

1. From the Power BI Home page (after Section 6), click AI Monitoring Performance Report under Recommended or Recent.
2. The report opens on the Cover page with the Filters pane on the right.

The Date filter controls the reporting window. Power BI offers four filter types; pick the one that best fits the request.

Use when the requestor provides explicit start and end dates.

1. In the Filters pane, expand Date.
2. Set Filter type to Advanced filtering.
3. Configure Show items when the value (top condition). Operators available: is, is not, is after, is on or after, is before, is on or before, is blank, is not blank.
4. Choose And to combine with a second condition (typically is on or before for the end date).
5. Click Apply filter.

Use when the requestor needs to cherry-pick specific calendar dates.

1. Set Filter type to Basic filtering.
2. Tick the individual dates required. Use Select all to include every date in the dataset.

Use for rolling windows that should auto-shift each time the report is regenerated.

1. Set Filter type to Relative date.
2. Choose the operator: is in the last, is in this, or is in the next.
3. Enter the numeric value (e.g., 30).
4. Choose the unit: days, weeks, months, years, calendar months, or calendar years.
5. Tick Include today if the current day must be included.
6. Click Apply filter.

Use only for intra-day reports. Resolution is hours or minutes.

1. Set Filter type to Relative time.
2. Choose the operator: is in the last, is in this, or is in the next.
3. Enter the numeric value (1 to 10000).
4. Choose the unit: hours or minutes.
5. Click Apply filter.

<note warning>Date filter caveat: Mixing filter types on the same field is not supported in a single render. If you switch from Advanced to Basic, the previous values are cleared. Confirm the date range with the requestor before switching.</note>

The Organization Name identifies the client whose data is being reported. The document uses “CDW Internal” as the example client.

1. Expand Organization Name in the Filters pane.
2. Tick the single organization in scope. Untick any others.
3. If “(Blank)” is shown, leave it unchecked unless the requestor explicitly asks for it.

<note warning>Always confirm exactly one organization is selected. Selecting multiple organizations will mix client data into the same report. This is a data-confidentiality violation. Reviewer must verify in Section 10.</note>

Select the AI Assets in scope. The report renders separate views per asset selected.

1. Expand AI Asset Name and set Filter type to Basic filtering.
2. Tick the assets in scope, or use Select all. Available assets include Azure MS Cloud Chatbot, CloudGenie - Chat, CloudGenie - KnowledgeBase, Jenkins AI Copilot, Onboarding Guide, SOW Analyzer, etc.

Choose which evaluators (metric families) should appear in the report.

1. Expand Evaluator Name and set Filter type to Basic filtering.
2. Tick the evaluators in scope, or use Select all. Typical evaluators: Groundedness, Hallucination, Prompt Relevance, Response Relevance, Readability, Latency.

Each evaluator can be computed by one or more methods. For example, Readability supports FRES and SMOG.

1. Expand Method Name and set Filter type to Basic filtering.
2. Tick the methods in scope, or use Select all. Available methods include Fluency, FRES, LLMaJ, Parallel, Sequential, SMOG.

<note>Method–Evaluator pairing: Some methods only apply to specific evaluators (e.g., FRES and SMOG only apply to Readability). Selecting an irrelevant method has no effect on the report but adds clutter to the filter audit trail. Pick only the methods aligned to the chosen evaluators.</note>

1. Wait for all visuals on every report page to finish rendering. The status bar should display Updated.
2. Navigate through the pages in the left-hand Pages pane: Cover, 1- Executive, 2- Evaluator Snapshots, 3- Details (1) … 6- Details (4), Legal Disclaimer, Glossary 1–3.
3. Visually confirm: (a) date range in the cover matches the requested window, (b) organization name is correct, © selected evaluators and methods appear on the details pages.

1. Click the download / export icon in the top toolbar (highlighted in red in Figure 13 below).
2. Choose the requested format: PDF (recommended for client delivery), PowerPoint, or Excel data.
3. Wait for the export job to finish. Power BI shows a notification with a download link.
4. Save the file using the naming convention defined in Section 9.

The AI Health Report shares the Organization / AI Asset / Evaluator / Date filter pattern with the Performance Report, plus two additional filters specific to health monitoring: Trending Days and is_normalized.

1. From the Power BI Home page (after Section 6), click AI Health Report - CDW.
2. The report opens on the Cover page with the Filters pane on the right.

Apply the following filters in order, using the same procedures as for the AI Monitoring Performance Report:

• Organization (or Organization(s)) — see Section 6.3.
• AI Asset (or AI Asset(s)) — see Section 6.4.
• Evaluators — see Section 6.5.
• Report Date Range — see Section 6.2 (use Advanced or Relative date as appropriate).

Trending Days defines the lookback window used to compute the Trend % column shown in the AI Health Snapshots table. The default value is 0.

1. Expand Trending Days in the Filters pane.
2. Set Filter type to Advanced filtering.
3. Set Show items when the value to is, then enter the desired number of days (e.g., 7, 14, 30).
4. Leave the And/Or row empty unless a second condition is required.
5. Click Apply filter.

<note>Choosing Trending Days: Pick a window proportional to the report's date range and the volatility of the metric. 7 days suits high-volume daily monitoring; 30 days suits low-volume monthly review. Avoid windows longer than the date range itself — the trend will be meaningless.</note>

is_normalized determines whether the Actual Normalized Score column on the AI Health Snapshots page uses normalized values.

1. Expand is_normalized in the Filters pane.
2. Tick True to show normalized scores, False to show raw scores, or both for comparison.

<note>When to use normalized scores: Use normalized scores (True) when comparing different evaluators or metrics that have different native scales (e.g., comparing Latency in milliseconds against Readability scores). Use raw scores (False) when the requestor needs the original measurement values.</note>

Navigate to the 2- AI Health Snapshots page. The table shows one row per Evaluator × Method × Data Type combination.

Columns to verify:

• Evaluator and Method — match the filters applied.
• Group — semantic grouping (e.g., Semantic Quality, Relevance & Coverage, Linguistic Quality, Efficiency, Harmful Content).
• Data Type — Manual or Generated.
• Actual Normalized Score — the score for the period (highlighted in Figure 16).
• Baseline Start Date / Baseline End Date — the reference window for trend computation.
• Trend % (N Day(s)) — directional change vs. baseline. Green up-arrow indicates improvement; red down-arrow indicates degradation.

Follow the same export procedure as Section 6.8. PDF is the default delivery format unless the requestor specifies otherwise.

Following are the details of Affirm support, in case of any Power BI Access Issue or Technical issue. Teams Channel: CDW AFFIRM Support Chat