Test Suite Run Workflow
Module: Intent-Based Diagnostics (IBD)
Feature: Test Suite Evaluations
Platform Version: AGILITY v4.37.0
Table of Contents
Overview
This guide walks you through the complete workflow for running a network test suite on the AGILITY Platform using the Intent-Based Diagnostics (IBD) module. By the end of this guide you will be able to:
Review existing test suite runs and their statuses
Submit a new test suite run using a PCAP file
View and download detailed IBD Evaluation Reports
Drill into scenario-level results
Required Inputs
Before starting the workflow, prepare the following inputs:
Input | Description | Example / Default | Required? |
|---|---|---|---|
PCAP File | A packet capture file (.pcap or .pcapng) containing the network traffic to be analyzed |
| Yes |
Test Suite Name | The name of the test suite to execute. Defines which scenarios and validation rules are applied |
| Yes |
Tip: The default test suite "VoNR Test Suite v1" is pre-configured with 13 scenarios across 6 domains: SIP Signaling, ENUM/DNS Resolution, Location Resolution [Cx], User Data Resolution [Sh], PCF Interactions [Rx], and SIP Headers Generated by the ASBC from [Rx].
Part 1: Review Existing Test Suite Runs
Step 1 — Open the AGILITY Platform
Navigate to the platform in your browser:
https://agility-stg.b-yond.com/cv/Log in with your credentials if prompted.
Step 2 — Expand the Sidebar
In the left navigation panel, click on Intent-Based Diagnostics to expand it. Sub-items will appear: Evaluations and Scenarios Catalog.
Left sidebar with Intent-Based Diagnostics expanded.
Step 3 — Click on Evaluations
Select Evaluations from the sidebar. This opens the Evaluations page, which defaults to the My Evaluations view.
Step 4 — Open the Test Suite Runs Page
In the upper-right area of the Evaluations page, click the TEST SUITES button, or navigate directly to:
https://agility-stg.b-yond.com/cv/evaluations/test-suites/The Test Suite Runs table showing all runs with status, progress, creation date, executed-by, and action buttons.
Step 5 — Understand the Table Columns
Column | Description |
|---|---|
TEST SUITE | Name of the test suite (e.g., "VoNR Test Suite v1") |
STATUS | This indicates whether the evaluation completed or failed to complete. This is not the actual intent validation result of the test suite. Possible values: |
PROGRESS | Fraction of scenarios completed, e.g., 13/13 or 12/12 |
CREATED | Date the run was initiated |
EXECUTED BY | User account that triggered the run |
Actions |
|
You can toggle between MY RUNS and ALL RUNS tabs at the top of the table.
Step 6 — Download a Report
Click the REPORT button on any completed run row to download the IBD Evaluation Report for offline review.
Step 7 — View the In-App Report
Click the VIEW button for a completed run to open the rendered report in the browser.
Top of the run page: Run ID, Status (Completed), Progress (13/13), Created date, and Download Report button.
Part 2: Submit a New Test Suite Run
Once you have a PCAP file ready, follow these steps to submit a new test suite run.
Step 1 — Click "+ NEW TEST SUITE RUN"
From the Test Suite Runs page, click the + NEW TEST SUITE RUN button in the top-right corner of the page.
The Test Suite Runs page with the '+ NEW TEST SUITE RUN' button highlighted in the top-right corner.
The New Test Suite Run dialog will open.
Step 2 — Select a Test Suite
In the dialog, click the Test Suite dropdown (labelled "Select a test suite...") and choose the test suite you want to run.
The Test Suite dropdown expanded, showing the available option: "VoNR Test Suite v1".
Select VoNR Test Suite v1 (or whichever suite is appropriate for your PCAP).
Step 3 — Choose How to Provide the PCAP File
After selecting a test suite, the dialog shows two tabs for providing the PCAP data:
Tab | Description |
|---|---|
SELECT ANALYSIS | Choose a PCAP file that has already been uploaded and analysed in the platform |
UPLOAD FILE | Upload a new PCAP file directly from your computer |
Option A: Select an Existing Analysis
Click the SELECT ANALYSIS tab (active by default). Use the "Search by file name..." dropdown to find and select an existing PCAP analysis from the platform.
The SELECT ANALYSIS tab with the file name dropdown open, showing previously uploaded PCAP files available for selection (e.g., 5G_VONR_OK_05.02.2026_TEST.pcapng).
Click on the file you want to use. The file name will appear selected in the field.
Tip: You can type part of the file name in the search box to filter the list.
Option B: Upload a New PCAP File
Click the UPLOAD FILE tab. A "CHOOSE A PCAP FILE" button will appear.
The UPLOAD FILE tab with the 'CHOOSE A PCAP FILE' upload button.
Click CHOOSE A PCAP FILE and select your .pcap or .pcapng file from your local file system. The file will be uploaded to the platform and used for this run.
Step 4 — Start the Run
Once the test suite is selected and a PCAP file is chosen (or uploaded), the START RUN button will become active.
Click START RUN to submit the test suite run.
The dialog with all fields filled in and the START RUN button active, ready to submit.
Click CANCEL at any time to close the dialog without submitting.
Step 5 — Monitor the Run Progress
After submitting, you will be returned to the Test Suite Runs table. Your new run will appear at the top of the list with:
Status: A running/in-progress indicator
Progress: A live counter (e.g., 3/13) updating as each scenario completes
Created: The current timestamp
Executed By: Your account email
The run will update automatically. Once all scenarios complete, the status will change to Completed (green) and the VIEW and REPORT buttons will become available.
Understanding the IBD Evaluation Report
Summary Metrics
Three summary cards provide an at-a-glance overview of the run:
IBD Evaluation Report: summary metrics (Total Scenarios: 13, Passed: 11, Failed: 2) and Summary by Domain bar charts.
Metric | Description |
|---|---|
Total Scenarios | Total scenarios evaluated (e.g., 13 across 6 domains) |
Passed (green) | Passed scenario count and step count (e.g., 11 scenarios, 102 steps) |
Failed (red) | Failed scenario count and step count (e.g., 2 scenarios, 4 steps) |
Summary by Domain
A horizontal bar chart shows the pass/fail ratio per network domain.
The list below is the relevant domains to the VoNR test suite that we selected earlier. Other test suites may include different domains.
Domain | Description |
|---|---|
SIP Signaling | Validates SIP messages, charging vectors, identity, routing, and contact parameters |
ENUM/DNS Resolution | Verifies DNS/ENUM resolution for E2U+sip and E2U+sips control flows |
Location Resolution [Cx] | Checks Cx interface location lookup procedures |
User Data Resolution [Sh] | Validates Sh interface user data retrieval |
PCF Interactions [Rx] | Tests PCF policy interactions over the Rx interface |
SIP Headers Generated by the ASBC from [Rx] | Verifies SIP headers generated by the Application Session Border Controller |
Each bar uses green for passed steps and red for failed steps. The P/F/Total count is shown at the end of each bar.
Failed Scenarios Section
Failed scenarios are listed in the FAILED SCENARIOS table:
Failed Scenarios table listing scenario name, domain, category, failed steps, and action links.
Column | Description |
|---|---|
Scenario Name | Full name of the failed scenario |
Domain | Network domain the scenario belongs to |
Category | Test category (e.g., Call Setup) |
Failed Steps | Step numbers (e.g., STEP 8) and a brief description of what was expected |
Actions | Link icon to open the full evaluation detail |
All Scenarios Section
A complete list of all scenarios, grouped by domain:
All Scenarios table showing all 13 scenarios grouped by domain with status, steps, and action links.
Column | Description |
|---|---|
Scenario Name | Full descriptive name |
Category | Test category (e.g., Call Setup) |
Status |
|
Sets | Number of test sets |
Callflows | Number of callflows analyzed |
Steps | Pass / Fail / Total (e.g., 11/1/12) |
Actions | Link icon to open full evaluation detail |
Tip: Click the DOWNLOAD REPORT button (top-right of the run page) to save the full report.
Scenario-Level Details
Clicking the action icon on any scenario opens the step-by-step evaluation detail view.
Step Detail Panel
Each step is an expandable row. Expanding reveals:
Expanded scenario step showing the matched network frame: Frame 552, tPCSCF to tSCSCF, SIP 183 Session Progress to INVITE, with full SIP attribute details and a failed validation indicator.
Field | Description |
|---|---|
Frame | Packet frame number in the PCAP file |
Time | Timestamp (in nanoseconds) |
Source NE | Source Network Element (e.g., tPCSCF) |
Destination NE | Destination Network Element (e.g., tSCSCF) |
Protocol | Protocol used (e.g., SIP, Diameter) |
Message | SIP message or response (e.g., 183 Session Progress to INVITE) |
Message Type | Classification (e.g., Response, Request) |
Attributes | Full list of extracted SIP headers and SDP parameters |
Pass / Fail Indicators
Green checkmark — Step validation passed
Red icon — Step validation failed, with message: "No matching messages found for defined criteria, closest match at..."
Tips & Troubleshooting
Common Issues
Issue | Likely Cause | Suggested Action |
|---|---|---|
START RUN button stays greyed out | Test suite or PCAP not yet selected | Ensure both the Test Suite dropdown and a PCAP file are selected |
Run status shows Failed | Scenarios could not be evaluated | Ensure the PCAP captures the full call flow and all required protocols |
Scenario shows "No matching messages found" | Expected message absent from PCAP, or NE names don't match | Verify PCAP completeness; check Network Element naming |
Progress stuck (e.g., 5/13) | Platform processing delay or connectivity issue | Refresh the page; if stuck >5 min, contact your AGILITY administrator |
REPORT button not available | Run has not completed yet | Wait for Completed status; the button appears only after full completion |
Best Practices
Use a full, untruncated PCAP file that captures the complete call flow from setup to teardown.
Check Others' Evaluations before starting a new run to avoid duplicating work.
Download the PDF report immediately after a run completes to archive it.
Use step-level detail to pinpoint exactly which network message or SIP attribute failed — faster than re-running the full suite.
AGILITY Platform · Intent-Based Diagnostics · Test Suite Workflow User Guide · v1.1 · March 2026
For support, contact your AGILITY administrator.