Offer Testing Framework

The offer testing framework allows you to simulate how an offer's pricing, underwriting, and eligibility rules perform under different scenarios. To access it, from the offer view, open the ellipsis menu (...) and select Run Tests.

From the offer's testing framework page, you can:

Run an Individual Offer Test Case from the UI

To run a test case from the offer test framework:

  1. Go to the Offer Manual Testing section.
  2. Enter a Test Name.
  3. Optionally, add a Description, and Reference Date.
  4. If your offer includes multiple offer cards, you can select specific cards that you want to test by ticking the corresponding card names.
  5. Fill in the audience rules inputs. The available inputs are generated dynamically based on the offer's configuration.

After providing the general test information, choose the type of test you want to run:

Test Offer Pricing

  1. In the Pricing tab, provide the pricing inputs for each product in the offer. The available inputs are generated dynamically based on the product configurations.
  2. You can optionally fill in the Expected Results section. If there are differences between the actual and expected results, the test case will fail.
  3. Click Test Pricing.
  4. Once the test is complete, a Test Output window is displayed that includes the pricing details in both HTML and RAW JSON format.

Test Offer Underwriting

  1. In the Underwriting tab, enter the underwriting rules inputs for each product in the offer. The available inputs are generated dynamically based on the product configurations.
  2. You can optionally fill in the Expected Results section. If there are differences between the actual and expected results, the test case will fail.
  3. Click Test Underwriting.
  4. Once the test is complete, a Test Output window is displayed that includes the underwriting rules results in both HTML and RAW JSON format:

Test Offer Eligibility

  1. In the Eligibility tab, enter the eligibility rules inputs for each product in the offer. The available inputs are generated dynamically based on the product configurations.
  2. You can optionally fill in the Expected Results section. If there are differences between the actual and expected results, the test case will fail.
  3. Click Test Eligibility.
  4. Once the test is complete, a Test Output window is displayed that includes the eligibility rules results in both HTML and RAW JSON format:

Import and Execute Multiple Offer Test Cases in Bulk

If you need to run multiple offer test cases, you can streamline the process by uploading all the test data using an Excel template based on the offer configuration and running all the tests in one go:

  1. Click Get Template to download the Excel test template.
  2. Open the template file and fill in the data for your pricing, underwriting, and/or eligibility test cases.
    • The template is customized based on your offer configuration: mandatory fields are highlighted in red and entries are validated as you type. For detailed instructions, read the template's Instructions sheet.
    • The template includes formatting for 20 test cases, but this number is not mandatory. You can leave test cases empty or add columns for additional test cases depending on your requirements.
    • Use the Pricing Input sheet to fill in your pricing test cases.
    • Use the Underwriting Input sheet to fill in your underwriting test cases.
    • Use the Eligibility Input sheet to fill in your eligibility test cases.
    • You can optionally fill in the Pricing Result, Underwriting Result, and Eligibility Result sheets with your expected results for various test cases. If there are differences between the actual and expected results, the test case will fail.
  3. Save the test file.
  4. Go back to the offer test framework page, click Upload & Run Tests, and select the test file you saved earlier.
  5. Once the tests are finished, an Excel file with the test results is generated. To view the results you can:
    • Scroll the offer testing framework page to the Executions History section, click the > icon to expand the test, and click the eye icon next to a test case to display its output (see Inspect the Full Execution History for All the Test Runs for details).
    • Open the test results Excel file and browse the Pricing Result, Underwriting Result, and Eligibility Result sheets to view the pricing, underwriting, and eligibility tests results respectively.

Manage Mandatory Test Cases for Offer Approval

For specific offer test cases, you can designate a minimum number of successful tests that is required for offer approval (see Offer Life Cycle) and iterate through them until enough tests execute successfully.

Configure the Minimum Number of Successful Offer Test Cases

The minimum number of successful offer test cases is controlled by the following Configuration Manager key:

kv/<environment>/mkexp-pfapi/app-settings/minNumberOfSuccessfulTestsForOffersApproval

The default value is 0, allowing offer approval without enforcing any minimum number of successful tests.

Run the Offer Approval Test Cases

  1. Scroll the offer test framework page to the Execution History section.
  2. Expand the test run that contains the required test case by clicking the > icon.
  3. Tick the In Approval Test Pool checkbox next to the desired test case. This adds the test case to the Approval Test Pool section.
  4. Once you have populated the Approval Test Pool section with all the required test cases, you can adjust and re-run them until enough test cases execute successfully. See Test Case Actions for details on how to reload a test case.
NOTE  
Test cases executed from the Approval Test Pool section are not recorded in the Execution History. This lets you repeat critical tests as needed without adding excess entries to the execution log.

Offer Approval Test Pool Batch Actions

In the Approval Test Pool section, the following batch actions are available:

  • Run All Tests - Executes all selected test cases at once.
  • Download Test Cases - Exports an Excel test template containing the input data of all selected test cases. You can then use this file to run the tests in their current state or after modifying the test inputs (see Import and Execute Multiple Offer Test Cases in Bulk).

Offer Approval Test Pool Individual Actions

Each individual test case in the Approval Test Pool section includes the following actions:

  • Reload the test case - Click the pencil icon to edit the test case inputs and re-run it (see Test Case Actions for information on how to reload test cases).
  • Run the test case - Click the play icon.
  • Remove the test case from the list of selected test cases - Click the trash can icon.

Review Aggregated Test Analytics

To view summary metrics for your test runs, scroll the offer test framework page to the Test Analytics section. This area displays:

  • A distribution of test outcomes pie chart (passed, failed, incomplete)
  • A number of tests by category bar chart (pricing, underwriting, eligibility)
  • A day-to-day trend of executed tests line chart.

Use the From Date and To Date selectors to limit the analytics to a specific time interval.

Inspect the Full Execution History for All the Test Runs

Scroll the offer test framework page to the Execution History section to inspect all the offer's test runs.

Test Actions

  • Click the > icon to expand a test row and view its individual test cases.
  • Click the Download button to download an Excel test template populated with the test's input data. You can use this file to re-run the test either in its current form or after modifying the test inputs (see Import and Execute Multiple Offer Test Cases in Bulk).

Test Case Actions

  • View test case output - Click the eye icon case to display the test case output.
  • Reload the test case - Click the pencil icon to edit the test case inputs and execute it again:
    1. This scrolls the page to the Offer Testing section and populates it with the test case's input data.
    2. Update the inputs as needed and save them.
    3. After saving, the test case runs automatically and its results are added to the original test in the test execution history.
  • Archive the test case - Click the folder icon. Archived test cases cannot be reloaded or marked as selected for offer approval (see Manage Mandatory Test Cases for Offer Approval). Click the icon again to unarchive the test case. If archived test cases are hidden, use the Archived Status filter to display them (see Filter the Test Execution History for details).

Filter the Test Execution History

You can click the Show Filters button to filter the displayed test cases. You can filter test case results by:

  • Calculation Type - Pricing, Underwriting, Eligibility, or All
  • Status - Success, Failed, Pending, Running, or All
  • Time Interval - Date From and Date To controls
  • Source - Manual, Automatic, or All
  • Test Case Name
  • Archived Status - You can either show or hide archived test cases.