Skip to main content

Training and Activating a Model

This guide walks you through training an AI model on one of your rule plans, enabling it so it scores your transactions, and reviewing the results. For what the AI risk score means, see How AI Risk Scoring Works.

Before You Start​

You need:

  • A Finalised rule plan - Models can only be trained on a Finalised plan. A plan becomes Finalised when you save it with Save as Finalised, or automatically when it is set as an Active Plan. See Rule Plans.
  • The right role - Training and enabling models requires the Org Owner/ Org Admin/ Rules Manager role. The Rules Viewer role can view models but not train or enable them.

Step 1: Start Training​

You can start training from either of two places:

  • AI Models page - Choose the plan in Select rule plan and select Train Model. Training starts immediately, without a confirmation step.
  • Rule Plans page - Expand a Finalised plan and select Train Model, then select Start Training in the Training Confirmation dialog.

Either way, you can keep using the application while the model trains.

One Pipeline at a Time

Your organisation trains one model at a time. If you start training while another pipeline is in progress, the new pipeline waits as Pending and starts automatically once the earlier one finishes. A rule plan can only have one pipeline in progress, so training a plan that is already being trained shows an error and does not start a second pipeline.

Step 2: Monitor the Pipeline​

Training runs as a training pipeline. Follow its progress in the Active Pipelines panel, on the AI Models page or in the expanded plan on Rule Plans. The panel refreshes every 10 seconds and shows each step - Initializing, Generate Synthetic Data, Extract Features, Train Model and Complete - along with the overall Progress. Its status badge shows where the pipeline is:

  • Pending - The pipeline is waiting to start, including while an earlier pipeline in your organisation finishes.
  • Running - The pipeline is working through its steps.
  • Retrying - A step is being retried automatically. You do not need to do anything.
  • Failed - The pipeline ended without producing a model, for example because its retries ran out, it stopped reporting progress or it waited too long to start. Start a new training run for the plan.
  • Cancelled - FraudShield support stopped the pipeline before it completed. Start a new training run if you still need the model.

Once the pipeline finishes, it disappears from Active Pipelines and appears in the Pipeline History table.

Step 3: Review the Result​

In the Pipeline History table on the AI Models page, check the finished pipeline:

  • Accuracy - How well the model distinguishes risky transactions from safe ones, measured on test data held back from training. Higher is better.
  • Rules in Plan - The number of rules in the plan the model was trained on.
  • Time Taken - How long training took.

Step 4: Enable the Model​

Training a model does not put it to use. To enable it:

  1. On the Rule Plans page, expand the plan and select Select Trained Models.
  2. In the Trained Models dialog, tick Enable Production, Enable Test or both against the model you want to use. Each environment can have one model enabled, and unticking the enabled model disables it.
  3. Select Save.

If the plan is the active plan in either environment, a confirmation dialog lists the changes. Type Confirm and select Confirm to apply them. Batches already in progress keep using the model assigned to them.

Active Plans Only

A model only scores transactions while its plan is the Active Plan for that environment. If you enable a model on a plan that is not active, activate the plan in that environment to start scoring.

Test a new model in the Test environment first, in the same way as a new rule plan. Once you are happy with the scores, enable it for Production.

Step 5: Turn On Auto Case Creation (Optional)​

By default, cases are only created by your rules. To also create cases for transactions that trigger no rules but score within a range you choose, turn on Auto Case Creation Based on AI Risk Score in the Case Settings on the Cases page.

This setting needs a different set of roles: Org Owner/ Org Admin/ Business Manager. It applies to both environments.

Step 6: Review the Results​

As transactions are processed, look at the scores in:

  • Transactions - The AI Risk score on each scored transaction, with View Rule Breakdown leading to the AI Risk Assessment.
  • Cases - Sort by AI Risk Score, or use the Min AI Risk Score and Max AI Risk Score filters to focus on high-scoring cases.

Pay particular attention to high-scoring transactions that triggered no rules - they are the main reason to use AI risk scoring.

Retraining and Switching Models​

A Finalised plan cannot change, so a trained model stays a match for its plan and retraining is optional. You can train several models for the same plan and switch the enabled model at any time from the Trained Models dialog, for example to use a model with higher Accuracy.

When you want to score transactions against a different set of rules, create and finalise a new rule plan, train a model on it, and enable that model before or after activating the plan.

What's next?

That covers training and activating AI models in FraudShield. Head to the Features section to explore the full range of pages available in the application, or revisit How AI Risk Scoring Works for how to read the scores.