AI Test Tool

The AI Test Tool is an essential component of the developer tools for Copilot in Business Central. It focuses on data-driven test automation to ensure that AI systems are accurate with various inputs, maintain the trust and security of our customers and their data, and are resilient to changes in AI model versions.

With the AI Test Tool, you can:

• Create tests for your prompt dialogues using a familiar framework in AL and Visual Studio Code.
• Organize your test datasets based on purpose and prompt variations.
• Implement automatic verification depending on your feature's use case.
• Run tests as fully automated or semi-automated.

Step-by-step: using the AI Test Tool

This goes over how to use the AI Test tool to set up a test suite, execute it, and view the results.

Prerequisite: You already created and published test codeunits for your AI Tests and created the necessary datasets. See Creating datasets and Writing AI tests.

Step 1 - upload datasets

1. Open the AI Test Suite page.
2. Open the Test Input page using the Input Datasets action.
3. Import the dataset using the Import data-driven test inputs action and uploading your dataset.

Once the dataset is uploaded, you can open and view the dataset in Business Central. From here, you can also view the test input of each line by clicking the test input. You can also set the dataset to Sensitive by toggling the Sensitive toggle, which hides the test input and output by default, both in the dataset view and results view.

Step 2 - set up the test suite

1. Open the AI Test Suite page.
2. Upload the previously created datasets for the tests.
3. Create a test suite using the New action.
4. Open the newly created test suite.
5. Configure the test suite (see the following example).

Specify a unique code for the test suite and description. On the header level, select the input dataset that was previously uploaded. On the test line, add the test codeunits previously created. Optionally, specify a line-specific input dataset for each line that requires a different input dataset than the header-level input dataset.

Step 3 - preconfigure the test app

These steps make sure that the test suite is preconfigured whenever the test app is installed.

1. Open the AI Test Suites page.
2. Export the previously created test suite using the Export action.
3. In your test app, add the datasets and test suite to the /.resources folder.
4. Create an install codeunit that imports the test suite and dataset when installed.

codeunit 50201 "Marketing Text Simple Install"
{
    Subtype = Install;

    trigger OnInstallAppPerCompany()
    var
        DatasetPaths: List of [Text];
        TestSuitePaths: List of [Text];
        ResourcePath: Text;
    begin
        // Load Datasets
        DatasetPaths := NavApp.ListResources('Datasets/*.jsonl');
        foreach ResourcePath in DatasetPaths do
            SetupDataInput(ResourcePath);

        // Load Test Suites
        TestSuitePaths := NavApp.ListResources('TestSuites/*.xml');
        foreach ResourcePath in TestSuitePaths do
            SetupTestSuite(ResourcePath);
    end;

    local procedure SetupDataInput(FilePath: Text)
    var
        AITALTestSuiteMgt: Codeunit "AIT AL Test Suite Mgt";
        FileName: Text;
        ResInStream: InStream;
    begin
        // Get the filename from the path
        FileName := FilePath.Substring(FilePath.LastIndexOf('/') + 1);

        NavApp.GetResource(FilePath, ResInStream);
        AITALTestSuiteMgt.ImportTestInputs(FileName, ResInStream);
    end;

    local procedure SetupTestSuite(Filepath: Text)
    var
        AITALTestSuiteMgt: Codeunit "AIT AL Test Suite Mgt";
        XMLSetupInStream: InStream;
    begin
        NavApp.GetResource(Filepath, XMLSetupInStream);
        AITALTestSuiteMgt.ImportAITestSuite(XMLSetupInStream);
    end;
}

Step 4 - run the test suite

1. Open the AI Test Suites page.
2. Open the previously created test suite.
3. Run the AI Test Suite using the Start action. Alternatively, run them one line at a time using the Start action for each line. Each test method is executed for each dataset line.
4. The tests doing the evaluation in AL either fail or succeed based on the condition.
5. Test Output must be generated for all the tests, which needs to be evaluated externally.

Step 5 - inspect the results

1. Open the AL Test Tool and switch to the created test suite to execute each test manually.
2. Open Log Entries page to see the result of each execution.
3. Download the test output, which generates the .jsonl file or export the logs to Excel.
4. Open the AL Test Tool and switch to the created test suite to execute each test manually.

Related information

Business Central Copilot Test Toolkit
Build the Copilot capability in AL
Test the Copilot capability in AL
Create datasets
Write AI tests
Best practices for testing the Copilot capability