Share via

CodeVulnerabilityEvaluator Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Evaluates code vulnerability for a given query and response for a single-turn evaluation only, where query represents the user query or code before the completion, and response represents the code recommended by the assistant.

The code vulnerability evaluation checks for vulnerabilities in the following coding languages:

  • Python

  • Java

  • C++

  • C#

  • Go

  • Javascript

  • SQL

The code vulnerability evaluation identifies the following vulnerabilities:

  • path-injection

  • sql-injection

  • code-injection

  • stack-trace-exposure

  • incomplete-url-substring-sanitization

  • flask-debug

  • clear-text-logging-sensitive-data

  • incomplete-hostname-regexp

  • server-side-unvalidated-url-redirection

  • weak-cryptographic-algorithm

  • full-ssrf

  • bind-socket-all-network-interfaces

  • client-side-unvalidated-url-redirection

  • likely-bugs

  • reflected-xss

  • clear-text-storage-sensitive-data

  • tarslip

  • hardcoded-credentials

  • insecure-randomness

Note

If this evaluator is supplied to the evaluate function, the metric

for the code vulnerability will be "code_vulnerability_label".

Constructor

CodeVulnerabilityEvaluator(credential, azure_ai_project, **kwargs)

Parameters

Name Description
credential
Required
TokenCredential

The credential for connecting to Azure AI project. Required
azure_ai_project
Required
Union[str, AzureAIProject]

The Azure AI project, which can either be a string representing the project endpoint or an instance of AzureAIProject. It contains subscription id, resource group, and project name.
kwargs
Required
Any

Additional arguments to pass to the evaluator.

Attributes

id

Evaluator identifier, experimental and to be used only with evaluation in cloud.

id = 'azureai://built-in/evaluators/code_vulnerability'