Skip to main content
Mindgard provides a Burp Extension for testing applications via Burp Intruder. This operates as an alternative to the Mindgard CLI for single-turn tests.

Downloading and Installing the Mindgard Extension

You can download Mindgard Burp Extension jar file from Mindgard’s Github under the assets of the latest release. You can find details on how to manually install a custom extension in Burp’s documentation. Once installed, your Burp Suite should have an additional Mindgard tab. Burp menu with Mindgard tab to the right of Burp's default items

Configuring the Mindgard Extension

There are two pages with options related to configuring a test. At the bottom of each is a Save All button that saves the work in both panes when clicked. Be sure to save your changes before moving to the Burp Intruder tab for testing.

Login Options

On the extension’s Login pane configure the Mindgard URL to match the URL used to login via the Web UI. Mindgard URL setting The default Audience, https://prod-pov-demo-orchestrator.com, and Client ID, mBSDAk4R283BeGC17Uvbyo3vSAAGL67J, are associated with the default Mindgard URL, https://demo.us.mindgard.ai. If you are using a different URL please contact support@mindgard.ai for the proper values. These two options will be automatically configured and removed from the interface in a future release.

Test Configuration

This pane holds many of the same options that are configured via CLI arguments or within a configuration file. At a minimum, Selector and Project ID must be specified before a test is run. Mindgard Test Configuration options

Selector

The Selector is a JSON Path expression (https://jsonpath.com), that tells Mindgard how to identify your Model’s response within the API response. Your browser devtools may be useful to observe the structure of your API response to determine what this should be set to.

Project ID

The project ID used to group your test results. See the projects page for more information.

System Prompt

You can specify the system prompt for the AI model. If you’re testing a model inference API directly, you may wish to include the real system prompt used by your application here to simulate its performance as part of the wider application. When a custom system prompt is not relevant to your testing this field can be left blank. When not specified the default prompt, “Please answer the following question”, is used.

Dataset

Focus testing on a specific area of risk using one of Mindgard’s AI Safety Datasets.

Custom Dataset

Alternatively to selecting a dataset, you may provide your own custom set of prompts. Mindgard attacks will then use your prompts, and only your prompts, with each technique. Instructions to setup a file with custom datasets can be found here

Prompt Repeats

This is the number of times each prompt in the dataset will be repeated against the model. The default is 1. This is useful for testing models that are not deterministic and may return different results for the same input.

Running a subset of attacks

Similar to the CLI, two options exist to limit the types of attacks used in a test.

Exclude attack(s)

Execute a subset of attack techniques by excluding one or more methods. Specify attacks by name separating multiple attacks with commas. All other methods will be used.

Include attack(s)

Use a subset of attacks by specify which techniques should be used. The attack names can be provided as a comma separated string to use multiple techniques. No other attacks will be used.

Parallelism

Parallelism controls the maximum number of parallel attacks we will launch against your model. If your chatbot is stateful, or otherwise would be confused/overwhelmed by overlapping attacks, leave this at the default of 1.

Testing with the Mindgard Extension

Mindgard tests are run through Burp Intruder where Mindgard can be selected as a payload generator. You will also need to craft the attack payload which is similar to the CLI’s request-template option. While Intruder offers several attack options, only Sniper attack is useful for Mindgard testing. Other test types will fuzz data in ways that Mindgard’s analysis is not prepared to interpret.
1

Set Intruder's payload type

In the Intruder payload options set the payload type to Extension-generated from the pulldown.Setting Burp Intruder payload type to Extension-generated
2

Set Selected generator to MindgardGenerator

At the bottom of the payload options set the selected generator by clicking Select Generator and choosing MindgardGenerator from the popup dialog’s pulldown.Selecting MindgardGenerator from the select generator pulldown
3

Set the target URL

Set Intruder’s target option to the URL of the application or model you wish to test. Do not include the URL path in this setting. If you have tested with Mindgard’s CLI you may be used to configuring a complete URL. In Burp, the target URL only describes the protocol, hostname and port. For an application hosted locally running on port 4200 using HTTP the url is http://localhost:4200.Setting Intruder URL to http://localhost:4200
4

Craft the attack payload

The final step before testing is to configure Intruder’s payload. In this pane you will need to craft the request type, URL path, any headers required, and the payload. For example, an application hosting a model at http://localhost:4200/api/v1/chat/completions that accepts prompts via POST requests, the first line of the payload could start POST /api/v1/chat/completions HTTP/1.0. The next lines should contain any headers required to send for your application and a blank line to mark the boundary between the request headers and body. The body of your request is the same as the CLI option, request-template.Mindgard CLI request-template pasted into Burp Intruder as an attack payloadIn the above image a request-template value has been pasted into Burp as the HTTP body of the attack requests. Before testing however, the variables {prompt} and {system_prompt} need to be identified to Burp as payload positions. Click and drag over each variable one at a time, and click Add §. Below both variables have been turned into payload positions and the configuration is ready for testing.Burp Intruder attack payload with two Mindgard configuration variables added as payload positions
5

Start the attack

Click Start attack to begin the attack. A new window will pop up allowing you to view the requests and responses. Mindgard’s web UI will also begin to populate results in the test’s project page as it runs.Burp Intruder running a test using Mindgard