Testing via Burp
Mindgard provides a Burp Extension for testing applications via Burp Intruder. This operates as an alternative client to the Mindgard CLI.
You can download Mindgard Burp Extension jar file from Mindgard’s Github under the assets of the latest release.
You can find details on how to install a custom Burp extension in the Burp documentation.
Selecting the Mindgard Extension
Once installed, you can select the Mindgard extension as a payload generator.
Configuring the Mindgard Extension
Selector
The Selector is a JSON Path expression (https://jsonpath.com), that tells Mindgard how to identify your Model’s response within the API response.
Your browser devtools may be useful to observe the structure of your API response to determine what this should be set to.
Test Name
An human readable identifier for your Mindgard test.
System Prompt
You can specify the system prompt for the AI model. If you’re testing a model inference API directly, you may wish to include the real system prompt used by your application here to simulate its performance as part of the wider application.
If the system prompt is not relevant to your tests, the default prompt used will be “Please answer the following question”
Dataset
You can select from a range of provided datasets for a given domain including CustomerService, Finance, Legal, Medical, Injection and XSS
Custom Dataset
Alternatively to selecting a dataset, you may provide your own custom set of prompts. Mindgard attacks will then use your prompts and only your prompts with each technique.
Instructions to setup a file with custom datasets can be found here
Prompt Repeats
This is the number of times each prompt in the dataset will be repeated against the model. The default is 1. This is useful for testing models that are not deterministic and may return different results for the same input.
Running a subset of attacks
Similar to the CLI, if you only wish to run a subset of attacks, you can do so by either excluding the attacks or including only the attacks that are required. Attack names and attack categories to exclude or include can be found here.
Exclude attack(s)
The attack names or attack categories can be provided as a comma separated string, e.g. EvilConfidant,skeleton_key
Include attack(s)
The attack names or attack categories can be provided as a comma separated string, e.g. jail_breaking