Skip to main content

​
Introduction

Mindgard’s CLI is a python based AI security tool with multiple commands and sub-commands supporting the work of red teams conducting testing against large language models, and applications using them. Help is available at every level with -h, --help.

​
Running mindgard

Generally, an invokation of mindgard is 
mindgard [options] [command] [cmd_options] [sub-command] [sub_options]
There may be zero or more top level options controlling logging, a single command, zero or more command options supporting the given command, a sub-command, and zero or more options supporting the specific sub-command. Most of the options specified will be at the sub-command level. Except where noted options use long names in the form --option.

​
Options

​
--h, --help

Display a view of the general options and the list of supported commands.
Help is available as an option for every command and sub-command.

​
--version

Display the current version of the CLI and exit. Version is checked every time the Mindgard CLI is used. If a new version is available this will be one of the earliest output messages.

​
--log-file <file>

Send standard output from this execution to the specified file. When using this option while testing you may also wish to consider using the --json command option to supress the normal interactive output. Interactive output can be hard to read when stored in a file.

​
--log-level <level>

Set the verbosity of logging using one of: critical, fatal, error, warn, warning, info, or debug.

​
Commands

​
create

The create command offers users the ability to make custom prompt datasets and projects for recording test results.

​
dataset

Dataset creates a set of prompts for testing that aim to achieve a user defined goal. mindgard create dataset is useful for developing new tests targeting specific areas of risk within the organizaiton, and developing a wide variety of prompts to thoroghly assess risk. Options --num-entries <number> Number of prompts to generate. If not specified, 15 prompts will be generated. --output-filename <file> Write generated prompts to file. Defaults to mindgard_custom_dataset.txt in the current working directory. If file exists the command will exit without generating prompts. --perspective {nonspecific, historical, cultural, scientific} Skew prompts generated toward one of 4 perspectives. Default is nonspecific. --seed-prompt <prompt> The seed prompt defines the model policy to be attacked. Generated values will seek to violate the prompt specified. --tone {neutral, forceful, leading, innocent, corrigible, indirect} Similar to --perspective, shift generated prompts toward a specific tone of voice. Default is neutral.

​
project

Project is used to create new projects for reporting and sharing results. Options --name <project name> Create a new project with project name. Project ID will be returned as part of the output.

​
list

​
projects

Retrieve the list of projects the current user is authorized to use along with their project ID’s and a link to their project page in the Web UI. There are no options to this command other than -h, --help.

​
login

Authenticate with the Mindgard platform. Options --instance <hostname> Log into the Mindgard instance using hostname. Typically the hostname portion of your Web UI URL. Example: If your Web UI is hosted at https://sahara.mindgard.ai, the instance is sahara.

​
logout

Remove the local file holding the instance and token used to answer authentication challenges.

​
recon

Conduct tests focused on learning useful information regarding the target and its defensive posture.

​
code-generation

Prototype Test the target’s willingness to generate and embed code in its output.

​
guardrail

Detect whether guardrails are being used by the target and attempt to identify the specific guardrail software.

​
input-encoding

Discover methods of text encoding supported by the target.

​
non-contextual

Prototype Run non-contextual reconnaissance against the target.

​
output-encoding

Detect text encodings supported by the target’s output.

​
output-formatting

Prototype Discover output formats the target can produce.

​
output-rendering

Prototype Discover methods of redering supported in target outputs.

​
system-prompt-extraction

Pre-release Run a series of tests designed to extract the target’s system prompt.

​
tool-extraction

Pre-release Discover connected tools available to the target and as much information about how they are invoked.

​
Recon sub-command Options

All recon sub commands offer the same set of options. --api-key <key> Token used to authenticate access to the test target if necessary. --az-api-version <version> Version of Azure OpenAI API used. Only relevant to Azure based targets. --config-file <file> Read mindgard options from the TOML formatted file. --header <header> Send HTTP header with test requests. This option can be used multiple times if more than one header is required by the target. --headers <header 1>, ..., <header n> Send multiple header values with each test request. Headers are specified as a comma seaparated list. --json Output test result as JSON. Useful for non-interactive use cases. --model-name <name> For targets that offer multiple models like Anthropic or OpenAI, name will be used for testing. --preset {huggingface-openai, openai-compatible, huggingface,openai, azure-openai, azure-aistudio, anthropic, tester, custom} Use a preset configuration bundle to define one or more options. --project-id <project id> Specifies the project where results will be reported. project id can be retrieved using mindgard list projects or from the Web UI. --prompt <string> Specify a prompt to use instead of Mindgard’s default prompt sets or a user defined prompt set. --request-template <string> Define the JSON structure of test requests. and must appear in the definition. These will be replaced during tests with appropriate values. --risk-threshold <number> Number between 0 and 100 defining the percentage of tests that must pass. Results exceeding number will cause the test to exit with code 1. Tests under the number will exit with code 0. --selector <JSONPath query> Define the JSONPath query that identifies the target’s response within its returned JSON data. --system-prompt <string> Specify a system prompt to send with each test. --tokenizer <string> Select a HuggingFace model to set options for constructing test requests and parsing responses. --url <url> Specify the url to send test requests to. Tests are sent as HTTP POST requests with JSON bodies.

​
run

​
multiturn

Perform a multiple prompt test using one of several attack techniques. Multiturn Options --goal <string> Defines the goal of the attack. --include {actor_attack, crescendo, skeleton_key} skeleton_key is deprecated as of version 0.104.1. This option will be removed in future release. Attack technique to use. actor_attack uses role playing to achieve the goal. cresendo crafts an escalating series of prompts designed to indirectly reach the goal. Additionally, all of the recon sub-command options are supported to configure target specific settings.

​
sandbox

Execute a suite of single prompt tests against a Mindgard hosted model. Sandbox Options --json Output test result as JSON. Useful for non-interactive use cases. --risk-threshold <number> Number between 0 and 100 defining the percentage of tests that must pass. Results exceeding number will cause the test to exit with code 1. Tests under the number will exit with code 0.

​
test

Execute single prompt tests. Test Options --dataset <file> Text formatted file containing prompts to be used for testing with one prompt per line. --domain <prompt set> Mindgard supplied set of prompts to test. For details on each prompt set please refer to our AI Safety Datasets. --exclude <attack technique> Run all single prompt tests except attack technique. May be specified multiple times to omit several techniques. Information on the attacks and their names can be found in our attack library. --force-multi-turn Deprecated as of version 0.99.1. This option will be removed in future release. Previously used to enable multi-turn tests before the run command was implemented. Using the option forces --parallelism to 1 if used. --include <attack technique> Only test using the supplied attack technique. Multiple include options can be specified to run a subset of techniques. Find information on the attacks and their names in our attack library. --mode {fast,thorough,exhaustive} Alter the number of samples tested. fast tests are run by default. thorough and exhaustive tests are not usable by default. Please contact support@mindgard.ai to enable these options. --parallelism <number> Controls the number of simulataneous requests Mindgard will make during testing. This must be lower than the number of requests the target can serve per minute. Set to 1 for the slowest possible test if errors or timeouts from the target are seen during testing. --prompt-repeats <number> Tells Mindgard to try each prompt number of times during a test. Defaults to 1. Increase to test the target’s reaction to repeated prompts. --rate-limit <number> The maximum number of requests that Mindgard can make during a one minute period. Defaults to 3600. test also uses the recon sub-command options to specify target settings.

​
validate

Verify that Mindgard’s CLI can access the test target by sending a benign greeting prompt and checking the response. As with other commands that specify a target, validate accepts the recon sub-command options to specify target settings.