Skip to content
AI Illuminate

Jailbreak T2T Benchmark v0.5

The AILuminate Jailbreak benchmark evaluates AI system resistance to jailbreaking attempts across different attack scenarios. These results show the AILuminate Safety benchmark as the foundational, "naïve Safety" reference and the post-prompt injection attack safety rating as the Jailbreak score.


MLCommons applied the AILuminate v0.5 Jailbreak benchmark to a variety of publicly available AI systems from leading vendors. Results have been de-identified for the v0.5 release.
Benchmark results visualization

Benchmark Highlights

  • As with the AILuminate v1.0 benchmark, no SUT received a safety grade of Excellent. Three SUTs were graded Very Good, which means that they performed somewhat better than the reference SUT.
  • No SUT received a security grade better than Good.
  • Of the 39 SUTs tested, no SUTs scored better for jailbreak resistance than for safety.
  • Out of 39 SUTs tested, only four SUTs did not receive a lower grade for jailbreak resistance than for safety.
  • Of 35 SUTs that were graded lower for jailbreak resistance than for safety, 29 were reduced by one grade level and 6 were reduced by two grade levels (five from Good to Poor and one from Very Good to Fair).
These initial results clearly illustrate the impact of jailbreak techniques on safety. The jailbreak techniques used negatively affected the grade of all but four of the SUTs.

De-Identified System 1

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 2

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 3

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 4

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 5

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 6

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 7

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 8

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 9

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 10

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 11

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 12

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 13

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 14

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 15

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 16

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 17

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 18

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 19

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 20

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 21

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 22

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 23

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 24

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 25

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 26

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 27

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 28

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 29

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 30

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 31

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 32

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 33

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 34

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 35

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 36

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 37

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 38

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

De-Identified System 39

Poor
Fair
Good
Very Good
Excellent

Security Naive Hazard 0.5 En Us Demo

Security Jailbreak Hazard 0.5 En Us Official

Score %

100
46.3
23.1
7.7
0.1
0
Worse
Reference
Better

For support questions, contact: ailuminate-support@mlcommons.org