SeanHarrington/jurorsim-mistral24b

Model Card for Juror Simulation Fine-Tuned Model

This model is a fine-tuned version of Mistral-Small-24B-Base-2501 designed to simulate juror decision-making behavior in criminal trials. The model has been trained to predict how jurors with different demographic characteristics would rate the likelihood of a defendant's guilt based on trial evidence.

Model Details

Model Description

This model simulates juror decision-making by taking demographic information (sex, ethnicity, political affiliation, economic status, education) and trial scenario details as input, then outputting a rating on a 0-10 scale indicating how much more likely the evidence makes the defendant's guilt. The model was fine-tuned using LoRA (Low-Rank Adaptation) with FP16 precision to efficiently adapt the base Mistral-24B model for this specific legal reasoning task.

• Developed by: Sean A. Harrington, Director of Technology Innovation, University of Oklahoma College of Law
• Funded by [optional]: University of Oklahoma College of Law
• Shared by [optional]: [More Information Needed]
• Model type: Fine-tuned Large Language Model (LoRA adaptation)
• Language(s) (NLP): English
• License: MIT License
• Finetuned from model [optional]: mistralai/Mistral-Small-24B-Base-2501

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Direct Use

This model is designed for legal research and jury simulation studies. It can be used to:

• Predict how jurors with different demographic backgrounds might evaluate evidence
• Study the impact of demographic factors on juror decision-making
• Assist in legal research on jury bias and decision-making patterns
• Provide insights for legal professionals and researchers studying jury behavior

Downstream Use [optional]

The model can be integrated into:

• Legal research platforms
• Jury selection analysis tools
• Legal education and training programs
• Academic research on juror decision-making

Out-of-Scope Use

This model should NOT be used for:

• Making actual legal decisions or recommendations
• Replacing human jurors or legal professionals
• Determining guilt or innocence in real cases
• Making predictions about specific individuals
• Any use that could influence actual legal proceedings

Bias, Risks, and Limitations

The model inherits biases from both the base model and the training data, which includes demographic information that may reflect societal biases. The training data focuses on a specific legal scenario (armed robbery with co-defendants) and may not generalize to other types of cases. The model's predictions are simulations and should not be considered as actual juror behavior.

Recommendations

Users should be aware that this is a research tool for studying juror decision-making patterns, not a tool for making legal decisions. The model's outputs should be used for academic and research purposes only. Results should be interpreted with caution and in the context of the specific scenario and demographic factors represented in the training data.

How to Get Started with the Model

Use the code below to get started with the model.

from openpipe import OpenAI

client = OpenAI(
    openpipe={"api_key": "YOUR_OPENPIPE_API_KEY"}
)

completion = client.chat.completions.create(
    model="openpipe:juror4scenarios",
    messages=[
        {
            "role": "system",
            "content": "You are a juror simulation assistant."
        },
        {
            "role": "user",
            "content": "Demographics:\n- Sex: Male\n- Ethnicity: White\n- Political Affiliation: Independent\n- Economic Status: $50,000 - $74,999\n- Education: Bachelor's degree\n\nScenario: [Your legal scenario here]"
        }
    ],
    temperature=0
)

print(completion.choices[0].message.content)

Training Details

Training Data

The model was trained on 1,199 examples of juror responses to a specific legal scenario involving armed robbery with co-defendants. The training data includes:

• Demographic information: sex, ethnicity, political affiliation, economic status, education level
• Legal scenario: Joint trial for armed robbery with two co-defendants
• Human ratings: 0-10 scale indicating likelihood of guilt based on evidence
• Data format: OpenAI-compatible JSONL format with system/user/assistant message structure

The training data was converted from CSV format to JSONL format suitable for fine-tuning, with each example containing demographic context and the corresponding human juror rating.

Training Procedure

Preprocessing [optional]

The training data was preprocessed to convert from CSV format to OpenAI-compatible JSONL format. Each training example includes:

• System message defining the assistant's role
• User message containing demographic information and legal scenario
• Assistant message containing the human rating in JSON format

Training Hyperparameters

• Training regime: LoRA fine-tuning with FP16 precision
• Base model: mistralai/Mistral-Small-24B-Base-2501
• Fine-tuning method: Parameter-Efficient Fine-Tuning (PEFT) using LoRA
• Precision: FP16 (16-bit floating point)

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model was evaluated using the same dataset structure as training data, with demographic variations and the same legal scenario to test consistency and demographic sensitivity.

Factors

Evaluation factors include:

• Demographic variations (sex, ethnicity, political affiliation, economic status, education)
• Rating consistency across similar demographic profiles
• Response patterns to the specific legal scenario

Metrics

• Rating accuracy compared to human responses
• Demographic sensitivity analysis
• Response consistency and reliability

Results

[More Information Needed]

Summary

The model successfully learned to predict juror ratings based on demographic information and legal scenario details, demonstrating the ability to simulate juror decision-making patterns.

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

The model uses LoRA (Low-Rank Adaptation) fine-tuning on the Mistral-Small-24B-Base-2501 architecture. LoRA allows efficient adaptation of large language models by adding small rank decomposition matrices to existing weights, significantly reducing the number of trainable parameters while maintaining performance.

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

• Framework: PEFT 0.15.2
• Base Model: Mistral-Small-24B-Base-2501
• Fine-tuning Platform: OpenPipe
• Precision: FP16

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

• LoRA (Low-Rank Adaptation): A parameter-efficient fine-tuning method that adds small rank decomposition matrices to existing model weights
• FP16: 16-bit floating point precision used for training to reduce memory usage and increase speed
• PEFT: Parameter-Efficient Fine-Tuning, a library for efficient adaptation of large language models
• JSONL: JSON Lines format, where each line contains a valid JSON object, commonly used for fine-tuning datasets

More Information [optional]

This model is part of a research project at the University of Oklahoma College of Law focused on understanding juror decision-making patterns and demographic influences on legal reasoning.

Model Card Authors [optional]

Sean A. Harrington, Director of Technology Innovation, University of Oklahoma College of Law

Model Card Contact

[More Information Needed]

Framework versions

• PEFT 0.15.2