metadata
base_model: google/flan-t5-xl
datasets:
- 2024-mcm-everitt-ryan/job-bias-synthetic-human-benchmark
language: en
license: apache-2.0
model_id: flan-t5-xl-job-bias-seq2seq-cls
model_description: >-
The model is a multi-label classifier designed to detect various types of bias
within job descriptions.
developers: Tristan Everitt and Paul Ryan
model_card_authors: See developers
model_card_contact: See developers
repo: https://gitlab.computing.dcu.ie/everitt2/2024-mcm-everitt-ryan
compute_infrastructure: Linux 6.5.0-35-generic x86_64
software: Python 3.10.12
hardware_type: x86_64
hours_used: N/A
cloud_provider: N/A
cloud_region: N/A
co2_emitted: N/A
direct_use: |2-
```python
from transformers import pipeline
pipe = pipeline("text-classification", model="2024-mcm-everitt-ryan/flan-t5-xl-job-bias-seq2seq-cls", return_all_scores=True)
results = pipe("Join our dynamic and fast-paced team as a Junior Marketing Specialist. We seek a tech-savvy and energetic individual who thrives in a vibrant environment. Ideal candidates are digital natives with a fresh perspective, ready to adapt quickly to new trends. You should have recent experience in social media strategies and a strong understanding of current digital marketing tools. We're looking for someone with a youthful mindset, eager to bring innovative ideas to our young and ambitious team. If you're a recent graduate or early in your career, this opportunity is perfect for you!")
print(results)
```
>> [[
{'label': 'age', 'score': 0.9883460402488708},
{'label': 'disability', 'score': 0.00787709467113018},
{'label': 'feminine', 'score': 0.007224376779049635},
{'label': 'general', 'score': 0.09967829287052155},
{'label': 'masculine', 'score': 0.0035264550242573023},
{'label': 'racial', 'score': 0.014618005603551865},
{'label': 'sexuality', 'score': 0.005568435415625572}
]]
Classification Report:
precision recall f1-score support
age 0.72 0.57 0.63 81
sexuality 0.84 0.79 0.82 81
disability 0.70 0.60 0.65 81
masculine 0.64 0.62 0.63 81
feminine 0.84 0.89 0.86 81
general 0.28 0.44 0.34 82
racial 0.62 0.86 0.72 78
micro avg 0.63 0.68 0.65 565
macro avg 0.66 0.68 0.66 565
weighted avg 0.66 0.68 0.66 565
samples avg 0.31 0.35 0.32 565
model-index:
- name: flan-t5-xl-job-bias-seq2seq-cls
results:
- task:
type: multi_label_classification
dataset:
name: 2024-mcm-everitt-ryan/job-bias-synthetic-human-benchmark
type: mix_human-eval_synthetic
metrics:
- type: loss
value: 0.6297690868377686
- type: accuracy
value: 0.724596391263058
- type: f1_micro
value: 0.6541737649063032
- type: f1_macro
value: 0.6649871410336159
- type: f1_samples
value: 0.7891104779993668
- type: f1_weighted
value: 0.6641255154887347
- type: precision_micro
value: 0.6305418719211823
- type: precision_macro
value: 0.6632750205440888
- type: precision_samples
value: 0.8839822728711617
- type: precision_weighted
value: 0.6628306019545424
- type: recall_micro
value: 0.679646017699115
- type: recall_macro
value: 0.6810192216696281
- type: recall_samples
value: 0.8638809749920862
- type: recall_weighted
value: 0.679646017699115
- type: roc_auc_micro
value: 0.8232934760843503
- type: roc_auc_macro
value: 0.8239776029004718
- type: runtime
value: 228.4627
- type: samples_per_second
value: 4.609
- type: steps_per_second
value: 0.578
- type: epoch
value: 1
Model Card for flan-t5-xl-job-bias-seq2seq-cls
Model Details
Model Description
The model is a multi-label classifier designed to detect various types of bias within job descriptions.
- Developed by: Tristan Everitt and Paul Ryan
- Funded by [optional]: [More Information Needed]
- Shared by [optional]: [More Information Needed]
- Model type: [More Information Needed]
- Language(s) (NLP): en
- License: apache-2.0
- Finetuned from model [optional]: google/flan-t5-xl
Model Sources [optional]
- Repository: https://gitlab.computing.dcu.ie/everitt2/2024-mcm-everitt-ryan
- Paper [optional]: [More Information Needed]
- Demo [optional]: [More Information Needed]
Uses
Direct Use
```python
from transformers import pipeline
pipe = pipeline("text-classification", model="2024-mcm-everitt-ryan/flan-t5-xl-job-bias-seq2seq-cls", return_all_scores=True)
results = pipe("Join our dynamic and fast-paced team as a Junior Marketing Specialist. We seek a tech-savvy and energetic individual who thrives in a vibrant environment. Ideal candidates are digital natives with a fresh perspective, ready to adapt quickly to new trends. You should have recent experience in social media strategies and a strong understanding of current digital marketing tools. We're looking for someone with a youthful mindset, eager to bring innovative ideas to our young and ambitious team. If you're a recent graduate or early in your career, this opportunity is perfect for you!")
print(results)
```
>> [[
{'label': 'age', 'score': 0.9883460402488708},
{'label': 'disability', 'score': 0.00787709467113018},
{'label': 'feminine', 'score': 0.007224376779049635},
{'label': 'general', 'score': 0.09967829287052155},
{'label': 'masculine', 'score': 0.0035264550242573023},
{'label': 'racial', 'score': 0.014618005603551865},
{'label': 'sexuality', 'score': 0.005568435415625572}
]]
Classification Report:
precision recall f1-score support
age 0.72 0.57 0.63 81
sexuality 0.84 0.79 0.82 81
disability 0.70 0.60 0.65 81
masculine 0.64 0.62 0.63 81
feminine 0.84 0.89 0.86 81
general 0.28 0.44 0.34 82
racial 0.62 0.86 0.72 78
micro avg 0.63 0.68 0.65 565
macro avg 0.66 0.68 0.66 565
weighted avg 0.66 0.68 0.66 565
samples avg 0.31 0.35 0.32 565
Downstream Use [optional]
[More Information Needed]
Out-of-Scope Use
[More Information Needed]
Bias, Risks, and Limitations
[More Information Needed]
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
Training Details
Training Data
[More Information Needed]
Training Procedure
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
- Training regime: [More Information Needed]
Speeds, Sizes, Times [optional]
[More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: x86_64
- Hours used: N/A
- Cloud Provider: N/A
- Compute Region: N/A
- Carbon Emitted: N/A
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
Linux 6.5.0-35-generic x86_64
Hardware
[More Information Needed]
Software
Python 3.10.12
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
See developers
Model Card Contact
See developers