---
language:
 - en
tags:
 - retrained
 - SpanBERT
---


SpanBERT

This is the SpanBERT model from:

Mike Zhang, Kristian Nørgaard Jensen, Sif Dam Sonniks, and Barbara Plank. __SkillSpan: Hard and Soft Skill Extraction from Job Postings__. To appear at the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). 2022.

This model is pre-trained from scratch on the BookCorpus and WikiData. To pre-train from scratch we use the code from Splinter: https://github.com/oriram/splinter. 
On our job posting dataset, we found that our `spanbert-base-cased` model works better than the original.

More information can be found in the paper (which should be released when the NAACL proceedings are online).