Link
Search

NER Models

Table of contents

System Performance on NER Corpora

In the table below you can find the performance of Stanza’s pretrained NER models. All numbers reported are micro-averaged F1 scores. We used canonical train/dev/test splits for all datasets except for the WikiNER datasets, for which we used random splits.

The Ukrainian model and its score was provided by gawy. The Armenian model was provided by ShakeHakobyan. The Polish model was provided by Karol Saputa

LanguagelcodeCorpusTypesF1Def?SinceDoc
AfrikaansafNCHLT480.08 
ArmenianhyARMTDP1887.961.5.0
ArabicarAQMAR474.3 
BulgarianbgBSNLP 2019583.211.2.1
ChinesezhOntoNotes1879.2 
DanishdaDDT480.951.4.0
DutchnlCoNLL02489.2 
DutchnlWikiNER494.8 
EnglishenCoNLL03492.1 
EnglishenOntoNotes1888.8 
FinnishfiTurku687.041.2.1
FrenchfrWikiNER492.9 
GermandeCoNLL03481.9 
GermandeGermEval2014485.2 
HebrewheIAHLT1283.9soon!
HungarianhuCombined4-1.2.1
ItalianitFBK387.921.2.3
JapanesejaGSD2281.011.4.0
KazakhkkkazNERD2594.941.4.1
MarathimrL3Cube684.191.4.1
MyanmarmyUCSY795.861.4.0
Norwegian‑BokmaalnbNorne884.791.4.0
Norwegian‑NynorsknnNorne880.161.4.0
PersianfaArman680.071.4.0
PolishplNKJP688.731.4.1
RussianruWikiNER492.9 
SindhisdSiNER1184.741.5.0
SpanishesCoNLL02488.1 
SpanishesAnCora488.6 
SwedishsvSUC3 (shuffled)885.661.4.0
SwedishsvSUC3 (licensed)882.541.4.0
ThaithLST201079.651.4.1
TurkishtrStarlang581.651.4.0
Ukrainianuklanguk486.05 
VietnameseviVLSP482.441.2.1

Notes on NER Corpora

We have provided links to all NER datasets used to train the released models on our available NER models page. Here we provide notes on how to find several of these corpora:

  • Afrikaans: The Afrikaans data is part of the NCHLT corpus of South African languages. Van Huyssteen, G.B., Puttkammer, M.J., Trollip, E.B., Liversage, J.C., Eiselen, R. 2016. NCHLT Afrikaans Named Entity Annotated Corpus. 1.0.

  • Bulgarian: The Bulgarian BSNLP 2019 data is available from the shared task page. You can also find their dataset description paper.

  • Finnish: The Turku dataset used for Finnish NER training can be found on the Turku NLP website, and they also provide a Turku NER dataset description paper.

  • Hebrew: The IAHLT corpus has labels on the knesset portion of its UD dataset. The labels are actually to be released as part of UD 2.15, but as of July 2024, they are available [in the github]. Zeldes, Amir, Nick Howell, Noam Ordan and Yifat Ben Moshe (2022) A Second Wave of UD Hebrew Treebanking and Cross-Domain Parsing. In: Proceedings of EMNLP 2022. Abu Dhabi, UAE, 4331-4344,

  • Hungarian: The dataset used for training our Hungarian NER system is a combination of 3 separate datasets: Business, Criminal, and NYTK. Two of these datasets can be found from this Szeged page, and the third can be found in this NYTK-NerKor github repo. A dataset description paper can also be found here.

  • Italian: The Italian FBK dataset was licensed to us from FBK. Paccosi T. and Palmero Aprosio A. KIND: an Italian Multi-Domain Dataset for Named Entity Recognition. LREC 2022

  • Marathi: The L3cube-MahaNER dataset was used for Marathi NER training. The original dataset was curated as a part of the MahaNLP initiative by L3Cube Pune. The MahaNER along with other Marathi resources is shared on the Marathi NLP page and described in their dataset description paper.

  • Myanmar: The Myanmar dataset is by special request from UCSY.

  • Sindhi: The Sindhi dataset SiNER is available on github

  • Swedish: The SUC3 dataset has two versions, one with the entries shuffled and another using the original ordering of the data. We make the shuffled version the default in order to expand the coverage of the model.

  • Vietnamese: The Vietnamese VLSP dataset is available by request from VLSP.

Tag category notes

  • For packages with 4 named entity types, supported types include PER (Person), LOC (Location), ORG (Organization) and MISC (Miscellaneous)
    • The Vietnamese VLSP model spells out the entire tag, though: PERSON, LOCATION, ORGANIZATION, MISCELLANEOUS.
  • For packages with 18 named entity types, supported types include PERSON, NORP (Nationalities/religious/political group), FAC (Facility), ORG (Organization), GPE (Countries/cities/states), LOC (Location), PRODUCT,EVENT, WORK_OF_ART, LAW, LANGUAGE, DATE, TIME, PERCENT, MONEY, QUANTITY, ORDINAL and CARDINAL (details can be found on page 21 of this OntoNotes documentation).
  • The BSNLP19 dataset(s) use EVENT, LOCATION, ORGANIZATION, PERSON, PRODUCT.
  • The Hebrew IAHLT dataset uses ANG (Language), DUC (Product), EVE (Event), FAC (Facility), GPE, LOC, ORG, PER, TIMEX, TTL (Title), WOA (Work of Art), and MISC
  • The Italian FBK dataset uses LOC, ORG, PER
  • The Marathi L3Cube dataset uses ED (Designation), NED (Date), NEL (Location), NEM (Measure), NEO (Organization), NEP (Person), NETI (Time)
  • The Myanmar UCSY dataset uses LOC (Location), NE (Misc), ORG (Organization), PNAME (Person), RACE, TIME, NUM
  • The Japanese GSD dataset uses 22 tags: CARDINAL, DATE, EVENT, FAC, GPE, LANGUAGE, LAW, LOC, MONEY, MOVEMENT, NORP, ORDINAL, ORG, PERCENT, PERSON, PET_NAME, PHONE, PRODUCT, QUANTITY, TIME, TITLE_AFFIX, WORK_OF_ART
  • The Kazakh KazNERD dataset uses 25 tags: ADAGE, ART, CARDINAL, CONTACT, DATE, DISEASE, EVENT, FACILITY, GPE, LANGUAGE, LAW, LOCATION, MISCELLANEOUS, MONEY, NON_HUMAN, NORP, ORDINAL, ORGANISATION, PERCENTAGE, PERSON, POSITION, PRODUCT, PROJECT, QUANTITY, TIME
  • The Norwegian Norne dataset uses 8 tags for both NB and NN: DRV, EVT, GPE, LOC, MISC, ORG, PER, PROD
  • The Persian Arman dataset uses 6 tags: event, fac, loc, org, pers, pro
  • The Polish NKJP dataset uses 6 tags: date, geogName, orgName, persName, placeName, time
  • The Sindhi SiNER dataset uses 11 tags: ART, EVENT, FAC, GPE, LANGUAGE, LOC, NORP, ORG, OTHERS, PERSON, TITLE
  • The Thai LST20 dataset uses 10 tags: Person (PER), Title (TTL), Designator (DES), Organization (ORG), Location (LOC), Brand (BRN), Date and time (DTM), Measurement unit (MEA), Number (NUM), and Terminology (TRM)
  • The Turkish Starlang dataset uses 5 tags: LOCATION, MONEY, ORGANIZATION, PERSON, TIME