Link

Model Performance

Table of contents


Here we report the performance of Stanza’s pretrained models on all supported languages. Again, performances of models for tokenization, multi-word token (MWT) expansion, lemmatization, part-of-speech (POS) and morphological features tagging and dependency parsing are reported on the Universal Dependencies (UD) treebanks, while performances of the NER models are reported separately.

System Performance on UD Treebanks

In the table below you can find the performance of Stanza’s pretrained UD models as of version 1.5.1 on UD 2.12.

Older versions of this chart are also available:

Stanza v1.0 on UD 2.5.

Stanza v1.3 on UD 2.8.

All models are trained and tested with the Universal Dependencies v2.12 treebanks. The following scores are reported for models with charlm, when possible, but not with transformers.

Note that all scores reported are from an end-to-end evaluation on the official test sets (from raw text to the full CoNLL-U file), and are generated with the CoNLL 2018 UD shared task official evaluation script. For detailed interpretation of these scores and the evaluation scripts we used, please refer to the CoNLL 2018 UD Shared Task Evaluation page. For details on how we handled treebanks with no training data, please refer to our CoNLL 2018 system description paper.

If the models for a particular dataset are the default for that language, it is marked with a green checkmark: For languages which have a “combined” family of models, that will be the default, and those models are not represented in this table.

License information for models built from the UD data is unclear, but users are encouraged to click on the git links below and check the license for the relevant data if they are unsure of the licenses for the data used to build these models. To the extent that The Trustees of Leland Stanford Junior University have ownership and rights over these language packs, all these Stanza language packs are made available under the Open Data Commons Attribution License v1.0.

LanguageTreebanklcodeDefaultTokensSentencesWordsUPOSXPOSUFeatsAllTagsLemmasUASLASCLASMLASBLEX
​Macro Avg 99.5089.2398.9594.6894.3991.6689.1094.2883.6879.0274.8167.4771.29
AfrikaansAfriBoomsaf99.9999.6599.9998.6095.7798.4195.6698.2989.5486.1480.1076.7177.53
Ancient GreekPROIELgrc100.0052.00100.0097.4297.8292.3091.0697.1882.7779.0272.4763.9470.00
Ancient GreekPerseusgrc 99.9998.9399.9992.4185.1391.1184.8187.8679.4673.9767.9953.9357.20
Ancient HebrewPTNKhbo100.0098.1893.6090.2090.5385.4084.6392.1476.1272.7263.1054.5461.98
ArabicPADTar99.9883.5698.0395.1892.4092.5292.1893.5483.9979.7776.7271.1073.25
ArmenianArmTDPhy99.9299.4699.4883.0599.4866.2264.3091.3574.0461.8052.5229.5849.05
ArmenianBSUThy 99.9198.2399.8695.9499.8689.1688.2994.9984.3177.2071.5660.2967.48
BasqueBDTeu99.98100.0099.9896.0799.9892.9591.1696.3886.6583.2080.8073.4277.71
BelarusianHSEbe98.4388.3998.4396.7296.1092.1291.1594.5985.1682.6879.2872.7075.20
BulgarianBTBbg99.9696.5099.9699.1696.9797.8496.6297.2893.9891.0988.0084.8884.59
BuryatBDTbxr100.00100.00100.0075.16100.0074.5167.9777.7856.2140.5231.5824.8825.84
CatalanAnCoraca99.9799.5799.9598.7095.9398.2795.6799.0693.5591.6687.2484.7286.41
Chinese (Simplified)GSDSimpzh-hans92.1499.1092.1488.8288.6191.4787.9592.0773.4170.6567.3963.9667.35
Chinese (Traditional)GSDzh-hant92.7999.1092.7989.0688.8092.0188.0792.7072.9469.9866.6562.6066.62
Classical ChineseKyotolzh97.8549.5797.8589.1688.1791.3885.8997.6371.3265.8064.5461.9464.43
CopticScriptoriumcop100.0034.0988.6385.7785.6986.3784.7785.9566.3764.4356.3953.0855.19
CroatianSEThr99.9697.0899.9698.2194.8995.2794.6096.6090.2986.9584.1478.1980.25
CzechCACcs 99.82100.0099.5798.5294.5993.6492.9297.4691.0988.7186.7880.2784.87
CzechCLTTcs 99.2398.2399.2397.3888.2588.2287.4295.7883.5379.7176.5266.3274.12
CzechFicTreecs 99.9899.0799.9798.4595.1596.0094.7798.3293.2890.8088.5582.8586.53
CzechPDTcs99.9794.6299.9798.8895.1795.8494.6598.7392.2290.1888.5283.4987.26
DanishDDTda99.9791.8199.9798.3099.9797.8797.1497.8888.6186.6183.8080.0681.27
DutchAlpinonl99.8790.0899.8796.7595.5097.0094.9795.3391.4288.6783.9479.6578.12
DutchLassySmallnl 99.8778.6199.8796.6795.9296.8395.2095.8688.2585.3280.0076.7275.15
EnglishAtisen 100.0087.55100.0099.15100.0098.7498.1999.6594.4592.6190.0887.4989.72
EnglishESLSpoken 100.0092.81100.0098.2898.32100.0097.88100.0093.1290.7389.4387.4589.43
EnglishEWTen 99.1789.1698.9296.2095.7596.3994.6496.6388.9086.7784.1180.2781.54
EnglishGUMen 99.7895.3499.6997.5997.4297.5296.5298.2290.4988.2484.5281.7182.68
EnglishLinESen 99.9688.6399.9697.4296.5797.0494.5298.2688.6785.0481.0276.5779.11
EnglishParTUTen 99.72100.0099.6396.7096.4095.7694.5097.3191.1588.3683.6278.2281.20
ErzyaJRmyv99.6396.9299.0884.8483.2975.8869.5284.7270.0857.3552.1036.0944.27
EstonianEDTet99.9792.8899.9797.1997.9495.6494.3995.9386.6083.8182.2377.0278.05
EstonianEWTet 98.7979.5798.7993.1495.0292.1189.3590.7777.2772.9470.3463.8763.75
FaroeseFarPaHCfo99.6593.9499.6496.5191.5692.9290.8699.6484.3980.0772.2864.4872.28
FinnishFTBfi 100.0090.1399.9896.8896.0997.1295.2997.1090.9988.6386.2482.6084.20
FinnishTDTfi99.7390.7099.7097.5198.0996.1395.3995.9890.3788.3187.1583.0183.42
FrenchGSDfr99.7195.0399.5097.6399.5097.7497.0598.2093.2791.2887.4484.4585.78
FrenchParTUTfr 99.88100.0099.4296.9296.5894.7393.9697.3591.9389.5885.4878.4682.69
FrenchParisStoriesfr 99.7692.8099.5697.0899.5694.9993.4097.6678.1173.9569.6762.7667.87
FrenchRhapsodiefr 99.8199.8299.5297.3199.5296.0895.0697.7485.2280.7274.7369.6372.74
FrenchSequoiafr 99.8887.4699.6498.8099.6497.8797.5298.5093.2291.5787.7484.8886.36
GalicianCTGgl99.9199.3099.3797.1196.9099.1996.5398.0184.5881.9575.7269.7274.39
GalicianTreeGalgl 99.6888.0298.5393.8991.5193.1190.6494.5277.2771.8664.3858.1460.30
GermanGSDde99.6283.8499.6095.6197.3689.7887.1997.2385.8081.8077.0862.8174.04
GermanHDTde 100.0097.36100.0098.3098.2092.5792.1798.0495.5993.5689.8878.8187.18
GothicPROIELgot100.0040.33100.0096.0696.6290.4388.6396.1079.3472.8767.4960.6465.16
GreekGDTel99.9093.6699.9097.7197.7194.7294.0196.0591.2288.8784.1977.9979.08
GreekGUDel 99.8998.1699.8996.5295.6092.5089.0092.6389.8185.3380.2868.9369.75
HebrewHTBhe 99.98100.0092.2989.9989.9988.6888.0389.7179.3877.1871.9565.3568.81
HebrewIAHLTwikihe 99.6496.7894.6191.6991.6986.8886.1492.5183.2381.1975.5864.8873.14
HindiHDTBhi100.0099.52100.0097.9597.5394.4192.7496.8895.2692.4689.3880.2688.12
HungarianSzegedhu99.8697.2199.8695.8099.8693.5492.7493.8183.5778.6177.0268.1970.84
IcelandicGCis 99.7095.8099.7092.6876.4681.1074.8991.0680.4175.4571.1752.5563.45
IcelandicIcePaHCis99.9093.0799.8496.3692.2790.4285.4895.7886.7982.8577.2764.7873.63
IcelandicModernis 99.2494.4099.2496.6593.9585.8783.0395.3482.7079.4073.7658.2669.99
IndonesianCSUIid 99.6590.7699.2695.9096.1296.6695.5096.3683.2278.5576.0773.2373.39
IndonesianGSDid99.8893.0499.6494.1694.6995.5889.3898.2887.0580.8376.9871.7876.03
IrishIDTga99.7597.2499.7596.0194.5589.4186.5295.3286.8181.5176.3463.3871.94
IrishTwittIrishga 98.5156.9898.5188.5998.5198.5188.5987.9675.8367.6060.0553.4252.16
ItalianISDTit 99.9399.0799.7998.4198.3298.1497.6998.2393.2991.6187.3285.0585.17
ItalianMarkITit 100.0099.1299.6796.2996.3494.2892.3487.0885.5880.6374.4865.3570.41
ItalianParTUTit 99.81100.0099.8298.4298.3798.0497.5197.7193.5391.4786.6284.2884.06
ItalianParlaMintit 99.5995.8399.0196.8395.9695.2893.7495.6485.1981.4574.7668.8870.30
ItalianPoSTWITAit 99.6161.7199.3496.3796.1596.3295.0596.6383.6079.3073.6670.1471.68
ItalianTWITTIROit 99.1857.7898.9194.7094.2293.8291.7392.9179.2173.5565.1559.5859.88
ItalianVITit 99.9495.9399.6197.8297.0097.6196.0698.3590.4087.4082.3378.7380.70
JapaneseGSDLUWja 96.3299.7296.3295.1094.6796.3294.6494.6788.2687.3680.7478.7178.14
JapaneseGSDja97.3799.7297.3796.3895.7297.3695.4596.0290.0189.2984.9983.8683.34
KazakhKTBkk99.42100.0096.8585.5886.1578.7074.5089.2160.5551.3849.4538.4044.48
KoreanGSDko 99.9096.2999.9095.9890.4299.7288.1693.5586.5682.4880.7979.0775.39
KoreanKaistko100.0099.93100.0095.7485.81100.0085.8194.1087.6385.5083.6580.9378.78
KurmanjiMGkmr99.38100.0099.3891.9692.3783.3079.5992.7873.4068.4564.3846.5857.53
KyrgyzKTMUky99.0698.1499.0687.6587.9460.2257.9188.0974.9558.7752.6028.4747.22
LatinITTBla99.9980.6699.9998.7695.7296.6994.6399.0988.8286.8085.7181.4785.31
LatinLLCTla 99.9999.3299.9999.5796.6596.7996.4798.0996.1394.8893.5389.4991.50
LatinPROIELla 99.9843.1099.9896.9597.1591.5890.5996.8677.9873.9968.8961.9667.12
LatinPerseusla 99.9798.9399.9791.4078.0981.8376.3783.5772.0763.1658.2745.1448.05
LatinUDantela 99.6698.2299.3090.0874.5581.6473.0786.9568.0058.8448.7437.4642.52
LatvianLVTBlv99.8198.1599.8196.7089.7294.7389.2996.1288.9185.7783.3776.7180.17
LigurianGLTlij100.00100.0098.0089.1498.0088.2585.5992.6877.6169.1860.3351.2453.72
LithuanianALKSNISlt99.9489.9399.9493.7186.4888.1685.5492.9577.8272.7570.2360.5365.21
LithuanianHSElt 97.8652.7597.8682.3081.1670.9166.8276.7950.2138.2533.7921.9026.25
Maghrebi Arabic FrenchArabiziqaf96.918.4493.7777.4070.7383.7570.2751.0356.5047.8839.8433.2323.73
MalteseMUDTmt99.8585.3599.8595.7295.6199.8595.3599.8583.6278.2270.7667.2070.76
ManxCadhangv99.8799.7497.5593.8297.5595.6392.7692.6788.5584.7981.4877.6976.15
MarathiUFALmr97.9989.3692.8985.7892.8973.5371.5782.8468.8759.5654.7337.0446.09
NaijaNSCpcm99.99100.0099.9997.5599.9998.8497.0399.1492.3989.5688.6086.4388.21
North SamiGiellasme99.7199.6599.7190.7392.7788.0883.6988.5472.2666.6364.1957.2156.46
NorwegianBokmaalnb99.8897.7099.8898.3899.0497.0996.3898.6193.2691.8589.1185.1387.53
Norwegian NynorskNynorsknn99.9795.4899.9798.1398.9196.8795.9397.8791.6389.9486.8482.3384.30
Old Church SlavonicPROIELcu100.0040.27100.0096.1496.4589.6888.1894.6578.8774.6470.0062.0466.83
Old East SlavicBirchbarkorv 99.9326.8599.9383.8999.1267.8865.3473.6960.0250.8240.1827.2329.53
Old East SlavicRNCorv 99.1489.2999.1491.0487.5372.3964.4081.0468.1060.8151.9033.2540.36
Old East SlavicTOROTorv100.0040.67100.0095.1295.1589.0687.2292.7978.6674.0668.8660.6364.19
Old FrenchSRCMFfro99.89100.0099.8996.6496.5697.6495.7299.8590.8087.0184.1180.7284.07
PersianPerDTfa99.9699.7399.4697.1797.1497.3195.3598.5592.9090.8088.8785.4887.83
PersianSerajifa 100.0099.2599.7297.6997.6397.7097.3198.1890.9887.9685.4383.7683.76
PolishLFGpl 99.9299.9199.9298.9794.7395.8094.2797.2496.7195.2493.7988.6790.38
PolishPDBpl99.8598.1999.8398.6394.5994.7893.8997.4793.7391.7389.8283.5086.95
PomakPhilotisqpm100.0095.05100.0098.76100.0095.3795.1397.9087.8882.8777.2070.2974.96
PortugueseBosquept99.8593.1099.7697.4599.7696.4495.3398.0490.5987.7582.6677.0780.59
PortugueseCINTILpt 100.0085.3499.9397.1995.7495.4892.9498.5684.0880.0974.7568.9073.19
PortuguesePetroGoldpt 99.7195.8399.6698.5999.6698.5197.9499.0294.4893.2890.2688.1089.57
RomanianNonstandardro 98.9697.5398.9695.6090.8589.7188.3294.9688.9784.8880.5167.1776.58
RomanianRRTro99.7395.7299.7397.5096.8097.1096.7497.4290.7086.8682.9979.9080.83
RomanianSiMoNERoro 99.77100.0099.7797.9897.5597.1996.9998.6993.4591.4488.5084.3687.34
RussianGSDru 99.5897.3399.5897.6397.4193.6392.7695.5189.5685.8983.9177.0079.48
RussianSynTagRusru99.7498.4399.7498.0699.7492.7892.4497.6593.2690.9089.4780.3687.17
RussianTaigaru 99.2189.6799.2196.3499.2193.2892.2094.4182.3678.5375.9169.2071.30
SanskritVedicsa100.0032.93100.0089.19100.0080.7076.9188.9262.5250.6449.9043.2746.84
Scottish GaelicARCOSGgd99.6564.5498.8694.6290.2491.9189.4295.8182.7377.8172.8666.5370.20
SerbianSETsr99.9998.4699.9998.4494.1694.3193.7696.3091.4488.3685.9278.8281.74
SlovakSNKsk99.9684.3199.9696.3488.1990.9587.3594.0889.1386.5884.4374.7278.62
SlovenianSSJsl99.9399.2699.9398.4195.8896.0995.3797.6992.4590.8088.2883.8085.67
SlovenianSSTsl 99.9726.6299.9794.1587.9288.1785.8294.3261.7354.7549.2442.4146.63
SpanishAnCoraes99.9798.4999.9698.9696.1098.5495.6599.2493.0991.3087.8085.4986.94
SpanishGSDes 99.9495.6099.8097.0799.8096.4294.7798.4292.0189.5784.5676.8682.66
SwedishLinESsv 99.9487.9199.9497.3495.0790.6887.9597.1989.0285.2582.2869.8279.13
SwedishTalbankensv99.9797.4999.9798.2897.1397.2996.2497.9290.3287.8585.9181.8883.42
TamilTTBta99.2798.7489.6178.5177.5379.6373.0382.5558.3950.9748.8241.4043.87
TeluguMTGte99.7998.6299.7993.9793.9799.1093.9799.7990.0981.6478.1374.5078.13
TurkishAtistr 100.0086.93100.0098.77100.0098.5798.3498.9889.0386.9186.3184.6085.56
TurkishBOUNtr 99.5890.0896.3289.9285.7481.6673.0390.3771.5365.1763.7948.4958.96
TurkishFrameNettr 100.00100.00100.0095.71100.0091.8991.1495.1692.5083.7881.3771.6976.07
TurkishIMSTtr99.8698.1697.7593.4493.2791.0689.0795.5573.3964.0760.9654.7059.51
TurkishKenettr 99.9998.4599.9993.6099.9991.2790.3593.2282.9870.5569.6260.8464.00
TurkishPenntr 100.0094.52100.0096.24100.0094.3793.4694.5885.0971.9568.8361.1564.27
TurkishTourismtr 100.00100.00100.0099.08100.0094.9094.6498.9096.9991.3188.7481.6087.58
Turkish GermanSAGTqtd99.9599.4498.9985.7898.9975.2869.8390.2465.1152.7447.2634.1343.41
UkrainianIUuk99.7997.2199.7697.5293.0193.1192.0796.7288.6085.8782.7775.6379.55
Upper SorbianUFALhsb100.0086.36100.0095.87100.0078.4877.1793.7079.5774.1365.7145.3660.36
UrduUDTBur100.0098.88100.0094.7892.9884.0680.4995.5887.9682.7677.1359.1274.79
UyghurUDTug99.4284.6999.4289.2691.1987.5880.0695.3074.8063.4258.9648.0155.81
VietnameseVTBvi88.0299.2588.0281.4880.6188.0280.5681.6658.8251.4946.6042.5842.47
WelshCCGcy99.9097.7999.6895.8694.6090.1187.8394.8586.3280.2574.8562.8169.32
Western ArmenianArmTDPhyw99.9599.5699.9196.4299.9192.7092.0496.7087.7382.8377.3368.8974.38
WolofWTBwo99.9594.3399.3594.3694.1693.1991.3894.6383.2877.3271.2264.8967.65