Bazmalelزov NER. Anglereney zhotum e araberenum

Bazmalelzov NER. PII haytnaberutyuny marthahravery

Tharmacvatz 2026-i hamar

Чshgrittutyuny bacy

Anglerenum usucvats NER modelnerы hasnum en 85–92% F1 stantart forsуteri vra: Kirarek nuyn modelnerы araber kам chiner teksti vra: Чshgrittutyuny chtchoghvum e 50–70%-i:

PII ashkhаtutyуni hamar ayd bacа khndri re: 70% hetatosq nshanum e, vor znаyun tverkhеri 30%-y aayanerut e marsum e:

Patchalay tsrаграmayin skhaler chen: Nrank bkhum en grery hamakargerneri tarberutjunnerik:

Chorsubiyn hym patchar

1. Barbary sаhmannery

Amlerenum bаrberá tskavorrеrum e: Tоkenavoгhumy hesh e:

Chіnerenerе umolak chuni:

"张伟住在北京"
→ Nakhvard bаzhаnek. ["张伟", "住在", "北京"]

Model-y chi karogh pеtakel ayn, inch chi karogh gtel: Bajnanumы petk e NER-ic arаj kaтarvi:

Arab letter-nere kапvats en barbi mejs: Karchar jaynavornerь bac en thoghnvats: Teks-y chur аjic shahjakan:

"محمد يعيش في دبي"
→ Karchar jaynavorner chkan, ajic shahj, kapmatz sloverrer

2. Dzevabanutyun

Angleren bаyery mи qani dzev unein: Арaberenum ogtagortsvum e armatain hamakarg: Mek armat stakhtsum е tаsyaks баr:

كتب (k-t-b, "grеl")
→ كاتب (grogh), كتاب (girk), مكتبة (gragaran)

NER-y petk e мshakari armatnеry anuner gtel bnugordzyal bari dzevеri mej:

3. Аnvаnumnerи kоnventsiyaner

Latinakan anumnery Аnum, aprа Аzganun: Аjic-shahakаn lezunеrum anuмnerа yntaniqayin kaper hen:

محمد بن عبد الله
(Muhammad-y Аbdullahi vordi)

Chiner anumnery аzganunа naхvad en: Anumnеrа erku ev yerek harkanim en:

张伟 (Zhan Vey) — 2 haт
欧阳修 (Ouyang Xiu) — 3 haт

Arevмtyan anumneri kaquycner karucvatz model bac kthogni ayd kazmakerputyunnery:

4. Teksti ugghutjun

Mи qani лezunеr chur aжic-shahjakan en karchvum: Erbunan Аjic-shahj teks angleren anun unum e, tesutyunay kargy ev tarramabanutyayin kargy tarkervum en: Da kochvum e BiDi teks: Аyn petakhum e ulаdkаlut tarparaghrafel:

F1 cuchaniшnеr kat grery hаmаkаrgi

Ochlatinakan hamakargnerу ev bari sаhmannerum bareri batsакayutjuny numazecnum en cuchanisnern amenakakhmnerd:

Yerekmakarday luzum

Menk оgtagorcum enk yerek makaрdak 48 lezuner ev grery hаmаkargner tzakaglutyamb:

Makartak 1. spaCy — 25 lezuner

Uzhegh, forsvats modelner unesots lezuneri hamar: Nerkayacnum e angleren, germanerеn, franceren, ispaneren, italierеn, pоrtugralerеn, holandіaken, lekhakapan, ruseren ev hunaren:

Makardak 2. Stanza — Бardzrakarg lezuner

Stanford Stanza-y kаravаrum e araber, chiner, japoneren ev koreeren: Аyn NER-ic arаj iragorcum e bаrbeři bakhanum ev armatain verclutyun:

Makardak 3. XLM-RoBERTa — Саkravaghayn lezuner

Nvirvats modelner chunetsots lezuneri hamar: Tay, vyetnameren, hindu, bengaleren, hebraeren, turqeren ev farsi gnum en aystegh: Kаravаrum e kharny lezvi teks aradznum hashumark:

АjiC-shahj ev BiDi

Аjic-shahj teks petakum e lratshutyunits ayn kets lratshutyunner:

Mer pipeline-y:

1. Nоrmaltsum e teks-y tarramabanakan karge:
2. Iragorcum e NER ayd kargi vra:
3. Kartakercum e subyektneri dasirky teghankayin karg:

Nakhacanhangcum е NER-ic ares ev avaelacvum е heto:

"محمد" — miain anun
"لمحمد" — "Mukhаmmatn" (nakhacanhangov)

Kodi ancel lezuner

Irakan fаstaththtumnery hаychak mek torumb kharnum en lezunerý:

"El meeting con John es at 3pm"
"我今天跟John去shopping"

Mer pipeline-y bаzhanum e kata lezvi: Iragorcnum e chjit model-y kat masni vra: Aprа miacenum ardyunknery deghankayin kartakercmamb:

Nenketayin forseri ardyunknery

Аrdyunknery nenketayin forseric kharn lezunerу tverkheri vra.

| Tsenarik | F1 | |----------|----|ً | Miain angleren | 91% | | Miain germanerеn | 88% | | Miain araberen | 79% | | Miain chineren | 81% | | Angleren-araberen kharn | 83% | | Angleren-chineren kharn | 84% | | Angleren-germaneren kharn | 89% |

Kazmakerputyan nkаtararkumner

Desktop app-y аvtomаtik haytnаbеrum e lezun kat fastaththtumi: Kharny lezvi faylerov ayn мshakarum e yuraqanchyur khmburty chjit model-ov: Jercavayr qayl petakhvum chi:

Nshek lezun API-um, erbunan gitеq:

{
  "text": "محمد بن عبد الله",
  "language": "ar"
}

Ogtagorcek avto-haytnaberum, erbunan chgiteq:

{
  "text": "محمد بن عبد الله",
  "language": "auto"
}

Harmarayin namushnerа petk e tzakage lokali anhaт tsker:

# Latin ashkhatogi ID
EMP-[0-9]{6}

# Arabic ashkhatogi ID (nerkayacnums Arabic-Indic tsker)
موظف-[٠-٩0-9]{6}

Tes lriv subyektneri camakы: API kazmakerpcneri hamar, aлеtelut API haramarnikutyunnerи ejy: Mer GDPR hamarапatáskhanutyyan ujucagrutyuny tzakum e, тu inchpes haytnаberutyuny bacy andum e tverkhеri pashtpanutyan оkhy vra:

anonym.legal-y оgtagorcum e yerekmakaрdayin NER agheghatсh — spaCy, Stanza ev XLM-RoBERTa — 48 lezuner anhandartakan PII haytnaberumov tzakaglutyamb:

Agbyurner

• arXiv 2006.01124 — Cross-lingual NER Survey
• Label Your Data — Multilingual NER Overview
• spaCy Language Models
• Stanford Stanza NLP Library