LREC-COLING 2024Chinese spelling correction · Retrieval
Error-Robust Retrieval for Chinese Spelling Check
A misspelled character is a poor retrieval query. RERIC combines pronunciation, shape, and surrounding context to find useful training examples without retraining the correction model.
- Result
- SIGHAN15 sentence-level correction F1: RERIC 80.0 vs 78.5 for ECOPO, the strongest baseline, and 77.8 for its unretrained base model, REALISE.
- New vs prior
- kNN-LM (Khandelwal et al., 2019) keys a datastore on context representations and stores single next tokens; RERIC keys on pronunciation, glyph shape, and weighted context, and stores corrected 3-grams for reranking.
- Limitation
- On SIGHAN13 RERIC only matches ECOPO (correction F1 85.2 vs 85.1), and F1 declines once more than about 16 neighbors are retrieved.
The erroneous character should not dominate the search
Chinese Spelling Check (CSC) corrects misspelled characters, which are usually confused with characters of similar sound or shape. In CSC, part of the retrieval query is wrong: a key that encodes the mistaken character can retrieve fluent uses of that character and reinforce the error.
Strong correctors also underuse their training data: in Table 1, REALISE, a multimodal CSC model, leaves 设 (set) uncorrected where 这 (this) is meant, although that error appears in similar contexts in the training set. RERIC turns the training set into a datastore that the corrector consults at inference.
Closest prior work
Multimodal Chinese Spelling Check (CSC) models such as SpellGCN, PLOME, and REALISE build phonetic and glyph information into the corrector during training. Retrieval methods such as kNN-LM and kNN-MT (Khandelwal et al., 2019, 2020) instead attach a datastore of training examples to a trained model and interpolate its predictions at inference. RERIC, which the authors describe as the first retrieval-augmented CSC method, adapts that pattern to queries that may contain the error: keys combine pronunciation, glyph shape, and distance-weighted context, and candidates are reranked by 3-gram overlap.
Much of the gain comes from avoiding unnecessary corrections
80.0 F1
SIGHAN15 sentence-level correction
RERIC versus 78.5 for ECOPO, the strongest baseline, and 77.8 for REALISE, the base model whose representations build the datastore (Table 4).
78.2 F1
Retrieval with ordinary hidden-state keys
Using REALISE's standard token representation as the key leaves SIGHAN15 correction F1 near the 77.8 of no retrieval; RERIC's error-robust keys and 3-gram reranking reach 80.0 (Table 5).
On SIGHAN15, the 2015 test set of the SIGHAN Chinese spelling-check shared tasks, nearly all of the gain over REALISE comes from precision (Table 4): RERIC makes fewer wrong corrections rather than finding more errors. In the case study, REALISE changes a correct 带 (take) to 戴 (wear); RERIC, with many correct uses of 带 in its datastore, does not.
The key design, not retrieval itself, carries the gain: replacing the key's context part with the token's semantic representation falls below no retrieval (Table 5). Retrieval adds about 6% inference time and a 1.3 GB datastore; adding about one million error-free Chinese Wikipedia articles to the datastore raises SIGHAN15 correction F1 from 80.0 to 80.9 without retraining (Table 7).
Table 4 (selected columns), SIGHAN15 test set. Sentence-level detection F1, and correction accuracy, precision, recall and F1; higher is better. REALISE is the base CSC model used to build RERIC's datastore; RERIC adds retrieval at inference without retraining it. Source ↗
| Model | Detection F1 | Correction Accuracy | Correction Precision | Correction Recall | Correction F1 |
|---|---|---|---|---|---|
| FASpell | 63.5 | 73.7 | 66.6 | 59.1 | 62.6 |
| SpellGCN | 77.7 | - | 72.1 | 77.7 | 75.9 |
| PLOME | 79.4 | - | 75.3 | 79.3 | 77.2 |
| ECOPO | 80.0 | 84.2 | 76.1 | 81.2 | 78.5 |
| REALISE | 79.3 | 84.0 | 75.9 | 79.9 | 77.8 |
| RERIC | 81.2 | 85.6 | 79.9 | 80.1 | 80.0 |
Method proposed in the paper
Scope. RERIC is tested only on top of REALISE, on the SIGHAN13–15 test sets converted to Simplified Chinese; retrieval hyperparameters are tuned by grid search on a held-out 10% of the training data. Read the study ↗
Read the abstract
Chinese Spelling Check (CSC) aims to detect and correct error tokens in Chinese contexts, which has a wide range of applications. However, it is confronted with the challenges of insufficient annotated data and the issue that previous methods may actually not fully leverage the existing datasets. In this paper, we introduce our plug-and-play retrieval method with error-robust information for Chinese Spelling Check (RERIC), which can be directly applied to existing CSC models. The datastore for retrieval is built completely based on the training data, with elaborate designs according to the characteristics of CSC. Specifically, we employ multimodal representations that fuse phonetic, morphologic, and contextual information in the calculation of query and key during retrieval to enhance robustness against potential errors. Furthermore, in order to better judge the retrieved candidates, the n-gram surrounding the token to be checked is regarded as the value and utilized for specific reranking. The experiment results on the SIGHAN benchmarks demonstrate that our proposed method achieves substantial improvements over existing work.
Full paper ↗Citation BibTeX
@inproceedings{yin-etal-2024-error,
title = {Error-Robust Retrieval for Chinese Spelling Check},
author = {Yin, Xunjian and Hu, Xinyu and Jiang, Jin and Wan, Xiaojun},
booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
year = {2024},
pages = {6257--6267},
url = {https://aclanthology.org/2024.lrec-main.553/}
}