~/divyam/stories  lexigrade-text-readability-classifier

the lab  ·  models and research

LexiGrade: A Text Readability Classifier

A calibrated NLP pipeline that combines LightGBM, sentence-transformer embeddings, and classical readability indices to classify text by reading level.

when
April 2026
role
ML Engineer
stack
LightGBMall-MiniLM-L6-v2PythonScikit-learnNLP

Overview

LexiGrade predicts the reading level of a passage by combining semantic and linguistic signals. Rather than depending only on formulas like Flesch-Kincaid, it fuses sentence-transformer embeddings with classical readability features.

The project was designed for skewed real-world data, where expert-level text is underrepresented and naive classifiers tend to underperform on the hardest class.

Problem

Classical readability indices are useful but shallow: they cannot fully capture semantic complexity. Transformer embeddings add semantic signal, but on small expert-class samples they can overfit. The challenge was to combine both without letting label imbalance dominate the model.

Solution

I trained a LightGBM regressor on a combined feature vector: all-MiniLM-L6-v2 embeddings plus handcrafted linguistic features. A stratified split and 2.5x expert-class weighting helped the model pay attention to the rare upper-grade examples.

A calibrated fusion layer blends the ML score with FRE, FKG, FOG, SMOG, CLI, and ARI. An ML trust override catches short high-FRE sentences that classical formulas incorrectly mark as easy.

Highlights

  • LightGBM over transformer embeddings and handcrafted linguistic features
  • Stratified split with 2.5x expert-class sample weight boosting
  • Fusion layer across FRE, FKG, FOG, SMOG, CLI, and ARI readability indices
  • ML trust override for high-confidence expert-level edge cases

Outcome

The calibrated fusion approach improved expert-class recall compared with either the ML model or classical indices alone, showing that semantic embeddings and readability formulas capture complementary signals.