~/divyam/stories smart-price-prediction
the lab · models and research
Smart Price Prediction
An Amazon ML Challenge 2025 project: NeoBERT fine-tuned to predict product prices from noisy catalogue text, with a metric-matched SMAPE loss and a CLIP image variant.
- when
- October 2025
- role
- AI/ML Engineer
- stack
- PythonPyTorchTransformersNeoBERTCLIPMLflow

Overview
This project was built for the Amazon ML Challenge 2025, where the task was to predict a product's price from its catalogue listing. The training set held 75,000 products, and their descriptions were noisy and inconsistently formatted, which makes ordinary regression on raw prices unstable.
The approach fine-tunes NeoBERT with a small regression head, trains on log-scaled prices with a loss that matches the competition's SMAPE metric, and adds a multimodal variant that also reads the product photo.
Problem
Raw price targets amplify outliers, and product descriptions are often too sparse or inconsistently formatted to provide a clean pricing signal. The model needed to learn useful patterns without being dominated by extreme prices.
Solution
I fine-tuned NeoBERT with an MLP regression head on its [CLS] embedding. Prices were log-transformed before training, and a custom SMAPE loss, computed on the recovered prices, kept training aligned with how the challenge was scored.
A second training script fuses text with product photos: image features from a frozen CLIP ViT-B/32 meet the NeoBERT text features through 8-head cross-attention, with image augmentation during training.
Training used mixed precision (FP16), linear warmup, gradient clipping, weight decay and early stopping, with every run tracked in MLflow. An ensembling script combines model predictions four ways: weighted, geometric, median and rank averaging.
Highlights
- NeoBERT with an MLP regression head, trained on 75,000 Amazon products
- Log-scaled targets and a custom SMAPE loss that matches the challenge metric
- Multimodal variant: frozen CLIP ViT-B/32 image features fused through 8-head cross-attention
- Mixed precision, warmup, gradient clipping and early stopping
- MLflow experiment tracking and a four-method ensembling script
Outcome
The result is a working text-only pricing model and a complete, tracked pipeline around it, from data loading through MLflow-logged experiments to ensembled predictions, with a multimodal variant ready to compare against it.