CNN-based socioeconomic level (NSE) predictor built on credit bureau data. Achieved +30% accuracy improvement over the previous logistic regression baseline. Migrated to Hadoop (Impala/Hive) for batch scoring at scale across multiple geographies.
Socioeconomic level (NSE) is a critical signal for financial product targeting, credit limit assignment, and marketing segmentation — but it's rarely available directly. The challenge: infer NSE reliably from credit bureau variables alone, without self-reported income or demographic surveys.
This model was built at Equifax as part of a portfolio of generic scores deployed to clients across multiple geographies (Argentina, Paraguay, Colombia).
| Version | Model | Accuracy | Delta |
|---|---|---|---|
| NSE 2.0 (baseline) | Logistic Regression | baseline | — |
| NSE 3.0 | 1D CNN | +30% | +30% |
Additional milestone: full migration of scoring pipeline to Hadoop (Hive + Impala) for scalable batch inference.
Standard tree models treat features independently. A 1D CNN applied over ordered feature groups captures local interactions between adjacent variables — for example, the relationship between utilization rate, payment behavior, and delinquency flags within a credit product group.
Feature groups (treated as channels):
[revolving_credit_vars] → Conv1D → MaxPool
[installment_loan_vars] → Conv1D → MaxPool
[delinquency_history] → Conv1D → MaxPool
[bureau_summary_stats] → Conv1D → MaxPool
└────────────────── Flatten → Dense → NSE class
This structure lets the model learn feature-group-level patterns before combining them globally — something neither logistic regression nor standard MLPs do efficiently.
┌─────────────────────────────────────────────────────────────┐
│ DATA LAYER │
│ Hadoop / Hive — credit bureau snapshots │
│ Impala — batch query for feature extraction │
│ Monthly refresh cycle │
└─────────────────────┬───────────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────────┐
│ FEATURE ENGINEERING │
│ 4 feature groups × N variables per group │
│ Normalized, missing-value imputed, outlier-clipped │
│ Reshaped as (batch, n_groups, vars_per_group) tensor │
└─────────────────────┬───────────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────────┐
│ CNN CLASSIFIER │
│ │
│ Input: (batch, 4, vars_per_group) │
│ Conv1D(32, kernel=2) → BatchNorm → ReLU → MaxPool │
│ Conv1D(64, kernel=2) → BatchNorm → ReLU → MaxPool │
│ Flatten → Dense(128) → Dropout(0.3) → Dense(n_classes) │
│ Output: NSE class probabilities (softmax) │
└─────────────────────┬───────────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────────┐
│ SCORING & DEPLOYMENT │
│ Batch scoring via Hadoop / Impala │
│ Output: NSE label + probability per customer │
│ Monitoring: accuracy drift + class distribution shift │
└─────────────────────────────────────────────────────────────┘
| Layer | Technology |
|---|---|
| Model | TensorFlow / Keras (1D CNN) |
| Baseline comparison | scikit-learn (LR, RF, XGBoost) |
| Feature engineering | pandas, numpy |
| Big Data pipeline | Hadoop, Hive, Impala, PySpark |
| Monitoring | Class distribution shift, PSI |
socioeconomic-score-cnn/
├── data/
│ └── sample/ # Synthetic bureau data (public schema)
├── notebooks/
│ ├── 01_EDA_bureau_features.ipynb
│ ├── 02_feature_grouping.ipynb
│ ├── 03_baseline_models.ipynb
│ ├── 04_cnn_classifier.ipynb
│ └── 05_model_comparison.ipynb
├── src/
│ ├── features/
│ │ ├── bureau_features.py # Feature extraction from bureau variables
│ │ └── feature_groups.py # Group assignment + tensor reshaping
│ ├── models/
│ │ ├── nse_cnn.py # CNN classifier
│ │ ├── baselines.py # LR, RF, XGBoost baselines
│ │ └── calibration.py # Probability calibration
│ ├── pipeline/
│ │ ├── hive_extractor.py # Hive query → pandas
│ │ └── impala_batch_score.py # Batch scoring via Impala
│ └── evaluation/
│ └── metrics.py
├── configs/
│ └── model_config.yaml
├── requirements.txt
└── README.md
Credit bureau variables are organized into 4 semantic groups — each treated as a "channel" for the CNN:
FEATURE_GROUPS = {
"revolving": [
"revolving_utilization", "revolving_balance",
"revolving_limit", "revolving_payment_ratio",
"revolving_account_count", "revolving_delinquent_count",
],
"installment": [
"installment_balance", "installment_monthly_payment",
"installment_remaining_term", "installment_account_count",
"installment_delinquent_count", "installment_paid_ratio",
],
"delinquency": [
"ever_30dpd", "ever_60dpd", "ever_90dpd",
"max_dpd_12m", "months_since_last_delinquency",
"delinquency_trend_6m",
],
"bureau_summary": [
"total_accounts", "open_accounts", "account_age_months",
"inquiries_6m", "bureau_score_raw", "time_on_bureau",
],
}import tensorflow as tf
class NSEClassifierCNN(tf.keras.Model):
"""
1D CNN for socioeconomic level classification from grouped credit bureau features.
Input shape: (batch, n_groups, vars_per_group)
Output: NSE class probabilities (A, B, C1, C2, C3, D, E)
"""
def __init__(self, n_groups: int, vars_per_group: int, n_classes: int = 7):
super().__init__()
self.conv1 = tf.keras.layers.Conv1D(32, kernel_size=2, padding="same", activation="relu")
self.bn1 = tf.keras.layers.BatchNormalization()
self.pool1 = tf.keras.layers.MaxPooling1D(pool_size=2, padding="same")
self.conv2 = tf.keras.layers.Conv1D(64, kernel_size=2, padding="same", activation="relu")
self.bn2 = tf.keras.layers.BatchNormalization()
self.pool2 = tf.keras.layers.MaxPooling1D(pool_size=2, padding="same")
self.flatten = tf.keras.layers.Flatten()
self.dense1 = tf.keras.layers.Dense(128, activation="relu")
self.dropout = tf.keras.layers.Dropout(0.3)
self.output_layer = tf.keras.layers.Dense(n_classes, activation="softmax")
def call(self, x, training=False):
x = self.pool1(self.bn1(self.conv1(x), training=training))
x = self.pool2(self.bn2(self.conv2(x), training=training))
x = self.dropout(self.dense1(self.flatten(x)), training=training)
return self.output_layer(x)The scoring pipeline was migrated from SAS to Hadoop (Hive + Impala) to handle increasing data volume and enable monthly batch scoring across multiple client databases.
from impala.dbapi import connect
def batch_score_impala(
model,
connection_params: dict,
query: str,
output_table: str,
batch_size: int = 10_000,
) -> None:
conn = connect(**connection_params)
cursor = conn.cursor()
cursor.execute(query)
while True:
rows = cursor.fetchmany(batch_size)
if not rows:
break
X = preprocess_batch(rows)
scores = model.predict(X)
write_scores_to_impala(cursor, output_table, scores)
conn.close()| Model | Accuracy | Macro F1 | Notes |
|---|---|---|---|
| Logistic Regression (v2.0) | baseline | baseline | Previous production model |
| Random Forest | +12% | +10% | Better but slow to score |
| XGBoost | +21% | +18% | Good but no group structure |
| 1D CNN (v3.0) | +30% | +27% | Production model |
Public reference: Give Me Some Credit — credit bureau variables
Synthetic NSE labels generated via income-bracket proxy rules for demonstration.
Production model trained on proprietary Equifax credit bureau data.
git clone https://github.com/lbruno086/socioeconomic-score-cnn
cd socioeconomic-score-cnn
pip install -r requirements.txt
# Generate synthetic data
python src/data/generate_synthetic.py
# Train baselines
python src/models/baselines.py
# Train CNN
python src/models/nse_cnn.py --epochs 30 --n_classes 7
# Compare models
jupyter notebook notebooks/05_model_comparison.ipynb