Building a Per-Payer Claim Denial Predictor from 835 Remittances

Overturning a denied claim costs far more than never getting denied in the first place. That makes pre-submission denial prediction one of the most practical pieces of AI medical billing you can build — and, notably, it is mostly classic supervised learning, not LLM magic.

This post walks through how we approach it at TechCirkle: where the labels come from, which features matter, why one global model is usually the wrong call, and how to turn a probability into a flag a billing specialist can act on.

The data you already have

Every provider organisation receives the raw material for this model already: 835 electronic remittance advice files. Each 835 tells you, claim line by claim line, what the payer paid, what it adjusted, and why — via:

• CARC (Claim Adjustment Reason Codes) — the primary reason for an adjustment or denial.
• RARC (Remittance Advice Remark Codes) — supplementary detail explaining the CARC.
• Group codes (CO, PR, OA, PI…) — who is responsible for the adjustment.

Join those back to the 837 claims you submitted and you have a labelled dataset: claim features in, outcome out.

The hard part is not the algorithm. It is the join. In practice we see remittances that cannot be matched to the original claim, payer names spelled five different ways, and denial codes recorded inconsistently across systems. Models trained on that learn noise. Budget real time for a normalised claims layer before you write any training code.

Defining the label

Start simple. A binary target works well for a first model:

-- One row per submitted claim line, labelled from the matched 835
SELECT
  c.claim_id,
  c.line_no,
  c.payer_id,
  c.plan_id,
  c.cpt_code,
  c.modifiers,
  c.icd10_primary,
  c.rendering_npi,
  c.place_of_service,
  c.prior_auth_number IS NOT NULL      AS has_prior_auth,
  c.submitted_at,
  CASE
    WHEN r.group_code = 'CO'
     AND r.carc IN ('16','50','96','97','197','4','11')  -- tune to your mix
    THEN 1 ELSE 0
  END                                   AS denied,
  r.carc,
  r.rarc
FROM claims_normalised c
JOIN remits_835 r
  ON r.claim_id = c.claim_id
 AND r.line_no  = c.line_no
WHERE c.submitted_at >= NOW() - INTERVAL '18 months';

The CARC list above is illustrative — pick the codes that represent preventable denials in your data (missing information, medical necessity, authorisation, bundling) and exclude pure patient-responsibility adjustments. Keep carc and rarc in the table even though they are not features; you will use them later to generate human-readable reasons and to train a secondary "which category" model.

Features that actually carry signal

From projects we have delivered, the features that consistently matter:

• Payer × plan × CPT interactions — the single most predictive family.
• Modifier combinations — a payer that started rejecting a specific combination is exactly what a static rules engine misses.
• Authorisation presence vs. whether the CPT/plan pair historically required one.
• Rendering provider documentation patterns — some providers are routinely insufficient on certain procedures.
• Diagnosis–procedure pairing frequency — rare pairs deny more.
• Recency-weighted historical denial rate for the same payer/CPT/modifier bucket over the last 30/90 days.
• Time since last policy change if you track payer bulletins.

Avoid anything that leaks the outcome: adjudication dates, paid amounts, or any field populated after submission.

Why per-payer models

Payer behaviour differs too much for a single global model to serve everyone well. One payer's medical-necessity logic has nothing to do with another's bundling rules, and a global model tends to learn the behaviour of your biggest payer and apply it everywhere.

Our default: one model per payer where volume allows, with a pooled fallback model (payer as a feature) for the long tail of low-volume plans.

import lightgbm as lgb
import pandas as pd

MIN_ROWS = 20_000  # below this, fall back to the pooled model

def train_models(df: pd.DataFrame, features: list[str]) -> dict:
    models = {}
    for payer, grp in df.groupby("payer_id"):
        if len(grp) < MIN_ROWS:
            continue
        train = grp[grp.submitted_at < grp.submitted_at.quantile(0.8)]
        valid = grp[grp.submitted_at >= grp.submitted_at.quantile(0.8)]
        m = lgb.LGBMClassifier(n_estimators=400, learning_rate=0.05,
                               num_leaves=63, class_weight="balanced")
        m.fit(train[features], train.denied,
              eval_set=[(valid[features], valid.denied)],
              categorical_feature=["cpt_code", "plan_id", "modifiers"])
        models[payer] = m
    pooled = lgb.LGBMClassifier(n_estimators=400, learning_rate=0.05)
    pooled.fit(df[features + ["payer_id"]], df.denied,
               categorical_feature=["cpt_code", "plan_id", "modifiers", "payer_id"])
    models["__pooled__"] = pooled
    return models

Note the time-based split. A random split will flatter you, because it lets the model see the future of payer behaviour.

Retraining on a schedule

Payer policies shift, often quarterly, and a model trained on last year's denials degrades quietly — nobody notices until denial rates climb. Build retraining in from day one:

• Retrain on a fixed cadence (monthly is a reasonable start) on a rolling window.
• Monitor per-payer calibration and precision at your review threshold, not just AUC.
• Alert on sudden spikes in a CARC for a payer/CPT bucket — often the earliest sign of an unannounced policy change.
• Version every model and log which version scored every claim, so any decision can be reconstructed during an audit.

Explainable flags, not scores

"Risk: 0.83" is useless to a billing specialist. "Missing prior authorisation number for this CPT under this plan" is actionable.

We generate reasons in two layers:

1. Per-claim attributions (for example SHAP values) identify which features pushed the score up.
2. A mapping layer translates the top contributing features into plain-language reasons, using the CARC/RARC categories historically associated with that payer/CPT bucket.

The flagged claim then lands in a work queue inside the tool your team already uses, with the reason attached. A flag nobody owns is worse than no flag — it trains the team to ignore the system.

Measure the loop, not the model

AUC is a development metric. The business metrics are first-pass acceptance rate, denial rate by category and days in accounts receivable. Roll out to a subset of payers or teams first so you have a comparison group, and compare against a pre-launch baseline.

If you want the bigger picture — where denial prediction sits alongside coding assistance, appeal agents and HIPAA design — read our full guide on AI medical billing. And if you are building this in-house and want a hand with the data pipeline or monitoring, our AI development services team does exactly this kind of regulated-domain ML work.

Frequently Asked Questions

Do I need an LLM to predict claim denials?

No. Denial prediction is a tabular classification problem, and gradient-boosted trees on well-engineered claim features are usually the right tool. LLMs are more useful downstream, for drafting appeals once a denial has happened.

Where do the training labels come from?

From your own 835 remittances, joined to the 837 claims you submitted. The CARC, RARC and group codes on each line tell you whether and why a claim was denied or adjusted.

How much history should the training window cover?

Enough to capture each payer's current behaviour — often 12 to 18 months — but weight recent data more heavily, since older denials may reflect rules the payer has since changed.

What if a payer has too few claims for its own model?

Use a pooled model with payer as a categorical feature for low-volume plans, and graduate a payer to its own model once it crosses your volume threshold.

How do I keep PHI safe during model development?

Train only on the fields the model needs, strip direct identifiers where possible, restrict access by role, and log model versions and inputs so every score is auditable.

How do I know the model is still working six months later?

Track per-payer calibration, precision at your review threshold and CARC spikes continuously, and retrain on a fixed cadence. A silent drop in precision is usually the first sign a payer changed a policy.