SutraAudit

SutraAudit addresses an important AI safety and socio-economic fairness problem: AI-driven credit scoring systems can still produce unfair denials or rejections even when explicit demographic features are removed. The project’s framing around proxy leakage in Indian fintech is strong, especially because alternative credit signals such as UPI transaction patterns, device metadata, rural markers, and informal income footprints may encode socio-economic status in ways that standard input scrubbing cannot fully remove.

One suggestion is to include feature importance analysis. The dataset appears to distinguish between a Formal Baseline group, associated with urban and formal banking footprints, and an Informal Target group, associated with high-frequency small-value UPI transactions, lower-tier device metadata, rural markers, and informal income signals. Since XGBoost binary classifiers can provide feature importance during training, this analysis would help show whether the model is learning realistic lending signals or relying too heavily on proxy traits that separate the baseline and target groups.

The paper should also describe the data generation process more clearly, because the baseline and target groups seem to be built using predefined heuristic assumptions. Clear documentation would make it easier to assess whether the synthetic assumptions themselves create false positives and false negatives across groups. For example, the model may learn a rule such as “lower-tier phone + rural pincode + dialect messages + no salary slip = risky borrower,” even when the applicant’s true repayment behavior does not justify that conclusion. Reporting precision, recall, F1, demographic parity difference, and equal opportunity difference would help evaluate whether these assumptions create uneven harms across groups.

The report would also be stronger with an ablation study on the data generation process and fairness-constrained training methods. Comparing different proxy features, fairness constraints, and steering mechanisms would make the claimed advantage of SutraAudit more convincing.

Finally, the discussion could expand beyond binary loan approval to address broader structural factors mentioned in the introduction, such as caste, non-traditional rural income streams, and linguistic variation. These factors may affect how creditworthiness is represented and judged in real-world Indian fintech systems.

Overall, SutraAudit is a practically relevant project. It identifies a neglected Global South credit-safety problem and offers a promising framework for continuous fairness auditing, but it would benefit from clearer data documentation, stronger validation, feature-level analysis, and a more detailed explanation of the alignment mechanism.

The problem you picked is very good and not many people work on it. Fairness in lending for India, with caste, informal income, UPI and phone type, this is important and your report is clear and well written. My main worry is the test. You made the data so both groups are equally creditworthy, then you show the model treats them differently, then a tool removes the gap. When you decide the answer in advance the test cannot really prove anything about the real world. Please try it on real data, or at least on data that is more messy and realistic. Also the fix goes through one paid tool, but you never explain what it actually does, so nobody can repeat your result without that product. The tone in some places sounds like a sales pitch. Very good topic, but the proof is not there yet.

You've picked a well-targeted problem and tying it to the RBI's Seven Sutras is a smart governance framing; holding true repayment ability identical across groups was also a thoughtful design choice for isolating bias. My core concern is that the central result is largely true by construction: you generated data in which group membership is encoded in the proxy features, trained a model free to use them, and then "discovered" proxy discrimination. That outcome follows from the setup rather than revealing anything about real lending systems, and the fix has the same circularity, since the Blueprint tool was told to suppress exactly those two channels. I'd also gently flag that routing the entire mitigation through one commercial product, with no comparison against standard open methods, sits awkwardly with the independent-audit framing. Here's a suggested redirection: audit what the models actually produce as mandated by the regulator. It would be great to see it tested on messy data with unknown proxy structure, and reframed around the accountability regime you correctly identified.