Why Affirm is incorporating LLM tech into its underwriting

• Key insights: Affirm is using transformer-based machine learning — the same technology that powers the world's most popular large language models — in its underwriting models. 
• What's at stake: The point-of-sale lender says the tech allows it to better predict a consumers' ability and willingness to pay. 

The success of large language models such as ChatGPT, Claude and Gemini has always rested on their ability to predict what comes next. 

Processing Content

Affirm is incorporating that same transformer-based technology into its underwriting models with the hopes that it will be able to underwrite more loans that it would have previously not funded.

"In underwriting and in our use of it, we're looking at more credit and payment and repayment behavior as a sequence that gives a good indicator of what's going to come next," Affirm President Libor Michalek told American Banker. 

LLMs run on transformer neural networks, a deep-learning architecture that processes sequences of data as a whole rather than step-by-step. In its most basic sense, it is what allows someone to query a computer in natural language and get a response in kind. 

"Unlike our previous generation of models, which are where each signal is hand engineered to basically provide a piece of information for the model, here we're able to just take sequences of historical information about how the consumers have behaved over time, and what behavior leads to good predictions about what they're going to do in the future around both their ability to repay and their willingness to repay," Michalek said. 

The new  transformer-based model can detect patterns with less direction from engineers compared with the old model, he said.  

The old model "was able to detect patterns that the engineers told it to look for," Michalek said. "The difference is that now they're able to say more generally, 'Hey, look for patterns in repay purchases and repayments and credit history events,' but not the specific pattern to look for." 

For example, a transformer-based model could now identify a missed payment as transient, rather than indicating that the potential customer is on a downward trend. 

Affirm is not the only fintech using transformer-based underwriting. Stripe-owned Parafin, an embedded finance infrastructure company, uses a transformer-based underwriting model to underwrite small business credit based on cash flow.

"Predicting the next number in a sequence is structurally the same as predicting the next word in a sentence," Parafin said in a blog post. 

Plaid also has a transformer-based underwriting model. 

Combining different types of machine learning models is a well established practice in the data science community, according to Aaron McPherson, principal at AFM Consulting, who noted that there is a benefit to having more lenders use a transformer-based model. 

"I see this as being more of an attempt by Affirm to get other lenders to adopt the same approach, because it is important to maintain consistency in credit reporting and analysis," McPherson told American Banker. "One of the disadvantages of LLMs, and the reason why you can't just rely on them alone, is that they are 'black boxes,' in that it is difficult to see inside to understand why the model gave the prediction it did." 

Using LLMs to find new features or characteristics that can be modeled through a more transparent technique gets around this problem, and having a consistent approach across the industry would improve accuracy by ensuring that the same data is being used everywhere, he said. 

"It is a competitive advantage, but one which other lenders can easily duplicate, so it's more important for how it advances the field as a whole than for how it affects one company," McPherson said.