NLP in Fintech: How Large Language Models are transforming the future of Fintech
Jayeeta Putatunda30:402023
Fintech can skip building NLP from scratch, but not the human in the loop
Language models have grown from millions of parameters to almost trillions in a couple of years, and some open-source versions can now be retrained for commercial use. For a regulated industry like finance, that opens a faster route to question answering and summarization without reinventing the wheel. Jayeeta Putatunda, Senior Data Scientist at Fitch Ratings, shows where that shortcut works and asks what it takes to use it safely.
See how to fine-tune open-source models and build a DistilBERT question-answering pipeline
Jayeeta Putatunda, Senior Data Scientist at Fitch Ratings, explains how open-source large language models and transfer learning let regulated fintech teams build NLP products without starting from scratch. She maps fintech use cases, explains when to freeze or fine-tune model layers, and demonstrates a DistilBERT question-answering pipeline with Hugging Face, closing with a warning against over-promising automation.
“And we are not talking about like models with like a million parameters, we are like almost in trillions now.”
“It's definitely a crucial thing because changing in the semantics of the prompt, like nuances of how you frame the question can, can affect the outcome of the answers.”
What you will learn
Identify where NLP adds value in fintech, from chatbots and churn analysis to credit rating, fraud detection and report generation.
Decide when to use a pre-trained model off the shelf and when to freeze early layers and fine-tune later ones on domain data.
Build a basic extractive question-answering pipeline with Hugging Face Transformers and DistilBERT.
Combine an extractive model with an instruction-tuned model such as Flan-T5 to add reasoning and validate answers.
Set realistic expectations for LLM projects by evaluating data and models and keeping a human in the loop.
Key takeaways from the speaker
NLP in Finance use-cases
Implementation of a Question Answering system using open-source Large Language Models
Pitfalls to avoid while building an end-to-end NLP pipeline
Details
Level
Intermediate
Format
Tech talk
Technical depth
Practitioner
For
Early career, Mid-career, Senior individual contributor
Length
30:40
Published
June 2023
Questions this talk answers
When should a team use transfer learning instead of training a model from scratch?
When labeled data is scarce or expensive to create with subject matter experts, and a pre-trained model exists on a large data set with properties similar to the target task.
What is the difference between off-the-shelf use and off-the-shelf plus augmentation?
Off the shelf, you freeze the early layers and replace only the last layer for your task; with augmentation, you also selectively retrain later layers on nuanced data such as legal or patent documents.
Where is NLP being used in fintech today?
In customer service chatbots and virtual assistants, CRM optimization and churn analysis, credit rating from qualitative text, governance, fraud detection and generating executive summaries and reports.
When is DistilBERT the right choice and when is Flan-T5 better?
DistilBERT is extractive and pinpoints the exact span that answers a question, while instruction-tuned Flan-T5 can add a reasoning or one-line explanation, so the two can be chained to validate answers.
How should organizations set expectations for LLM projects?
They should not promise that manual workflows will be fully automated at human level; instead they should evaluate data and models, keep a human in the loop at all stages and test several model flavors with SMEs.
About this talk
1\ Overview of AI in Finance - where it is and the path forward
2\ Best practices to consider as organizations begin implementing end-to-end ML models
3\ A case study on how to use open-source NLP large language models (LLMs) to build a domain-specific Question-Answering model
Speaker: Hi, everyone. Thanks for joining. I think we're going to give a couple more minutes for everyone to join in and then we'll get started. But thank you for joining in the session. I hope you're having a great, you know, session experiences with Women Tech Networks conference and attending a lot of good sessions. I'm sure I have, I was not able to make it to quite a few, but then I kind of saved them just to watch them later.
Speaker 1: I hope you can do that too. Speaker 1: We'll just wait for a couple more minutes by that time. If anybody have any questions or, you know, particular concerns or areas about today's topic and I'll be in Fintech and you want to put that in the chat, feel free to do it and we can, you know, take those questions later on.
Speaker 1: I think we can get started and then we pause if there's any other additional questions that comes in, in the chat. But Speaker 1: Hello everyone. Thank you again for joining the session today. My name is Jayita. I work as a senior data scientist at Fitch Ratings. I'm based out of New York. You can connect me via LinkedIn. If any questions I'm not able to answer today or there's no time, please feel free to reach out and we can connect later on as well. The topic for today that we're covering is like, you know, I'm, I'm sure all of you in the data space have heard about the tremendous growth of large language models. And so, I think that's really important. I think that's really important.
Speaker 1: Yeah, I think that's really important. I think that's really important. I think that's really important. Speaker 1: Yeah. You can connect me via LinkedIn. If any questions I'm not able to answer today or there's no time, please feel free to reach out and we can connect later on as well. The topic for today that we're covering is like, you know, I'm sure all of you in the data space have heard about the tremendous growth of large language models.
And we will look into how that the expansion of that domain happened over the years. But I wanted to touch upon how in the, you know, the fintech industry, which usually we see as a very governed and very regulated industry, and rightly so, it needs to be, is starting to adopt some of these technologies in terms of saying, Hey, how can we leverage all these massive amounts of data that we have and leverage some of these open source technologies, but make it in a more safer and regulated way so that we can get to our end goal or KPIs faster versus, you know, trying to, you know, reinvent the wheel from scratch every time, which may or may not work out.
Right. So let's just jump into it and talk through that. So, yeah, I'm going to cover a little bit on the NLP sort of trends, some of the challenges, what is happening in the NLP fintech space, how we are using transfer learning of NLP in solving some of these use cases, and then a very quick sample code demo just to show that how easy it is to, like, you know, start getting integrated into your systems.
I'm going to talk about what is NLP today. I'm sure all of you are aware of how we are leveraging some of these textual large language data based models in feeding in some of these open source models, and like, you know, trying to understand meaning text information, even using images from there. So if you look into this chart of growth of NLP, and I have one more to show, because I ran out of space to kind of keep adding the details into there, you see that in the last couple of years, the growth in terms of the number of parameters, that's the size of the model that were released versus the companies that did exponentially grew.
And we are not talking about like models with like a million parameters, we are like almost in trillions now. And that all happened within the last couple of years of research and boom in the space. And this chart actually is like, I really like the look of it. It's really messy. But if you see the timeline, it kind of shows the different flavors of the similar models.
Like we see at the end, like in the open source GPT-4 category, there are not one, but like almost six to seven different flavors of a similar kind of model. Some are closed source, like you cannot, you can use it for like research purposes, but cannot use it for commercial purposes like Lama. But then we already have like a couple of options like Dolly that were released by Databricks that you can implement commercially by retraining or fine tuning using your commercial official or I would say entity data.
So this kind of opens up the arena of how much can be leveraged from the open source and keeping in mind that we are like keeping a note of the training data that goes into this model so that we are not creating any bias into our financial systems and different models that we're building in the Fentac domain.
So this image kind of shows like the kind of task that some of these large language models are able to do and achieve. And this is a very nice infographic from one of the blog that Google released about the Palm model. And it kind of shows that we started off from like just very basic language understanding, summarization, code completion.
And now we are in a phase where we are like, it's complete like a general knowledge or like a complete knowledge base that you can leverage to kind of get started, kickstarted versus trying to build everything from scratch. I'm not sure if you remember the time, like when we were building word to work models from scratch and they were like so prone to errors, they were out of vocabulary, word issues.
Because we are in an era that have completely taken out those challenges and have completely created a different domain of a baseline model that we can get a jumpstart from. So one of the very recent releases from Google that is a very, I would say, a powerful model. If you have like a good amount of data set and you want to build an application service that can do summarization, question answering, can also do inference or even like inference on reasoning that you give it multiple cases and say, why do you think that this is true?
So it can also find out those reasons and logically attempt to answer from the text context that you have provided, right? And we will see some examples later on. Here, FLANTE5 is an instruction-tuned LLM. When I say instruction-tuned, it means that you have to give multiple inputs. First input would be your context. Like in this image, you see the text input that we have provided is that, you know, could Geoffrey Hinton have had the conversation with George Washington?
That's the question. So that's the text input that you're providing. And then there's like instruction-tuned and you are asking the model to answer based on the instruction that you're providing that give the rational before like answering. So it creates that step-by-step answering process and can guide you to get to that stage sooner to get your answer generated versus trying to build, you know, from scratch like an ER model or an extraction model and then kind of trying to loop that together as that why or what is the reason behind it.
Just one application, I also wanted to show some applications from the Google's palm model and what are some of the features that it can do, like counterfactuals, emoji game, conceptual understanding, as well as the synonym game. So you can ask various prompts based on those various prompts, whatever responses or options you would provide from in the context, it would be able to identify the correct response based on your prompt.
So just a very quick example is that from the emoji movie game example, you can see that the prompt is what movie does this emoji describe, there is a robot, a cockroach and a word, and then there are multiple options. And then the model shows wall E as one of the better fitted answer, why from the context again, and from the prompt again, because in the prompt, we provided robot as being a synonymous word, or one of the contextual factors that the model would take into consideration for answering, right?
Great. So we're going to just go very quickly into some of the applications of NLP that I've experienced or seen in the FinTech industry. Before that, as you all know, I'm sure you've all played around with ChatGPT, there's also an open source version of ChatGPT that Hugging Face recently released called Hugging Chat. So feel free to explore, it's just a first iteration, feel free to look into that and like, you know, play with it and see how that works out against ChatGPT.
So I was just asking, like, you know, highlight some use cases of NLP and FinTech, again, the same concept that we're talking about today, and like, you know, make it look like a conference. It's titled. So if you see, you see that I got five outputs, that talks about uncovering financial fraud with NLP, case study in FinTech, personalizing investment experience, a FinTech approach, it looks very formalized standard, but talks about like, again, the use cases itself as a conference title.
And so it's always fun to play around with it and see what kind of, you know, input pointers it gives us. And yeah, and you can just start to see how you can incorporate it in your daily work that you do or like any other personal side works you do. Another tidbit is that you can also create your recipes through it.
I tried and it definitely works. So great. So some of the core areas are like, you know, customer service, building chatbots, virtual assistants, CRM optimization, like when you have, you know, thousands and thousands of sales, end-to-end sales communication channels that are not centralized. How do you create value or find pattern as to why you are like maybe losing a client?
Why is the churn ratio something like this? So you have to create that contextual parameter that using LLMs can be very useful and helpful in this scenario. And then there's also credit rating, like, you know, quantify matrix to kind of identify analyzing qualitative text and give it a weightage as to the types of matrix that we try to tag it to.
And then, of course, there is governance, fraud detection, as well as content generation, building executive-based summary, report generation, abstracted summaries. And that is why when I mentioned Flan T5, Flan T5 is also an abstractive-based model, which means that not only can it extract information from your content, but it can also generate its own. I would have to say that it's not as strong as, like, the GPT-based models, and rightly so, right?
Because it's a much smaller model. The purpose is not that. But you can still create, like, you know, one-line summaries. If you have five paragraphs, you can create, like, a topic of what the five paragraphs are talking about. And, like, you can create, like, multi-line summaries as well. But just to make sure that we fine-tune and, like, tone our model parameters so that it doesn't start to hallucinate and write incorrect information as an output.
Great. So how are we doing it? There's, of course, machine translation, conversational AI, intent classification, knowledge tree, question-answering, all falls under that bucket of conversational AI. There's natural text generation, as well as legal document understanding, where we are trying to, you know, understand what kind of entities are mentioned, what are the relations, or what is the, you know, graph of knowledge that we can identify between multiple entities or for, like, say, a particular topic that you want to explore.
And then there's also text-based applications, like I mentioned, fraud detection, threat detection, et cetera, which are very common in quite a few of the industries that I have worked and seen in. This concept, before we go into the code and actually talk about the models, this concept of transfer learning is something that have made this shift in the industry, in the mind space of all professionals as well as researchers, is that how can we leverage that have already been pre-built on some data, use those weights, understand their meaning, and then apply it for our data set.
If our data set and that data set that it was worked on matches up correctly, maybe we can leverage without even fine-tuning, just using off-the-shelf, versus if your data set is very, you know, distinguished, has a nuanced understanding or a meaning to it, like legal documents or, I don't know, patent-based applications, those kind of documents, then you definitely need to have a fine-tuned version of it.
But you can still utilize those pre-built weights and learn from them as your knowledge, general knowledge database and build on top of that versus starting from scratch. So when to use transfer learning, like I said, if there's a scarcity of label data, like you do not have enough to start training your neural network or big LLM models.
So creating level data is definitely expensive. You need SMEs and experts that need to spend time to kind of create a good data quality, and that may or may not be enough to get you to the model KPIs that you are targeting for. So in those scenarios, then there's like a pre-trained model with a huge data set that already exists, and it has the same kind of properties to the task that you are trying to solve, right?
And we will see why. So this is the reason why. So it can either be, you know, same domain or different task. And in this case, I took an example of an image because it's like a little easier to explain, is that the tasks are different because you, in one, the first task that it was trained on, which is T1, it's like talking about only classification of general object labels.
But then your target task is that you have to identify urban versus rural, which is the basic domain is same, that the image is image based problem, you're trying to distinguish objects, but then the task is different. The another variety or flavor of it is that the domain is different, but the task is the same. Here, the task is translation, but in first one, you are doing like, you know, a PO, sorry, the task is POS tagging, which is like the same for both the use cases.
But why the domain is different, because in the first, a pre-trained model that was trained on was the source language was German, but you are applying it to an English document for inferencing. In that case, the task gets, the domain gets different. And like I mentioned, like if your base task parameters or properties are similar to what you are leveraging for your target task, then off-the-shelf pre-trailed models can be definitely used as feature extractors, what you need to just do is like, you know, keep like, freeze all the weights of the previous or early on layers, like in neural network, we have multiple layers, right, and similar concept in LLMs, we have multiple layers of weights, when we learn about the particular data format, or like, you know, what are like the baseline embeddings, what are like the associated embedding maps,
in association with those embeddings, these are some things that will not change too much, so we freeze all those weights of those model layers and then, during training, we just replace the last layer to change it up for your use case. Like if you want to utilize like say, just an example like a distal bird model, but you don't want, the downstream task is for, not for classification, but for Q and A, so you change the last layer to kind of update on that.
Or, if the data sets are completely different, you have nuanced data, like again legal data set or different kind of patent based applications that have very strong languages that you wouldn't find in very generic documents on which these LLMs were trained like blogs and Wikipedia, etc. So, we kind of do off-the-shelf plus augmentation, which kind of means that the initial layers, they capture the generic features, they still remain the same, like the meaning of the words wouldn't change that the model has already learned, right, but then for the later layer we focus more on like the specific tasks and we selectively retrain the models with our newer data sets and that's the process of fine tuning.
Right, so we have well defined targets, right, so we have well defined targets, we want to like maybe do a Q and A on a legal data set, and we are building or fine tuning on top of like say an example, a distal bird model that was already trained on a Q and A task, but we are adding our company specific or your company specific data to make it more valuable and accurate in terms of understanding the wordings, the nuances, the differences between different concepts in a much better way.
So, two options, use off-the-shelf as well, or you can do off-the-shelf plus augmentation. So, this is just a visual representation to show the exact difference that I was talking about in the previous slide. Here, you have like, you know, this entire feature learning that happens in the first stage from all the training images. You have this source task to classify if it's a cat, tree or a lake, and you see all these convolutional layers and fully connected layers.
So, what happens in transfer learning is that you have a different task, which is the training images from urban and rural images, you keep C1 to C5 frozen because you are not touching them like actually C1 to F7 frozen because those are very similar to your input task and you don't want to change them at all. And then you add a new layer, and then you add a new layer, which is FC8E that you see here, is that where you are adding that last final layer to identify now as urban versus rural versus doing the object detection from the pre-trained last layer model.
So, that's how we kind of learn from what the model has initially learned and then utilize our sets of data to identify or update the last layer so that we can use it for our use case. Great. And why do we want to do that? Like, what's like, you know, the go or what's the benefit of it?
So, definitely, it gives you a higher start, like better initial skill of the source model. Before fine tuning, you're not starting from scratch. Like I mentioned, if you remember building Doc2Vec and Word2Vec models from scratch, initially, you would have like an out of vocabulary problem. There were like situations where models wouldn't understand or represent embeddings correctly because there were not enough instances or examples of commonality of the sentences that you are passing on to the model.
And it would not understand that relationship well. So, this gives you a better start or a better initial skill of the model. There's also higher slope, like higher learning rate. It is able to learn faster using a small number of training data set, as well as higher accuracy. And it's definitely fast and cheap to kind of train versus, you know, just to give an example, like, there were cases where I have fine tuned models on maybe 2000 per label data set.
And it worked out with like a very good number of RRTs that we met the benchmark for accuracy scores and just imagine if we had to build it from scratch that would have required like, you know, I don't know, like thousands of par label documents just to create like, you know, baseline model. Some of the applications are like, you know, information retrieval, that this is part of the QA model, it may or may not be the only application.
But just to give an idea that the flanty five model that I was talking about before, if you give a context out like a context from like a paragraph or like a PDF that you're processing. And then there's a question that you want to ask and the question can be a free flowing natural language question right. And then the goal for the model is to identify where, what is the right sentence that have the correct answer, and what is the exact answer.
So with flanty five, we can do both, we can identify or we can ask it to give, give a prompt saying that, hey, write one line summary of the correct answer, and it can do so for us. And then it extracts the same correct answer from the text as like, you know, particular extractive text value. So let's look at a sample QA.
I'm sure all of you have like, you know, explored hugging face library, they are like, you know, the epitome of making AI accessible. They have like a lot of good models like up there, there's data sets, there's spaces, communities that people are every day using and uploading their models for. So you have, if you have not explored, I would highly encourage to kind of look into hugging face.
And you can like, you know, download all these models or large LL models from hugging face for your test cases. So what we are doing here is we're just installing like transformers, we're installing torch. And then from transformers, we're calling in distilbert tokenizer, which is like a pre built tokenizer from the distilbert model that was trained. So that means that whatever data that have gone into the distilbert models training, whatever it has learned about the tokenizer, the embedding process is saved there.
And we can utilize that to also tokenize our, on our data set. So we don't need to create a new tokenizer, right. And then there's distilbert for question answering, which is the final modular head for that model. So we're calling in tokenizer, calling in from the pre-trained, and you see there are multiple flavors of distilbert model. There's distilbert base uncased, there's base cased.
That means that the data that it was trained on have like, you know, case sensitive data. That means like if it's a noun, the first letter will be a capital letter. Nothing is like, you know, turned to, and in case of uncased, everything has been turned to lowercase. So for some use cases that can be very valuable as well, because we're trying to, you know, do maybe any are extractions and we would want to retain some of these nuanced embedding understanding into the models understanding weightages.
So we are calling both the tokenizer and the model. And then, and then, and then that's it, that's all you have to do to kind of, you know, get the model to an inference stage. And the next part that you see here talks about the same, like a context that I provided from like, say, a very generic document from Apple's sustainability, or I think it was an annual report.
And it was talking about how Appalink, you know, achieved return on average. It invested on some of these areas, et cetera. And ROI improved. So, and this context can be longer. I think this has a context length of 512 tokens, like 512 tokens. We can break down bigger chunks of context into smaller chunks, create that linkage so that if the answer exists in a different chunk, after you have tightened, or you have made the split of a bigger context, we don't lose, you know, vital information.
Or like we don't lose information or like we don't lose information half here and half in the second context parameter that shouldn't happen. Right. So after just providing this context, I ask a very simple question that what did Apple achieve? Of course, there's like a small function where I am calling the model, the input tokenizer, and then just, you know, asking to extract the information.
So it goes into Q81, which you do not see in the slide, but the output that comes out or the complete answer that comes out was that return on average invested assets on 4.31 percentage in fourth quarter. So why did Apple, what did Apple achieve? This was their achievement that they achieved a return on average invested. So why did Apple's ROI improve?
Again, the last answer is like, because they had a net income growth. So you see, since DistilBird is a more extractive based model, it is able to, you know, identify very clearly the area of the context that answers your particular question. And you can test any kind of, you know, context information with more numbers, with more numerical data, or like, you know, or the questions can be more open ended as well.
One caveat to that is that for answering open ended questions. It can get a little tricky, because if you're asking to write like, did it do it? Yes or no. To answer that, Bert is not prepared. Bert needs to find first, DistilBird first need to find the answer. And then pre-process and you can have like maybe a post-process function to say if the answer was found and matches with the source of truth, just for the initial training.
You can say yes, and then that could be your final output version. So you can still work with different flavors of question type based on your business use case and leverage this to kind of have and have like a post-processing output clearing format just to make sure that it aligns with your question types. Right. One other example was like, so what is the status of total ranking?
So status of total ranking deteriorated. Again, in some use cases, we see that identifying only this one keyword or deteriorated as the answer may or may not serve the purpose. So you would want, you know, more associated context to support your answer of why deteriorated. So in those cases, Slantify would be a better model because you can, you know, add prompts and say that, okay, write one line about why the status of total ranking.
What is the status of total ranking and give a reasoning for your answer. So in that case, Slantify, like I mentioned before, since it's an instruction tuned model, it will be able to better identify and give a logic to why it ended up with that answer. Right. But this will be a very small user friendly, production friendly and deployment friendly model in instances where we can quickly deploy it.
It definitely gives you an edge to make sure that you are starting up, setting it up and not wasting too much time in R&D or trying to set up those prompts, standardize those prompts because prompt, like I'm sure all of you have heard about this term prompt engineering. It's definitely a crucial thing because changing in the semantics of the prompt, like nuances of how you frame the question can, can affect the outcome of the answers.
Right. So we need to be very careful, especially when we are applying it to a business use case and the end users are going to see the results. So you don't want to be creating unnecessary friction there. So, but Nisselberg can still get you there, identify the location exact answer, and then you can utilize Slantify maybe as an associated next step to kind of identify more context or more information so that you have two answers to validate, and then create like a chain of model answer with the downstream task at hand. Right.
And then again, just to show that it can also identify matrix. Is that like how much the serial of the total ranking deteriorated? It's like 56 to 60, which we saw in the context before that it deteriorated. I think the text is missing from the screenshot, but yeah, that was the answer that was correctly taken into account.
Great. So I think my ending note would be, I think, since we are seeing so much of a boom in the LLM space with so many different kinds of models and flavors that's coming out. It's, it's, it's very interesting to see that how, you know, leaders and business representatives are finally like being aware of that. And the chat GPTI definitely helped kind of open up that discussion domains, but we also need to be very conscious in how we promise.
And we shouldn't like, you know, over promising that, hey, this complete workflow that was happening manually, and then we can automate it and it'll like, you know, reach human intelligence benchmark. So that is, uh, at least in all the use cases that I have seen is like a super over promise and we shouldn't go to that space at all.
So it is definitely tricky. We need to make sure that the models that we use, the data that we are using is highly evaluated. And there's a human in the loop for QA at all stages. It helps in keeping the data at check, the models output at check before pushing the final results out to the end users. Right.
So, uh, like always review the use cases, have like a better business sense working with SMEs. And most of the times we have seen like, you know, open source is huge. So we get confused as to which model we have to use that again, comes down to testing, testing different flavors of models, testing what kind of data sets you have.
And then finally getting to that final answer of what works best for you. Um, right. And I think that's, that's all from my end that I had. Uh, I wanted to keep some time for the end if there's any additional questions. Uh, if you have any questions later on, or, uh, you want to reach out later, you can feel free to reach out.
Uh, and yeah, I will, uh, stop there and see if there's any questions I can answer, uh, specifically or, or this was great. Thank you for joining in. Thank you.
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
How can regulated fintech teams use open-source LLMs without starting from scratch?
Jayeeta Putatunda explains how transfer learning, careful model choice and a human in the loop make large language models practical for finance.
In short
Jayeeta Putatunda, Senior Data Scientist at Fitch Ratings, argues that financial firms no longer need to build language models from scratch. Open-source large language models and transfer learning let teams reach their goals faster, as long as they track training data, test models carefully and keep humans in the loop. She maps fintech use cases, explains when to freeze or fine-tune model layers, and walks through a short Hugging Face demo of a DistilBERT question-answering model.
Key takeaways
NLP use cases in finance
Implementing a question-answering system using open-source large language models
Pitfalls to avoid while building an end-to-end NLP pipeline
From millions of parameters to almost trillions
Putatunda frames the talk around a tension. Fintech is a heavily governed and regulated industry, and rightly so, yet it holds massive amounts of data that open-source models could help unlock. The question is how to use those models in a safer, regulated way instead of reinventing the wheel every time.
She starts with the scale of change. Using charts of model releases over time, she shows how quickly model size has grown in just a few years of research.
"And we are not talking about like models with like a million parameters, we are like almost in trillions now." (3:48)
That growth has produced many flavors of similar models. Some, like LLaMA, can be used for research but not commercially, while others, like Dolly from Databricks, can be fine-tuned on company data and used commercially. Putatunda stresses keeping track of the training data that goes into any model so teams do not introduce bias into financial systems.
What modern models can do, and where fintech uses them
Drawing on Google's PaLM infographic, she shows how models have moved from basic language understanding, summarization and code completion to something close to a general knowledge base. She highlights Flan-T5, an instruction-tuned model that takes a context and an instruction, and can reason step by step before answering.
In fintech, she sees NLP in customer service chatbots and virtual assistants, CRM optimization (finding patterns in scattered sales communications to understand churn), credit rating work that turns qualitative text into weighted metrics, governance, fraud detection and content generation such as executive summaries. Flan-T5 can write short abstractive summaries, though it is much smaller than GPT-based models, so tuning matters.
"But just to make sure that we fine-tune and, like, tone our model parameters so that it doesn't start to hallucinate and write incorrect information as an output." (11:02)
Transfer learning: freeze what generalizes, retrain what is specific
Transfer learning, Putatunda says, is the concept that shifted how practitioners and researchers work: reuse weights a model has already learned, then apply them to your own data.
"So when to use transfer learning, like I said, if there's a scarcity of label data, like you do not have enough to start training your neural network or big LLM models." (12:58)
She explains two kinds of mismatch: the same domain with a different task (classifying general objects versus urban and rural images) and a different domain with the same task (a tagging model trained on German applied to English). When the source and target are similar, teams can use a model off the shelf, freezing the early layers and replacing only the last one. When data is nuanced, as with legal or patent documents, they add augmentation: the early layers that capture generic features stay frozen while later layers are selectively retrained.
The benefits are concrete.
"So, definitely, it gives you a higher start, like better initial skill of the source model. Before fine tuning, you're not starting from scratch." (18:33)
She adds a faster learning rate, higher accuracy and lower cost, noting from her own work that models fine-tuned on around 2,000 examples per label met accuracy benchmarks.
A question-answering demo with DistilBERT
For the demo, Putatunda turns to Hugging Face.
"I'm sure all of you have like, you know, explored hugging face library, they are like, you know, the epitome of making AI accessible." (20:52)
After installing Transformers and PyTorch, she loads the pre-built DistilBERT tokenizer and its question-answering head, explaining the choice between cased and uncased versions. She feeds in a passage from an Apple report and asks questions such as what Apple achieved and why its return improved; the extractive model pinpoints the exact answer span. She explains splitting long text into linked chunks so answers are not lost at a boundary, and adding post-processing for yes or no questions.
When a single extracted word is not enough, she suggests chaining DistilBERT with Flan-T5, which can add a reasoning step. She warns that prompts need care.
"It's definitely a crucial thing because changing in the semantics of the prompt, like nuances of how you frame the question can, can affect the outcome of the answers." (27:07)
Promise less, evaluate more
Putatunda closes with a caution. ChatGPT has opened up conversations with leaders, but teams should not claim that a manual workflow will be fully automated at human level.
"So that is, uh, at least in all the use cases that I have seen is like a super over promise and we shouldn't go to that space at all." (29:07)
"We need to make sure that the models that we use, the data that we are using is highly evaluated. And there's a human in the loop for QA at all stages." (29:27)
Her advice: review use cases with subject matter experts, and test different model flavors against your own data until you find what works best.
Watch the talk
"NLP in Fintech: How Large Language Models are transforming the future of Fintech" by Jayeeta Putatunda (31 minutes): https://www.womentech.net/video/nlp-in-fintech-how-large-language-models-are-transforming-future-fintech
About the speaker
Jayeeta Putatunda is a Senior Data Scientist at Fitch Ratings in New York, with industry experience in natural language processing, statistical modeling, product analytics and machine learning solutions for specialized use cases. An avid NLP researcher, she has spoken at ICML 2022, ODSC East and MLConf EU, and mentors through Women in Data Science, Girl Up and WomenTech Network.
Questions this talk answers
When should a team use transfer learning instead of training a model from scratch?
What is the difference between off-the-shelf use and off-the-shelf plus augmentation?
Where is NLP being used in fintech today?
When is DistilBERT the right choice and when is Flan-T5 better?
How should organizations set expectations for LLM projects?
Drafted with AI from the talk transcript. Editor review pending.
Frameworks and ideas mentioned
Transfer learning
Fine-tuning
Instruction tuning
Prompt engineering
Human in the loop
Resources mentioned
Hugging Face (models, datasets and spaces)
HuggingChat
Hugging Face Transformers library
Google blog infographic on the PaLM model
Cite this talk
Jayeeta Putatunda. "NLP in Fintech: How Large Language Models are transforming the future of Fintech." WomenTech Network, 2023. https://www.womentech.net/video/nlp-in-fintech-how-large-language-models-are-transforming-future-fintech