Machine learning holds tremendous potential for FinTech but it’s not easy getting a model from research to production. A survey of the state of enterprise machine learning in 2020 found that half of all organizations hadn’t put a single model into production.
Cutting edge research models like GANs and GPT-3 create tremendous hype and showcase the amazing potential of AI but what’s left out of the hype cycle is the incredible cost and time that went into building those models. The research teams behind them often built their own infrastructure to train and deploy that model from scratch. For AI to work for the rest of us we need an enterprise stack that we can build our models on. We don’t want to be inventing the car, while also building the roads to drive it on at the same time.
CI/CD for Machine Learning Data
But unlike a car, a machine learning model isn’t finished once it rolls off the assembly line. Our models need to keep learning and adapting to new data. Not only is the data changing, our models and the code are changing simultaneously. Even a small change to how we preprocess our inputs can have drastic consequences when it comes to model quality. And this discrepancy is most noticeable in the development and deployment stages of machine learning development frameworks.
Even after we deploy our models, we begin to see our own research-to-production gap. We start seeing the biases of our training environment, the edge cases that we didn’t think about, or even performance issues with our code.
So what can we do?
Try Pachyderm for FinTech
In our FinTech Kit, we show you how to scale your machine learning across your organization, taking you from data ingestion all the way to serving. We also illustrate how to prep, train and deploy a state-of-the-art NLP classifier, using Pachyderm and Seldon.
Download the FinTech Kit to turn your AI potential into reality today.