Pachyderm proudly supports independent researchers battling COVID on the front lines with data. That’s why we’re happy to announce that we’ve helped the COVID Alliance, a project of the Center for New Data, by donating strategic machine learning infrastructure support to their fight against this dreaded disease. In these extraordinary times, we’re facing complexities like never before, and data is the only tool we have to break through the noise, see more clearly and make better decisions.
The COVID Alliance project is a team of volunteer data scientists and software engineers sharing their time and expertise to build actionable intelligence for governments and research groups around the world. Most recently they’ve used machine learning to track super spreader events in the United States as detailed in a comprehensive Washington Post report. Hundreds or thousands of big events take place all over the US every day, and sorting through them to find potential outbreaks is a massive task. The Alliance is doing it faster than ever before, getting crucial alerts to governments in near real time, so they can move quickly to stop the spread.
Pachyderm proved essential to scaling their analysis across big data sets so they can spot trends fast and get out alerts to key government and research institutions faster. In the early days of the epidemic, it took multiple days to process the data and by then the damage was done. The event had already happened and the people had filtered back into their life with no hope of getting in touch with them. The Alliance’s super spreader detection app uses Pachyderm to deliver analysis in only a few hours, helping key government agencies and medical professionals make better decisions on where to deploy their teams.
The data on the virus and the real world is changing all the time and that’s why versioning and lineage proved critical to the team’s efforts. Sometimes that data is imperfect or has errors that are subtle and easy to miss. Data lineage lets data scientists and MLOps support teams pinpoint where those errors happened and roll back in time to reprocess that data and rebuild their models to get them back on track. It also lets advanced teams test multiple models at the same time, on different branches of the dataset, so they can maximize their prediction efficiency, which can make all the difference when life and death are on the line.
Pachyderm is an AI/ML company but like any company we’re made up of people and we’re a part of the world around us. We feel it’s vitally important to give back to the community and that’s why we support AI for good causes. We’re proud to bring hope with our outreach efforts to non-for-profits and research teams who are working hard to fight outbreaks now and in the future. We’re excited to play a even small part in cutting edge research that’s helping save lives and get the economy back on track.
