Machine Learning won’t reach its potential without the human element

We’re excited for Sarah Catanzaro to join us this week! Sarah is an investor at Amplify Partners.

Sara and I spoke this week on topics ranging from machine learning’s impact on social media to the role it plays within companies.

Background

Sam: Sarah! Thanks so much for your time. I’m pumped to talk. Can you start by shedding light on your background and the types of companies you invest in?

Sarah: Of course. I recently joined Amplify Partners, a seed stage VC that invests in technical founders solving technical problems. We focus primarily on Machine Intelligence, Analytics, and Enterprise Infrastructure. At a high level, my career has been focused on leveraging computational models to better understand complex data sets.

Not all problems are suited for machine learning

Sam: A common refrain amongst VC’s is, “all startups will eventually be machine learning startups.” What do you think?

Sarah: I think it’s reasonable to say all companies will leverage machine learning because there is a broad enough swath of problems to which machine learning can apply.

But, not all problems are suited for machine learning. It should and will be a tool in everyone’s toolkit, but ML is best suited to a subset of problems where you have high volume, highly dimensional data and a well defined objective. There are many other problems, for example when only small datasets are available, where other methodologies may be more appropriate.

Better predictions don’t guarantee adoption

Sam: Do you have any opinions about machine learning that would be unpopular amongst the VC crowd?

Sarah: I don’t think that adoption of AI will hit a tipping point until we have more effective mechanisms and interfaces for consuming machine learning results. These mechanisms and interfaces must enable us to interpret and reason about uncertainty associated with our models. In other words, they should show what an algorithm based a decision on and how confident it is in its decision.

For example, people get excited about academic research on medical diagnoses, then jump to the conclusion that these methods will be adopted by physicians. What people fail to recognize is the output of machine learning models must be used by people, employed by organizations, with responsibilities and liabilities. A physician, who faces malpractice risk, won’t accept a diagnosis just because your model is “more precise.”

He/she is going to want to know how the model arrived at the diagnosis, why it might be wrong, and who is most likely to be affected if it is wrong.

We don’t yet trust AI and I don’t think we will or can trust it until it can “admit” what it doesn’t know.

For example, consider a system that recommends certain drug treatments based on medical records. The model underlying this system was developed on a training dataset. If the system encounters a patient who is unlike the patients profiled in the training set — that is, who lies outside its data distribution — it may make an unreasonable recommendation about a treatment plan. In this case, we would want to know that the system is not confident in its recommendation.

Likewise, think about an autonomous vehicle, which is trained to identify trucks, motorcycles, and cars. If this vehicle encountered a new object, say a scooter, which it could not classify with certainty, we might want it to alert the user to take control of steering. People won’t use self-driving cars or accept treatment from doctors aided by black-box algorithms if they can’t ask questions like “how do you know this” or “why are you sure?”

Medicine and autonomous vehicles are extreme examples, but there are other use cases like real-time bidding or inventory management. Businesses want to understand why and when their model might be wrong so they can do scenario planning and think sensibly about possible futures. If they don’t their decisions can have very negative, cascading effects.

As we learn to interpret algorithmic decisions, we learn to help people in new ways

Sam: What happens when we can interpret the algorithms?

Sarah: By enabling interpretability, we can better understand and even change our own behavior. For example, interpretable machine learning could clarify bias in our datasets (and therefore in our society), which we didn’t know existed. Also, by understanding what features machines use, we could improve our own learning experiences.

For example, Marco Tulio Ribeiro and other researchers at the University of Washington are finding that with interpretable machine learning models of Chinese characters, they can provide better guidance to students learning to read Chinese.

So, it’s not just about super negative repercussions if we can’t explain models. But if we can understand them, we can enable and improve human behaviors.

Machine learning on social media can affect policy

Sam: Allow me to pull you back to super negative repercussions :). There’s been so much talk recently of the research coming out about the negative impacts of social media. How does machine learning make this worse?

Sarah: Machine learning on these platforms determines what we see and pay attention to. Even things as simple as the order in which news articles are displayed can impact what we read first, which can impact what we pay the most attention to, and could even affect our behavior that day. For example, an article that is prioritized in a news feed may elicit a stronger reaction. If we’re made to feel strongly enough, we might decide to write to our congressman, which could ultimately impact policy.

There’s a huge power mismatch associated with data. On one hand, algorithms are trained using data collected by tech behemoths about our preferences, social networks, buying patterns. However, we have no transparency into when and how those algorithms are applied. So perhaps there’s a case to be made for interpretability here too.

For example, how might I respond to news article ranking if I knew it was generated algorithmically? Would I respond differently if I knew that the author’s ethnicity was used as a feature in the ranking algorithm? Why not allow me to specify the features that are applied to my ranking? If we’re more transparent about our algorithms, we can give more agency back to the consumer.

Machine learning cannot be isolated within companies

Sam: How then do we make sure that machine learning is built ethically within organizations?

Sarah: As machine learning gets commercialized, it won’t be the sole responsibility of machine learning engineers and mathematicians to ensure AI safety but of all people who build products — from UX designers to product managers to sales and marketing.

While I don’t think that everyone needs to be able to implement machine learning, everyone in the company should understand it and should become more data literate so that they can reason about the consequences of machine learning. We have to hold everyone in an organization responsible for AI safety, not just the machine learning engineers.

Truth be told, machine learning is broken in a lot of organizations. Too often, machine learning researchers are isolated from the rest of the organization. As a former Data Product Manager, I think the way product managers interact with machine learning developers is often broken.

They think of machine learning as a silver bullet that produces a completely clean wound. Product needs to get smarter about how machine learning is wielded and about its hidden costs; PMs need to provide more direction about how its output will be consumed.

It’s right to hold machine learning developers responsible for optimization metrics (although even they need to think beyond precision and recall), but product managers need to understand how certain design decisions can impact the user perception, including of precision and recall since these perceptions actually affect how the user responds.

Machine learning has been a bit too isolated. It needs to be brought into the fold, and results need to be tied back to organizational directives.

The power of machine learning is the power to make humans better version of themselves

Sam: What excites you most about the future of machine learning?

Sarah: Honestly, I’m most excited by how we use machine learning to understand and improve ourselves, including through augmented intelligence. Machine learning can help us better understand our own biology and psychology and if applied responsibly, it can also help us behave better and make more conscientious and less biased decisions.

Machine learning can help us personalize and accelerate education and could even one day facilitate business conversations and negotiations. I know it sounds idealistic, but I really want us to use machine learning to become better people.

Today, there is a lot of paranoia about AI and automation replacing jobs. I think, and perhaps hope, people will start to react to this fear, by considering augmented intelligence. How can we use machine intelligence to make ourselves more efficient workers (so that we won’t lose our jobs to robots and so that we can have more productive economies)?

Sam: That is an interesting question indeed! Thanks again for your time, Sarah. Looking forward to doing this again.

Sarah: Talk soon!