Should We Trust AI 100%? Read This Before It Misled Us
Up to now, we tend to hear the success story of AI (Artificial Intelligence) or Machine Learning implementations. Some from many: Netflix (founded in 1997) is an entertainment company that consistently utilizes its analytics to track customers’ behavior from time to time, which has grown from $5 million in revenues in 1999 to $8.3 billion in 2016, with as much as 93 million subscribers in 190 countries worldwide. In 2013 Netflix initiated the production of the new original series House of Cards as a result of predictive effort, employed analytics to increase the likelihood of its success by analyzing customers’ likelihood of the program, along with popular lead-man and well-liked producer/directors. This series helped Netflix add 4 million new subscribers in Q1 2014 and Netflix stock price went up higher since then, from $27 and reaching $160 in 2018.
In the automotive industry, Tesla produced an autopilot electric car and have been delivered not less than 269,000 autonomous cars on the road until Q1 2018 according to a researcher from MIT Human-Centered AI. This autonomous electric car heavily relies on a subset of AI technology called Deep Neural Network that enables per-camera networks to analyze raw images to perform semantic segmentation, object detection, and monocular depth estimation, hence the driver could drive effortlessly (as we saw a lot in social media, that many of Tesla’s drivers even can read the newspaper while the automation system drives the car, thanks to their full build of Autopilot neural networks involves 48 networks that take 70,000 GPU hours to train).
Apart from those companies, Amazon as one of the tech titans of FAANG companies (Facebook, Amazon, Alibaba, Netflix, Google), a company that was founded in 1994 become one of the companies that stays resilient and survived the storm of the Bubble Dotcom Crash back in early 2000, thanks to their proven business model in the online book store, shown by their first profit in Q4 2021. Moreover, during the 21st century, Amazon has led the way on many technological fronts, from cloud computing to e-readers, online grocery shopping, video streaming, and gaming. Most are, not incidentally, ways to receive, store information, and utilize the data they have and implement AI approach on its business pipeline, from AI bot Alexa, cashier-less Amazon Go, to Amazon’s recommendation engine, which generates 35% of the company’s revenue.
Seeing these marvelous successful stories of AI implementation might drive our thoughts that AI will never fail. But is our thought correct? Or have we been misled by this kind of information and unconsciously become our confirmation bias in regards to our belief in AI?
On the other hand, have you ever heard about the failures of AI products? If yet, bear with me, grab your coffee and let me tell you the stories of the dark side story of failure in AI projects.
- IBM on cancer treatment of recommender system
A well-known example of an AI project failure is IBM’s partnership with The University of Texas M.D. Anderson Cancer Center to develop IBM Watson for Oncology to improve cancer care in 2013. The project named ‘moon shot’: diagnose and recommend treatment plans for certain forms of cancer using IBM’s Watson cognitive system. According to a report by the University of Texas System Administration, the project was put on hold after costs topped $62 million- and the system had yet to be used on patients.
- Tesla autopilot car accident
In 2018 there was a fatal crash in Mountain View, that killed the driver of a Tesla Model X sport utility vehicle. Tesla’s Autopilot driver-assistance system and a driver who relied too heavily on it are likely to blame for a 2018 crash in California in which the driver died, as reported by Federal Safety Agency.
- Tay — Twitter Bot by Microsoft Corp.
Tay was an artificial intelligence chatterbot that was originally released by Microsoft Corporation via Twitter on March 23, 2016, the aim was to see how it interacted with humans. Tay was able to perform several tasks, like telling users jokes or offering up a comment on a picture you send her, for example. But she’s also designed to personalize her interactions with users while answering questions or even mirroring users’ statements back to them. However, the bot only lived for 16 hours, due to its racist and inappropriate tweets, Microsoft decided to shut down Tay Bot right away after those controversial tweets and the company apologizes for ‘offensive and hurtful tweets from its AI bot.
- Zillow’s Financial Collapse due to heavily relying on its AI’s House Price Prediction
Zillow is an American tech real-estate marketplace company, that recently shuttered its Zillow Offers business because of failed iBuying algorithms.
A pandemic causes many things around us to get changed including the real-estate market. The drastic changes in the real estate market cause the machine learning models used to predict house prices don’t work anymore.
A detailed algorithm on property valuations led the company to have $304 million in Q3 losses and expects to reduce its workforce by 25% over future quarters to compensate for the impact on its business.
And hundreds of stories to come…
Considering the tremendous risk of failure in AI implementations, it shows that AI not only can produce a beneficial result, but also could bring a huge risk for humans, businesses, and society.
So, what can we, as a society learn from AI failures?
- Understand that AI is not magic, never be perfect, and is far from an overnight process.
AI is not a super-intelligence machine that can do anything that surpasses 100% of human capacity. AI is not more than an automated cognitive, as a result of long hours brain of humans works to formulate the problem, prepare the models, determine the appropriate training data sets, eliminate the potential biases induced by the data, and so on. Then, they have to adjust the software in light of its performance. - Comprehend that data can be subjective and AI can be biased too. Cathy O’Neil in her book “Weapons of Math Destruction” shows there was a lot of AI bias in the real world that amplifies the inequality in society, and affects large numbers of people, increasing the chances that they get it wrong for some of them. Some of the projects that algorithms claim to quantify important traits: teacher quality, recidivism risk, and creditworthiness but have harmful outcomes and often reinforce inequality, keeping the poor poor and the rich. They are often proprietary or otherwise shielded from prying eyes, so they have the effect of being a black box. One example is in the 2016 ProPublica study found that a recidivism risk algorithm (Recidivism risk assessments refer to automated decision making that aims to predict the likelihood of an individual’s future criminal behavior) called COMPAS was almost twice as likely to rate a Black defendant as high-risk for reoffending. White defendants, on the other hand, were more likely to be mislabeled as low-risk. The study found that two years after the predictions were made, they were accurate in predicting violent crime only 20 percent of the time. When non-violent misdemeanors were added, the predictions fared slightly better than a coin flip at 61 percent accuracy. This is very possible to happen when the datasets of non-white races are significantly imbalanced, and this issue has not become a leader’s priority to handle. As Cassie Kozyrkof, Chief Decision Scientist — Google, often remind
“Textbooks reflect the biases of their authors. Like textbooks, datasets have authors. They’re collected according to instructions made by people.” Thus, always mind Garbage in, garbage out. - Embrace imperfection in any AI product, and acknowledge that in some cases, humans are needed to validate especially in an ethical way.
In every AI project, there are always numeric metrics to measure the error from AI modeling, this means nearly impossible for any AI product is flawless without unwanted results. Even if evaluation performance shows perfect measurement, we might face considerable ethical dilemmas. Google, Amazon, and other technology companies have been under scrutiny for years for biases within their artificial intelligence systems, particularly around issues of race. Studies have shown that facial recognition technology is biased against people of color and has more trouble identifying them, leading to incidents where Black people have been discriminated against or arrested because of this AI bias.
Hence, always give room when errors or unethical terms occur.
When we comply with these terms, we’d understand that there might be unwanted occurrence events, even on the most recent and advanced AI technology on the market.
With that, we understand that AI is not the only basis for the decision-making process we should look at, but also the context.
4. Send immediate feedback when the AI products do not work in certain cases or harm the desired outcome or society.
As in the iterative process, when the AI product result does not suit the desired outcome/harm the society, we need to alarm the AI providers that something wrong happened and their call to adjust the iterative process of AI deployment to improve the outcome or even shut down the service.
As a close statement, there’s a precious quote from HBR:
“AI has many potential benefits for business, the economy, and for tackling society’s most pressing social challenges, including the impact of human biases. But that will only be possible if people trust these systems to produce unbiased results. AI can help humans with bias — but only if humans are working together to tackle bias in AI.”
References:
- Thomas H. Davenport, Jeanne G. Harris (2017). Competing on Analytics: New Science of Winning. Harvard Business Review Press.
- Michael E. Porter and James E. Heppelmann (2019). HBR’s 10 Must-Read on AI, Analytics, and the New Machine Age. Harvard Business Review.
- Cathy O’Neil (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Penguin Books Limited.
- Coded Bias. Directed by Shalini Kantayya (2020). Netflix: watch here or here for the Educational Discussion Guide
- Iterative AI deployment by Deeplearning.ai
