Skip to content
Open

my pull #1488

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
f2e195f
changed website name
praveensplayground Aug 1, 2023
dea1344
my post
praveensplayground Aug 1, 2023
9692c4a
added thunder.jpg
praveensplayground Aug 1, 2023
9fcf9df
Update 2023-08-01_why models fail.md
praveensplayground Aug 1, 2023
0fafc57
Update 2023-08-01_why models fail.md
praveensplayground Aug 1, 2023
7a972bf
Update index.html
praveensplayground Aug 1, 2023
517a5db
Rename 2023-08-01_why models fail.md to 2023-08-01-why models fail.md
praveensplayground Aug 1, 2023
04d9ecf
changed subtitle
praveensplayground Aug 1, 2023
adc484b
changed avatar
praveensplayground Aug 1, 2023
8a3dbf3
avatar pe change
praveensplayground Aug 1, 2023
72483cd
Update 2023-08-01-why models fail.md
praveensplayground Aug 1, 2023
0f81509
Update aboutme.md
praveensplayground Aug 4, 2023
08d7bdd
Update aboutme.md
praveensplayground Aug 4, 2023
4cb6fd5
economic methods
praveensplayground Aug 5, 2023
268bfcb
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 5, 2023
ed0b19a
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 5, 2023
a081acc
Add files via upload
praveensplayground Aug 5, 2023
39a66a2
Add files via upload
praveensplayground Aug 5, 2023
5211c03
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 5, 2023
d5480d1
Add files via upload
praveensplayground Aug 5, 2023
1fd80e2
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 5, 2023
8f2fd18
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 5, 2023
6077e42
Add files via upload
praveensplayground Aug 12, 2023
b769118
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
4cd13d2
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
c0b27bf
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
8bd5b95
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
5774464
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
ee4e414
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
a8dcff5
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
4662e47
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
8a217d4
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
d8a5a51
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
af0835a
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
9213a5e
Add files via upload
praveensplayground Aug 12, 2023
6363fae
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
489b5ec
Update 2023-08-05-econometric_methods_impact_assesment.md
praveensplayground Aug 12, 2023
c1632d0
Add files via upload
praveensplayground Aug 13, 2023
69f6ace
Update 2023-08-15-the_beauty_of_boosting.md
praveensplayground Aug 13, 2023
f54a58d
pic upload
praveensplayground Aug 13, 2023
0e475b3
Update 2023-08-15-the_beauty_of_boosting.md
praveensplayground Aug 13, 2023
77bbc89
Update 2023-08-15-the_beauty_of_boosting.md
praveensplayground Aug 13, 2023
42e30cc
Update _config.yml
praveensplayground Aug 14, 2023
c280cb0
Update _config.yml
praveensplayground Aug 14, 2023
9f42c97
Update _config.yml
praveensplayground Aug 19, 2023
db85586
Update _config.yml
praveensplayground Aug 20, 2023
42d66ab
Update aboutme.md
praveensplayground Aug 20, 2023
bb99ad1
Update aboutme.md
praveensplayground Aug 20, 2023
fbc333f
Update aboutme.md
praveensplayground Aug 20, 2023
46b302d
Update aboutme.md
praveensplayground Aug 20, 2023
0b514e7
Update aboutme.md
praveensplayground Aug 20, 2023
637741b
Update aboutme.md
praveensplayground Aug 20, 2023
e23e70e
Update aboutme.md
praveensplayground Aug 20, 2023
bc18701
Update aboutme.md
praveensplayground Aug 20, 2023
67668e6
Update aboutme.md
praveensplayground Aug 20, 2023
04deb89
Update aboutme.md
praveensplayground Aug 21, 2023
8b6bce0
Update _config.yml
praveensplayground Aug 30, 2023
a6327e1
Update _config.yml
praveensplayground Aug 30, 2023
98a31de
Update aboutme.md
praveensplayground Aug 30, 2023
5266a67
Update aboutme.md
praveensplayground Aug 30, 2023
19c5dbf
Update aboutme.md
praveensplayground Aug 30, 2023
35d0d49
Update aboutme.md
praveensplayground Aug 30, 2023
ac577fe
Rename 2020-02-26-flake-it-till-you-make-it.md to 2020-50-50-flake-it…
praveensplayground Sep 24, 2023
17788fa
Rename 2020-02-28-sample-markdown.md to 2020-50-50-sample-markdown.md
praveensplayground Sep 24, 2023
602f901
Add files via upload
praveensplayground Oct 14, 2023
1f04434
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
dc2f91c
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
aa93825
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
62ba9ec
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
1d3b297
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
72e8807
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
c1f8dfc
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
2882502
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
afb7c2f
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
8e356a7
add files
praveensplayground Oct 14, 2023
3449324
add
praveensplayground Oct 14, 2023
d82ddc9
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 14, 2023
3c9a601
Rename 2020-50-50-flake-it-till-you-make-it.md to 2020-12-10-flake-it…
praveensplayground Oct 14, 2023
0e4b311
Rename 2020-50-50-sample-markdown.md to 2020-12-10-sample-markdown.md
praveensplayground Oct 14, 2023
af8cf42
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 15, 2023
3f9812a
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 15, 2023
3d37300
Update 2023-10-10-make unit testing your friend.md
praveensplayground Oct 21, 2023
a3979d2
Update _config.yml
praveensplayground Nov 5, 2023
b25af34
Add files via upload
praveensplayground Jun 18, 2025
9e6c2c9
Add files via upload
praveensplayground Jun 18, 2025
53d6816
Add files via upload
praveensplayground Jun 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,17 @@
############################

# Name of website
title: My Website
title: Hello, I am Praveen

# Your name to show in the footer
author: Some Person
author: Praveen Ekanayake

###############################################
# --- List of links in the navigation bar --- #
###############################################

navbar-links:
About Me: "aboutme"
My Projects: "aboutme"
Resources:
- Beautiful Jekyll: "https://beautifuljekyll.com"
- Learn markdown: "https://www.markdowntutorial.com/"
Expand All @@ -32,7 +32,7 @@ navbar-links:

# Image to show in the navigation bar - works best with a square image
# Remove this parameter if you don't want an image in the navbar
avatar: "/assets/img/avatar-icon.png"
avatar: "/assets/img/avatar-icon-pe.png"

# By default, the image is cut into a circle. You can disable this behaviour by setting 'round-avatar: false'
round-avatar: true
Expand All @@ -50,22 +50,22 @@ round-avatar: true
# You can change the order that they show up on the page by changing the order here.
# Uncomment the links you want to show and add your information to each one.
social-network-links:
email: "someone@example.com"
rss: true # remove this line if you don't want to show an RSS link at the bottom
facebook: deanattali
github: daattali
twitter: daattali
patreon: DeanAttali
youtube: c/daattali
email: "praveen.ekanayake@gmail.com"
rss: false # remove this line if you don't want to show an RSS link at the bottom
# facebook: deanattali
github: https://github.com/praveensplayground
# twitter: daattali
# patreon: DeanAttali
# youtube: c/daattali
# medium: yourname
# reddit: yourname
# linkedin: daattali
linkedin: https://www.linkedin.com/in/praveen-ekanayake-58793541/
# xing: yourname
# stackoverflow: "3943160/daattali"
# snapchat: deanat78
# instagram: deanat78
# spotify: yourname
# telephone: +14159998888
telephone: +947171 00 888
# steam: deanat78
# twitch: yourname
# yelp: yourname
Expand All @@ -89,14 +89,14 @@ rss-description: This website is a virtual proof that I'm awesome

# Select which social network share links to show in posts
share-links-active:
twitter: true
facebook: true
twitter: false
facebook: false
linkedin: true
vk: false

# How to display the link to your website in the footer
# Remove this if you don't want a link in the footer
url-pretty: "MyWebsite.com"
# url-pretty: "MyWebsite.com"

# Excerpt word length - Truncate the excerpt of each post on the feed page to the specified number of words
excerpt_length: 50
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ The truth is that no one else can definitively know the path we are here to walk

At the heart of the struggle are two very different ideas of success—survival-driven and soul-driven. For survivalists, success is security, pragmatism, power over others. Success is the absence of material suffering, the nourishing of the soul be damned. It is an odd and ironic thing that most of the material power in our world often resides in the hands of younger souls. Still working in the egoic and material realms, they love the sensations of power and focus most of their energy on accumulation. Older souls tend not to be as materially driven. They have already played the worldly game in previous lives and they search for more subtle shades of meaning in this one—authentication rather than accumulation. They are often ignored by the culture at large, although they really are the truest warriors.

A soulful notion of success rests on the actualization of our innate image. Success is simply the completion of a soul step, however unsightly it may be. We have finished what we started when the lesson is learned. What a fear-based culture calls a wonderful opportunity may be fruitless and misguided for the soul. Staying in a passionless relationship may satisfy our need for comfort, but it may stifle the soul. Becoming a famous lawyer is only worthwhile if the soul demands it. It is an essential failure if you are called to be a monastic this time around. If you need to explore and abandon ten careers in order to stretch your soul toward its innate image, then so be it. Flake it till you make it.
A soulful notion of success rests on the actualization of our innate image. Success is simply the completion of a soul step, however unsightly it may be. We have finished what we started when the lesson is learned. What a fear-based culture calls a wonderful opportunity may be fruitless and misguided for the soul. Staying in a passionless relationship may satisfy our need for comfort, but it may stifle the soul. Becoming a famous lawyer is only worthwhile if the soul demands it. It is an essential failure if you are called to be a monastic this time around. If you need to explore and abandon ten careers in order to stretch your soul toward its innate image, then so be it. Flake it till you make it.
43 changes: 43 additions & 0 deletions _posts/2023-08-01-why models fail.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
layout: post
title: Why Machine Learning Models Fail
subtitle: Steering Clear of Machine Learning Misstep
cover-img: /assets/img/thunder.jpg
thumbnail-img: /assets/img/thunder.jpg
share-img: /assets/img/path.jpg
tags: [books, test]
---

It is often said that most data science projects fail to achieve the value they promised to deliver. Gartner, technological research and consulting company says that almost half of Chief Information Officers plan to deploy AI and machine learning in their companies. However, they predict that 85 percent of AI projects will deliver erroneous outcomes due to bias in data, algorithms or the teams responsible for managing them. This may sound as a very disheartening message to data scientists and any company looking forward to adopting data science in their businesses. However, this does not occur due to the weaknesses in machine learning itself. This occurs due to how machine learning is implemented. This can be avoided if the practitioners are careful.

## Lack of good initiation process
Machine learning is an exciting technology for any company. The inclination would be to apply them as soon as possible without thorough assessment of feasibility. They assume that the problems that come in their way will take care of themselves. Such approaches often lead to unrealized value from machine learning. Before any analytics is carried out the practitioners should answer some fundamental questions to ensure the value is derived. First, make sure the ROI is measurable and justifiable. There should be clear sources of value and the business process in place to execute the interventions. Secondly, answer if the use case can be scaled to different parts of the business. If the current use-case can be scaled, the value generated from an analytics intervention can be doubled. Therefore laying strong ground-work will invariably lead to solid results.

## Machine learning specific failures
The underlying premise of machine learning is that a statistical model will successfully be able to learn the variations of a training dataset and apply it to the real world. We assume that the model’s performance on test data is an accurate representation of how the model will perform on unseen, real world data. This assumption is fundamentally incorrect. The distribution of training data is unlikely to be similar to that of testing data. The training data is a finite sample of observations whereas the real-world data is virtually infinite. It is multi-faceted and there are infinite number of transformations. The divergence could be due to a simple change in HTML encoding, resulting in erroneous web-scraped data that is fed into a model. In this situation when we use the same feature engineering code, our models fail or predictions will be inaccurate. This error is known as the “Train-serving skew”.
The training and testing datasets are time-bound whereas the real-world data is not. The real-world is often unpredictable and often affected by unforeseen events. For example, when someone from the United States searched for Ukraine, this could have been to retrieve travel information. Since February of 2021, a search for Ukraine could be to know more about a conflict between two countries. An online personalized recommendation engine could easily suffer in such a scenario. If the model is already in production, it could be giving sub-par results. These divergences from train and test data need to be constantly monitored so long as the model lives in production.

## Long feedback loops
It is very important to have some feedback about how your deployed model is working in the real world. A data scientist would ideally want to know about the performance of the model in order to update the model. In some instances it is very straightforward and the feedback is received instantly. For example, if you deployed a model to forecast the time taken to get to a location in a ride hailing application, the actual time taken becomes clear instantly when passengers complete the trip. When you have deployed an online advertisement recommendation system, you will know whether the respondent liked the advertisement instantly when they click on it. However, in some cases the feedback systems are very long or virtually non-existent. For example, if you have deployed a fraud detection system, it will be very long before the activities are proven to be actually fraudulent. In such instances getting a feedback is never guaranteed and depend on many external circumstances. Perhaps an even better example is how inaccurate Google Translate is. The users of Google Translate have to endure comically inaccurate translations because in most cases the creators of the algorithm may never know how accurate their translations are. As most users are new to the language thus unaware whether the translation is accurate, there is very little room to create a good feedback loop. For these reasons, after years of in existence, Google Translate still is painfully inaccurate. Therefore, having too long feedback loops lead to machine learning models failing silently.

## Edge cases
Imagine that you have developed an algorithm to identify tumors. You have an impressive 99.9% accuracy. However, the 0.1% times you misclassify a malignant tumor as benign which could prove fatal to a patient. A hospital will be reluctant to use such an algorithm no matter how impressive the numbers are. This phenomenon is often explained in self-driving cars too. Imagine there is a self-driving car where it would drive safely in 99.99% of the times but could bring catastrophic results in the rest of the instances. Although this level of accuracy is better than the performance of an average human, the miniscule possibility of a catastrophe could be very unappealing to the prospective buyers. This is not common only to human safety related algorithms. The performance on edge cases could prove irksome in other less threatening applications too. For example, a company produced a virtual personal assistant which uses NLP technology to recognize human speech and provide assistance to people. Although it is impressively accurate, it occasionally produces racist or sexist comments. Such ML products could go out of business for a 0.01% failure.
It is understood that the only way to solve this problem is to better the model’s performance. The self-driving car companies are trying to improve the accuracy on edge cases. In order to reduce the misclassification, the IA practitioners are using multiple models to get to an accurate decision.

## The usual suspects
There are many common mistakes any analyst should be careful about. These are the errors that can be avoided if we can be more vigilant in our day to day work. For example, dependencies of a model not being present in the running environment or an earlier version of a model being deployed. The failures in the ML platform due to a runtime error also result in many model failures. Although these errors may sound trivial, these are the errors that account to a large majority of all the ML failures.

## Conclusion
The above only highlight some of the common failures and there are many other ways that cause ML systems to fail. These impediments can be avoided in many ways. For example, following proper best practices help overcome many oversights related to model deployment. However, for many companies data science is a relatively new field and best practices are yet to be properly established. Many organizations are increasingly focused on laying good groundwork and co-creating ML solutions with the businesses. When it comes to edge cases, it has proven extremely difficult to avoid them especially in autonomous vehicles. However, the companies are always researching to improve the performance in corner cases and provide the best outcome for its customers. The broken feedback loops can be mended in many ways. For example, Google Translate has placed a great emphasis on allowing reviewers to provide the ideal translation when the translations are off. This helps improve their translation by mending the feedback loop. These developments have been promising and it will improve the perception of machine learning for the consumers in the years to come.

( _This post originally appeared in Medium account of Octave-John Keells_ )

Further readings:

https://snit.ch/blog/why-gartner-says-your-machine-learning-models-will-fail/

https://huyenchip.com/2022/02/07/data-distribution-shifts-and-monitoring.html

https://support.google.com/translate/answer/2534530?hl=en

https://dataloop.ai/blog/autonomous-edge-case/
Loading