[Supervised ML] Feature scaling - 5

1. Intuition

---

House example:

 

Lets say we are predicting the price of a house based on its size and number of bed rooms. Number for size of a house is relatively larger than the number of bed rooms. This large difference between numbers will make it hard to accurately predict the price. 

 

How is this related to gradient descent? 

Since the number for size is large, it takes smaller w value to make a change. 

Meanwhile, it takes larger w value to make a change for the number of bedrooms due to its small number.

If you were to run gradient descent, it will bounce back and force, finding hard to find the minimum cost.

 

How do you solve this problem?

By tranforming and rescaling the data

 

2. Rescaling data

---

2.1 Feature Scaling

 

We can rescale the problem by dividing the number of its maximum range number.

 

2.2 Mean normalization

 

Mean normalization = x1 - mu1(average) /  max - min

 

Through mean normalizaton, we can make data to be centered at 0. 

 

2.3 Z - score normalization

We can rescale the data by using z - score or standard deviation of the data.

 

ex)

 

With the large data, it will take longer time for gradient descent.