My Progress

1. Details of training #Step 1 : Specify how to compute output given x and parameters w, b (define model)#Math : f_w,b(x) = ?#Logistic Regressionz = np.dot(w,x) + b#Step 2 : Specify loss and cost#Math: L(f_w,b(x), y)# J(w,b) = 1/m sigma L (f_w,b(x(i)), y[i])f_x = 1 / (1 + np.exp(-z))loss = -y * np.log(f_x) - (1 - y) * np.log(1 - f_x)#Train on data to minimize J(w,b)w = w - alpha * dj_dwb = b- ..

1. Neurons and the brain Origins: Algorithms that try to mimic the brain Development RoadMap: speech -> images -> text(NLP) -> ... Since the development of Large Neural Network and GPU, we could perform many different task using the "Big Data". 2. Example - Demand Prediction We use sigmoid function to solve the classification problem. We called the function we use f(x). In the neural network, we..

This is the overview of supervised learning. Based on the characteristics of problems, we can divide problems into regression and classification. To solve the problem, we use linear regression for regression and logistic regression for classification problem. We have to get the best linear or logistic regression formula that fits the datasest. The way to find the formula is the cost function. By..

1. Intuition Regression Example Underfit / High bias : does not fit the training set well Overfit / High variance: Fits the training set extremely well Generalization: Fits training set pretty well Even though overfitting example might have a high accuracy on the training set, it will not have a high accuracy for new test dataset. 2. Addressing overfitting Option 1: Collecting more examples By c..

1. Cost function1.1 IntuitionWe use logistic regression / sigmoid function to estimate the data's label or category. How do we choose w and b?For linear regression, we used squared error cost.Linear regression is a convex form, so we could use standard gradient descent equation to figure out local minimum. Since logistic regression is a non-convex form, it has multiple local minimum. Thus, we ca..

1. IntuitionLinear Regression is not a good method for Classification Problem.Why?One outlier on the right changes the linear regression function. It moves decision boundary dramatically. 2. Logistic Regression2.1 FormulaLogistic function allows the function to be curved unlike the linear regression. This is a sigmoid function aka logistic function. x-axis represents the number z. It outputs a v..

1. What is Feature Engineering? Using intuition to design new features, by transforming or combinging original features. Example) House price. If we are given the dimension of the house such as length and width, we can create a new variable to include for price prediction of a house. For example, we create a area variable with length and width. 2. Feature engineering in Polynomial Regression We ..

1. Checking Gradient descent for convergence How to check if Gradient descent is working well? 1.1 Graph This graph is called Learning curve. As the iteration increases, the minimum cost should decrease. If the graph remains constant after enough iterations, we call that it has converged 1.2 Epsilon / Automatic convergence test Let epsilon be 0.001 If the cost function decreaes by epsilon in one..

1. Intuition House example: Lets say we are predicting the price of a house based on its size and number of bed rooms. Number for size of a house is relatively larger than the number of bed rooms. This large difference between numbers will make it hard to accurately predict the price. How is this related to gradient descent? Since the number for size is large, it takes smaller w value to make a ..

1. Intuition Instead of prediction house price with only the size of a house, we can use multiple features of a house to make a better prediction This is a standard notation for multiple linear regression W and X can be represented in the form of vectors. This is the other format of writing multiple linear regression equation. Multiplication of vectors are called "Dot Product" 2. Vectorization D..