Loss Functions For Regression Task

by jeza   Last Updated August 14, 2019 17:19 PM - source

I am trying to understand the idea of Loss functions For Regression Task perfectly.

I have read many textbooks and articles, and I came up with questions related to this subject.

Several different uses of loss functions can be distinguished.

a) In prediction problems: a loss function depending on predicted and observed value defines the quality of a prediction.

b) In estimation problems: a loss function depending on the true parameter and the estimated value defines the quality of estimation.

c) Many estimators (such as least squares or M-estimators) are defined as optimizers of certain loss functions which then depend on the data and the estimated value.

Now, since my focus is on Loss Functions For Regression Task

My questions are as follows.

1- Should I write the loss function formula as a function of the parameter or of the variables (L($\theta-\hat\theta))$ OR (L($y-\hat\y))$?

2- Should I consider the Loss function formula for one point or Not (with sums or not)?

Note

My thought is to introduce Loss function first and then to use the standard notation for all Loss functions (least square, absolute value and Huber Loss, Quntile Loss and so on).



Related Questions



How can I create a meaningful weighting for RMSE?

Updated January 11, 2019 04:19 AM

Solving an inverse problem with machine learning

Updated April 08, 2017 17:19 PM