A loss function is a part of a cost function which is a type of an objective function. All that being said, thse terms are far from strict, and depending on context, research group, background, can shift and be used in a different meaning.
What are common cost functions used in evaluating the performance of neural networks? Details (feel free to skip the rest of this question, my intent here is simply to provide clarification on nota...
machine learning - A list of cost functions used in neural networks ...
40 I am doing the Machine Learning Stanford course on Coursera. In the chapter on Logistic Regression, the cost function is this: Then, it is differentiated here: I tried getting the derivative of the cost function, but I got something completely different. How is the derivative obtained? Which are the intermediary steps?
Regularization - penalty for the cost function, L1 as Lasso & L2 as Ridge Cost/Loss Function - L1 as MAE (Mean Absolute Error) and L2 as MSE (Mean Square Error) Are [1] and [2] the same thing? or are these two completely separate practices sharing the same names? (if relevant) what are the similarities and differences between the two?
There is a subtle and practical reason why MSE is preferred over the sum of squared-errors cost function, and it has to do with iterative descent methods. More details in my answer below.
Why use MSE instead of SSE as cost function in linear regression?
I am applying change-point detection with the PELT algorithm and an RBF cost function on annual RBI banking NPA data (sample size ≈ 28). I tried with different penalty values and chose the smallest penalty (pen=3 in my case) that gave stable and interpretable breakpoints across nearby values.