• objective function or criterion - some function 𝑓(𝑥) we want to minimize or maximize
  • cost function or loss function or error function - an objective function 𝑓(𝑥) we want to minimize
• critical points or stationary points - are points where 𝑓’(𝑥) = 0
  • local minimum - a critical point 𝑥 where 𝑓(𝑥) is lower than at all neighboring points, so it is no longer possible to decrease 𝑓(𝑥) by making infinitesimal steps
  • local maximum - a critical point 𝑥 where 𝑓(𝑥) is higher than at all neighboring points, so it is no longer possible to increase 𝑓(𝑥) by making infinitesimal steps
  • saddle point - a critical point that is neither maxima nor minima
  • global minimum/maximum - a critical point 𝑥 that obtains the absolute lowest/highest value of 𝑓(𝑥)