Clustering is an important category of machine learning methods and a main form of unsupervised learning. Clustering is essentially distinctive and substantially different from the other dominant form of machine learning, classification, in that it does not rely on training (supervised learning). In principle, clustering represents any method that tries to identify and distinguish groups […]

## Posts tagged math

## Matrix-based implementation of neural...

Of the most basic forms of a machine learning system based on neural networks is the one in which training is accomplished using back error propagation, or simply back-propagation. Back-propagation is a technique, like many others, that targets the minimisation of a cost function during a learning process by following the descending gradient of the […]

## Logistic regression for multi-class c...

Machine learning is a research domain that is becoming the holy grail of data science towards the modelling and solution of science and engineering problems. I have already witnessed researchers proposing solutions to problems out of their area of expertise using machine learning methods, basing their approach on the success of modern machine learning algorithm […]

## Expectation Maximization for gaussian...

This post serves as a practical approach towards a vectorized implementation of the Expectation Maximization (EM) algorithm mainly for MATLAB or OCTAVE applications. EM is a really powerful and elegant methods for finding maximum likelihood solutions in cases where the hypothesis involves a gaussian mixture model and latent variables. Introduction EM is connected with the […]

## Differentiation cheat sheet

This post serves as a cheat sheet for differentiation. It just includes the most basic of the rules to be remembered when computing derivatives. First a little reminder on the notation being used in differentiation. Leibniz’s notation for the first derivative of a function f(x) is Newton’s notation for the first derivative of a function […]

## On critical points of functions

There is a really large number of applications in engineering, in which the identification of critical points of a function is crucial for the analysis and modeling of a process or a system. Generally, one may identify three kinds of critical points, that is (a) maxima, (b) minima and (c) saddle or stationary inflection points. […]