Machine Learning and Algorithmic InformationThe induction problemInduction is one of the oldest philosophical problem.
And Algorithmic Information Theory solved it!
As an example, we will show how Kolmogorov complexity can be used to make analogies.
Minimum Description Length The most common way to perform induction in practice is called MDL: Minimum Description Length.
Prototype-based models and compression The central topic of this chapter is that machine learning translates into compression.
This claim will be illustrated with prototype-based models.
Conversely, compression provides a criterion that can be used to guide machine learning. It is illustrated in the context of clustering.
Anomaly detectionAnomaly detection is an important and difficult topic of AI. Anomalies will be shown to be events that are abnormally simple.
Universal induction Are you dreaming of an AI system that would beat us all?
AIT offers precisely this, at least in principle.
First as an ideal way to perform prediction, using what is called the universal distribution.
Then as an ideal way to perform action: we will briefly consider AIXI, an ideal version of reinforcement learning based on algorithmic information.