Today, machine learning methods are routinely used in high-energy particle physics to accelerate the discovery of new phenomena. Typically, standard classification algorithms are used for signal/background separation both in the online selection (trigger) step, and in the offline analysis. The first goal of this thesis to answer some of the theoretical questions raised by these unorthodox machine learning applications, and to design new algorithms that improve the analyses. In the second theme we propose to go beyond the standard setup of ``manual'' feature extraction followed by classification and to investigate the applicability of recently developed techniques on unsupervised representation learning. Both themes are motivated by concrete particle and astroparticle physics experiments, the future International Linear Collider (ILC), and the Pierre Auger Experiment (Auger). Data provided by these experiments will be a natural testbed for methodolgies developed in the thesis.