Wednesday Apr 5
11:30 AM
Room 5209
Seminar speaker: Hugo Cui, École Polytechnique Fédérale de Lausanne (EPFL)
Title: Bayes-optimal learning of deep random neural networks
Abstract: We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width are proportionally large. We derive a closed-form expression for the Bayes-optimal test error, for regression and classification tasks, and for the test error of a number of empirical risk minimization methods - including logistic/ridge regression, kernel regression, (deep) random features. We find, in particular, that optimally regularized ridge regression, as well as kernel regression, achieve Bayes-optimal performances, while the logistic loss yields a near-optimal test error for classification. We finally discuss how things differ when the number of samples is much larger than the dimension.
The seminar will be followed by lunch. Please register here to attend.
Host: Francesca Mignacco (Center for the Physics of Biological Function)