ASSET Seminar: Lockout: Sparse Regularization of Neural Networks (Gilmer Valdes, UCSF)

PRESENTATION ABSTRACT 

Many regression and classification procedures fit a function f(x;w) of predictor variables x to data 〖{x_i,y_i}〗_1^N based on some loss criterion L(y,f(x;w)). Often, regularization is applied to improve accuracy by placing a constraint P(w)≤t on the values of the parameters w, where P is a monotonic increasing function of the absolute values of the parameters (e.g. Lasso, Ridge). Although efficient methods exist for finding solutions to these constrained optimization problems for all values of t≥0 in the special case when f is a linear function, none are available when f is non-linear (e.g. Neural Networks). Here we present a fast algorithm that provides all such solutions (path) for any differentiable function f and loss L, and any differentiable constraint P that is an increasing monotone function of the absolute value of each parameter. Applications involving sparsity inducing regularization of arbitrary Neural Networks are discussed. Empirical results indicate that these sparse solutions are usually superior to their dense counterparts in both accuracy and interpretability (sometimes strikingly better). This improvement in accuracy can often make Neural Networks competitive with, and sometimes superior to, state-of-the-art methods in the analysis of tabular data. Specific applications to Medicine, in particular, Microarray data are discussed.

BIO

Dr. Gilmer Valdes is an associate professor with appointments in the Departments of Radiation Oncology, Epidemiology and Biostatistics at UCSF and in the joint program in Computer and Precision Health between UCSF and UCB. After completing his doctoral training in medical physics at UCLA, Dr. Valdes completed a fellowship and clinical residency in Therapeutic Medical Physics at UPENN. His work during early training focused on development of machine learning algorithms and advanced modeling to predict patient outcomes, tailor treatments based on the type and quality of data available (particularly for prostate cancer patients), and more recently, on development of methods that permit inspection and interpretation of machine learning models (MediBoost, The Additive Tree, Expert Augmented Machine Learning, Lockout). Finally, he was fortunate to have been awarded an NI BIBK08 career award to support the development of accurate and interpretable machine learning algorithms for their application to medicine.