Google Scholar
Fred Roosta 

Fred Roosta

 

  • Associate Professor
  • School of Mathematics and Physics, University of Queensland, Australia
  • Email: fred.roostauq.edu.au
  • Office: Priestley Building (67) - Room 754
  • Phone: +61 7 336 53259

News

2024

  • February, 2024:  Our paper on an adaptive gradient descent appeared in the Transactions on Machine Learning Research (TMLR).
  • January, 2024: Our new paper investigates conjugate direction methods, such as conjugate gradient (CG) and conjugate residual (CR), across a wide range of settings, including inconsistent ones. In particular, we demonstrate that the widely celebrated but more complex MINRES method is essentially equivalent to the much simpler CR method!
  • January, 2024: In our new paper, we developed a new efficient algorithm for approximating leverage scores of a tall matrix, with applications to big time series data.

2023

  • December, 2023: I will be serving as an area chair for the 41st International Conference on Machine Learning (ICML, 2024).
  • November, 2023:  A new paper on a variant of gradient descent in which the step-size is automatically tuned using a novel local first-order smoothness oracle, which generalizes the Lipschitz continuous gradients smoothness condition.
  • November, 2023:  A new paper on PAC-Bayesian bound for a general class of models, characterizing factors which influence generalization performance in the interpolating regime.
  • October, 2023:  I will be serving on the program committee of the 2025 SIAM Annual Meeting, to be held in Montreal, Canada, July 26 - August 2, 2025.
  • October, 2023: In this paper, we show that for inconsistent linear systems, it is possible to obtain the pseudo-inverse solution in a variety of settings using a straightforward one-dimensional projection method.
  • September, 2023: I will be serving as an area chair for the 12th International Conference on Learning Representations (ICLR 2024).
  • August, 2023:   Our recent paper on the iteration and operation complexity analysis of non-convex Newton-MR  algorithm with inexact Hessian information is now available.
  • July, 2023:  In our new paper, we develop a (Bayesian) information criterion for interpolating overparameterized models that, among other things, captures the sharpness (flat minima) heuristic commonly used in deep learning.
  • May, 2023:  I will be serving on the program committee of the 2024 SIAM Conference on the Mathematics of Data Science (SIAM MDS), to be held October 21-25, 2024, in Atlanta, Georgia, USA.
  • May, 2023: Our paper on uncertainty quantification of deep learning models applied to oncology has appeared in Nature's Scientific Reports.
  • March, 2023: Our paper on double descent phenomenon in Gaussian processes has been accepted in ICML 2023.
  • March, 2023: I will be serving as an area chair for the 37th Conference on Neural Information Processing Systems (NeurIPS) 2023.

2022

  • December, 2022: I will be serving as an area chair for the 40th International Conference on Machine Learning (ICML) 2023.
  • October, 2022:   New paper on double descent phenomenon in Gaussian processes. 
  • September, 2022:   Congratulations to my outgoing PhD student, Yang Liu, for landing a postdoc position at the Oxford Mathematical Institute as part of the Centre for Intelligent Multidimensional Data Analysis (CIMDA). 
  • August, 2022:  Our paper on the complexity analysis of non-convex variants of Newton-MR is now available.
  • July, 2022:  Our paper on a variant of the Newton-CG method with inexact gradient and Hessian information has been accepted to IMA Journal of Numerical Analysis (IMAJNA).
  • July, 2022:  Our paper on a variant of the Newton's method with MINRES as sub-problem solver for invex optimization has been accepted to EURO Journal on Computational Optimization (EJCO).
  • June, 2022:  Our paper on negative curvature detection and monotonicity properties of MINRES has been accepted to SIAM Journal on Optimization (SIOPT).
  • April, 2022:  I will be serving on the organizing committee of the SIAM Conference on Optimization (OP23), to be held May 30-June 3, 2023, in Seattle, Washington, USA.
  • March, 2022: I will be serving as an area chair for the 36th Conference on Neural Information Processing Systems (NeurIPS) 2022.

2021

  • December, 2021: I will be serving as an area chair for the 39th International Conference on Machine Learning (ICML) 2022.
  • November, 2021: Our new paper on a novel invexifying regualrization framework for non-linear least squares problems is available.
  • September, 2021: Our new paper on complexity analysis of variants of Newton-CG algorithm with inexact gradient and Hessian for non-convex optimization is available.
  • January, 2021: With Albert S. Berahas (University of Michigan), we will have a two part mini-symposium on "Beyond First Order Methods in Machine Learning"  during the EUROPT 2021, the 18th international workshop on continuous optimization.

2020

  • July, 2020: Our paper on Newton-ADMM, a distributed GPU-accelerated second-order optimization method, has been accepted in the Proceedings of the ACM/IEEE Supercomputing Conference (SC20) - 18% acceptance rate.
  • June, 2020: Our paper on central limit theorems and concentration inequalities for out-of-sample extensions of the adjacency and Laplacian spectral embeddings has been accepted to the Journal of Machine Learning Research (JMLR), with minor revision.
  • June, 2020: Our paper on distributed non-convex Newton-type optimization methods has been accepted in ICML, 2020! Congratulations to my PhD student Rixon Crane for his second paper in an A* conference!
  • February, 2020: Our recent paper on stochastic normalizing flows, an extension of continuous normalizing flows for maximum likelihood estimation and variational inference, using stochastic differential equations is available.
  • February, 2020: Our new paper on studying Gaussian processes arising from infinitely wide neural networks with ELU and GELU activations as well as analysing the fixed-point dynamics of iterated kernels is available.
  • January, 2020: Our paper on the theory and application of the reproducing Stein kernel for a posteriori correction of Monte-Carlo sampling algorithms is now available on arXiv.
  • January, 2020: Our paper on the theoretical and practical properties of Monte-Carlo sampling algorithms by implicit discretizations of Langevin SDE has been accepted to the Journal of Machine Learning Research (JMLR), with minor revision.

2019

  • December, 2019: Our paper on an empirical study of second-order optimization methods in deep learning has been accepted to the SIAM International Conference on Data Mining (SDM20) - 19% acceptance rate.
  • December, 2019: Our new paper, which extends the existing results on Gaussian process iterpretation of multi-layer perceptrons to richer families of priors, is available.
  • November, 2019: Our new paper on the application of RandNLA and leverage score sampling within the context of big-data time series is now available.
  • November, 2019: I will be serving as a senior program committee member for the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020).
  • October, 2019: Our paper on central limit theorems and concentration inequalities for out-of-sample extensions of the adjacency and Laplacian spectral embeddings is now available.
  • September, 2019: Our paper on stability analysis of Newton-MR with respect to Hessian approximations is now available.
  • September, 2019: DINGO Does Vancouver! Our paper on distributed Newton-type methods for optimization of invex objectives has been accepted to NeurIPS 2019!
  • June, 2019: Congratulations to my PhD student, Rixon Crane, for being awarded the Best Poster Award at AMSI Optimise this year.
  • June, 2019: I was selected as a top 5% reviewer for ICML 2019.
  • May, 2019: After 17 months and 21 days, our paper on non-convex Newton-type methods with inexact Hessian information has been accepted to Mathematical Programming.
  • May, 2019: Our paper on the invariance kernels of multilayer perceptrons during training has been accepted to the International Joint Conference on Artificial Intelligence (IJCAI 2019) - 18% acceptance rate.
  • April, 2019: Our paper on the theoretical and practical properties of Monte-Carlo sampling algorithms by implicit discretizations of Langevin SDE is now available on arXiv.