Presentations

Note: The following presentations were given at conferences where no paper was published in the proceedings. For conference presentations with an associated proceedings paper, see the publications page.

I. McInerney, M. Wang, B. Stellato, V. Bansal, and A. Solomon, “OSQP with GPUs & FPGAs: Accelerating quadratic programming on heterogeneous systems,” presented at the 2024 INFORMS Annual Meeting, Seattle, WA, USA, Oct. 2024.

The rise of machine learning has led to increased availability of accelerators, including GPUs in workstations and cloud services like AWS offering GPUs and FPGAs. However, traditional optimization solvers have largely not leveraged this computational power. In this talk, we present our recent work in the OSQP (Operator Splitting Quadratic Programming) 1.0 release, which implements end-to-end solver acceleration on both GPU and FPGA platforms. At the core of this acceleration is a linear algebra abstraction layer that decouples the core optimization steps from numerical operations, enabling a unified OSQP solver API regardless of the computational backend. This layer allows for seamless backend selection at build-time, including CPU backends with sparse operations or Intel MKL, a GPU backend using cuSPARSE, and an FPGA backend using RSQP. OSQP libraries with these backends are packaged with high-level interfaces, such as the OSQP Julia and Python packages, enabling users to easily solve quadratic optimization problems on GPUs.
```
@inproceedings{McInerney2024_INFORMS,
  address = {Seattle, WA, USA},
  author = {McInerney, Ian and Wang, Maolin and Stellato, Bartolomeo and Bansal, Vineet and Solomon, Amit},
  event = {2024 INFORMS Annual Meeting},
  title = {OSQP with GPUs & FPGAs: Accelerating quadratic programming on heterogeneous systems},
  month = oct,
  year = {2024},
  author+an = {1=highlight}
}
```
I. McInerney, V. Bansal, P. Goulart, and B. Stellato, “Recent Advances in the OSQP Solver: Differentiable Optimization, Accelerated Linear Algebra, and More,” presented at the SIAM Conference on Optimization, Seattle, WA, USA, May 2023.
Slides

OSQP is a fast, light, and widely popular open-source solver for quadratic optimization based on the alternating direction method of multipliers. We present the latest changes, improvements, and additions in OSQP 1.0. We introduce a new modular linear algebra framework to seamlessly run OSQP on three different platforms: default single-threaded for embedded applications, the Intel Math Kernel Library for multicore CPUs, and CUDA for GPUs. We discuss new OSQP code generation routines to create a light embeddable version of the solver. In contrast to prior OSQP code generation, our new implementation runs directly in C and can be called from any high level OSQP interface, including Python, Matlab, and Julia. In addition, OSQP now supports differentiable optimization and it can efficiently compute, directly in C, the derivatives of its solution with respect to the problem parameters. With convenient wrappers in JAX and Pytorch, OSQP is, to our knowledge, the first solver with internal automatic differentiation capabilities for the optimization problem itself. We illustrate the benefits of the new functionalities in OSQP on various numerical examples.
```
@inproceedings{McInerney2023_SIOPT,
  address = {Seattle, WA, USA},
  author = {McInerney, Ian and Bansal, Vineet and Goulart, Paul and Stellato, Bartolomeo},
  event = {SIAM Conference on Optimization},
  title = {Recent Advances in the OSQP Solver: Differentiable Optimization, Accelerated Linear Algebra, and More},
  month = may,
  year = {2023},
  author+an = {1=highlight}
}
```
I. McInerney and N. J. Higham, “Chopblas: Simulating Mixed-Precision and Stochastically Rounded Linear Algebra,” presented at the SIAM Conference on Computational Science and Engineering, Amsterdam, NL, Mar. 2023.
Slides

In this talk, we present the ChopBLAS MATLAB library for simulating basic linear algebra operations using mixed precision and non-standard floating-point formats such as BFloat16, stochastic rounding, and custom formats. Existing simulation frameworks use MATLAB classes and operator overloading to implement custom precision data types, which makes implementing mixed-precision computations difficult and leads to decreased computational performance due to the overhead of MATLAB classes. Instead, ChopBLAS uses operation-level rounding, where instead of a custom data type, we store all data as double precision floating-point values, use double precision floating-point arithmetic, and then round the data to the desired precision using either the chop or cpfloat rounding function. By operating on double precision values we are able to exploit the built-in vectorization capabilities of MATLAB operations, leading to a speed-up of nearly 90x and 1050x when simulating stochastic rounding using chop and cpfloat, respectively, compared to a reference BLAS implementation in MATLAB with operation level rounding.
```
@inproceedings{McInerney2023_CSE,
  address = {Amsterdam, NL},
  author = {McInerney, Ian and Higham, Nicholas J.},
  event = {SIAM Conference on Computational Science and Engineering},
  title = {Chopblas: Simulating Mixed-Precision and Stochastically Rounded Linear Algebra},
  month = mar,
  year = {2023},
  author+an = {1=highlight}
}
```

I. McInerney, E. C. Kerrigan, and G. A. Constantinides, “Circulant Preconditioning of the Fast Gradient Method for Predictive Control,” presented at the 7th IMA Conference on Numerical Linear Algebra and Optimization, Birmingham, UK, Jun. 2022.

Extended Abstract Slides

@inproceedings{McInerney2022_IMA,
  address = {Birmingham, UK},
  author = {McInerney, Ian and Kerrigan, Eric C. and Constantinides, George A.},
  event = {7th IMA Conference on Numerical Linear Algebra and Optimization},
  title = {Circulant Preconditioning of the Fast Gradient Method for Predictive Control},
  month = jun,
  year = {2022}
}

I. McInerney, E. C. Kerrigan, and G. A. Constantinides, “Closed-Form Preconditioner Design for Linear Predictive Control,” presented at the 21st IFAC World Congress, Berlin, Germany, Jul. 2020.
Extended Abstract Institutional Repository Slides Video

Model Predictive Control (MPC) with linear models and constraints is extensively being utilized in many applications, many of which have low power requirements and limited computational resources. In these resource-constrained environments, many designers choose to utilize simple iterative first-order optimization solvers, such as the Fast Gradient Method. Unfortunately, the convergence rate of these solvers is affected by the conditioning of the problem data, with ill-conditioned problems requiring a large number of iterations to solve. In order to reduce the number of solver iterations required, we present a simple closed-form method for computing an optimal preconditioning matrix for the Hessian of the condensed primal problem. To accomplish this, we also derive spectral bounds for the Hessian in terms of the transfer function of the predicted system. This preconditioner is based on the Toeplitz structure of the Hessian and has equivalent performance to a state-of-the-art optimal preconditioner, without having to solve a semidefinite program during the design phase.
```
@inproceedings{McInerney2020_IFAC,
  address = {Berlin, Germany},
  author = {McInerney, Ian and Kerrigan, Eric C. and Constantinides, George A.},
  event = {21st IFAC World Congress},
  title = {Closed-Form Preconditioner Design for Linear Predictive Control},
  month = jul,
  year = {2020}
}
```
I. McInerney, E. C. Kerrigan, and G. A. Constantinides, “Bounding computational complexity under cost function scaling in predictive control,” presented at the International Conference on Industrial and Applied Mathematics (ICIAM), Valencia, Spain, Jul. 2019.
Slides

We present an upper bound for the computational complexity of first-order optimization algorithms for constrained LQR problems with state, input and cross-term weights. We derive horizon-independent bounds on the condition number and extremal eigenvalues of the Hessian matrices via a system-theoretic analysis of appropriate Toeplitz operators. Comparing the iteration bounds with the closed-loop performance shows that small improvements in performance come at the expense of greatly increased computational cost.
```
@inproceedings{McInerney2019_ICIAM,
  address = {Valencia, Spain},
  author = {McInerney, Ian and Kerrigan, Eric C. and Constantinides, George A.},
  event = {International Conference on Industrial and Applied Mathematics (ICIAM)},
  title = {Bounding computational complexity under cost function scaling in predictive control},
  month = jul,
  year = {2019}
}
```

I. McInerney, E. C. Kerrigan, and G. A. Constantinides, “Modeling round-off error in the fast gradient method for predictive control,” presented at the 28th Biennial Numerical Analysis Conference, Glasgow, Scotland, UK, Jun. 2019.

Extended Abstract Slides

@inproceedings{McInerney2019_NACONF,
  address = {Glasgow, Scotland, UK},
  author = {McInerney, Ian and Kerrigan, Eric C. and Constantinides, George A.},
  event = {28th Biennial Numerical Analysis Conference},
  title = {Modeling round-off error in the fast gradient method for predictive control},
  month = jun,
  year = {2019}
}

Copyright disclaimer: The documents contained in this page are included to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.

Ian McInerney

Ian McInerney

Presentations