Speakers

Many high-dimensional estimation problems involve matrices whose effective complexity is much smaller than their ambient dimension, but whose rank is unknown a priori. In this talk, I will present a rank-adaptive approach based on singular-value thresholding, which automatically balances the statistical cost of estimating additional singular directions against the approximation error of discarding them.

I will first discuss near-optimal guarantees and instance-dependent limits for general high-dimensional matrix estimation, and then show how the same principle leads to a rank-adaptive system identification method based on a thresholded Ho–Kalman algorithm. The resulting procedure can recover the system order and estimate its dynamics without prior knowledge of the order, while achieving finite-sample guarantees comparable to methods that know it in advance.

Joint work with Yassir Jedra (Imperial College) and Frederic Zheng (KTH).

The term "dual control" was introduced by Feldbaum in 1960 to describe the tradeoff between exploration and exploitation in the context of systems whose parameters are initially unknown and must be learned by active probing.

The lecture will compare recent work on minimax dynamic games for dual control with two dominating branches of past literature: self-tuning adaptive control and statistical regret minimization. In particular, we will show that Feldbaum's hyperstate, which combines the physical state with an information state, has a particularly clean representation in the minimax case, enabling explicit expressions for optimal dual controllers.

Many control problems are high-dimensional decisions that must respect hard constraints and be solved in seconds: rebalancing a fleet, routing inventory through a distribution network, keeping a power system within limits, driving a robot to a goal pose, etc. End-to-end reinforcement learning is fast at inference but does not, by itself, produce feasible actions. Optimization does, when the problem is feasible, but formulations that also model stochastic, non-linear dynamics get expensive over long horizons. In this talk, I describe a hierarchical alternative: a learned policy specifies a desired next state, and a convex program computes the action that best achieves it under the operational constraints. In our experiments, this is markedly more sample-efficient than end-to-end RL, returns actions that satisfy the constraints we encode, and ensures that graph policies transfer across network sizes and topologies. However, learning offline is more challenging, since the desired next state is typically not recorded, and the logs may come from a controller that was never hierarchical. To address this, we propose OHIO (Offline Hierarchical RL via Inverse Optimization) - a framework that uses the known structure of the lower-level optimizer to reconstruct the targets that explain the logged actions, or the logged transitions where actions are absent, yielding a dataset for off-the-shelf offline training. I will show results on network optimization and robotic problems, including online fine-tuning, and close on identifiability and on what a per-step feasibility guarantee does not say about the trajectory.

In this talk, I will discuss a series of results on Probably Approximately Correct (PAC)-Bayes learning for stochastic dynamical systems, focusing mainly on linear time-invariant (LTI) stochastic systems in state-space form with inputs. The results extend PAC-Bayes analyses beyond static models and independent data to dynamical systems learned from non-i.i.d. time series. I will present generalization bounds whose dependence on the system’s memory is captured through mixing coefficients and which converge to zero as the amount of data increases. I will also consider partially observed LTI systems and show how PAC-Bayes bounds can provide finite-sample guarantees for both prediction and parameter estimation. These results offer a step toward PAC-Bayes learning guarantees for recurrent neural networks.

In this talk we discuss the standard RL algorithms DQN and PPO. We discuss new ways of interpreting the algorithms that allow us to derive convergence results in the tabular setting but also help to better understand those algorithms.

Most algorithms in online learning either assume that the environment is stationary or that it is adversarial. The latter guarantees ultimate robustness, whereas the former exploits the simplicity to achieve faster convergence to the optimal policy. But what if it is unknown whether the environment is stationary or adversarial? Or what if it only somewhat deviates from stationarity? I will present best-of-both-worlds online learning algorithms, which guarantee robustness against adversarial environments while simultaneously achieving faster convergence to optimality when environments happen to be stationary, with no need of prior knowledge about the nature of the environment. They also provide graceful interpolation guarantees for a range of intermediate environments. I will also present some open questions concerning trade-offs between restricting modeling complexity of an environment and making assumptions on its stationarity.