The empirical spectral distribution (ESD) of the loss function of a neural network contains important qualitative information about the learning process and can be used to estimate the rate at which the loss approaches its minima. However, the ESD is typically very difficult to compute. Generalizing work of Liao and Mahoney, we have developed a recursive algorithm to estimate the ESD of a single layer network with many-dimensional output. In this talk, I will introduce the concepts underlying our research, and discuss avenues for future study. This talk is based on joint work with Aren Martinian, under the guidance of Professor Federico Pasqualotto. The research was partially funded by the SURF summer fellowship for Berkeley students.

In a general Banach algebra, or even a matrix algebra, a product of exponents need not equal the exponential of the sum. Nonetheless, the Lie-Trotter formula famously asserts that alternating products of exponentials do converge to the exponential of the sum. We show that (in many circumstances) such behavior is typical: for almost all permutations of the factors, the products of exponentials converge. In a matrix algebra, the result holds if the norms of the matrices do not grow too fast. In a general Banach algebra, it holds if n matrices fall only into o(n / log n) distinct types. The methods involve elementary estimates and concentration inequalities. The results are an outcome of undergraduate projects with Austin Pritchett and Anh Nguyen.

An important analytical tool for computing the free multiplicative convolution of two probability measures is Voiculescu's S-transform. Several works have progressively extended its domain of applicability. It was first introduced by Voiculescu (1987) for distributions with nonzero mean and compact support, and later studied by Bercovici and Voiculescu (1993) in the case of probability measures on ℝ≥0 with unbounded support. Subsequently, Raj Rao and Speicher (2007) defined an S-transform for measures with zero mean and all moments finite. Their approach is combinatorial, using Puiseux series, and cannot be extended to the general case. Arizmendi and Pérez-Abreu (2009) considered the case of symmetric measures with unbounded support. In this talk, we will explain a recent result with Hasebe and Kitagawa, in which we extend Voiculescu's S-transform to the case of general probability measures on ℝ with possibly unbounded support. Our approach is analytical and is based on a reformulation using the T-transform, defined as the reciprocal of the S-transform. Unlike previous works, our construction does not require any assumptions on symmetry, mean, or boundedness of the support. This new definition generalizes previous approaches while preserving key properties such as the existence of the limit and non-vanishing in a natural domain.