Abstract: At the 1900 International Congress of Mathematicians, Hilbert claimed that the Riemann zeta function is not the solution of any algebraic ordinary differential equation its region of analyticity. In 2015, Van Gorder addresses the question of whether the Riemann zeta function satisfies a non-algebraic differential equation and constructs a differential equation of infinite order which zeta satisfies. However, as he notes in the paper, this representation is formal and Van Gorder does not attempt to claim a region or type of convergence. In this talk, we show that Van Gorder's operator applied to the zeta function does not converge pointwise at any point in the complex plane. We also investigate the accuracy of truncations of Van Gorder's operator applied to the zeta function and show that a similar operator applied to zeta and other $L$-functions does converge.