The asymmetric simple exclusion process with open boundaries, which is a very simple model of out-of-equilibrium statistical physics, is known to be integrable. In particular, its spectrum can be described in terms of Bethe roots. The large deviation function of the current can be obtained as well by diagonalizing a modified transition matrix, which is still integrable: the spectrum of this new matrix can also be described in terms of Bethe roots for special values of the parameters. However, due to the algebraic framework used to write the Bethe equations in previous works, the nature of the excitations and the full structure of the eigenvectors remained unknown. This paper explains why the eigenvectors of the modified transition matrix are physically relevant, gives an explicit expression for the eigenvectors and applies it to the study of atypical currents. It also shows how the coordinate Bethe ansatz developed for the excitations leads to a simple derivation of the Bethe equations and of the validity conditions of this ansatz. All the results obtained by de Gier and Essler are recovered and the approach gives a physical interpretation of the exceptional points. The overlap of this approach with other tools such as the matrix ansatz is also discussed. The method that is presented here may be not specific to the asymmetric exclusion process and may be applied to other models with open boundaries to find similar exceptional points.