Unleashing the predictive power of molecular dynamics (MD), Neural Network Potentials (NNPs) trained on Density Functional Theory (DFT) calculations are revolutionizing our ability to simulate chemical systems with unprecedented accuracy and efficiency. Electrolyte solutions are a natural initial system to apply this tool to because they are critically important for a wide range of applications and their properties cannot currently be predicted. Unfortunately, however most DFT approximations are not sufficiently accurate to predict many practically relevant properties of electrolytes. Additionally, tracking the position of every atom in a system during molecular simulations is inherently limited, even with NNP-MD. Here, we use a state-of-the-art DFT approximation to demonstrate highly accurate all-atom NNPs with minimal training data. We demonstrate that NNPs can reliably be recursively trained on a subset of their own output to enable coarse-grained continuum solvent molecular simulations that can access much longer timescales. We apply our technique to simulate lithium chloride, potassium chloride, and lithium bromide in water. We reproduce key structural, thermodynamic, and kinetic properties of these solutions in agreement with experimental data. The formation of a previously unknown Li cation dimer is observed, along with identical anion-anion interactions of chloride and bromide. Finally, the coarse-grained model is capable of reproducing crystal phase behavior and infinite dilution pairing free energies despite being trained solely on moderate concentration solutions, disproving the notion that NNPs are only useful for interpolation. This approach should be scalable to determine the properties of electrolyte solutions over a much wider range of conditions and compositions than is possible experimentally.