Ballinger made a name for himself both as librarian and bibliographer. Although he was not a Welsh-speaking Welshman, he must be credited with the formation of a good Welsh library in Cardiff. He was fortunate to have the assistance and advice there of such people as James Ifano Jones and Professor Thomas Powel (q.v.). With the help of Ifano he arranged for publication in 1898 a catalogue of the Welsh and Celtic portions of the contents of the Cardiff Public Library; evidence of Ifano's assistance is also seen in what Ballinger published on Vicar Prichard of Llandovery (1899), the Trevecka printing press (1905), and The Bible in Wales (1906).

In view of the great success achieved at Cardiff in building up the Welsh collection — yet not at the expense of starving the other part of that city's library — it is not surprising that the new library at Aberystwyth flourished so greatly under his care.

Ballinger's contemporaries in the world of libraries in Britain thought highly of him as librarian and administrator; he was, for example, chosen as the president of the Library Association in 1922. He served for a time as chairman of the Press Board of the University of Wales; that body arranged for him to prepare new editions of Rhann o Psalmae Dafydd Brophwyd … 1603, Basilikon Doron … 1604, Y Llyfr Plygain … 1612, Yr Ymarfer o Dduwioldeb … 1630, and Carwr y Cymru … 1631. He prepared the introduction for a new edition (1927) of Sir John Wynne's History of the Gwydir Family; he published Gleanings from a Printer's File in 1928, and ‘Katheryn of Berain’ in Cymm., xl. He served as editor of the Journal of the Welsh Bibliographical Society for some years. He was awarded the honorary degree of M.A. by the University of Wales, in 1909, became a C.B.E. in 1920, and was made a knight bachelor in 1930, in which year he retired. In 1932 he was awarded the medal of the Hon. Society of Cymmrodorion for his services to Wales in the realms of librarianship and bibliography. He had m., in 1888, Amy, daughter of Captain D. Boughton, Cardiff; they had three sons and one daughter.