The APU provides electrical power but only when there's fuel for it to burn. In the case of the Gimli Glider, it therefore would not have worked.
If we look at the case of Cactus 1549 [1], though, we see Capt. Sullenberger turned on the APU immediately after both engines failed. This has been hailed [1,2] as a good decision -- no doubt because the A320 relies so heavily on electronics for control and it could have helped them restart the failed engines.
Commercial jets certainly do also carry traditional batteries for various uses. Take the Boeing 787 as an example; its onboard batteries originally presented a fire hazard [3].
No, an APU is (in most aircraft that have them) actually a small jet engine that GENERATES electrical power, as opposed to storage. In some aircraft, the distinction between the onboard battery and the APU is quite clear. I know one jet that this is true of the the A320 family.
The startup procedure for that aircraft relies on either a tether from the ground to provide external electricity, or on the battery to provide electrical power to the instruments/HUD before the APU is spun up.
Isn't "backup batteries" a loose description of an APU? https://en.wikipedia.org/wiki/Auxiliary_power_unit