One thing about using a superCAP or several of them to add capacity. You can charge them far faster than they discharge, unlike traditional rechargeable batteries.
I just connected two 2.5F 5.5 volt superCAPs to a 39 ohm resistor and three parallel LED's. Feeding that I have a diode (to block reverse flow to the power source). I feed the network with 5.5 volts. The caps easily charged up from a 1A current limited supply in about 25 seconds, and the LED's were burning brightly. I killed the input power and the three LED's burned for over three minutes at a nice brightness. At the two minute mark, I still had over 4 volts across the superCAP bank and the LED's were burning brightly.
The caps are $4/ea, a regulated power supply is $3-4, and you just have to wire up the LED's.
If you want more LED's, you could add another cap or two.