But wait, there's more! 
In the 2 birds with 1 stone category. For just over $1 w/free shipping (from Asia) you can get an amplifier module with rotary volume control. Obviously the volume control can double as an on-off switch in that turning to minimum volume effectively turns the music off. The additional module is powered by the 3.8V from the AC-to-DC converter.
Also, I show the actual recommended AC-to-DC converter in the previous post. It's very similar to the one in the video but always good to see proof of concept with the real McCoy!
In the fine-print department, the MP3 cube player has a +VOL and -VOL button. It remembers the last set volume between use...but if you want to change the volume it can be inconvenient to access the silver mini "buttons". Hence for the modest additional cost, real-estate, and just a few additional solder connections, I recommend the volume control module. This also goes to the occasional OGR topics where someone wants to know how to turn down the volume on some particular layout accessory.
