If someone was really adamant about having NiMH batteries and wanted them to be easy to replace, the battery holder is the way to go...
When it comes to the 3 volt PS2 boards, I'm with John on this one. Supercaps are cheap and easy to make. Plus they are smaller than a pair of AAA batteries!
Image credit goes to GUNRUNNERJOHN from a previous post.