In your application, what provides power to the player/amp? And what voltages are available?
For the player, the added 5 cent capacitor was attached to the + and - pads of the battery (which was removed). A battery provides a relatively steady voltage so without the battery there, the capacitor a similar smoothing function which aids with audio quality.
For the amplifier, the added 5 cent capacitor is added as shown int he photo; it appears you have the identical board. The function here is a bit more nuanced. In this case it affects the startup time of the amplifier which helps to demote the initial pop/click you frequently hear when turning on an audio amplifier. It additionally assists with audio quality.
It's a nuisance if your player does not start playing when you turn it on. So what exactly happens and how do you get it to start playing? Does the display come up with a scroll-able list of all the .MP3 files stored on the SD card? Then even if only 1 choice, it makes you select it for it to start?
I'm sure we can eventually figure something out to get your $1.79 player to work, but is procuring the 99 cent player I used an option you'd consider?
For example, in addition to the automatically-start "feature" that I think is handy, another sleuthing task is to determine if the player module can directly accept 5V in "play" mode if one has removed the ~3.8V Lithium battery (see comment below). Some MP3 players let you transfer your MP3 files into the microSD card via the USB cable while the USB is charging the battery. Others let you stream MP3 files over USB. And so on. I've not found datasheets for the magic chips used in these low-cost player to understand the voltage limitations or modes of operation.
Separately, as a general comment about these $1-2 MP3 players. Some listings on eBay have a warning about NOT overcharging the battery - such as don't leave it plugged in to the USB 5V port for more than some number of hours or something like that:
I have indeed had these tiny rechargeable Lithium batteries bulge and fail in my messing around with these modules. Note that these devices have very simplistic battery-charging circuits unlike the sophisticated chargers in smartphones or whatever that know when to stop pushing power into the cells. In any event, I recommend simply removing the battery for DIY track-side sound accessories when using these modules. There is a separate discussion about rolling-stock sound and if/how a battery might be applicable but that's for another day.
