It's possible you may not be connecting the negative lead from the power supply to the correct negative battery terminal.
The batteries would normally be connected in series (electrically daisy chained positive end to negative end from one battery to the next) when installed in the holder. If connections are made in the middle of a broken chain, the electricity won't flow to the LEDs.
I would suggest checking connections as follows:
The terminal marked positive should be correct. Only one of the two marked negative will work.