As already mentioned, going from "not enough RAM" to "enough RAM" is way more noticeable than any speed difference.

If your mainboard can run two different sticks in dual-channel mode (see manual for where to put them if you have four slots, and you might have to manually configure them both to the same values in the bios) then the speed increase from dual-channel might even offset the lower frequency.

The CPU might also run some parts connected to the memory controller at different frequencies depending on RAM frequency which can also make a slight difference. But still, having enough memory is the most important issue.