Caltrain created 5-car train-sets to accommodate peak hour demands. Many trains do operate near capacity during peak hours. Caltrain recently has also expanded on-board bike capacity which puts more demand for rail cars.
Because Caltrain cars do not have motors, changing train length is a laborious and time consuming operation compared to rail systems that operate EMUs (including light rail). It would add cost for Caltrain because of extra labor needed to change train size. So even if Caltrain could save money on fuel, the savings would be offset by an increase in labor cost.
With electrification, the new EMU train-sets will be able to increase and decrease its length similar to light rail cars and BART cars. Caltrain will then have the option to run shorter trains more frequently during off-peak hours.