The freight trains are regularly delayed by issues along that line. It runs through pedestrian plazas, marshland, hairpin turns in hills, etc. A straight shot east for a few miles would land it in flat, unpopulated areas. They'd lose freight access to the salt evaporation plant north of milpitas, which seems to be the remaining freight stop on that corridor (it is being decommissioned for environmental reasons, so by the time the rail rework was done, it wouldn't matter)

Surface Amtrak trains have a much higher top speed than BART, and are nicer. If the freight lines were owned by Amtrak, and maintained to commuter rail standards (instead of freight standards), the existing trains could roughly double their cruising speed for most of the miles of that line, and be much faster than BART. Also, the Amtrak trains have bars and restrooms. They are quiet and don't stink.

Edit: Also, building out a multilane freight rail from the port to east of Oakland would allow the port operator the option to increase port capacity.

They could move containers by rail to a rail yard outside of the bay area commuting zone. They could load trucks there, cutting hours of stop and go truck drive time during commute hours.