Though, the body will experience lesser weight when you don't eat or when the stomach is empty but don't think it is healthy to swim with an empty stomach. Since, the essence of food is to gain energy. And that energy is needed to move both hands and legs correspondently while swimming.. If such energy is not enough, you might loose the control along the line.
Also in health perception, food is actually a basic requirements by human body and must always be eaten at regular interval. If you refuse to eat accordingly aand chosed to enter the river or swimming pool to swim,you might actually be tempted to drink the undrinkable water and even loose the strenght to actively control well. Also more water tends to enter the body since the system willbw dialated. Unclean water will penetrate.