If you want to swim in the Potomac then you are advised to take significant precautions -- don't swallow the water, don't swim if you have open cuts or sores, shower shortly after swimming, and avoid swimming after storms or other runoff events. So yeah, I don't think people are exactly lining up to swim in the Potomac, other than for sanctioned events like triathlons.
Whether the Potomac is cleaner or dirtier than other urban rivers requires one to control for a lot of different factors. So it's tough to make a direct comparison. It is true, though, that raw sewage flows into the Potomac during even light rainfall.