There are lots of choices and all rest roosm mentioned here are public.
When the water is on in Riverside Park you can use the bathrooms at the 119th Street Tennis Courts in Riverside Park. The water is on from middle April to middle October.
All year long there are bathrooms in the Columbia University building that houses the Dear Mama cafe/coffee shop.
All year long there are bathrooms when its open at The Wallach Art Gallery at Columbia University Lenfest Center of the Arts. That Art Gallery is on the 6th floor I believe (ask one of the receptionists). Google Maps says it opens at 12pm.
During the summer the Department of Parks and Recreation puts out two portable toilets at about Marginal and 134th in West Harlem Piers near the Williams/Transco gas repressurization facility on the bike path which is the East Coast Greenway.