You shouldn't have to go that far but, since that's the largest in the building, it's probably easier. The library has toilets scattered throughout the building and not terribly well signposted. They don't have any on the ground floor for some reason or close to the cafe on the ground...