Click on Participants in the left sidebar and open the Referees tab. Then select the referee you want to delete, by ticking the box left of the referees' name. You can now click on Delete to delete the referee.