I've never been a fan of public speaking, but I was able to do it somewhat until I went to college. There I had one instructor(grad student) who favored the guys over the girls, and nothing I did was right. The other instructor who really messed it up for me said I was doctrinally incorrect on what I was teaching in front of my entire class.
Since then I've absolutely hated it. I've given one talk at church, and like you said, I didn't look at anyone, I didn't look up from the paper I had in front of me.