You have a valid point! Joseph Smith was human much like the rest of us and had things in his life that could cause for questioning, but his heart was pure. He was a prophet or he wasn't, he either saw God or he didn't. There is is no half way when it comes to Mormons. Mormons are either the biggest fraud in the world or they are the only True church. When question arise I revert back to one thing THE BOOK OF MORMON. There is no way Joseph Smith could have written that book. I have a testimony of the Book of Mormon and know it is true therefore everything else falls into place. Since the Book of Mormon is true Joseph Smith is a prophet it is the Keystone to our religion. Jeffrey R Holland gave a fantastic Talk
* October 2009 Sessions
* Safety for the Soul
Elder Holland lays it out perfectly and it is the Voice of a Modern Day prophet.