LDS members: please take this survey for my Master's Degree research project

[BluePlastic]

Hello :) I was hoping some people here could help me out! I am conducting a research survey for my special project for my master's of library science degree.

The survey is for LDS Church members only, about the use of media by Church members.

If you could go to https://www.surveymonkey.com/s/YVCVR5M and take the survey, I would really appreciate it.

It's only 19 questions and does not collect any information that could uniquely identify you. I am trying to get responses from all over the place, and I thank everyone who takes the time to take the survey in advance!

[pam]

I just took. Very good survey.

[selek]

I took your survey- but I genuinely wonder of what use the results will be to you.

Your stated purpose was to assess the utility and effectiveness of Church produced media and the new media options (such as blogs and podcasts).

What troubles me is that your primary demographic (this group especially) is already fairly tech-savvy and our mere presence here makes it more likely that any of us is more likely to both have used- and to respond to- such media.

How, then, are you controlling for confirmation bias and obtaining a representative (unbiased) sample?

[BluePlastic]

I took your survey- but I genuinely wonder of what use the results will be to you.

Your stated purpose was to assess the utility and effectiveness of Church produced media and the new media options (such as blogs and podcasts).

What troubles me is that your primary demographic (this group especially) is already fairly tech-savvy and our mere presence here makes it more likely that any of us is more likely to both have used- and to respond to- such media.

How, then, are you controlling for confirmation bias and obtaining a representative (unbiased) sample?

I'm not really comfortable with discussing particulars of my research methodology while I'm still conducting my research. I am taking steps to address the things you have concerns about, though.

Thank you for taking the survey :)

[Windseeker]

Sorry this survey confused me, are you only measuring Iphone/Ipad apps? You say Iphone/Ipad apps when referencing the Gospel Library which is also is available on Android, BlackBerry, webOS, Nokia, Java and Windows. So did you mean Mobile Apps or are you specifically tracking the Iphone and Ipad use?

[NeuroTypical]

Survey all taken.

It was just a 1 page survey, yes? After I clicked 'done', there wasn't any 'thanks for taking our survey' page - so I wanted to make sure I didn't break something.

[BluePlastic]

Yes, it's just that one page :)

And I meant all the mobile apps. I will change the wording, thank you for your feedback. :)

[EarlJibbs]

Done. I enjoyed it. Thanks for sharing.

[pam]

I took your survey- but I genuinely wonder of what use the results will be to you.

Your stated purpose was to assess the utility and effectiveness of Church produced media and the new media options (such as blogs and podcasts).

What troubles me is that your primary demographic (this group especially) is already fairly tech-savvy and our mere presence here makes it more likely that any of us is more likely to both have used- and to respond to- such media.

How, then, are you controlling for confirmation bias and obtaining a representative (unbiased) sample?

Keep in mind that some of the questions were asked about feelings after watching a video or reading a particular scripture. You don't have to be tech savvy to watch or respond to that.

I found the survey extremely refreshing after some of the ones in the past I've done.

[selek]

Keep in mind that some of the questions were asked about feelings after watching a video or reading a particular scripture. You don't have to be tech savvy to watch or respond to that.

Agreed, but the problem is that this poll is targeting a self-selecting minority, rather than a general population; generally a huge no-no in any kind of stastical analysis.

By targeting a comparatively small (and self-selecting) subset of Mormons, rather than Mormons generally, the OP has made it extremely unlikely that her results will be representative of the whole.

Put it another way: how well do you think this poll would reflect the LDS people if it were answered only by fans of the HBO series "Big Love"?

My guess would be "not very well" as only a tiny minority bother to watch that show at all, let alone consider theemselves "fans". By the same token, very few Mormons have the time, patience, or OCD to establish themselves here. There is a very real possibility that our presence here represents a receptiveness (a quite bluntly, access) to technology not shared by LDS as a whole.

There is no (obvious) control for such a bias, which may very well render the results invalid.

I'm not saying it's a bad poll: I'm saying that the sample demographic may skew the results.

There is also the possibility that the poll is measuring one thing under the pretext of another (not an uncommon tactic in polling) and what I perceive as a bug is actually a feature.

Time will tell, I suppose.

I found the survey extremely refreshing after some of the ones in the past I've done.

Agreed, but not my point.

[pam]

Agreed, but the problem is that this poll is targeting a self-selecting minority, rather than a general population; generally a huge no-no in any kind of stastical analysis.

By targeting a comparatively small (and self-selecting) subset of Mormons, rather than Mormons generally, the OP has made it extremely unlikely that her results will be representative of the whole.

Put it another way: how well do you think this poll would reflect the LDS people if it were answered only by fans of the HBO series "Big Love"?

My guess would be "not very well" as only a tiny minority bother to watch that show at all, let alone consider theemselves "fans". By the same token, very few Mormons have the time, patience, or OCD to establish themselves here. There is a very real possibility that our presence here represents a receptiveness (a quite bluntly, access) to technology not shared by LDS as a whole.

There is no (obvious) control for such a bias, which may very well render the results invalid.

I'm not saying it's a bad poll: I'm saying that the sample demographic may skew the results.

There is also the possibility that the poll is measuring one thing under the pretext of another (not an uncommon tactic in polling) and what I perceive as a bug is actually a feature.

Time will tell, I suppose.

Agreed, but not my point.

Just quit over thinking it. It's her survey. Just take it and enjoy it.

Edited June 3, 2013 by pam

[EarlJibbs]

Agreed, but the problem is that this poll is targeting a self-selecting minority, rather than a general population; generally a huge no-no in any kind of stastical analysis.

By targeting a comparatively small (and self-selecting) subset of Mormons, rather than Mormons generally, the OP has made it extremely unlikely that her results will be representative of the whole.

Put it another way: how well do you think this poll would reflect the LDS people if it were answered only by fans of the HBO series "Big Love"?

My guess would be "not very well" as only a tiny minority bother to watch that show at all, let alone consider theemselves "fans". By the same token, very few Mormons have the time, patience, or OCD to establish themselves here. There is a very real possibility that our presence here represents a receptiveness (a quite bluntly, access) to technology not shared by LDS as a whole.

There is no (obvious) control for such a bias, which may very well render the results invalid.

I'm not saying it's a bad poll: I'm saying that the sample demographic may skew the results.

There is also the possibility that the poll is measuring one thing under the pretext of another (not an uncommon tactic in polling) and what I perceive as a bug is actually a feature.

Time will tell, I suppose.

Agreed, but not my point.

Assuming that this is the only place the author has asked anyone to take the survey. You could be correct. But as the OP stated, they are trying to get responses from "all over the place"

Edited June 3, 2013 by EarlJibbs

[selek]

Just quit over thinking it. It's his survey. Just take it and enjoy it.

For starters, per the information on the Consent To Take a Survey Page, "him" is a "her".

Second, as stated above, I DID take it before I started asking questions about its built-in bias.

Finally, now you know why my kids hated it when I helped them with their homework.

"If you're gonna do it, do it right.":whistling:

@Earl: You are correct: my argument assumes that this survey is limited in its scope and sample. "All over the place" is necessarily a vague and unscientific statement.

The problem is that the same method used in multiple locations is not a control against bias and does not automagically ensure an unbiased whole.

Even a small tainted sample can skew the results: "A little leaven leaveneth the whole lump", and all that- or as many of our tech friends would argue, "Garbage In, Garbage Out".

What valid methodological lessons can the young Miss Starr take from a (potentially) poorly designed a/o unrepresentative survey?

As I understand it, one of the primary reasons for doing this poll is to be familiar with the practice and methodologies involved for use in the future.

Are we doing her any favors by simply playing along and ignoring (what I at least perceive to be) flaws in her methodology?

Isn't a lesson learned or a new thought added better than a complacent nod and a belly rub, even for an old sea dog?

From her response, BluePlastic is at least aware (if she was not before) of the potential problems with her methodology. That's a plus in my book.

If the built-in problems with her survey don't skew the data, more power to her.

If, however, they do, she needs to be aware of that effect before she engages in real-world (as opposed to classroom) exercises, wouldn't you agree?

There seems to be a consensus here that I am nit-picking Blue for mere pedanticism's sake.

Has it crossed none of your minds that I genuinely want to help her achieve all that she can?

Isn't that the point in participating in the survey in the first place?

Edited June 3, 2013 by selek

[Guest]

You must be a scream at parties.

[selek]

You must be a scream at parties.

Depends largely upon the party...

Edited June 3, 2013 by selek

[MarginOfError]

Without knowledge of her research hypotheses, it's difficult to say if the convenience sample she is drawing here is inappropriate or not. If she is targeting a technologically proficient population of mormons, then this isn't too terribly far off the mark. I would argue that I have insufficient information with which to make meaningful comment about the validity and generalizability of her sample.