 |
| Author | Post |
|---|
CarolynLawrence
Administrator
| Joined: | Sun Mar 5th, 2006 |
| Location: | USA |
| Posts: | 3077 |
| Mana: |    |
|
Posted: Tue Nov 6th, 2007 04:44 am |
|
| The National Survey of Student Engagement, conducted annually by the Center for Post-Secondary Education at Indiana University, is a great way to get insights into how actual students experience their education and social experiences on college campuses. Unfortunately, finding NSSE data about specific colleges has been difficult. Today, however, USA Today, NSSE, and the National Association for College Admissions Counseling launched a new collaborative effort that includes a database of selected NSSE indicators for about 245 colleges and universities. I put the links on my blog (AdmissionsAdvice.com) this evening. I wish that they had included more data, and definitely more colleges (about 1200 actually participate in NSSE), but this is a good start.
|
mackinaw
Member
| Joined: | Mon Mar 6th, 2006 |
| Location: | Michigan |
| Posts: | 750 |
| Mana: | |
|
Posted: Tue Nov 6th, 2007 01:37 pm |
|
(Edited and clarified later)
I also wish more colleges would participate. At the same time, my skepticism about data will begin to come more into play as more colleges participate and especially if the survey data come to be taken very seriously by a wide spectrum of students and parents. Why the skepticism? Sample sizes -- or rather the completed number of interviews -- are small, and both sample selectivity and substantial random year to year variation seem likely.
I took a look at the NSSE website and it lists sample sizes, which is the number of students who are contacted, but not the completion rates and the number who are actually interviewed.
If, say, they complete 100 interviews of freshmen, and 100 of seniors, then that means percentages or means calculated from responses might have a (random) sampling error of +/- 10%, and if they complete 200 interviews at leach level the sampling error would still be about 7%. At 400 interviews completed at each level, the sampling error would be +/- 5%. What this means, for example, if there are 200 completed interviews of freshman and seniors (each), is that you could get 50% who think placement services at a given college are swell in 2008, and 44% who say they are swell in 2010, and as a result many parents and students might conclude the placement services are going to hell. But such a 6% drop would be within the margin of sampling error and not statistically significant. Not 5 in 100 people will realize this fact even if they knew the sample size; probably not 1 in 100 will know the sample size.
But suppose you are comparing the results from a handful of colleges you are considering, and the number of respondents differs greatly across colleges, from, say 200 per class level to 400 per class level? Are you going to take into account the sampling error range for the different schools? I seriously doubt it.
At this website (http://nsse.iub.edu/faq/ifaq.cfm ) I see the much larger "samples" for web-only surveys compared with paper surveys. This is because they expect a much lower yield (number of completed surveys) from the web surveys than from the paper surveys. That's typical. But what is their actual expected yield? I would be surprised if they get over 30% from mail surveys or over 20% from web -- unless the schools themselves send messages to the potential respondents encouraging them to respond to the mail/email message they are receiving from NSSE. And if that happens, you can expect the college's own message to stimulate not only higher yields but also to impart a "support your school" bias to the responses. So higher yields would be correlated with more favorable assessments.
From the survey summaries I've seen, the number of interviews is often omitted. But for the 523 colleges that participated in the 2006 NSSE, a total of 260,000 students were interviewed (http://www.elon.edu/e-web/news/nsse/2005/ ). This works out to about 500 students per college. When we divide those into freshmen and seniors, that's about 250 students per school at each class level on average. I imagine some schools have much larger, and some much smaller totals. But with 250 interviews at each level, the sampling error is roughly +/- 6% at each level. I hope the number of interviews is larger than this in future.
Last edited on Fri Jan 4th, 2008 07:14 pm by mackinaw
|
Current time is 09:03 am | |
|
|
|
|
