Processes for User Surveys

Dr. Diane Beaudoin, Director of Assessment for the College of Engineering at Purdue, has served as
NCN Director of Assessment for two years. She leads the effort to formalize nanoHUB assessment and
user survey processes. Last year we had begun a systematic user survey process. We have
categorized registered users by their usage patterns: 

1) one-time, 
2) nonsimulation, and 
3) heavy users. 

[[Image(userdiscovery.gif, 360, class=align-left)]] 

The so-called “One-time” users utilize nanoHUB content for a single visit only and never return. We have also devised specialized user surveys for these groups. One interesting result of the specialized survey that went to the “one-time” users (which was accommodated by a large 10% return of survey requests) is that these one-time users are overall quite satisfied with what they received from nanoHUB. Another surprising result was that users want to interact with other users more. This has driven nanoHUB component developments that enable connections to social network sites.

This reporting year we contacted 1,431 users in December 2009 who had registered their account within the prior 3 months. Also on this survey we received a very high 9% response rate with 130 people. 52% were not using nanoHUB as part of a course. Of these, 49% were graduate students, 21% were professional scientists/engineers, and 16% were faculty members. Users who utilize the nanoHUB primarily in the context of coursework – we call course users. Of these course users, 90% were undergraduate students. The majority of our new non-course users (users of nanoHUB for purposes other than coursework) discovered nanoHUB by surfing the web. This result reconfirmed our effort that we must improve our presence and linkage to other web sites like Wikipedia and iTunes U, which helps our Google ranking.

Components of these surveys address research questions that the Education Research team has posed. Finally, we are also working with external groups that have approached the nanoHUB team because of a desire to study nanoHUB as a virtual organization and coordinate the user populations that are being surveyed.

How users found out about nanoHUB

Independent Studies by VOSS Projects

In the spring of 2008, NSF solicited proposals for studies on “Virtual Organizations as
Sociotechnical Systems (VOSS)”. A portion of the program announcement reads as follows: “A virtual organization is a group of individuals whose members and resources may be dispersed geographically, but who function as a coherent unit through the use of cyberinfrastructure. Virtual organizations are increasingly central to the science and engineering projects funded by the National Science Foundation. Focused investments in sociotechnical analyses of virtual organizations are necessary to harness their full potential and the promise they offer for discovery and learning. The Virtual Organizations as Sociotechnical Systems (VOSS) program supports scientific research directed at advancing the understanding of what constitutes effective virtual organizations and under what conditions virtual organizations can enable and enhance scientific, engineering, and education production and innovation. …..”
Two proposal teams, one each from Northwestern (PI, Noshir Contractor) and Purdue (PI, Michael Beyerlein), approached NCN to provide nanoHUB user and usage data and access to nanoHUB users for interviews. Both teams were successful, and are working with each of them. From these collaborations, we expect to gain further insight into our own virtual organization.

VOSS Survey Documents Impact of nanoHUB on Research

The Purdue VOSS team distributed an online survey to 3,940 nanoHUB users who have been active in the last three years. There were 278 respondents to the survey of which 186 completed the full survey. The survey asked users about how nanoHUB has impacted their work and the results is charted below.

[[Image(vossprojects.gif, 360, class=align-left)]] 

Seventy percent of respondents noted that nanoHUB meets their research needs; 50% note that nanoHUB has accelerated their research work. In addition, around 50% say that nanoHUB changes are increasing nanoHUB value to them. The dominantly neutral response on the issues of feedback and response to feedback shows room for improvement.