nanoHUB Assessment

by Gerhard Klimeck, Gregory James Perigo

Search

Search pages

Wiki

Tools

Version 1
by (unknown)
Version 2
by (unknown)

Deletions or items before changed

Additions or items after changed

1 -
Processes for User Surveys
+
== Open Usage Statistics ==
2 +
3 +
The idea of science portals that enable the rapid dissemination of scientific and engineering results (and that enable other researchers and educators to use these results) has been pursued by many organizations since the early-to-mid-1990s. The Purdue University Network Computing HUB (PUNCH) was one of the first. We believe there are five critical elements of a successful science gateway:
4 +
5 +
1) Connection to outstanding science/engineering.
6 +
2) Willingness to make the results useful to others, outside the core community.
7 +
3) Efficient, dependable infrastructure operations.
8 +
4) Technology that enables rapid development and deployment.
9 +
5) Open assessment and usage statistics.
10 +
11 +
We believe that while most science gateways are based on criterion 1), most struggle to
12 +
meet points 2) and 3) and are lacking 4) and 5). HUBzero can help to address 3), 4) and
13 +
5) as discussed above. Even extremely well–funded NSFbased infrastructures, such as TeraGrid or NEES, do not make their usage data and usage patterns as available as does nanoHUB. Usage data guides nanoHUB technology development, and its availability to our contributors and
14 +
users bolsters our engagement efforts. We believe that the detailed study and openness of the
15 +
46 nanoHUB usage statistics has advanced nanoHUB capabilities and has given nanoHUB a standing as the premier science gateway. nanoHUB can now provide contributors with usage and impact statistics that can be used in proposals as hard evidence of their impact on a community.
16 +
17 +
[[Image(vasileska_impact_9736.gif, 360, class=align-left)]]
18 +
19 +
The figure above for example shows the monthly and cumulative numbers of user served by Prof. Dragica Vasileska at Arizona State University.
20 +
21 +
== Processes for User Surveys ==
22
23 Dr. Diane Beaudoin, Director of Assessment for the College of Engineering at Purdue, has served as
24 NCN Director of Assessment for two years. She leads the effort to formalize nanoHUB assessment and
25 user survey processes. Last year we had begun a systematic user survey process. We have
26 categorized registered users by their usage patterns:
27
28 1) one-time,
29 2) nonsimulation, and
30 3) heavy users.
31
32 [[Image(userdiscovery.gif, 360, class=align-left)]]
33
34 The so-called “One-time” users utilize nanoHUB content for a single visit only and never return. We have also devised specialized user surveys for these groups. One interesting result of the specialized survey that went to the “one-time” users (which was accommodated by a large 10% return of survey requests) is that these one-time users are overall quite satisfied with what they received from nanoHUB. Another surprising result was that users want to interact with other users more. This has driven nanoHUB component developments that enable connections to social network sites.
35
36 This reporting year we contacted 1,431 users in December 2009 who had registered their account within the prior 3 months. Also on this survey we received a very high 9% response rate with 130 people. 52% were not using nanoHUB as part of a course. Of these, 49% were graduate students, 21% were professional scientists/engineers, and 16% were faculty members. Users who utilize the nanoHUB primarily in the context of coursework – we call course users. Of these course users, 90% were undergraduate students. The majority of our new non-course users (users of nanoHUB for purposes other than coursework) discovered nanoHUB by surfing the web. This result reconfirmed our effort that we must improve our presence and linkage to other web sites like Wikipedia and iTunes U, which helps our Google ranking.
37
38 Components of these surveys address research questions that the Education Research team has posed. Finally, we are also working with external groups that have approached the nanoHUB team because of a desire to study nanoHUB as a virtual organization and coordinate the user populations that are being surveyed.
39
40 -
How users found out about nanoHUB
+
== How users found out about nanoHUB ==
41
42 -
Independent Studies by VOSS Projects
+
== Independent Studies by VOSS Projects ==
43
44 In the spring of 2008, NSF solicited proposals for studies on “Virtual Organizations as
45 Sociotechnical Systems (VOSS)”. A portion of the program announcement reads as follows: “A virtual organization is a group of individuals whose members and resources may be dispersed geographically, but who function as a coherent unit through the use of cyberinfrastructure. Virtual organizations are increasingly central to the science and engineering projects funded by the National Science Foundation. Focused investments in sociotechnical analyses of virtual organizations are necessary to harness their full potential and the promise they offer for discovery and learning. The Virtual Organizations as Sociotechnical Systems (VOSS) program supports scientific research directed at advancing the understanding of what constitutes effective virtual organizations and under what conditions virtual organizations can enable and enhance scientific, engineering, and education production and innovation. …..”
46 Two proposal teams, one each from Northwestern (PI, Noshir Contractor) and Purdue (PI, Michael Beyerlein), approached NCN to provide nanoHUB user and usage data and access to nanoHUB users for interviews. Both teams were successful, and are working with each of them. From these collaborations, we expect to gain further insight into our own virtual organization.
47
48 -
VOSS Survey Documents Impact of nanoHUB on Research
+
== VOSS Survey Documents Impact of nanoHUB on Research ==
49
50 The Purdue VOSS team distributed an online survey to 3,940 nanoHUB users who have been active in the last three years. There were 278 respondents to the survey of which 186 completed the full survey. The survey asked users about how nanoHUB has impacted their work and the results is charted below.
51
52 [[Image(vossprojects.gif, 360, class=align-left)]]
53
54 Seventy percent of respondents noted that nanoHUB meets their research needs; 50% note that nanoHUB has accelerated their research work. In addition, around 50% say that nanoHUB changes are increasing nanoHUB value to them. The dominantly neutral response on the issues of feedback and response to feedback shows room for improvement.