nanoHUB: Social Network of Citations

by Gerhard Klimeck, Gregory James Perigo

Version 2
by (unknown)
Version 3
by (unknown)

Deletions or items before changed

Additions or items after changed

1 == nanoHUB Citations in the Scientific Literature ==
2 === nanoHUB citations in nano-Research, Education, and Cyberinfrastructure ===
4 [[Image(reference_type_44in.png, 360, class=align-left)]]
5 We have identified 575 citations to nanoHUB in the scientific literature with 56% of these citations attributed to investigators who are not in any way affiliated with NCN. The citation map provides a visual representation of affiliated and non-affiliated investigators. Non-affiliates appear outside of the dashed line. Researchers affiliated with but not funded by NCN are responsible for many of the remaining citations. NCN clearly is strongly networked through research papers, with networks being developed outside of NCN. Some of the outside networks are completely decoupled from NCN. 509 (89%) of the citations appear in peer-reviewed journals, conference proceedings, book chapters. Ph.D. or Master’s thesis. 469 (82%) pertain to research in the nanotechnology field. Thus, the ''vast majority of citations refer to actual use''.
7 [[Div(start, class=clear)]][[Div(end)]]
9 === nanoHUB h-index ===
10 Two years ago we began charting citation network maps to address the question of whether nanoHUB can indeed be used for research. The documented citations and their extent into the non-NCN-affiliated nano community exceed those of any other science gateway we are aware of. Next the question arose: “Is it good research?” In the past year we have begun to address that question by asking: “Are the papers that cite nanoHUB subsequently cited by other authors?” We have interacted with Luo Si, professor of computer science at Purdue, who helped us to mine the Google Scholar service to obtain the secondary citations to the nanoHUB citations. Let’s imagine nanoHUB as the author of the 575 papers citing nanoHUB. We have found 3,251 citations to these 575 primary papers such that the h-index is 27. That means 27 of the primary papers have at least 27 citations. Considering that the first primary papers appeared in 2000, the “beginning of nanaoHUB’s scientific career,” nanoHUB exceeds the typical value of 10 for the h-index of a professional with 10 years of experience.
12 [[Div(start, class=clear)]][[Div(end)]]
14 === nanoHUB use by Experimentalists ===
16 [[Image(research_and_exp_datateal_44in.png, 360, class=align-left)]]
17 Documenting the impact achieved by researchers who are using a remote cyberinfrastructure is a challenging task and part of our assessment effort. This year we have re- examined all 575
18 -
nanoHUB citations in the literature to find out if the citation is either given by experimental group that has clearly designed or improved an experiment and utilized nanoHUB resources along the way, or, which is a bit easier to identify, if the paper is plotting real experimental data. In the cosmos of 469 citations that reference nanoHUB usage in nano research, we have identified 55 (12%) papers that are clearly driven by experimentalists, and 142 (30.3%) papers that plot experimental data. We consider these numbers to be a strong evidence of extensive use of nanoHUB by experimentalists doing experimental nanotechnology.
nanoHUB citations in the literature to find out if the citation is either given by experimental group that has clearly designed or improved an experiment and utilized nanoHUB resources along the way, or, which is a bit easier to identify, if the paper is plotting real experimental data. In the cosmos of 469 citations that reference nanoHUB usage in nano research, we have identified 55 (12%) papers that are clearly driven by experimentalists, and 142 (30.3%) papers that plot experimental data. We consider these numbers to be a strong evidence of extensive use of nanoHUB by experimentalists doing experimental nanotechnology.
19 +
20 +
We have also determined that 41 of the papers are written by authors with industrial
21 +
affiliations. This is just about 9% of the 469 nano research papers. As a concrete example we reference an experimental Phys. Rev. B paper where the authors use the conceptual model of molecular conduction in the tool MOLCtoy by Supriyo Datta to explain fundamental conduction results.
22 +
23 +
24 +
25 +
The authors state in the abstract: “A fewparameter scalar model for ballistic current flow
26 +
through a single energy level is sufficient to describe the main features observed in scanning tunneling spectra of individual Mn12 molecules and offers a deeper insight into the electronic transport properties of this class of single-molecule magnets.” The authors plot their data and MolCtoy results in their paper on the same chart as shown in the figure here. The authors logged into nanoHUB some 65 times and ran around 280 simulations with MolCtoy consuming less than 20
27 +
minutes of CPU time. This paper was published in Oct. 2008 and already has been cited
28 +
three times.
29 +
30 +
As another example we list the use by Judy Hoyt’s MIT research group of nanoHUB FETtoy tool to examine their experimental Si/Ge slabs. The work is published in IEEE transactions electron devices. Prof. Hoyt, who is a very well respected experimentalist in the nanoelectronic community and a Fellow of the IEEE, is planning to attend the NCN site visit virtually through a teleconference link to describe her nanoHUB interactions.
31 +
32 +
A newly developed nanoHUB tool that modelselectron transport in high mobility InAs/InGaAs based transistors. The particularly important modeling capability is the mapping of the non-parabolic bandstructure to a simplified model and the ability to compute the gate tunneling. The visualization of the current flow drove a nanoHUB development that now enables an intuitive representation of gate tunneling. The new OMENfet code has now been released and is now available
33 +
openly on the nanoHUB. The scientific results generated in this experimental and theoretical collaboration has been published in a co-authored IEDM proceedings article involving the experimental MIT group of Jesus del Alamo and the theory group of Klimeck at Purdue. The funding for the science was obtained through leveraged grants of the FCRP / MSD center where del Alamo and Klimeck participate.
34 +
35 +
36 +
37 +
38 +
39 +
The analysis of our external literature citations revealed some examples where external
40 +
theory-driven researchers utilized a nanoHUB tool that has been contributed to nanoHUB
41 +
from outside of the NCN. For example the authors Kureshi and Hasan working at a
42 +
university in India published their study CNT bundles as possible copper replacement
43 +
interconnects in the “Journal of Nanomaterials” in April 2009. In their work they utilized
44 +
the tool “Carbon Nanotubes Interconnect Analyzer (CNIA)” for over 1,600 simulations in
45 +
the year 2009 alone. CNIA was contributed by Tanachutiwat and Wang of the University
46 +
of Albany. We cannot see any relationship between the two research groups and NCN
47 +
other than the contributed, efficiently served, and utilized tool CNIA. The same CNIA
48 +
tool has previously also been cited in a Master’s thesis at the University of Cincinnati in
49 +
27 the year 2005. The concept of community contributions coming from outside the NCN
50 +
and being useful for an unrelated research group is becoming reality.
51 +
52 +
[[Div(start, class=clear)]][[Div(end)]]
53 +
54 +
=== Lowering Barriers to Simulation ===
55 +
56 +
[[Image(rapptureintroncn_tools.png, 360, class=align-left)]]
57 +
58 +
Accelerating Deployment of Scientific Software
59 +
60 +
In June 2005, NCN released Rappture 1.0, a new, open toolkit to facilitate the rapid development and robust deployment of scientific simulation packages. Designed to work as a library for a variety of programming languages, including Matlab, C/C++, Fortran, Python, Perl, Tcl, and Ruby, Rappture automatically generates a graphical user interface from a description of the inputs and
61 +
outputs of the simulation program. Rappture has become the main vehicle for software deployment on nanoHUB. Students across the NCN network are using Rappture to enable simulation applications for use on Simulation usage on nanoHUB has grown seven-fold to over 7,900 users
62 +
in less than four years and we attribute this increase to the friendly interfaces afforded by
63 +
64 +
65 +
Prior to Rappture, a significant portion of nanoHUB users downloaded software and installed it on their own computers. However, since the introduction of Rappturized applications, the number of software downloads has all but vanished – Rappturized software is user-friendly. Rappture is also central to NCN’s long-term strategy to link simulations in complex workflows. Because of the scale of the NCN initiative, we expect that Rappture will have broad impact on scientific computing outside NCN. There are currently over 282 Rappture software development projects under way with 318 developers. In all, 140 tools have been deployed on nanoHUB, with all but seven using
66 +
Rappture. Although we do not force developers to use Rappture, approximately 95% of the deployed projects use it, which indicates developer buy-in and the ease of development with Rappture technology.
67 +
68 +
[[Image(schred_ref_type_44in.png, 360, class=align-left)]]
69 +
70 +
[[Div(start, class=clear)]][[Div(end)]]
71 +
72 +
=== Leadership in Assessment and Virtual Organizations ===
73 +
74 +
Open Usage Statistics
75 +
76 +
The idea of science portals that enable the rapid dissemination of scientific and engineering results (and that enable other researchers and educators to use these results) has been pursued by many organizations since the early-to-mid-1990s. The Purdue University Network Computing HUB (PUNCH) was one of the first. We believe there are five critical elements of a successful science gateway:
77 +
78 +
1) Connection to outstanding science/engineering.
79 +
2) Willingness to make the results useful to others, outside the core community.
80 +
3) Efficient, dependable infrastructure operations.
81 +
4) Technology that enables rapid development and deployment.
82 +
5) Open assessment and usage statistics.
83 +
84 +
We believe that while most science gateways are based on criterion 1), most struggle to
85 +
meet points 2) and 3) and are lacking 4) and 5). HUBzero can help to address 3), 4) and
86 +
5) as discussed above. Even extremely well–funded NSFbased infrastructures, such as TeraGrid or NEES, do not make their usage data and usage patterns as available as does nanoHUB. Usage data guides nanoHUB technology development, and its availability to our contributors and
87 +
users bolsters our engagement efforts. We believe that the detailed study and openness of the
88 +
46 nanoHUB usage statistics has advanced nanoHUB capabilities and has given nanoHUB a standing as the premier science gateway. nanoHUB can now provide contributors with usage and impact statistics that can be used in proposals as hard evidence of their impact on a community.
89 +
90 +
[[Image(vasileska_impact_9736.gif, 360, class=align-left)]]
91 +
92 +
The figure above for example shows the monthly and cumulative numbers of user served by Prof. Dragica Vasileska at Arizona State University.
93 +
94 +
Content Characterization by Usage, User Feedback, and Community Involvement
95 +
96 +
As nanoHUB content increases, we find that users are struggling to find the high quality content. As a result, we have continued to improve the search mechanisms on nanoHUB to enable rapid information retrieval. One key element in this effort is to characterize each content item by a
97 +
variety of criteria that ultimately influence the ranking of the resource. Each simulation tool is
98 +
characterized by:
99 +
100 +
1) A Google-like ranking based on user reviews and use.
101 +
2) A target audience rating, or, the expertise level expected from the user.
102 +
3) An indication if this is an NCN Supported tool, or a community supported tool
103 +
4) Data including number of users and simulation jobs, average run time, and average
104 +
number of stars awarded in reviews.
105 +
5) Number of citations in the scientific literature–this indicates the vetting of the tool
106 +
and its use in research.
107 +
6) Number of questions, indicative of the liveliness of the community. A large number
108 +
of open questions suggests a poorly supported tool. Conversely, large numbers of
109 +
closed questions indicates a live code with tool owners interested and dedicated to its
110 +
support. The introduction of a virtual economy has proved to have a positive
111 +
influence on the question and answer forum.
112 +
7) A wishlist enables users to express tool improvement wishes and the tool
113 +
development team to handle tool improvement processes.
114 +
8) User reviews: anyone can give a 0- to 5-star review and submit written comments.
115 +
9) Users can also declare nanoHUB content items as their favorites, which they can later
116 +
easily find again on their favorite list. Furthermore, they can share their favorite
117 +
nanoHUB items on six different social network sites, including Facebook, Twitter,
118 +
and Google.
119 +
10) A list of associated and recommended documents that support this tool.
120 +
121 +
Processes for User Surveys
122 +
123 +
Dr. Diane Beaudoin, Director of Assessment for the College of Engineering at Purdue, has served as
124 +
NCN Director of Assessment for two years. She leads the effort to formalize nanoHUB assessment and
125 +
user survey processes. Last year we had begun a systematic user survey process. We have
126 +
categorized registered users by their usage patterns:
127 +
128 +
1) one-time,
129 +
2) nonsimulation, and
130 +
3) heavy users.
131 +
132 +
The so-called “One-time” users utilize nanoHUB content for a single visit only and never return. We have also devised specialized user surveys for these groups. One interesting result of the specialized survey that went to the “one-time” users (which was
133 +
accommodated by a large 10% return of survey requests) is that these one-time users are
134 +
overall quite satisfied with what they received from nanoHUB. Another surprising result
135 +
was that users want to interact with other users more. This has driven nanoHUB
136 +
component developments that enable connections to social network sites.
137 +
This reporting year we contacted 1,431 users in December 2009 who had registered their
138 +
account within the prior 3 months. Also on this survey we received a very high 9%
139 +
response rate with 130 people. 52% were not using nanoHUB as part of a course. Of
140 +
these, 49% were graduate students, 21% were professional scientists/engineers, and 16%
141 +
were faculty members. Users who utilize the nanoHUB primarily in the context of
142 +
coursework – we call course users. Of these course users, 90% were undergraduate
143 +
students. The majority of our new non-course users (users of nanoHUB for purposes
144 +
other than coursework) discovered nanoHUB by surfing the web. This result
145 +
reconfirmed our effort that we must improve our presence and linkage to other web sites
146 +
like Wikipedia and iTunes U, which helps our Google ranking.
147 +
Components of these surveys address research questions that the Education Research
148 +
team has posed. Finally, we are also working with external groups that have approached
149 +
the nanoHUB team because of a desire to study nanoHUB as a virtual organization and
150 +
coordinate the user populations that are being surveyed.
151 +
How users found out about nanoHUB
152 +
153 +
• Independent Studies by VOSS Projects
154 +
In the spring of 2008, NSF solicited proposals for studies on “Virtual Organizations as
155 +
Sociotechnical Systems (VOSS)”. A portion of the program announcement reads as
156 +
follows: “A virtual organization is a group of individuals whose members and resources
157 +
may be dispersed geographically, but who function as a coherent unit through the use of
158 +
cyberinfrastructure. Virtual organizations are increasingly central to the science and
159 +
engineering projects funded by the National Science Foundation. Focused investments in
160 +
sociotechnical analyses of virtual organizations are necessary to harness their full
161 +
potential and the promise they offer for discovery and learning. The Virtual
162 +
Organizations as Sociotechnical Systems (VOSS) program supports scientific research
163 +
directed at advancing the understanding of what constitutes effective virtual organizations
164 +
and under what conditions virtual organizations can enable and enhance scientific,
165 +
engineering, and education production and innovation. …..”
166 +
Two proposal teams, one each from Northwestern (PI, Noshir Contractor) and Purdue
167 +
(PI, Michael Beyerlein), approached NCN to provide nanoHUB user and usage data and
168 +
access to nanoHUB users for interviews. Both teams were successful, and are working
169 +
with each of them. From these collaborations, we expect to gain further insight into our
170 +
own virtual organization.
171 +
• VOSS Survey Documents Impact of nanoHUB on Research
172 +
The Purdue VOSS team distributed an online survey to 3,940 nanoHUB users who have
173 +
been active in the last three years. There were 278 respondents to the survey of which
174 +
186 completed the full survey. The survey asked users about how nanoHUB has
175 +
impacted their work and the results is charted below.
176 +
We are excited to see that for 70% of the respondents nanoHUB meets their research
177 +
needs and that for 50% nanoHUB has accelerated their research work. We are thrilled to
178 +
learn that around 50% say that nanoHUB changes are increasing nanoHUB value to
179 +
180 +
them. The dominantly neutral response on the issues of feedback and response to
181 +
feedback shows room for improvement.
182 +
• nanoHUB Virtual Economy Shows Impact
183 +
The concept of a virtual economy is most commonly associated with multi-player online
184 +
gaming communities. has borrowed the concept to assist in its own
185 +
community building, resource management, and sustainability goals. As of March 2008,
186 +
the system allowed users to earn points by asking and answering questions in the Answers
187 +
forum. They could spend points on merchandise in the store, or spend
188 +
them by asking questions in the forum and assigning a point reward for the best answer.
189 +
In January 2009, an assessment
190 +
project was conducted as part of
191 +
a nanoHUB staff member’s
192 +
Master’s thesis with a goal to
193 +
evaluate the early impact of the
194 +
new virtual economy system at
195 + on user behavior,
196 +
as well as to establish a generic
197 +
assessment model for the
198 +
nanoHUB virtual economy. The
199 +
study analyzed site usage data
200 +
for equal time periods before
201 +
and after the introduction of the
202 +
first few components of the
203 +
system. The analysis showed
204 +
that there was a substantial increase in the number of Q&A contributions per user in the
205 +
post time period in all categories and groups of users. The maximum (9.6 times) increase
206 +
was seen in a category of heavy simulation users (users with over 30 and below 350
207 +
simulation runs in a 6-month period). Expert users (those with over 350 simulation runs
208 +
in 6 months) had highest ratios of Q&A contributions per user, going from over 0.22 to
209 +
almost 0.97 in the post period. Similarly, there was an 8.3 times increase in proportion of
210 +
contributing users per all users within the category of heavy users.
211 +
As a result of the study we have further expanded our reward system to encourage good
212 +
behavior on the site. Points are now rewarded for meaningful comments and ratings –
213 +
other users have to give these ratings a “thumbs-up” before any points are rewarded. We
214 +
also fully introduced point rewards for contributed tools and seminars as they are being
215 +
used on the nanoHUB.
216 +
• nanoHUB Community Building by Job Announcements
217 +
NCN also created a new area of to manage job applications for the
218 +
nanotechnology community. Any member can post a resume as part of their profile.
219 +
Professors and other employers can also post specific job advertisements. Students can
220 +
browse the ads and apply for particular jobs, either directly on or off on the
221 +
employer’s web site.
222 +
223 +
We created this capability not only to help forge connections within the community, but
224 +
also to experiment with a new model for sustainability. Many institutions pay significant
225 +
fees for job advertisements. Posting a single job ad to, for example, costs
226 +
$950. Unlike Google AdSense or other advertising programs, having job ads would not
227 +
hurt the scientific integrity of, and would be viewed by users as a service
228 +
to the community.
229 +
As part of the jobs component, we
230 +
developed an order fulfillment system for
231 +
subscription services. Right now, we are
232 +
offering a free “basic” subscription for job
233 +
postings, which allows one job posting per
234 +
month. We intend to experiment with other
235 +
subscription models during the coming
236 +
year. An advanced subscription would
237 +
support several job postings at once and
238 +
unlimited access to our resume database for
239 +
a monthly fee. We may continue to offer the free
240 +
subscription as a community service for professors who
241 +
want to advertise a post-doc position or summer
242 +
internship at no cost. We plan to expand the use of subscription services to many other
243 +
areas of the site as well, so users can pay for additional disk space, faster
244 +
turn-around time for simulations, etc. We will establish ways for users to pay with the
245 +
points they earn while working on the site, and also with real dollars.