Content Characterization by Usage, User Feedback, and Community Involvement
Virtual Organizations are institutions whose members are geographically widespread but share a common set of resources and operating conditions. As a virtual cyber-community engaged in theory, modeling and simulation, nanoHUB serves over 115,000 researchers, educators, students and professionals every year. nanoHUB has nearly 700 contributors, and 575 scholarly papers directly cite nanoHUB.org.
As nanoHUB content increases, we find that users are struggling to find the high quality content. As a result, we have continued to improve the search mechanisms on nanoHUB to enable rapid information retrieval. One key element in this effort is to characterize each content item by a variety of criteria that ultimately influence the ranking of the resource. Each simulation tool is characterized by:
1) A Google-like ranking based on user reviews and use.
2) A target audience rating, or, the expertise level expected from the user.
3) An indication if this is an NCN Supported tool, or a community supported tool.
4) Data including number of users and simulation jobs, average run time, and average number of stars awarded in reviews.
5) Number of citations in the scientific literature–this indicates the vetting of the tool and its use in research.
6) Number of questions, indicative of the liveliness of the community. A large number of open questions suggests a poorly supported tool. Conversely, large numbers of closed questions indicates a live code with tool owners interested and dedicated to its support. The introduction of a virtual economy has proved to have a positive influence on the question and answer forum.
7) A wishlist enables users to express tool improvement wishes and the tool development team to handle tool improvement processes.
8) User reviews: anyone can give a 0- to 5-star review and submit written comments.
9) Users can also declare nanoHUB content items as their favorites, which they can later easily find again on their favorite list. Furthermore, they can share their favorite nanoHUB items on six different social network sites, including Facebook, Twitter, and Google.
10) A list of associated and recommended documents that support this tool.
nanoHUB Virtual Economy Shows Impact
The concept of a virtual economy is most commonly associated with multi-player online gaming communities. nanoHUB.org has borrowed the concept to assist in its own community building, resource management, and sustainability goals. As of March 2008, the system allowed users to earn points by asking and answering questions in the Answers forum. They could spend points on merchandise in the nanoHUB.org store, or spend them by asking questions in the forum and assigning a point reward for the best answer.
In January 2009, an assessment project was conducted as part of a nanoHUB staff member’s Master’s thesis with a goal to evaluate the early impact of the new virtual economy system at nanoHUB.org on user behavior, as well as to establish a generic assessment model for the nanoHUB virtual economy. The study analyzed site usage data for equal time periods before and after the introduction of the first few components of the system. The analysis showed that there was a substantial increase in the number of Q&A contributions per user in the post time period in all categories and groups of users. The maximum (9.6 times) increase was seen in a category of heavy simulation users (users with over 30 and below 350 simulation runs in a 6-month period). Expert users (those with over 350 simulation runs in 6 months) had highest ratios of Q&A contributions per user, going from over 0.22 to almost 0.97 in the post period. Similarly, there was an 8.3 times increase in proportion of contributing users per all users within the category of heavy users.
As a result of the study we have further expanded our reward system to encourage good behavior on the site. Points are now rewarded for meaningful comments and ratings – other users have to give these ratings a “thumbs-up” before any points are rewarded. We also fully introduced point rewards for contributed tools and seminars as they are being used on the nanoHUB.
nanoHUB Community Building by Job Announcements
NCN also created a new area of nanoHUB.org to manage job applications for the nanotechnology community. Any member can post a resume as part of their profile. Professors and other employers can also post specific job advertisements. Students can browse the ads and apply for particular jobs, either directly on nanoHUB.org or off on the employer’s web site.
We created this capability not only to help forge connections within the community, but also to experiment with a new model for sustainability. Many institutions pay significant fees for job advertisements. Posting a single job ad to HPCwire.com, for example, costs $950. Unlike Google AdSense or other advertising programs, having job ads would not hurt the scientific integrity of nanoHUB.org, and would be viewed by users as a service to the community.
As part of the jobs component, we developed an order fulfillment system for subscription services. Right now, we are offering a free “basic” subscription for job postings, which allows one job posting per month. We intend to experiment with other subscription models during the coming year. An advanced subscription would support several job postings at once and unlimited access to our resume database for a monthly fee. We may continue to offer the free subscription as a community service for professors who want to advertise a post-doc position or summer internship at no cost. We plan to expand the use of subscription services to many other areas of the nanoHUB.org site as well, so users can pay for additional disk space, faster turn-around time for simulations, etc. We will establish ways for users to pay with the points they earn while working on the site, and also with real dollars.