Scholarpedia:Invitation-only policy

Summary of suggested ideas (not all of them are related to the invitation-only policy):

• To register, a new user needs 1 invitation from a curator or 2 invitations from other existing users.
• When you invite a new person, your Scholar index may go up or down depending on the activity of those whom you invited.
• Articles have "interest groups", which discuss possible modifications before contacting the curator to approve/decline them.
• There is a barrier for certain articles: To modify them, you need high Scholar index.
• Curators with high Scholar index can reserve and write an article that is (somehow) determined to be in their area of expertise by the graph-theoretic approach (based on the set of articles where they earned the high Scholar index).
• Change of registration policy must be announced to the community.

Discussion prior to March 21, 2008

The process of authoring an article in Scholarpedia is quite selective ("invitation-only"), but registration in Scholarpedia is not. After a short "probation" period, a newly registered person can start "improving" articles anyway he/she wishes. I propose an "invitation-only" method for registration too. This is similar to the Society for Neuroscience membership policy.

Main idea

Once the number of registered users in Scholarpedia is large enough, the process of registration should be "invitation-only". That is, to create a new account, a person needs to be invited by \(m=2\) (this is a parameter) already registered users. Each registered person has a credit of how many new users he/she could invite, e.g., 100. While a registered user invites a new user, the registered user can share some of the credit, e.g., give the new user the ability to invite 5 more people (so the registered user will have the remaining 100-1-5 = 94 credit-invitations). This way, people will not agree to spend their invitations for strangers, but only agree to invite those whom they personally know. If a spammer gets into the system, the 2 people who invited him will be penalized - their credit will be frozen or removed, and the users who invited these two users (the grand-fathers of the spammer) will get a note.

Depending on the Scholar index of a user, he/she can accumulate invitation credits. People's scholar index may reflect the activity of the users whom they invited, etc. I did not think about this yet.

Of course, editors and assistant editors would have full authority to invite anybody without any invitation (i.e., only one invitation from an assistant editor is needed for a new person to register).

For now, I keep this project a secret and we were quite fortunate and honored to get full support of the best scientists and minimal interference from spammers. This may change in the near future, if Science or Nature or some other big journal finds out about this project. We may be flooded with unwanted users whose only goal is to self-promote themselves or sabotage Scholarpedia.

Discussion

Put your discussion here.

How about starting out with a smaller credit? I think it'd be a good idea to even have an initial credit of 0 for a newly registered user, and only increase it once the scholar index exceeds a certain (small) threshold. That way, all users are forced to contribute if they want Scholarpedia to grow. This will help carry the "spirit" of this project on to newly registered users.

I like the idea of "reverse inheritance" of scholar index increments, i.e. the scholar index of a user is incremented by a small fraction of the scholar index increments of the users he/she invited. However, this fraction has to be sufficiently small in order to ensure that the resulting index doesn't grow too fast without the corresponding user actually contributing. (If you know that I mean... I probably should formalize it.) --Bouchain 21:11, 9 March 2008 (EDT)

How about this recursive approach:

Scholar i element in {1, .., total number of scholars};

For each scholar i, there is a child generation, a set N_i of childs {N_i(1), .., N_i(n_i)} which is a subset of {1, .., number of all scholars};

Number of scholars in i's child generation n_i;

Scholarindex a_i of scholar i;

Self collected points b_i of scholar i;

a_i = b_i + 1/2 * (1/n_i) * ( a_(N_i(1)) + .. + a_(N_i(n_i)))

--Denninger 09:22, 10 March 2008 (EDT)

I hope you don't mind me taking the liberty of LateX'ing the above:

Scholar \(i=1\ldots m, m\) total number of scholars.

For each scholar \(i\), there is a child generation, a set \(N_i\) of childs \(\left\{N_i(1), .., N_i(n_i)\right\}\) which is a subset of \(\left\{1,\ldots, m\right\}\);

Number of scholars in \(i\)'s child generation \(n_i\);

Scholarindex \(a_i\) of scholar \(i\);

Self collected points \(b_i\) of scholar \(i\);

\[a_i=b_i+\frac{1}{2}\cdot\frac{1}{n_i}\cdot(a_{N_i(1)} + .. + a_{N_i(n_i)})\]

If I broke your formulas somewhere, feel free to just fix it. The recursion of the calculation is inherent to the tree-like structure of the user-base, once the invitation-only regime is activated. What you do here is, you weigh the increment by \(\frac{1}{n_i}\). This is not a good idea in my opinion. For example, say scholar \(A\) has invited one user who's very active, and one user who only does very few or no contributions at all. Scholar \(B\) has only invited one user who's also very active. Now, whenever the ambitious users have their scholar index incremented, scholar \(B\) receives twice as much increment on her own scholar index than scholar \(A\). This somewhat punishes scholar \(A\) for inviting a "lazy" user.

Do we want that? As Eugene mentioned, users who invite spammers should be punished somehow. This is a good idea (and necessary). Users that just don't contribute (just like me up until recently), though, don't hurt the project. So, I suggest a simplified version of your formula\[a_i=b_i+\varepsilon\cdot\sum_{k=1}^{n_i}a_{N_i(k)}\], with \(\varepsilon\ll1\)

I also noticed that children's scholar indexes should only affect the parent's index until a certain value is reached, just to prevent a scholar from becoming an editor without really contributing. Let me know what you think.--Bouchain 10:52, 10 March 2008 (EDT)

The last point is very important. After all, the Scholar Index should reflect the expertise of a given user, and not how easy he/she is in sending invitations to other experts. We want participation in this project to be selective, based on the academic and scholarly merits and not on the "social networking". User:Eugene Izhikevich

Precisely. A simpler way to reward users for their invitations might be to have the contributions of the invited users reviewed by "someone", and make a small increase of the scholar index up to their judgement. That, however, needs manpower. Or, as Abdellatif suggests below, maybe invitations shouldn't be rewarded at all.--Bouchain 14:21, 10 March 2008 (EDT)

Also, keep in mind that we do not want to encourage newcomes to edit all articles, just to get increase their scholar index. If an author, say John Hopfield, receives email alerts every week that there is a minor change to his article and he has to maintain it constantly, he will quite pretty soon to be the curator of the article. I would like to minimize the burden on the authors, especially senior ones, like Ichiji Tasaki (who is 97) or Rita Levi-Montalcini (who is 98). In other words, it is better to have fewer changes, but good changes, than many small and questionable changes to the articles. I do not know yet how to achieve this. Many authors, before agreeing to contribute, ask me how extensive the workload is after the article is written. So far, I reply, not that much - maybe an hour per year total. It is not clear whether it will be much more once the project really starts growing. Indeed, everybody in the field of computational neuroscience knows about Scholarpedia, yet articles in this field are not modified every day (or even every week). Thus, if/when Scholarpedia grows to neuroscience and applied math, there will be the same ratio of readers-to-articles, so hopefully, there will be the same rate of modifications (on average, a few modifications per year).

Again, a manpower approach: Instead of sending the pending contributions directly to the curator, they could be sent to "some group of users" who would then decide on whether to discard the changes, accept them right away (e.g. typos), or to actually send them to the curator (changes in content/meaning). It's an honor that we have to many gurus on here writing articles, and we shouldn't bother them any more than absolutely necessary.--Bouchain 14:21, 10 March 2008 (EDT)

To summarize, I want the exclusivity not to keep the numbe of modifications down, and not to increase it, but to make the modifications more useful and thoughtfull. Izhikevich

The edge of a knife. We want to encourage users to invite people who can contribute to Scholarpedia, but we also want users to only invite people who will contribute. It's a big issue actually, thanks for bringing it up.--Bouchain 14:21, 10 March 2008 (EDT)

--

Dear all, I would vouch for a simpler approach. I don't see the fact of inviting people as worthy of an award such as an increase in scholar index. If someone invites people who have some negative impact on Scholarpedia, their inviting privileges should be frozen for a period (2-3 months). If the invited scholars have no impact, or a positive impact, it's all good, and the inviting scholar should retain his inviting privileges. Ten invitations (accepted ones) seems a good number. When this credit is used, it could be renewed after someone (assistant editor?) checks how the 10 invitations were used.

On the other hand, I like the co-optation by 2 users as a basic rule. Scholarpedia could function as a scientific society (it already does, somehow). I guess curators will have the same privileges as editors and assistant editors in that respect? I wouldn't be too comfortable with seeing a curator penalized by a bad call for an invitation, or in need of another user's support to invite someone. -- Nemri 13:48, 10 March 2008 (EDT)

We can have the following rule to register:

• one invitation from a curator or an assistant editor (who are curators already)
• two invitations from any other user

The simplest way to implement that is the following: When a new person clicks 'create account' link, he/she sees a window to typein 2 keys (two 8 symbol hex numbers) and a short explanation of the policy and how to get the keys. The person need to go through the list of all users and find those whom can voucher for that person. Then, he/she contacts those two people, who go to a special place in Scholarpedia (while logged in) and generate the key. Each key is stored in a database, and sent to the registered user, who then forwards it to the new applicant. Once the new applicant gets 2 keys, he/she goes to Scholarpedia, provides the keys and then creates a new account. The keys are stored in the database, so there is a history trace and they cannot be used again. Izhikevich

Sounds good to me. To answer another of your points (in your email), I think you should keep the invitation-only policy for writing an article. --Nemri 15:04, 10 March 2008 (EDT)

I like David Bouchain's idea of "interest groups" for some articles. Small modifications could be pre-filtered by the students before a Nobel laureate is contacted to approve/decline a major one. The interest group can have a discussion in the 'reviews' page. Actually, the group members may get a kick out of this - the possibility to interact with a living legend. I just need to figure out how to implement that.

Regarding rewards, I also do not think that getting rewarded (via Scholar index) for invitations is important or useful. Often, being asked to send an invitation is rewarding enough, at least for me. Izhikevich

I think the policy for user invitation is a very crucial step regarding the future development of Scholarpedia. It is therefore worth to think and discuss in-depth about this issue. The idea behind the approach I suggested was following: the growth rate of Scholarpedia in terms of registering users could be seen as a trade-off between speed and quality. The aim of the suggested approach is slow (linear) growth with highest quality. To achieve this, following invitation strategy of each scholar gets rewarded: try to invite the most promising person. This is achieved by rewarding the inviting scholar with a certain fraction of the mean of the scholar indices of the persons invited by him. Motivation to invite a further person is given only, if the inviting scholar expects that person to obtain a scholar index higher than the mean scholar index of the already invited persons and that way to increase his reward.

This approach for invitation reward might be of course combined with the idea of the two invitations that are needed and with the limit of possible invitations to prevent from spamming. --Denninger 16:35, 10 March 2008 (EDT)

Tobias, you treating Scholar Index as some kind of a money. I think you are too serious about it. Its major goal is to figure out who is to inherit a particular article when the curator passes away or quits (the latter never happened so far). Originally, the whole mechanism of Scholar index was invented so that the project can grow without any supervision vis contributions of postdocs and junior professors. However, soon it became clear that it would not be that difficult to invite the best people to write articles, so I enabled the invitation-only regime for articles. I plan to remove the "invitation-only" mechanism once most of the important articles are distributed among the best experts. I just need to figure out how I can limit this to certain topics. For example, I would not like the best expert whose Scholar index is 100 in computational neuroscience to reserve the article "Religion" or "Cosmos" or something like this. That person should select the article within the field where he/she is an expert (where he/she raised the Scholar index). It is easy to say, but difficult to implement, at least at the present stage; but I will solve this problem, possibly with the help of assistant editors Izhikevich.

Eugene, thanks for clearing that up a bit. I thought about the scholar index more in terms of a monetary system, too, but it makes indeed more sense to invite/promote users based on their respective expertise as opposed to their number of contributions, which might be scattered all over Scholarpedia anyways. So then, yes, I can very much agree that rewarding parent users for the contributions of their child users won't really affect the quality of Scholarpedia. The above formula might make a good measure for "meta contributions", i.e. contributions improving the user base and infrastructure, but for finding experts among registered users it now seems more or less irrelevant.--Bouchain 10:13, 11 March 2008 (EDT)

An idea regarding limiting the burden for curators: If the work-load becomes an issue for a curator an artificial barrier for article modification permission could be introduced.

The barrier might be a number and different for each article. The barrier number defines a certain set of criteria the editing user has to meet. The higher the number, the more difficult for the editing user to meet the criteria which might be: Minimal quality of the recent contributions of the editing user, maximal number of modifications of the editing user to limit the edit rate, competence criterion (editing user is curator of an article in the same category), etc.

The curator can determine which maximal amount of workload he agrees. The barrier number might be self-adapting, meaning that if the modification rate exceeds the workload chosen by the curator, the barrier gets elevated, that way decreasing the modification rate. --Denninger 06:35, 11 March 2008 (EDT)

An idea regarding competence specificity: A distance matrix for the articles could be defined. The distance matrix might be based on distance between multidimensional vectors that gets computed for each article. Those vectors might depend on the set of incoming links, outgoing links, editing user, viewing user etc. Of course for a non-existing article, maybe only the incoming links exist. (Having a discussion process before the author determination might be a possibility to obtain other values though). The reservation privilege of an article could then depend on the quality of contributions in the neighbourhood of the respective article, given by the distance matrix, independent of high quality contributions of more distant articles. --Denninger 07:02, 11 March 2008 (EDT)

Eugene: The idea of a threshold for modification is good. The threshold may be just the scholar index; that is, any user with a Scholar index above, \(x\), may edit the article, but not below. The number \(x\) could be the total score that the author assigned to previous modifications divided by 10; that is, if there were 20 modifications to the article and each one was graded as 'improvement', the total score is 20, so \(x=2\). This way, to modify the article, the user needs to work his way to Scholar Index=2 (making other useful modifications) before bothering the curator of the article.

Concerning Tobias' suggestion, it is not that difficult to make the graph-theoretic analysis and determine a topic's neighborhood for an existing article, which existed for some time and links to which were incorporated into many other articles. But how can you do that for a completely new article? For example, the article "space" does not exist, and may not exist for a long time. However, practically every other article mentions this keyword, so how do you determine that "space" should be assigned to somebody with high Scholar index in physics, and not, say in psychology. Izhikevich

For a completely new article, incoming links might already exist (the "red" links) which provide a vector of dimension n (n number of articles) and elements 0 or 1 (incoming link exists or doesn't exist). The completely new article might be a relevant topic in several contexts, whether this is case can be found out by analyzing the vector cloud of the articles in the neighbourhood. If there is only one dense cluster, the context will be clear, if there are several clusters, the article might be relevant for different contexts, and there can be an article "space (physics)" and another one "space (psychology)". --Denninger 11:54, 11 March 2008 (EDT)

If it's not clear, another idea to find out who might be the best expert for a certain article, might be to have a anonymous discussion procedure about contents of the article, before the decision for the author. The discussion postings can be evaluated (weighted by the participants competence), and the discussion participant with the highest evaluation might decide about who is going to be the author (or to be author him/herself). --Denninger 12:05, 11 March 2008 (EDT)

I see that in this discussion, we deviated from the focus topic - whether or not having "invitation-only" policy for registration of new users is useful. Let me return to this question.

• Right now, there are almost 4,000 users. What is the threshold for implementing the invitation-only regime? 5,000? 10,000?

I would suggest the first spammer problem as the start signal. Or 2009. Whatever is decided, it will be subjective  :)

Seems reasonable. Let's hope Scholarpedia grows some more before this starts to be a problem. See below.

• Do we require one invitation initially? Or do we start with the requirement of 2 invitations (from other users) and one from a curator?

Two invitations sound good.

I second that. However, the barrier to registration is huge compared to only one invitation. I mean, what are the odds of knowing two users personally, with the relatively small user base we have now?

• Do we advertise that Scholarpedia will have a stricter registration policy soon and give people the last chance to register freely, or do we just close it one day and implement the policy?

I agree with Tobias, some communication would be appropriate before the change of policy. However, it kind of conflicts with your wish to keep the project as secret as possible. Also, it would be good to define the scope of Scholarpedia and allow people from all fields to join freely before closing the door. Otherwise, the neuroscientists, mathematicians and astrophysicians would be favored over other disciplines. --Nemri 18:05, 11 March 2008 (EDT)

Again, agreed. Eugene, how exactly would you advertise this, or Scholarpedia as a whole? (This might be a discussion on its own.)--Bouchain 20:50, 17 March 2008 (EDT)

An alternative to the invitation-only-policy or an add-on: In order to become a new user, one has to submit a reasonable article/author suggestion. Then someone has to decide whether the suggestion is reasonable. Sorry for deviating again..

I think the new registration policy should be anounced, otherwise people might be made insecured about what will happen next. --Denninger 12:43, 11 March 2008 (EDT)

Newly registered people must make title/author suggestions

Uppend your discussion here in the chronological order.

I'd just like to remark that, given the current growth rate, I estimate the 4000th user to register during March and the 5000th user during July. Denninger