Increasing voter turnout in AC reviews with incentives

[anssiko]

There has been a soft rule that 5% of the members of the Advisory Committee need to support a proposal (typical case: new/revised charter) in its AC review to pass.

However, experience has shown it often requires an active push with a direct connection from either W3C Staff or other active participants to ACs to get there. This is not surprising. Voting means work for the AC reps, who in many cases are busy and need to conduct a legal review internally and possibly do other paperwork before casting a vote.

Can the W3C Process create better incentives for AC reps to cast a vote? I'd prefer carrot over stick. Example proposal: active ACs get a badge of honor next to their profile ("Pro") if they have voted in X% of AC reviews annually.

[dwsinger]

I also prefer carrot over stick, but the tricky part is finding something that is one but not the other. In this case, the lack of a "pro" sticker might be seen as implying a criticism, an "amateur" status. We wondered about marking voting statistics, but decided that had a greater shaming effect than incentive. We wondered about modifying the guidelines to say "5% of the normally active AC Reps" but then we're back with the marking (and calculating) problem.

[nigelmegitt]

Do we have any data about what AC members feel their AC reviews should be testing? I'm guessing most of the AC reviews are for CR->PR transitions, and I know from experience that getting a high voter turnout is difficult. Perhaps we need to check the things we're asking AC to vote on during reviews are things they care about and want to have a say in?

[palemieux]

I think it is expected (and healthy) to (usually) see low engagement from the AC, which is a diverse group:

• during charter review, since new projects are often in novel areas where only few AC members have expertise. I would in fact argue that, for charter reviews, the question really is does anyone object to this new work item?

• during PR transition review, since the technical work is complete and the AC should not delve in detailed technical comments. In fact, I would argue that the AC review should be focused on whether the process was followed.

[chaals]

There's not much incentive to vote for something unless it matters to you. It takes work to analyse any issue, so if it isn't directly relevant to your company, you might find it hard to justify the work. I find it equally hard to justify expressing an opinion that has been hastily formed about something that matters little to me but much more to others, as a "vote".

I am not sure this should really change. The soft rule has the (slightly perverse) effect of meaning that well-known individuals can start work more easily, because the important question becomes how many AC reps you can rally to support it. It probably should not become a hard-and-fast rule:

At least as long as we have a technical director and staff, it makes sense to allow W3C a certain amount of latitude to direct some resources to work they consider will be important to the Web. But I think if we have a mechanism in place for W3C to lean on the scales somewhat, we will end up with an actual rule for determining when we commit the resources that members fund, to work that is intended to be issued and adopted with the recommendation of the collective membership behind it.

I conclude that this isn't a problem we can solve without insisting people do what amounts to busywork in order to simplify our own lives. That doesn't seem like a good plan.

[TzviyaSiegman]

I was going to say what Chaals said. I don't think we should push people to vote on topics with which they are not familiar. (Separate topic, I think we need to simplify the voting process.)

[dwsinger]

The tricky part is that there is a constant undercurrent of two complaints:

• the W3C allocates resources (the most obvious being staff time) in the wrong emphases or the wrong way
• voting seems pointless, it's all decided anyway

These are both dangerous perceptions.

I agree we can't expect every AC member to care about every subject. But we should make it easier to care about questions on charters like

• is this an appropriate resource allocation?
• are there general questions about this charter (e.g. the decision policy, who is chairing, its overall scope) that are of non-specialist concern?

Unfortunately changing the overall emphasis on use of resources by voting on charters is trying to adjust the direction of a ship by hundreds of small nudges. Each individual action might seem OK or not important, but the cumulative effect is important.

On Recs, I suppose we should have the formal ability to say 'no' on the PR transition, but it should be a formal rubber stamp. Has the AC ever said no? Saying No at this stage is brutal; a WG has completed work, been through reviews, and so on.

Yet going into CR is not currently a question for the AC. Is this a suitable CR, good enough to be implementable, something we ought to be asking for implementation experience of, a quality addition to our stable of work, congruent with other specs and our architectural directions, and so on?

So concretely, maybe Charter review should tease apart 'generalist' questions, such as "are you comfortable with the resource allocation?" and "are you comfortable with work being done in this general area?" and "is the overall structure and the operational part of the charter acceptable?", and specifically separate out "I care about this charter and this work" questions.

Mostly thinking out loud here.

[nigelmegitt]

Seems like AC review turnout needs to be thought about separately for a) CR -> PR reviews and b) Charter reviews [and by extension, for c) anything else].

CR -> PR reviews

Reasonable grounds for saying "no" should be limited to things like:

• due process has not been followed.
• AC disagrees with WG's handling of objections, e.g. if a Chair has decided to proceed despite an objection, but disagrees with the rationale or the outcome.
• some kind of "force majeure" scenario where events have overtaken the work but the WG has decided to proceed despite them, or process was initiated before them, and AC thinks it would be better to shelve the work anyway.

Force majeure examples?

I suspect this has never happened - I'm not sure what a good example would be, maybe [thinks out loud] a spec based around an encryption technique for which a new mechanism has been recently found to break it, and the WG thinks "it's still worth advancing for cases where the new mechanism is impractical" but the AC thinks "sorry we have to go back to the drawing board now we have this new mechanism".

Unreasonable grounds for saying "no" might include technical objections, which should have been raised earlier by interested parties.

Turnout? Default vote?

The assumption should be silence is assent.

The need for a minimum number of AC reviews seems pointless. However, this could be made stronger by entering a default vote for AC members, of "no objection" unless it is deliberately overridden, with one of the override options being "abstain". This would nudge AC members to express a view, if they have one, but give minimal work if they do not.

Charter review

Reasonable questions to ask the AC are:

• Do you agree with the strategy team that W3C should be doing this work next?
• Do you agree that we should spend this much team resource? (and if no, what would be a better amount?)
• Is this Charter adequate and appropriate in terms of scope, specificity, architectural approach (TAG review of the scope of Charter deliverables, anyone?), proposed timeline, links with other groups, internal and external to W3C, success criteria?
• Are the proposed policies acceptable in breadth and detail, e.g. Decision Policy?

Turnout?

Here I think it makes more sense to require a minimum voting level, and would not enter a default vote. Positive support at the Charter stage is much more useful than at the PR transition stage.

[dwsinger]

Note that charters are normally subject to greater scrutiny when they fail the appallingly low 5% threshold.

[palemieux]

Positive support at the Charter stage is much more useful than at the PR transition stage.

I expect few explicit votes of support at charter review: a more diverse membership means a broader range of interests and relatively fewer experts in each domain, especially if the domain is on the cutting edge. Some groundbreaking efforts sometime start with just a few visionaries.

I would therefore assume approval unless explicit objections are raised.

I would also suggest that these objections need to clear a high-bar, e.g. "contrary to the mission of W3C", "outside of the scope of W3C", "cannot result in a deliverable under any circumstance", etc... In the case of charter renewal, lack of measurable progress could be grounds for objection.

[dwsinger]

we are going to leave this to practice, guidelines and the AB and not change the formal process