DAOrayaki: A more valuable DAO reputation quantification model than SBT
Original author: TalentDAO@xrex
Original Title: Quantifying Reputation in DAOs
Quantitative models of reputation may make more sense than the single qualitative data points provided by SBT. This article mainly discusses how to quantify the reputation of contributors in the DAO network. In the next post, we will explore the potential impact of this quantitative modeling framework on DAO governance.
several assumptions
A person's power in an organization is equal to his reputation;
Reputation equals contribution;
Contributions can be enumerated, measured and quantified;
The overall quantification of contributions to the DAO enables a fully transparent, trustless and fair governance system.
reputation is multidimensional
Reputation is made up of many factors. Which factors we choose to observe and how we choose to measure them can have a huge impact on the decisions we make about a person; one needs to be wary of subjective measurements involving moral and ethical considerations. Subjective measures such as peer reward circles (e.g. Coordinape) can capture contributions that are not fully enumerated, while also incentivizing emerging growth through new endeavors. However, I will do my best in this article to stick to objective measurements and think from logical first principles.
If new hires in the organization already have a reputation; then it is because of contributions they made elsewhere – whether in their personal or professional lives. These contributions give them a transferable reputation.
In a web3 world where anonymity, privacy, censorship-resistant, trustless, and permissionless systems rightly reign supreme, we can often only rely on reputation when evaluating new organizations or individuals. Therefore, reputation is very important in determining how much power someone has in an organization. The good thing is that we can track actions and thus contributions on-chain.
DAO contributors can build reputations by participating in chats, forums, voting, writing proposals, voting, attending meetings, completing bounties, and countless other ways. These behaviors can be quantified in various ways and rewarded with tokens. Non-transferable NFTs, or "soul-bound" tokens (SBTs), have over the past year become the gold standard for issuing reputation credentials on-chain. However, this approach is insufficient to adequately represent the complexities of human reputation, especially in digital environments.
Some methods of creating numerical representations of human reputation do involve a different set of input data sources. Reputation models can be based on contributions drawn from various data sources both on-chain and off-chain, then aggregated or compared to calculate a given metric and certificate eligibility (“badge of merit”). A few examples of this are Gitcoin Passport and Orange Protocol's reputation NFT.
Using models allows decision makers to fine-tune their choices of inputs. On the one hand there is a complete lack of authentication and on the other hand a comprehensive evaluation of all enumerated variables. Quantitative models of reputation may make more sense than the single qualitative data points provided by SBT.
Why Organizations Use KPIs
A KPI is a means of measuring performance towards an objective. Improving KPIs should not be the primary goal of any initiative. It's like a dog chasing its tail. Optimizing for KPIs does not solve the original problem and leads to "training the model" on irrelevant datasets. Humans are inherently biased and subjective. Humans lie not only to others but to themselves, often unintentionally. The method of control is the use of objective data and analysis.
What is a contribution?
Proof of work with unlimited design space.
Contributions include:
an action/behavior (job)
result(value)
Documentation (proof of work)
Four Contribution Types
Governance: Proposal, Voting, Operation, Review
Finance: investments, grants, acquisitions
Effort: Activities based on performance and deliverables, time spent in meetings, and project work tracked
Social: chat and forum ideas, feedback, meetings, promotions
contribution score
By tracking and quantifying contributions in various ways, it is possible to create an algorithmic and overall system that is robust against fraud and more representative, fair, and inclusive with less input.
Different levels of accreditation (credentials) and acquisition rules need to be established to differentiate between individual contributors and communities. Everyone's actions need to be enumerated. This seems like a daunting task until we impose some structure on the process.
Individual contributions can be normalized using a transformation function, followed by time filtering to adjust their relative weights. The weights of any particular contribution can then be aggregated according to a metafunction that scales categories of contribution types according to each community-specific distribution (further described below). The end result is a single aggregated value, which we describe as a "contribution score".
Conversion function: data conversion
Linear: x = n
"The graph is a function of a straight line, that is, a polynomial function of degree zero or one."
Quadratic: x = 2^n
"A polynomial function of one or more variables in which the highest-order term is second order."
logarithm: x = log2(n);
The inverse function of exponentiation. "\"Another fixed number [...] to which exponent must be raised"
time filter
Each input can be tuned to focus on the relevant time range or weighted to determine which contributions are included in the counts sent to the transformation function. This section and concepts were originally inspired by the model used in the SourceCred grain distribution.
SourceCred has three policies for how projects allocate rewards ("grains") for contributions - recent, immediate, and balanced. The model presented here extends this to include beliefs—a policy of increasing the weight of inputs over a given period of time.
The concept of temporal filters can be applied to compute the contribution score for any given snapshot.
1. Instant
"This will distribute rewards equally based on each participant's contributions in the last week. (This policy ignores all contributions from previous weeks and is intended to provide quick rewards for active participants)."
2. balance
“This distributes rewards based on lifetime contributions and lifetime reward earnings. A balanced temporal filter attempts to ensure that everyone in the project receives a total reward payout consistent with their total contributions throughout their participation period.
For example, suppose a contributor has contributed a small amount in the past and thus received a small reward. However, the community has recently changed the weights, or added a new plugin, such that contributors now have more records for their contributions.
The balance policy thinks this contributor is underpaid, so it pays extra money to "catch up" to everyone else on the project. Conversely, contributors may 'overpay' and they will be rewarded with less until the payout is balanced. "
3. recently
"This distributes rewards based on recent contributions, using exponential decay to prioritize recent reputation." The recentWeeklyDecayRate parameter determines how much you want to focus on recent contributions. If recentWeeklyDecayRate is set to 0.5 (i.e. 50% discount), as in the above example, the strategy will calculate 100% of the contribution generated last week, 50% of the contribution generated the previous week, and 25% of the contribution generated this week. 12.5% for the week, and so on. "
4. Faith
There is a fourth possible mode popular as a modifier of governance that can be used as a time contribution filter. Conviction voting incorporates the concept of increasing commitment to a decision over time. The same concept can be extended to compute contribution scores based on weights that decay continuously over time rather than a fixed period of time.
output
image description
Figure 1: Illustrative model components with a rich input set
metafunction
A metafunction is defined here as a compound function composed of aggregate functions. A property of aggregate functions is that the dimensional output of a reduction computation is reduced relative to a set of input functions. A meta-function can be used to compute a normalized contribution score that reasonably approximates reputation-based participation in a community. Computing a normalized contribution score from a set of output values (Y) representing scaled and normalized quantifications of each individually contributed input source requires three steps.
Determine the weight of each contribution type.
Governance: XX%
Finance: XX%
Efforts: XX%
Social: XX%
Note: If contributions are assigned to more than one type, they must be calculated separately.
Determines the conversion function used to combine contribution types.
(a) + (b) + (c) + (d) = total weight
Note: It may be simpler to use linear transformations for all metafunctions and avoid redundant data transformations. Finer control can be obtained by transforming each individual input with an appropriate transform function.
The final contribution score is normalized by taking the average of the total weights.
Total Weight / # Contribution Type = Normalized Contribution Score
Write at the end:
In the following, we explore how the contribution scoring model can be applied to DAO governance, caveats and limitations, and potential implications.
