Integrating managerial preferences into the qualitative multi-criteria evaluation of team members

Managers can find it challenging to assess team members consistently and fairly. The ideal composition of qualities possessed by good team members depends on the organization, the team, and the manager. To enable managers to elucidate the qualities they require, we make use of an innovative methodology. This methodology is based on a multi-criteria decision aiding process, starting with the identification and definition of the dimensions that will be used to evaluate team members, then the inference of the manager’s preferences through a multi-step protocol combining multiple types of preference models, and finally extracting a set of rules that can support the manager in his/her tasks. We illustrate this methodology in the case of free/libre/open source software development teams, where we were able to elicit the characteristics of a good, acceptable or bad contributor based on multiple managers’ perspectives. We additionally provide an example on how to reproduce this experiment using the MCDA package for the R statistical environment.