Why we do not always simulate

By Vincent de Gooyert, Inge Bleijenbergh, Hubert Korzilius, Brigit Fokkinga, Monic Lansu, Stephan Raaijmakers, Etiënne Rouwette, Merel van der Wal

With great interest we have been following and participating in discussions on what ‘good’ and ‘appropriate’ system dynamics modeling is. The system dynamics society, including its conferences and the wiSDom blog provide a stimulating environment for such discussions. However, we noticed with growing discomfort over the last few years a more and more vocal expression of two axioms in these discussions: the idea that every system dynamics project should be accompanied by the collection of quantitative data, and the idea that every system dynamics project should be accompanied by a quantified computer simulation model. The main argument being that not including these two elements is detrimental for the project’s rigor. We fully support the continuous strive for rigor, but we contend that system dynamics projects without quantitative data and quantified computer simulation models can be just as rigorous. In this memo we explain why, in an attempt to advance the debate.

Our main argument is that the answer to the question what ‘appropriate’ system dynamics modeling is, depends on the research question being asked. For some research questions, quantitative data and models are indispensable. However, for other research questions they are not relevant at all, let alone that they would increase rigor. We think that the call for data and models comes from overlooking this latter category of research questions. We are hesitant to use terms as ontology and epistemology, as these terms often come with a wealth of obfuscating language, but we believe this is exactly where the views differ. We see a group of system dynamicists with a positivistic stance striving for rigor in terms of quantitative data and models, and a (smaller) group of system dynamicists with a constructivist approach working on a research agenda that is substantially different.

In the positivist research tradition, different methods are used to get ever closer to an objective reality. Triangulation is an important means to increase validity and computer models and quantitative data provide great ways to check the validity of mental models of decision makers. After all, there is a myriad of biases and heuristics that distort mental models, and the human brain is notoriously incapable of calculating the implications of feedback, delays and non-linearities. In the constructivist tradition there is less interest in the ‘essence’ of objective reality. Rather, it focuses on the meanings that people ascribe to their environment, and how this socially constructed meaning continuously is both the cause and consequence of their behavior. While system dynamicists with a positivist stance triangulate to decrease subjectivity, system dynamicists with a constructivist approach are interested in exactly that subjectivity that positivists try to avoid. Perceptions are valuable, not as a distorted image of reality, but as a complete answer to a research question as for example ‘what meaning do members of an organization ascribe to a particular problem?’ Collecting quantitative data and running computer models are not helpful to answer such questions, it is even completely irrelevant as this would take us further away from an answer to our question how people perceive their environment.

Group model building can be a very exciting part of SD research with a constructivist approach, as it allows researchers to witness the social construction of reality first-hand. Mapping the structure of a system is a great way of sharing worldviews, surfacing and resolving conflicting worldviews, and collaboratively making sense of an equivocal environment. Critical studies, a subgroup of constructivist research, explicitly give voice to worldviews from minority groups. An important aspect of understanding social reality as socially constructed is that not all worldviews get equal space and attention. Powerful actors tend to dominate the social construction of reality. Group model building is one way to challenge the hegemony of dominant worldviews by being inclusive in terms of the stakeholders that are heard and the space each stakeholder gets to contribute to the discussion. This may help to challenge taken for granted assumptions. Sometimes collecting quantitative data supports putting dominant worldviews under discussion, but it is good to be aware that these dominant worldviews affect how qualitative data is translated into quantitative data.

Some suggest that deciding whether to translate a qualitative model into a quantitative model is a trade-off between additional resources and additional insights: is the added value of the quantitative model worth the additional investment? We find it important to note that our objection against the call for quantitative data and models is of a different nature. As stated, for us the difference is in the research questions. Some research questions require data and model runs, but if the research question is about the perception of reality itself, these tools add rigor nor relevance. The terms rigor and validity get a different meaning when used in constructivist research. For us, validity is not about ‘is this model a good representation of objective reality’, but about ‘is this model a good representation of how these participants give meaning to their reality’. There are various ways of increasing confidence in the validity of our models in this sense, including giving participants ample room to adjust the model, and regularly asking the question whether the participants see their own worldview represented in the way it is visualized.

A positivist system dynamicist may respond to our argumentation above by saying: I agree that the confusion stems from a difference in the kind of questions that we try to answer, and to avoid that confusion I would like you to refrain from using the labels ‘system dynamics’ and ‘group model building’ when doing research in the constructivist tradition, let’s confine the use of those labels to ‘proper’ positivistic system dynamics. To those we say: no can do. After all, we are interested in the dynamics of systems, and we build models with groups. We are educated, motivated, inspired by and emotionally engaged with the rich past and future of system dynamics and use these terms to connect to it. We would find it worrisome if dogmas would prevent researchers from answering new types of questions, and from creatively using parts of our body of knowledge for their ends. It might be confusing to have both positivist and constructionist flavors of system dynamics, yet it is also enriching. After all, system dynamics is a field in which we all work with and for people who are primarily meaning giving creatures. The language of system dynamicists with a constructivist approach is still evolving and we are discovering what it is that we do while we do it, we welcome any reflections that readers might have. To all system dynamicists with a constructivist approach we say: let’s work together and collaboratively improve our understanding of the opportunities and challenges of qualitative and quantitative system dynamics, and become more proficient in describing and justifying our approach.

6 thoughts on “Why we do not always simulate

  1. As a vocal proponent of simulation-based (traditional) SD, I naturally feel the need to respond to this post. I don’t want to get into a debate, but rather to make two requests about the use of language; specifically, the terms positivism and model.

    The authors suggest that traditional SD is positivist. It is not. Strict positivists would say that the only variables one should include in a model are those that have already been well-defined and measured. Even the most data-based SD modeler will sometimes include variables for which no good data yet exist but where values could be inferred logically from other variables or evidence. So, please don’t refer to traditional evidence-based SD as positivist. The use of that term here strikes me as disparaging. If you want to use another term for traditional SD, it would be “evidence-based” or “scientific”.

    The authors also use the term model when referring to their group-based causal-loop diagrams. The accepted term for such a diagram is a map, not a model. I realize that some people talk about “qualitative models”, but that is a misnomer. The term model (at least in the field of SD) should be reserved for frameworks that can be tested and compared with real-world evidence (as opposed to unsubstantiated perceptions). This is fundamental and goes back to the founding of the field.

    Like

  2. Thank you very much Jack for your reply. I agree that it is not fair to position traditional SD as positivist. Mental models have been an important part of SD since its origin and that in itself makes the approach much more attentive to subjective views than many other research approaches. Nevertheless, I think it is useful to distinguish between different traditions within SD by pointing out that some are relatively more positivist than others. Suggesting that traditional SD is “scientific” suggest that constructivist SD is not scientific, with which I strongly disagree. The term model refers to a simplified version of reality (“a usually miniature representation of something”, Merriam-Webster dictionary) and I don’t see why a qualitative causal loop diagram does not qualify as a simplified version of reality.

    Like

  3. Hi
    Do the authors of the paper, talk about classic constructionism that considers that there is no intrinsic knowledge but knowledge influenced by the people who know especially in social sciences?

    Like

  4. As a newcomer to the field of System Dynamics looking to contribute to advancement of the work (via a focus on SD education and training programs) I would like to propose that there is an opportunity in this dialogue to consider this: what does it mean to be a system dynamicist? That is probably not a simple question, and certainly the answer(s) are neither simple nor singular, judging from the lively conversation here. But this blog and the comments that follow begin to unpack the values, assumptions, and practices behind what it means to be a system dynamicist. Understanding that richness has in it wonderful implications for determining specifically the valued capabilities that lead to development of a system dynamicist. In fact, several different capabilities emerge from the discussion, sometimes overlapping and sometimes different depending on the positivist or constructivist approach to model building presented here! I look forward to learning more about what it means to be a system dynamicist – perhaps multiple meanings of the role – and imagining how that translates into or informs the learning goals, instruction, and assessments for SD education programs.

    Like

    1. Hi Vincent

      I answered to your post because of the difficulty for system dynamic people to model the reality and the second difficulty to share this understanding with those concerned by that reality and are often a part of it, mainly the decision takers. See the post about the optimal size of a model.

      Best

      Like

  5. Hi Lauble, we used the label of constructivism mainly to point out that there are different research traditions, with some focusing more on the meaning people ascribe to their environment and how this socially constructed meaning continuously is both the cause and consequence of their behavior.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.