[The Electronic Oracle] ② The Questions to Ask When You Face a Model: A Checklist for Citizens and Policymakers

 

[The Electronic Oracle] ② The Questions to Ask When You Face a Model: A Checklist for Citizens and Policymakers


Before we turn to the technical yardsticks for evaluating models (Post ④—research) and the philosophy of assumptions (Post ③), we need to learn how to ask questions. A computer model becomes a modern “oracle” when it dulls our judgment. To prevent that, here are three questions you should ask every time you meet a model.

This checklist is not mainly for model builders. It is for model users—citizens, journalists, policymakers, and administrators.

Question 1: “What number is currently serving as our ‘authority’?”
(Focus: guarding against false certainty)

A model becomes an oracle when its output—stripped of context—poses as the essence of reality.

Checklist

  • Is the number accepted without proof?
  • Is this number used as the sole justification for a policy decision?
  • Have the conditional statements ("if") attached to the number been cut from meetings, reports, or documents?

Examples

  • "Next year's growth rate is 1.2%." — Did the condition "if exchange rates soar" disappear, leaving only the figure?
  • "The reproduction number (R) is 1.3." — Is the realistic limitation of "data aggregation delays" hidden behind this digit?

๐Ÿ‘‰ To understand deeper: This question connects to the concept of 'Precision vs. Accuracy' in Post ④. 

Question 2: “Are the assumptions public? Who holds the authority to modify them?
(Focus: Securing transparency and democracy)

A model's conclusion is ultimately a product of its Assumptions. If assumptions remain in a black box, verification, rebuttal, and modification become impossible. At that moment, the model becomes an authority outside the public sphere. It becomes a dictatorship.

Checkpoints:

  • Is the Structure (causality, delays, feedback) publicly available?

  • Are the Numbers (initial values, coefficients, delay times) publicly available?

  • Is the Boundary (what was excluded) clearly explained?

Case Study:

  • Behind the conclusion of a "teacher surplus for the next 10 years," does a rigid assumption hide: "the number of students per class will not change"?

๐Ÿ‘‰ To understand deeper: The specific types of assumptions (boundary, structure, parameters) are covered in Post ③. How this relates to responsibility is detailed in the distinction between 'Exogenous vs. Endogenous' variables in Post ④.

Question 3: “What is deleted when translated into policy language?”
(Focus: Restoring context)

When a modeler's complex output is translated into a "one-line summary," critical information evaporates. We must bring these lost elements back to the table.

The Deletion List (What is often lost):

  • Conditional Sentences: "If A, then B" becomes "It is B."
  • Range of Uncertainty: Scenario variations are collapsed into a single average.
  • Time Delays: Today’s effects get airtime; five-year side effects vanish.
  • Feedback/Side Effects: The structure explaining "why it happens" is removed, leaving only "we must do it."

Case Study:

  • Behind the one-line conclusion —“We expect an electricity shortage, so let’s build power plants” — has an alternative scenario—“efficiency policies could reduce demand”—been deleted?

๐Ÿ‘‰ To understand deeper: How do we restore this deleted context? By monitoring through Externalization (Post ③) and Structure (Post ④).

[Epilogue] These three questions are not intended to neutralize models. Instead, they aim to dethrone models from their status as "mysterious prophets" and transform them into "useful tools" we can wield proactively. With these questions in mind, let us now examine the three standards that pierce the essence of modeling (Post ④) and the philosophy of assumptions (Post ③).

Comments

Post a Comment

Popular posts from this blog

Thinking in Systems Guide (1) – Preface

Image
   The Beginning of Systems Thinking: A New Lens on the World Taught by a Slinky "If a factory is torn down but the rationality which produced it is left standing, then that rationality will simply produce another factory." - Robert Pirsig (Here, “rationality” can be understood simply as the reason or purpose behind it.)   Why do we always face the same problems? Have you ever had this experience? You worked hard to solve a problem at work, but before long, a similar issue appeared again in a different form. Or you may have tried to break a bad habit, succeeded for a while, and then suddenly found yourself back at square one. These kinds of repetitions happen because we focus only on the symptoms of the problem. The true cause—the underlying structure —remains untouched. The Slinky Experiment: The First Insight of Systems Thinking Donella Meadows, who taught system dynamics at MIT, often conducted a fascinating classroom experiment. She used nothing more than a Slinky—t...
Read more

The + and - Symbols: Simple Enough for Elementary Students, Yet the World's Most Confusing Concept

Image
Let me talk about the most challenging concept in the world—at least for me. It’s so simple that even elementary school students understand it. But believe me, it gets you when you least expect it. I’m talking about... + and -. And yes, there’s a twist. Actually, there’s a twist to the twist—so buckle up.  In my systems thinking lectures, I typically explain that when cause and effect move in the same direction, we use the + symbol and refer to it as a positive causal relationship. When they move in opposite directions, we use the - symbol and call it a negative causal relationship. I always emphasize firmly that "positive" and "negative" here are purely mathematical concepts—they have nothing to do with good or bad. Elementary School Level   For example, when the birth rate increases, the total population also increases. Since both birth rates and population changes move in the same direction, we write:  Births → Population (+)  Conversely, when the death rate incr...
Read more

[The Electronic Oracle] ③ Assumptions Aren’t "Guesses": A Proposal for Transparent Democracy

Image
  [The Electronic Oracle] ③ Assumptions Aren’t “Guesses”: A Proposal for Transparent Democracy When I teach System Dynamics or computer modeling, I often encounter a word that instinctively annoys both students and practitioners: "Assumption." The dictionary reveals the source of this discomfort. Assumption :  “a thing that is accepted as true or as certain to happen, without proof .” (Oxford Languages) For scientists, the phrase "without proof"  can sound like an insult. So when someone says, “That’s just your assumption,” scientists rush to defend themselves: "No! I have evidence!" In this post, I want to restore the honor of this word. Modeling assumptions are not mere tricks to patch missing data; they are influential political acts that transform secret dictatorships into transparent democracies. This article distinguishes two layers of assumptions: The Assumption as Belief (Type A)   hides in the mind and operates on authority The Model Assumption (T...
Read more