Decision Science

2. Business context, decisions, and strategy

Mihai Bizovi

VP of Decision Science @AdoreMe

2024-12-13

Three critical questions

Status Quo, Desired and Feared Trajectories in systems’ dynamics. Source: Kim Warren

• What happened (data / facts) and Why (inference)?
• Where are we likely going if we do things as before?
  • Is it a feared scenario?
  • What contributes to it?
• How to achieve the desired trajectory? Is it realistic?

The topic of how to measure performance is complicated and deserves a deep-dive.

Trajectories and SWOT diagnosis

What would an apparel e-commerce want?

(\(\max\)) Top line (revenue) | bottom line (EBITDA) | customer satisfaction | LTV ¹

• Status Quo: what is most likely trajectory? What contributes to it?
  • looks good \(\implies\) strengths
  • looks bad or unsatisfactory \(\implies\) weaknesses
• Feared trajectory (shocks, risks, macro environment, competition):
  • scenario looks bad \(\implies\) threats
• Desired trajectory. Is it reasonable and realistically achievable?
  • if yes \(\implies\) opportunities

Discussion: There are better tools than SWOT, coming from management consulting. ²

We can’t avoid tradeoffs

• What if it’s a startup that received big funding?
• What if it wants to capture market share?
• What if the goal is to have sustainable profitability?
• What if they position themselves as luxury?

The question we asked is too generic. We need a strategy and possible decisions, constraints in their value chain

What is the optimal tradeoff between objectives? In which part of value chain should they take action? How to make the most out of limited resources it has?

Business Analysts’ Workflow

Source: Adam Fleischhacker; This process is highly iterative and depends on having good feedback and collaboration

Characteristics of this process

• Outcome-focused: What’s the point otherwise?
• Strategically-aligned: Not all outcomes are equal!
• Action-oriented: Biggest pitfall of any AI/ML initiative – when it’s not actionable!
  • Needs clear and persuasive communication
• Computationally rigorous:
  • Correctness, reproducibility and maintainability
  • Accesible: idealy in an app which users explore

Data scientists often forget about outcomes and actionability. Students get stuck in EDA

What is a strategy anyways?

NOT just aspiration towards goal or a vision or a target.

Step	Outcome	Characteristics
Honest diagnosis	Identify obstacles	Few critical, relevant aspects
Guiding policy	General approach to overcome obstacles	Focus on most promising
Coherent actions	Support policy with action plan	Coordinated and focused

When it comes to setting good goals, you might’ve heard of SMART criteria

Other methodologies to be aware of

• Statistics and experiment design (12 steps) ³
  • The scientific process is much larger than this
• Causal inference and probabilistic graphs ⁴
• CRISP-DM, Tuckey’s Exploratory Data Analysis ⁵
• Machine Learning (12 steps) ⁶
• Software Development: Agile, DDD, TDD, XP, Design Thinking
• AI Products: People+AI, AI Governance, Event Storming

Methodologies are NOT recipes, NOT step-by-step instructions, NOR bureaucracy. Ways of thinking which help you avoid pitfalls, be systematic and effective in problem-solving.

Value Chain meets Decision Science

Source: bayesianquest – Data Science Strategy Safari.

Roles in firms: stuff data people do

• Data Engineering – pipelines and infrastructure
• Data Analysts – detectives, decision support
• BI – infrastructure for reporting, clean, modeled data
• ML Engineer – builds ML models and deploys them
• Data Scientist – jack of all trades, often lots of stats
• Product Analyst – cares about experiments
• Decision Makers & Domain Experts are usually the clients

Everyone deals with data, even BE, FE, Decision-Makers, QA, DevOPS, domain experts

Conversation: fields & use-cases

• What are the fields in which data science methods are extensively used? e.g. finance, genomics, psychology, …
• What are some products that use AI, data science, data-driven systems? What are their use-cases? e.g. uber …

Think in terms of reverse engineering

When using those products, how do you think those systems were designed?

• What were the goals and user/client needs? What were the firm’s objective?
• What constraints did they hit? Why is it a difficult problem?
• What are some potential approaches they settled on? What is a naive solution?

Remark on course philosophy

• Why is something important (method, idea, model …)
• Develop conceptual understanding and intuition
  • Theoretical rigor only where necessary
• Use simulations as a safe playground
• Practical and realistic applications
  • problem formulation: focus on decision-making
  • start with simplest models
  • deal with messy data and introduce more realism

There is no shame in going back to the fundamentals to master them

The danger of thinking in buckets

Here is R. Sapolsky’s argument about studying different aspects of human behavior:

• Our brains think about stuff in buckets / boundaries
• These buckets influence our memory, language, behavior
• We stop seeing the big picture:
  • Bad at differentiating facts within buckets
  • Exagerrate differences between buckets
• Tempting to claim that a bucket is the only, true explanation
• Some of the most influential scientists fell into this trap

We’ll walk across many buckets

• Problem space: the CAS of a firm, but not only
• Cognitive science: intelligence, rationality, foolishness
• Probability Theory: Reason under uncertainty, DAGs, DGPs
• Statistics: formulating hypotheses, experiment design
• Machine Learning: next year we focus on predictions
• Computer Science: how to make the stuff usable
• Philosophy: ethics, epistemiology, phil. science
• Mathematics: elegant abstractions and tools

Footnotes

1. some industries have specific metrics like daily users, revenue per mile per sq.m

2. e.g. McKinsey MECE + hypotheses, root cause analysis, modeling processes to find bottlenecks, impact analysis, design thinking, event storming, etc

3. We will discuss research methods and experiment design in the following lectures

4. We sketched out the process in the previous lecture

5. You might be familiar with this

6. We will dedicate two lectures on this, since you will need it until year two.