What is AI, Data Science, Cybernetics?
It’s undeniable that there is a lot of excitement when it comes to AI, ML, and data science – to the point of calling it the sexiest job of 21st century. Data science is an umbrella term, with interdisciplinary at its core, drawing inspiration from multiple fields, sets of tools, practices, methods and it continuously evolves. 1 It is designed to help us tackle increasingly more complicated problems at a large scale. There is also a reasonable worry about ways in which these systems can go wrong or awry.
1 There are many questions still unanswered: How does this landscape of Data Science look like? What are the roles and jobs? What is the process for building smarter, data-driven software systems; drawing more reliable inferences and conclusions from data and theory? How does a day in data scientist’s life look like?
You will often see a Venn diagram where data science sits at the intersection of mathematics – statistics, computer science – software engineering, and domain knowledge. I think this is not sufficient to characterise data science, therefore, will try to elaborate what it does, and how (which is as important).
For all pragmatic intents and purposes, especially in businesses, AI is about Decision-Making under Uncertainty at Scale 2.
One important keyword here is uncertainty, as there is no point in building AI solutions based on complex models if we don’t have uncertainty. We have to be able to change our mind and actions in the face of evidence.
On the other hand, scale is the reason ML and Deep Learning is so powerful, because you can take lots of small decisions in an automated way, with little curation or guidance from humans. This is why many traditionally “paperwork” industries like legal and accounting embrace digitalisation and automation now.
2 C. Kozyrkov - AI is decision-making at scale
Ultimately, why would I build a system which predicts demand for products in a direct-to-consumer ecommerce like Allbirds, Macy’s, or AdoreMe? Either in a marketplace like Emag or Amazon? Why would I try to find out the factors which contribute to a successful advertisement?
- In order to allocate resources to the stuff which generates growth and profit. Avoid being scattered around (which I would call bad strategy), resulting in costs over targets and inefficiency.
- In short, we attempt to allocate resources and efforts efficiently.
- We can view information as a competitive advantage, anticipating and predicting so that we can plan and prepare, outperform competitors.
When we talk about uncertainty, it’s important to recognize its sources: 3 one coming from incomplete information, that we always work with samples in one way or another. For example, even if at a certain point in time we might have real-time data, everything evolves and quickly becomes outdated. Everything is in a state of flux and change. Even in the current state, we don’t know for sure where we stand – sometimes, in economics, this problem is called nowcasting. When talking about the future, making a good prediction is one of the most difficult things.
3 We will talk more formally about sources of uncertainty in the next lecture, while reviewing the fundamentals of statistics.
For example, who would’ve predicted the pandemic and all its implications on the supply chain and society? It’s important to note the difference between this kind of black swan events and the irreducible, fundamental uncertainty, which can’t be captured by any explanatory factors.
In a happy case, we can quantify and reduce it by conditioning the model, that is, joint distribution of random variables with a given structure, on data. That would result in inferences and evidence with respect to our hypothesis and model of the world.4 At the very least we can try to quantify how uncertain are we.
4 We want to say something intelligent about the population, technically, to generalize. However, there is ambiguity, as objectives and the meaning/semantics of data fields are not always mathematically clear or without conflict.
So, we still have to make decisions. Those have to have a level of robustness and resilience to shocks, in the face of uncertainty. I would go even further, to suggest that we should aim for antifragility, meaning, the system improves after a shock or negative event, but that is very hard to implement and operationalize, therefore, it falls outside the scope of this course – to the realm of systems’ design.
As a though experiment, imagine we have an equation or program, with well-defined rules, which perfectly predicts the price on stock markets, or perfectly predicts how many items will a client buy and how she will respond if we change the price (an intervention). We won’t need machine learning, causal inference, or AI there.
Of course, we don’t have that kind of program. It’s only somewhat true in cases when we have a well-tested theory, which stood the test of time and went through the scientific process to become the best theory with respect to all others. For example newtonian physics, relativity, quantum mechanics, evolution.
However, when we talk about human behavior, we should resist the temptation and arrogance to say that we have a well-defined theory, be it normative or positive. Our preferences change, and we can “decide” in which direction they change or persist.
Regardless of the business we work at or own, the place in the value chain, we’ll have to deal with human behavior: customers, employees, decision-makers, engineers. We need other kind of tools to infer perceptible regularities and patterns in their behavior. We will be forced in one way or another to learn from data and observation.
We need models to make sense of the world around us, because it is so complex and uncomprehensible if we are to represent it faithfully in a simulation. Therefore, we focus on relevant, interestig, essential aspects to us, we simplify by baking in domain knowledge, assumptions, and data into the models and algorithms.
So, we can collect data, apply algorithms to train models, in order to make inferences about some relevant quantities. That will help us in making evidence-based decisions which gets us closer to our objective in an efficient way.
By now, you probably figured out that we’re not talking about General AI, trying to surpass human intelligence in general reasoning and problem-solving. Thus, we’re talking about weak or specialized AI, which depends very much on the domain.
AI in an a fashion e-commerce, like AdoreMe, where we sell lingerie, will have a very different flavor from the tools and methods used in genomics, medicine, social science or psychology.
Despite the fact that there are a lot of shared fundamentals, when it comes to the principles of building models, it is not straightforward to take something which works in one domain and apply it in another. Significant tweaks and adaptations are needed, which are dependent on the specificities of that domain.
The good news is that when these transdisciplinary groups of people work together and successfully adapt a method, it is often a breakthrough in the field borrowing the theory and technology.
Cybernetics is what we call AI
At this point we have a working definition of Weak AI. At a first glance it might be hard to see what does it have in common with Cybernetics and its study of Complex Adaptive Systems.
I’m not trying to equivocate those two, but argue that weak AI is how Cybernetics evolved and is mostly used in practice now. I will give a definition from P. Novikov, which I found tremendously useful, then explain it. Can you spot the parallels of “decision-making under uncertainty at scale” in this definition?
The science of general regularities of control and information processing in animal, machine and human (socitey)
- Control means goal-directedness, the ability to reach the goals and objectives by taking action and stirring the system towards a trajectory. The objective can also be perserving the structural-functional organization of the system itself, an autopoesis.
- Information Processing could be pattern recoginition, perception, how you understand and model the world, what inferences do you draw, what “data inputs” are used
- General regularities means what is true and plausible of control and information processing across fields and a variety of complex systems, not only in particular cases.
- Animal refers to applications in biology, machine – in engineering, and human – in our society and behavior.
In economic cybernetics, we’re concerned with economics, society and human behavior, rather than engineering, biology, or natural science applications.
To explain how Cybernetics evolved into Weak AI, there is a conglomeration of fields which went a bit out of fashion and favor: Game Theory, General Systems’ Theory, Agent-Based Modeling, Systems’ Dynamics, Complexity and Chaos, Evolutionary Algorithms. This stuff is fascinating and inspired many other breakthroughs, but it is extremely difficult to implement in practice.
So, we kind of settled on a more pragmatic set of tools, which is dominated by pattern recognition and optimisation, in one form or another trying to learn from data (ML, DL, Causality) and act optimally (Dynamic Programming, Reinforcement Learning). .
Wait. What’s going on here? Am I saying that we did a bachelor’s degree in AI under the term of Economic Cybernetics? For me, personally, after having this epiphany – everything I studied makes so much more sense in retrospective.
You can make sense of the terminology and general confusion of terms, by reading M. I. Jordan’s brilliant article 5, which tells the history of “AI” and how this confusion arose. He also points out how many of the claims in the field, as of today are a stretch (i.e. the revolution hasn’t happened yet) 6.
I highly encourage you to read the articles by M. Jordan, but until then, here are a few ways people understand AI:
- Cybernetics and Weak AI, which we discussed before
- General AI is a titanic project. It interweaves with Philosophy, Cognitive Science, in order to understand what makes us intelligent and conscious. On the other hand, trying to build general-purpose problem solving machines.
- Symbolic AI, is still relevant in a few niches, especially in automated proofs and logical reasoning.
- Augmentative AI, like VR, augmenting human capabilities, human-machine interactions
6 M. Jordan - Artificial Intelligence: The Revolution Hasn’t Happened Yet
5 K. Pretz - Stop Calling Everything AI, Machine-Learning Pioneer Says
In practice, if you’re a data/business analyst, ML/data engineer, data scientist, statistician, product manager – Cybernetics is a way of thinking in systems and formulate problems well. When it comes to implementation, we mostly use data and the tools, models, methods discussed in this course.