Winner of the New Statesman SPERI Prize in Political Economy 2016

Saturday, 3 February 2018

Large models, small models and Brexit

Non-economists with no interest in modelling techniques can skip to paragraph starting 'How is this all related to Brexit'.

I promised to look at some of the other papers in the OxREP volume “Rebuilding macroeconomic theory” besides my own, but as usual other things - including Brexit - got in the way. In this post I want to talk about the paper by Haldane and Turrell, which is about Agent Based Models, or ABMs. Right at the end of this post, however, I will come back to Brexit.

As a result of the microfoundations hegemony, any paper talking about a different modelling strategy often feels it must start by describing some drawbacks of that hegemony, and this paper is no exception. I might talk about that some other time, but instead I want to recommend what I think is one of the most realistic discussions of what ABM can or cannot do I have read.

As you might guess from the name, ABMs model the economy as a collection of a large number of different agents, each of which behaves in a specified way. The authors generalise the idea of a choice between internal and external consistency that I talk about in my paper to also include a degree of heterogeneity.

As you can see, ABMs are all about allowing as much heterogeneity as you wish. This is not to say that other methods cannot do heterogeneity (they can), but ABMs major in this dimension, and in practice often keep the behaviour of agents relatively simple compared to a DSGE. (A slight quibble: I would argue that as DSGEs are internally consistent by definition, the orange square representing them should be a slimmer and perhaps taller rectangle.) ABMs (within the bounds of tractability) owe no allegiance to any school of thought: the paper has a nice table of the many different types of consumption function used in a range of ABM studies.

As the macroeconomy is indeed made up of many different types of agents who may be doing different things, and whose interaction may produce unexpected results, it seems like ABMs can only be a good thing. But this additional freedom brings a large cost. Because, and unlike some hard sciences, there is a large amount of uncertainty about how people actually behave, we cannot treat any model as a black box, the output from which has to be accepted without question. No civil servant or central bank economist can go to politicians or governors and simply say it is what the model said.

Exactly the same problem can arise with SEMs, simply because of their complexity or disaggregation. It could also arise from a complex DSGE. The first question any economist asks when seeing an output from any large and complex model is does the result make sense given the smaller theoretical models they carry around in their head. It is why I proposed for SEMs the process I called theoretical deconstruction, where model properties were either reduced to familiar results from simpler models, or show the limitations of those simpler models. Again, as the paper notes, a similar process needs to, and in some cases has, happened with results from ABMs.

How is this all related to Brexit? The results showing how different degrees of Brexit would do the economy damage to different extents that I talked about in my last post were produced by trade theory’s equivalent of ABMs, called computable general equilibrium (CGE) models. These allow for considerable heterogeneity (across sectors and countries) in modelling trade. As Chris Giles recounts in this excellent piece, the model is more complex than anything the Treasury had before Brexit, and was built specifically to help with Brexit.

As Chris writes
“It must have come as a bit of a shock to government economists that the moment some results of this new model were leaked this week, ministers rushed to deny the usefulness of the tools they commissioned. Such models are “always wrong”, declared Steve Baker, a junior Brexit minister, on Tuesday.”

As I note in a postscript to my last post, he went further on Thursday to suggest that civil servants had deliberately cooked the model to sabotage Brexit.

How do we know that this didn’t happen, apart from the implausibility that so many civil servants could concoct such a conspiracy. Precisely because in this case the results from a highly disaggregated model broadly agrees with most other studies, and also common sense: the more difficult you make trade, the less there will be and the more costly that will be for UK output. Chris ends with some words that should be sent to every journalist in the country.
“Ministers now have a choice. They can opt for an honest Brexit in which they argue in public that people should pay an economic price for their policies. Or they can opt for a dishonest Brexit, pretending they have a secret plan for economic nirvana and trashing their own internal economic evidence. Ministers’ initial reaction in disowning the analysis suggests deception is the government’s central Brexit strategy. People talk about a crisis in economics. After this episode, it is the crisis in politics that should really concern us.”


  1. I do wish that prominent BRexiters such as Rees-Mogg would be challenged to produce their trade and economic forecasts as a result of BRexit.

    I discount Minford, as he is the 'climate denier' version of economics. But if BRexiters could produce a decent well-accepted model that supports their fantasies, then surely Remainers would have a more difficult time?

    I won't be holding my breath however...

  2. I can't help but feel that that last paragraph should have been written two years ago. With or without this latest model, the underlying conditions were still much the same. The only difference was that the 'dishonest Brexit' took the form of false words and numbers written in white on the side of a red single-decker bus.

  3. In the aftermath of the debacle of economic predictions of immediate consequences of a Leave vote, economic models have become irrevocably toxic. They are seen not as works with any technical merit, but as part of a concerted effort by a group of people to overthrow a democratic decision they didn't like and replace it with rule by an unelected self-appointed group of technocrats. Your models may be right, they may be wrong, they may even be more right than wrong, but what they are not is considered impartial by anyone apart from economists.

    As a someone with a background in the physical sciences I find the hubris and arrogance of economists about their work to be completely misplaced. The thing about coming up with models that describe nature is that it is really hard. Nature has a habit of not agreeing with the wisdom of scientists. I find no such respect for facts or reality amongst economists or other social scientists who appear to simply torture the data until it agrees with their prejudices.

    Finally, consider large economies; they are evolutionary. And the thing about evolutionary systems is they are largely completely unpredictable. No-one could have forecast where we are now ten years ago, so it is hard to see why we should now believe forecasts of ten years in the future, particularly given that economists show a marked inability to forecast economic growth one year ahead in stable conditions and we are now entering a period of significant change.

  4. These days I think it most sense to regard economists as shamen-like figures, or witch doctors. Rulers drag them out inspect the entrails of sheep or some other such phenomena and then pronounce that the gods support the rulers and that terrible things will befall those who challenge the regime. You can't object because if you are not a witch-doctor, well, you just don't understand the special magic that only witch doctors have.

    1. Your "STALINISTS!" shtick was more entertaining, please go back to that one.

  5. Any politician worth their salt should see an output that has "no scenario under which Brexit is positive" and immediately be skeptical. There are plausible but unlikely political scenarios in the EU which could throw the block into turmoil (e.g. Le Pen in France, Five Star in Italy) - yet these sophisticated mechanical models, no matter how complex, apparently take no account of these political economy outcomes, instead focusing purely on economic man with his potential demand and output functions.

    I think most people recognise that these unlikely political scenarios, if they occurred, would have extremely large but unquantifiable (apparently) 2nd/3rd order effects and Brexit would on balance be positive in those scenarios. Brexiteers see those outcomes as more likely than the rest of us so their implicit expectations are for completely different model results.

    The reaction might be childish but it is not necessarily wrong.

  6. Dipper,

    You're simply wrong about macroeconomic forecasting during periods of economic stability. Even the simplest approaches are usually coreect to within a reasonable margin of error. It's shocks that are impossible to predict, by definition, and difficult to model.

    The truth is, there simply isn't enough date to develop empirically sound macro models.

  7. From false microfoundations to true macrofoundations
    Comment on Simon Wren-Lewis on ‘Large models, small models and Brexit’

    Anyone can climb on a soapbox and tell the world how they would save the country or humanity ― except an economist. An economist needs the true theory “In order to tell the politicians and practitioners something about causes and best means, the economist needs the true theory or else he has not much more to offer than educated common sense or his personal opinion.” (Stigum)

    This, then, is the methodological challenge “The highest ambition an economist can entertain who believes in the scientific character of economics would be fulfilled as soon as he succeeded in constructing a simple model displaying all the essential features of the economic process by means of a reasonably small number of equations connecting a reasonably small number of variables. (Schumpeter)

    Economists do not have the true theory/model. The failure of economics had been programmed by the founding fathers with the definition of the subject matter as social science and latter on with this very specific guideline “It is a touchstone of accepted economics that all explanations must run in terms of the actions and reactions of individuals. Our behavior in judging economic research, in peer review of papers and research, and in promotions, includes the criterion that in principle the behavior we explain and the policies we propose are explicable in terms of individuals, not of other social categories.” (Arrow)#1

    Arrow’s definition covers General Equilibrium Theory, Marshallian partial analysis, Behavioral Economics, DSGE, Agent Based Models, and verbalized/common sense/ad hoc/special purpose models.

    The common denominator of these approaches is that they take individual/social behavior as a starting point and then try to explain the behavior of the economy as a whole. The methodological defect of the microfoundations approach is that NO way leads from the explanation of Human Nature/motives/behavior/action to the explanation of how the economic system works. Economics does not conform to Aristotle’s general definition of science “When the premises are certain, true, and primary, and the conclusion formally follows from them, this is demonstration, and produces scientific knowledge of a thing.”

    Because the premises/microfoundations are NOT certain/true/primary a paradigm shift is necessary. Economics has to move from microfoundations to macrofoundations. For a graphical summary of the methodological basics see Wikimedia.#2

    Methodologically it holds, (i) all microfoundations approaches are axiomatically dead, (ii) all Keynesian approaches are defective with regard to the definition of macroeconomic profit/income,#3 (iii) if it isn’t macro-axiomatized it isn’t economics,#4 (iv) lacking the true theory, economists have not more to offer than educated/computer-aided common sense.

    Egmont Kakarot-Handtke

    #1 This translates into the neo-Walrasian axioms “HC1 economic agents have preferences over outcomes; HC2 agents individually optimize subject to constraints; HC3 agent choice is manifest in interrelated markets; HC4 agents have full relevant knowledge; HC5 observable outcomes are coordinated, and must be discussed with reference to equilibrium states.” (Weintraub)

    #2 Wikimedia

    #3 Is Nick Rowe stupid or corrupt or both?

    #4 The core of macroeconomic premises reads: (A0) The objectively given and most elementary systemic configuration of the economy consists of the household and the business sector which, in turn, consists initially of one giant fully integrated firm. (A1) Yw=WL wage income Yw is equal to wage rate W times working hours. L, (A2) O=RL output O is equal to productivity R times working hours L, (A3) C=PX consumption expenditure C is equal to price P times quantity bought/sold X.

  8. «how different degrees of Brexit would do the economy damage to different extents»

    The political difficulty with that is that all such damage is in opportunity costs, with respect to a hypothetical alternative.
    Even a halved growth rates for 10 years after exit, plausible as it is, can be hard to prove after the fact, never mind before.

    It is a bit like the impact of immigration on wages and employments of natives: the constant argument by "sell-side" Economists and many "Remain" economists is that it has not led to a fall in the average wage, and so is irrelevant, ignoring the arguments that below average wages have fallen, and that what matters as to impact on average wage is whether they would have been higher had immigration been lower.

    Now the question that then matters politically is not "would have GDP growth been higher without exit?", but "will GDP fall because of exit?", and no model predicts that exit will come out with a GDP lower than in june 2015-june 2016, and even if a model predicted that, no "sell-side" Economist (affiliated to the government or to the City) would ever "talk down the economy" to that extent.

  9. Dipper: with respect, you know, or you should know, that your criticism is unfounded and full of that arrogance you accuse economists of. Economic models may have high degrees of inaccuracy when dealing with such things as political decisions and so on, but at least the attempt, usually quite successfully, to explain how the economy works and guide policy. Failing to model at all means you have no idea whatsoever how, ceteris paribus, one factor affects another. Are you suggesting future economic policy be guided, Brexiteer-like, by Boris and Gove's "gut feel" or do you have another suggestion? As an analogy: I can estimate how long it will take me to get from A to B allowing for traffic, weather and so on. I can model it with various factors such as average speed in peak traffic in my car and assume average weather and so on. I will still almost certainly be wrong to varying degrees. I can also know when someone else says they can get there in a tenth of the time I estimate based on "optimism" that they are probably wrong... how do you plan your journeys? How would you set about analysing economic policy decisions?


  10. So "additional freedom [of ABM] brings a large cost. (...) there is a large amount of uncertainty about how people actually behave, we cannot treat any model as a black box."

    We don't know, if ABM gets different agents' behavior correctly, but we know representative agent DSGE doesn't even try. So I don't see a cost here.
    Especially that it is an opportunity to end with interdisciplinary closure and use some knowledge about how people behave from other sciences.


Unfortunately because of spam with embedded links (which then flag up warnings about the whole site on some browsers), I have to personally moderate all comments. As a result, your comment may not appear for some time.