Lecture 6: Dynamics and Programming

Introduction

Robert Townsend introduces the lecture and provides an overview of the reading list and study guide.

Lecture Overview

• The lecture will focus on dynamics and programming, specifically in the context of a storage problem.

• Chapter 3 of "Medville" is starred on the reading list and fills out the material for the lecture.

• The study guide includes questions related to Decision Making under Uncertainty and Linear Programming.

Dynamics and Programming

Robert Townsend explains that today's lecture will feature dynamics and what happens over various dates. He also introduces Chapter 3 of "Medville."

Storage Problem

• Today's lecture will focus on a storage problem where people were close to starving to death, as in medieval villages.

• This problem will be discussed in the context of dynamics and programming.

Reading List

• Chapter 3 of "Medville" is starred on the reading list because it fills out even more than what can be covered in one class.

Study Guide Questions

Robert Townsend discusses a question from the study guide related to consumption sets.

Consumption Sets

• A consumption set with a discrete number of points is not convex because you cannot consume lunch in a mixture of two cities.

• Drawing a line between those two points converts that set into something that is convex using lotteries which put mass on these discrete points.

• Lotteries represent weights, with non-degenerate lotteries putting probability between zero and one on each endpoint.

Risk Aversion

• If you are risk averse, then you have diminishing marginal utility of income.

• The utility function should be concave, typically weakly concave.

• Risk aversion means strictly adverse to lottery.

New Section

In this section, the speaker asks for a description of the complete commodity space for a consumer by combining states and time of the world. The speaker is looking for volunteers to describe the tree of possibilities that was presented in the previous lecture.

Complete Commodity Space for a Consumer

• The complete commodity space for a consumer involves combining states and time of the world.

• The tree of possibilities is oriented upwards with time increasing as you go up in this tree.

• The state of the world at any date includes not just possible realizations of contemporary shock but also includes history shocks up to and including the contemporary date.

• When delineating all possible histories and all possible states that can happen in any given date, it's important to consider whether you're already at the current date or whether you're at date zero imagining what could happen in the future.

New Section

This section focuses on dynamics and dynamic programming. It discusses an application for medieval villages on storage or inventory, carryover, as well as seed.

Dynamics and Dynamic Programming

• Calibration is finding parameter values appropriate for a model so that when simulated, it fits reasonably well with data.

• Most lectures up to this point were static except for interest rate and income substitution effects and consumption set diagrams.

• Time plays a big role here; therefore, if we are going to solve a problem or program, it's going to be dynamic programming.

• This lecture talks about actual and potential use of risk reduction arrangements such as carrying over grain from one year to another.

Alternative Ways of Reducing Exposure to Risk

• The picture of the estates of the Bishop of Winchester was spread out over space, arguably to diversify.

• The stylized facts show that there is not a lot of carryover in the same Bishop of Winchester accounts. There are rare spikes where carryover jumps up, and then it's down.

Neoclassical Growth Stabilization Model

In this section, the speaker discusses a neoclassical growth stabilization model that includes two types of storage technologies: grain in the bin and seed in the ground. The speaker explains how they will choose parameters consistent with available data and look at numerical predictions to match the fact that carryover was not common.

Two Types of Storage Technologies

• The two types of storage technologies are grain in the bin for storage and seed in the ground.

• Only three out of 18 states had carryover in a good year, which is why variability in consumption is one to one with variability in output.

• Grain can be stored after harvest or put into the ground as seed planted for next year's crop.

• Grain can be stored by putting it into a bin, but it may spoil due to mildew or being eaten by rodents.

Depreciation Rate

• Grain has a depreciation rate called delta, which is why it goes bad over time.

• McCloskey gets a depreciation rate of 30% from grain price inflation rates through the year. Whatever you store at one date, you get back 1 minus delta that number, say, the following month.

Profit Maximization

In this section, the speaker discusses profit maximization and constant returns to scale.

Profit Maximization

• The formula for profit maximization is maximize P1 times y minus i, where i is investment, plus P2 times Y2 plus i multiplied by 1 minus delta.

• Constant returns to scale means that the solution is either infinite or indeterminate or zero. However, in this case, it's not indeterminate nor zero because they were storing from month to month.

Economic Intuition

• To compensate for the loss of the crop rather than selling it today at a certain price, they can sell it tomorrow when they're going to have 70% left, a 30% loss. So the price has to accumulate at 30% over the year.

Seed Planting and Harvesting

In this section, the speaker discusses seed planting and harvesting.

Yield-to-Seed Ratio

• The yield-to-seed ratio is about 2.6 for one of these manners and 1.67 for the other one. This means you get roughly two units of output for every unit of input.

Technology

• If you planted Kt units-- put Kt seeds in the ground at t minus 1--the output would be a function f of that amount of seed, the harvest with some shock epsilon. Alpha times whatever seed got put in the ground is equal to output where alpha could be two and epsilon makes it random and move around if we assume linear technology.

Decision Variables

• You can't plant an unbounded amount of seed since there's a limit to how much land you can plant. The seed per unit land is constant, so it's really a decision about how much land to plant. K bar is going to be the most land that you can plant. You plant all of your land and if you were to try to put down more seed, it's like throwing it away. The decision variables at t are labeled t plus 1.

Key Equation

In this section, the speaker discusses the key equation.

Output Function

• Output f, harvest is a function of the seed planted last year plus an influence of epsilon with something in storage as well. If they had done both things, then they put some seed in the ground and had some storage. You're going to get 1 minus delta times that amount of storage back and that would be the end of the story for consumption except they have to plan for next year.

Consumption

• This is the amount that's available; this is the amount that you squirrel away for next year. And the difference between what's available and what you store in one way or another is consumption.

Finite Time Horizon

The speaker introduces the concept of a finite time horizon and discounted utility. Beta is a discount rate, beta some number less than 1. So you care about the present more than you care about the future, present utility more than future utility.

• With beta less than 1, geometric.

• As the horizon gets further and further away, you care less and less relative to today's consumption.

• Concave utility when we introduced the concept of risk.

• They would try to avoid variability in consumption.

Calibrating Parameters

The speaker discusses calibrating parameters for constant relative risk aversion and setting an upper bound on storage.

• Depreciation rate calibrated at 30%.

• Discount rate set at 0.95.

• Constant relative risk aversion set at 0.5.

• Upper bound on storage set at 1.

• Uncertainty is put in with epsilon.

• Yield-to-seed ratio calibrated to deliver disaster every 12 years on average.

Dynamic Decision Problem

The speaker presents a dynamic decision problem where total available this year is plotted against savings decisions of inventory or seed.

• Total amount available going from 0 to 6 is varied.

• Solution to a max problem at calibrated parameters.

• Model version of what they were doing in fact.

Solving the Problem

The speaker discusses solving the problem using Matlab or something similar.

• Picture could be anything but it's pinned down because it's the solution to a max problem at calibrated parameters.

• If total amount available was a little less than 1, then they were not planting all the land.

Savings Decisions

In this section, the speaker discusses how savings decisions are made in agricultural societies.

Savings and Storage

• Savings decisions are made based on the amount of harvest available each year.

• Seed is usually put into the ground, but not when the amount available is less than 3 or when storage is zero.

• Consumption is the difference between the amount available and savings decisions.

• The 45-degree line plays a role in determining consumption.

Starvation and Seed Corn

• When there's less than 1.1 units available, people still eat but don't plant all their land.

• Eating seed corn guarantees zero for tomorrow; it's better to gamble on getting a reasonable return.

• They cut back on seed consumption but still eat some of it.

Dynamics of Total Available Units

In this section, the speaker talks about how total available units change over time based on savings decisions and stochastic output.

Amount Available Next Year

• Total available units can be saved or eaten this year.

• The amount available next year depends on endogenously determined savings decisions and stochastic output (epsilon).

• If three units are available now, then two will likely be available next year with no storage possible.

Steady State

• Two in and two out is the most likely outcome if you start with two units.

Carryover and Storage

In this section, Robert Townsend discusses the concept of carryover and storage in agricultural production.

Carryover

• Carryover refers to the amount of crop that is stored from one year to the next.

• Carryover is rare because it depends on getting a high yield for two consecutive years.

• If a farmer has carryover one year, they may not have it the next year if they get a low yield.

Storage

• If a farmer has two units available and gets a high yield, they will be storing their crop.

• With four units available, there is definitely storage going on.

• If depreciation rate goes down, inventory line will move up.

Parameters

• Beta matters. If beta goes to zero, you don't care about the future.

• All parameters matter in agricultural production including risk aversion and depreciation rate.

Conclusion

In this section, Robert Townsend concludes his discussion on agricultural production by acknowledging that some parameters were picked but succeeded in producing data.

Cheating

• The model had a finite horizon but was ignored during the discussion.

Working Back from the End

In this section, the speaker discusses how to solve a two-period problem by working back from the end.

Two-Period Problem

• The speaker explains that solving a two-period problem involves starting at the last date and moving to the next-to-last date to solve the maximization problem.

• As you move farther away from t equal 1, you are closer to solving an infinite horizon problem.

• The speaker introduces some notation for a sequence of two-period problems.

Bellman Problem

• The Bellman problem involves choosing control variables such as consumption and storage.

• To solve an infinite horizon problem, start with the last day and do the two-period optimization. Keep iterating until there is no change in solution.

• There is something called a contraction mapping that helps things contract.

Contraction Mapping

• A contraction mapping refers to T, which is not time but rather a mapping from utility from next period on to utility today.

• Things will contract as we keep successively applying that mapping to an initial guess value of v. We will converge to something called v* when T of v* is equal to v*.

Applying the Contraction Mapping

In this section, the speaker explains how to apply the contraction mapping technique to solve optimization problems.

Steps for Applying Contraction Mapping

• Start with a guess for v0, which is the solution to the last period problem.

• Optimize the value as if you were at the last date by substituting in the maximized utility number.

• Iterate again with that new guess for the second period and keep iterating until you get to the infinite horizon solution.

Maximizing Utility in a Contemporary Setting

In this section, the speaker discusses how to maximize utility of consumption in a household facing stochastic earnings and limited savings.

Maximizing Discounted Expected Utility

• Maximize discounted expected utility over an infinite horizon.

• The choice will be over how much to eat and how much to save.

• Resources available today at T can either be eaten or used to buy a bond and carry it over to next period.

Storing Money Over Time

In this section, the speaker compares storing money over time through riskless bonds or treasuries with storing grain in medieval times.

Storing Money Over Time

• A household can store money over time through a savings account in a bank at some interest rate or through riskless bonds like treasuries.

• This is similar to storing grain in medieval times.

• The timing of decisions is different from state variables.

Infinite Horizon Optimization Problem

In this section, the speaker explains how an infinite horizon optimization problem can be solved using discounting techniques.

Solving Infinite Horizon Optimization Problem

• Maximize discounted expected utility over an infinite horizon where beta is less than one.

• The choice will be over how much to eat and how much to save.

• Resources available today at T can either be eaten or used to buy a bond and carry it over to next period.

Borrowing Constraints

In this section, the speaker explains how borrowing constraints work in an optimization problem.

Borrowing Constraints

• Borrowing is allowed but limited.

• If savings are negative, then borrowing occurs.

• This formulation of the problem allows for borrowing but with an upper bound.

Self-Insurance and Risk-Sharing

In this section, Robert Townsend discusses self-insurance and risk-sharing. He explains that self-insurance is what they have been studying in the lecture, which is essentially one person or village being in isolation. However, there is still a bond or borrowing possibility at a fixed interest rate. This partial equilibrium problem allows agents to borrow or lend with the rest of the economy.

Self-Insurance vs Risk-Sharing

• Self-insurance is what they have been studying in the lecture.

• Partial equilibrium problem allows agents to borrow or lend with the rest of the economy.

• If there were multiple people like this, some people had high income at a given date while others had low income at a given date, then we're back to thinking about whether they could make deals with each other.

• Explicit risk-sharing insurance.

Medieval Villages and Thai Temple Scheme

In this section, Robert Townsend compares and contrasts two different ways to handle risk - medieval villages with scattered strips and Thai temple scheme.

Comparison between Medieval Villages and Thai Temple Scheme

• The very first two example economies were medieval villages with scattered strips that we've been studying now implicitly or the Thai temple scheme.

• The Thai temple scheme was an active risk-sharing scheme where people contributed their crops to the monks under the auspices of the temple with monks handing it out to people that had either drought or flood.

• Comparing and contrasting these two different ways to handle risk.

Pareto Optimality and Efficiency

In this section, Robert Townsend talks about the upcoming lecture on Pareto optimality and efficiency.

Pareto Optimality and Efficiency

• Next Tuesday is a lecture on Pareto optimality, which is finally the full-blown general equilibrium.

• The lecture will fully embrace it and talk about efficiency.