Did you ever wonder how this seemingly simple cup of coffee ended in front of you? Would it taste any better if you’d know the efforts made by 25 million people to bring it to your table?
Well, this was my line of questions one morning. As a data scientist, I thought that I have the analytical tools that would help to explore this commodity market that provides a living to 25 million people and see how much could be understood. On the other hand, I felt that I needed a little extra to get this done. Here, the combination of an analytical breakdown and a systems analysis will help us build some background in three stages:
The first step is to understand some building blocks of the market, but how to break down something that large? Holistic thinking is about being able to build a big picture from iterative cycles around three notions that are:
We will broach on these three in this section and expand on space and time in sections II and III.
Coffee production worldwide is mostly based on two closely related species: Coffea arabica (Arabica) and Coffea canephora (Robusta). They both require tropical conditions but harbour differences that have a large impact on the final product. Arabica is more delicate to cultivate, as it is thriving at higher altitudes with a certain proximity to adjacent vegetation and grows better within a 3°C interval. On the other hand, Robusta is much more resistant. It can grow in larger environmental conditions and survive warmer temperatures (see figure above).
Both species have differences in terms of composition that affect the final taste. Arabica tends to have a more aromatic and acidic taste that is often considered as more elegant than Robusta. It also has larger quantities of chlorogenic acids (an anti-oxidant), but less caffeine. With the roasting, chlorogenic acids are lost and, at some point, caffeine amounts start to decrease too.
Roughly, if you want a caffeine shot, drink a dark strong espresso made from Robusta with plenty of caffeine. Otherwise, if you are interested in the composition, potential health benefits and aromatic taste, go for Arabica, freshly ground and prepared with a French press.
It all starts in the South…
The tropical producing countries are grouped into the so-called coffee belt. Differences in temperature, humidity, etc. influence when coffee can be harvested as shown in the map below based on data from ICO. This explains why some coffees are not available all year long.
Once planted, coffee trees start to produce flowers after 3 to 4 years. These flowers can then become coffee cherries that are harvested but, getting the beans requires to remove most of the fruit.
Two major processes can be used to remove the pulpous part of the fruit. In the dry process, the cherries are dried under the sun before the beans can be picked from cracked fruits. In the wet process, the pulpous part is washed away with costly equipment that isolated farmers are unlikely able to buy individually. In both processes, sorting and cleaning steps are performed to ensure that only mature and undamaged cherries are kept. The export starts by bagging the green beans in 60kg bags.
… and traditionally continues in the North
Large amounts of green coffee are then exported to the drinking countries, but some are also processed locally.
Beyond the location, the following step is to roast the beans so that they harbour this specific colour and shape that we are used to seeing. The duration and the temperature of the roasting have a large effect on the final taste (e.g. bitterness) and composition (e.g. chlorogenic acids) of the product.
Three major types of products are then assembled that address different markets:
This is roughly outlining the transformation of coffee beans into a beverage, but who are the 25 million people making this happen?
The supply chain is necessarily the ghost profile behind the production steps that we described before:
As shown below, most of the world coffee is produced by three countries: Brazil, Viet Nam and Columbia. The fourth is Indonesia, with approximatively 5% of the production. Consequently, when events affect Brazil capacity to produce coffee (such as droughts), the whole market is affected.
Historical consumer markets such as Europe, USA and Japan represent a large share of the market. For instance, Europe and the USA represent 78.4 % of imports in US dollars. Even if this is massive, the market has a limited growth:
This is in stark opposition to emerging markets, like in south-east Asia (see chart below), where countries like Indonesia and Viet Nam show important growth. Interestingly, some historically drinking countries (China and Russia) have increased their imports of green beans too.
In this first part, we saw some of the building blocks of the coffee market. We answered questions such as: what is it? How is it produced and, more importantly, to what end? Now, how do these blocks fit together in time and space?
Below, we can see coffee price variations along 45 years, with its large highs and lows. Observing the smoothed moving average curve shows impressive trend variations. How is coffee’s volatility explained? All commodities have this in common that it takes time to produce them. For instance, from the day you plant the seeds, you have to wait 3-4 years before it yields any fruits. This means that in front of any demand variation, the supply can not adjust rapidly. Similarly, if suddenly the price of 1kg of coffee is much lower, most people will not suddenly buy 4kg of coffee. Together, it explains coffee prices volatility due to inelastic demand and supply. But is that all?
Regulation of the market occurred between 1965 to 1989 to reduce this volatility. The strategy was to limit the exports to keep higher prices that would make a more balanced market, beneficial for all. Unfortunately, a shift in consumers tastes prevented the renewal of the agreement in 1989. The new pricing balance was not favouring the producing countries (especially Brazil, the main producer) and contradicted US requests which led to its breakdown.
In 1990, begins the free market period and, you can see on the chart two extremely low price periods that endanger the coffee producers as they have to sell below production costs. If we take the second one, called the coffee crisis, can we understand its leading causes? One of the key components was that Brazil but also Viet Nam were producing larger than expected amounts of coffee. Increasing the supply, prices naturally decrease. This is probably the “elephant in the room”, the whole situation is bound to be more complicated. For example, a United Nations Conference on Trade and Development paper argues that oil price might have long-term effects on coffee prices. Curious readers will find a lot more looking at the International Coffee Organisation reports that study, for example, the coffee market in general or the first consequences of Covid19 and subsequent lockdowns.
If you have multiple sources of income, the consequences might be manageable, but coffee farmers often have a single one. How does it go for them?
Hundreds of pages are written on the network of coffee and its connections. In many of the reports I read, there was a lot of text and not too many visualizations. Words are sequential - they necessarily provide a linear unfolding of the story. Building a holistic perspective requires a little extra. What if you present everything in a diagram? Diagrams can show relationships at least in two dimensions and easily illustrate non-linear situations. For example? Feedback loops can be described or shown. In the following illustration, I took the example of mapping some elements that affect a coffee farmer income.
Still, any good picture deserves some explanations. I will focus here on three elements: the climate, the standard product market and the differentiated one.
Climate variations are affecting coffee harvests. In Brazil, for example, droughts have been known to induce huge variations in the production of coffee. Droughts decrease the number of produced cherries which leads to a loss for the farmer (for example see BBC news). In a broader scale, if the average temperature keeps rising as predicted, it will favour Robusta trees as they are comfortable up to 26°C. Arabica needs a lower temperature to grow (18-21°C). The direct consequence is that Robusta will make a larger share of the market with the downside of lower prices. Climate has intricated ways to affect the production of coffee that are hard to predict.
Higher temperatures might be a problem of lesser importance to importers and roasters. They have indeed learned to process Robusta in ways that can decrease the bitter taste that is not successful with mass consumers. R&D allow them here to provide coffee with a decent taste to the market but buying cheaper raw materials. Besides, they benefit from an oligopolistic situation. Producers have less leverage as buyers can shop somewhere else. The consequence is that, in this quick analysis, producers and transformers goals are not aligned creating discrepancies in wealth distribution.
Finally, the differentiated market. In recent years, an increasing proportion of coffee drinkers have had concerns about the quality of the coffee and the way it is produced. Consequently, this population is asking for better coffee that is produced in fair conditions for both the environment and the farmer. If it is “better”, it could be sold at higher prices, which would go into the producer’s pocket. Unfortunately, it is not that easy. First, the market is constantly looking for new flavours to elate educated palates. Producers are then expected to change quickly the seeds they use. Those will need 3/4 years before they yield any fruits and, by then, maybe the taste has changed? Some indirect outcomes are also that some of these require costly equipment and certificates, further decreasing the producer’s share. Finally, if by any chance, many would start to produce a speculated variant, it would drive the selling price down as more would be available in the supply/demand balance.
These examples show that the power, knowledge and wealth equilibrium are rather unbalanced between the farmer and the different actors along the supply chain.
Getting to understand the coffee market is not straightforward. Multiple elements are interconnected with different scales of impact. Here is only presented a first overview. Once this first level of interconnections is understood, it almost begs for more questions. These questions are priming cycles of deepened and enlarged understanding of the system. For example, the next natural question that comes to my mind is: what are the constraints of the other actors in the supply chain? Here, it is almost easy to empathize with farmers dire conditions, but it is likely to be more nuanced than that. What would be your next line of questions?
Aside from the curiosity to understand a product that many of us drink, we learned some elements of systems thinking. Building a big picture can feel unnatural, difficult to start but, function, structure and processes iteratively placed in an expanding context help to do that. I would not argue that it should be the work of a single mind. Building hybrid teams that present expertise in both data and business fields is important to solve real problems.
From a data scientist perspective, I think building this background is important. The systems overview brings clarity and a notion of the problems complexity. It is opening doors into the design of features to build models but also, makes you realize how modest you have to be. Trying to build models that “know” everything seems futile to me. How to anticipate things such as the consequences of Covid19 lockdowns based on past data to predict market prices? I would rather build a simpler model with established limits that helps to decide than trying to get the perfect model which would trick someone into blindly believing its results. What about you?
The following ressources helped me to write this post: