In 1977, Jimmy Carter made an improbable journey from Georgia peanut grower to Democratic president in part by playing on his humble roots and receiving support from America’s farmers. Yet this bedrock voting constituency abandoned a fellow farmer to back Ronald Reagan four years later, after Carter punished Moscow for invading Afghanistan by cutting off grain sales to the Soviet Union. U.S. farmers were already struggling with collapsing crop prices, and the embargo may have been the final straw. Farmers threw their support behind Reagan, who had promised to lift the hated restrictions.

This might seem like ancient history, but it’s still relevant to — and well-remembered in — today’s Farm Belt, where farmers are once again torn between devotion to a president and anger at the tariff policies he’s imposed that are set to wreak havoc on agricultural prices. Although farmers overwhelmingly supported Donald Trump in 2016, discontent over tariffs and trade policy could induce farmers to oppose his bid for re-election — and their strategic location might just enable them to thwart it.

First, Trump did really well in the states that have more farmers:

In particular, there are three states that Trump won by narrow margins in which a mass farmer defection could prove pivotal: Michigan, Wisconsin and Pennsylvania. In each of these states, the number of farmers far exceeds the president’s margin of victory in 2016. If all three states saw significant ag defection, a Democratic challenger could pick up a total of 46 Electoral College votes — enough to tip the balance even if Trump performs up to his 2016 standards in every other state in the union.

Of course, 2020 is unlikely to be an exact replay of 2016. But these swing states in the Midwest are a good bet to be competitive again, making any erosion of Trump’s support among farmers a big worry for the president.

So what’s getting in between Trump and farmers?

Trump raised farmers’ alarm bells when he bandied about the idea of pulling out of NAFTA and cutting off free trade with Canada, one of the biggest importers of U.S. agricultural products. As farmers’ groups bought pricey airtime to run ads that attempted to dissuade policymakers from exiting the North American trade deal, a more immediate threat to the farm economy appeared on the horizon this month: In reaction to protectionist manufacturing tariffs leveled by the U.S., China — another big destination for U.S. agricultural products — retaliated, placing 15 percent tariffs on numerous agricultural exports and a 25 percent markup on pork products.

When Trump responded by announcing plans for $100 billion in additional tariffs, China threatened more of its own, including one on the U.S.’s top agricultural export, soybeans.

In a recent Senate Finance Committee meeting on Trump’s tariffs, Iowa Sen. Chuck Grassley said that farmers in his state “still remember President Carter’s grain embargo.” Grassley reached the Senate in those same 1980 elections. “That’s 38 years ago, but that’s still in the memory of farmers,” he said.

Indeed, the Iowa senator knows his audience. The average age of U.S. farmers has been increasing over the past several decades, rising from 51 in 1982 to 58 in 2012 — the most recent year for which data is available. The average age is now likely to be close to 60 years old, which means that the bulk of U.S. farmers were old enough to vote when they abandoned Carter as a group in 1980.

Data on how farmers voted at that time is elusive, but anecdotal evidence suggests that the share of farmers who backed Reagan was high. A member of Reagan’s agricultural transition team put the number at “at least two-thirds,” and The Washington Post described the farm sector as “one of the most solid blocs for Reagan.” The New York Times concluded that “many farmers who had voted for Jimmy Carter in 1976 supported Mr. Reagan” in 1980 “because they felt Mr. Carter had broken a campaign pledge never to establish an embargo on agricultural products except in a case of extreme national emergency.”

Flash forward to 2016, when 67 percent of farmers voted for Trump. Farmers were no doubt drawn to his deregulatory message and motivated by their perception that any Obama-era economic recovery had left them behind. In all regions, farmers are overwhelmingly Republican voters, regardless of the size of the operation.

But after months of controversy over farm labor visas, falling commodity prices and withdrawal from free-trade agreements, a recent survey conducted by Agri-Pulse, one of America’s leading agricultural news sources, adds weight to Grassley’s warning. The survey — conducted from Feb. 26 through March 9, before the most recent tariff battles — indicates that farmers are losing patience with the president. Although 67 percent said they had voted for Trump, only 45 percent said they would support his re-election.

There are fewer farmers today than there were in 1980. The number has shrunk by about 15 percent — from the approximately 2.4 million farms dotting the U.S. countryside to about 2 million today. Farmers now represent around 1 percent of the U.S. population. But the geographic distribution of these farmers, as well as their voting habits, could turn them into a powerful anti-Trump force ready to pop up in 2020 and alter the balance of the elections.

These Midwestern states are typically extremely close in presidential elections. They certainly were in 2016. And Trump would be more vulnerable there in 2020 if farmers abandoned him. Farmers are good voters in general, and in 2016, an estimated 74 percent of them turned out to vote in competitive states in the presidential elections. In Michigan, Trump won by 10,704 votes, and there are approximately 80,000 farmers in the state. And somewhere around 67 percent of them (let’s call it 40,000) voted for Trump if they were consistent with national projections. All it would take is for slightly a little more than one in four of those farmers to jump ship to turn the tables against Trump. The numbers are similar in Wisconsin. (Trump won Pennsylvania by a bit more.)

We can use the Agri-Pulse poll to look at this in a slightly different way. If only 45 percent of Michigan farmers vote for Trump, he would get 27,000 farm votes — a loss of 13,100. Without any other change, Trump just lost Michigan and has been reduced to a razor-thin margin of victory Wisconsin. If 45 percent of Wisconsin farmers voted for Trump, he’d get about 37,000 farm votes, a loss that would reduce his lead to around 5,000.

Elections, of course, don’t work this way — one group moving while all others vote the same way — but the math shows that there are enough farmers in these crucial states to make a difference should they fully sour on Trump.

One thing we can be sure of is that these Chinese tariffs will have a negative impact on the agricultural sector in all three states. The first round of tariffs included two of Michigan’s top agricultural products — hogs and apples — and two of Wisconsin’s — hogs and cranberries. Also, soy is the third-largest agricultural product for both states. They both also count corn among their top 10 products; while the first round of tariffs imposed a 15 percent duty on ethanol, a key application for the U.S. corn crop, the second round of threatened tariffs included corn as well. While corn is a major export product for U.S. farmers, the Chinese market represents a very small market share for edible corn.

Farmers may not be the massive voting bloc they once were, but as Jimmy Carter learned nearly four decades ago, presidents ignore farmers’ needs — and erode farmers’ markets — at great peril.