An Interpretable Produce Price Forecasting System for Small and Marginal Farmers in India using Collaborative Filtering and Adaptive Nearest Neighbors
Authors:
Wei Ma,
Kendall Nowocin,
Niraj Marathe,
George H. Chen
Abstract:
Small and marginal farmers, who account for over 80% of India's agricultural population, often sell their harvest at low, unfavorable prices before spoilage. These farmers often lack access to either cold storage or market forecasts. In particular, by having access to cold storage, farmers can store their produce for longer and thus have more flexibility as to when they should sell their harvest b…
▽ More
Small and marginal farmers, who account for over 80% of India's agricultural population, often sell their harvest at low, unfavorable prices before spoilage. These farmers often lack access to either cold storage or market forecasts. In particular, by having access to cold storage, farmers can store their produce for longer and thus have more flexibility as to when they should sell their harvest by. Meanwhile, by having access to market forecasts, farmers can more easily identify which markets to sell at and when. While affordable cold storage solutions have become more widely available, there has been less work on produce price forecasting. A key challenge is that in many regions of India, predominantly in rural and remote areas, we have either very limited or no produce pricing data available from public online sources.
In this paper, we present a produce price forecasting system that pulls data from the Indian Ministry of Agriculture and Farmers Welfare's website Agmarknet, trains a model of prices using over a thousand markets, and displays interpretable price forecasts in a web application viewable from a mobile phone. Due to the pricing data being extremely sparse, our method first imputes missing entries using collaborative filtering to obtain a dense dataset. Using this imputed dense dataset, we then train a decision-tree-based classifier to predict produce prices at different markets. In terms of interpretability, we display the most relevant historical pricing data that drive each forecasted price, where we take advantage of the fact that a wide family of decision-tree-based ensemble learning methods are adaptive nearest neighbor methods. We show how to construct heuristic price uncertainty intervals based on nearest neighbors. We validate forecast accuracy on data from Agmarknet and a small field survey of a few markets in Odisha.
△ Less
Submitted 12 December, 2018;
originally announced December 2018.
Toward Reducing Crop Spoilage and Increasing Small Farmer Profits in India: a Simultaneous Hardware and Software Solution
Authors:
George H. Chen,
Kendall Nowocin,
Niraj Marathe
Abstract:
India's agricultural system has been facing a severe problem of crop wastage. A key contributing factor to this problem is that many small farmers lack access to reliable cold storage that extends crop shelf-life. To avoid having leftover crops that spoil, these farmers often sell their crops at unfavorable low prices. Inevitably, not all crops are sold before spoilage. Even if the farmers have ac…
▽ More
India's agricultural system has been facing a severe problem of crop wastage. A key contributing factor to this problem is that many small farmers lack access to reliable cold storage that extends crop shelf-life. To avoid having leftover crops that spoil, these farmers often sell their crops at unfavorable low prices. Inevitably, not all crops are sold before spoilage. Even if the farmers have access to cold storage, the farmers may not know how long to hold different crops in cold storage for, which hinges on strategizing over when and where to sell their harvest. In this note, we present progress toward a simultaneous hardware and software solution that aims to help farmers reduce crop spoilage and increase their profits. The hardware is a cost-effective solar-powered refrigerator and control unit. The software refers to a produce price forecasting system, for which we have tested a number of machine learning methods. Note that unlike standard price forecasting tasks such as for stock market data, the produce price data from predominantly rural Indian markets have a large amount of missing values. In developing our two-pronged solution, we are actively working with farmers at two pilot sites in Karnataka and Odisha.
△ Less
Submitted 7 December, 2017; v1 submitted 28 October, 2017;
originally announced October 2017.
