“How many FPOs are there in India?” This seemingly simple question marked the beginning of an unexpected challenge when I began working on the Tata-Cornell Institute’s project on FPO-Led Small Farm Market Access Models in 2019.

Farmer producer organizations (FPOs) are collaborative entities that unite small and marginal farmers, enabling them to pool resources and enhance their bargaining power in the marketplace. Established by The Companies (Amendment) Act, 2002—an amendment to the Companies Act of 1956—these entities, often referred to as Producer Companies or FPOs, combine the desirable attributes of cooperatives with the efficiencies of corporate structures to support primary producers, particularly small and marginal farmers.

Despite their critical role, comprehensive data on FPOs was startlingly limited. The absence of basic demographic and financial data concerning the formation and performance of FPOs significantly hampers effective decision-making and policy development. This lack of data often perpetuates the status quo, hindering efforts to make markets work more effectively for farmers and increase their accessibility and benefits.

In this blog post, I narrate TCI’s journey to develop India’s first FPO data dashboard, a dynamic platform designed to enhance the visibility of FPOs and facilitate deeper analysis of the broader FPO ecosystem.

Navigating data challenges: from accessibility to availability

The discovery phase of my research revealed a sparse FPO data ecosystem, characterized by fragmented sources without a single credible, consolidated source of data. This fragmentation complicated a true assessment of the growth of aggregation models in India, further necessitating the need for a single comprehensive database.

The challenge extended beyond merely gathering data; making research usable also meant making it accessible and actionable. This task required navigating a variety of data sources, including state and central government records often found in unstructured formats like printed, scanned, and uploaded PDF documents, which are nearly unusable for data analysis.

The phase required diving into 13 different data sources, each presenting its own format challenges, half of which were not computer readable. A detailed process to systematically digitize every record was undertaken, thereby creating an exhaustive “universe” of FPOs with each variable coded into clear, reusable header formats. My efforts aimed at not only cleaning the data and assigning information into the correct variables but also managing the translation of records from local languages. For instance, a standardized crop table was developed to categorize 83 different crops into 13 distinct groups, simplifying a crucial data field for future analysis.

The foundation for this data transformation was established by a robust ETL (extract, transform, load) pipeline. This was essential for ensuring that the dashboard could manage complex queries efficiently and remain scalable for future enhancements.

Validation of the first MVP

The culmination of my intensive data organization and transformation efforts led to the creation of the project’s first minimum viable product (MVP)—a PowerBI dashboard. This initial MVP targeted two key user groups: internal users needing a robust tool for research data access, and a broader external audience that would benefit from transparent access to data on FPOs across India.

This strategic approach enabled the functionalities of the dashboard to be thoroughly tested and validated with minimal resources, ensuring readiness before scaling up with more complex features. With a solid data model as its foundation, the dashboard was developed to provide immediate, actionable insights into the FPO universe derived from the compiled data.

The first internal iteration of the dashboard facilitated an understanding of the distribution of FPOs and their crop patterns across the country.

This MVP was critical for testing the dashboard’s user interactions and gathering feedback from both internal and outside users, helping us define the dashboard’s primary functionalities. It also played a key role in revealing the relationships between FPO distributions and agricultural practices, enhancing our comprehension of the sector’s dynamics.

Development of the FPO Dashboard

After the successful development of the PowerBI dashboard, a pivotal quarterly review with our donor agency, the Walmart Foundation, showcased the potential for broader application. My presentation was designed to position the PowerBI dashboard not just as a robust internal tool but also as a potential digital public good—a dedicated FPO resource that is openly available, universally accessible, and significantly enhances information transparency.

By classifying the dashboard as a digital public good, we committed to a path that would democratize data access, enabling stakeholders from various segments—government bodies, policymakers, researchers, and the farming community—to leverage this tool for information transparency in the FPO universe and potentially improved market access outcomes.

Encouraged by support from both TCI leadership and the Walmart Foundation, the decision was made to scale the solution to reach a wider audience. This necessitated the development of a web-based platform that we named the Database for Indian FPOs (the dashboard has since been rebranded as the FPO Platform for India). This transition, though complex, involved several critical steps:

1. Identifying the technology partner: We embarked on a thorough selection process to find a technology vendor capable of translating our vision into a public-facing web dashboard. Selecting the right partner was critical. A partner was needed who not only possessed the technical expertise to build scalable web solutions but also understood the nuances of a social-impact product. Ultimately, the partner chosen demonstrated a profound understanding of how technology could be leveraged to enhance accessibility and empower stakeholders.
2. Designing public-facing prototypes: In collaboration with the technology partner and a design specialist, prototypes were developed that replicated the original intent of the product and added a layer of user-friendliness for the public. Additionally, internal user layers were built to be able to make complex data integrations seamless. These prototypes were developed to ensure that the user experience was intuitive, catering to individuals with varying levels of tech savviness. This phase was crucial for setting the foundation of a dashboard that could effectively serve a diverse audience.
3. Finding credible sources of financial information: Adding verified financial data was yet another essential step for deepening the dashboard’s credibility. This step involved intricate processes to validate the existence and operational status of FPOs through their Corporate Identification Numbers (CINs). A key ETL step came in handy here—as many data sources provided names in local languages, there were discrepancies in names due to the translation from local languages. This required the implementation of advanced fuzzy-matching techniques, where the company names were searched in parallel with their available addresses to identify the correct CIN number of over 99% of the records. The rest had to be matched manually. This process not only ensured accuracy but also enhanced the reliability of the data with a layer of detailed financials for available FPOs.
4. Expanding filters: In addition to basic filters like crops and promoting agencies, based on user testing, we introduced capabilities to filter FPOs by their financial profiles, including active vs inactive status, exact year of formation, and completeness of financial records. These filters were designed to assist stakeholders in identifying their target FPOs based on their specific needs and interests.

A foundation for future growth

As I reflect on my journey from a simple question—“How many FPOs are there in India?”—to the development of India’s first comprehensive FPO data dashboard, it is evident that this project extends beyond technical advancements. We’ve made significant strides towards enhancing the accessibility of the FPO ecosystem through data transparency. The creation of this dashboard laid a robust foundation for empowering stakeholders across the agricultural sector with the critical data needed to make informed decisions.

This journey—from a collection of unstructured PDFs to a sophisticated web app—was both challenging and exhilarating. The launch of the Database for Indian FPOs in September 2021 marked a significant milestone for our team at TCI. Since its debut, the dashboard has garnered over 200,000 impressions and received numerous inquiries from stakeholders eager to engage with and expand the dataset, proving its value as a tool for transparency and as an instrument to assess the foundational models of FPOs.

As of March 2024, there are 44,000 FPOs registered in the country, with countless cooperatives, underscoring the complexity of obtaining reliable data on their sustainability and business operations. This highlights the critical need for our dashboard as a Digital Public Good that supports transparent and accessible data. Such platforms are essential for fostering our understanding of sustainable aggregation models and enhancing the economic viability of smallholder farmers through an improved digital presence.

As we continue to refine and expand this tool, the next phase of our work will focus on further assessing user needs and expanding the dashboard’s capabilities, ensuring it remains a valuable resource in the digital transformation of India’s agricultural landscape.

Stay tuned for more details on the complete product development lifecycle and the evolution of the dashboard into a comprehensive FPO Platform for India in our next blog. We are committed to continuously developing and enhancing our platform to better serve the Indian FPO ecosystem and contribute to India’s agricultural development.

Tanuj Chawla is a research support specialist at TCI, responsible for building and maintaining the FPO Platform for India. 

Featured image: A member of the Baliraja Krishak Farmer Producer Company in Sangamner, Maharashtra explores other FPOs in Maharashtra as part of TCI’s FPO platform user study, conducted in January 2023. (Photo illustration by Tanuj Chawla/TCI)