MoSPI Proposes to Set Up National Data Warehouse for Big Data Analytical Tools to Improve Official Statistics

The Ministry of Statistics and Programme Implementation (MoSPI) is giving an increased focus on data quality and assurance, as the statistics ministry proposes to set up a "National Data Warehouse". Announced earlier this week, the proposed data warehouse is aimed at leveraging big data analytical tools to further improve the quality of macro-economic aggregates.

As a ministry of Government of India, the MoSPI says, efforts are also on to evolve a legislative framework under which the National Statistical Commission (NSC) may function with independence and give holistic guidance for improving the national statistical system. In its announcement, the ministry further noted that as far as the statistical reforms are concerned, it is important to note that system reforms are an ongoing process and are necessary for ensuring responsiveness to the changing needs of society.

Demands for Statistics

The ministry points out, "Over a period of time, there have been increasing demands on the statistical system for the production of relevant and quality statistics. The Ministry has been accommodating these demands by optimising the available resources and use of technology."

As in any system, the advent of technology necessitates reforms in statistical processes and products with an aim to synergise the existing resources so that the system remains responsive, they added.

Merger Under MoSPI

Last month, the government had approved the merger of the National Sample Survey Office (NSSO) with the Central Statistics Office (CSO) under MoSPI. Ahead of this amalgamation, MoSPI said that the recent step for the merger of CSO and NSSO was aimed at leveraging the strengths of the two organizations so that it can meet the increasing demands.

In some sections, though, the Ministry of Statistics and Programme Implementation has been criticized for the quality of macro-economic data.

Responding to the questions raised on the credibility of India's official statistics, the ministry says that India has adopted the United Nations Fundamental Principles of Official Statistics (FPOS) in May 2016 and is committed to ensure and secure the autonomy and independence of the statistical system to produce appropriate and reliable data.

It also emphasizes, revision in GDP estimates occur when data coverage from administrative sources improves over time and these improvements get well documented. Consequently, the initial estimates of GDP tend to be conservative.

So, for the improvement of this, it would require concomitant changes in the sectoral data flows and associated regulatory framework in the data source agencies to facilitate the use of more macro modelling techniques, the ministry's release noted.

As all these reforms are in the continuing process, it is significant that the readers and users understand and appreciate the limitations of data and the challenges in estimation. The ministry states, "While undertaking these reforms, it is important to realize that newer data sets and survey results will invariably be used and it would be incorrect to comment that old processes were better than the new."

However, it further adds, the reforms being undertaken in MoSPI will lead to better datasets and better estimates in the future, and will be duly deliberated on by the ACNAS during the Base Year revision.

Ministry of Statistics and Programme Implementation (MoSPI)

Currently headed by Rao Inderjit Singh, the Ministry of Statistics and Programme Implementation is a ministry of Government of India, concerned with coverage and quality aspects of statistics released. It came into existence as an Independent Ministry in the year 1999, after the merger of the Department of Statistics and the Department of Programme Implementation. It has two wings, first Statistics, and the other Programme Implementation. The surveys conducted by the Ministry are based on scientific sampling methods.