GitHub, for instance, shows component-level uptime percentages such as Git Operations at 99.83% and Pull Requests at 99.55%. This information is presented alongside an account of what went wrong. A simple description of platform performance allows users to know when things are broken and when they are fixed too. For consumers of these platforms, this creates trust as many businesses now run with the help of these platforms.

Github has been facing unprecedented load since the boom in AI-assisted software development where the volume of software code being written and pushed to Github has increased exponentially. Consequently, GitHub spent a large part of last month in internet infamy, with every hiccup laid bare for the world to see — all enabled by its own transparency tool. Yet, throughout the turbulence, the status page remained online, holding the company accountable until the dashboards eventually turned green again.

Now open the Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS) platform, the Pradhan Mantri Gram Awaas Yojana platform, the Jal Jeevan Mission platform, UPI or Centralised Public Grievance Redress and Monitoring System. What you will find is what these platforms choose to show: scheme targets, funds released, houses built, piped water connections created. What you will not find is how reliably the portals themselves work, how quickly grievances are actually resolved, or what a citizen should do when the system fails them. The performance of the scheme is reported while the performance of the platform is not.

Without these hard numbers, what you are left behind with are hearsay on the slowness of pages, servers being down etc. But, every time you would try to raise these issues with the engineering teams, they will be set-aside as mere gossip, or one-offs or repudiated by loading the page right in front of you in 100 MBPS leased web lines and saying “see it works”!

We’ve agreed that we need evidence-based development, but where is evidence-based software development?

Pun aside, technology is playing an increasingly important role in welfare delivery. It is no longer an accessory but the medium through which benefits and services are being delivered. If these platforms don’t work optimally, the downstream services and benefits get impacted adversely. From registering job demands on NREGASoft, tracking housing instalments on PMAY-G to filing water connection requests on the JJM portal, millions of citizens are now nudged to access entitlements through digital platforms. The reliability of these systems is no longer just a technical concern that can be ignored or surpassed offline. It is a welfare and service delivery concern. A portal that is down during wage week, or a grievance that loops in a closed status for months, has direct consequences for the families depending on it and those delivering it to them.

This is an important gap to fix. To fix it, we need to know it’s broken, in a more quantified manner. While uptime metrics is a great start, in our context we’ll need a little bit more.

What the metrics should look like

Metrics that could be designed to show a platform’s performance does not have to be complex for the team managing the platform as well as the citizen accessing it. Adapted for welfare portals, a public-facing status dashboard could show the following:

1. Uptime — the percentage of time the portal was accessible and functional, shown over the past 30 and 90 days. This could be broken down by key functions such as application, tracking, payment or grievance.

2. Average grievance resolution time — not just whether grievances were closed, but how long it actually took, tracked against the stated 21-day limit that CPGRAMS itself commits to. CPGRAMS has reportedly seen a 74% reduction in pendency since 2021, but citizens have no way to see what that means for their specific complaint category or state.

3. Payment disbursement lag — the average gap between entitlement approval and actual transfer, visible at different levels of disaggregation such as district, block and village.

4. Incident log — a simple record of when the system experienced failures, what was affected, and how long resolution took. This includes Aadhar authentication failures where necessary.

Why this matters beyond transparency

As studies have shown, lack of trust in government platforms is a common challenge in India. A public status page does not solve this problem entirely, but it addresses one specific part of it: it tells citizens whether the system they depend on is actually working.

There are also benefits for the system itself. Frontline workers, namely Data Entry Operators, Rozgar Sahayaks, block and district-level officials, spend significant time navigating portal errors they cannot explain. A live status page gives them a reference point when systems are down. It tells them whether the problem is on their devices or the system’s end. This also helps them redress the large volume of queries that citizens bring to them during outages.

It isn’t just the citizens and frontline staff, engineering teams that run these systems and Union and state-level bureaucrats overseeing these systems stand to benefit too. This evidence can help them identify recurring bottlenecks, improve coordination, and build more reliable public systems over time.

The 2024 OECD Digital Government Index specifically highlights reliable and resilient digital public infrastructure as a foundational requirement for sustainable digital governance. It is important to note that reliability here is not a feature, but a fundamental baseline.

The government already measures scheme output. It is time to also measure and openly share platform performance. If the portals that deliver welfare can tell us how many houses were built, they can also report whether the portal was up when the beneficiary came to check if the funds to build their house were disbursed or if the OTP came when an engineer was trying to use the inspection mobile app to inspect the road built by the contractor.

This was first posted on Foundation for Responsive Governance’s website and written in collaboration with Sidharth S from ResGov. They are a fantastic org, doing a fantastic job in building the state’s capacity and ability to ensure government’s policies reach people. If you are in the space of public policy, there is a good chance you’ve read work from someone or the other in their team.

Out beyond the ideas of rightdoing and wrongdoing, there is a field. You should go speak to users there. If you can't, open YouTube.

Rumi, probably.

A large problem with working in gov-tech is the lack of bottom-up feedback loops. In an ideal world, gov-tech teams would have PMs and Designers who would speak to distant end-users and do field-research to inform decisions shaping IT systems which are in-turn shaping or being shaped by policy. But currently, Delhi Aur Developer Dur hai (Delhi and its developers are far). Most gov-tech projects have unreasonably small teams working on them and largely staffed with only engineers and overseeing bureaucrats. These tiny teams commendably pull weight akin to a dung beetle but most of that strength goes into rolling the same ball uphill: keeping existing systems running, fighting fires, and fixing bugs (sic). There’s little bandwidth left for the kind of ground-up discovery work: talking to end-users, iterating on design, rethinking flows or things that would make these products genuinely work for people as opposed to just barely ‘work’.

If you find yourself in such a team, wanting to know what your end-users actually need, but stuck in the Sisyphean sustainment of existing systems, take a break.

Open YouTube.

One of the first tips I give our Build for Bharat fellows and new colleagues in gov-tech is to search the gov-tech app or website they’ve been assigned on YouTube. It is the fastest way to do some discovery especially if it is a G2G app or actually relevant G2C app. If you happen to work on a reasonably important and well-used gov-tech system, you will surely find yourself on YouTube.

Take the example of the GST Portal:

or Karnataka’s e-Swathu Portal:

or something that is often in the news, EPFO:

[How to check balance or login videos have views in the millions]

Overall, you’ll find videos of one of the following types:

- How-to videos

- Work-around videos

These are videos made by the power-users within your system. These noble souls fill a gap left by the lack of user-friendly official training, which needs to exist in the first place because of the lack of user-friendly systems.

You can find the following signals from these videos:

1. Understanding which modules are not very user-friendly and have induced the demand for explainer videos. Sometimes they are simple explainer videos, sometimes they are work-around videos.

2. Bugs in your systems and how people work-around them. Think of these as ‘Something went wrong’ unboxing videos.

3. How people are gaming/by-passing your system for either unscrupulous reasons or because the IT systems are enforcing a poorly informed policy [bug in policy].

The first two are pretty straightforward, the third one is very interesting, and worth a real example.

A couple of years ago, when Richa and I were working in the Ministry of Rural Development, one of the projects we were assigned to was to alleviate the emergent issues arising from the transition from regular bank-based transfer of wages to an Aadhar Enabled Payment System.

The shift was not simple. A worker had to seed their bank account with Aadhaar, which in turn required getting mapped on the NPCI database. Depending on the bank, the state government’s staff, the existence of multiple or deactivated accounts, this caused stress, anxiety and loss of wage for many. The Ministry issued multiple deadlines, each extended on the last day. Delhi thought this was a straightforward process. But straight and forward is rarely the motion experienced by India’s majority gov-tech end-users, who remain suspended in Brownian motion set in motion by brown sahib technocrats.

As NREGA extends a right to work, a worker cannot be denied work. What the Ministry did was to modify the software such that workers without AEPS wouldn’t be paid for their work until they transitioned but they could continue working.

Now, work in NREGA happens via what is called a muster-roll, where a group of workers are added and then are assigned a site to work. Post working, this muster-roll is then converted into a wage-list, and the wage-list then processed for payment. Given the payment isn’t processed individually but in a group, the software was modified such that the entire wage-list wouldn’t be processed even if a single person in it was not enabled for AEPS.

“Work is disappearing!”

Once this change was initiated, news and murmurs came that workers are complaining that their work has disappeared. Delhi didn’t give these any weight as it knew it hadn’t done such a change in the system and assumed that these were vested interests resisting the transition.

And then a group of well-wishers forwarded us a video that shed light on the situation.

If you understand Hindi and have followed this story till here, I urge you to watch it:

The video shows a workaround: as a Gram Rozgar Sevak (village employment assistant), your village has a wage-list that gets stuck entirely because of 1 or 2 non-AEPS workers in it. You are facing pressure from the remaining workers within the list who have AEPS but are not getting paid because of their fellow villagers who are not yet AEPS-enabled. What do you do? You can go to the muster-roll and delete the non-AEPS workers from it and therefore erase the record of work done and wages entitled to them.

Imagine. You worked hard for 14 days, tilled a farm or dug a pond. You are owed by the government 14 days of wage. But as per the Ministry, you cannot be paid till the AEPS process is completed. So while you go to the bank to sort it out, you find your name and therefore work has been erased from that particular 14 day period’s records. Gone. Poof.

What strikes me is how technocratically the video explains the process of deleting the entry from the muster rolls after people have worked. Clinical, cold and without emotion. Machine-like. Someone’s hard earned entitlements, robbed away in a click of a button.

I don’t blame the Gram Rozgar Sevaks who might have faced pressure from fellow villagers to get their ‘straightforward’ payments processed, or the villagers who are expected to organize wherein they all decide that none of them will receive payment till all of them can.

This is what the third type of video can give you that no internal review will: ground truth about how your system is being bent in the field, narrated by the people bending it. Delhi was dismissing the “work is disappearing” complaints as resistance. The video showed it was a feature of the design.

On getting to know this, we advocated that the option to delete entry from muster-roll should be removed from the system. But this would just work for one cycle; eventually workers without AEPS might stop being added to muster-rolls at the beginning itself to avoid blocking other people’s payments.

The other ad-hoc solution was for such entries to be considered as failed payments, which can happen for individuals within a wage-list and don’t block the whole list and are routinely re-tried. This would ensure once the worker had completed the process of AEPS, their older payments would get picked up by a cron job and re-processed without manual intervention.

But that’s all patchwork. The real issue was the incentives being created purposely or without thought that led to such issues, and that’s something we only saw clearly because someone made a YouTube video about it.

In our work with NREGA, we ended up making official videos ourselves and embedding them as links in the respective modules. These authoritative videos got a lot of traction and positive feedback, corroborated by the increasing view-counts.

You have to remember: in most trainings that happen when a new software module is released, the intended users of IT systems visit, not the actual users. The actual users of many systems are Data Entry Operators, MTS staff, and Computer Operators who login on behalf of the intended user to complete the digital tasks. By embedding YouTube videos within actual workflows, you can actually reach your actual user.

A poor way to do this job is to remain in blissful ignorance, imagine a simpler system and make technocratic decisions departed from reality. The right way to do this job requires a lot more, and sometimes that starts with watching YouTube.

First, our project in partnership with Ministry of Statistics and Programme Implementation went live:

Second, National Highway Authority of India’s redesigned website dedicated to purchase of FASTag Annual Pass went live [see it].

The former was largely received with disbelief and surprise that government and MCP could go together. For those that don’t know, Model Context Protocol allows third-party apps to connect to your favourite AI agents. That’s how Claude can connect to your Google Calender because Google exposed their service via an MCP server. We worked with Ministry of Statistics and Programme Implementation to expose Indian Government’s official statistics (CPI, National Income, PLFS etc) to your AI Agents via an MCP server so you can query, analyze them etc in your existing workflows.

The latter was NHAI’s consumer facing interface for FASTAG, highway complaints and more recently the purchase of FASTAG Annual Pass. I would urge you to open the site to understand why it is different from regular government websites. The tweet above was part of a discussion sparked by a US government’s affordable medicine website launch and people asking why our government couldn’t do such a thing.

Both arrived the same week. Both provoked the same underlying reaction: surprise that government technology could actually be good anchored by an example.

That surprise is the problem.

It is not that we don’t know better is possible. 9 to 6, most of us in government experience a certain flavour and quality of technology. 6 to 9, we experience what’s possible in technology otherwise, as we use private apps which never crash midway, have OTPs that come on time. Externally too, citizens show dismay from time to time, but it doesn’t convert into heightened expectations or comparisions. In our heads we have compartmentalized both experiences as separate realities. Rather, these dismays are categorized as things that will never change. Sarkari hai, aur kya ho sakta hai. (It’s government, what to expect?).

The compartmentalization is too strong. The imagination of a better reality has to come from within the government ecosystem.

Last year, Richa and I, co-founded Bharat Digital, a non-profit dedicated to solving gov-tech as a systems problem. Increasingly, policy outcomes depend on the state’s capacity to implement or leverage technology. But the government’s capacity (or ability) to do this properly cannot be assumed across its length and breadth.

Both of us have combined 15 years inside the government implementing technology for some of India’s largest programs. So we have some first-hand experience on what are the problems but there is no easy fix here.

We’ve bucketed the problems in talent, systems and culture and we have experiments ongoing across all three buckets. But an emerging theme that cuts across is the need to shift gov-tech’s Overton window - for those not perennially online, is the range of policies and ideas considered acceptable to the mainstream at a given time. We need to shift the Overton window on government technology. We need to change what people expect from gov-tech and what is considered acceptable. People here includes citizens, gov-tech engineers and managers, bureaucrats, frontline staff — everyone.

We are trying to enable this shift in two ways: micro and macro.

For the micro, we run the Build for Bharat fellowship where we identify, equip and embed early career technologists and designers within government departments. The fellows work alongside existing government officials on strategic, meaningful gov-tech projects and augment the department’s capacity. To get a flavor of the projects, fellows work on, you can see the demo-day presentations from the previous cohort here. They range from using AI to reduce incident response times at the 112 emergency helpline, to redesigning Tamil Nadu’s online citizen facing services portal or attempting to predict Monthly Per Capita Consumption Expenditure which is calculated every 5 years, by using data from other fast-moving surveys.

Our fellows shift the culture of how software should be thought about and crafted within the department itself. In their own ways, they show and not tell. Through proximity and osmosis, we see senior stakeholders pick up the language of design, user-centered work, or modern tooling like GitHub Actions, CI/CD and Figma from the fellows. The culture moves not because anyone gave a lecture, but because someone in the room was simply doing things differently and people start expecting their own existing staff to function that way.

At the macro level, when some of these projects go live, it shifts the expectation of the larger public and other government departments: that this is doable and should be expected from the government. Its not limited to the projects our fellows pursue during the fellowship, but experiments which come out of Bharat Digital’s Labs. The MoSPI MCP server came out of Labs and the NHAI FASTag website was done by the fellows last year during the fellowship. These are macro moments. They exist publicly. Other departments can see them. Citizens can point to them. They become counter-examples and shift the window on the what is possible and expected.

We have many such projects under-way now - from reimaging frontline tech beyond top-down surveillance, bringing more accessible modalities to government open-data or designing more agentic workflows within departmental monitoring.

The goal here is to create these surprises, until they are no longer surprising. Not because the work becomes less impressive, but because good gov-tech becomes what people expect. That’s the window moving.

I will not assume you’ve followed what I’ve written about all this time, but my broad theme has been that of lamenting about the status-quo and hope that young people in the sarkaar can really affect change.

I lament about fellowships where engineers who wish to do good in the world, decide to leave engineering and join these fellowships where they offer general program management expertise. I wish more engineers did good, by doing good engineering.

I lament that the greater the physical separation between the policy-maker, developer and end-user, the greater the divergence between policy objectives, IT solutioning, and ground-reality.

But I am also optimistic. I am optimistic that technology does actually improve governance when done right. I’ve seen this over and over in my time in the government. Things do change.

I know that to be a good technologist in the government, you need more than engineering skills, you need to be translators, be okay with doing little things and that change takes time.

I’ve had a fortunate journey in the government, it started by happen-stance, I got incredible mentors who gave me opportunities far beyond my rank, I had devoted colleagues and my circumstances allowed me to stay inside long-enough to see change.

A little storyline:

• In 2016, when I was looking for a new job that was in the intersection of data-science and impact, my worldview was limited to NGOs and development sector organization who were hiring data scientists to analyze RCTs.

• I just didn’t think of government as a place of outsized technological impact. My journey started with happenstance, I was applying for a Data Scientist position at EPoD via IFMR, and Pratibha saw my CV and asked me to apply to a different position with the Ministry of Rural Development.

• I was hired as an experimental Data Scientist in 2016 with rural skills division at the Ministry of Rural Development. didn’t have a lot data, so I worked on helping design an ERP system + unlearning and learning about government. I had excellent mentorship and starting conditions (Tanya, Senthil, Simer, Manish, Anisur, Anir Sir, Santosh Mathew and Alka ma’am). I went to the field consistently and was introduced to public administration and practical policy.

• I attended UChicago’s Data Science for Social Good fellowship where my understanding of public-interest data science was made. They knew what they were doing and when zoomed-out most public-sector data science interventions followed a structure. The fellowship was rigourous, structured and for me professionally, life changing.

• I had phenomenal 3-months working with Dutch Government’s version of NHAI. I worked on predicting accidents on Dutch highways to pre-emptively asking patrol cars to drive in that direction. I was able to see what is possible. Sometimes you really need to see what is possible and being done already in other parts of the country/world to get the drive to push change where you are.

• I asked Rayid, the director of the fellowship, “I don’t think 3 months is enough to bring any change on the problems we are tackling”. He said and I paraphrase, “Ofcourse, the aim of the fellowship is not for you to solve things, it is to give you a taste so that you all eventually pursue public-interest and bring change”. Boy, he was right.

• I moved to rural-roads in 2018. They had a lot of data and problem statement. Next 3 years, I implemented the public program where an algorithm was integral to scheme and made me really understand what tech and policy together can achieve and how it can go wrong. Those 3 years remain my most rewarding years. In all the AI hype, that case-study is years ahead of its time even now. [Paper][Talk]

• Around this time, I learned two things: First, difference between Project Management and Product Management. Second, GIS is under-utilised in public sector and QGIS is all you need. We started by creating two Product Manager position for PMGSY and one GIS Specialist.

• We ended up hiring Richa for PM. I learned from her that good product-management is the most pressing skill we need to fix-gov tech.

• Having proved how data science can significantly improve governance, my boss and I pitched the head of the Ministry of Rural Development that we need a pan-Ministry interdisplinary Data and Product team.

• 2020-2023 [Secured funding for the first year of the team, hired a good team, applied data and product interventions across NREGA, PMAY, NRLM, PMGSY etc]. We have many use-cases deployed with product, data and ML (in that order). And I believe strongly for that to be the case with most departments.

• Departments choosing to focus on AI, while they have broken products or stores of data with which decisions are not being made, are misguided willfully or otherwise.

• Moved to NHAI as Advisor-IT. Setup a similar team of Product Managers, Data Scientists and GIS folks. The team has incredible impact over the last 2-3 years and I can say with humility it has changed NHAI’s digital trajectory.

• I started focusing on engineering systems [led a SOTA Drone-AI project, co-wrote the design and policy for Advanced Traffic Monitoring Systems [the cameras you'll increasingly see on highways] and was part of the core team designing FasTAG 2.0 [no more toll plazas], started designing and evaluating procurement tenders etc. This is a whole different game and expertise. Engineers have an important role to play here. Gov-tech is an engineering-systems play.

• NHAI revamped its hiring policy for tech roles. Really good pay, work from home, study leaves, promotion tracks etc]. It is near optimal but still no health insurance etc. More to do, but a gold-standard for other government agencies to follow.

The reason to recap is to make two main points:

Conclusion #1: Introducing people with good skills, right mindset and right starting conditions is the one of most solid inteventions you can do to improve government.

Coming to these conclusion is easy, but following through on it has been hard.

Over my tenure, I’ve maybe conducted more than 200 interviews for public-tech positions. While, I have been lucky meeting and hiring some incredibly talented and nice people, it has been increasinly hard.

Most good hires (with few exceptions) were via outbound reach or people reaching out to me on my socials. That’s not sustainable at all.

This problem is widespread. Everyone I know in government who is hiring for tech positions is struggling with talent. The pay is also decent, the problem statements are challenging but we suffer from a supply-side problem. Not many people want to work in government on tech, out of pre-concieved notions about #sarkari or simply not being aware of such opportunities.

Conclusion #2: Before we move on with DPI/DPG/AI, we need to fix our basics, urgently. Fixing legacy is boring, complex and wicked hard. But it needs to be done.

Well, we want to fix this now.

It is criminal that we struggle to fill these positions and pick sub-optimal talent to work on such mission-critical systems. Not only do we need more skilled technologists, we need more than that, we need a new breed of technologists.

We need Public technologists that are highly skilled technologists who understand the realities of public service, policy, and the unique challenges of implementing tech solutions responsibily within complex social frameworks.

And if we can’t find these people, we need to create this pipeline.

Introducing, Build for Bharat Fellowship:

The Build for Bharat fellowship aims to identify and train early-career public-interest technologists who can strengthen, modernize and humanize the digital technologies used to address India’s most urgent public and governance challenges.

We’re looking to place India’s top engineering and design students at the heart of some of the nation’s toughest governance initiatives, bridging the gap between state-of-the-art technology and genuine public service.

What does the Fellowship offer?

1. Real-World Problem-Solving: For 2.5 months this summer, Fellows will be placed in government departments in partnership with our program. Here, they’ll work on high-impact tech projects that either improve the lives of millions or enhance the effectiveness of government operations.
   

2. Orientation: Before and throughout your placement, you’ll undergo a rigorous orientation program—diving deep into how public systems operate, why large-scale tech solutions can fail, and how to build resilient, responsible systems that account for India’s societal and policy complexities.
   

3. Expert Mentorship: You’ll have access to one-on-one office hours with leading experts in databases, security, distributed systems, open-source technology, and design—drawn from top tech companies, academia, and the public sector.
   

Eligibility: Students graduating in 2025 or 2026

Stipend: Rs 40,000 per month

Source: https://bharatdigital.io/fellowship

The experience is very carefully being designed based on our own reflections of what works and doesn’t work in the governemnt. Hence the slightly inudulgent storyline in an otherwise very depersonalized newsletter.

It is cohort based, so you have company, bureucracy is easy to endure with friends.

You have mentorship from Industry experts: Because tech-mentorship is limited inside government, and this internship isn’t a compromise on technical learning. Often, technologists in the public/dev-sector turn out to be sub-par, good-intentions cannot mean a compromise on skills.

Problem statements will be neatly scoped by our core team and the partner government departments before the interns arrive. This will also ensure projects are value-aligned.

Orientation: The core of this fellowship is to shape the future public technologist. They should be thoughtful, responsible, understand public sector complexities and how tech can do good/harm in our very specific setting. It is a very specific mindset and training which you need to thrive in the public sector. It is the starting condition and that will be our MVP.

Selectivity: We are not going to chase people to join. Its going to be a very selective fellowship and you need to have some commitment to public or social-good before you join.

We are excited and nervous but we know people make things happen. And we need more people.

The fellowship will be housed under an initiative called Bharat Digital. The fellowship is just one of the things we are doing. Gov-tech is a wicked problem and few countries have been able really solve it. We need to tackle it at three levels: systems, culture and talent:

We are staring with Talent, but other things are cooking.

How can you help?

• Share the word! https://bharatdigital.io/fellowship

• Like our LinkedIn page and share the post (that’s where the talent is)

• If you have thoughts on what skills a public technologist should have, help us curate their Orientation curriculum.

• Volunteer with us with the fellowship (scoping, vetting etc) or with Bharat Digital in-general.

• In any way you think you can support us in partnerships, fund-raising etc.

With love,

Submitted Please

So you go and talk to the lead bureaucrat. Tea is ordered. You ask for green tea thinking black coffee might bring you more attention than you need. You want to center the stakeholder. 2024, baby. They see you have some leverage (either donor money, deep tech talent etc). They start telling you issues from the top of their minds where tech can be applied. You have their attention while files with more complex issues are put aside. You sip your green tea. You are abhorred that it came with sugar. Before you know it, the discussion is no longer about problems but about possible solutions.

'Agar aisa kuch banayege, toh accha ho jata' (If you made something like this, it would be really good)

Wishes are made. Fantasies are fantasized.

You go back feeling good about the meeting. You have a way forward, this is something they want you to build, it is an internal ask, stakeholder be participating.

In the very off-chance you end up actually building said thing, it is highly likely it wont be used.

Why so?

I don't mean to be patronizing and say the user doesn't know what she wants. Depending on where the user is on the decision-maker vs decision-implementer axis, she might have useful hints.

But a pattern I've repeatedly seen in conversations between head bureaucrats and technocrats (you) is that solutioning takes the liberty to be in the realm of fantasies or outsized ambition. Often bureaucrats will pitch more riskier, loftier or ambitious problem statements to people who have leverage (external capital, tech skills etc). It makes sense, you think you have internal machinery to pursue your day to day problems, you can leverage people from outside for more fancier or ambitious bets. After all, the external technocrat is unencumbered by the administrative complexities and constraints of the everyday. You'll be tasked with silver bullets that kill mythical werewolves, while the kingdom faces an ongoing murky rat-infestation.

But what should you building? Is there even a werewolf problem? Should you not be improving the rat traps?

You need to be careful after all. Proper project scoping is everything. If you don't get it right, why bother with the rest. It is already an arduous long journey to get 'anything' implemented within sarkari constraints, at least you should be convinced about the end goal. Imagine among his existing troubles, if Sisyphus was later told that the hill itself was wrong?

So what do you do to evaluate whether a solution being pitched is solving a genuine need? Will it be used by stakeholders once made? Is there an organic demand?

Here, Paul McCartney, noted civic technologist, has a thing to add:

Hey Jude
Don't make it bad
Take a sad song and make it better
Remember to let her into under your skin
Then you begin to make it better

Supplementary reading, I recommend opening in a new tab while you read the rest:

I think what he wanted to say was that if you want your tech intervention to be used, identify an existing implemented process, understand it deeply and then improve it. And yes, don't leave it worse than how you found it.

Simple and profound.

Don't try to introduce new behaviors, just take an existing behavior with proven intent and make it better.

What does it mean?

So for example, if a bureaucrat wants you to build a fancy state-of-the-art AI model to identify fraud, first see if they are already implementing rudimentary things and taking action. Are they closing the loop? Suspending people? If not, then AI isn't going to change anything. The limitation on action isn't the efficiency or scale that algorithms bring.

Or, say you are writing an algorithm that equitably distributes funding across women community groups. Is there an existing defined process where they are deciding this in a rudimentary manner?

Or, you are asked to create a solution that identifies 'activity' in classrooms using AI on the CCTV footage, but are they currently acting on audit reports on teacher quality, citizen complaints or manually auditing footage on sample and taking corrective action?

Or, you are asked to use drones to identify potholes on roads, but how robust is the existing pipeline for reporting, tracking and closing defects identified in regular manual inspections?

So if the machinery isn't acting on something already in a rudimentary or manual manner, there is little evidence it will be acted upon later when scaled and made fancier.

A key part of scoping a project is mapping what actions will be taken based on the output of your intervention. But action depends on many factors such as political economy, existing capacities, IT infrastructure etc. Many of these are not in your control or fit in your timeline of change. So it is easier to take processes where the output-action pipeline is already in place and focus on just improving the efficiency/quality of the output.

I am not being overly unimaginative. Having seen dozens of technocratic projects both successful and failed, I try to have measured ambition. This is not to say technologists shouldn't pitch projects that would possibly change behaviors. My prized projects are those. That should be the goal and exponential gains often come from such projects. But for this, you need patience (multi-year or decadal). You need to be clear that this is what you are getting into. There will be moments where the circumstances align and you'll be able to pitch a completely new behavior-inducing tech solution and even be successful but the stars need to align, and stars there are many. In the meantime, there are rat traps that can be improved.

So in summary:

Look for existing actions and not intent (relationship advice as well)

It is not what they say they'll do if they had your silver-bullet, it is what they are already doing with rat-traps and can that be improved by you.

So what do you say when someone asks you to chase the next technocratic/goal-post shifting/silver bullet?

Na-Na-Na Na-Na-Na-Na Na-Na-Na Hey Judee.

PS. More than 50% of the song is the Beatles going 'Na Na Na Na' and I think that makes 'Hey Jude' the perfect tech-scoping song. A good technocrat should be saying no more often than yes if they are doing their job well.

In the previous post,

I explained that default maintenance contracts are typically based on item rates, where the contractor issues bills according to the volume of work completed (both material and labor) for tasks such as filling potholes, repairing cracks, and clearing shrubs. There is a predetermined cap on the maximum amount that can be billed in a year. The process involves the contractor raising a bill, which then triggers the action of the government engineer. Upon receiving the bill, the government engineer is supposed to visit the site to verify the contractor's efforts. However, if the engineer fails to visit and document sufficient evidence in time, new potholes may emerge or shrubs may regrow by the following month, thereby complicating the verification of the work performed.

In summary, what are the characteristics of road maintenance?

1. Difficult to estimate work needed and depends on external factors (sudden rains, farm water spillage, heavy trucks, etc.)

2. Low ticket bills

3. A high number of recurring bills (60 bills over a 5 year period)

4. Difficult to prove work (not) done if inspections are not done on-time

5. Difficult to prove work was done upon future scrutiny

For government engineers, processing a maintenance bill is not only a difficult and risky task but also requires timely and regular assessments. These challenges are compounded by the context of poor state capacity, where government field officers are often overburdened and under-resourced. Additionally, they are frequently tasked with duties unrelated to their primary responsibilities, further complicating their ability to manage road maintenance effectively.

….

Therefore, there is a need to simplify, de-risk and unburden maintenance.

Introducing Outcome and Performance-Based Maintenance Contracts (OPMBC). In these contracts, the yearly estimate is divided equally into monthly amounts, and the contractor bills the full amount each month, regardless of the volume of work completed in that time period. The engineer's role shifts to evaluating the condition of the road on the day of their visit, focusing on the outcome rather than the quantity of work done by the contractor. This approach emphasizes the results of maintenance efforts and not the effort itself, thereby aligning contractor incentives with the achievement of desired maintenance outcomes.

The key building blocks for OPMBC-based routine maintenance are:

• The contractor submits a full bill

• There is a quantified marking criteria that measures the performance of the contractor or the maintenance outcomes.

• Based on the marks, there is a mathematical formula that calculates the money to be deducted from the monthly bill.

For example, if the annual contract amount is Rs 12,00,000, this is divided into equal monthly payments of Rs 1,00,000 each. After say February, the contractor submits a bill for Rs 1,00,000, not based on specific efforts or materials but as part of the regular billing cycle. The engineer then evaluates the road in the first week of March, rating its condition on a scale from 0 (completely battered road) to 100 (perfectly maintained road).

Should the engineer rate the road's condition at 85 out of 100 for February, the contractor will receive a prorated amount of Rs 85,000. To discourage poor maintenance, a rule is established that if the road scores below 80, no payment will be made to the contractor for that month.

This approach eliminates the need for the contractor and engineer to quantify or verify the labor and materials for minor maintenance tasks, such as filling potholes. The contractor is responsible for submitting a full bill regardless of the specific work done, and the engineer assesses the road's condition, awarding marks based on the outcome at the time of the visit. The evaluation criteria for rural roads include a 100-mark rubric covering various aspects of road maintenance like main carriageway condition (30 marks), shoulder maintenance (20 marks) and jungle clearance etc.

Another, issue with earlier item-rate contracts was this:

Further, if the contractor doesn't raise a bill, it also reduces the engineer's incentive to inspect the road, leading to a situation where while no undue money is paid to the contractor, the road remains pothole-ridden, and the public suffer.

So if the contractor doesn't raise a bill, there is nothing for an engineer to measure and validate, and if the contractor then raises it later, the engineer is in a fix as to what to measure now. In many cases, this leads to maintenance bills just piling up for years, and engineers hoping they'll get transferred before a decision has to be made for those bills.

In this case, the engineer can keep recording the performance of the road at the end of each month, irrespective of whether the contractor will raise a bill or not. As long as the performance/outcome of the road is recorded every month, payment can be calculated whenever the bill is submitted.

But still, this doesn't solve all our issues. How does the engineer de-risk himself from future auditors? How do we de-risk future audits and add legitimacy to the measurement of performance?

Without the above problems resolved, PBMC remains a risky endeavor for the engineer.

Funnily, the Standard Bidding Document of PMGSY stated that the maintenance had to be PBMC more than a decade before it started getting implemented across the country.

Why?

One of the reasons was that the policy, while good on paper, didn’t have an adjoining IT intervention.

So how do we save evidence of performance?

As rural road routine maintenance largely deals with defects that are visible on the exterior of the surface, it makes it easier to record the condition of the road. You don’t need to click pictures of individual potholes or record measurements etc that were (un)filled, you just need to capture the general state of the road at the time of the visit. Stand in the middle of the road at a random location and click a picture. That would capture whether it is in good condition or not - at least for narrow rural roads.

So let’s design an IT intervention.

• What if we allowed contractors to generate and submit bills online at a click of a button? As the bills are always the same fixed amount, this should be possible.

• What if the engineer’s monthly inspections were through a mobile app that proposed system-generated locations on a road from where the engineer had to click geo-tagged pictures as proof of performance?

• What if engineers could record marks online against the geo-tagged photographs captured during field inspections from the mobile app?

• What if the final due amounts per month were auto-calculated by the system taking the above marks?

• What if the bills were converted into payment vouchers with necessary taxes and state duties?

• What if payments were deposited directly into the bank accounts of contractors?

So if we incorporate all these features, we get the Ministry of Rural Development’s solution called eMARG.

eMARG is designed to cater to the maintenance of 6,00,000 km of rural roads built under the Ministry of Rural Development’s PMGSY program. That’s a little less than 10% of India’s entire road network (urban and rural combined).

A quick video which explains everything I said till this point. 👇

A little history: eMARG was designed and developed by Madhya Pradesh Rural Development Agency or MPRRDA which is the agency that implements PMGSY in Madhya Pradesh. While the construction cost under PMGSY is shared by the state and central government, the maintenance cost of roads is to be purely born by state governments. eMARG was developed by MPRRDA through its state office of NIC to solve the very problems we’ve been discussing. It was earlier only for rural roads in MP. It so happened that the boss-lady IAS officer incharge of MPRRDA, Ms Alka Upadhyaya, was transferred to Ministry of Rural Development to eventually head the National Agency in charge of PMGSY. She immediately decided to scale eMARG nationally so it can be used across all states for PMGSY.

Not only did the fate of maintenance of rural roads in India change thanks to her, but my fates changed too as Alka ma’am went on to become my mentor and the best-boss I could have for many years that have came after.

It all seems rosy but let’s tackle some caveats:

Q. Aren’t we rewarding the contractor with money even if he possibly didn’t do much work that month?

A. Well, as long as the road is in good condition during the engineer’s visit, the contractor will get the full amount whether they filled 0 or a 100 potholes. It’s a neat trick. Often the contractor who constructed the road is given the maintenance contract for the first 5 years (Defect Liability Period). That means if the contractor constructed a high-quality road, it will have fewer potholes and therefore will be rewarded with maintenance money with less efforts. But if the road is poorly built, the contractor will end up losing money during maintenance as well. This adds to the incentive of building good roads.

Q. Isn’t the performance criteria a little subjective? For eg. out of the 100 marks, 30 are for main carriageway. The policy doesn’t describe in detail how to distribute these 30 marks which could lead to subjectivity, especially given the rule that scoring less than 80 marks overall leads to no payment.

I had the same concerns when the project was being scaled from Madhya Pradesh to a national project. But, I was told that the subjectivity was strategic. The discretion was necessary and part of the design. If item-rate was precise, PBMC in the case of rural roads was designed to allow discretion and promote easier cash-flow.

You see there was already little incentive or money to be made by the contractor in maintenance. Compounded by the paralysis exhibited by engineers, most maintenance bills were either not submitted or cleared till the end of the contract. Additionally, you are trying to get a digitally-averse population (civil + rural contractors) to a new shiny IT system. There was dire need to make maintenance a liquid and free-flowing proposition and the policy-backed discretion provided the needed lubricant.

It was then decided that once maintenance was re-claimed to be an attractive proposition and contractors lured into the system, we would increasingly reduce the discretion (It did happen, we now run AI models on the images that get collected during inspection and are slowly moving towards a video regime).

Moving rural contractors and engineers into this new system was no mean task and took us some years and neat tricks but as both contractors and engineers benefited from this system, it slowly but surely got embraced.

Why is eMARG one of my top favourite e-governance projects?

• It takes a good policy on paper and helps implement it on the ground.

• It reduces administrative burden for both the contractor and the engineer. Amongst a sea of fraud-monitoring projects, this project targets the cause of poor maintenance and not the symptoms.

• It was devised closer to where the action is. It took something devised and implemented at the state level and scaled it nationally (bottom-up). I’ve written previously about how the distance between the developer and how it leads to poor e-gov projects
  

• It led to tangible higher expenditure on rural road maintenance multi-fold the level of expenditure pre-eMARG.

• Multiple states have moved to get their 100% state-funded rural roads too under eMARG as well. As the project is built by NIC and public-funded, we are sharing eMARG as a platform for other state schemes as well.
  

  

• The act of digitizing the processes and generating transactional data of the processes, enabled deeper reviews and solidified maintenance in New Delhi’s and State HQs review agendas.

• eMARG also got the second prize at the Government’s top e-governance awards for the year of 2021.

Parting Thoughts:

One would imagine that such important projects would have already been developed in India as it moves ahead with its digital ambitions. But this project is only 4-5 years old. Road maintenance is an age-old dusted topic, often spoken about in dining rooms, and helps garner votes too. But still, it continues to have scope for such policy and digital reforms.

This is my biggest concern with national digital imaginations being carried away with AI and other Bangalore market-led projects such as UPI, AA, etc. I believe they are extremely important as well, but there remains much left to do with boring old e-Governance and we shouldn’t move on too fast.

There is much left to do. eGov needs to be made cool again.

Next Part:
Does this work for National Highways? Simple visual performance criteria cannot work for National Highways with complex routine maintenance requirements and massive pay-outs. What are they up to?

While the privileged class can hide behind enclaves and the optionality of private services to escape most public service failures, it cannot yet do away with the need to move. While it can exchange overfilled trains for slightly busy check-in lines at the airport, it still needs to use the same pothole-ridden roads as the rest. Sure, roads in better parts of the city are sometimes better maintained, but there are many contrarian examples in favour of uneven roads as class levellers where even high-profile parts of the cities or highways are found in poor condition. If class solidarity is not able to fix roads, then there must be either larger powers at play that ensure poorer roads, or maintaining infrastructure is indeed a hard problem.

The political economy of roads and contractors is a dinner table conversation at family meetups where the men argue passionately “ki sab mile hue hai” while the women clean dishes and put the kitchen back together after the meal, hoping a similar nexus existed with their partners in this economy of maintaining social relations and household chores. As I do in those situations, I aim to steer clear of dismissive and reductive attitudes that blame everything on the political economy.

What if, with all good intentions on all sides, maintaining roads is intrinsically or rather administratively a hard problem? Beyond intentions, does it require reform?

To understand that, we first need to understand how roads are built. Typical road construction contracts are item-based, i.e., there is little creativity upfront on how to make a road. A road that is proposed to meet certain traffic can only be built in a certain way if costs are to be optimized. The load-bearing strength of the existing soil, the estimated traffic projected for a decade, and some other objective parameters available on the site decide the thickness of various layers that make up a road. Each State has a statement of rates (SOR) which dictates the cost of labour, construction materials like bitumen or sand and machinery like road rollers. Based on the design and the SOR, the state government comes up with an estimate for the road, which includes a set margin for the contractor as profit. The competition is among contractors on who can forego a higher margin.

During construction, payment is usually made as the road is constructed and reaches milestones, or on a monthly basis. The contractor submits bills claiming the volume of work done. As roads are constructed in layers, their length, width, and depth are measured physically to determine the volume of work done in that particular layer, which is then multiplied by the rate approved in the statement of rates. Whether it is a milestone-based contract or a monthly billing contract, the number of bills is limited, and each is of a significant amount. When a bill is raised by the contractor, the government engineer responsible for processing the bill must generate and record adequate evidence to demonstrate their satisfaction that the work has been completed and that payment can be made. This requires careful attention and exposes them to future audit scrutiny and vigilance in case norms are not followed. Having processed payments within the government myself, I know it is a meticulous task. The government engineer will typically conduct tests and calculations to verify the claims made by the contractor. This may involve digging a pit on the road to check the thickness of different layers that make up the road to validate the quantities being claimed. This evidence is then documented in a manner that would satisfy any auditor in the future that adequate care was taken by the engineer before processing the payment. If the contractor submits bills months after the actual construction, the engineer can dig a pit and assess quantities. The same holds true if a third-party or government auditor visits at a later date; they can dig a pit and verify whether the bill was processed correctly or if undue public money was paid to the contractor. Due to the risks involved, the entire exercise of validating and processing bills typically represents a high administrative burden.

So, in summary, road construction has:

1. Fixed Design and Predictable Work

2. High ticket bills

3. A low number of bills (Max = Months of Construction ~12-24 months)

4. Easy to prove work done as expected (dig a hole, check thicknesses are as per design or not)

5. Proof remains intact for future scrutiny (dig a hole again, probably get the same answer)

What about road maintenance?

Well, road maintenance has two components: periodic renewal and routine maintenance. The former is when a new layer is put on an existing road due to wear and tear, and the latter involves routine tasks such as filling potholes or cracks, clearing shrubs growing on the side of the road, refixing road signs, or kilometre stones, etc. Routine maintenance is basically maintaining the road, giving monthly care so that the life of the road is elongated, and daily commute is easier. Unattended cracks become potholes, unattended potholes get bigger and expose lower layers of the road, which once filled with water can shift subsurface layers enough to cause more potholes or even cave-ins. The point being, roads require routine/regular care on a daily/weekly basis, and routine maintenance is key.

But how does a typical item-rate contract look for routine maintenance of roads? There is no exact science or estimate on how much material and labour will be needed to maintain the road over a typical 5 year contract. The cost estimate for fixing a pothole is easy once it has formed. But it is difficult to estimate the exact cost of maintaining a road simply because you can't predict the number of potholes you'll get. Typically, contracts are made with an upper limit as to what amount can be paid to the contractor on an annual or monthly basis. This amount is very small compared to the cost of construction. The maintenance contractor is expected to raise regular monthly bills based on the number of potholes they repaired, cracks they filled, and the material and labour involved in clearing the shrubs that grew on the road. Once submitted, the government engineer has to visit and verify the work (similar to construction) and process the payment. If the engineer delays vetting by a month, new deformities would have arisen, or shrubs grown back, or kilometre stones missing. This would make it difficult to assess the work done by the contractor. Even if the engineer vetted everything on time and found it satisfactory, an auditor visiting the road at a later date can find newer deformities formed after the engineer's vetting and accuse the engineer of corruption and processing bills when work is incomplete. Hence, it is a risky proposition for the engineer as well. Further, if the contractor doesn't raise a bill, it also reduces the engineer's incentive to inspect the road, leading to a situation where while no undue money is paid to the contractor, the road remains pothole-ridden, and the public suffer.

In summary, what are the characteristics of road maintenance?

1. Difficult to estimate work needed and depends on external factors (sudden rains, farm water spillage, heavy trucks, etc.)

2. Low ticket bills

3. A high number of recurring bills (60 bills over a 5 year period)

4. Difficult to prove work (not) done if inspections are not done on-time

5. Difficult to prove work was done upon future scrutiny

From the point of view of the government engineer, not only is processing a maintenance bill hard and a risky endeavor, but it also requires them to assess things on time and regularly without fail. This is to be read in the context of poor state capacity (government officers in the field are overburdened and under-resourced) and time and again are given tasks that are unrelated to their primary duties. Related Read is below:

Compounding matters, any grease that lubricates the system and makes it function is also limited due to the low-ticket nature of the bills being processed. Knowing well that bills aren’t getting cleared on time, the incentive for the contractor to maintain the road also reduces and they can focus on bidding on construction contracts in the locality which there has been no dearth of lately.

Therefore, there is a need to simplify, de-risk and unburden maintenance. Many initiatives have been taken in this direction in the road sector and there are learning at-large for other infrastructure domains which are afflicted by similar problems.

In the next two posts: I’ll cover initiatives taken by the Ministry of Rural Development for rural road maintenance and National Highways Authority of India for highway maintenance. Both initiatives are a mix of contractual and meaningful digital innovations.  Stay tuned and subscribe to be notified 🛣️

PS. If this landed in your email - please forward it to someone who would like this.

There were times in my family's tryst with financial struggle when I, as a teenager, used to think that if a random wealthy individual would just swoop in and give us a certain amount of money to clear our debt, our problems would be solved, and we could start afresh. For the ultra-wealthy individual, it wouldn't mean much, but for us, it would be a lifesaver. There was nothing structurally wrong with our situation, just a poorly planned debt that was slowly but surely draining all assets accumulated up to that point.

That thought has stayed with me. While I may not be in a position to unflinchingly clear debts of the size my family was afflicted with, I can still part with money (that I consider a small amount) and give it to others for whom it could be significant. After all, it is relative.

So, a few years ago, I started my own personal cash-transfer journey. I decided I would give away a certain amount of money in cash each month. But to whom?

I found the practice of evaluating whether a person is needy enough to be an uncomfortable proposition—who am I to judge a competition of misery?

So, I decided on two things: As long as certain criteria were met, I wanted to give the money away randomly without asking (to women, the elderly, small businesses etc). I didn't want to give it to non-profits, as I assumed that population is taken care of by others; I wanted a certain randomness or entropy in the selection process.

A sucker for scale, I thought if only there was a database of all Indians, you could randomly pick one out of a billion, and then send them money. After all, we, the readers of this newsletter, are in the top 1% or less by income in India.

But I can imagine this makes people uncomfortable too: people want to make sure their money has the most impact. If I pick an Indian randomly from the census data, they will be less privileged than I, but are they the most in need?

Aside, for some reason, the accountability we expect from our philanthropic donations doesn't apply when it comes to tax money, public expenditure, and voting, but that's a different topic.

Anyway, so how do I ensure our cash transfer reaches those most in need? In our day-to-day urban life, our face-to-face interactions with people less privileged than us are limited and self-selecting. There needs to be a way to leap beyond the immediate.

Hypothetically, I would want to click anywhere on the map of rural India and be able to select a household and send a cash transfer to them.

Let's say this random point I zoomed into in Jharkhand.

Source: https://maps.app.goo.gl/CtMkZ4FNP4x2UwGd6

Someone residing in the remote hamlet of Ganarbhara in Bishnugarh, Hazaribagh, Jharkhand, could undoubtedly make better use of the money I might otherwise spend on a fancy dinner in South Delhi. There's also a good chance that India's burdened civil society isn't reaching this area, especially given the absence of a paved road. I would, without much hesitation, proceed with a cash transfer, but how?

While identifying a household remotely is possible, I still need a mechanism to transfer money. If it's hard cash, I would need someone to physically deliver the money at a set frequency. If it's digital, I still require bank details for a one-time transfer.

Both methods involve associated costs and, even if feasible for one case, scalability is a concern.

Therefore, we need a readily available database of needy individuals and an inexpensive mechanism for money transfer.

The minimum viable information for this database would include:

• Block, District, etc., to identify underdeveloped areas and Aspirational Districts

• Gender, as cash transfers to women in the household lead to better outcomes for children and nutrition

• Payment Information

• It should be at the population scale, utilizing an existing database because creating a new one at this scale is not feasible. Filtering people from a population database is more fair than building one from scratch for a program, as the latter will have many exclusion errors.

I imagine an online platform where anyone can select an amount to donate via UPI. The system would randomly select a recipient from the database as described above and pair the payment directly to them. The only fees involved would be for payment processing by a third-party, facilitating a pure peer-to-peer transaction. Simple, efficient, and direct.

However, the simplicity of this idea hinges on having a ready database. Creating this database through field surveys, means tests, etc., would compromise its viability at the scale I envision.

My ambition seemed too high until last year when I inadvertently found myself involved in creating such a database. The Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS), one of the world's largest welfare programs, decided to make the Aadhaar Based Payment System (ABPS) the primary source for wage payments to its beneficiaries.

For those unfamiliar, MGNREGS entitles workers to 100 days of paid work from the government in labor-intensive tasks. The assumption is that only those in dire need would opt for MGNREGS over better-paying jobs. Payments are made directly into bank accounts every 15 days. Last year, ABPS was prioritized and mandated as the method of payment. The idea is to use Aadhaar as a universal bank account number, simplifying the process for both the government and beneficiaries. Once a bank account is linked with Aadhaar, the National Payment Corporation of India stores this information, allowing payments to be made by simply entering the Aadhaar number.

Although theoretically neat, this shift to ABPS had its challenges, including confusion and protests on the ground. Despite these obstacles, as of this writing, 90% of those who have worked under MGNREGS in the last three years are receiving payments through ABPS. That's 129 million people, or about 9% of India's population.

With the database now including gender, geography, and a robust measure of need, MGNREGS could serve as a platform for a quasi-universal basic income (UBI). The government already has the IT infrastructure for bi-weekly payments, with 450 million transactions processed this financial year alone.

However, concerns with this database include the potential for false positives and data contamination if MGNREGS were used for UBI purposes. Additionally, there are still areas unaware of MGNREGS despite its widespread presence.

The randomly chosen hamlet in Jharkhand? MGNREGS work has indeed taken place there, indicating the program's potential for facilitating direct cash transfers to those in need.

What do you think?

PS. If this landed in your inbox and you liked it - please forward it to your friends?

Technology or data science is often leveraged within the government to prevent fraud from happening or for increasing legibility at the cost of increasing administrative burden. Even when we fix one leak, it seems like another one pops up elsewhere. So, if we narrow our focus to specific forms of leakage, many interventions seem successful but if broaden the system boundary, I am unsure if the overall system's leakages have reduced to the extent desired. A senior technocrat once explained to me that corruption is like matter, you can't destroy it. You'll plug a leak and it'll take a new form. It will mutate or displace but it won't be destroyed.

Eg There is a subsidy program wherein beneficiaries to be given money to buy certain goods. In the cash regime, it was noticed that the distributing officer kept a cut with them and disbursed less money than as mentioned in policy. The government decides to directly transfer money into the beneficiary’s bank account thereby reducing leakage. This should work, right? But now the officer will take money in advance from the beneficiary before the whole amount is transferred directly into your bank account. Or the cut may be moved elsewhere in the supply chain. Say at the time of entering your name of the beneficiary in the list of recipients.

Although, I've not done enough years to warrant this cynicism as compared the technocrat earlier, my optimism in technology cyclically wavers, as it should.

Often quoted by insiders and in research, preventing fraud is a cat and mouse game signifying the constant lively chase between the people trying to prevent corruption (upper level bureaucrats) and the one’s being corrupt downstream.

This description poorly paints the fight as fair. Instead of a balanced lively match, the cat is over-burdened, aging and operates with a limited toolbox in Delhi, whereas the mice are many, decentralized and many rabid.

Take any central intervention to fraud prevention initiative and you'll be surprised by the creativity and diversity at work in bypassing it. Obviously, technical interventions make it harder for someone to bypass rules, but in doing so you make the system hard for everyone involved including those who were being honest. Increasing administrative burdens mixed with overworked capacity will push even the honest ones into breaking rules. (This is something we term Counterproductive Computing in this paper, which I am hoping to do a post on later)

The problem with centralized approaches to prevent fraud is that fraud is decentralized, more agile, has better capacity and creativity. Centralized approaches also are standardized, work with limited information and often toothless.

So what do you try then?

Let's see.

So, we've seen centralized mechanisms fail against decentralized corruption. What if we centralize corruption? We can deliberately shape mechanisms such that corruption moves higher up in the chain thereby reducing the number of bad actors. This might decrease interfaces of corruption and thereby making it more amenable to standardized centralized fraud prevention mechanisms from Delhi. Arguably, by centralizing it a little, you reduce the variance in strategies employed by the corrupt as well.

The Direct Benefit Transfer example I provided earlier may be a good example of centralizing corruption to an extent. Earlier, money would disbursed in cash by the field functionaries say at the Panchayat level. With DBT, the act of disbursement is moved up further to the Block Level Officer who approves the transfer on her Management Information System. There are ~6000 Blocks and therefore Block Level Officers making this decision as opposed to ~250,000 Panchayats. I think US might be another example of centralized corruption which could be further examined.

What about decentralized mechanisms to prevent decentralized corruption?
If the fraud is decentralized, we need to decentralize the ability to catch it too. This isn't news, people favoring local government/democracy have been saying this for long. The closer accountability is to the physical manifestation of governance, the better. Village A may be facing a different kind of fraud than Village B. The villagers in those respective villages will be better equipped at preventing their kind of fraud than someone in Delhi trying to identify patterns.

Well, why aren't the villagers doing it already with much success? It isn't that extant hot favorite centralized approaches are competing with decentralized approaches. Well, maybe they are. I think centralization efforts have weakened decentralized capacities. As more and more governance shifted to MIS, two things have happened. First ironically, an information asymmetry has been created by digitization wherein information that could be accessed locally on paper is now locked within logins or public reporting labyrinths. Truly, information is being managed. Secondly, the ability to rectify mistakes or positively adapt processes locally has been removed as centralized MIS have to standardize processes and cannot incorporate nuances if they have to serve all parts of the country (Delhi Aur Developer Dur Hai)

How do you then decentralize mechanisms to prevent fraud using technology? In my experience, I've found that opening up data is the least controversial method of democratizing anti-corruption efforts. It's simple: sunlight is the best disinfectant, and there's plenty of it. One can argue, RTI did more for the cause than a million mobile apps and MIS validations. Public on the ground while closer to the site of corruption, still suffer from a power dynamic with the local state. Data or technology should be used to address this power imbalance. Opening data or freeing information has a great multiplier effect as well. Extending the cat-mouse analogy before, opening data and empowering the public is akin to have your own diverse cat army on the ground.

More broadly, any move that empowers people to hold the state accountable should be encouraged. Another example is empowering social-audits. Many argue social-audits often don’t lead to meaningful outcomes. Can technology provide more legibility here?

The elite bureaucracy often sees preventing leakages as their exclusive domain. While camps are vehemently split into the role of the state and the market in different activities, I've been coming to the conclusion that the role of keeping the state accountable is better served by the market of local pressures or more simply by the people. While, the state should be making policies and establishing frameworks of regulating itself, I believe the role of keeping the state itself accountable is a ripe case for publicisation

PS. I am glad to have completed 15 posts. If you liked this, feel free to subscribe, if you have thoughts you can reply to this email and if you know someone who’ll like this post please share 😊

I would usually call her once a month, asking her nonchalantly if she's made up her mind to transition working for the government as a software developer. I told her if she were interested there would be few opportunities which would provide her such comfortable onboarding into public sector tech. Still hesitant, she said she could start remotely and then see how it unfolds.

Overtime, I had become mindful not to push or try to convince any potential hires. It usually backfires. An ideal hire should require a very gentle push, anything more and we'll have a mismatch of expectations eventually when they join.

On one of the calls, she mentioned she won't be able to start working remotely as she had decided to help a leading dev economist do field surveys on government programs. Moreover, she had applied to CMGGA.

At that moment I decided the break my rule of gentle pushing. Not another disenchanted engineer leaving code to get into policy. We honestly have a few too many of them occupying space within the dev/public sector pretending to do specialist jobs as generalists. We need engineers to do good by doing good engineering within the government. Yes, you can be on the field reporting about how government apps are failing or you can come help the government write better code and re-engineer processes. In any case, the former is better served by people who understand social realities and policy and not engineers on guilt trips. I became more aggressive in my convincing, not wanting to see the same story unfold again, especially for someone who liked code, but was seeking a broader purpose.

Fast forward, she joined us, and a few thousand lines of code later, she wrote the citizen information module for Janmanrega app, empowering potentially crores of MGNREGA workers to fight for their rights by knowing real-time status of their workdays, payments and attendance. Simply an engineer, doing good by doing good engineering.

Submitted Please.

PS. Due to changes in my life, I promise to write more often. Subscribe if you haven’t already :)

Delhi Aur Developer Dur Ast Axiom (Delhi & Developer Are Still Far)
The greater the physical separation between the policy-maker, developer and end-user, the greater the divergence between policy objectives, IT solutioning, and ground-reality. 

Nizamuddin Auliya, Probably. 

Quite simply, if your developer or decision maker sits in New Delhi and the end-user sits in the Gram Panchayat office in District Dumka, there is a higher chance that what is being developed is cut-off from ground-reality due to difficult feedback loops in all directions. 

The week before, a high-level joint team of senior NIC and Data & Insights Unit (our team)  visited two different states to examine the complaints we had been hearing about one of our key IT systems.

While I've been avoiding travelling outside of Delhi for work (the “field”) over the last two years, this 6-day trip ended up being my most cherished work-trips. The fact I was sick the entire week after I returned and had myself googling early onset of knee issues doesn’t dampen one bit the prior week.  

The week started with us being in a make-shift conference room packed with District and Block officials in the Trivandrum office of society incharge of social audits in Kerala and ended with an immersive meeting in Pipili Block Office in Odisha where the senior-most technocrat in-charge of our government program sat next to the Data Entry Operator for more than an hour as the operator navigated the IT system and tried to complete their tasks, stopping to show the issues they were facing etc. To emphasize the beauty of this moment and personal cherish, this exact frame for which I have no photos, is one of the top highlights of my career with the Ministry of Rural Development.

After the meeting, as the rest went to the airport on their way to Delhi, Richa and I sat on the stairs of the beautiful Shanti Stupa in Dhauligiri trying to reflect upon the week we’ve had. We had booked an extra day in case we needed to do more fact-finding, but we both were satisfied with what had already been seen and discussed across the weeks. There wasn’t any need for convincing or additional facts.

While we were reflecting about the trip, we both were convinced how significant this trip was. This was important work - work that needed to be done more often. There was a reason I limited going to the field once I moved from PMGSY to the Data & Insights Unit (DIU). The DIU is set up within the IT Division of the Ministry of Rural Development, so while it can work across all schemes (DDUGKY, PMGSY, NREGA etc), it isn’t placed within any programme division (with a scheme) thereby limiting the operational change it can affect. You are away from the daily operations, so while it allows for more strategic distant work, you are cut-away from being in the thick of it and solving immediate problems.  When I was in PMGSY, if I was on the field and saw some issue related to the MIS or operations, I could simply call the head developer, or go back and attempt to fix it with the relevant team without the necessary baggage of presentations, approvals, follow-ups. I was an insider. That’s not true to the same extent for the DIU and while that is by-design, it is still to my-annoyance. The few visits I did to the ground as part of DIU, I soon realized I was uncomfortable talking to functionaries about the particular niche problem statement we could solve and not be able to do anything about the issues they had prepared already about their day to day. 

This trip was different though. The head technocrats in-charge of the IT systems were travelling with us and were explicitly here to listen to issues and find solutions. Our role was meticulous note-taking, facilitating and translating between the end-users and the technocrats such that neither side is misheard. It was an extremely positive and inspiring trip to see the conversations unfold. All of us were aligned that we were here just to listen. We weren’t thinking about how we’ll go back and convince or re-create the bugs, the decisions were being taken right there and then by the technical heads. It was beautiful. It was enough to warm up slightly a jaded heart which had decided to strategically remain in bliss ignorance. 

Technically such trips should happen more often, but the senior technocrats visiting probably couldn’t spare a week like this often as all other work that depended on them back in Delhi halted as also seen by how often their phone rang. And while the intermediate is to atleast ask senior developers, if not heads, to visit states/districts once a month to remain empathetic, our minds wandered to a dedicated role just designed for this.

Enter, the Travelling Product Manager. To me teammates, it is no secret that PM’s are my favourite roles in the government but this one is slightly different.

The Travelling Product Manager is a sub-category or niche of the PM role.

Given the Delhi Aur Developer Dur Ast Axiom reiterated below:

The greater the physical separation between the developer, policy-maker, and end-user, the greater the divergence between policy objectives, IT solutioning, and ground-reality. 

The task then of the Travelling Product Manager Problem is as follows: Given an IT product, and the distances between end-users, developers and decision makers, what is the shortest number of trips between each party, that the product manager has to do to bridge the divide between parties and get the least compromised product improvements approved. Rated NP Hard or Notoriously Problematically Hard.

Instead of explaining the whole idea, I’ll just write down the hypothetical job post below:

Title: Travelling Product Manager
Reporting: Joint Secretary (scheme in-charge) 
Office: With the technical team and developers.

Responsibilities: 

• 75% of the time will be spent to travel to various parts of the country and spend time with Users of the MIS /Apps including Data Entry Operators, Panchayat Secretaries, District and State Information Officers.

• Conduct Training Workshops at various levels explaining different modules in the MIS and their intended uses.

• Identifying MIS/App related problems faced by different functionaries on the ground not limited to lack of modules to do certain tasks but also how difficult it is to conduct certain tasks.

• For any issue identified, the TPM will on the fly create detailed Product Requirement Documents (PRD) within the existing framework of the MIS and submit them for development on returning to New Delhi.

• While e-ticketing systems exist, the TPM will target going to most lagging behind GPs/Blocks/District and trouble-shoot problems instantly either themselves or by being on call with senior developers in Delhi. While solving issues on the ground in the micro, this also helps in consolidating and framing larger issues to be solved when you return back to Delhi.

• TPM will not limit the technical issues but also bring back the operational and administrative difficulties on the ground which limit the implementation of the technical solutions being rolled out.

Minimum Qualification Requirements:

• Technical Background (has coded) or has product managed technical teams closely. 

• Understanding of Policy/Administration functioning on the ground.

• Deep empathy plus reading between the lines (of code, of policy and literal spoken languages)

• Mission Driven & Solution Oriented

• Can convert findings on the ground into Product Requirement Documents & Figma Mockups

• Fancy Powerpoints & Report Writing

Notes:

• Technical background important? Unfortunately, without a technical background it is difficult to win legitimacy from the existing technical teams and thereby limiting your ability to get action on the findings. Submitting a report is not enough, they need to write a PRD or make figma mockups to solve for the problem identified. They’ll need to sit with the technical team, test the requirements if needed and be easily accessible in case the developer has a doubt.

• Our objective here is not to identify issues but to also be part of the conversation where their solutions are discussed or even when new modules are being designed. Someone who has a technical background (web/app development) or understanding of how gov-tech architectures operate will be able to better string together isolated issues into architectural issues or be able to parlay with technical teams when presenting these findings. If you are completely non-technical, you’ll have to take whatever explanation the developers will give you when you present your findings. I’ve been on both the receiving and dealing end of “technical explanations” and I can assure you a good percentage of them are meant to diffuse the situation and not truthfully explain the situation.  

• Why sit with technical teams and not policy teams? The role is not to find flaws in the technical team’s work, but to co-fix things and reduce the information symmetry between the developers and the ground realities and policy objectives. Also by being situated with the technical teams, they’ll have a better understanding of the system architecture and better contextualise the problems they see on the ground. 

• Reiterating, TPM’s personal OKR is in getting things fixed, not pointing out issues.

• If being within the technical side of things is so important, why report to the administrative head or policy maker? There has to be a chinese wall between the person who writes the code and the person whose insights may actually increase the amount of code that will need to be written. While the TPM needs to have a foot in both the policy and technical sides, they need to lean slightly on the policy side of things as technology is in service of the policy objectives and it is important for the TPM to remain grounded in that. 

• Public Administration/Policy Experience? A lot of technical issues are administrative issues and vice-versa.  You should be able to identify that the lack of take up of an app on the ground is linked to the lower administrative caps on travel related spending currently provisioned to frontline engineers in the state budget. As you report to the administrative head of the program, such insights from the ground will be extremely valuable. 

If any donor agency is reading this, here is your cue to create a fellowship for just Travelling Product Managers in the Government - we’ve had enough of generic project management type fellowships. In an alternative life, where my duties could be limited to just this, I would take this job in a blink of an eye.

Submitted Please.

It’s 6:00 AM and Imran is ready to leave for work. Their 2 year old son isn’t awake yet. His wife has packed a steel tiffin for the day. It’s not enough for the whole day but he decides to eat on a dhaba on the way. He had joined the state engineering services about 2 years ago and holds the position of a Joint Engineer, an entry level position for overseeing public engineering works in a rural district of Bihar. He struggles to kick start his motorbike. It’s freezing cold and he doesn’t have a departmental vehicle assigned to him. The two cars allocated to the district are being utilized to chauffeur his superiors’ personal families. The local contractor had offered to buy him a new bike but he had declined the offer on principle. His bike finally breathes into life. Ready and well armed with multiple power banks, he leaves for the day. His first stop is at the local petrol pump. He fills a full tank knowing well in advance there will be no reimbursement for his bills. The administrative cap for travel allowances for his district was surpassed months ago. He opens the official whatsapp group to find the apk shared by app developers in Delhi and installs the android application. A survey needs to be done. It’s “urgent” as per his superiors, a word that has little meaning left when almost every task given to him is classified as urgent. He starts the survey and drives from one village to another. The internet is patchy but he has been informed that the mobile application works offline as well. Not informed about the intended use of the data, he makes his own assumptions about what should and shouldn’t be surveyed. Occasionally he checks in on the whatsapp group to see if someone has posted any tips on the application usage. Occasionally someone jokes about not being paid enough for this work. No wonder some of his colleagues have found other sources of illicit income. As he drives on dilapidated roads, he is careful not to drop his phone. Many of the roads which he passes are due for routine inspections as part of his regular duties. A proper inspection would require about 2-3 hours per road but instead he makes a mental note about the visual condition of the road and decides to extrapolate the same and write a desk based report later. It will be inaccurate, but that’s the best he can do given the circumstances. He did the same last month as he was occupied with Panchayat election duty. Before that the engineers were called in to help set up a district mela on behest of the local MLA. There is always something. Today he has to cover 100 villages. At around 11:00 PM he reaches home. His wife applies balm to his back to soothe a recurring ache. Their 2 year old kid is fast asleep. Imran’s wife is concerned about the stress and exhaustion. “Wasn’t the government job supposed to be easy?” The next day he reports to the office to find that a percentage of photographs weren’t uploaded to the online system because of system outage at the server end in Delhi and that he was forwarded an outdated version of the app. Others in the whatsapp group have reported similar issues. He tries contacting the helpline number provided by the software development team in Delhi but they are helpless as well. Imran’s superiors decide that it’s best for him to go to and re-survey the villages for which the data was lost. This goes on for a week till the survey work is complete. He never heard back about how the data was utilized or got access to the data he collected through the app.

Meanwhile in Delhi - the survey data from many engineers like Imran across the country are aggregated to fit a single bar graph with a 2 line caption in a government commissioned, multilateral bank funded, big-4 consultancy authored report. The report is titled “Data driven Innovation in Rural Infrastructure” which among many things concludes that state capacity is limited, corrupt and pushes for further privatization.

Disclaimer: This is a fictional write up loosely inspired by ground realities.

If you finished reading this and are still here:

This was originally posted on my blog on March 28, 2020. I was at the lag-end of the implementation of third phase of PMGSY. Central to PMGSY-III, was an algorithm I had proposed and developed which with it brought in a lot of costs and burden on the existing capacity. Central to it was the data-preparation and collection costs associated with running the algorithms and checks. Almost routinely, the core team would stop and question ourselves, whether all of this effort put by everyone is worth it? Are we really getting closer to better selection of the roads than status quo? The post above was written then as outlet of frustrations, from my own guilt but also what I’ve seen over and over again play out.

Very often, if not always, projects I’ve been involved in or pursued have been convoluted in their costs and benefits. Whenever the careful negotiation of costs and benefits is put under question, doubts creep in and motivations dropped. Very rarely has there been a project which I’ve not questioned midway its implementation whether it was really worth it or whether it did achieve the goals we planned.

The only project that I’ve encountered where I found myself doubt-free and pure in its merits and enabled dogged conviction was opening the PMGSY Rural Connectivity GIS Datasets. These datasets were for a long time only accessible to less than 10 people, and all of us were convinced without doubt how incredible these datasets were. While the idea to open it was in our heads for a long time, we were waiting for the stars to align. After more than 2 years of waiting and pursuining, the data was opened up liberally in January, 2022. One year later, the data has more than 20,000 downloads and has been deployed for usecases the dozen people with early-access could never imagine.

Technologically, data collection is intensive but opening it up is turn-key. Other than pursuing the decision at the higher levels, opening the data required little physical effort from me but when I look back I count it as one of my most pure, cherished and impactful projects in the Government. Not that it absolves us of costs inflicted on many people like Imran, nor does it help alleviate what they endure, but it does help a little in making sure their efforts are given some broader meaning because very little will ever justify the cost we technocratically inflict on our frontline.

Submitted Please.

Previous notings: https://sarkari.substack.com/archive?sort=new

Short Tenures

Bureaucrats at the helm have short tenures - time before they are transferred to a different department. Let's say you have 12 months and you want to show progress. Any complex policy reform needs time - understanding the problem statement, figuring out the solution, getting approvals, notifying and then waiting to see it get adopted. The space in which a non-technological policy solution is deployed is a complex one. If you have a short relationship, you try to pack as much as you can in easy explicit forms of affection. Fancy, I-tattoed-your-name, yes but possibly not sustainable. Slow love is difficult.

The higher up you go, ironically lesser power/control you have

As one moves up further into the hierarchy and in the central government, many claim to lose direct control to affect change as compared to when they were in the state or in the district level.

“I argue that India is today a flailing state---a nation-state in which the head, that is the elite institutions at the national (and in some states) level remain sound and functional but that this head is no longer reliably connected via nerves and sinews to its own limbs”

Pritchett, Lant. 2009. Is India a Flailing State?: Detours on the Four Lane Highway to Modernization.

While, I don't agree to the blanket statement, there is some merit to the flailing state argument not necessarily whether one part of the body is more sound or functional or delusional for that matter, but that there is definitely a long-distance relationship between different parts of the government with miscommunication and same-page problems.

Higher up, things start to abstract and aggregate. Further, with Centrally Sponsored Schemes, a central government officer's role is limited to policy, 60% funding and monitoring at a high-level. People at the center feel they have little control on the last-mile excecution. On the other hand, people in the state governments often remark that people at the center are a little cut-off from the realities of what it takes to actually implement things and not everything is as hunky-dory as it appears in air-conditioned bhawans of Delhi.

So, if you find yourself at the center, you can flag the rate of road construction is slow, but apart from writing a D.O letter or conducting a review there isn't much you can do. Apart from financial releases, there aren't many carrots or sticks to exert control. Policy too is largely enshrined in scheme guidelines and what remains under control is day to day operational policies.

So the levers for change available to you are limited, and change one must as there is no glory in maintenance of existing systems apparentely.

So within the context of having shorter tenures, feeling powerless and having limited direct control on the functionaries that execute the actual work on the ground, technical fixes are a great way to exert control. A technological fix is not only under one’s control to initiate and deploy in a short period of time, it allows me to exert control without the necessary need to write a letter or conduct a review or negotiate and as bonus it generates KPIs from day one for to measure whether I am moving in the right direction or atleast in some direction. It is in-situ exertion of power or control as an infrastructure, so sublime that it disappears in the background.

A technological solution or a fix can be designed with little context (dangerously at times), it can be deployed within weeks/months and is live almost instantly. Sure, if it is a new app or something, it will take time for people to get on-boarded but still asking people to download an ed-tech mobile app is easier than a policy reform that fixes teacher training quality or salaries. Latter is messier and has plenty human factors, former is easy. Right? Well, also wrong. This assumption that technological fixes are simpler and deterministic is one of the reasons we are seeing ill-thought half-baked technological solutions disrupting a lot of the front-line. While deploying a technological solution is easy i.e. the means, the end isn't straightforward. Eventually, you pick up the wires and follow where they lead, you'll find a human at the other end of the rainbow coloured cable puncturing the techno-utopia. Whatever you've built is going to interact with a human, most policy isn't purely implemented in 1s and 0s.

Contagion not limited to Government

Okay, so if feeling of powerlessness and urgency to show results leads to technocracy, is this phenomenon only limited to government? I think not. It seems to also be exhibited by organizations doting the streets of Munirka, Hauz Khas and Koramangala. Development consultancies, donor agencies etc too seem to be flocking to technocratic solutions to wicked problems. The powerlessness they feel is by-design, they are not the government and are here to advise. But at times they exhibit the ambitions to have more control than presenting policy briefs to every-changing bureaucrats. These organization do also feel the pressure to show impact even if they do not personally have shorter tenures. Be it donor pressures to show impact or the brief period in which they are matched with a government counterpart who is complying or entertaining them - the emphemeral window where they want to get the most done.

The last wave of transformation saw evidence collecting dev organizations pivoting to implementing/consulting orgs which are now further pivoting into becoming code shops for the government.

While, this transformation happens and move towards technical solutions happens within and in the periphery of government, the larger question remains, who is really getting their hands dirty and working on systemic solutions to the larger problems?

I don't necessarily think longer tenures will reduce technocracy, but it at least creates space and time for one to explore non-technological fixes. As for the the flailing state and the top's perceived powerlessness, I am not sure what organizational design allows for the top (center) to exert control on the bottom (front-line) wherein you are in a situationship with the middle (state) but the answer may lie somewhere in the concurrent list and in re-visiting the the role of the center and state governments.

Submitted Please.

Question seems simple but the answers really long.

So, I could tell them "government", but I don't think what they want.

I don't want to tell them more, because anything I say is wrong.

- Riz Ahmed as a consultant in Government (Listen to this for reference).

People who’ve worked in the government in any capacity are privy to these questions either as the ones asking or being asked.  Immigration and working in the government seem to have a lot more in common wherein how you attained the *entry* qualifies your allegiance and place within the social hierarchy.

Working in/for the government

The social hierarchy.

The creme de la creme are the officers (afsars). Within the officers too there is hierarchy depending on which services they are part of (administrative, forest etc) irrespective of whether they now hold the same posts.

When people ask if you are a government employee they wish to know whether you are a permanent employee or a contractual employee. Note: You are still an employee - but that's not what they are asking. Permanent employees have won their positions the hard way and they'll not let you in their networks so easily. The distinction is reminded to you very often.

The hierarchy is shown to you in interesting ways. First of all, there is @ias.nic.in and @gov.in. Then there is a @nic.in.  A couple of years ago, contractors would also get @nic.in but eventually they introduced @govcontractors.in to demarcate contractual/third-party employees. The much touted new IT platform for capacity building still doesn't accept govcontractors.in as a valid email-address for sign-up leaving out all recent Young Professionals and contractual hires.  

People who are contractual try to demarcate between themselves whether their salary comes from the government or it comes from a third-party.  Where does your salary come from may also define the incentive structures that you operate within.  Note that for the permanent ones, you all are the same so it doesn't matter.

In third-party, there are again two kinds: people who've just deputed to the government as a pass-over agreement, having lost their consulting firm's identities and those that are operating in teams setup by consulting firms and within their incentives.  Within those, there are people who operate as back-office consultants, swooping in for presentations and submitting reports. It may appear that it is a divided house, but hating on the distant consultant unites almost everyone else.

It is not that the hierarchy is maintained by people within only, interactions *outside* also go similarly.  When a group of "Young Civil Engineers" employed by PMGSY went to a bar in Connaught Place, the security guard made fun of the title "Young Civil Engineers" on their government issued employment ID cards. When meeting relatives or friends, often the conversation about working in the government is about the nature or perks of employment and not the nature of what I've been employed for? They are more interested in if I have a *white gaadi* versus what is that I really do. Once a friend really wanted to confirm whether my salary came from the Consolidated Fund of India.

Why is the hierarchy enforced so rigidly?

When someone says they work in tech, you expect high salaries and perks.

When someone says they work in an NGO, you expect social good work and low salaries

When someone says they work in the government, what's your first thought?

Stability and Stature?

In that definition, only the *afsars* fit and hence they remain coveted and on top of the pyramid.  In a country with widespread poverty and discrimination on social grounds, a government job at least promises an irrevocable almost ticket out of everything. The *sarkari* dream.  Permanent positions have reduced over time and people spend a significant part of their youth in preparation of these exams. There is a significant amount of work done before the job starts. So when it is finally achieved, it is natural to preserve its status and keep it separate from those that seem to share the same office and sometimes the same work but joined more *easily*.

Another reason I think there is a tiff between permanents and consultants is because consultants parachute into problem statements, report to officers that the permanent employees double their age don't get to and are allowed to function in an environment free of administrative baggage and accountability. Often they create a mess that is then cleaned by the *permanent* employees. At the same time, I also know departments where almost all of the _everyday_ that makes government function (letters, presentations, briefs) are purely done by consultants or where consultants have been able to successfully move the needle against inertia. If they were not there, things would be different.  A good management consultant uses their positioning to listen to existing veterans and voice their ideas packaged in palatable presentations.

Quagmires

Root of the problem I think is that government hiring has been a quagmire.  The government gets specialist talent in a myriad of ways - most of them precarious. During the time I've been here, I've been subjected to and have experimented with different methods of hiring, each with their own flaws and levels of precarity. Enough attention over the years has been spent on reforming the administrative services, but I think people need to revisit government hiring, progression and retention in positions other than the coveted ones. Eliminating the precarity in hiring contractual positions will also hopefully reduce the power dynamics and make organisations more cohesive. Now, the tension is often palpable.  Consultants/Contractual employees are either from such a privileged background that they have an appetite for precarity which also limits diversity. On the other hand, people without the privilege will always be worried about their jobs, thereby censoring the stands they can take for fear of not getting extensions.

Forced Dichotomy

Yamini's article creates a distinction too between the permanent employees and everyone else as 'consultants'. Is the state only made up of permanent employees though? What makes you eligible to be called as an employee of the state? While I do think there are forms of 'consultants' that operate in perverse incentives, but as we saw above, there is much more to it than a forced dichotomy between permanent and non-permanents. Pick any mid-size government department and you'll find people from all different kinds of hiring mechanisms sitting together and working on a supposed common goal and often doing similar work. From the naked eye, you’ll not be able to spot differences (except for chairs with towels). But look closer, you’ll find tensions, anxieties and divisions and any administrative or organizational reform that doesn't address this divide holistically will leave much off the table.

At the first government department I worked, I was lucky to have stumbled upon Mr X. He welcomed me into operations, mentored me and helped me transition into government. He was extremely passionate about the work he did and went above and beyond and get things done often at the expense of his work and displeasure of his family.

A few months into the work, I heard that he was leaving the department. I was heart broken. I thought the department would never be the same and the government program will actually suffer.

"Everyone is replaceable and things always go on" I was assured by veterans in the department.

And things did go on. The government program continued to deliver and nothing broke. While the person absolutely drove the task he was assigned at the department, when he left things continued to happen. Years later, having seen multiple churns in the government, I still don't understand this dichotomy. At the micro level, I know the person was making a difference in the everyday, at a macro-level their absence didn't leave a hole and things continued to function.

How can the absence of someone apparently important have no noticeable difference in government?

I have a few theories:

This would center the "systems" and "processes" in government and say that particular individuals don't matter to the extent imagined. You can imagine this to be by design in a system where people keep getting transferred and individual people shouldn't be wielding too much power anyway. Plus there are so many stakeholders in government (centre, state, districts, blocks, private players etc) that ability to affect change per stakeholder is limited irrespective of position in hierarchy. Governments by-design make people cogs in larger machines, replaceable and depersonalized.

But, Mr X did bring in his own flavor and style of working. Theorizing aside, I did see with me own eyes that work happened under him, that he cared, he followed up and got the processes mentioned above completed with quality. But these things aren't measurable so easily. The fact that he maintained good relations with the state counterparts, had a cordial work environment and mentored his juniors, got bureaucratic blockages fixed etc can't be measured in broad KPIs at a national level. While, these things did have an impact, they weren't measurable or probably lost in the larger KPIs (Number of Candidates Trained, Number of KMs road built etc). So when Mr X left, larger KPIs remained more or less untouched (or appeared so) and things did go on and people moved on. This also begs us to think about the incentives in government? If you aren’t moving big number or launching new things, how will you be remembered? There is no remembrance in maintaining the system and keeping the cogs well oiled. This may help us understand the appeal of launching new app, road projects, projects, policies or schemes while steering clear from brownfield maintenance.

But this conclusion that its hard for an individual to affect change in government and invisibility of important yet broader KPI agnostic work is a little disappointing. If individuals can't impact the big picture of government, I've been telling myself lies for the time I've spent in the government or to the people I've tried to hire.

I think it is both humbling and relieving. Humbling because its very important to know your place. One needn't overestimate the impact they've had individually. Yes, it takes an entire village to get anything done but also the whole system minus you is still pretty much the same system. This doesn’t mean the work you do isn’t important. No job is small, there is merit in keeping things running. Even if you are a cog, you can be a cog that cares. "Relieving" because it takes off the burden. Things have been like this for a long time, they'll continue to be like this for a long time. You can try to leave your imprint, maybe it'll work and maybe it won't. But, I shouldn't take the outcomes personally.

I would like to believe that the individual conversations and projects people have worked on in the government have had an impact. Did those changes radically change how governments operate, not at all. And I am glad, I don't think individuals should be able to affect radical changes in government so easily. Things are slow for a reason. I think the problem isn’t importance of work but rather attribution. I think the work we do and conversations we start are passed on. You build a foundation, or you take someone’s work, you build over it and you pass it on. But if you are seeking clear attribution to impact, it will be a disappointing ride. I think the objective isn’t to stand on the shoulders of giants, there are few anyway. It is less glamourous. It is to pass the baton on, from one cog to another, hoping against hope that you are progressing in the right direction.

The PowerPoint slide displayed a picture of confused but cute Labrador dog with paint all over it. The audience laughed - a much needed respite in a full-day technical workshop. Almost a hundred officials from Ministry of Rural Development - from MIS executives to the Secretary of Rural Development sat diligently through an internal workshop where Richa (our star product manager) gave a master-class on visualizations, user-centered design and enabling Programme Divisions (PDs) to build their own analytics and dashboards. For the purpose of this piece, PDs are the business units which administer policy (domain, business, bureaucrats etc).

Traditionally data analytics and dashboards have had a strong dependency on IT teams. IT has access to databases, can write code to derive analytics or write JavaScript to build charts. Personally, I've not held high opinions of dashboards, having built many and having rarely seen any used. But Richa has made me a fence-sitter again on this topic. While, I may still remain skeptical about "dashboards" within the context of public sector, I do believe analytics should be in the hand of domain experts and not IT.

Having been intimately involved in various part of the data pipeline in the government, I've seen a pattern on how "everyday" data science decisions are taken. By everyday data science, I mean:

• ad hoc policy decisions (a list of poorly performing projects are to be identified to issue warnings)

• RTI requests (many RTI requests ask for data which is not readily available publicly and need some mining )

• Parliamentary questions (same as above, but heavily concentrated when parliament is in session)

• Review PPTs

Aakash and I have a not-yet-public working paper describing "everyday" data science in government which describes in detail one instance of this practice.

Traditionally - this how an "everyday" decision is taken:

The time gap between the data request and data receipt decides whether the decision will be data-driven or data-backed. The PDs aren’t the the only victims here. The technical teams can’t focus on development activities and have to context switch to address these often urgent and one-time data requests. This in context of already being overburdened, under-capacity and struggling to meet routine development timelines.

This is how a self-service model would look in comparison:

This definition from Gartner (sue me) on self-service analytics is quite exhaustive:

Self-Service Analytics is a form of business intelligence (BI) in which line-of-business professionals are enabled and encouraged to perform queries and generate reports on their own, with nominal IT support. Self-service analytics is often characterized by simple-to-use BI tools with basic analytic capabilities and an underlying data model that has been simplified or scaled down for ease of understanding and straightforward data.

With equal amounts of astonishment and begrudging, I have to admit that this executive gibberish actually makes sense and is completely on point.

Last year, I was pushing for hiring dedicated data analysts into programme divisions. But now from the vantage point of my current team, I understand that the blocker isn't specialized skillsets but instead agile data access. I was overestimating the need of expertise to do ad-hoc analysis, probably coloured by my own journey within the Government. But most everyday data science tasks within the government require just about adequate tooling i.e. Excel and PowerPoint. Even fancier dashboards can be developed without writing any code with PowerBI or Tableau etc. So the larger issue is that of agile access to data. Programme Divisions though cannot and should not be given access to databases in their raw form. Database generally evolve over time and carry their quirks. We need simplified versions of databases that can be shared with Program Divisions who in-turn can access the data either through BI or SQL or Python. These simplified versions of databases are common in digital private sector firms either by the name of data models/warehouses/marts.

Some challenges to this:

1. Designing and creating simplified data models is an art and continuous maintenance; maintenance is a hard problem to solve in public sector be in digital or physical goods.

2. There will be a lot of initial confusion with what's ITs role and what's PDs - note we aren't eliminating IT's role from data reporting; long-form reports and ETL still is with IT. Only adhoc analysis is being moved to PD.

3. Culture change: Boils down how much interest PD wants to take - while data access becomes faster, it does bring additional responsibilities. You can no longer shirk data-driven decision making because of data accessibility issues.

Creating these data models also solves the problem of single-source of truth. Most government MIS end up with reports that are built over time and sometimes have the same columns displaying differing values. For eg. total funds disbursed to contractors may have two different values depending on the report opened and the developer who built it. This happens because for each report the business logic is re-written and the same definition isn't honored.

Ministry of Rural Development has embarked on the journey of moving towards self-serving analytics across all its schemes. Very ambitious and it is definitely going to take a while but nonetheless a step in the right direction. I am optimistic about what self-service paradigm can achieve. It's not some lofty futuristic goal. While I was preparing the presentation for the workshop, I realized I've already been part of government program which unknowingly and by happen-stance operated in a radical version of self-service in 2015-16 but that's a story for later.

Submitted please.

It is understandable. You join fresh with ideals, armed with degrees and here you are doing routine file-work. It is frustrating.

At first, it is a problem of expectation setting. There isn't a single fix or magic trick that's going to fix things which have been wrong for decades and globally. Rather, governments or rather other big organizations are kept running by showing up and doing the routine and mechanical. Sometimes you are being your most useful when you are doing small-jobs. So one approach is to suck it up and just do it. The routine may not be your idea of change, but much good comes out of just keeping the systems running.

Many juniors are tasked with writing minutes of meetings. It is not an enviable task. But if I care about a project, I go out of my way to ensure I get to write the minutes. A task which many are happy to offload. Writing actionable and precise minutes is an art. Granted, I've not seen people referring to minutes in consecutive meetings unless there is a conflict. But rest assured, conflicts are many. Governments don't use JIRA to manage software requirements or other task management tools for actionables. In that context, minutes are powerful levers to exploit.

PowerPoint presentations too are routine work and generally juniors get tasked with first drafts and managing revisions. These are generally review presentations. Deciding how and which information will be presented, KPIs, narrative are powerful policy/data levers. Granted, you don't get a complete control on the presentation, but you do get a say. Some presentation templates or KPIs which I initiated in my first department are still being used years later and have become for now, systemic.

The routine is not only can be the site of change itself, but it also can open meaningful and grounded pathways to larger changes. I got most of my cherished wins at work because of doing such 'small' work. The work closest to my heart was kick-started because I happened to be the fastest typist in the room.

In 2018, I was hired at PMGSY to explore the use of GIS algorithms in selection of rural road investments. They had almost completed the digitization of the rural road network & habitations on GIS and the new leadership wanted to find a meaningful use of it for a new Rs 80,000 Cr scheme under ideation. My typing speed made me the ideal candidate to type and maintain changes in the overall policy guidelines as seniors/committees discussed and deliberated. Having such intimate access to the process allowed me to understand the whole picture and also chip in changes in the overall policy which impacted the planning process and ensure my algorithmic intervention was within context and backed by policy.

A strong part of data/policy work is to scope out problem statements and rarely people vocalize the root causes explicitly. As someone who has come from outside of the system and probably on a small tenure, catching up to the context is really important. There is much to be unlearned and learned about the government. These "small jobs" allow juniors to be in rooms they wouldn't be in generally, learn and absorb information. The key is to read between the lines and listening to murmurs. This makes the “routine” an important way of initation.

Routine doesn’t always open pathways though. I have a core domain expertise (tech, data science) which lets me step away from the *operational* when I wish to because few in this office would be able to do my core work. But I know colleagues who've been overtime spending most of their time in such work and remain unsatisfied or feel underutilized.

There lies one of the problems with hiring young people in the government: retention and job satisfaction. While, NITI might have figured out the branding and gets many people to apply, I know first-hand many people who were hired but now feel under-utilized and frustrated. While, there is an expectation setting problem, the government also needs to setup pathways for young people to move higher up or get responsibilities matching their skillsets. Why listen to an external management consultant with minimal context of your exact problem when you can ask help from your own intiated young professionals who’ve done the work and may have ideas. This needs to be fixed soon, otherwise government will remain just a launch pad for careers in think tanks, management consulting or grad schools.

Submitted please.

When I had joined the government back in 2016, the then COO of DDU-GKY had a very strict policy for newcomers: Stay quiet till you’ve actually visited the field. Obvious right? How can you make solutions or policy without ever having visited the first mile where it gets operationalized? Still, the field remains a romantic idea and still not routine for people who work at headquarters. I don’t talk about my meetings with the bureaucrats in Delhi with the same tone as my visits to the field, and therein lies the problem. With our team at the Ministry, we’ve tried to carry the same philopsophy as my COO and have made the first mile insights a priority. We also have a dedicated HCI Researcher to bring in deeper qualitative insights and critical inquiry into the tech work done by the team.

The following post is Dibyendu’s reflection from some of his visits to the first mile, written from the lens of a data scientist:

A few months into my job as a data scientist in the government, I found myself seated near the overburdened desk of a clerk in Rajasthan who was responsible for the management of the MIS system of an entire district. While on a tight deadline for the data entry and report making for a government scheme, coordinating with frontline officers on the phone and also prepping for a sudden visit from the chief minister, they were trying with their patchy internet to take us through their day to day on the MIS system in an effort to help us with our scoping work. Needless to say, we were also amongst the many ‘tasks’ that they had to attend to that day. While the exercise was helping us scope our project and collect qualitative insights, it was also revealing and ‘making real’ the complexities and ambiguities behind producing a single datapoint on my screen. There was hard labour, intricate negotiations, competing values and perspectives involved in the journey of what would become a neat little cell on my dataframe. Having gone through scheme guidelines, circulars and been in spaces discussing and working on human computer interaction, I had still become a disconnected actor. Data, related metrics, and code have this very ability to disguise these convoluted realities in the form of spick and span ‘data realities’. This not only argues for an introspection of a data scientist’s agency and responsibilities in the entire system but also modifications in the data science life cycle itself. 

A few months later and for a different government program, on a rather long visit to government offices at various administrative levels and self help groups in  Jharkhand, the extensive and arduous journey of the implementation of a scheme came into light (something that can also remain hidden from a data scientist). What seemed to take a few kBs of my laptop’s RAM, took up thousands of square kilometres of land in reality  encompassing lakhs of people. Solutions had now become objects being used by real people with their own complex positioning and objectives. In separating the process into tidy stages, the data left out some very important points of concern. Could my solution space then be sufficient by being blind to these complexities or without taking into account each actor’s needs ? Would my job as a data scientist be complete by just trying to improve my ML model’s accuracy or helping accomplish a certain metric? Have I not been given immense power by getting to decide on these? Sitting in a remote underserved adivasi village, it was becoming increasingly difficult to imagine the sparkling tech solution proposed by a consultant that had caught everyone’s fancy. Can one really design for someone without interacting with them and getting to know their lived experiences? 

As has been mentioned in previous notings by Harsh, data scientists/engineers are given a large and shiny seat on the table. They are often at the centrestage of formulating problems and deciding on them. Their inherent ideologies and biases frame solutions and thus the existence of  a ‘neutral and apolitical’ data scientist is a fallacy. Infact, this very stance is quite a conservative one. Their decisions could lead to beneficiaries being unfairly denied resources.  One could argue that a lot of these could be taken care of by making the data scientists talk to the product managers and researchers on the team but without recognising their political position, empathising with the beneficiaries, and being thoughtful partners in the process, a lot of their inherent biases would creep up into the system which would be hard to keep checks and balances for. The life cycle of a data science project can’t hence end at iterating through scoping, collecting data, training a model and deploying it. It must incorporate the process of designing with (in opposition to designing for) the actors and beneficiaries and adapting to their needs at every step, sometimes even scraping the entire project if deemed necessary. 

P.S: Everything being said, data scientists should always tag along field visits and not lead them. The qualitative/policy researchers are better trained to do them. The post only encourages thoughtful collaboration and does not endorse taking up more space as the field has been weirdly doing for some time now. 

Submitted Please.

Usha wakes up at 5:00 AM. Her world and one at large is still asleep. She reminds herself that she chose this life. She starts making breakfast for the family. Poha should do? She has been hesitant since her mother in law taunted her about the lack of love in her food. She wonders what other than love would have her making Poha at 5:00 AM. The crackle of the mustard seed distracts her from answering her own question. She doesn’t want to know. There’s breakfast, lunch, and kid's school work. Miles to go while everyone sleeps. 

She reaches the office by 9:00 AM. She’s often the first one in her sarkari office and the only one on time. Her male counterparts slowly pop-in. She isn’t pals with Shukla who marks the attendance. She’s had a hard time socialising with the men at work. She doesn’t smoke or eat paan either. You never know what people will think? Surely, Patel will think more than two things. She likes to keep distance from Shukla. She has heard things too.

The staff is talking about the branch closing. Liberalisation people say. But her job has been liberating, but this isn’t about her. When is anything ever? Almost always as per her husband.  She likes her job. Account Officer for the state telecom department. Any other job and her family wouldn’t have let her work. 

The smell of samosas comes from the left. Bhatia is having his breakfast. An innocent looking man patiently sits opposite and waits for Bhatia to attend to his query. He seems like a familiar face, maybe comes here frequently? Then again, over time the faces of almost all people visiting the government office have mashed into one: powerless eyes and nervous postures. She used to feel bad for them before, not anymore. 

She knows the drill, Bhatia will redirect the person to one of them and they’ll redirect further. The existence of the office depended on a fine balance of action and inaction: constant moving around of people and hoarding of files in one place for as long as possible. Yin and Yang. They don’t know what happens if this balance changes. The seniors have theories. A world put together by two games: hot potatoes of people and treasure hunts for files. She did feel a little bad for the poor chap, poor potato.

Was she always like this? Was it always like this? She thinks of the time when she had just joined; fresh out of college with a CA degree and brimming with idealism. Happy to be able to work. She would be on time, process her files quickly and relate to the people who sat across her desk. They were the same as her, often from the same neighbourhoods? When did they become different? When did she become better than them?

She remembers a person similar to the chap at Bhatia’s desk who wanted to get a new telephone connection. She’ll never forget the name: Musaddilal. It was in the first month of her joining the department. She was quick to process his file when it came to her but she had no control on what happened at the other desks. She got invested in this person. She had pestered Bhatia to move his file quickly. There were legit reasons for slowness. Lack of staff, changing bosses, pending files, bloated paperwork, VIP requests, computers not working etc. Not everything was in her control. But she felt personally responsible for Musaddilal every time he would come to office. In the office, he became Usha’s Musaddi.  So much power, but also so little. Mussadilal got a telephone after 6 months, but she decided to grow a thick skin there-on. Be helpful but not get too invested. Overtime, she forgot to care as well. She started taking bribes too. Everyone was doing it. She didn’t want more attention. She didn’t want to stand out. 

The poor chap across Bhatia is now patiently waiting for her across her desk. She is cutting vegetables on her desk. She has a house to run as well. It is part of the deal so she can continue working for the government. The poor chap has a complaint about a wrongly calculated phone bill. Usha knows what went wrong - a simple printing mistake, easy to fix. But she has vegetables to cut and many pending files to process already. She acts as if she doesn’t know what’s wrong with the bill. She redirects him to Shukla. She buys time. She is always buying time, but she no longer remembers what for? Does she do anything at all? This reminds her of her mother-in-law. She gets back to kaddu in front of her: chop-chop.

A simple example, our team generated PDF reports for printing for Block level functionaries to complete a task on the ground. It had recommendations for each edge cases listed row-wise. One of the recommendations was "Form a new VO". Our product manager reported back from the field stating the functionary was struggling to read the word "form" and thought it meant "from". That would change the meaning of the recommendation. While the visit by the team was to get feedback on the format and its columns etc, we missed the basic fact that the whole of it was in English. There are other examples where "Landless" was interpreted as "Landline" leading to Kafkaesque results which I'll save for a later date.

Most central government websites are eventually available in Hindi or English. Some attempt to cater to more languages but generally limit their exercises to static public interfaces. If you go to COWIN and select Oriya, you'll find the resulting page is a mix of English and Oriya.

Often departments create word to word mapping dictionaries when they wish to make their content available in other languages. These mappings are often cumbersome one-time exercises and don't cater to new content or dynamic reports. The process involves creating a spreadsheet of word to word mappings, sending it to the states for vetting and if we get a reply then uploading that on our website. This is a broken system.

You can't use google translate either as it often breaks formatting but more importantly it doesn't understand when to translate vs transliterate.

Let's take an example of a common MIS report which has the name of beneficiaries, their home districts and the status of their entitlements (With Block for Approval, Delivered, Not Submitted). Here the District and Beneficiary name columns would need to be transliterated but the Status column would need to be translated. You don't want beneficiary name "Harsh" to be translated into "कठोर" (ie. rude/mean) but remain has "हर्ष ". At the same time, you want "Not Submitted" to become "प्रस्तुत नही किया है".

While people have figured out language models which accurately translate and transliterate separately (https://github.com/libindic), it still doesn't solve our problem entirely.

Plus, many IT systems are legacy. You can do a word to word mapping for the public static interfaces, but the workflows behind logins are a house of cards. Plus, backend systems vary across schemes. A solution for SQL Server might not work for websites built on Spring or Angular.

A possible solution is an open-source browser plugin specifically trained to operate on sarkari MIS vocabulary and idiosyncrasies. It will be client side so it won't impact our legacy systems. And narrowing the use case to government vocabulary makes it an easy problem to solve. A mix of unsupervised and rule based logics could solve the problem. Some custom rules on identifying column types in dynamic sarkari reports, hard coded cases for translation/transliterations on common government vocab and leaving the rest to the algorithm's best guess.

This would truly be a digital public good. But do solutions with no eventual pathways for market profits count anymore? :P

Submitted please.

PS. While the first noting on substack, I am continuing the numbering from the blog. We all hate it when people start new sarkari files without back referencing.

Previous notes are available at my blog and you can check my tweets for more #sarkari micro-musings