OpenGov Voices: Investing in open data and analytics in San José

Kevin Miller, senior executive analyst for data analytics in the city of San José, Calif.

For years, Silicon Valley has led the nation and the world in tech innovation. Yet, the city of San José — the Capital of Silicon Valley and the nation’s 10th-largest city — has not been at the forefront of civic technology, data analytics and open data. This is due to significant fiscal constraints over the last decade; most notably, the number of city employees now roughly equals levels in the mid-1990s, even though San José’s population has increased by 25 percent. Though the city transformed many of its operations to more efficiently deploy remaining resources and employees, reductions were still required in core public services, such as public safety, neighborhood services and infrastructure maintenance. Investments in centralized analytics and open data across the organization were limited until 2015, when the city created the Data Analytics Team.

That is not to say that city data went unused before 2015. City staff have used data in a variety of interesting ways to improve service delivery, such as optimizing the scheduling of sewer main cleanings to prevent overflows. What was missing, however, was an organization wide strategy to operationalize insights from data analytics and scaling successful departmental data initiatives to the citywide level. This new effort will build upon the good work done in departments and increase our ability to implement data-driven decision-making.

When I joined the city in May of 2015 to lead the Data Analytics Team, I wasn’t thinking much about open data or its relationship to a successful, centralized analytics shop. This began to change in the fall of 2015 when the City Manager’s Office and the Information Technology Department began drafting an open data policy, which became a catalyst for broader conversations about how to engage the community around this issue and leverage open data as a resource. Because the Data Analytics Team was already conducting a citywide data inventory initiative as part of our initial work plan, and given that we relish in all things data, the responsibility of guiding San José’s Open Data Initiative forward fell to us by default – and it stuck.

The skyline of San Jose. (Photo credit: The city of San Jose)

The more entrenched my team has become in the city’s open data endeavor, the more we have come to understand its importance and relevance for our analytics work and its benefits for city staff and the community. While its promises of enhanced transparency and accountability and its potential for creating economic opportunity are often touted, we hear less about benefits to internal operations and public policy formulation. Encouraging and institutionalizing the ability to conduct more complex data analysis across an array of departments will allow for more informed decision-making from both city staff and elected officials.

We have also observed that our efforts around open data and analytics have worked symbiotically to facilitate change management in our organization. The more we have data-related conversations and the more we press upon our colleagues the importance and value of using data in new ways, the more progress we make. Having such conversations in the contexts of open data and analytics has helped reinforce a shared, multi dimensional understanding of the value of data. Here in San José, most of these conversations started around the data inventory initiative, which people widely interpreted as a move toward opening up our data. As we’ve progressed, the conversations around analytics and project scoping have become more streamlined. As we move to focus on open data — our policy passed City Council on April 5, publishing information through an open portal by default — we are seeing fewer instances of pushback and growing support.

Much of this has to do with a key factor for success in this field: relationship building. Acquiring and opening departmental data and moving toward a deeper level of analytics and data enthusiasm requires trust. When opportunities arise to collaborate with departmental staff to help them progress toward a goal, or when we have chances to act as their advocate, we almost always take them. This has helped us cultivate key relationships, which will make our jobs easier down the line. The importance of this and the return on the time investment required to cultivate relationships should not be underestimated.

While the city struggled — and still does to a lesser degree — to devote enough resources to key public services, this is actually a strong argument for investment in open data and analytics. Core principles of the open data movement, including enhanced transparency, accountability, and economic opportunity and innovation, are increasingly seen as strategic investments aimed at maximizing the return on taxpayer dollars and encouraging informed public policy and citizen engagement. Consider the case of San José: Residents demand investments in infrastructure and public safety, but the city has an inherently limited pool of resources from which to draw. Open data helps us communicate where our investments are being made and what the impacts of those investments are. Similarly, analytics help optimize the allocation of limited resources and maximize their impact. In limited resource scenarios, these are investments worth making.

San José is making these investments. Analytic capacity is improving citywide, and our Open Data Initiative is moving forward rapidly. But we aren’t ready to check the box. The key question is: How do we create an imperative in the community to engage with open data? Much of the local community here is enthusiastic about open data, and in the heart of Silicon Valley, tech talent and civic entrepreneurship are abundant. This makes for a powerful opportunity, and one which San José will seek to fully leverage and cultivate.

Interested in writing a guest blog for Sunlight? Email us at

Source : ADEC - Open data