Spring / Summer 2024

The City of Pittsburgh’s PGH Lab

Program Evaluation

The City of Pittsburgh wants to evaluate the current state of PGH Lab and create concrete steps to improve the program’s outcomes

Role
Researcher

Time Frame
8 weeks

Team
Marlaina Kaine, Civic Innovation Designer

Introduction

PGH Lab is an innovation pilot program run by the City of Pittsburgh, created to connect local startups with city government. The program pays companies $25,000 to test their innovative products and services over a 6 month period, addressing civic challenges like sustainability, public safety, and energy efficiency. It was launched as part of Pittsburgh's efforts to foster community-driven innovation of city services.

Why evaluate?

In 2024, after 49 companies had participated across 9 cohorts, only one had secured a long-term contract with a city department. No other piloted solutions were implemented.

As a new hire and co-program manager of PGH Lab, I was tasked with conducting a structured evaluation to understand why the program was falling short of its goal to foster long-term innovation.

Research Methods

Participant Interviews

I spoke with 7 entrepreneurs who had participated in the program across 4 different cohorts
Documentation Review

I reviewed all documentation that had been used to plan the most recent cohort
Stakeholder Interviews

I spoke with 6 internal stakeholders about their experience overseeing or participating in the program

“The startups should’ve been in the room for the feasibility discussions. So we could say, ‘Hey, here's what we can do and here’s what we can’t do’”

— Participant Entrepreneur from Cohort 6

Participant Quotes

“The problem is that for our [product], you can't sit down in it. And their workers would hop out of the back of the truck, go down the street, and they'd jump back in the truck. And so the feasibility of like putting the [product] on and taking it off, really wouldn't work for them.”

— Participant Entrepreneur on why his pilot was unsuccessful

“We had pretty strong signals in advance that [our product] was a good fit because the department had already made public announcements around needing sustainability in procurement”

— Participant Entrepreneur on why her pilot resulted in a contract

So, what’s going on?

PGH Lab’s current model is pitch-based — startups present their solutions based on their understanding of city needs. This model was originally chosen to introduce unique, community-driven solutions to city problems.

The review committee then evaluates the pitches and pairs the startup with a department for a pilot. The issue is that this process lacks alignment — startups don’t have insight into departmental challenges, and the city review committee doesn’t fully understand the startups' capabilities.

As a result, pilots are based on assumptions, failing to address the department's most urgent needs, which leads to few long-term contracts.

Recommendations

Adopt a Problem-Focused Approach

Shift the program’s focus towards addressing specific challenges faced by city departments. Departments should identify issues and communicate these to the Civic Innovation team. The team will then conduct design sprints to understand department problems and identify the solution space.

Publicize Identified Problems

Once issues and potential avenues for solutions are identified, the city should publicize this information or directly contact businesses that could offer relevant solutions. This targeted approach will attract companies with products that have a higher chance of fitting departmental needs.

Assess Feasibility Early

Many pilots have failed due to practical challenges that could have been discovered with an in-depth conversation prior to the start of the pilot. The Civic Innovation team should engage departments and startups early in discussions about feasibility to prevent mismatches and increase pilot effectiveness.

Future of the Program

Based on these findings, the program is undergoing a redesign. The 2025-2026 cohort will adopt a problem-focused approach, with members of the Civic Innovation team studying human-centered design methods to guide this shift. We are also re-evaluating our processes and communication strategies to better align startups with city needs and ensure more effective outcomes.