Managing for Results in Regulatory Initiatives – Dec. 5, 2002
This summary reflects highlights from the December 5th PMN
workshop entitled Managing For Results in Regulatory Initiatives. For more information on this event, or to
join our emerging community of practice in this area, please email communities@pmn.net and note ‘Regulatory’
in the subject heading.
Why participants attended
§
Tying practical application with strategic planning
§
Want to learn about regulatory p.m. and MFR
§
Strategies for p.m.
§
To learn about what others are doing
§
To bring p.m. to our project and to share the
knowledge with our 13 other ministries
§
Comprehensive measurement
§
Want to be more proactive in implementing p.m. into
our programs
§
Scoping p.m.
§
Causal links
§
“smart regulations”
§
Environmental assessment
§
Integration
§
To share and learn
§
Regulatory attribution
§
Behaviour change
§
Risk modeling for inspection planning
§
P.m. versus risk
§
Multi stakeholder collaboration
§
To determine if p.m. can help to bring together a
heavily regulated industry into soft regulations
§
Looking for helpful communication tools in MFR and
risk management
The group was made up of
individuals representing
§
Smart regulations
§
Environmental assessment
§
First year working in the government
§
Strategic projects and planning
§
P.m. strategies
§
Program managers
§
Evaluators
§
Inspections
§
Operations
§
P.m. officers
§
P.m. managers
§
Enforcement
§
Compliance (infrastructure
on anti-terrorism)
§
Working with heavily regulated organizations
Smart Regulations
§
is results-based and not process-based
§
looks are regulatory burden
§
1 stop shopping for regulators and inspectors
§
strategy being developed by external advisory
committee
§
is about streamlining processes
§
coming out of the “red tape” commission in Ontario
Government
Q. Has there been any work
on how to have meaningful attribution?
A.
§
According to Sparrow, you can’t ‘roll-up’ the big
picture.
§
Need to make sure you collect the right data,
appreciate timelines, focus on specific gaps or problems and have a narrow
scope of what you’re measuring
§
You need to believe that attribution is
possible. Attribution must be implicit
in MFR
§
Need to continually scan the environment
§
Need to remember that at times we need to change our
focus
§
Need to be flexible!
Steve Montague : Managing for Results in Regulatory Initiatives
Key highlights:
§
There are two analytical frameworks that can assist
us to better understand MFR in regulatory initiatives. These are:
§
Type of performance orientation
§
Level of implementation
Key lessons learned:
§
We need to build better communities of practice
§
We are ultimately working towards a goal of positive
behaviour change
§
A problem-solving approach can help to create a
practical results focus
§
The use of a logic model or chain of results has
been helpful for groups to focus on appropriate outcomes
§
We should measure only the important indicators
§
Patience and persistence pay off
Lise Davey: Industry
Key highlights:
§
“Smart Reg” compels us to talk to others
§
It’s impossible to measure everything
§
We need to take things one step at a time
§
Marketplace and Competition Bureau have used results
logic at project and program levels
§
Marketplace has also found the Reach vs. Resources
and Results Step Model and the Conformity Continuum useful
Steve
Adams: Learning How to Learn
Key background:
Mr. Adams has been working
in performance measurement for approximately 6 years. He looks at eco-system management (from a
holistic approach). He wears two hats – he is an analyst for performance
measurement in
§
§
Three to four years into the merger, the Agency
started to look at performance measurement (especially in terms of resources
and enforcement actions). Our resources
weren’t equal to our enforcement costs, so we needed to reevaluate our
priorities. We wanted to move beyond looking at activities. Now our activities are geared towards
behaviour change which interact with specific resources.
§
We use Sparrow’s “problem-solving” approach in many
facets of what we do
Slide 4:
§
In developing our performance measurement strategy,
the Secretary communicated that she wanted our activities to be
actionable. So, we used staff analysis
with indicators of her concern. From
this, the categories of ‘good’, ‘watch’ and ‘focus’ were developed.
Key lessons:
§
Organizational change hasn’t been supported as well
as it could have been
§
We needed to really define our core business
processes
§
Need to be able to link measures back to their
sources
§
Information systems need to be firmly rooted in
performance measurement (if it has been in our case, we would have saved a lot
of resources – including $ and time)
§
Our next steps:
§
Our logic model needs to be made more explicit
§
Need to manage information as a corporate resource
Laura Pasquale:
Key highlights:
§
When thinking about pm, we need to simplify the
system but remain meaningful
§
To best develop a meaningful performance measurement
system, you need to holistically look at what your system really needs and
effectively determine that you’ll be able to properly manage the whole process
(i.e. IT, performance measurement, resources).
Steve Adams: US Environmental Protection Agency Environmental Indicators Initiative
Key highlights:
§
This program was initiated by Governor Christie
Todd-Whittman. She made it clear that
the EPA could use these indicators to increase transparency.
§
The Initiative builds from draft indicators through
a dialogue with key partners (e.g., government, NGOs, international bodies,
stakeholders, etc.). Our State Agency
partners act great sounding boards.
§
The overall report is approximately 100 pages (and
accompanied by a hefty archival report which will probably be disseminated in a
CD-ROM). The document will remain in
‘draft’ form.
§
We will be seeking the feedback from our key partners
from around to country and will be asking them to evaluate it based on it’s
relevance and the degree to which the indicators are sufficient
§
In
§
The pyramid demonstrates the different strategies
used in the Chesapeake Bay Program
Q. How do you move from
measuring compliance to moving to environmental effects?
A. Laura Pasquale
§
The logic model is a good vehicle to demonstrate
this evolution
§
You should look at emerging themes and what’s going
on around you
§
Need to understand the cause-and-effect
§
Based on what you’re observing, re-evaluate your
activities
See: Harry Hatry – Urban Institute
Q. How do you know exactly
what to measure? When working in compliance and
you enforce, how do you measure your results after the fact?
A. Laura Pasquale
As noted in the
§
Instead of looking for a full regulatory solution,
they went with an information approach
§
They sent out a book, and the dry cleaners had to
sign a form to confirm how they would comply
§
This approach had a significant effect and provided
positive results
§
The main lesson here is that you need to think about
what you’re doing before you go ahead with it because you don’t want to waste
precious resources
§
The ‘Mindset’ is important
Q. How do you deal with
IM/IT struggles in terms of pm? Do you
have any strategies?
A. Edwin Levine
§
The problem is when you have too many systems and
too many discreet definitions that are looking at the same things but have
competing infrastructures
§
The system needs to be appropriately linked to your
needs
§
If we’re going to learn, we have to think about how
we’ll link our pm systems to our resources
§
Need to have senior buy-in
§
Need to be able to deal with the ‘human element’ of
this (i.e. competing interests for what is measured / monitored)
§
Need to make sure you have people who are regularly
reviewing the data (is it accurate? Are we collecting the right information?)
Q. Could you tell us about how you have been integrating the Balanced Scorecard approach into your pm activities?
A. Edwin Levine
§
We looked at a lot of pm programs and the Balanced
Scorecard approach seemed to make sense for our support functions
§
Many of our branches and business lines now use the
Balanced Scorecard approach
§
We use it to frame our IM / IT strategies
§
It is appropriate for our ‘customer’ focussed
approach (though stakeholder is a much better word than customer)
§
We need to be mindful that what we do directly
affects our stakeholders
§
You really need to know your target audience
Q. How do you handle good
indicators that demonstrate bad results?
A. Steve Adams
§
Some indicators are better tailored for ‘internal’
reporting purposes
Sharon MacDonald: Ensuring
Compliance Today and Tomorrow
Key highlights:
§
People relate to talking about issues and
discussions, not developing logic models
§
You know when compliance will be an issue if your
auditors aren’t auditing
§
Box and wire logic models aren’t as strong as
non-linear models because its all about how it fits together, not what you can
sign off on
§
We need to shift from the ‘gotcha’ mindset to
exhibited behaviour investigations
§
A situation or Needs Assessment vs. A Chart of
Results can help tell the story (horizontal logic)
Small Group Work
Each group was asked to develop a results performance framework using
the spheres of influence (tier model).
The groups worked on the following cases:
§
Competition Bureau Information Centre
§
Office of Boating Safety
§
Dying from falling off ladders
§
Environmental assessment of regulations of
substances found in agricultural products
Key Lessons from the Workshop
§
Learning focus
§
Logic-based structure
§
Problem-risk orientation
§
Pilots build incrementally but persistently
§
Involvement of ‘Doers’
§
Collaboration IS a legitimate outcome and a
key part of the performance story!
Note: We
are attempting to build a community of practice in this area. Please feel free to e-mail us with questions, concerns and
contributions. (communities@pmn.net, www.pmn.net)