DESCRIPTION OF LHD: The Boston Public Health Commission (BPHC) is the governmental public health agency for the City of Boston. BPHC provides a range of critical services, including Boston's main ambulance service, substance abuse treatment facilities and some of New England's largest homeless shelters. Boston is a majority-minority city, where over 50% of Boston's 600,000 residents do not identify as White. Boston's Black and Latino residents experience higher rates of preterm births, asthma-related emergency department visits, heart disease and other poorer health outcomes than their White counterparts. BPHC over 40 programs have been in the front line of providing public health services and working with partners to address the numerous health disparities experienced by the city's residents. While there is a lot of evidence on the value of using quality improvement principles to improve essential public health services and address health inequities, the practice was not known to the organization's over 1,100 staff members until the QI program was created. In 2013, BPHC completed an accreditation self-assessment which revealed an average of 70% compliance in all the eleven domains except domain nine (quality improvement) which was 8% compliance with the standards and measures. In 2015, BPHC established the Office of Accreditation and Quality Improvement (AQI Office) to create a formal organization wide Quality Improvement Program to with an aim of improving the organization's culture of improvement and help staff understand how QI could be used to improve public health services and health equity. GOALS AND OBJECTIVES: In August 2015, BPHC developed its first ever Quality Improvement (QI) Program. The program's plan was implemented by a newly created Office of Accreditation and Quality Improvement and supported by an organization wide Quality Improvement committee. This overall goal was to create an organization where every staff member contributed to the continuous improvement in their work resulting in improved public health services and advancing health equity. The program had two main objectives: (1) Develop a sustainable culture of learning and continuous quality improvement practice to advance health equity (2) serve as a model for other agencies on quality improvement. HOW WAS THE PRACTICE IMPLEMENTED? The program involved (a) delivery of numerous QI training session (into to QI, basic QI training, QI Toolbox Series), (b) coaching for teams conducting QI projects, (c) hosting QI committee meetings and (d) ongoing feedback and internal communication on process and outcomes. RESULTS AND OUTCOMES: The results from Objective 1 - Develop a sustainable culture of learning and continuous quality improvement practice to advance health equity: 1) 75% of staff surveyed agreed the culture of QI has improved in the last year” (2) 12-percentage point increase (positive direction) in from the proportion of staff who agreed with the same statement in two years (3) BPHC received PHAB accreditation and the Quality Improvement Program was considered an area of excellence (4) 77% of staff (847 people) received QI training (5) Twenty-two programs (more than 50%) in all of the organization bureaus completed at least one formal QI project in the last two years. The results from Objective 2 - serve as a model for other agencies on performance management and quality improvement: (1) One of the program's materials was published in the NACCHO Compendium of tools and resources in 2016. (2) The program staff was selected to speak at nine national conferences in the last two years (3) The program staff showcased the programs work through 10 poster presentations at national conferences (4) The program staff submitted numerous publications of completed quality improvement projects and a paper in an international journal Perspectives in Public Health”. (5) The program staff has fielded requests and shares its tools and resources with twenty-nine (29) health departments across eighteen (18) states. (6) In September 2018, the program received a 2018 Health and Human Services Service Excellence Award from the Office of the Mayor of Boston, Marty J Walsh. WHAT FACTORS LEAD TO SUCCESS? Factor 1: Strong support from the AQI Committee (BPHC's QI champions) and the executive leadership. Factor 2: Investing time in New trainings and coaching in QI. Factor 3: The program's ability to be creative and embed equity considerations into QI practices (4) Strong organizational commitment to Health Equity. PUBLIC HEALTH IMPACT: Quality Improvement program has supported improvements in numerous public health essential services program including, but not limited to: Improving referrals to a home visiting program, reducing client grievances at the women's homeless shelter, improving community engagements with the organization social media channels, improving the HIV delivery among partner organizations.
DESCRIPTION OF LHD: The Boston Public Health Commission (BPHC) is the governmental public health agency for the City of Boston. BPHC provides a range of critical services, including Boston's main ambulance service, substance abuse treatment facilities and some of New England's largest homeless shelters. Boston is a majority-minority city, where over 50% of Boston's 600,000 residents do not identify as White. Boston's Black and Latino residents experience higher rates of preterm births, asthma-related emergency department visits, heart disease and other poorer health outcomes than their White counterparts. BPHC over 40 programs have been in the front line of providing public health services and working with partners to address the numerous health disparities experienced by the city's residents. While there is a lot of evidence on the value of using quality improvement principles to improve essential public health services and address health inequities, the practice was not known to the organization's over 1,100 staff members until the QI program was created. In 2013, BPHC completed an accreditation self-assessment which revealed an average of 70% compliance in all the eleven domains except domain nine (quality improvement) which was 8% compliance with the standards and measures. In 2015, BPHC established the Office of Accreditation and Quality Improvement (AQI Office) to create a formal organization wide Quality Improvement Program to with an aim of improving the organization's culture of improvement and help staff understand how QI could be used to improve public health services and health equity. TARGET POPULATION: Boston is a majority-minority city, where over 50% of Boston's 600,000 residents do not identify as White. The Quality Improvement Program was an organization-wide effort that required engagement across BPHC's 1100 or so employees spread across 40 programs. Like the population they serve, BPHC's staff are racially, ethnically and educationally diverse. The AQI Office had to take these factors into consideration when developing and implementing the QI Program. According to the national 2017 Public Health Workforce Interests and Needs Survey (PH WINS) administered by the Association of State and Territorial Health Officials (ASTHO): 60% of BPHC's workforce identifies as nonwhite and 65% identify as female. 40% of BPHC's staff has a master's degree or higher, over 35% have not completed a four-year degree program. 42% of staff have been at BPHC for less than 6 years, 34% indicated that they are considering leaving their position in the next year. Public health programming varies broadly at BPHC, including epidemiologists, health inspectors, emergency medical services, administrative staff and 24/7 direct care services for clients in recovery or in need of housing. According to PH WINS, BPHC has a lower proportion of clinical and laboratory staff compared to the national average for local health departments, likely in part because of the numerous Boston hospitals that help fill that role. Instead, BPHC has a higher proportion of staff engaged in social services, an area with fewer established quality improvement best practices compared to more clinical fields. Past attempts to address the issue: In 2012 BPHC published its Racial Justice and Health Equity Initiative which focused on rates of obesity, Chlamydia infection and low birth weight as the key measures of progress, because they are indicators of overall health and longstanding racial health inequities. Various campaigns were developed to support this goal, such as the Healthy Beverages Campaign and the convening of the Chlamydia Advisory Group. Bi-monthly newsletters were distributed to communicate progress to these goals. However, these efforts were not systematically applied across the organization and did not result in fundamental improvements. Why the current practice better? Boston has long been at the forefront of piloting and implementing initiatives where health outcome data has been used to measure the success of a program. For example, BPHC helped advise and implement by Boston Children's Hospital to reduce the burden of asthma on low-income children in Boston—. the program had several successes including a 41% drop in missed school days for children with asthma and was called a shining example” by The Atlantic. However, data collection and reporting of progress for quality improvement was not standard practice. There were few internal resources, a lack of organizational knowledge and no central team coordinating quality improvement efforts. The end effect was a thin patchwork of QI knowledge and understanding of value of continuous improvement. The 2015 establishment of the Quality Improvement Program and the delivery of the 2015-2018 QI Plan was better, because it mobilized a Commission wide move towards using a Quality Improvement practice and it filled a skills gap in the organization. This was important because it equipped BPHC staff with skills, tools and a process to further impact the health of all Boston residents. The QI Program also gave staff access to creative customized resources and coaching, which was never provided before to effectively work in addressing disparities. These elements were the foundation of a new way of thinking that will continue to improve public health service for the people of Boston. The creative use of Quality Improvement practice to improve public health services and health equity: According to the Centers for Disease Control and Prevention health equity is ‘achieved when every person has the opportunity to attain his or her full health potential and no one is disadvantaged from achieving this potential because of social position or other socially determined circumstances'. BPHC's overall mission aligns with this definition of health equity. When developing the QI Program, the AQI Office was concerned that equity considerations are not explicit in traditional QI methods, therefore the program may have limited impact on public health outcomes. It is the view of the Office that, while effective, Quality Improvement may not benefit all populations equally. This is not to say that traditional QI can't be applied to a health equity improvement project – it can, but the Office wanted to be more creative and embed equity considerations into all QI practices, whatever the project. BPHC Quality Improvement Program adapted existing industry quality improvement practices and customized them to specifically address health equity which is a new practice to the best of our knowledge. Our example, teams were encouraged and taught to develop *SMARTER aims* Identifying a SMART aim is best practice in Quality Improvement. SMART is an acronym that stands for Specific, Measurable, Achievable, Realistic/Relevant, and Timely. In pursuit of aligning QI with BPHC's mission statement the AQI Office adopted a new acronym: The ‘SMARTER” Aim (an example below). This stands for Specific, Measurable, Achievable, Realistic/Relevant, Time bound, and ‘Equity Revision'. SMARTER aims are used at BPHC to ensure public health services not only improve, but they improve for all. Example: SMART - Increase the # referrals to Mom and Baby home visiting program by 15% in the next 6 months. SMARTER - Increase the # referrals to Mom and Baby home visiting program by 15% and reduce the Black-White referral gap in the next 6 months. The program staff customized numerous other traditional quality improvement tools for example Tool 1: A Swim lane diagram is a method of process mapping that groups the components of a process in distinct lanes to distinguish roles and relationships between components. This is taught as part of the QI Program because it is effective at visually representing complex processes and ideal for mapping public health services. When teaching process mapping, the importance of flagging any disparities between components in a process is emphasized. This means being intentional about mapping where the process flow and quality will change due to race/ethnicity, gender, primary language and neighborhood or community. Qi Tool 2: Another traditional QI tool is a fishbone diagram, or an Ishikawa diagram which outlines contributing factors of a specific issue. When investigating contributing factors, it is common place to consider factors using the 5 P's approach (people, process, place, product, policies and procedures). Due to the nature of public health programs and services the Office trains staff to use the ‘5P' but has adapted this best practice to be ‘5P+E'. The E refers to the contributing factors from equity/inequity. Similarly, when training on how to use a prioritization matrix staff are encouraged to include equity as a criterion and to weight it appropriately for the project. *Adapting QI games for health equity* The Plan, Do, Study, Act (PDSA) exercise using Hasbro's Mr. Potato Head toy is an effective game that is commonly used in QI training. Developed by Dr. Dave Williams Improvement Advisor and founder of DMWAustin it teaches rapid testing of change ideas and measurement to improve process. The AQI Office at BPHC have adapted the game to create the Mr. and Mrs. Potato Head Game. In this version participants are also given a Mrs. Potato head to teach about the equity impact of quality improvement. It is played in a similar way to the traditional game; build Mr. and Mrs. Potato head as quickly as possible with all the right pieces in place and in the right position. However, as well as measuring efficiency (time taken to build) and effectiveness (all the pieces in the correct position) the Office ask participants to measure equity (quantify the disparity in the build between Mr. and Mrs. Potato Head). The program also creatively adapted the 5S numbers game to resonate with a public health workforce. 5S is a Lean Methodology game to teach about using standardization and visual management improve efficiency. This game asks participants to cross out numbers on page (1 to 49) in correct sequence to show that standardization of the visual display helps to work quicker. However, at BPHC, participants are asked to find the ‘public health' on a page of letters to teach the same principals. The evidence for Quality Improvement as an effective public health practice: The program staff extensively conducted the systematic reviews published on the Guide to Community Preventive Services database when developing the QI Program because it is a collection of evidence-based findings from the Community Preventive Services Task Force. There are currently 40 systematic reviews provide evidence on how quality improvement as a practice can support public health activities like vaccination programs, diabetes management and efforts to tackle obesity. The QI program was also identified as an area of excellence during the organization accreditation site visit based on the Public Health Accreditation Board (PHAB) national standards. PHAB has worked with leading experts to develop its accreditation standards which grew from the CDC framework of the Essential Public Health Services. The standard for Domain 9: Quality Improvement is to ‘Develop and Implement Quality Improvement Processes Integrated into Organizational Practice, Programs, Processes, and Interventions'. The AQI Office felt confident that the evidence and guidelines would ensure the QI Program was effective because the National Association of County and City Health Officials (NACCHO) resources for developing a QI Program are also evaluated against the PHAB standards. The NACCHO reviewed, PHAB standards for Quality Improvement suggests health departments focus on three areas (below) and these were considered by the AQI Office: 1. Developing a quality improvement plan that describes, (a) key elements of the quality improvement plan's governance structure, (b) types of quality improvement training available, (c) project identification (d) goals, objectives, and measures with time-framed targets (e) the approach to how the quality improvement plan is monitored (f) communication activities (g) evaluation process to assess the effectiveness of the quality improvement plan and activities. 2. Collecting evidence of quality improvement activities from program areas and administrative areas. 3. Collecting evidence of staff participation in quality improvement activities based on the QI plan. NACCHO also provide guidance on QI training and resources in the Compendium of Quality Improvement Training Resource. BPHC work has been published in the NACCHO Compendium as an example to support and guide other health departments.
QI PROGRAM Goals(s) and objectives: In August 2015, BPHC developed its first ever Quality Improvement (QI) Program. The program included an Office for Accreditation and Quality Improvement and the delivery of a QI plan. This was to support QI to become an organization-wide practice that improved public health services and health equity. The program had two main objectives: (1) Develop a sustainable culture of learning and continuous quality improvement practice to advance health equity (2) serve as a model for other agencies on performance management and quality improvement. How the QI Program achieved the goals and objectives: The program objectives were achieved with 75% of staff agreeing that the QI culture has improved in the last two years and an a significant improvement in the NACCHO QI Culture Assessment result. The Program has also shared his materials and tools to twenty nine health departments across eighteen states. Major activities were (a) delivery of QI training session (Introduction to QI, Basic QI training, QI Toolbox Series), (b) coaching for teams conducting QI projects and administering community wide grants, (c) monthly QI committee and champion meetings and (g) developing and promoting internal communication materials.
*QI training* The training delivered as part of the QI program was designed to help those working at BPHC at all levels and in all areas. Five types of training were designed to ensure the training met the needs of diverse range of participants. The Program used a multi-method approach to learning, varying the skill level (i.e. introductory to practitioner) and the method of learning (classroom-based lecture to structured team learning). Training opportunities included: (1) New Hire Orientation: Quality Improvement - Introduce employees to the organization's quality improvement approach and importance the organization places on a continual improvement mindset. Audience: New employees. Length: 60 minutes. Frequency: Every 2 months. Method: Class based presentation. (2) Introduction to Accreditation and Quality Improvement - Introduce employees to national standards as set by the Public Health Accreditation Board, the organizations approach to quality improvement and the value of a culture of continuous improvement. Audience: All employees. Length: 60-90 minutes. Frequency: As requested. Method: Class based presentation. (3) Accreditation and National Standards - Introduce employees to national standards as set by the Public Health Accreditation Board, share progress of BPHCs accreditation journey, and help staff prepare for milestones such as PHAB's site visit. Audience: All employees. Length: 60 minutes. Frequency: As requested. Method: Class based presentation. (4) Basic Quality Improvement Training - Highly participatory and hands on training to improve staff under-standing of the organization's improvement approach, as well as provide skills-based training on various quality improvement methods. Audience: All employees. Length: 6 hours (2 days). Frequency: Quarterly. Method: Facilitated team-based learning. (5) Quality Improvement Toolbox Series - A multi-class series of highly participatory and hands on training that provides participants with resources, coaching and opportunities to learn, theory, apply quality improvement tools to create measurable and impactful change to their work and reflect on their practice. Audience: Self-referred employees with pre-identified projects. Length: 18 hours (6 sessions). Frequency: Quarterly. Method: Facilitated team-based learning, independent study, and coaching. *Developing QI games for Public Health* As described previously the program used games as key way to support staff to learn quickly and remain engaged with Quality Improvement actively. Traditional QI games were creatively redesigned to focus on improving health equity so that QI could be an effective practice that advanced BPHC's mission of health equity. *Staff engagement and communications* The program used a multi method approach when delivering the QI Program. This included standard communication practices such as an internal newsletter and updates on the BPHC intranet. One nontraditional method was the design of a mascot. ‘Accreditaurus' is the Office mascot, a colorful green dinosaur that is easily recognizable—important considering BPHC is a large organization with multiple initiatives. It was useful because the mascot was an extremely cost-effective marketing tool. ‘Accreditaurus' symbolized the spirit of the program, that QI could be fun as well as effective. Working with multidisciplinary AQI Committee members was also key in delivering the program. BPHC is based across several sites and a large geographic area. This means it's difficult to reach staff working in all programs. By establishing relationships with key champions in each public health bureau it made sharing information and referring people to Quality Improvement opportunities more effective. The AQI Committee were also key to making sure the program worked for all. *Quality Improvement coaching* There were office hours and coaching sessions for staff completing QI projects. This supported QI practice outside training sessions and application of QI to improve existing work. The coaching relationship also supported a culture of continuous learning by empowering staff to keep trying to make improvements and learn from feedback from the Office. Who received the practice: There were no criteria for those who wanted to engage in and learn about Quality Improvement practice. The approach to developing a culture of continuous improvement across the organization was to provide opportunities to learn about QI to anyone who showed an interest. This was one reason why the program succeeded in reaching a significant number of BPHC staff with 1000+ episodes of participation and over 77% (847 employees) of staff trained in three years. The exception to the lack of criteria was the Quality Improvement Toolbox Series. This opportunity is a practitioner level training so only those with an idea for a Quality Improvement project were accepted on to the course. Similarly, because the course was over 6 days and a total of 18 hours, it required a supervisor's approval to ensure full participation. The timeframe for the practice: BPHC completed a health department accreditation readiness assessment in 2013, which revealed a need for a stronger quality improvement presence at BPHC. Efforts formally began with the creation of BPHC's Office of Accreditation and Quality Improvement in 2015. Shortly after, BPHC establishing the Accreditation and Quality Improvement Committee in May 2015 and published its first QI Plan in August 2015. Since then, the Office and the Committee have delivered dozens of opportunities in line with the plan. Stakeholders and their role in planning and implementation process: BPHC's Accreditation and Quality Improvement (AQI) Committee is a team of 8-12 BPHC staff with representatives working at all levels from programs across BPHC: (1) Infectious Disease Bureau (2) Homeless Services Bureau (3) Bureau of Recovery Services (4) Community Initiative Bureau (5) Emergency Medical Services (6) Child Adolescence and Family Health Bureau, (7) Executive Office (8) and at least one representative of Office of Health Equity and the administrative offices. Committee members lead the organization's accreditation and quality improvement activities. Selected by Bureau and Division directors to serve on the Committee, each member was tasked with representing their Bureau, championing QI and the accreditation process, and evaluating related plans. Committee members also co-deliver training sessions with a member of the Office of Accreditation and Quality Improvement, including Introduction to Accreditation and Quality Improvement and the New Hire Orientation. Committee members' roles and responsibilities include: (1) Quality Improvement Collaborate to implement the Program and revise the Quality Improvement (QI) Plan Serve as Bureau's Quality Improvement Champion by facilitating Quality Improvement projects Assist with development and facilitation of QI professional development activities, training, and presentations Participate in ongoing coaching and training. (2) Accreditation/Reaccreditation Serve as Accreditation Champion Suggest and collect relevant accreditation/reaccreditation documents from Bureau and programs Review and make edits to accreditation documents based on accreditation standards Identify and document current gaps in based on accreditation standards and BPHC's Accreditation Report Collaborate with BPHC staff to draft new policies, procedures or provide templates based on accreditation standards Inform and gain support of Bureau and Division Directors for AQI activities. The Committee was crucial to the success of accreditation, as they helped collect, review and edit the hundreds of documents necessary to submit for accreditation. (3) Staff Engagement Engage staff on Accreditation, Quality Improvement and Performance Management. The committee meets the third Thursday of each month for 90 minutes. At each meeting Committee members give their own presentations about progress with accreditation and quality improvement efforts in their respective Bureau. A committee member is allocated to host the meetings each quarter. This means the member chairs the meetings held in their office space. The member also provides a tour of their Bureau and overview of their Bureaus priorities. This is crucial, as it allows the Committee to have a better understanding of each other's work, so they can engage teams in Quality Improvement more effectively. Other stakeholders include the BPHC Executive team. The Deputy Director sits on the AQI Committee as the member from the Executive office and the 2015-2018 QI Plan was formally endorsed by BPHC's Executive Director. How the AQI Office at BPHC fostered collaboration with community stakeholders to further practice goals: The AQI Office fosters collaboration with community stakeholders by embedding equity considerations into QI practice. In pursuit of aligning QI with BPHC's mission statement of health equity and racial justice the AQI Office created a new acronym: The ‘SMARTER” Aim (example below). This stands for Specific, Measurable, Achievable, Realistic/Relevant, Time bound, and Equity Revision. The significance of including Equity Revision as a criterion is that it ensures public health services not only improve, they improve for all. How to create a SMARTER aim statement is taught at all QI trainings delivered at BPHC. Similarly, when teaching process mapping the AQI Office also focus on the importance of identifying components of swim lane diagrams that can flag any disparities in the process of providing public health services. These practices encourage all staff engaged in QI practice to think about how their relationship and how collaborate with the local community. The AQI Office also fosters collaboration with community stakeholders by supporting commission wide initiatives like the BPHC Community Engagement Plan: 2016 – 2019 created by the Office of Health Equity. Through the QI Program, the AQI Office provides coaching for staff conducting QI projects. Teams are supported to think about how they can collaborate with community stakeholders to brainstorm change ideas or prototype change ideas. For example, the QI project with a Women's Shelter was supported to convene the Shelter Client Advisory Group in order to understand client grievances and how they process of resolving grievances can be improved. The 2014 Public Health Workforce Interests and Needs Survey identified the need for improved communications at BPHC and the AQI Office collaborated with external stakeholders to implement a QI project of improving communications between senior leadership and front-line staff. The program staff also collaborated with experts in the Quality Improvement, community representatives and Public Health field including the Institute for Healthcare Improvement (IHI), Cambridge Public Health Department, Worchester Public Health Department, New Bedford Public Health Department, Boston Ryan White Part A planning council and the Infant Mortality Community Action Network which includes representatives from community-based organizations and healthcare. The program staff went further to explore how non-health care partners optimize improvement trainings including a partnership and site visits to a local Applebee's restaurant franchise to learn about how they train staff in process improvement and use data visualization and dashboards. The learning was used to develop the QI training opportunities. The program had two city funded employees, and a Fellow serving at BPHC through the Global Health Corps. The staff formed the Office of Accreditation and Quality Improvement which delivered QI Program and promoted a culture of continuous improvement for the over 1100 employees. The program staff also supported the development and implementation of the organization's Strategic Plan, Community Health Improvement Plan and the successful accreditation from the Public Health Accreditation Board. The program had a non-personnel budget of approximately $5,000 annually which covered cost of materials, food, training supplies etc.
Impact and results of the QI PROGRAM and how far goals were achieved: In August 2015, BPHC developed its first ever Quality Improvement (QI) Program. The program included an Office for Accreditation and Quality Improvement and the delivery of a QI plan. This was to support QI to become an organization-wide practice that improved public health services and health equity. The program had two main objectives: (1) Develop a sustainable culture of learning and continuous quality improvement practice to advance health equity (2) serve as a model for other agencies on performance management and quality improvement. RESULTS: Developing a culture of continuous improvement: A 2018 cultural assessment survey for Program Directors (n=51) found that 75% of staff agreed the culture of QI has improved in the last year”. This was a 12-percentage point increase from the proportion of Directors that agreed when the same assessment was conducted in 2015. The results of same assessment also found that between 2015 and 2018: *There was 10 percentage point increase in agreement that there is an ‘Improvement in effective processes', *10 percentage point increase in agreement that there is an ‘Improvement in continuous learning', *13 percentage point increase in agreement that there is an ‘Improvement in empowering staff'. The national Public Health Workforce Interests and Needs Survey (PH WINS) was conducted at BPHC in 2014 (n=238) and 2017 (n=305). The 2017 survey found a 15-percentage point increase in staff stating there were familiar with the concept of a QI culture compared to the 2014 survey. In 2014 the percentage of staff acknowledging they were familiar with the concept of a QI culture (64%) was like the PH WINS national average (66%). By 2017, the BPHC response (79%) improved by 15 percentage points and was above the PH WINS national average for that year (68%). Staff working at different levels of the organization responded to the survey to state the importance of a specific competency for their individual roles. 90% of staff indicated that Quality Improvement skills were of high importance. In October 2018 towards the end of the 2015-2018 QI Plan 26% of vacancies that were publicly advertised by BPHC stated quality improvement experience in the minimum criteria. 22 teams completed an improvement project some examples Improving referrals to an home visiting program, decreasing client grievances, increasing the percentage of invoices paid, increasing engagements in the organization's social media channels, improving HIV care delivery among partner organizations etc. In November 2017 BPHC received accreditation status from PHAB with the QI program highlighted as an area of excellence
RESULTS: Objective 2: BPHC is a model for external agencies: The program staff Improvement has been selected as speakers at 9 national conferences in the last two years The program staff has showcased its work through 10 poster presentations at national conferences in the last two years Multiple online publications of QI projects and recently authored a paper in the Perspectives in Public Health Journal. The Program materials are also available on the NACCHO's Compendium of tools In the last two years, 27 health departments across 18 states have requested tools and resources from the program to implement in their organizations. QI Program evaluation: All the information documented between 2015-2018 was evaluated to see how the QI Program impacted the use of Quality Improvement as an integral public health practice at BPHC. The evaluation was both summative and formative in order to do the following: Measure the outcome - Understand how far the program went towards achieving the objectives set out in the initial logic model. Explore the impact - Explore how the design and implementation of the 2015-2018 plan impacted BPHC's performance as an organization which continually improves its public health services Inform decision making - Based on lessons learned from the delivery of the 2015-2018 plan, recommend key considerations for the future. By addressing these areas, the evaluation answered the following questions: 1. How effective was engagement and improving staff understanding of quality improvement? 2. How did the QI plan impact employees and the commission overall? 3. How did the QI program impact external agencies? Data sources: Program staff collected primary data from people who attended the QI training opportunities. Participant satisfaction and experience was routinely surveyed in all four years of the QI Program. Surveys were mainly distributed in person immediately after training sessions, which resulted in a total of 766 responses regarding satisfaction with the training delivered. To understand how the QI Program was supporting staff to learn, pretest-posttest assessments were distributed to Toolbox Series participants. The Office collected data on how many correct answers were given to these questions on basic quality improvement principals. Data was also collected on over 22 QI projects completed through the 2015-2018 plan. This included data from a wide range of measures, from response times, referrals to services and number of social media engagements depending on the QI project. The Continuous Improvement Culture at BPHC was assessed as part of the QI Program. The first assessment took place in November 2015 and was based on responses from employees working at director level. The survey used a Likert agreement scale on various questions about continuous improvement culture. The second assessment was done in May 2018. Secondary data sources: The national Public Health Workforce Interests and Needs Survey (PH WINS) administered by the Association of State and Territorial Health Officials (ASTHO) helps benchmark BPHC's internal staff needs/strengths with other similar local health department. The survey was conducted in 2014 with 238 staff responses and in 2018 PH WINS when data was collected from 305 BPHC staff. The survey measured staff awareness and attitude towards the importance of QI and compared it the survey national average. QI Program performance measures: A summative evaluation of the QI Program was planned for the end of 2018. The evaluation assessed the impact the plan had on the organization and how far the AQI Office had achieved the programs objectives. The evaluation also informed decisions about the future of QI at BPHC. Between 2015-2018 the AQI Office measured attendance and satisfaction at QI training opportunities as the process measures. This was important as an aim of the program was to develop a culture of continuous improvement, and therefore it was important to measure the spread and perception of QI across the organization. The process measures monitored were: Percentage of staff introduced to Quality Improvement (Target = 75%, Result = 77%). Percentage of staff participating in Quality Improvement Projects (Target = 10%, Result = 15%). Number of quality improvement projects supported by the AQI Office (Target = 15, Result = 22). Number of ‘small changes' short improvement stories celebrated (Target = 25, Result = 34). 95% of participants responded as extremely or very satisfied with the effectiveness of training presenters (n=412). 92% of participants believed the training was extremely or very valuable (n=40). 92% of participants would recommend the training to a colleague (n=24). 88% of participants surveyed said the QI training games played were very effective or extremely effective in improving their understanding of QI (n=167). A pretestposttest design was used by the AQI office to measure whether understanding improved following Toolbox Series training. In all area's participants showed better understand posttest (n=17) *Model for improvement - 50 percentage point improvement. *Process mapping - 15 percentage point improvement. *Root cause analysis - 50 percentage point improvement. *Prioritization of change ideas - 30 percentage point improvement. *Defining SMARTER aims - 68 percentage point improvement. Analysis of results and data: The survey response data from QI training sessions were analyzed to see how participants were receiving the training. The pretestposttest design was one method used by the AQI office to measure the degree of change in understanding and knowledge because of the QI Toolbox series delivered as part of the QI Plan. While a control group was never used (where the test was shared with people who did not take part in the training), the evidence shows that the training improved participant knowledge in key QI areas. The data collected from QI projects was analyzed to see how the QI program impacted the delivery of public health services and programs. Run charts and other data visualizations were used to see whether the QI efforts where improving the services staff provided to Boston residents. The difference in the spread of agreement responses was compared between the 2015 Continuous Improvement Culture survey and the 2018 survey. The percentage point difference in the proportion of responders who agreed with statements was used to see if there was a change in continuous improvement culture. The PH WINS survey data was analyzed to investigate trends in attitudes between non-supervisory, supervisory and executive staff when it comes to how important QI is for these types of roles. Practice monitoring and MODIFICATIONS: The Program staff noticed that the program was behind with regards to the outcome measure: percentage of staff introduced to Quality Improvement. Forecasting forward, the staff was concerned that it would not reach its target for introducing staff to Quality Improvement. The Office decided to modify its approach to delivering the Introduction to Quality Improvement and Accreditation 60minute training and the Basic Quality Improvement Training. Instead of scheduling independent training sessions and marketing the opportunity for staff to attend, the Office worked with members of the AQI Committee to identify existing team meetings and bureau meetings which could be used to train whole teams at a time. This modification resulted in reaching 77% of staff at BPHC.
LESSONS LEARNED IN RELATION TO THE PRACTICE: The QI Program was evaluated to determine lessons learned and provide recommendations for the 2019-2021 QI Plan. These recommendations included: *Improve access* - The QI Plan was successful in reaching a significant proportion of the BPHC workforce (over 77% of 1100 employees). A lesson learned was that in order engage the harder to reach parts of the workforce, more flexible access is necessary in order to continue to spread QI practice. Creating online introductory trainings and resources have been built into a new 2019-2021 QI plan. This would allow staff working in 24-hour services and evening shifts to take part in training opportunities and contribute to the developing culture of improvement, the program staff is also actively pursuing continuing education credits for public health nurses, social workers and community health workers. *Set specific goals appropriate for a diverse workforce* - BPHC has a diverse workforce that provides a vast range of services to support the health of Boston. Between 2015-2018, whether a QI project was delivered was determined by which members of staff had learned about QI and expressed an interest in receiving support from the program staff. It has been agreed that the new QI program should review what the specific needs and gaps are per public health bureau. Outputs and outcomes for future bureau-specific logic models will reflect how each bureau can better use QI to improve its services. *Better understand where the program is having an impact* - The Program staff learned a lot about how to collect, record and analyze data from the QI Plan activities. Capturing the impact of QI projects through storyboards was an excellent way of recording how public health programs were improving. It also means the program could share the impact QI practice has on public health at conference. A method for data collection will be included in the future QI program to expand the type of data collected such as bureau, job title and recording when individuals take part in more than one opportunity. These options will provide additional incentive for new staff to participate in the programs and make the program more sustainable. The Quality Improvement Plan is one of many plans supporting the growth and transformation of BPHC. The program is also referenced as one of the objectives in BPHC's new strategic plan which is a nod to ongoing commitment and support from BPHC Executive Office and Senior Leadership Team. stakeholder commitment to sustain the practice: The program staff has worked with other programs to develop their program specific QI plans including with the Boston Ryan White HIV/AIDS care delivery program and the Boston Health Start System Program. These will expand the program reach with work with over forty other agencies and thereby improving or supporting the QI culture of partner organizations.
At a NACCHO conference | Colleague in my LHD | E-Mail from NACCHO