Our commitment to helping our partners improve outcomes and advance equity has led us to conduct an annual assessment. During this process, we evaluate the work of our cradle-to-career partnerships in four areas:

• Shared community vision
• Evidence-based decision making
• Collaborative action
• Investment and sustainability

These are the four pillars of our Theory of Action. Success in these areas gives a good idea of how close communities are to building the civic infrastructure necessary to support every child from cradle to career. Our civic infrastructure assessment also collects partnership information and data for each of the seven outcome areas that form our cradle-to-career approach.

Over the past four months, we have been digging into lots of exciting data to understand more about the partnerships in our Cradle to Career Network. We especially want to know how to best support communities toward achieving our bold new goal of transforming the systems that surround our kids to serve them more effectively.

Here is what we have learned:

Network demographics

We now have a more accurate idea of the reach of our work — the StriveTogether Cradle to Career Network impacts more than 13 million children! And nearly 9 million are children of color, with most of the students (37%) in kindergarten through 12th grade being Latinx. That’s a vital piece of data for our work moving forward and validates our focus on centering equity in every aspect of our work.

There are many takeaways from this information, but one that stands out to me is the need for targeted universalism. Our Network must have a positive impact on students with diverse identities, and one-size-fits all strategies will be less successful than those tailored to support the unique needs and interests of every student.

Outcomes data

I’m thrilled to share that 90% of the Cradle to Career Network provided multiple years of data for all students. And 60% of the network reported multiple years of data disaggregated by race for our seven cradle-to-career outcomes.

Nearly half of network members reporting disaggregated data for at least two years are seeing improvements for African American and/or Latinx students in at least one outcome area.

We still have room for improvement — we need stronger data to identify the best strategies to close gaps and get better results — but we were able to determine from the assessment that many network members are seeing outcomes improve for students in their communities over time. Even though we have a lot of hard work ahead of us, that’s truly something to celebrate!

Theory of Action progress

The civic infrastructure assessment shows how partnerships are progressing across the Theory of Action and what benchmarks they are meeting or missing. In 2018, partnerships in the Network met a total of 168 additional benchmarks — indicating significant progress over the past year!

The most met benchmarks involve communication, like informing the community of progress and communicating a consistent message across partners. As the Cradle to Career Network has matured, a focus on communication strategies and metrics has enabled partnerships to develop a stronger shared vision and community alignment.

Benchmarks related to data and continuous improvement are less likely to be met. Although both appear largely technical (e.g., databases, data-sharing agreements, continuous improvement processes, etc.), they require a great deal of trust and capacity — neither of which are built overnight. Check out two examples of how partnerships have built data capacity and continuous improvement capability.

These insights into opportunities for improvement allow us to develop support in the areas where partnerships need help the most. Feedback in the past has resulted in current new offerings, including the StriveTogether Equitable Results Programs, topic-based convenings, communities of practice and virtual supports.

Next steps

The 2018 civic infrastructure assessment produced the most robust data that StriveTogether has had about the Cradle to Career Network. With an advisory council of network members, we are determining ways in which this data can help us toward our bold goal of 24 communities in systems transformation by 2023. It gives us a picture of our network’s strengths and opportunities and helps us identify the support and resources we must provide in service of our 2023 goal.

We are thankful to network members for taking the time to complete the assessment, have learned a great deal about collecting this data and will work closely with partnerships to improve the process. Please leave a comment if you have ideas on how to improve the assessment!

The post Our 2018 civic infrastructure assessment shows Cradle to Career Network’s strengths and opportunities appeared first on StriveTogether.

In Boston, 75% of students graduate from high school — a promising start to their postsecondary futures. But for those who go on to college, just 52% complete their degrees. How can data help predict which students will complete college and what support others need to reach that milestone? Cradle to Career Network member

Why focus on college, career and life readiness?

Our college completion numbers show that our young people are not ready to be in college when they get there. Based on our data, we need to do a better job of thinking about what it means when you graduate from high school to be ready for what’s next — whether what’s next is college or entering a workforce training program or becoming an entrepreneur.

In 2017, we gathered a cross-sector group of individuals in Boston — from workforce development, education, philanthropy and more. Together, we created a definition of what it means to be college, career and life ready.

How do you measure what it means to be ready?

In our definition, we agreed on five important readiness metrics that we found in a review of national research. In our new report, we wanted to see if these metrics played out for students in Boston. If we look at our data over time, do these measures help predict if students will be successful when they graduate?

We tested three of the metrics where we had access to systems-level data, which were having a 2.7 or higher GPA, 94% attendance and access to rigorous coursework. With support from Bob Balfanz, a researcher from the Everyone Graduates Center at the Johns Hopkins University School of Education, we found that the metrics were strong predictors of success. The young people who had combinations or all three of these success factors completed degrees at much higher rates than those who don’t, and much higher rates than our overall completion rates. 

How did you get the community involved?

We involved our community partners so that everyone is oriented to where we’re trying to go and becomes part of how we’re going to get there. From the start, the school system has been at the table — not just the superintendent, but people at every level, from the data accountability office, to the school improvement office, to the high school office. We also brought in higher education and researchers from multiple institutions, and we involved philanthropy from the very beginning.

How will this report make a difference in Boston?

Boston Public Schools, along with Boston’s charter and Catholic schools, are going to be using this report to make policy changes. They’re going to be using it as a guide for school improvement and redesign. At Boston Public Schools, the superintendent has appointed six working groups that are looking at college-, career- and life-readiness supports that go beyond academics, because our definition is much broader than that. Our Catholic and charter schools are going to be doing similar work based on the needs of their students.

What else is next?

We’re going to learn and adjust as we go, but this report gives us a starting point for the things that we know need to happen. In May, we’re going to publish a citywide goal so that we can track our progress on getting students ready. After the goal is announced, we’re going to launch a campaign to help parents, students and teachers understand why this work is necessary. We’re moving forward knowing that how we’re measuring success is valid, because we’ve shown it through the research that the community was involved in.

Kristin McSwain is executive director of the Boston Opportunity Agenda, pursuing a future in which all of Boston’s children and youth are prepared to succeed in college, career and life. Read the new college, career and life readiness report online here. 

The post Helping students get ready for what’s next in Boston appeared first on StriveTogether.

Before we all held Global Positioning Systems in our hands, we used maps and compasses to navigate from point A to point B. And research shows that we had a better handle on directions when we weren’t reliant on a British woman’s voice providing turn-by-turn directions.

Communities need navigation sometimes too. One of the benefits of being a part of the StriveTogether Cradle to Career Network is that you can use the learnings and support of 70 other partnerships across the country to chart a course to advance your own work. Thirty-nine Network members — from Rochester, New York, to Salt Lake City, Utah — contributed to the development of the StriveTogether Data Maturity Model through a series of surveys, focus groups and site visits conducted by The Impact Lab. The model — intended as a compass, not a roadmap — guides communities through the technical, political and cultural aspects of sharing, connecting and using data to drive change and improvement.

Our theory is that to change systems in a meaningful way for every child across the cradle-to-career spectrum, “student-level data must be accessible and used regularly by relevant partners to inform actions to improve outcomes and narrow disparities.”

Porter-Leath maps the process to share and access real-time data across key stakeholders to improve kindergarten readiness.

We’ve seen examples of how intentional and consistent use of data across multiple stakeholders can result in better outcomes for kids. For example, at Harlandale Independent School District in San Antonio, teachers, social workers and counselors pulling daily attendance data resulted in 44 percent of students no longer being chronically absent and 72 percent improving their attendance. In Memphis, a community-based nonprofit Porter-Leath built a data culture across its organization over the last year — spreading the use of data from the quality improvement team to teachers, on-the-ground staff and parents. From 2016 to 2017, they saw an increase in kindergarten readiness from 71 percent to 82 percent, which they attribute to teachers having more frequent data to craft interventions earlier and parents being energized by data pep rallies.

And yet, data managers know all too well the roadblocks that prevent education leaders and practitioners from accessing easy-to-digest data to drive decisions: FERPA. Buy-In. Time. Siloed data sets. Trust (or lack thereof.)

The Data Maturity Model allows community partners to self-assess against 14 key aspects of creating a data culture. Those self-assessments can spur conversations amongst relevant stakeholders about potential paths forward in each category, recognizing that there is no single path to data maturity. Each community’s route is shaped by local context and history; it is not necessarily linear and unfortunately no one is going to warn you that there is a traffic jam ahead that will delay your data-sharing agreement by at least three weeks. Rather, the model will show you your true north. If you meet a roadblock, you can return to it to identify alternative routes — but only if at least one of you says in a British accent, “recalculating!”

The post Navigating the choppy (but not uncharted) waters of data maturity appeared first on StriveTogether.

Lee is standing in front of 17 data managers from across the StriveTogether Cradle to Career Network, all of whom are intent on identifying the correct answer. They offer several suggestions in response to his question — pie charts, bar graphs, scatter plots. With a knowing look, Lee states, “The best way to visualize data is whatever is most effective to achieve your result.” Even in the world of data — a world that can often seem rife with technical solutions — we see that achieving result is fundamentally rooted in adaptive solutions.

The data managers were gathered in February for a convening of the second cohort of its data analytics fellowship program offered in partnership with Tableau Foundation. The fellows are using Tableau data software as a tool to help build a continuous improvement culture and data-driven decision-making in their local community partnerships. They met at the Tableau corporate offices in Austin, Texas, to learn strategies and best practices for maximizing impact in data visualization. The fellows also had the opportunity to work with experts like Feinberg — and one another — to address key challenges and advance their own visualizations.

Over the last year, Feinberg has provided pro bono training support to the fellowship and lends his expertise to help the program participants use data visualization to tell their stories in a compelling manner. Key takeaways from his talk include:

• Focus on intent before content. The purpose and informative capabilities of a “viz” should come before the data is actually shared.
• Keep the “viz” simple. Get rid of components that are challenging for people to understand.
• Help create insights for viewers. Think about your end user and determine if the “viz” makes sense for informing the decisions they need to make.

Although it’s not a “silver bullet” solution for the sophisticated data infrastructure needed to improve educational outcomes for kids, Tableau helps data managers communicate complex information in a simple way. One data visualization can be worth a thousand data points.

The Adams County Youth Initiative (Adams County, Colo.) is using Tableau to highlight attendance data to decrease chronic absenteeism. Identifying and targeting supports to local transient families is a focus area for 90% by 2020: Anchorage United for Youth (Anchorage, Ala.). The P16Plus Council of Greater Bexar County (San Antonio, Texas) is creating annotated run charts that depict completion rates for federal college student aid applications over time and the interventions that improve those numbers.

The fellowship program participants are halfway through their eighth-month journey to use data to drive action and help partners involved in their local work make more informed decisions. The cohort will meet in person again in April and June.

The post One data visualization is worth a thousand data points appeared first on StriveTogether.

Using data to define problems, develop plans and monitor progress may seem straightforward, but continuous improvement has never been among the competencies required in the youth development, direct service or intermediary agencies that make up most collective impact partnerships.

I learned that the hard way.

When the Seeding Success continuous improvement director negotiated a data-sharing agreement for the partnership, we were presented with an opportunity and a challenge. Our direct service partners would be able to measure progress for the children they served, and we would be able to aggregate up to the partnership level. Our monitoring would be as regular as new data were available.

We were responsible for data security and ensuring partners would use the data for continuous improvement. I worked closely with my teammate and developed a training that covered it all.

We adopted language from the Data Quality Campaign. We examined other approaches to continuous improvement training for nonprofits and designed something appropriate for our audience.

Or so we thought.

We made far too many assumptions about what our partners knew. Some staff had basic literacy and numeracy challenges. Others struggled to apply the concepts to their work. There were significant failure rates on the training assessments. Access to data was tied to proficiency, which means we were standing between our partners and very sought-after data. Things got a little tense.

We had skipped an important step. We never assessed our starting point. If we had, we would have found that we needed to lay a foundation before attempting to build partner competency.

As our work evolved, a significant part of my job became focused on capacity-building. The better we got at it, the more need we saw. It has since become a focus of local philanthropy because we know that an agency’s ability to pursue outcomes depends on capacity. Thanks to regular conversations with the StriveTogether team and the Annie E. Casey Foundation’s results-based leadership faculty, I have interrogated my definition of capacity-building. Since my departure from Seeding Success, I have shifted my language to competence.

Defining and assessing competencies is achievable. It has been happening in the nonprofit management world for years. What has been missing is a systematic way of assessing competencies for continuous improvement in the sector.

Ultimately, we are engaged in a professionalizing of direct service work in the health and human development sector beyond the arena of licensed social workers and like professionals. We are asking child care, youth development, social service and intermediary leaders to commit to a core set of knowledge, skills and dispositions and be held accountable for them.

I am both excited about the possibilities and nervous about the implications. This is a corporate and somewhat scientific approach. Such methods privilege ways of knowing that have marginalized people of color and limited access for people living in poverty and in the working class. Professionalization sounds great, but it also suggests a weeding out of those who are unable (or unwilling) to see their work as systematic and scientific.

For some people, caring is more important than competence, no matter the outcomes. And in some circumstances, those people may be right.

Adriane Johnson-Williams, Ph.D., was the founding facilitator for Seeding Success, a StriveTogether Cradle to Career Network partnership in Memphis, Tenn. She now works in philanthropy. She is a native Memphian committed to improving outcomes in her hometown.

The post Checking the foundation for collective impact appeared first on StriveTogether.

All of the outcome indicators that you would like to track as a partnership may not be readily available. Part of the work is directly with partners to help develop shared measures and determine the best way to start tracking them. One example from StrivePartnership’s early work was in selecting our outcome indicator for goal one: Every child will be prepared for school.

There were a number of indicators that we could potentially track (infant mortality, low birth weight, pre-K experience, etc). But we knew the best measure would be one that is a close proxy to the goal, and the one we landed on was percent of children who are assessed as ready for school when they enter kindergarten. But this data wasn’t consistently available. We worked closely with the Success By 6® early childhood networks and the school districts to land on an assessment and begin tracking the data on a regular basis. The early childhood networks were meeting on a regular basis, and I remember that in one of the Covington network meetings, we brainstormed a list on flipchart paper of about 20 different assessments that were being used by partners. There are no common measures for kindergarten readiness and there aren’t even standard definitions of it. Many factors influence a child’s readiness for school including cognitive development, physical well-being, language use, approach to learning, motor development and social/emotional skills. But we needed to land on something as a population-level measure, even if the measures are imperfect, in order to advance the conversation. “Don’t let perfect be the enemy of good” was a mantra, and we had three good measures in our three geographic areas: Cincinnati, Covington and Newport.

Cincinnati is using a brief assessment tool called the Kindergarten Readiness Assessment – Literacy (KRA-L), which helps teachers identifies early reading skills. It is an assessment that has been adopted by the state of Ohio. Newport started out using the Developmental Indicators for Early Learning (DIAL-3) screen tool. The DIAL-3 provides scores for motor, concepts and language. Covington started out using the Dynamic Indicators of Basic Early Literacy (DIBELS), which measures letter-naming fluency and initial sound fluency.

The work of a data manager could involve getting into the weeds on all these measures with members of the collaborative action networks where necessary, helping to sort through the various measures.The partnership can highlight the discrepancies in the ways school readiness is measured and help advance the conversation around the importance of good data and in pushing toward common measures. Covington and Newport agreed on and began reporting a common measure (the Dial-3) a couple of years after the initial baseline report. And just recently, Kentucky adopted a new statewide assessment that all districts in the state will be using.

As mentioned earlier, there are three primary areas where I found myself spending time on any given day: digging into data, building relationships and consensus with stakeholders, and supporting the data needs of collaborative action networks.  A short list of competencies for a data manager would include:

• Knowledge and demonstrated success in data collection, management and analysis; knowledge of education and community data resources
• Ability to build relationships and work with key partners to develop a comprehensive community accountability system that incorporates data across the cradle-to-career education pipeline
• Ability to address and overcome uncertain and complex issues to achieve desired results
• Plans for the collection, analysis and reporting of data to measure the partnership’s impact and to facilitate evidence-based decision-making and continuous improvement
• Develops and cultivates relationships with community partners and stakeholders, including data and research professionals in education, business, faith, nonprofit, philanthropic and civic sectors

See A day in the life of a data manager: part one

The post A day in the life of a data manager: part two appeared first on StriveTogether.

As a data manager for a StriveTogether Cradle to Career Network partnership, there are three primary areas where I found myself spending time on any given day: digging into data, building relationships and consensus with stakeholders, and supporting the data needs of collaborative action networks.

Evidence-based decision-making is an underpinning of a collective impact partnership, and one of the first things to tackle after establishing the shared vision and goals is to establish a set of shared outcome indicators to help measure progress toward the shared goals. And so a data manager needs to dive head first into the data itself and really understand all the sources, variables and caveats to how the data may be collected and presented. We started with a list of over 75 potential measures, and so the data manager really needs to understand the data landscape in order to be able to help steer the data team and partnership in getting to consensus in narrowing that list down. The manager also needs to become the local education data “expert” and help build credibility for the partnership by being one of the go-to people for questions related to education data and results in the community.

Building relationships and consensus, however, is just as important as the data analytic skills. A partnership’s director and local champions will definitely help with building relationships among partners and advocating for data transparency, but the data manager also has to be able to forge relationships with the key data partners and build trust with them. One of our first efforts was to form a data committee comprised of all the data experts from key partners at the table: the school districts, postsecondary institutions, early childhood professionals and other community data experts. As a committee, we came to a list of 10 shared outcome indicators together, using a set of criteria that we developed, to take back to the executive committee as a recommendation.

It is important that this process is done with your key partners as opposed to it feeling like you are producing a report about your key partners. And so establishing relationships and building trust are key ingredients in this. Landing on the indicators is a back-and-forth process of presenting ideas and getting feedback until you have built something together that everyone feels ownership of. As a result, when we released the first report, executive committee members could speak with confidence about it, knowing that they had truly helped to create it.

Coming next: Working with networks to define indicators where no clear ones exist and key competencies of a data manager: See A day in the life of a data manager: part two.

The post A day in the life of a data manager: part one appeared first on StriveTogether.

The question posed around how to choose the right outcome indicators was a really good one that I’ll explore a little here. And it’s one that comes up a lot. How do you know that you have the right indicators? What if there is another critical measure that you missed at sixth grade, for example?  The reality is that there are a lot of important indicators that can be tracked and reported on. We started off with a list that was close to 100 potential indicators, but this is of course way too many. Not only is it too many to feasibly track on a regular basis, but the fewer indicators you have, the more focused you can become and the easier it will be to focus a wide collective impact effort to drive a core set of outcomes.

One of our mantras, espoused by the partners in creating the first StrivePartnership report card in 2008, was don’t let perfect be the enemy of good. This was a critical disposition, because for the outcome indicators that we settled on (we started with 10 in the first report), there were any number of reasons why a particular indicator didn’t really tell the whole story. As one example of many, the kindergarten readiness assessment data in Ohio only measures literacy. However, many will argue that we need to be measuring more than this to understand if children are prepared for school. So, we’ll work on advocating for a better measure (and the report card also becomes a good tool for this purpose), but we won’t let this stop us from publishing a good indicator that still gives a strong proxy to the ultimate goal that every child be prepared for school. We created a data committee to help develop the report and one of the very first things we did was to create a criteria for selecting indicators to help guide this process.

So how did we get from 100 indicators down to 10? We started with the Cradle to Career Roadmap to Success, which includes the benchmarks (the research-based competencies and experiences that act as predictors along the roadmap, indicating future success along the continuum). Then the key transition years are time periods along the continuum where students are often found to fall at risk if they are not developmentally on target. The combination of benchmarks and transition years led to the creation of the five big overarching goals (prepared, supported, succeeds, enrolls and graduates), and the indicators developed from these as measures of how well we, as a community, are achieving these goals.

Just because you don’t choose an indicator doesn’t mean that it’s not important, and it may be tracked in some of the strategic priority areas of the partnership. The outcome indicators that we are interested in are primarily the longer-term indicators that tell us if we’re getting to our goal. The short- and mid-term indicators are still important, but not the primary focus in choosing indicators for the report card.

And the development of the indicators was very much a collaborative process with all the partners. The data committee was comprised of the lead data people from each sector contributing data — early childhood, k-12, higher ed — as well as some key community partners, including United Way and Children’s Hospital. We had a series of many meetings in the first year in an iterative process to develop and refine our list of indicators. And we have refined (and hopefully improved) the indicators and reporting style in each subsequent year.

We have since reduced the 10 indicators down to eight. And the work of the entire StrivePartnership is focused on helping to move the dial on these core outcomes:

• Percent of children assessed as ready for school at kindergarten
• Fourth-grade reading achievement
• Eight-grade math achievement
• High school graduation
• ACT composite score
• Percent of students enrolling in college
• Postsecondary retention
• Postsecondary completion

Ultimately, choosing the right indicators is both a science and an art. A science in that you need to have a rigorous process and criteria for selecting indicators, and a good fundamental understanding of what data is available and what it can tell you. And an art in that it’s also about aligning this work under the shared vision and leadership of the overall effort, as well as building relationships with the data experts and owners in order to create the buy in and shared ownership to move the work forward together.

The post Developing shared measures appeared first on StriveTogether.