I admit: I didn't think it was that hard to get a survey translated. 

Over the past few weeks, I learned just how wrong I was -- and ate a big piece of humble pie in the process. 

With colleagues, I'm working on a landscape analysis of how families and educators in California feel about family engagement and the state's requirements for incorporating stakeholder feedback into district plans for improvement. We're designing a training program around these topics, but to make sure our program will be relevant, we wanted to hear from the people who would be participating in it. We designed a survey and planned for focus groups, and I naively thought we were good to go.

Although Baltimore has a growing -- but fairly localized -- population of English Language Learners, the families at the schools where I worked were predominately Black and English-speaking.  When I worked at the district, we had a cadre of interpreters we regularly contracted with for events, and we used large-scale survey software that easily facilitated (mostly adequate) translations. 

So when we decided to translate the California survey into nine additional languages, I didn't anticipate just how difficult that would be. 

Our survey was fairly basic and brief, so I built it out in Google Forms ... only to learn that despite the widespread availability of their free translation technology, there was no mechanism for translating surveys in their tool. (I'm honestly still scratching my head about this.) The most straightforward (ha!) way I found to create a multilingual survey in Google was to independently translate the survey into each language, build a separate page in the survey for each language, copy and paste each line of the translated surveys, and then use skip logic to direct people to the page with the language they selected. 

Umm what?

We gave up on Google. We found out that our client had a Survey Monkey account that included the ability to create multilingual surveys. I was excited. Finally - a logical way to complete this seemingly simple task! 

Nope. I was still wrong.

​While this platform at least offers a dropdown menu of languages on the survey page (thereby making it easier for respondents and avoiding the skip logic silliness on the back end), it turns out that this paid feature was just as cumbersome to use as the Google option. What I ended up having to do was download a coded text file for each language, pay to independently translate each of the languages (Thank you, Stepes Translation, for coming to our rescue!), copy and paste each line of the translations into specific sections of the text file, and then upload the translated file to the system. NINE TIMES.

With my hand cramping from all of that Ctrl-C and Ctrl-V action, I was stunned by how technically difficult and frankly, inaccessible the survey translation process was. Who is actually going to go through all this? More importantly, what does this mean for the voices of those who are not native English speakers? Without access to a large, institutional subscription to a powerhouse survey software, my gut tells me that very little translation is likely to happen. As a result, many important voices are being silenced. 

I don't have a solution to offer here, but I'm glad that this is a lesson I learned. This has opened my eyes to the institutional roadblocks that prevent equitable language access in our country... and I know I've just scratched the surface. Translation services, albeit not 100% reliable, are widely accessible and free online, yet they are not integrated into lower-cost survey platforms. This not only causes a huge headache for survey designers, but it inhibits the ability to hear from non-English speakers about important issues. As I seem to say in a lot of my blog posts, we have to do better. 

If anyone has a better solution than the relay race I just ran, please share in the comments! I do hope that a more accessible and user-friendly option exists.

It's easy to feel discouraged and upset when you turn on the news these days. So much is happening to progress the fight for racial equity and justice in this country that even a global pandemic seems to have taken the backseat. The truth is that without a precedent for our current events, we are all making sense of them as they come. We face tremendous uncertainty in the days and months ahead, especially in the education sector. No one knows what school should or will look like when the 2020-2021 year begins in the fall. And that is scary. 

But here's why I'm feeling encouraged. Without a doubt, the Black Lives Matter movement is bringing critical and often unheard voices to the forefront. I'm also starting to see this happening in schools, with many districts really lifting up the voices of parents and families as they make decisions for what reopening schools will look like. I've seen multiple districts just this past week sharing surveys with families about reopening. How can we truly serve children and families if we don't know what they fear, what they want, or what they need?

So if your school, organization, or district is trying to imagine what school will look like in August, and you haven't talked to families and students, now's the time to use some simple evaluation strategies to give power to their perspectives. Here are a few tips to get started. 

Think about what you really need and want to know from your stakeholders. Make a list of what your team is wondering about or what the impact of proposed plans might be before you draft your survey questions. For example, many districts are considering alternate schedules to accommodate all students in socially distant ways. Here are a few things to think about: 

• Do families have access to childcare and meals for when their kids would not be in school?
• How do their work schedules align or conflict with proposed school schedules?
• Would entire families be on the same schedule, or would parents and guardians have to juggle multiple schedules?
• What are their fears and concerns (and those of their children) about returning to school?

We can turn these internal questions into survey questions for students and families to share their thoughts and influence these critical decisions. 

Encourage your survey respondents to commit to an answer. Whenever I take a survey, and I don't really know or care about the answer, I always select the non-committal, middle option. Most people do - it's human nature. However, during this especially important time, we can't risk having a whole bunch of middle of the road responses. Consider using a four-point (instead of a five-point) scale that encourages respondents to indicate if they're feeling negatively or positively about what you're asking. Instead of a neutral/not sure answer choice in the middle, have them choose from a scale like this: Strongly Disagree, Disagree, Agree, or Strongly Agree. Include a "not applicable" option if you feel that's relevant - we don't want to force answers that don't make sense - but this type of scale will give your team a better sense of which way your stakeholders are leaning and help you make more informed decisions.

Make it equitable and accessible. Hopefully it goes without saying that not all students and families can access a survey that is online and only in English. To embrace and reflect the diversity of our nation's school districts, we must try to reach our stakeholders in multiple ways. Of course, an online survey is the easiest way to collect information, and many families can at least access the internet on their phones. However, some families cannot, and to truly understand what your families and students are feeling about reopening, we need to try to reach them as well. Think about mailing surveys or distributing them at food giveaways or other local gathering places. Or, if you're unable to translate the survey into every language spoken in your district, hire a few bilingual staff members or outside interpreters to do brief phone surveys with families whose native language isn't English. 

For the most successful reopening possible in the fall, districts need to know what families and students are thinking now. Brief surveys are an easy, cost-effective way to reach a large percentage of your stakeholders. Schools and districts need to think creatively to hear from as many families as they can and make their understanding of student and family needs as inclusive and diverse as possible. You'll be amazed at how much more informative your results can be!

It's funny how things work out sometimes. 

Tamara Hamai and I have been sowing the seeds for our new program, Evidence for Engagement, for months. Our partnership happened so organically - a meeting of the minds for two evaluators who have experience with and a passion for organizations that serve youth and families. We'd been toying with the best way to support the organizations that we serve and help them use evaluation to improve their access to funding and the children and families they serve. 

Then COVID hit. 

The pandemic has caused all of us to pause and re-evaluate how our work fits into a very new, very different reality. Tamara and I know that small organizations, especially those who work in schools, are struggling right now. Their access to the people they serve has been essentially cut off. We realized that organizations may need our help even more than before. 

Our solution: We're running a totally free, three-week email series that will help small youth- and family-serving organizations build their evidence base (which is required under the Every Student Succeeds Act for any organization receiving federal education funds). Through videos, worksheets, frameworks, and success stories, Tamara and I will walk participants through the process of becoming evidence-based organizations and help them see this as an opportunity, not a burden. 

The goal: We want to help vital, community-based organizations plan for the future, open themselves up to new opportunities, and become more sustainably funded. We're hoping that this opportunity will help them better serve youth and families, not only during this difficult period of time, but also for a long time afterward. 

For us, this is also about equity. We know that for many community-based, minority-owned organizations, budgeting for evaluation is out of the question. We also know that these grass-roots organizations are having a profound impact on their communities -- and that their communities need all the support we can give. We're hoping that we can get more small, local organizations approved as evidence-based programs in their districts and begin to level the playing field. 

If you think this program will benefit you and your organization, sign up! If you know of someone else who could use this support, encourage them to join. Feel free to share this link widely: bit.ly/evidence4engagement ​

It's not always easy to measure the impact of family and community engagement efforts. Some aspects of education -- like test scores or report card grades -- (notwithstanding the wide variety of controversies around their use) are pretty straightforward to measure. They're already quantified. They're known entities. We can tell that story. But when we talk about measuring the impact of, say, a super successful family science night, our minds go blank. 

Thinking of our present situation, in which family and community engagement is more critical than ever, how can we tell the story of how a school or organization engaged and supported its students and families and even its neighbors during the pandemic? How do we collect and measure both the individual or personal impacts and the large-scale results?

Erica Green and Lola Fadulu did such an effective job of this in a recent New York Times story about how school cafeterias across the country have become "relief kitchens" for not only students and their families, but also community members in need. The cafeteria manager featured in the story effortlessly rattles off the incredible numbers of people she and her team have served and also shares stories of individuals she's encountered. Not only is she a hero in her community, but she's a perfect example of one of my core beliefs -- that you don't need to be a statistician to effectively use data in your work.

If your team is not sure how to start measuring your COVID-19 impact and telling its story, here are three steps to get started: 

1. Brainstorm the different types of data you already have. There will be more than you thought! Traditional sources of school data, such as attendance records, event sign-in sheets, or academic data, may be less available right now, so get creative. For example, think about all that you can learn from a log of phone calls made to families. You'll know which numbers worked and which didn't (which is basically a list of hard-to-reach families who need additional support), you'll know which staff members have made the most calls, you'll know how the families who were reached are doing, and you'll know where subsequent outreach is needed. Pop that information into a spreadsheet, add some labels or color-coding to indicate who needs what, and you've got yourself a working database for the students in your school. If you add in all of your other data sources, your team will be data-rich and able to act.
2. Consider collecting other data that you need. There are many ways to collect information from students, families, and community members right now. Think of all of your touch points with your key stakeholders. Students may be logging in for online class, families may be checking social media for updates, and all of your stakeholders may be accessing meal sites, like the one I mentioned above. At all of these venues, you can easily ask about needs, satisfaction with the school's efforts, or any other questions you may have. You could have teachers post a poll for their students on Zoom, you could post an online survey on social media (or on the district website, like my home district in Columbus has done), or you could have surveys on half-sheets of paper available at meal sites. For families you can't reach in one of these ways, there are always mailed surveys or text message options. You could also invite families you do reach to virtual focus groups, to learn more about their experiences and collect their insights. 
3. Make data analysis a team effort. Using these rich sources of data should not be the sole responsibility (or luxury!) of the principal or district evaluators. Building a culture of data means getting everyone involved and modeling for them how to use data and why it's so important. Consider hosting virtual staff meetings to work together on this data. I've seen a lot of districts hosting virtual PTA or PTO meetings as well - that's the perfect place to get families involved. You could even make posters of important data and hang it on the walls at your meal sites. If you need some ideas for how to engage stakeholders in data analysis and interpretation, here is a blog post from the American Evaluation Association and here is a great list of resources from Better Evaluation about hosting data parties. 

This is just a short list of ways to use data to measure your impact and involve your community in doing so. Of course, if your school or organization needs some guidance, I'm here for you. I offer three different packages to support schools, districts, and organizations with this work. I'm also happy to do phone consultations to help you brainstorm. 

Regardless of how you do it, this is the time to make data a priority. What will be your story?

I had some great conversations this week with colleagues about establishing a culture of data in organizations and training organizations who are new to evaluation and data. These conversations reminded me about one of my favorite old blog posts, that I originally wrote for the National Association for Family, School, and Community Engagement (NAFSCE) in 2017. Given this week's discussions, it felt like a good time to bring it back into the rotation (with a few updates!). 

When I was a teacher, conversations around instructional data were baffling to me. Fresh out of policy school, I was eager to use what I had learned about data analysis to monitor how my students were performing, but as a social studies teacher, this task was more difficult than I had anticipated. I was required to keep a data binder, and administrators would periodically check to confirm that, well, it existed. However, I struggled to figure out what to put inside of it. My administrators did not help me understand how – absent standardized test data – I could track progress on specific standards outside of my grade book. It often felt like the conversation ended after the word “data” was uttered.

As I have focused my career on family engagement efforts, I have seen how conversations about using data to improve engagement are often greeted by the same blank stares I encountered as a teacher of a non-tested subject. On other days, talking about data elicits looks of panic or skepticism. At one particularly memorable training, community school coordinators were led in a debate about the utility of data. Sitting from my seat on the pro-data side of the room, I was amazed by arguments from the anti-data group. What resonated most is that these capable and talented colleagues understood data to simply be numbers on which their performance review was based, not as a tool to discover context and unlock insights about the families being served.

I think this belief system exists for a number of reasons. First, many educators are tired of increasing demands for data without sufficient training. Professionals need to understand how data can be collected, ways in which it should be analyzed, and how it can actually make their work easier. I have found that on-the-ground staff are often the last to receive the proper supports and professional development around understanding and using data. It becomes a symbol for all of the things we don’t like about accountability instead of the asset that it truly can be.

Perhaps more importantly, the work of engaging families – understanding needs, forming trusting relationships, and helping people when they are vulnerable – is incredibly difficult to quantify. Often, we know we have made progress or achieved results – not because of a spreadsheet or heat map – but because a family had enough food for the weekend or because a child stopped acting out as much in class. How do we tell those stories? How do we show our value as professionals when these important markers seem impossible to put into a spreadsheet? These are the critical questions we need to answer.

For these reasons, it is my mission to help educators and professionals realize that data does not have to be scary or intimidating. It does not require complex coding skills or mathematical know-how to track how clients are being served. If you would have been sitting across from me in the data debate, here are some tips to get you started:

• Start with what you have. If you are trying to get more parents involved at the school, it’s helpful to know exactly who is already coming to events. Try making a spreadsheet of the information from your event sign-in sheets and see what patterns you find. For example, comparing the names on this list to a whole-school roster can help you figure out which kids have had little to no in-person parental involvement. Use this approach for other measures that you can track from the information and documents already sitting in your office.
• Leverage the expertise around you. Everyone knows who the go-to person is at their workplace when they have a technology question. Maybe that person (or someone else) also has some knowledge of Excel or other tools. Anyone can fill in a template. See if a colleague can help you design the tracking tool you know you need but do not know how to create.
• Do not be afraid to play around and make mistakes! The best way to learn how to manipulate and analyze data is to get your hands dirty and play around with it. Try different buttons, Google how to do things, and ask what colleagues at other schools and organizations are doing. This is why there is an “Undo” button! Of course, if you are unsure, you can always make a copy of your file so the original data is safe.

Using your organization's qualitative and quantitative data can give you amazing insight into both the ongoing needs and continuing growth of the students and families you serve. With a little less reticence towards this approach, we can make a lot more progress in engaging families to help their children succeed.

Of course, if your organization is unsure of how to get started in this area, I'd love to be of assistance. Learn about the new Build Your Evaluation Capacity training package!

It's funny where the most interesting and engaging conversations can happen. At the gym last week, I had a few random, but passionate conversations with other members of the educational (and martial arts!) community about youth experiencing trauma and how it impacts their ability to participate in school, learn effectively, and handle their emotions. 

Trauma is something I wish I had learned about when I became a teacher over ten years ago. Like many new teachers, I struggled with classroom management. It is probably not surprising to learn that as a 5'2", 22 year-old woman, I did not have a natural authoritative presence. However, there were many other reasons for the challenges I faced -- ineffective and harmful administrators, a lack of shared expectations among our middle school team, and my own battle with anxiety. These all contributed to what often felt like turmoil in my classroom. While I always empathized with the challenges my students faced in their own lives, I never fully understood the ramifications of the trauma and hardship they experienced -- nor did I know where to connect them or their families for additional support. 

I was lucky because the school had a wonderful social worker and part-time psychologist to whom I could refer students. They also served as great supports and sources of advice and knowledge for me as a new educator. Yet, there was little capacity and no infrastructure at the school for understanding and responding to trauma. I remember being told that students were experiencing homelessness, hearing stories about families who were involved in gangs, and seeing that students were extremely impoverished, but I had no tools to process these situations or fully support my students. I felt anguish about the situations they were experiencing, but I know I did not always respond effectively. 

Years later, after working in a community school and supporting many others, I know what more effective and comprehensive supports can look like. I have seen the benefits of wraparound services for students and families, including a full mental health team; meaningful enrichment and engagement opportunities for students and adults; connections to resources for basic needs such as housing, food, clothing, and adult education; efforts to track and review data on engagement, attendance, and supports provided; and most importantly, a loving and affirming approach to working with students and families with the greatest needs. Community schools -- schools that become a hub of the surrounding community and provide these wraparound services for students, families, and community members -- are well-supported by research. This model has been shown to be beneficial in reducing chronic absenteeism, improving school climate, increasing student achievement, and more. One of the findings from a 2017 Learning Policy Institute report states that:

"The evidence base provides a strong warrant for using community schools to meet the needs of low-achieving students in high-poverty schools and to help close opportunity and achievement gaps for students from low-income families, students of color, English learners, and students with disabilities."

​So when my friend at the gym, a high school guidance counselor, told me that his school effectively did not have any mental health clinicians available to students, I was taken aback. I talk to him frequently about his school and the amazing ways in which he supports his students, so I was shocked to learn that this large high school was so under-resourced in this area. The school seems to be on its way toward providing wraparound services, with a brand new food pantry and the dedication of counselors like my friend. Yet this conversation made me realize that the services I took for granted in Baltimore City schools -- which were still often insufficient to meet the high level of need -- did not exist everywhere.

This was a critical and humbling realization. Of course, if we could make students' barriers disappear -- or at least provide them with the resources that they need to overcome these barriers and thrive -- we would. But in the absence of a magic wand, what can we do? From my lens, this is where needs assessments and data tracking can play a huge role. Imagine if we collected stories from students, families, and staff about the challenges faced by the school community, each stakeholder group's perceived needs, and their recommendations for meeting those needs. Now think about if we used factual data about the community and student population to support those stories. What we would get is an intensely compelling, collective narrative about what this community needs and how its members feel those needs could be met.

Could a needs assessment or set of data instantly bring on a full slate of mental health workers at this school? Of course not. But when we tell our story effectively, people (read: funders, decision-makers, influencers) listen. And when people start listening, we can inspire them to make change. 

This was an unusual Thanksgiving for my family. My parents have always hosted Thanksgiving at their house; it is one of the few family traditions that remains since my Grandma passed away. All was set to continue as usual this year, until last week, when my Grandpa -- the effervescent, 89 year-old patriarch of the family -- began experiencing significant back pain. It quickly became evident that the trip to my parents' house would be too much for him. Like good evaluators, we continually monitored the situation and used the data we gathered to determine what changes we could make to ensure that the family would be together and that my Grandpa would be as comfortable as possible.  

We ended up bringing Thanksgiving to him. Different family members were nervous for different reasons. My parents now had to factor in a one-hour drive into their cooking schedule, and others had to adjust their travel plans. Some were skeptical about having Thanksgiving in a different location, and given that we all adore my Grandpa, we were worried about how he was feeling. Afterwards, when I reflected on how the day went, I realized that most of the questions on everyone's minds had been fairly quantifiable:
​

• Would the turkey be ready on time (It wasn't.) and survive the car ride? (It did, thanks to some duct tape and clever packaging on my parents' part.)
• Would we have the food warmed at my Grandpa's house in time for everyone's arrival? (We didn't.)
• Would all of our family members make it (We were down one cousin.) and arrive on time? (Everyone was early, except for my parents with the turkey that wouldn't cook.) 

To determine the success of our Thanksgiving dinner, we could track these metrics or others, such as the number of family members who came (11) or the amount of time everyone spent at my Grandpa's house (approximately three hours). These are all important details that help us craft the story of our Thanksgiving dinner. Yet these facts could never describe the contentment on my Grandpa's face as he sat at the head of the table and listened to the family's jokes and conversations. A series of data points would never suffice to explain how happy we all were to see him acting like his usual, wonderful self because he had just had a wonderful day. No spreadsheet in the world -- and I LOVE a spreadsheet -- could have ever captured the hope that my Grandpa had after a day with much less pain and far more happiness than he had had in a few weeks. 

Of course, I would never methodically evaluate our family dinner -- but isn't this exactly what we try to do when we try to measure family engagement in our schools? We host events and programs, offer services and supports, and build relationships like pros, but we struggle to figure out how we can capture the impact that we have made and share that impact with others. We count names on sign-in sheets and have participants rate their satisfaction, and we hope to show that we made a difference in our students' and families' lives. Those pieces of information are critical -- we need to track those metrics to track our progress over time and demonstrate growth in our programming and our reach within school communities. Yet, our standard documentation cannot capture the joy on a mother's face as she gets a relaxed moment to play and learn with her child or the pride that a father exudes when he has just completed his first professional resume. These are two examples from my work as a Community School Coordinator that have stayed with me to this day -- and neither came from one of my beloved spreadsheets. 

This is why stakeholder voices and qualitative data are so important. Certainly, if the turkey never cooked or if no one showed up, our Thanksgiving would have been very different. However, what made Thanksgiving so special for me was watching my Grandpa's joyful reactions during dinner and hearing him speak happily on the phone for days afterward. For those of us in the field of family and community engagement, these are the types of reactions we live for and the motivation we need to keep doing the tough work. To best capture our impacts and tell our stories, we must take this mixed-methods approach, enhancing our traditional measures with rich stories and perspectives from our most important stakeholders. For me, this is what makes evaluation -- even of a Thanksgiving dinner -- most powerful, interesting, and even fun.