|
You should know by now that I'm a bit of a data nerd. I love spreadsheets. I love organizing data and using it to illuminate patterns. I love the "ah-ha" moments when clients realize how much their own data can tell them about the kids and families they're serving. So it may surprise you that I'm here to say that numbers and spreadsheets don't tell us everything. That doesn't mean that numbers (or quantitative data) are irrelevant. It just means that they are even more informative when paired with stories, quotations, or anecdotes (qualitative data). (See the box for a quick refresher on the difference between the two).
Here's an example. Yesterday, I was re-reading an article from The Columbus Dispatch, my local paper, about the spike in youth violence that has occurred during the pandemic. It's been horrible to hear about how many children and teens (well, really anyone, for that matter) have been victims of gun violence since the spring. The article cites a number of statistics -- that the number of children treated in Columbus for gunshot wounds this spring and summer was double the rate from 2019 (from 16 to 32); and that children from racial or ethnic minorities are twice as likely to be shot than white children. Those are AWFUL statistics - and they certainly help me see that there is a dire situation here. But then, the article talks to a teacher whose student -- an eight year-old boy -- was killed. Here's what the article shares about (and from) the teacher: Thalgott has lost a handful of former students during her 20 years of teaching on the South Side. She's seen even more students who have lost a parent to gun violence. Having lost some former students or their family members to gun violence -- either as victims or perpetrators -- this quote really gets to me. This quote conjures up such raw emotions that suddenly it puts the statistics they cited into context. Those 32 children are somebody's child, somebody's sibling, somebody's student, somebody's mentee. Hearing from a person who actually experienced that loss made a big difference in how I processed this article. I imagine it did for you too. Quantitative data can be so powerful, but its impact is amplified when we lift up the voices of those we are serving or studying. Qualitative data -- gathered through interviews, focus groups, open-ended survey questions, or observations -- can sometimes more effectively communicate the experience of what is happening in your school or community. I'll be doing a series of posts on qualitative data over the next few weeks -- how to collect it, how to use it, and how using a combination of data can truly help you tell your story.
0 Comments
I started writing a completely different blog post for this week, but when I read the news this morning, I knew I had to shift gears.
CNN featured a story yesterday called "Teachers and social workers search for students who are 'missing' in the pandemic." The word "missing" made my heart drop. It immediately reminded me of this story, of a young girl from Washington, DC named Relisha Rudd. I heard about her story in 2014, and it broke my heart -- at the time, I was a community school coordinator and led our school's charge for attendance and engagement tracking and interventions. Staff from Relisha's school tried to track her down, only to find that a man working at the homeless shelter where she was staying had been impersonating her doctor to the school. She is still missing to this day. As a community school coordinator, her story lit a fire under me to do everything I could to make sure that we knew, to the best of our ability, that our kids were safe and able to come to school. I have thought of Relisha over the years and find the tragedy of her story to be a call to action for schools, districts, and other youth- and family-serving organizations. How can we make sure that no other students fall through the cracks? When I read the story this morning about the Robla School District in California doing home visits and trying everything they can to find their students "missing" from online school, I had so many thoughts: Labeling students as "missing" drives home the gravity of the situation our country is in. Families are truly struggling because of the virus and the economy, but honestly, lacking access to the internet, to stable housing, and to consistent work have been challenges for so many families for so long. The fact that things are only getting worse is upsetting and shows us that we have so much work to do. Literally going into neighborhoods searching for children is heroic, but also emotionally grueling. I remember the disappointment and worry of having a string of unsuccessful home visits -- you gear yourself up for making a difference, only to find that addresses were incorrect or have changed, or worse -- you just don't know where students and families are living. That is scary, and it is emotionally taxing for educators. While data tracking can't help us physically locate a family, it can help us focus our efforts where they are needed most. You may have seen that last week, I released a guide for how to Track Your Engagement Data in 4 Simple Steps. I believe strongly that using simple functions in Excel can help educators pinpoint exactly which students and families need additional support -- whether that's with attendance, engagement, or academics. (I know it can work because I've seen the impact it has had on my own work in schools!) So in honor of Relisha and in commitment to the well-being and success of students who may be "missing" from online school today, let's get tracking. To learn more about data tracking, visit my Engage with Excel page or sign up below.
Sometimes you launch a survey, and you're blown away by the number of responses you get.
And sometimes, you're not. I had one of these moments last week. I was SUPER excited to try something new with my blog and launch a survey to hear what readers wanted to see in future posts. I sent out my blog to my email list, waiting with bated breath (okay, maybe I'm being dramatic) for all of the responses to pour in. ... And then I realized that the survey I embedded didn't even show up in the email. INSERT FACE PALM EMOJI. Let's just say that I didn't get the response rate I was hoping for. But here's the thing - it's okay to have a survey fail. All hope is not lost. If you don't get the response rate you were hoping for, take a step back and consider:
For me, technical issues definitely got the best of my survey attempt, but I also think a reminder wouldn't hurt. So here's my plug: I'd love to know more about what you want to learn! I'd appreciate if you could take a minute to share your thoughts and preferences with me. I'll report the results and use them to make this blog even more beneficial for you. The survey is embedded below, but if it's not loading for you, click the button to go right to it.
We have all taken TONS of surveys in our lifetime.
We get surveys when we make an online purchase, when we speak with a customer service agent, when we want to get a free gift card, and even when we go to the hospital. We're all pros at taking surveys... and we all know when we're taking one that's TERRIBLY designed. For me, if I don't feel like I can answer the questions, or if it gets too long or overly annoying, I'm out. And that organization just lost a respondent. I don't want that to happen to you -- because in education, surveying our stakeholders is SO important. It shows that we value our stakeholders' opinions, feedback, and experiences. We can't afford to lose respondents because of iffy survey design. Here are a few of my tips for upping your survey game: 1. Ask only what's really important. Make a list of what your team is wondering about or what the impact of your proposed projects/plans might be before you draft your survey questions. Keep it short and sweet ... if it's not related to those things, don't include it. 2. Reach respondents where they are. Think of all of your touch points with your key stakeholders. Students may be log in for online class, families may check social media for updates, and all of your stakeholders may access meal sites. At all of these venues, you can easily ask about needs, satisfaction with the school's efforts, or other questions you may have. You can also get feedback through polls in Google Classroom, Zoom, via text message, or even on social media. 3. This may seem obvious, but ... make it easy for respondents to actually answer your questions. Keep the language clear and simple so a person of any reading level can understand it. Never ask about more than one topic in a single question, and try to avoid giving a neutral middle answer option when you can. (In both of these cases, it's very hard for you to actually learn anything from the data.) And of course, if you work with communities who speak languages other than English, find a way to translate your survey into their language. Translation is a much tougher process than it should be, but it is essential for making all of your families feel valued and for hearing from your entire community, not just one subset. All that being said, I feel the same way about you - my colleagues, clients, and readers. I want to know what's important to you and what would be helpful for me to cover on the blog. I hope that you've been inspired by this post and will take my brief survey below. I appreciate your feedback and will use it to generate future content for you!
Growing up in New Jersey, the day after Labor Day always marked the start of a new school year ... and the day I finally got to wear the new outfit I had carefully planned and crack open my new, pristine notebooks.
If you couldn't tell, I have always loved the excitement of returning to school. Unfortunately, for many children, families, and educators, this year felt different. Some of the usual excitement and jitters have been replaced by trepidation about what to expect from a year like no other. Concerns about health and safety, academic progress, and schedule juggling have been abundant in my conversations with teachers and the staff and family surveys I have analyzed. So how will schools and districts know if they are adequately addressing their stakeholders' fears? Well... they've got to ask them. Colleagues in a number of recent conversations have been discussing the use of continuous improvement cycles. If you're not familiar with continuous improvement, its hallmark is the Plan-Do-Study-Act (PDSA) Cycle. 
Alicia Grunow of the Carnegie Foundation for the Advancement of Teaching explains the PDSA cycle:
More simply put, schools and districts need to:
Then, the cycle starts all over again ... quickly. We're not talking about huge, multi-year studies here ... This is a relatively quick and simple process! Make a plan, implement the plan, figure out if the plan worked, and if not, adjust and try again! With school kicking off, schools and districts have already put a short-term plan in place and are putting it into action. And this year, short cycles of trial and error are going to be key, as even our modes of schooling could change multiple times throughout the year. So how can schools and districts get feedback from their stakeholders NOW to see if their plan worked? Instead of a lengthy formal survey, think of creative ways to ask for feedback:
Asking one or two questions at a time in interactive ways will make it easy for stakeholders of all groups to participate, prevent them from getting tired of surveys, and give you real-time data about how people are feeling.*
*Just make sure the platforms you choose will allow for translation. Now here's the kicker: once you collect data, you have to complete the cycle ... ACT! Make it clear for students, families, and staff that you valued their feedback and are going to put it to use ... and tell them how! Start this crazy school year off right by lifting up the voices of your stakeholders in fun and easy ways and demonstrating that their feedback will guide your next round of planning and action. What are your creative ideas for hearing from stakeholders? Share them in the comments!
I've written a lot lately (here and here) about WHY tracking your school or organization's data is so critical right now.
As schools begin to reopen, I've heard from educators of so many different configurations for what school is going to look like: some totally virtual, some in person, some in a hybrid model... That's a lot to keep track of! It's hard enough to make sure that no child falls through the cracks when school is operating as usual -- now that task seems herculean. So how can we be sure that we know where each student stands, even if we don't see them every day? How can we know that we're doing our best to meet their needs (and that we even know what those needs are)? That, my friends, is where some simple data tracking strategies come in. Here is one trick to help you assess, at a glance, which students or families need a little more support: conditional formatting in Excel. Conditional formatting allows you to set criteria to automatically color-code or highlight values that you need to take action on. You'll see that my (totally fictitious) dataset below is just a typical spreadsheet -- nothing stands out at a glance, and it's hard to see which of these students really needs the most attention. We can use conditional formatting to visually highlight some of the students who need some support. We can find it by clicking selecting the text we want to format, and then clicking on the circled icon below. 
Once you click on the icon, you'll see a drop-down menu pop up with all of the options you have for automatically color-coding your data. I think the easiest way to start is by selecting "New Rule."
In the menu that pops up to manage your rules, you'll set the criteria that will determine what gets color-coded and how you want to code it.
This is where things get fun. 
The simplest option is "Classic." I always choose "Format cells that contain" and set either a text or numerical value that I want to highlight. Below is how I highlighted the students whose Quarter 3 grades were below a 70.
And voila! Now you can see, at a glance, the students who were struggling in Quarter 3. No fancy statistical skills needed!
I could also create other rules for my high-performing students or those ones that fell in the middle of the class. 
There are TONS of other ways to conditionally format your data -- here are two other examples for the same data that address the whole range of grades.
On the left, you'll see that instead of the Classic mode, I used an Icon Set to do a little traffic light system for the student grades. On the right, I used Data Bars to help me see students' grades on a scale from the lowest to highest score.
With these formatting options, I can see the students that might need some extra support... and it only took a few clicks and a decision about the point at which I'd be concerned about a student.
This is just one strategy of many that can help you quickly and easily to make sure no students are falling through the cracks. Want to learn more?
None of these work for you? Schedule a free 30-minute consultation with me so we can figure out how to best get your data tracking system started!
When I started my doctoral program at Vanderbilt, I certainly didn't expect to get into a ... heated discussion, shall we say? ... with the professor of my first course.
We were discussing characteristics of effective leaders, and our professor mentioned that the Myers-Briggs Type Indicator, one of the most well-known personality tests, was essentially worthless. You see, despite its incredible popularity, there is actually no data to show that Myers-Briggs is a valid and reliable assessment -- that it measures what it intends to, and that you'd consistently get the same outcomes if you took it again and again. Now, I've always been a pretty introspective person, and I (still) love personality tests as a fun way to reflect on how I think, feel, and interact with others. I'd never taken them as a scientific assessment of my psyche, but Myers-Briggs especially had stood out to me as a somewhat revelatory framework for why people interact and act the way they do. I had always gotten the exact same result when I'd taken the Myers-Briggs (ENFJ, if you're curious), so when my professor started talking about how most people get quite different results each time they take it, and that there was no research to support its utility, part of me was bummed, and part of me was fired up. I argued (civilly, of course) that I didn't use it as a formal diagnostic tool, but instead as a helpful resource or an interesting way of looking at things. So why should it matter? (Newsflash: It does matter.) For fun, I recently read The Personality Brokers: The Strange History of Myers-Briggs and the Birth of Personality Testing by Merve Emre. Of course, she confirmed what my professor had said many years ago. However, it reminded me of something I see often in education. People who are passionate about helping children and families often feel that they KNOW that what they're doing is helping the communities they serve, even without any real data to back it up. We KNOW that our Family Science Night was a success because there were lots of families there, and everyone enjoyed themselves. We BELIEVE that a teacher is effective because the children love them. We FEEL the impact of an after-school program because, well, it's been in the community forever. Unfortunately, we can't rely on gut instincts, feelings, and beliefs alone to tell us if something is effective... just like I couldn't make decisions based on only an affinity for Myers-Briggs. Let me be clear: education, and family engagement in particular, tends to get kind of fuzzy. While we can't rely on intuition, it's also true that we can't rigorously test everything that happens in schools. We need to find a middle ground. But this isn't just my random interest in personality theory. When it comes to children and families, we need to make sure that what we're doing to try to help them actually works. Luckily, it's not that hard to get started. We can begin tracking data, analyzing trends, and ultimately, measuring our impact so that we know we aren't just THINKING that we're changing lives. We actually are.
I love a good spreadsheet. I mean, I really get excited about it. You may have read on the About page how my business evolved from the development of a really fancy spreadsheet. True story. Now I get to help others learn to use Excel to improve their work and watch them get excited about it too.
One positive thing to come out of the pandemic is an increased appetite for online professional development. Recently, I've gotten to connect with old friends and colleagues by providing a three-part Excel workshop series for the Family League of Baltimore. So on top of hanging out with my old network, I've gotten to teach them about all the fun stuff Excel can do. Win-win! Here's an overview of the series: Part 1: Excel Basics A lot of educators just haven't been trained in how to use data. They may be consumers of it, using someone else's spreadsheet to glean information, but often, they just don't know how to utilize Excel's features for themselves. The Excel Basics workshop starts from the top and discusses formatting, functions, and formulas that beginners can use to build their Excel capacity. Part 2: Creating and Using Templates in Excel In the engagement world, there is so much to track! This session built on what was covered in the Basics session and walked participants through the process of designing their own customized tracking sheets. We used breakout rooms to discuss how to track different topics, and we walked through some more advanced features and functions to make these tools as automated as possible. Part 3: Reporting and Visualizing Data in Excel Data visualization is a hot topic in evaluation right now, and I get why. When you're able to effectively show your data graphically, you can make your results accessible for a much wider audience. In this session, we talked about so many fun parts of Excel - PivotTables, creating charts and tables for reporting, and ... drumroll, please ... creating interactive dashboards! Did you know you can create dashboards like the one below to share with your team? 
Here's what some of my past workshop participants had to say about their experience:
Besides my obvious bias towards using Excel for ... well, everything ... I think it is even more important now for schools and districts to be effectively tracking their work. As we navigate through so many unknowns with school reopening, it will be critical to keep an eye on students who are at risk of falling through the cracks. Good news - Excel can help (and so can I!). I'd love to bring this workshop series to more places, so if your team could use a bit of an Excel boost, let's talk!
I admit: I didn't think it was that hard to get a survey translated.
Over the past few weeks, I learned just how wrong I was -- and ate a big piece of humble pie in the process. With colleagues, I'm working on a landscape analysis of how families and educators in California feel about family engagement and the state's requirements for incorporating stakeholder feedback into district plans for improvement. We're designing a training program around these topics, but to make sure our program will be relevant, we wanted to hear from the people who would be participating in it. We designed a survey and planned for focus groups, and I naively thought we were good to go. Although Baltimore has a growing -- but fairly localized -- population of English Language Learners, the families at the schools where I worked were predominately Black and English-speaking. When I worked at the district, we had a cadre of interpreters we regularly contracted with for events, and we used large-scale survey software that easily facilitated (mostly adequate) translations. So when we decided to translate the California survey into nine additional languages, I didn't anticipate just how difficult that would be. Our survey was fairly basic and brief, so I built it out in Google Forms ... only to learn that despite the widespread availability of their free translation technology, there was no mechanism for translating surveys in their tool. (I'm honestly still scratching my head about this.) The most straightforward (ha!) way I found to create a multilingual survey in Google was to independently translate the survey into each language, build a separate page in the survey for each language, copy and paste each line of the translated surveys, and then use skip logic to direct people to the page with the language they selected. Umm what? We gave up on Google. We found out that our client had a Survey Monkey account that included the ability to create multilingual surveys. I was excited. Finally - a logical way to complete this seemingly simple task! Nope. I was still wrong. While this platform at least offers a dropdown menu of languages on the survey page (thereby making it easier for respondents and avoiding the skip logic silliness on the back end), it turns out that this paid feature was just as cumbersome to use as the Google option. What I ended up having to do was download a coded text file for each language, pay to independently translate each of the languages (Thank you, Stepes Translation, for coming to our rescue!), copy and paste each line of the translations into specific sections of the text file, and then upload the translated file to the system. NINE TIMES. With my hand cramping from all of that Ctrl-C and Ctrl-V action, I was stunned by how technically difficult and frankly, inaccessible the survey translation process was. Who is actually going to go through all this? More importantly, what does this mean for the voices of those who are not native English speakers? Without access to a large, institutional subscription to a powerhouse survey software, my gut tells me that very little translation is likely to happen. As a result, many important voices are being silenced. I don't have a solution to offer here, but I'm glad that this is a lesson I learned. This has opened my eyes to the institutional roadblocks that prevent equitable language access in our country... and I know I've just scratched the surface. Translation services, albeit not 100% reliable, are widely accessible and free online, yet they are not integrated into lower-cost survey platforms. This not only causes a huge headache for survey designers, but it inhibits the ability to hear from non-English speakers about important issues. As I seem to say in a lot of my blog posts, we have to do better. If anyone has a better solution than the relay race I just ran, please share in the comments! I do hope that a more accessible and user-friendly option exists.
It's easy to feel discouraged and upset when you turn on the news these days. So much is happening to progress the fight for racial equity and justice in this country that even a global pandemic seems to have taken the backseat. The truth is that without a precedent for our current events, we are all making sense of them as they come. We face tremendous uncertainty in the days and months ahead, especially in the education sector. No one knows what school should or will look like when the 2020-2021 year begins in the fall. And that is scary.
But here's why I'm feeling encouraged. Without a doubt, the Black Lives Matter movement is bringing critical and often unheard voices to the forefront. I'm also starting to see this happening in schools, with many districts really lifting up the voices of parents and families as they make decisions for what reopening schools will look like. I've seen multiple districts just this past week sharing surveys with families about reopening. How can we truly serve children and families if we don't know what they fear, what they want, or what they need? So if your school, organization, or district is trying to imagine what school will look like in August, and you haven't talked to families and students, now's the time to use some simple evaluation strategies to give power to their perspectives. Here are a few tips to get started. Think about what you really need and want to know from your stakeholders. Make a list of what your team is wondering about or what the impact of proposed plans might be before you draft your survey questions. For example, many districts are considering alternate schedules to accommodate all students in socially distant ways. Here are a few things to think about:
Encourage your survey respondents to commit to an answer. Whenever I take a survey, and I don't really know or care about the answer, I always select the non-committal, middle option. Most people do - it's human nature. However, during this especially important time, we can't risk having a whole bunch of middle of the road responses. Consider using a four-point (instead of a five-point) scale that encourages respondents to indicate if they're feeling negatively or positively about what you're asking. Instead of a neutral/not sure answer choice in the middle, have them choose from a scale like this: Strongly Disagree, Disagree, Agree, or Strongly Agree. Include a "not applicable" option if you feel that's relevant - we don't want to force answers that don't make sense - but this type of scale will give your team a better sense of which way your stakeholders are leaning and help you make more informed decisions. Make it equitable and accessible. Hopefully it goes without saying that not all students and families can access a survey that is online and only in English. To embrace and reflect the diversity of our nation's school districts, we must try to reach our stakeholders in multiple ways. Of course, an online survey is the easiest way to collect information, and many families can at least access the internet on their phones. However, some families cannot, and to truly understand what your families and students are feeling about reopening, we need to try to reach them as well. Think about mailing surveys or distributing them at food giveaways or other local gathering places. Or, if you're unable to translate the survey into every language spoken in your district, hire a few bilingual staff members or outside interpreters to do brief phone surveys with families whose native language isn't English. For the most successful reopening possible in the fall, districts need to know what families and students are thinking now. Brief surveys are an easy, cost-effective way to reach a large percentage of your stakeholders. Schools and districts need to think creatively to hear from as many families as they can and make their understanding of student and family needs as inclusive and diverse as possible. You'll be amazed at how much more informative your results can be! |
AboutThe goal of this blog is to highlight relevant issues that impact students, families, and communities and spark engaging discussions about how to address those issues through evaluation. Categories
All
Archives
October 2020
|
RSS Feed
|
Get insights and updates right in your inbox. Thank you!You'll be hearing from me soon! |