I naively said to my family a few weeks ago that I was lucky that my business wouldn't be affected in the way that so many restaurants and other small businesses are by the pandemic. I couldn't see then, but it has become incredibly clear since that every business and every industry have been and will be impacted on a massive scale by this pause in how we normally function. I am so fortunate that I can continue much of my work from home; yet every day, my heart breaks for other small business owners who are doing everything they can to keep their businesses afloat during this unprecedented time. 

I spoke to one of my former clients recently, and I expressed how badly I felt that the cancellation of schools would drastically affect his work. His response to me was this: "This is a time to pivot!"

All week, I've been thinking about what he said and what a resilient attitude he had. I've also been pondering how this affects not only my work but the organizations and communities I serve. Take the issue of chronic absenteeism in schools. Chronic absenteeism is typically defined as missing 10% of more of a given school year, and it has been empirically tied to a host of negative outcomes for students, including reading levels, special education identification, suspensions, dropout rates, and more. Students from vulnerable and underserved populations are at the highest risk. The good news about this issue is that it is both preventable and reversible, and I've found in my work that regular and collaborative data tracking on student absences and related interventions can make a huge difference for kids and schools. ​A recent article, Chronic Absenteeism in the Time of Coronavirus, discussed the implications of shuttered schools on how schools and systems typically address attendance issues, but also on what closed schools mean for the students themselves. ​

In thinking of how to pivot from the typical accountability measures associated with attendance, as Jordan's article suggests, how can we in the education field work together during this time to address the root causes -- the underlying reasons why so many students miss school? I am loving all of the positive news articles out there about school districts employing bus drivers to deliver meals, offering wifi hotspots to those without internet access at home, and teachers driving around students' neighborhoods to help them feel connected. 

From my lens, I think this break from traditional schooling is an opportune time to go deep with our data and determine all that we can about which students are missing school the most and why. Once we've done that, we can get creative about our interventions for kids while they are in their homes and communities. I've got a few ideas in development for how my work can pivot to best serve our schools, districts, and non-profit organizations who have the most direct lines to children and families (more information on that to come!). In the meantime, if your organization is trying to better support its chronically absent students or more effectively engage with families during this time, let's chat. Shoot me an email here so that we can set up a conversation. 

How are you planning to pivot? Share your ideas in the comments below - I'd love to start a conversation and channel our collective creativity!

I've been training in Muay Thai (kickboxing) for a few years. I am always learning something new and being pushed outside of my comfort zone ... and I love it. However, I am and have always been a perfectionist. It's something I've struggled with my whole life: sometimes, I'm really proud that I've been lenient with myself, and other times, the perfectionism rears its ugly head. Lately, I've noticed it manifesting at the gym. As I'm trying to apply a new skill in work with a partner or coach, I've been getting frustrated and self-critical. My self-protective instincts (ironically not working to appropriately block a punch or kick) have made me think, "I don't like this aspect or that skill," instead of allowing me to see that this is a process of growth and that there is no place for perfection in that process.

I think organizations (and the people within them) can be the same way when it comes to evaluation. We get used to our routines, we think we've perfected them, and then one of a few scenarios happen that push us out of our comfort zones. Maybe we are required to learn a new system or skill, or -- even worse! -- we get feedback that doesn't match our own perceptions. Now at the gym, my feedback can be a simulated sparring round that doesn't end so well for me. But in our workplaces, while we are focused on serving the people we care about, feedback that we're not doing so well is upsetting to hear and painful to accept. That upset and pain is followed by questions -- "What could we do differently?" Why is so-and-so doing well at this when we're struggling?" or even "Is this feedback accurate or reliable?" Our self-protective instincts kick in.

The anticipation of negative feedback -- in whatever form -- is a huge barrier for people (including myself!) to try new things, reflect on their own performance, or seek help and other perspectives. Certainly, the accountability culture in education has only made these innate fears and insecurities worse. 

Today at the gym was different though. The past few days, I've been more reflective about why I'm getting so frustrated and how that is keeping me from truly learning and growing. So today, I tried to pay attention to the moments when I got frustrated (ie. I collected some data on myself!). I worked with my coach to talk through those negative feelings and develop some strategies I could try in those situations. Then, I practiced and stayed open to more feedback... and by the end of the session, I felt more resilient and confident in my skills than I had in awhile. 

Terms and methods like "continuous improvement" and "improvement science" get used a lot in both education and evaluation, and they are proven methods for making institutional (or personal) changes on all levels. I'm sure that what I did at the gym today was just a tiny Plan-Do-Study-Act cycle. Yet for me, these formal frameworks for self-assessment and reflection can sometimes be hard to grasp - and they can feel like another thing we're accountable for doing. However, we can look at them more simply: sometimes, all we need to do is recognize that we're passionate but not perfect, allow ourselves to be open to feedback, and develop an authentic plan for how we can improve. This is true for individuals and organizations.

As an evaluator, I love the moments when conversations about data lead to a-ha moments instead of feelings of defeat. (Data visualization is especially helpful here.) Sometimes, when we take a step back and think about why we're assessing or evaluating, we can see that it's not all about accountability and funding requirements (and not about our individual or collective insecurities either). Sometimes, it is just about putting our guard down (or up, if you're at the gym), remembering that we can always do better, and learning to see our imperfections as a sign of growth in the making.

I had some great conversations this week with colleagues about establishing a culture of data in organizations and training organizations who are new to evaluation and data. These conversations reminded me about one of my favorite old blog posts, that I originally wrote for the National Association for Family, School, and Community Engagement (NAFSCE) in 2017. Given this week's discussions, it felt like a good time to bring it back into the rotation (with a few updates!). 

When I was a teacher, conversations around instructional data were baffling to me. Fresh out of policy school, I was eager to use what I had learned about data analysis to monitor how my students were performing, but as a social studies teacher, this task was more difficult than I had anticipated. I was required to keep a data binder, and administrators would periodically check to confirm that, well, it existed. However, I struggled to figure out what to put inside of it. My administrators did not help me understand how – absent standardized test data – I could track progress on specific standards outside of my grade book. It often felt like the conversation ended after the word “data” was uttered.

As I have focused my career on family engagement efforts, I have seen how conversations about using data to improve engagement are often greeted by the same blank stares I encountered as a teacher of a non-tested subject. On other days, talking about data elicits looks of panic or skepticism. At one particularly memorable training, community school coordinators were led in a debate about the utility of data. Sitting from my seat on the pro-data side of the room, I was amazed by arguments from the anti-data group. What resonated most is that these capable and talented colleagues understood data to simply be numbers on which their performance review was based, not as a tool to discover context and unlock insights about the families being served.

I think this belief system exists for a number of reasons. First, many educators are tired of increasing demands for data without sufficient training. Professionals need to understand how data can be collected, ways in which it should be analyzed, and how it can actually make their work easier. I have found that on-the-ground staff are often the last to receive the proper supports and professional development around understanding and using data. It becomes a symbol for all of the things we don’t like about accountability instead of the asset that it truly can be.

Perhaps more importantly, the work of engaging families – understanding needs, forming trusting relationships, and helping people when they are vulnerable – is incredibly difficult to quantify. Often, we know we have made progress or achieved results – not because of a spreadsheet or heat map – but because a family had enough food for the weekend or because a child stopped acting out as much in class. How do we tell those stories? How do we show our value as professionals when these important markers seem impossible to put into a spreadsheet? These are the critical questions we need to answer.

For these reasons, it is my mission to help educators and professionals realize that data does not have to be scary or intimidating. It does not require complex coding skills or mathematical know-how to track how clients are being served. If you would have been sitting across from me in the data debate, here are some tips to get you started:

• Start with what you have. If you are trying to get more parents involved at the school, it’s helpful to know exactly who is already coming to events. Try making a spreadsheet of the information from your event sign-in sheets and see what patterns you find. For example, comparing the names on this list to a whole-school roster can help you figure out which kids have had little to no in-person parental involvement. Use this approach for other measures that you can track from the information and documents already sitting in your office.
• Leverage the expertise around you. Everyone knows who the go-to person is at their workplace when they have a technology question. Maybe that person (or someone else) also has some knowledge of Excel or other tools. Anyone can fill in a template. See if a colleague can help you design the tracking tool you know you need but do not know how to create.
• Do not be afraid to play around and make mistakes! The best way to learn how to manipulate and analyze data is to get your hands dirty and play around with it. Try different buttons, Google how to do things, and ask what colleagues at other schools and organizations are doing. This is why there is an “Undo” button! Of course, if you are unsure, you can always make a copy of your file so the original data is safe.

Using your organization's qualitative and quantitative data can give you amazing insight into both the ongoing needs and continuing growth of the students and families you serve. With a little less reticence towards this approach, we can make a lot more progress in engaging families to help their children succeed.

Of course, if your organization is unsure of how to get started in this area, I'd love to be of assistance. Learn about the new Build Your Evaluation Capacity training package!

You may notice that the website has gotten a bit of an overhaul. I wanted to better communicate what clients get from working with me and what makes Structured Solutions different from other small evaluation firms. 

The biggest change is that I've restructured the services I provide. There are now three distinct packages that clients can buy -- all still customizable, of course -- that will help you achieve your data and evaluation goals. The graphic below gives a quick overview, but I go into more detail below:

All three packages are designed as steps that organizations can take towards telling a rich, compelling story about the work they do for students and families and the impacts they have on their clients. Here are the details about each package: 

BUILD YOUR EVALUATION CAPACITY
For organizations who are truly at the start of their data journey, this package is an ideal starting point. Your organization will get a customized training and follow-up technical assistance in the areas you need. The goal is for your entire team to leave the training with confidence that they can work with and use data like a pro. 

CREATE A CULTURE OF DATA
The second package builds on the first and will include the custom trainings and support from Build Your Evaluation Capacity. After your team has been trained, we will get to work on developing data systems and procedures that are accessible to all skill levels. Depending on your organization's needs, we will create simple, customized systems for data collection, management, or visualization that allow for sustainability and continued attention to data in the future.

MEASURE YOUR IMPACT
In order to truly tell your organization's story, you must first know what impact you're actually having. The third package gives you all of the benefits of the first two but will culminate in a full-scale needs assessment or evaluation. As this type of work tends to be highly individualized and ongoing, we will work together to design a package that fits your needs and ensures that any evaluative requirements from funders or supervising organizations are met. 

If your organization is in the beginning stages of its data journey and could use some support, I'd love to set up a conversation with you. 

It's funny where the most interesting and engaging conversations can happen. At the gym last week, I had a few random, but passionate conversations with other members of the educational (and martial arts!) community about youth experiencing trauma and how it impacts their ability to participate in school, learn effectively, and handle their emotions. 

Trauma is something I wish I had learned about when I became a teacher over ten years ago. Like many new teachers, I struggled with classroom management. It is probably not surprising to learn that as a 5'2", 22 year-old woman, I did not have a natural authoritative presence. However, there were many other reasons for the challenges I faced -- ineffective and harmful administrators, a lack of shared expectations among our middle school team, and my own battle with anxiety. These all contributed to what often felt like turmoil in my classroom. While I always empathized with the challenges my students faced in their own lives, I never fully understood the ramifications of the trauma and hardship they experienced -- nor did I know where to connect them or their families for additional support. 

I was lucky because the school had a wonderful social worker and part-time psychologist to whom I could refer students. They also served as great supports and sources of advice and knowledge for me as a new educator. Yet, there was little capacity and no infrastructure at the school for understanding and responding to trauma. I remember being told that students were experiencing homelessness, hearing stories about families who were involved in gangs, and seeing that students were extremely impoverished, but I had no tools to process these situations or fully support my students. I felt anguish about the situations they were experiencing, but I know I did not always respond effectively. 

Years later, after working in a community school and supporting many others, I know what more effective and comprehensive supports can look like. I have seen the benefits of wraparound services for students and families, including a full mental health team; meaningful enrichment and engagement opportunities for students and adults; connections to resources for basic needs such as housing, food, clothing, and adult education; efforts to track and review data on engagement, attendance, and supports provided; and most importantly, a loving and affirming approach to working with students and families with the greatest needs. Community schools -- schools that become a hub of the surrounding community and provide these wraparound services for students, families, and community members -- are well-supported by research. This model has been shown to be beneficial in reducing chronic absenteeism, improving school climate, increasing student achievement, and more. One of the findings from a 2017 Learning Policy Institute report states that:

"The evidence base provides a strong warrant for using community schools to meet the needs of low-achieving students in high-poverty schools and to help close opportunity and achievement gaps for students from low-income families, students of color, English learners, and students with disabilities."

​So when my friend at the gym, a high school guidance counselor, told me that his school effectively did not have any mental health clinicians available to students, I was taken aback. I talk to him frequently about his school and the amazing ways in which he supports his students, so I was shocked to learn that this large high school was so under-resourced in this area. The school seems to be on its way toward providing wraparound services, with a brand new food pantry and the dedication of counselors like my friend. Yet this conversation made me realize that the services I took for granted in Baltimore City schools -- which were still often insufficient to meet the high level of need -- did not exist everywhere.

This was a critical and humbling realization. Of course, if we could make students' barriers disappear -- or at least provide them with the resources that they need to overcome these barriers and thrive -- we would. But in the absence of a magic wand, what can we do? From my lens, this is where needs assessments and data tracking can play a huge role. Imagine if we collected stories from students, families, and staff about the challenges faced by the school community, each stakeholder group's perceived needs, and their recommendations for meeting those needs. Now think about if we used factual data about the community and student population to support those stories. What we would get is an intensely compelling, collective narrative about what this community needs and how its members feel those needs could be met.

Could a needs assessment or set of data instantly bring on a full slate of mental health workers at this school? Of course not. But when we tell our story effectively, people (read: funders, decision-makers, influencers) listen. And when people start listening, we can inspire them to make change. 

This was an unusual Thanksgiving for my family. My parents have always hosted Thanksgiving at their house; it is one of the few family traditions that remains since my Grandma passed away. All was set to continue as usual this year, until last week, when my Grandpa -- the effervescent, 89 year-old patriarch of the family -- began experiencing significant back pain. It quickly became evident that the trip to my parents' house would be too much for him. Like good evaluators, we continually monitored the situation and used the data we gathered to determine what changes we could make to ensure that the family would be together and that my Grandpa would be as comfortable as possible.  

We ended up bringing Thanksgiving to him. Different family members were nervous for different reasons. My parents now had to factor in a one-hour drive into their cooking schedule, and others had to adjust their travel plans. Some were skeptical about having Thanksgiving in a different location, and given that we all adore my Grandpa, we were worried about how he was feeling. Afterwards, when I reflected on how the day went, I realized that most of the questions on everyone's minds had been fairly quantifiable:
​

• Would the turkey be ready on time (It wasn't.) and survive the car ride? (It did, thanks to some duct tape and clever packaging on my parents' part.)
• Would we have the food warmed at my Grandpa's house in time for everyone's arrival? (We didn't.)
• Would all of our family members make it (We were down one cousin.) and arrive on time? (Everyone was early, except for my parents with the turkey that wouldn't cook.) 

To determine the success of our Thanksgiving dinner, we could track these metrics or others, such as the number of family members who came (11) or the amount of time everyone spent at my Grandpa's house (approximately three hours). These are all important details that help us craft the story of our Thanksgiving dinner. Yet these facts could never describe the contentment on my Grandpa's face as he sat at the head of the table and listened to the family's jokes and conversations. A series of data points would never suffice to explain how happy we all were to see him acting like his usual, wonderful self because he had just had a wonderful day. No spreadsheet in the world -- and I LOVE a spreadsheet -- could have ever captured the hope that my Grandpa had after a day with much less pain and far more happiness than he had had in a few weeks. 

Of course, I would never methodically evaluate our family dinner -- but isn't this exactly what we try to do when we try to measure family engagement in our schools? We host events and programs, offer services and supports, and build relationships like pros, but we struggle to figure out how we can capture the impact that we have made and share that impact with others. We count names on sign-in sheets and have participants rate their satisfaction, and we hope to show that we made a difference in our students' and families' lives. Those pieces of information are critical -- we need to track those metrics to track our progress over time and demonstrate growth in our programming and our reach within school communities. Yet, our standard documentation cannot capture the joy on a mother's face as she gets a relaxed moment to play and learn with her child or the pride that a father exudes when he has just completed his first professional resume. These are two examples from my work as a Community School Coordinator that have stayed with me to this day -- and neither came from one of my beloved spreadsheets. 

This is why stakeholder voices and qualitative data are so important. Certainly, if the turkey never cooked or if no one showed up, our Thanksgiving would have been very different. However, what made Thanksgiving so special for me was watching my Grandpa's joyful reactions during dinner and hearing him speak happily on the phone for days afterward. For those of us in the field of family and community engagement, these are the types of reactions we live for and the motivation we need to keep doing the tough work. To best capture our impacts and tell our stories, we must take this mixed-methods approach, enhancing our traditional measures with rich stories and perspectives from our most important stakeholders. For me, this is what makes evaluation -- even of a Thanksgiving dinner -- most powerful, interesting, and even fun. 

In my career, I've always identified as an educator first. I have proudly rooted my analytical and consulting practices in my experiences directly serving students, families, schools, and communities. However, as my career and business have evolved, and as I gained the additional identity of "evaluator," I always felt that I did not quite fit into that community. Sure, I have training and experience in quantitative and qualitative data collection, analysis, and reporting. Sure, I work to help schools, districts, and non-profits complete evaluation projects and build out protocols and procedures for continuing this critical work. Sure, my job title has even been "Evaluator." Despite these things, something did not click for me on a deep level.

I never doubted my ability to relate to and help my clients with their evaluation needs. It was the pressure to label what I do or who I am that has caused me a lot of stress. When I attended previous conferences about research and evaluation, I always encountered career evaluators, whose methodologies were their bread and butter, and who yearned to understand the newest and most advanced statistical methodologies. That's not me, and for a long time, I thought that was a problem. 

This past week, I attended the American Evaluation Association's Evaluation 2019 conference in Minneapolis. I was excited to see a number of sessions and activities related to independent consultants and to my other areas of interest in the field -- creating data dashboards and evaluating programs serving families, to name a few. I was looking forward to connecting to new people and meeting up with those whom I already knew. What I did not expect was an experience that transformed how I see myself within the evaluator community and how I can use my refined identity to better serve my clients. 

Here were my key takeaways: 

• I do not have to be an educator OR an evaluator. I can be both -- I AM both -- and that is what my clients appreciate about me. Having my feet firmly planted in two fields gives me a perspective that many evaluators and researchers do not have and a sense of purpose for why this work needs to be done. 
• Evaluation comes in so many forms. My favorite clients are not the people looking for me to do large-scale, highly-complicated studies. They're the ones who are figuring out where to start. Maybe in a few years, they will be ready for more advanced methods or a larger sample size, but for now, they see the importance of evaluating their work and need some support with figuring out where to begin. This work of capacity-building is what most excites me, and I now see that it fits squarely along a continuum of evaluation, on which all points are valid, important, and potentially transformational.
• Just like my clients, I too can evolve and grow my evaluation abilities. That is, perhaps, the most fun part of all. Maybe I can start learning how to do social network analysis and incorporate it into my business. Perhaps I can expand my reach into new audiences, like the informal learning community. I could also just continue to strengthen and enhance the ways that I already serve my clients.

Thinking of my business as malleable -- as an opportunity for innovation and as a dynamic, growing entity -- is exciting. With this realization, it feels like the possibilities are endless for Structured Solutions. Thanks to AEA and the wonderful people with whom I conversed and interacted this week, I am happily embracing my new all-encompassing identity. What is your evaluation identity?