In my last post, I talked about how the recent American Rescue Plan Act will bring an influx of funds specifically for out-of-school time (OST) -- after-school and summer -- programs, as well as for community schools and wraparound services.
This is a huge win for those working tirelessly in family engagement and OST!
I also mentioned that to be on the safe side, it's a good idea to start building your evidence base now, in case these funds are earmarked for evidence-based programs under the Every Student Succeeds Act (ESSA).
Now that you're all familiar with the four evidence levels, let's dig into the most accessible one: Level 4.
So many grass roots, small, community-based organizations are at a disadvantage with ESSA's evidence requirement. Here's why: 1) evaluation services can get expensive, and 2) they often require technical know-how or an outside consultant to do them well.
Program staff are great at working with kids, families, and schools. That's why they do this work! They didn't sign up to be evaluators, so I get why the thought of doing an evaluation can send some program staff running for the hills.
But let's take a deep breath.
Here's the great news about Level 4: if you know that your organization is planning to evaluate your family engagement or after-school services but hasn't done so yet, you can demonstrate that there is a great likelihood that your services are impactful and still get access to those Title I and other federal funds.
That's it - demonstrating a likelihood! It's a great way to get your foot in the door with districts while working towards the bigger goal of becoming evidence-based.
So you may be saying - Amanda, that sounds great, but how do I show that my services are likely to have a positive impact on kids and families?
Here's what you need to apply for ESSA Level 4 approval:
1) A logic model for your organization
Essentially, logic models are a depiction of what you put into your program (resources, activities), what you hope to get out of it (short- and long-term outcomes), and how you'll know you're on track to do that (measures, benchmarks).
Check out my post about the ins and outs of logic models here.
2) Citations demonstrating the impact of similar programs
We can use online tools like Google Scholar to find existing evaluations and research studies that show that similar programs serving similar groups of kids or families had a positive impact.
So, if you're a program in a major urban center and you find a study demonstrating the effectiveness of a small, rural initiative, it's probably best to keep looking. We want to compare apples to apples here.
You'll also have to make sure that the studies you find meet the ESSA standards described in my last post.
3) A plan for your future evaluation
All you need to do is put together a plan for how you are going to measure your program's impact in the future. You'll have to share who you'll study, what you'll look at, and when you'll conduct this research.
You may need to chat with an evaluator for this one. Don't worry though - evaluations don't have to be a multi-year, super expensive endeavor!
I hope you noticed that none of those three requirements needed any program data!
So if you're started to track your family engagement or student data, Level 4 gives you time to get your systems up and running, while still giving you access to the funds you need and the students and families you want to work with!
If you want to know how to DIY the ESSA Level 4 process, sign up below for Evidence for Engagement, the free mini course from Tamara Hamai and me!
With weekly videos and worksheets, it will walk you through how to get your application ready for your local school district and get your foot in the door.
Here's a sentence we don't often get to say in education: the motherlode of funding is coming our way!
The recently-passed American Rescue Plan Act has set aside ... brace yourself ... over $2 BILLION for out-of-school time (OST) -- after-school and summer -- programs!
Not to mention, there is funding for community schools and all the wraparound services that so rarely get enough attention or funding but are absolutely critical for bolstering families in underserved communities.
This is a game changer for kids, families, schools, and OST/community school providers.
But we can't rest on our laurels and wait for the money to rain down on us. (Wouldn't that be nice?) It's time to be proactive!
I've had a lot of conversations with OST folks recently about the Every Student Succeeds Act (ESSA)'s evidence requirements.
Basically, if a school or district is going to purchase a program or services with federal funds (ie. Title I), they need to make sure that there is some evidence to show that what they're purchasing is effective.
Makes sense, right?
Unfortunately, that's not as easy or straightforward as it sounds.
Many small, community-based, minority-owned organizations don't have the capacity or funds to hire an evaluator. For me, this is a serious equity issue.
But for no money, there is a way to get your foot in the door.
To be deemed evidence-based, a program needs to meet one of the following levels:
For most programs, Level 4 is a natural place to start. With a little bit of guidance and planning, you can be on your way to accessing Title funds!
Now, the American Rescue Plan Act does not seem to specify that spending is limited to evidence-based programs ... but why hurt your chances of getting access to this lifeline?
Now is the time to position your program for maximum benefit from this upcoming funding opportunity.
If you want to get more information about Level 4 and becoming evidence-based, sign up for the FREE mini-course that Tamara Hamai and I developed. Each week you'll get emails with videos and graphic organizers to help you get ready to become an evidence-based organization!
It's hard to believe that a year ago this week, my husband and I returned from a trip to visit my family and went into lockdown for the first time.
At that time, the thought of staying at home for even two weeks straight sent my anxious brain into overdrive.
Now, it's been a year since we've been in a restaurant, a year since I've been to the gym, and a year since I've seen most of my friends. Somehow, we've made it through this year of change, fear, and loss.
On a more positive note, it's also been a year full of learning and growing as people and as educators.
In reflecting on this past year, here are the biggest takeaways that (I hope!) the education world has realized:
1. Families are critically important partners in the work of educating children.
In a recent teacher focus group, the challenges of connecting with families this year were their biggest concern.
No longer can we pretend that families and schools exist in separate planes.
Educators have seen just how important family engagement is for getting their own work done.
Teachers (and other critical school staff!) and families MUST be equal partners in the sometimes herculean task of educating children, especially those who come to school with a variety of needs and challenges.
2. Teachers and families both need to be commended for how well they adapted to educating children at home.
I am constantly in awe of teachers this year -- teaching is the hardest thing I've ever done, and I cannot imagine having to change basically every skill and strategy you've honed throughout your career in a flash. Their work was more challenging than ever during the pandemic, and in many cases, they put their health on the line to do the best they could for their kids.
Now, my dog is the only kid I've had to contend with this year (although he does make working from home challenging at times!). So I have great admiration for parents who have found ways to adapt their schedules, learn how to teach their children, and maintain positive, loving relationships in an unimaginable situation.
From the surveys and focus groups I've done over the past year, it's been clear how hard this has been for educators families, and both groups deserve our respect and gratitude now more than ever.
3. We can no longer rely on our memory or conversations with peers to know who we're reaching and who we're not.
I'm sure you anticipated this lesson ... but we MUST use data to drive our supports for children and families! Without kids and families in school buildings every day, a different approach was critical to figure out who was getting what they needed ... and who had barriers that the school should help families overcome.
I've gotten to work with a number of different schools, districts, and organizations on their use of data this year, so imagine my PURE JOY to hear that a staff member at one of these schools said that with the use of the data tracker we developed, "We are no longer guessing."
That's just it! We don't have to guess.
We can use simple, low- or no-cost strategies (Don't forget to download my free guide here!) to make sure we know EXACTLY what's happening with each of our kids.
No one's falling through the cracks on our watch.
By engaging families as equal partners, respecting the hard work that both educators and families contribute to the education of their kids, and by easily tracking how our families are engaging (sometimes more importantly, how we're trying to engage with them), we open up a new world of home-school collaboration and possibilities for success.
In the coming months, as we start to transition back to some semblance of normal life, I hope that we don't regress and forget all that we've learned this year.
I'm excited to find new ways to support educators and families in their journey towards more equitable, data-informed partnership.
I have a few things up my sleeve for 2021, so stay tuned!
Let's see how we can all continue to grow and learn together in this next phase of the year.
"Ugh, they're making us submit a logic model? What is the point of a logic model anyway?"
"I don't understand all that technical jargon. What am I supposed to put in a logic model?"
These are the thoughts I imagine my clients having when a funder or state department of education requires them to create a logic model ... and rightfully so.
Logic models are often overly complicated, far too technical, and not connected to ACTUAL practice - so it's no wonder that they are not intuitive for people doing "the work" in education.
It doesn't have to be that way though. Let's talk about what a logic model actually is and how it can help your organization.
I found the following definition of a logic model on the Community Tool Box (emphasis added):
“A logic model presents a picture of how your effort or initiative is supposed to work… Effective logic models make an explicit, often visual, statement of the activities that will bring about change and the results you expect to see for the community and its people. A logic model keeps participants in the effort moving in the same direction by providing a common language and point of reference.”
Here's what I love about this definition:
1) It clearly conveys that a logic model is a visualization of how your program operates, and
2) It helps teams see that a logic model can actually streamline their work and make sure everyone is on the same page.
The image below from the CDC is another great way to think about what a logic model can do.
So instead of an annoying task you have to complete for grant funding, think of a logic model as a dynamic map of your program and how you will collaboratively work towards achieving your goal.
Recently, I introduced this new way of thinking to a school district team I'm working with.
I'm designing a data dashboard (learn more here!) for them so that all of their family-serving teams can share data and serve their district's families more efficiently. (I'm really excited about this.)
When we met with each team individually, they were all saying the same things about how they engage with families and what data they already or want to track.
Yet, collectively, they couldn't see the forest for the trees. They didn't know how similarly each team was operating!
Let me be clear: this wasn't because they were not communicating or working together. It's because they didn't have a framework to guide their collective work and show where there was overlap across project teams.
So we got to work. Using Google Slides, we did an interactive work session where the teams brainstormed what they would put in each part of a logic model. Below is a screenshot of their "Activities" brainstorm.
Then we did a virtual "gallery walk" so they could see how much overlap there was. See how many "I do this too" stars there are in the image?
After this, it took no time to put together their ideas into a more traditional logic model format.
If your team is currently struggling with making a logic model, don't be afraid!
Reframing how we think about logic models can go a long way towards making them purposeful, usable tools to make our family engagement work more effective.
When I was a senior in high school, I thought the coolest job would be to serve as a member of Congress.
I was lucky to be part of an amazing course (in a public school, might I add!) called Government and Law-Related Experiences, affectionately known as GALRE.
Our teacher, Doug Martin, a hero of mine, is a Vietnam veteran who continued to serve his country by educating thousands of students about their civic rights and duties, in honor of his friend who didn't make it home from the war.
He taught us to be good citizens, but more importantly, he modeled for us how to be good people and to enjoy life in the process.
We had frequent guest speakers in the course -- GALRE alumni, community leaders, and elected officials on all levels -- who would come talk to us and answer our questions.
We also had to complete "outside experiences," where we immersed ourselves in the local community and political scene. I was canvassing and registering voters before I was old enough to vote myself.
The class culminated in a three-day, whirlwind trip to Washington, DC, where we got to see the federal government operate in real life.
GALRE inspired me to pursue a degree in Political Science and spend my career in service of others and the common good.
As I watched the horrific news of the insurgence of white supremacist domestic terrorists on the nation's Capitol yesterday, I found myself thinking back to our trip to DC, sitting on the steps of the Capitol building with my classmates, imagining working there one day.
More importantly, I keep thinking how safe I felt being with a teacher like Mr. Martin during times of national (9-11 occurred when I was a sophomore) and even personal times of crisis.
That level of security -- that my teachers could help me process what was happening and reassure me that, by learning from history, we would pull through any challenge -- is something I wish for all students.
Mr. Martin was a huge inspiration for me when I became a Social Studies teacher in Baltimore.
I brought in guest speakers, helped my students participate in a civics education competition, and took them on field trips to see where history happened in their local communities. Following the guidance of a veteran teacher at my school, I tried my best to teach my students the "real" version of early U.S. history and impart the lessons I learned from Mr. Martin.
As I watched yesterday's events unfold, I kept thinking about today's generation of students and all the challenges they have faced over the past year.
I worry about the quality of social studies education they are receiving.
Are they receiving the white-washed, textbook version of history that will only perpetuate the bigoted culture that we're seeing on display?
Or are they learning about the suffering and triumph of people from all racial and ethnic groups, the truth about how government has perpetuated inequity, and how they can play a role in making things better?
Worse, are they learning about history at all?
Most of all, I find myself wondering if they have a Mr. Martin in their lives to help them feel safe and make sense of what's tearing our country apart during this scary time.
I may not be a social studies teacher anymore, but the lessons I learned in G.A.L.R.E. and my time in the classroom have informed my work today.
I truly believe that we can all begin chipping away at the ills of our society through our own actions: how we support others, how we engage and empower children and families, and how we ensure that everyone gets what they need to thrive.
To all the social studies (or other!) teachers having difficult conversations with their students today and making them feel safe in an increasingly unstable world, thank you. You are making a difference.
It's the end of the year, when we reflect on the past year and look forward to new beginnings.
I haven't done any rigorous data collection about this, but I think it's fair to assume that most people would rate 2020 as a giant dumpster fire.
So let's take it back to 2019.
Last year was challenging for me in a different way. I had moved to Ohio from Maryland the previous year, and I naively thought that I could transition my business to my new home with relative ease since most of my work was remote.
As it took time for new connections to materialize into new contracts, I knew that things weren't headed in the right direction.
Now, I had a few metrics I used to measure my business:
At the time, those weren't pretty.
However, what was most telling for me was how I FELT.
I was discouraged, uncertain, and anxious. And in terms of my business, I didn't know what to do to make it better.
Certainly, the quantitative data was informing how I was feeling, but the numbers alone did not compel me to act.
I just wanted to feel better! I knew it was time to make a change.
Around that time, I met a new neighbor who specialized in website design and SEO. I heard about a marketing consultant whose approach resonated with me. I saw that the American Evaluation Association's (AEA) conference was featuring a lot of workshops and resources for independent consultants.
I sprang into action. I met with these new specialists, went to the conference, and got to work. And ultimately, I turned my business (and outlook) around.
While there is always room to grow, the numbers I mentioned earlier do reflect the changes I've made to my business. Yet I don't think they show HOW or WHY those changes happened.
From my neighbor, I learned something new and gained hope that some simple strategies could make my website work better for me.
From the marketing consultant, I felt understood and discovered a new way to communicate what I do and why I love it.
From AEA, I gained a large group of new colleagues, friends, and referral partners - but most of all, I felt accepted, validated, and supported.
To me, those feelings and networks are what helped me turn the page in my business - the fuzzy, not easily measurable, qualitative stuff.
Had I not reflected on those things, I might have stayed in my rut.
Maybe you already track your data, and maybe you don't, but if you're getting the feeling that something's not right, think about the qualitative data you can explore to see what's up.
How do your families feel when they interact with you? How do the staff feel? What is the tone of your interactions? How engaged are students in relationships with peers and staff and with their learning?
These things matter.
One thing I'm grateful for from 2020 is that a lot of educators are seeing just how vital family engagement is for student achievement.
So while we're reflecting on this crazy, crazy year, let's take a second to examine how our students, families, and staff FELT and how we helped them feel better.
If that's what we take with us into 2021, then I think we're off to a good start.
I started writing a completely different blog post for this week, but when I read the news this morning, I knew I had to shift gears.
CNN featured a story yesterday called "Teachers and social workers search for students who are 'missing' in the pandemic."
The word "missing" made my heart drop.
It immediately reminded me of this story, of a young girl from Washington, DC named Relisha Rudd. I heard about her story in 2014, and it broke my heart -- at the time, I was a community school coordinator and led our school's charge for attendance and engagement tracking and interventions. Staff from Relisha's school tried to track her down, only to find that a man working at the homeless shelter where she was staying had been impersonating her doctor to the school.
She is still missing to this day.
As a community school coordinator, her story lit a fire under me to do everything I could to make sure that we knew, to the best of our ability, that our kids were safe and able to come to school.
I have thought of Relisha over the years and find the tragedy of her story to be a call to action for schools, districts, and other youth- and family-serving organizations.
How can we make sure that no other students fall through the cracks?
When I read the story this morning about the Robla School District in California doing home visits and trying everything they can to find their students "missing" from online school, I had so many thoughts:
Labeling students as "missing" drives home the gravity of the situation our country is in. Families are truly struggling because of the virus and the economy, but honestly, lacking access to the internet, to stable housing, and to consistent work have been challenges for so many families for so long. The fact that things are only getting worse is upsetting and shows us that we have so much work to do.
Literally going into neighborhoods searching for children is heroic, but also emotionally grueling. I remember the disappointment and worry of having a string of unsuccessful home visits -- you gear yourself up for making a difference, only to find that addresses were incorrect or have changed, or worse -- you just don't know where students and families are living. That is scary, and it is emotionally taxing for educators.
While data tracking can't help us physically locate a family, it can help us focus our efforts where they are needed most. You may have seen that last week, I released a guide for how to Track Your Engagement Data in 4 Simple Steps. I believe strongly that using simple functions in Excel can help educators pinpoint exactly which students and families need additional support -- whether that's with attendance, engagement, or academics.
(I know it can work because I've seen the impact it has had on my own work in schools!)
So in honor of Relisha and in commitment to the well-being and success of students who may be "missing" from online school today, let's get tracking.
To learn more about data tracking, visit my Engage with Excel page or sign up below.
Growing up in New Jersey, the day after Labor Day always marked the start of a new school year ... and the day I finally got to wear the new outfit I had carefully planned and crack open my new, pristine notebooks.
If you couldn't tell, I have always loved the excitement of returning to school.
Unfortunately, for many children, families, and educators, this year felt different.
Some of the usual excitement and jitters have been replaced by trepidation about what to expect from a year like no other.
Concerns about health and safety, academic progress, and schedule juggling have been abundant in my conversations with teachers and the staff and family surveys I have analyzed.
So how will schools and districts know if they are adequately addressing their stakeholders' fears?
Well... they've got to ask them.
Colleagues in a number of recent conversations have been discussing the use of continuous improvement cycles. If you're not familiar with continuous improvement, its hallmark is the Plan-Do-Study-Act (PDSA) Cycle.
Alicia Grunow of the Carnegie Foundation for the Advancement of Teaching explains the PDSA cycle:
More simply put, schools and districts need to:
Then, the cycle starts all over again ... quickly.
We're not talking about huge, multi-year studies here ... This is a relatively quick and simple process!
Make a plan, implement the plan, figure out if the plan worked, and if not, adjust and try again!
With school kicking off, schools and districts have already put a short-term plan in place and are putting it into action.
And this year, short cycles of trial and error are going to be key, as even our modes of schooling could change multiple times throughout the year.
So how can schools and districts get feedback from their stakeholders NOW to see if their plan worked?
Instead of a lengthy formal survey, think of creative ways to ask for feedback:
Asking one or two questions at a time in interactive ways will make it easy for stakeholders of all groups to participate, prevent them from getting tired of surveys, and give you real-time data about how people are feeling.*
*Just make sure the platforms you choose will allow for translation.
Now here's the kicker: once you collect data, you have to complete the cycle ... ACT!
Make it clear for students, families, and staff that you valued their feedback and are going to put it to use ... and tell them how!
Start this crazy school year off right by lifting up the voices of your stakeholders in fun and easy ways and demonstrating that their feedback will guide your next round of planning and action.
What are your creative ideas for hearing from stakeholders? Share them in the comments!
I've written a lot lately (here and here) about WHY tracking your school or organization's data is so critical right now.
As schools begin to reopen, I've heard from educators of so many different configurations for what school is going to look like: some totally virtual, some in person, some in a hybrid model...
That's a lot to keep track of!
It's hard enough to make sure that no child falls through the cracks when school is operating as usual -- now that task seems herculean.
So how can we be sure that we know where each student stands, even if we don't see them every day?
How can we know that we're doing our best to meet their needs (and that we even know what those needs are)?
That, my friends, is where some simple data tracking strategies come in.
Here is one trick to help you assess, at a glance, which students or families need a little more support: conditional formatting in Excel.
Conditional formatting allows you to set criteria to automatically color-code or highlight values that you need to take action on.
You'll see that my (totally fictitious) dataset below is just a typical spreadsheet -- nothing stands out at a glance, and it's hard to see which of these students really needs the most attention.
We can use conditional formatting to visually highlight some of the students who need some support. We can find it by clicking selecting the text we want to format, and then clicking on the circled icon below.
Once you click on the icon, you'll see a drop-down menu pop up with all of the options you have for automatically color-coding your data. I think the easiest way to start is by selecting "New Rule."
In the menu that pops up to manage your rules, you'll set the criteria that will determine what gets color-coded and how you want to code it.
This is where things get fun.
The simplest option is "Classic." I always choose "Format cells that contain" and set either a text or numerical value that I want to highlight. Below is how I highlighted the students whose Quarter 3 grades were below a 70.
And voila! Now you can see, at a glance, the students who were struggling in Quarter 3. No fancy statistical skills needed!
I could also create other rules for my high-performing students or those ones that fell in the middle of the class.
There are TONS of other ways to conditionally format your data -- here are two other examples for the same data that address the whole range of grades.
On the left, you'll see that instead of the Classic mode, I used an Icon Set to do a little traffic light system for the student grades. On the right, I used Data Bars to help me see students' grades on a scale from the lowest to highest score.
With these formatting options, I can see the students that might need some extra support... and it only took a few clicks and a decision about the point at which I'd be concerned about a student.
This is just one strategy of many that can help you quickly and easily to make sure no students are falling through the cracks.
Want to learn more?
None of these work for you? Schedule a free 30-minute consultation with me so we can figure out how to best get your data tracking system started!
When I started my doctoral program at Vanderbilt, I certainly didn't expect to get into a ... heated discussion, shall we say? ... with the professor of my first course.
We were discussing characteristics of effective leaders, and our professor mentioned that the Myers-Briggs Type Indicator, one of the most well-known personality tests, was essentially worthless.
You see, despite its incredible popularity, there is actually no data to show that Myers-Briggs is a valid and reliable assessment -- that it measures what it intends to, and that you'd consistently get the same outcomes if you took it again and again.
Now, I've always been a pretty introspective person, and I (still) love personality tests as a fun way to reflect on how I think, feel, and interact with others. I'd never taken them as a scientific assessment of my psyche, but Myers-Briggs especially had stood out to me as a somewhat revelatory framework for why people interact and act the way they do.
I had always gotten the exact same result when I'd taken the Myers-Briggs (ENFJ, if you're curious), so when my professor started talking about how most people get quite different results each time they take it, and that there was no research to support its utility, part of me was bummed, and part of me was fired up.
I argued (civilly, of course) that I didn't use it as a formal diagnostic tool, but instead as a helpful resource or an interesting way of looking at things. So why should it matter? (Newsflash: It does matter.)
For fun, I recently read The Personality Brokers: The Strange History of Myers-Briggs and the Birth of Personality Testing by Merve Emre. Of course, she confirmed what my professor had said many years ago. However, it reminded me of something I see often in education.
People who are passionate about helping children and families often feel that they KNOW that what they're doing is helping the communities they serve, even without any real data to back it up.
We KNOW that our Family Science Night was a success because there were lots of families there, and everyone enjoyed themselves. We BELIEVE that a teacher is effective because the children love them. We FEEL the impact of an after-school program because, well, it's been in the community forever.
Unfortunately, we can't rely on gut instincts, feelings, and beliefs alone to tell us if something is effective... just like I couldn't make decisions based on only an affinity for Myers-Briggs.
Let me be clear: education, and family engagement in particular, tends to get kind of fuzzy. While we can't rely on intuition, it's also true that we can't rigorously test everything that happens in schools. We need to find a middle ground.
But this isn't just my random interest in personality theory.
When it comes to children and families, we need to make sure that what we're doing to try to help them actually works.
Luckily, it's not that hard to get started. We can begin tracking data, analyzing trends, and ultimately, measuring our impact so that we know we aren't just THINKING that we're changing lives. We actually are.
Ready to start your evidence journey?
Sign up below to get the Evidence for Engagement mini-course sent to your inbox.
You have successfully joined our subscriber list.
The goal of this blog is to highlight relevant issues that impact students, families, and communities and spark engaging discussions about how to address those issues through evaluation.