Disseminating research design and practice

I spend lots of time thinking about the best ways to disseminate research to impact practice. Here’s what Audrey Waters and Drew Davidson have to say about the topic.

Posted in Big Ideas, Small Talk | Tagged , , , , | Leave a comment

Improvement research

This post is about a type of research that’s applicable to all of us – improvement research. Teachers, admins, informal educators, designers and policy makers: keep reading because we can ALL use improvement research to assess changes in practices and get better at what we do.

Even though I’m a researcher, “improvement research” is still new to me. I used to think almost any research could count as improvement research. Don’t we all hope our research helps to improve something – student learning, teacher practices, or design of learning tools? After attending the Carnegie Foundation Summit on Improvement in Education earlier this month it’s all become much clearer. Both improvement research and traditional academic research are incredibly important for understanding, supporting and improving education and learning, but they are not the same.

What’s improvement research?
Improvement research is done to produce and assess changes in practice. Unlike traditional academic research, the goal isn’t to produce and test theories about the relationships between conceptual variables. Instead, it’s about testing hypotheses about how our processes affect outcomes we care about. For example, a teacher might hypothesize that using iPads to play a math game (the process) might help her students to understand how to add fractions (the outcome). Improvement research could be done to iteratively refine the use of iPads and the math game to support the learning of fractions. The teacher could share findings with others colleagues to improve practices across the school district.

Improvement research is designed to help make changes in practice, which means it’s very practical! It doesn’t have to involve excessive amounts of quantitative data (woohoo!), since you only need “just enough” data from small samples. Data collection can fit into existing routines, such as asking students to write down the answer to the same question before leaving class each day, in order to assess a specific learning goal. Anyone interested in improving education and learning, can make small and frequent iterations and use simple measures like this to assess change during practice.

What’s unique and accessible about improvement research is that it doesn’t require special training or large amounts of time or data, so anyone can do it – teachers, administrators, game developers, non-profit organizations…anyone, to assess changes in practices.

An example…
Several years ago I was a literacy coach in Chicago Public schools. Part of my job was to build literacy support tools that were embedded in science curriculum, and to support educators through observations and coaching. We started using these strategies in Environment Science classes, but eventually scaled to Chemistry, Physics, and Biology courses. I needed to know how the literacy tools were used to support student comprehension of content and how the design and use of the tools could be more effective. Based on evidence, the teachers and I made changes on a weekly, if not daily, basis to try and improve the work we were doing.

How did I know what changes to make? How did I know if the changes were effective in improving practices? I focused on asking one question, and I listened. I asked teachers to reflect on the conversations students had during the activities using the literacy tools. I also listened to student conversations and documented the types of questions they were asking. This information was critical for adapting the literacy strategies and how they were used.
We need precise aims and measures to improve our work.
Improvement research involves a continuous cycle of planning, doing, studying, and acting (PDSA).

Modified from Grunow, A. (2014). Measurement for improvement. Improvement Science Basics Workshop. Carnegie Foundation Summit on Education Improvement. March 10, 2014. Carnegie Foundation for the Advancement of Teaching.

At the start of each PDSA cycle, three questions should be addressed:

1. What are we trying to accomplish? Ideally, in improvement research we need a precise aim we’re trying to reach. This includes how much change, by when, and for what/whom. For the literacy strategy example an aim might have been: By the end of the semester, at least 6 higher-order thinking questions are asked during each reading activity that involves the use of the annotation literacy support tool in the Environment Science 101 course at Obama High School.

2. How will we know that a change is an improvement? We also need precise measures, which can be quantitative or qualitative, for assessing change. One measure of change in the science classrooms could have been the number of higher order thinking questions asked during a class period, tallied up on the chalk board by the teacher or an observer and tracked over the semester. Just answering one question each day can make it clear how changes are resulting in improvements (or not). One thing that this example reiterated for me is that measuring improvement and changes in processes is a lot easier when the data collection strategies are embedded in work we’re already doing.

3. What change(s) can we make that will result in an improvement? Changes can be made at various stages and to different parts of a system, but in this example they were primarily related to design (e.g. the literacy strategies) or implementation (e.g. how the strategies were used in class).

Improvement research is something all of us can do. Improvement research and traditional research both play an important part in improving education and learning. You don’t have to be a trained researcher or an expert on learning theory to do improvement research. You just need something you want to improve. That means that we all play a critical role in research, not just the researchers. To learn more about improvement research and strategies for scaling best practices, check out some of the useful resources from the Carnegie Foundation.

A version of this post was originally published on WorkingExamples.org.

Posted in Big Ideas | Tagged , , , , , , , , | Leave a comment

You Say You Want an Innovation: Practical Steps Towards Improvment – Louis Gomez

Louis Gomez gave a wonderful keynote at the Digital Media and Learning Conference last week. He talked about the importance of sharing how we do work and how we “make stuff happen”. This is something we think about a lot at Working Examples. It isn’t enough to just share what we create. Without understanding how work gets done, we can’t bring innovations to life and we can’t replicate or scale innovations. So, how do we share our process (i.e. the how) and continuously improve practices?

Louis starts speaking at 51:55 so you might want to skip ahead! He also talks about some of my work from the Family Share Project starting at 1:24:03.

Posted in Big Ideas, Conferences | Tagged , , , , , , , | Leave a comment

Conference strategy 101

It’s taken me years of attending conferences to realize I need a strategy. Sure, I planned what sessions to go to, but that was the extent of it. Being a relatively shy and sometimes awkward person, I avoided talking to people I didn’t know. I thought I could get everything I needed from listening to presentations. Sometimes I’d leave conferences really inspired, but most often I was bored and frustrated.

No matter what your reasons are for attending a conference, with a little planning you can get a lot more from the experience, personally and professionally. There are some things I’ve learned that have helped me to have more positive and productive experiences at conferences:

Don’t let the big names intimidate you. Most of them are normal, humble people who are just as interested in learning about you and your work. Think of them as friends, not superstars.

Schedule coffee chats or lunch in advance. Don’t underestimate the importance of networking. Is there someone you’d like to meet or talk to? Email them in advance and set up a time to chat. This will help structure a specific conversation and won’t leave anything to chance. I used to think it was so weird to email someone that didn’t know me, but it turns out they’re usually flattered and very interested in talking.

Plan what sessions to attend. Read through the schedule ahead of time. What sessions offer unique perspectives or information you can’t access from reading or surfing the web? Who is presenting that you might want to connect with?

Ask questions. If you’re shy or intimidated by breaking the ice with strangers, start a conversation by asking questions. You can actually carry a conversation for a long time by only asking questions! This is usually a good back up plan for when I’m nervous or just don’t feel like talking (it’s so exhausting for us introverts!).

Set goals. This has probably helped me more than anything. There is no right or wrong way to do this, but here are examples of some I’ve had in the past:

Introduce myself to 2 strangers today. It may sound crazy, but some of us have to start with the basics.
Take notes to share with my team back home.
Attend sessions that will help me to learn something I can directly apply to my work.

Take breaks.  Don’t try to attend every session. Your brain will turn to mush very quickly. Schedule in some “me time” or a quiet coffee break. Consider skipping a few sessions and attending social events instead. For me, that’s sometimes been where the best connections are made and most interesting conversations are had.

Now, to start planning which conferences to go to this year!

A version of this post was originally published on WorkingExamples.org.

Posted in Conferences | Tagged , , | Leave a comment

Pushing the boundaries of academic publishing


As a recent PhD grad, I’ve spent a lot of time thinking about what it means to publish as an academic.  I know I’m supposed to publish in peer-reviewed journals, the gold standard for academics. There’s often a silent attitude that writing for other mediums is a waste of time. Even books are considered second tier in many fields, especially for young scholars. For those who don’t work in academics, peer-reviewed journals require all submissions to go through a blind review process in which experts in the field anonymously critique the research ideas, methods, claims and the importance of the work. Only the highest quality research, making the most contributions to the field, should be accepted and published in the journals, although, arguably that isn’t always the case (e.g. due to political forces or inattentive reviewers). While journals typically contain the best research, the downside is that they’re inaccessible and expensive.

Publishing in peer-reviewed journals is incredibly important and can determine an academic’s future. Employment and tenure can be determined by the number of published peer-reviewed journal articles, and the quality and prestige of the journals they’re published in.

But we (academics) have a responsibility to do more than publish in peer-reviewed journals. That means writing in more accessible spaces and using accessible language for practitioners, designers and researchers alike. It means sharing our process, challenges, and lessons learned, rather than just finished results. The hope is that more people can learn from our work and build on it in interesting and meaningful ways.

Academic publishing is still really important, but we need to push the boundaries of what it means to publish as an academic. The peer review process and blind review have their merits, but could we benefit more from seeking open conversations about our work and crowd-sourced reviews?

There are a lot of unknowns about the future of academic publishing; however, changes are slowly on their way (e.g. Open access). Whatever the future brings, we need open dialogue about the research we’re conducting.

What do I mean by open dialogue? I’m referring to making research visible and accessible by sharing ideas and work-in-progress and inviting others to provide feedback and new perspectives. Not only do we need to make research visible and accessible, but also the conversations, feedback, and critiques relating to the work.

There are lots of options for creating open dialogue: blogs, commenting spaces, and of course Working Examples. Honestly, it doesn’t matter what the medium is as long as the space can serve as a hub for vetting ideas and crowd sourcing reviews.

In case you need some more convincing that open dialogue is really important, check out Henry Jenkins post from 2008 about why academics should blog. Jenkin’s post is blog-specific, but the rationale can apply to having an open dialogue in general.

It can help us reach a broader audience, including non-academic experts, the media, teachers, and other people our work is trying to reach. It’s not just an avenue for sharing information, but sharing it in a way that more people can resonate with. It’s a flexible space where we can express passion, feeling, and sense of discovery much easier than in academic publications. We can draw readers in with images, videos, and other media we can easily create and share.

It can help build our reputations as researchers and young scholars can build their reputation earlier in their careers. I imagine someone would have to do something really wacky or unethical for it to hurt their reputation.

It can make us proactive, rather than reactive to the media and other’s work in the field. This form of “just-in-time scholarship” means our ideas are heard in the moment rather than several months or years down the road, as is the case with publishing in peer-reviewed journals.

Additionally, it’s an invitation for feedback. It’s a great way to access new perspectives and work through challenges during a project rather than afterwards.

PhD student Megan Farnel is one scholar using a blog to vet ideas (and possibly crowd source reviews down the road). Recently, she began blogging about her dissertation, including notes to her committee and all the nitty-gritty details that go into developing a research study. Likely this will help her to document and make sense of a very complicated process and study, get feedback, and build credibility in the field. But, it’s also valuable for the rest of us. We can learn from her experiences, build on her ideas, and, of course, share them with her and the community.

Maybe writing blog posts and sharing Examples about our work won’t help us get jobs or tenure now, but I suspect soon it will – especially in departments that are forward-thinking and eager for change. Most important, open dialogue can give us something that a faculty job and tenure might not be able to provide: satisfaction that we’re doing everything we can to disseminate our work and make an impact.

A version of this post was originally published on WorkingExamples.org.

Posted in Big Ideas | Tagged , , , , , , , , , | Leave a comment

The open web & privacy: Highlights from Mozfest


I’m fully recovered from my very short (and intense) trip to London for Mozfest and am eager to share some insights. This was my first time at Mozfest and I’m happy to say it lives up to the hype. Forget the stuffy lectures – this event was all about brainstorming, collaborating and making. It was an amazing collection of creative thinkers and do-ers! Here are a few things Mozfest-ers got me thinking about:

Open access, open source, open data, open web…what’s the difference?
I’m a big advocate for open access. But, I’m embarrassed to say, I haven’t thought much about how “open web” or “open data” pertain to “open access” and “open source”. Mozfest gave me a crash course on the relationships between these terms. Here’s my take:

Open access refers to unrestricted web access to research, including peer-reviewed scholarly journal articles, theses, or books. Open access journals, like the Journal of Media Literacy Education are important for making research and information easily accessible for everyone.

Open source is a development model that encourages 1) sharing a product’s design (code, blueprint, etc.) through free license and 2) collaborating online to improve a product or design that is freely redistributed and frequently updated. Mozilla Firefox is an open source browser that has been modified to build new applications, such as Songbird. As a result of open source, better products can be created.

Open data. There is a belief that data, specifically government and scientific data, should be available to everyone to use or republish without copyright or patent restrictions. Data.gov provides data from the U.S. government in hopes of making the government “more transparent” and to encourage creative use of data to drive innovation.

– Open web. Open access, open source, and open data are all part of the open web. According to Mozilla rep, Santiago Hollmann, the open web is defined by 1) decentralization, 2) transparency, 3) hackability, 4) open development, and 5) accessibility. The open web is critical for pushing internet tools and content in new directions and for empowering many groups and individuals (not just companies or the government).

The Mozilla Open Badges Initiative promotes an open web by encouraging individuals or organizations to create, administer and verify badges. Check out this video from Doug Belshaw to learn more about open badges:

Don’t wait to share your ideas and work
So much great work is being done to make the internet open, accessible and meaningful for everyone. I went to a session hosted by the Open Knowledge Foundation, OpenStreetMap, MozillaWiki, Wikimedia UK, and Creative Commons where we discussed how to improve collaboration and sharing among communities who take more “open” approaches.
Something we all agree on: Share your ideas, challenges, process and products early – don’t wait till you’re “finished”! Sharing openly and early on will result in more innovative products, better research, and improved learning experiences.

Privacy concerns and open web
I’ve been wondering how the open web movement will impact our privacy. Turns out, many others are thinking about this too. There was a thread of Mozfest sessions dedicated to privacy, and during the festival Mozilla announced Lightbeam, a Firefox add-on that shows the relationship between the sites you visit and third-party sites, making our interactions and experiences much more transparent.

Through myshadow.org, you can calculate your “digital shadow” and discover ways to reduce traces of your activity. Check out the game Data Dealer if you’re interested in learning more about what’s done with your information online.

For more highlights from Mozfest, check out the Mozfest Tumblr page.

A version of this post was originally published on WorkingExamples.org.

Posted in Conferences | Tagged , , , , , , , , , , , , , , | Leave a comment

Connecting families and classrooms

I defended my dissertation this week…and passed!

Here are the slides from my presentation.

Take a look at my working examples to learn more about the Family Share Project and the research study. If you’re really inspired you can read the whole dissertation (hah!). I have a few publications in the works too…more about that later.

Posted in Big Ideas | Tagged , , , , , | Leave a comment