Understanding the IRB: How Does It Affect Us?

When I joined the INIT Lab, one of the first things I was tasked with was completing a series of IRB trainings. Like many others who don’t work in the research field (and had never done so before), I had no idea what the “IRB” was, or what purpose it served. I thought that it was just a set of rules and regulations we must follow to do our research. However, as I worked with the TIDESS group on several projects, I realized that, even after my trainings, I did not fully understand the purpose of the IRB. This may have been because I thought that most of the IRB information is hard to understand for someone who is unfamiliar with the field. The information presented within IRB trainings is verbose, and some of the terminology which is being used is specific to the research field. However, the thing that made understanding the IRB most difficult was my lack of context on what the IRB is, and the reasons it is important to our research. Topics such as privacy and safety are highly emphasized in the IRB trainings, but I could not fully understand the reasonings behind it, as I did not understand why safety/privacy was such a large topic within non-medical research labs such as ours. However, during my time as an undergraduate research assistant in the INIT lab, I have vastly improved my understanding of the IRB and the purpose it serves. In this post, I am going to explain exactly what the IRB is, and how it affects our projects on a large scale.

The IRB, or Institutional Review Board, is an administrative body that was created to protect the rights of participants in research activities. Each institution which is conducting research has their own IRB which reviews all research proposals, and either approves or denies them. Before anyone at an institution like UF can perform any studies/research involving human participants, they must get approval from the IRB. Every IRB is given latitude to interpret the federal regulations and create their own set of procedures. This blog post will cover what I have learned about UF’s IRB. Your own institution may do things similarly, or they may have differences.

The IRB review process begins with a research proposal, where a summary of the planned research is given. There are three categories of review that each proposal will fall into: Full Board, Expedited, or Exempt. An Exempt review means that the proposal will be reviewed by one IRB member, and is used when the proposal is low risk. An example of something that would be considered “Exempt” would be an anonymous survey. Expedited reviews are done on proposals which pose more risk than an exempt proposal (e.g., collecting height and weight data) and require either the chair of the IRB, or an experienced board member designated by the chair, to review. A Full Board review involves the entire IRB panel in the review process and occurs when there is “greater than minimal risk” involved in the study, such as in a study testing new medications. The risk level that a proposal is given is determined based on how identifiable the data is that you are collecting from the participants, what are the real and likely physical, emotional, or psychological risks of participating in the research, and whether any of the participants belong to any protected groups (like children) that may be vulnerable to coercion.

The research that we do as a part of the TIDESS project (and most INIT Lab research) typically falls into the Expedited category, as it poses no more than minimal risk to the participants. But what exactly does it mean for research to have “risk”? Research which many would think is perfectly safe, such as studying interactions with our touchscreen interfaces, actually involves possibly collecting many types of sensitive data from our participants. Even things such as contact information sheets can hold confidential identifiable data such as phone numbers, full names, and even addresses. However, following the IRB regulations assures that this information is seen only by those who need to see it and is stored in a secure environment. If information like this were to be handled unsafely, it could lead to personal data being leaked to the public.

One of my first moments seeing these regulations in action occurred when analyzing data of participants interacting with our sphere prototype (see Pufferfish). As part of our analysis, we were going to look at the videos of these participants interacting with the prototype, alongside written transcripts of everything they said. When I received access to these transcripts as a member of the team, every participant name was already replaced with a participant number, concealing any personal data that may have been present in the videos. Alongside this, the videos were stored on an encrypted hard drive, set to delete itself if the person who tried to access it did not have the correct password. Later, when making a presentation about the videos to the lab, we blurred out the faces of all the participants, when possible. All these security measures were done even though this information was just being utilized in the lab, and not in any way accessible by the public. If we were to have given this presentation to the public, we would take even more care to make sure anything that can be traced back to the participant would be removed. Any instances of participant names would be replaced with a number, and all faces would be blurred or even removed if possible. All audio would be checked/removed so that no personal information (names, etc.) is revealed. This would be done to make sure that nobody can use the information to identify the participants, fulfilling the privacy and protection goals of the IRB.

Another important factor of the IRB process is known as informed consent, where the participant is told exactly what the purpose of the study is, and the procedures they will have to do to participate. The goal is to ensure that they understand all the potential risks and benefits of the study to be able to decide for themselves if they would like to participate in the study. The participant has the right to withdraw from the study at any time (or choose not to participate), and the informed consent process assures that the participant is aware of their rights and understands them fully.

As a first-year student at UF, and a newcomer to the world of research studies, my time with the lab has changed my thoughts on the IRB. I realized that what I once thought were just excessive rules (e.g., blurring faces for in-lab presentations) are vital in protecting the participants of our studies. Privacy and protection plays a huge role in the research process, one which I was not initially aware of. The data we have access to is sensitive, and the IRB is there to assure that it is kept secure, in order to prevent dangerous consequences to our participants. Overall, the IRB allows us to conduct research in a way that prioritizes the safety and privacy of participants, and assures that they understand the full scope of what it is they are participating in.  

Advertisements
Posted in Uncategorized | Leave a comment

Strategizing the CSCW Revise and Re-submit submission process

This year, on the TIDESS project, we submitted a research paper to the ACM Conference of Computer-Supported Collaborative Work and Social Computing (CSCW). Unlike the five-page abstract we previously submitted and presented at NARST, for this submission we prepared a full-length paper (typically 13-15 pages). CSCW is a peer-reviewed conference seeking to publish research on topics related to collaborative and social computing. Unlike other ACM conferences such as the Special Interest Group on Computer-Human Interaction (SIGCHI), papers submitted to CSCW undergo two rounds of review cycles. After the initial round of review, a research paper can either get a revise and re-submit notification, or a reject notification along with detailed reviews from the reviewers. Reviewers are other people in the research community, generally experts in a specific research area, who read the submission to evaluate it based on its novelty, study methodology, and contributions to the research community. The research paper we submitted to CSCW got shortlisted for the second-round revision cycle, and we were given a two-week period to address comments provided by our reviewers. All the reviews we were given from our reviewer panel were very insightful and constructive, which helped us strengthen our paper. These reviews were specifically targeted towards ensuring that our research questions, motivation, and situation within the previous literature were clearer.

However, addressing all the reviewers’ comments, which required significant re-structuring of the paper, along with ensuring that all the research team members were on the same page with the way revisions are being incorporated, was a challenging task to complete in two weeks. Therefore, we followed a four-phase iterative process to address the reviewers’ comments as listed below:

(1) Conceptually grouping the reviews: To make the revision process smoother, we first took a detailed pass of all the reviews line by line and conceptually grouped these reviews based on the section of the paper they were targeted at. For example, if the reviewers’ comment said, “The paper needs to identify a gap in the existing studies”, we grouped it under a higher-level theme called “ground work in the literature.”

(2) Team discussion and planning: The second phase involved all the research team members walking through the conceptually grouped reviews together and discussing potential ways in which we can address the points raised by the reviewers. This phase mostly consisted of team discussions and planning. Conceptually grouping the reviews by paper sections helped us visualize all of the comments related to a particular section and facilitated the development of optimized revision strategies.

(3) Prioritize the reviews: Given that we had two weeks to address the suggested changes provided by our reviewers, and we would not have time to address every single comment, it was very important to strategize our revision process. We did this by making sure that the major revisions suggested by our reviewers, alongside revisions that require significant restructuring of the paper, were our priorities. In addition to ranking the revisions, we also documented our strategies related to how we were going to address the review and what places in the paper needed modification to accomplish this. This planning was especially necessary to make sure that the paper flowed well after the revision.

(4) Prepare a response document for reviewers: CSCW also asked authors to prepare a response document for the revise and re-submit cycle which described the changes made in the paper and the rationale behind these changes. Within this document, we made sure to highlight all the changes we made in the paper both at high and low levels of detail as we addressed reviewers’ comments. This document helped to show the reviewers where we have strengthened our paper.

I am a 2nd year Ph.D. student in the Human-Centered Computing Ph.D. program at the University of Florida. This was my first time submitting to CSCW, and I enjoyed the entire iterative process of revising and resubmitting the paper. Personally, addressing reviewers’ comments during the revise and re-submit phase helped me learn a lot about how to write a strong research paper. I think that approaching the revision process in a more strategic way definitely helped me and my team to come to a consensus and discuss the changes we were planning to make to the paper based on the reviews. Unfortunately, our paper ended up not making the final cut to be accepted, but we were able to resubmit to another conference recently and believe our paper was much stronger thanks to the thoughtful iteration from the CSCW community.

Posted in Uncategorized | Leave a comment

TIDESS Museum Learning Project Update: PufferSphere Prototype Development

The TIDESS Team has been investigating the natural and intuitive ways users interact with spherical displays by continuing to prototype on the PufferSphere. In my understanding as a new member of the team, the TIDESS project primarily aims to investigate ways in which children and adults interact with large touchscreen interfaces, such as spherical displays, to help us design interactive touchscreen exhibits for public science learning. More specifically, the project focusses on studying new types of touch interactions, or gestures, that allow users to manipulate and explore content displayed on the sphere.

To allow us to study how users interact with the PufferSphere and investigate natural user gestures, our team is in the process of developing a PufferSphere prototype to support future studies.  I have been primarily responsible for developing this prototype while part of this project this summer. In development of the prototype, I have faced issues with distortion of content shown on the sphere or more generally displaying 2D images and videos on a three-dimensional space. A simple solution that I have found is to limit videos and images to small windows on the sphere’s surface to decrease the effect of the distortion. I also learned that, when prototyping, especially on a new platform such as the PufferSphere, it is good practice to start your program with basic concepts and slowly add components. I found this idea to be helpful because, frequently, objects appear differently on the sphere than the desired outcome, and this allows for quick correction of any errors while preserving the already working components.

Currently, I have been attempting to add elements to the prototype that make it more visually appealing and provide motivation for participants to complete the study (e.g., “gamification” elements as popularized in Brewer et al, IDC 2013). I have also been working on logging touch data for the prototype. The PufferSphere already records touch information such as the start and end of a touch, as well as the longitude and latitude of the touch on the sphere. We want to record these pieces of data, with the time they occurred, in a CSV file to obtain the speed of a swipe/drag gesture or the length of time that a touch lasted. Logging touch data allows us to calculate this information after the study to do analysis across users. Other improvements to the prototype will be made after running pilot studies.

As an student from the IMHCI REU Program in the INIT at University of Florida, I have had many learning opportunities this summer. First and foremost, I have learned to be more proficient in task management. This was a necessity in my first research experience as I quickly saw that some of my assigned tasks had priority over others and deadlines, while at other times, I had independent/free time to read research papers relevant to the project. In the beginning of the summer, I was also timid to collaborate with others frequently. Now, I find it very casual to ask peers and mentors questions and even coproduce works for a multitude of assignments.  I am currently a third-year student at Elon University majoring in Computer Science (BS) and Mathematics (BS) with a minor in Physics. As the summer continues, I look forward to seeing how children will interact with the sphere during our study sessions.

 

References

  1. Brewer, R., Anthony, L., Brown, Q., Irwin, G., Nias, J., and Tate, B. Using gamification to motivate children to complete empirical studies in lab environments. Proc. IDC’2013, ACM Press (2013), 388–391.

 

Posted in Uncategorized | Leave a comment

Back in the swing for fall

Over the summer, there were several changes to our TIDESS project team. We said goodbye to Carrie, who won a prestigious award to help her finish her dissertation (defense scheduled for October 25!), and Peter, who graduated. We welcomed two REU students, Ian and Saylie, and had temporary help from grad students Shae, and now Sarah. We even said goodbye to an advisory board member, Betty Dunckel, who retired from a long career at FLMNH, and welcomed Julia Plummer from Penn State in her place, who we met at NARST. Whew!

It was a busy one, though, as we resubmitted a paper to a conference, the first time we’ve had a revise-and-resubmit process for computer science conferences. We also submitted our first paper on the education side of things for International Journal of Science Education, Part B. This was based in large part on the work we presented at NARST. We also submitted as part of a symposium of posters on scientific practices in informal science education settings for NARST 2019. We submitted our second NSF report, and we ran a gesture elicitation study, all while continuing to develop the software and exhibit for the sphere.

Now that fall semester has started (it’s already week 4!), our smaller team heading into our third year (the no-cost extension year of the project) will be focusing on wrapping up what we can of these studies and planning for future work. Hannah and I are really digging into the design of our next embodiment study, with Carrie’s help as she is available. We are hoping to get to run an in-museum study this spring or next summer, but with Betty’s departure from the museum, things there have been in flux in terms of personnel. So as always, stay tuned!

Posted in Uncategorized | Leave a comment

TIDESS Goes to NARST!

 

Pages from 2018_NARST_Clickable_Conference_Program

Recently, two TIDESS project members (Principal Investigator Katie Stofer and I) had the opportunity to attend and co-present our work at the NARST Annual International Conference in Atlanta, Georgia. In general, conferences provide a great opportunity to share what you’re working on and to get valuable feedback, to network with colleagues and identify opportunity for future collaborations, and to learn about associated work.

NARST describes themselves as “A Worldwide Organization for Improving Science Teaching and Learning Through Research”. This particular conference brings together educators and researchers who work on science learning from many different angles. The conference content is arranged by different “strands” and had 15 of these, including: History, Philosophy, and Sociology of Science; College Science Teaching and Learning; Cultural, Social, and Gender Issues; and many others. Our presentation was within Strand 6: Science Learning Within Informal Contexts. “Informal Learning” is a phrase researchers and facilitators often use to describe education and learning experiences that take place outside of formal structured contexts like classrooms. Other terminology, such as “free-choice learning”, can also be used to describe the process (you can read Katie’s paper detailing some of the nuances of this terminology here). Our work fits within this topic due to our focus on science museums.

For this particular presentation, we focused on the science content and science practices we saw discussed by participants within the TIDESS tabletop Before giving this presentation, we developed and submitted a five-page abstract that summarized some of the key ideas we would be presenting. This abstract was then peer-reviewed before we were accepted into the conference. I reference the process briefly in my post here. Both the process of developing the extended abstract and putting together the presentation has put us well on our way to writing up our results for future publication in an education journal. Another one of our current team members, Hannah, describes some of her experience with expanding the results into a more formal paper in her recent post.

We also were able to catch multiple exciting presentations in our own strand and others, such as Environmental Education and Educational Technology, incorporating ideas that might be of interest within the TIDESS project. For example, several other presenters were also exploring the concept of embodiment within their own research.  It was helpful to identify people within our specific research community working on the topic and to see how they are navigating the role of embodied cognition in a variety of different contexts.

Keep an eye on the blog as we will have upcoming posts detailing how our research is progressing and will undoubtedly have more news about developments from the human-computer interaction part of our team!

 

Posted in Uncategorized | 2 Comments

Getting my Feet Wet

The Fall semester got off to a quick start with the TIDESS project! New to the project, it was exciting getting up to speed. To introduce myself, I am a junior undergraduate student majoring in Political Science and Sociology. Just like Carrie, I might seem like an unusual fit for the TIDESS project, but with my studies in sociology, I am excited to apply my knowledge of how people engage and interact with their environments. I am interested to see how users learn and make meaning of the world around them by using data visualizations, and how they make connections to our planet by referring to their own experiences.

I entered the project in the heat of the team finishing a peer-reviewed conference paper submission. For the paper, I had the opportunity to work on the background section on embodied cognition, a theory we are using to ground some of our interpretations, which was a bit of a challenge. The education team and I spent a lot of time discussing what embodied cognition means to us and how users used embodied cognition while interacting with the tabletop. To find evidence of embodied cognition in the tabletop study, I had my first attempt at coding qualitative data. In a past research project I worked on, I practiced the fundamentals of coding, but it was an informal practice. With the TIDESS project, I put this idea to work formally. To find examples of embodied cognition, the education team studied transcripts from the study in order to pull examples of conceptual metaphors—a crucial part to identifying embodied cognition. It was a challenge at first to agree on what we thought were strong examples of embodied cognition, but once we narrowed it down to the two most apparent metaphors, we were able to define embodied cognition more clearly. Once we came to agreement, our second round of coding was much easier, and our team pulled strong examples for the paper.

It was a great learning experience to see how the conference paper was formed. Even though I have read various research papers before working on this project, I never knew how research papers were structured. I was also surprised at how much editing goes into formal research papers. After the first conference proposal, I have shifted my attention to writing a research paper for the NARST conference. The NARST conference is specifically for work on science education research. So far, I have had the opportunity to expand on our approved abstract and take my first couple attempts at writing the paper.

Looking forward to this spring semester, I am eager to get involved with the focus groups for our sphere study and possibly start a project of my own. With the upcoming focus group results, we will continue our research on museum exhibits while incorporating a physical model, the sphere, into our study. Since this study is focused on what people know or want to know about museum exhibits and oceans, I want to see how people conceptualize our global ocean systems without the use of an exhibit. Some brainstorming ideas include having participants draw a map of our oceans or color a map to portray their perceptions of ocean temperature. Clearly, a lot of research and planning still needs to be done to this study. Overall, I believe having an understanding about how people view our ocean systems can help us learn what would be effective in an interactive ocean museum exhibit.

Posted in Uncategorized | Leave a comment

One Way We Learn: Embodied Cognition

In a previous post, we mentioned that the TIDESS project had begun data analysis on our study to describe how people learn from interactive data visualizations. One of the underlying theories that we draw on to inform this work is the theory of embodied cognition. Embodied cognition is the idea that our body shapes our mind, or even performs some of the same meaning-making processes as our brain does. In other words, our body interacts with the environment, and we learn from that experience. Well, what does that mean exactly? In his paper, Embodiment and embodied design, Dr. Dor Abrahamson gives a great example:

File:A dog plays on a seesaw with children in Scotland,.jpg

Credit: William Reid, National Geographic

Imagine a child standing, as in the picture above, at the center of a seesaw. When this child shifts his weight from one side to another, the seesaw tips toward the side with more weight. Through embodied cognition, the child is learning about an idea of balance. He is physically experiencing that adding more weight to one side of the seesaw causes that side to be heavier than the other and thus move downward.

Although the example above involves learning from a physical activity, embodied cognition can also be involved for learning in other modalities, such as language. One key figure in the development of embodied cognition was George Lakoff, a professor at the University of California, Berkeley. He discusses in his book, Metaphors We Live By, that language is shaped from our body as well as shaped from our mind. For example, when we think of being “happy” or “sad”, we can associate those two emotions with the words “up” and “down”, respectively. That’s why we hear some people say, “Cheer up, mate” or “I’m feeling a little down today”. This association of up with happy and down with sadness is known as a conceptual metaphor. These metaphors shape our perception of our feelings, and thus we learn to use certain words associated with physical, bodily experiences to describe our emotions and experiences.

Within the context of learning, there are numerous studies that demonstrate the power of embodied cognition. By engaging learners through physical demonstrations and exercises, rather than stating information as in a lecture or text, learners are able to learn the concepts better. Embodied cognition also goes hand in hand with learning through visualizations. For instance, in our recent study, our interface displayed visualizations of Earth’s ocean temperatures. We then had our pilot participants engage with this interface. Our pilot participants used their perceptions from their senses and acted through their gestures in ways that could aid learning. It is these tasks that we hope to understand in our study.

Both our tabletop and sphere provide us the interactive technology for our participants to engage with and learn about the Earth’s oceans. As a senior in Biology at the University of Florida who joined the project this past summer, I’m fascinated to see if participants learn more about geoscience through embodied cognition by using their senses and performing actions on our tabletop and sphere.

Posted in Uncategorized | Leave a comment