Tag Archives: University

The Scientific Workflow: A Guide for Psychological Science

Leave a reply

When new students or postdocs enter into your lab, do you have a plan or a guide for them? I have a lab manual that explains roles and responsibilities, but I did not (until recently) have a guide for how we actually do things. So I’ve made it my mission in 2019 and 2020 to write these things down and keep them updated. The Lab Manual (see above) is about roles and responsibility, mentorship, EDI principles, and lab culture.

This current guide, which I call the Scientific Workflow, is my guide for doing psychological science.  I wrote this to help my own trainees after a lab meeting last where we discussed ideas around managing our projects. It started as a simple list, and I’m now making it part of my lab manual. You can find a formatted version here, and the LaTex files here.

IMG_20191029_075744

Nothing related to science here, but a beautiful picture of campus from our research building

Introduction

This is my guide for carrying out cognitive psychology and cognitive science research in my lab. The workflow is specific to my lab, but can be adapted. If you think this is helpful, please feel free to share and adapt for your own use. You can keep this workflow in mind when you are planning, conducting, analyzing, and interpreting scientific work. You may notice two themes that seem to run throughout the plan: documenting and sharing. That’s the take home message: Document everything you do and share your work for feedback (with the group, your peers, the field, and the public). Not every project will follow this outline, but most will.

Theory & Reading

The first step is theory development and understanding the relationship of your work to the relevant literature. For example, my research is based in cognitive science and I develop and test theories about how the mind forms concepts and categories. My lab usually works from two primary theories. 1) Prototype / exemplar theory , which deals with category representations; and 2) multiple systems theory (COVIS is an example) which addresses the category learning process and rule use. I follow these topics on line and in the literature.

Screen Shot 2020-01-18 at 6.49.03 AM

Paperpile is a great way to organize, annotate and share papers. See my article here.

You should keep up with developments in the field using Google Scholar alerts and its recommendations. I check every week and I recommend that you do as well. We want to test the assumptions of these theories, understand what they predict, test their limitations and contrast with alternative accounts. We’re going to design experiments that help understand the theory, the models, and make refinements and/or reject some aspects of our theories.

  • Use Google Scholar to find updates that are important for your research.
  • Save papers in Paperpile (or Zotero) and annotate as needed.
  • Document your work in Google Docs (or another note taking app).
  • Share interesting papers and preprints with the whole lab group in the relevant channel(s) in Slack.

Hypotheses Generation

Hypotheses are generated to test assumptions and aspects of the theory and to test predictions of other theories. The hypothesis is a formal statement of something that can be tested experimentally and these often arise from more general research questions which are often broad statements about what you are interested in or trying to discover. You might arrive at a research question or an idea while reading a paper, at a conference, while thinking about an observation you made, or by brainstorming in an informal group or lab meeting.

IMG_20190529_095011

A lab meeting with my student learning fNIRS

Notice that all of these assume you that put in some time and effort to understanding the theory and then allow some time to work over ideas in your mind, on paper, or in a computer simulation.

  • Work on hypothesis generation in lab meetings, our advisory meetings, and on your own.
  • Document your thoughts in Google Docs (or your own notes on paper, OneNote or Evernote).
  • Share insights in lab meetings and in the relevant channel in Slack.

Design the Study/Experiment

Concurrent with hypothesis generation is experimental design. In most case, we are designing experiments to test hypotheses about category representation and category learning and/or the predictions of computational models. We want to test hypothesis generated from theories and also carry out exploratory work to help refine our theories. Avoid the temptation to put the cart before the horse and come up with experiments and studies that will produce an effect for its own sake. We don’t just want to generate effects.

The design comes first. Consider the logic of your experiment, what you plan to manipulate, and what you want to measure. Avoid the temptation to add in more measures than you need, just to see if there’s an effect. For example, do you need to add in 2-3 measures of working memory, mood, or some demographic information just to see if there’s an effect there? If it’s not fully justified, it may hurt more than help because you have non-theoretically driven measures to contend with. I’ve been guilty of this in the past and it always comes back to haunt me.

  • Work on experiment generation in lab meetings, advisory meetings, on your own.
  • Document your work and ideas in Google Docs or a note taking app that you can share.
  • Use G*Power to estimate correct sample size.
  • Use PsychoPy or Qualtrics to build your experiment.
  • Test these experiment protocols often, on your self, on lab mates, on volunteers
  • Develop a script for research assistants who will be helping you carry out the study.
  • Share insights in lab meetings and in the relevant channel in Slack.
  • Organize tasks and chores in the relevant Trello board for your project.

Analysis Plan & Ethics Protocol

This is where we start to formalize things. An analysis plan will link together the hypothesis and the experimental design with the dependent variables and/outcome measures. In this plan, we’ll describe and document how the data will be collected, visualized, analyzed, stored and shared. This plan should describe how we will deal with outlier data, missing data, data from participants who did not complete the experiment correctly, experimenter error, malfunction, etc. This plan can include tentative predictions derived from a model and also a justification of how we intend to analyze and interpret the data. This plan can also be pre-registered with OSF, which is where we’ll plan to share the data we collect with the scientific community.

At the same time we also want to write an ethics protocol. This is a description of our experiment, the research question, and procedures for the University REB. This will also include standardized forms for information and consent, a policy for recruitment, subject safety, data storage and security. The REB has templates and examples, and our lab Slack channel on ethics can include examples as well. Use templates when ever possible.

Both of these documents, the analysis plan and the ethics protocol, should describe exactly what we are doing and why we are doing it. They should provide enough information that someone else would be able to reproduce our experiments in their own lab. These documents will also provide an outline for your eventual method section and your results section.

  • Document your analysis plan and ethics protocol work in Google Docs.
  • Link these documents to the project sheet or Trello board for your project.
  • Share in the relevant channel in Slack.

Collect Data

Once the experiment is designed, the stimuli have been examined, we’re ready to collect data or to obtain data from a third party (which might be appropriate for model testing). Before you run your first subject, however, there are some things to consider. Take some time to run yourself through every condition several times and ask other lab members to do the same. You can use this to make sure things are working exactly as you intend, to make sure the data are being saved on the computer, and to make sure the experiment takes as long as planned.

When you are ready to collect data for your experiment:

  • Meet with all of your research volunteers to go over the procedure.
  • Book the experiment rooms on the Google Calendar.
  • Reserve a laptop or laptops on the Google Calendar.
  • Recruit participants though SONA or flyers.
  • Prepare the study for M-Turk or Prolific
  • Use our lab email for recruitment.

After you have run through your experiment several time, documented all the steps, and ensured that the everything is working exactly as you intended, you are ready to begin. While you are running your experiment:

  • Document the study in Google Docs, Trello, and/or Slack (as appropriate)
  • Make a note of anything unusual or out of the ordinary for every participant in a behavioural study.
  • Collect signatures from participants if you are paying them.
  • Data should stored in text files that can be opened with Excel or Google sheets or imported directly into R. Be sure these are linked to the project sheet.
  • Make sure the raw data are labelled consistently and are never altered.
  • Be sure to follow the data storage procedures outlined in the ethics protocol.

Data Management

Your data plan should specify where and how to store your data. While you are collecting data you should be working on a script in R (or Python) to extract and summarize the raw data according to your plan. When you reach the planned sample size, ensure that all of that data are secure and backed up and do an initial summary with your R script.

As you work on summarizing and managing your data:

  • Make notes in the project sheet and/or Trello board about where the data are stored
  • Document your steps in an R Notebook (or Python Notebook).

Plots & Stats

Remember the photo of Dr. Katie Bouman, then a postdoc, when she first saw the rendering of the first photos of a black hole that her algorithms generated? That’s the best part of science: seeing your data visualized for the first time. When you have completed your experiment and taken care of the data storage and basic processing, it’s time to have fun and see what you discovered. The analysis plan is your guide and your analysis plan describes how you want to analyze the data, what your dependent variables are, and how to conduct statistical test with you data to test the hypothesis. But before you do any statistics, work on visualizing the data. Use your R notebook to document everything and generate boxplots, scatter plots, or violin plots to see the means, medians, and the distribution for the data.

Because you are using R Notebooks to do the analysis, you can write detailed descriptions of how you created the plot, what the plot is showing, and how we should interpret the plot. If you need to drop or eliminate a subject’s data for any reason, exclude them in from data set in R, do not delete the data from the raw data file. Make a comment in the script of which subject was dropped and why. This will be clear and transparent.

You can also use R to conduct the tests that we proposed to use in the analysis plan. This might be straightforward ANOVA or t-test, LME models, regression, etc. Follow the plan you wrote, and if you deviate from the plan, justify and document that exploratory analysis.

If you are fitting a decision boundary model to your data, make sure you have the code for the model (these will be on my GitHub) and you should do your modelling separately from the behavioural analysis. The GLM models are saved as R scripts but you should copy or fork to your R-Notebooks for your analysis so you can document what you did. Make sure that you develop the version for your experiment and that the generic model is not modified.

If you are fitting a prototype or exemplar model, these have been coded in Python. Use Python 3 and a basic text editor or JupyterLab. JupyterLab might be better as it’s able to generate markdown and reproducible code like R Notebooks. Or just call python from R Studio.

  • Follow your analysis plan.
  • Consult with me or your peers if you notice any unusual patterns with anything.
  • Make notes in the project sheet and/or Trello board about what analyses you’ve completed.
  • Document your steps in an R Notebook (or Python Notebook).
  • If you drop a participant for any reason, indicate in the comments of your R script (or other notes). We want this information to recorded and transparent.

Present and Explain Your Work

While you working on your analysis, you should present the interim work often in lab meetings for the rest of the group and we can discuss the work when we meet individually. The reason to present and discuss often is to keep the ideas and work fresh in your mind by reviewing manageable pieces of it. If you try to do too much at once, you may miss something or forget to document a step. Go over your work, make sure its documented, and then work on the new analyses, and repeat. You should be familiar with your data and your analysis so that you can explain it to yourself, to me, to your peers, end eventually anyone who reads your paper.

Use the following guidelines for developing your work:

  • Make your best plots and figures.
  • Present these to the lab on a regular basis.
  • Use RPubs to share summary work instantly with each other and on the web
  • Keep improving the analysis after each iteration.
  • You should always have 8-10 slides that you can present to the group.
  • Document your work in R Notebooks, Google Docs, Trello, and Google Slides.

Write Papers Around This Workflow

The final step is to write a paper that describes your research question, your experimental design, your analysis, and your interpretation of what the analysis means. A scientific paper, in my opinion has two important features:

  1. The paper should be clear and complete. That means it describes exactly what you wanted to find out, how and why you designed your experiment, how you collected your data, how you analyzed your data, what you discovered, and what that means. Clear and complete also means that it can be used by you or by others to reproduce your experiments.
  2. The paper should be interesting. A scientific paper should be interesting to read. It needs to connect to a testable theory, some problem in the literature, an unexplained observation. It is just as long as it needs to be.

I think the best way to generate a good paper is to make good figures. Try to tell the story of your theory, experiment, and results with figures. The paper is really just writing how you made the figures. You might have a theory or model that you can use a figure to explain. You can create clear figures for the experimental design, the task, and the stimuli. Your data figures, that you made according to you analysis plan, will frame the results section and a lot of what you write is telling the reader what they show, how you made them, and what they mean figures. Writing a scientific paper is writing a narrative for your figures.

Good writing requires good thinking and good planning. But if you’ve been working on your experiment according to this plan, you’ve already done a lot of the thinking and planning work that you need to do to write things out. You’ve already made notes about the literature and prior work for your introduction. You have notes from your experimental design phase to frame the experiment. You have an ethics protocol for your methods section and an analysis plan for your results. You’ll need to write the discussion section after you understand the results, but if you’ve been presenting your 8-10 slides in lab meeting and talking about them you will have some good ideas and the writing should flow. Finally, if you’ve been keeping track of the papers in Paperpile, your reference section should be easy.

Submit the paper

The final paper may have several experiments, each around the theme set out in the introduction. It’s a record of what we did, why we did it, and how. The peer reviewed journal article is the final stage, but before we submit the paper we have a few other steps to ensure that our work roughly conforms to the principles of Open Science, each of which should be straightforward if we’ve followed this plan.

  • Create a publication quality preprint using the lab template. We’ll host this on PsyArXiv (unless submitting a blind ms.)
  • Create a file for all the stimuli or materials that we used and upload to OSF.
  • Create a data archive with all the raw, de-identified data and upload to OSF.
  • Upload a clean version of your R Notebook that describe your analyses and upload to OSF.

The final steps are organized around the requirements of each journal. Depending on where we decide to submit our paper, some of these may change. Some journals will insist on a word doc file, others will allow for PDF. In both cases, assume that the Google Doc is the real version, and the PDF or the .doc files are just for the journal submission. Common steps include:

  • Download the Google Doc as a MS Word Doc or PDF.
  • Create a blind manuscript if required.
  • Embed the figures if possible otherwise place at the end.
  • Write a cover letter that summarizes paper and why we are submitting.
  • Identify possible reviewers.
  • Write additional summaries as required and generate keywords.
  • Check and verify the names, affiliations, and contact information for all authors.
  • Submit and wait for 8-12 weeks!

Conclusion

As I mentioned at the outset, this might not work for every lab or every project. But the take home message–document everything you do and share your work for feedback–should resonate with most science and scholarship. Is it necessary to have a formal guide? Maybe not, though I found it instructive for me as the PI to write this all down. Many of these practices were already in place, but not really formalized. Do you have a similar document or plan for your lab? I’d be happy to hear in the comments below.

Mindful University Leadership

Leave a reply

Academia, like many other sectors, is a complex work environment. Although universities vary in terms of their size and objectives, the average university in the United States, Canada, UK, and EU must simultaneously serve the interests of undergraduate education, graduate education, professional education, basic research, applied research, public policy research, and basic scholarship. Most research universities receive funding for operation from a combination of public and private sources. For example, my home university, The University of Western Ontario, receives its operating funds from tuition payments, governments, research funding agencies, and from private donors. Many other research universities are funded in similar ways, and most smaller colleges are as well.

IMG_20180728_204415-EFFECTS

Looking west over Lake Erie, Port Stanley, Ontario

Faculty are at the center of this diverse institution, acting as the engine of teaching, research, and service. As a result, faculty members may find themselves occasionally struggling to manage these different interests. This article looks at the challenges that faculty members face, paying particular attention to the leadership role that many faculty play. I then explore the possible ways in which a mindfulness practice can benefit faculty well-being and productivity.

Challenges of Leadership in the University Setting

Although many work environments have similar challenges and issues (being pulled in different directions, time management, etc.) I want to focus on the challenges that faculty members face when working at and leading the average, mid-sized or large university. The specific challenges will vary in terms of what role or roles a person is serving in, but let’s first look at challenges that might be common to most faculty members.

Challenge 1: Shifting tasks

“Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration.” — Donald Knuth

I love this quote from Donald Knuth, a professor of computer science, because it encapsulates the main challenge that so many of us have. We want to be on top of things (teaching, questions from students, cutting-edge research) but we also want to be on the bottom: digging deeply into a problem and finding a solution.

The average faculty member has, at a minimum, 2–3 very different kinds of jobs. We’re teachers, researchers/scholars, and we also help to run the university. Within these broadly-defined categories, we divide our teaching time between graduate and undergraduate teaching and mentorship. Research involves investigation, applying for grants, reading, investigation, analysis, writing, dissemination. And running the university can make us managers, chairs, deans, and provosts and as such, we’re responsible for hiring research staff, hiring other faculty members, and managing budgets.

These three categories require different sets of skills and shifting between them can be a source of stress. In addition, the act of shifting between them will not always go smoothly and this may result in a loss of effectiveness and productivity as the concerns from one category, task, or role bleed into another. Being mindful of the demands of the current task at hand is crucial.

For example, I find it especially difficult to transition after 2–3 hours of leading a seminar or lecture. Ideally, I would like to have some time to unwind. But many times, I also need to schedule a meeting in the afternoon and find that I have only a short amount of time to go from “lecture mode” into “meeting mode”. Worse, I might still be thinking about my lecture when the meeting begins (this is an even bigger challenge for me in 2020, because nearly everything is online, on Zoom, from my home office). Even among university leaders that have little or no direct teaching requirements, it is common to have to switch from and to very different topics. One day you might start the day answering emails (with multiple topics), a morning meeting on hiring negotiations, a meeting about undergraduate planning, then an hour with a PhD student on a very specific and complex analysis of data for their dissertation research, followed by a phone call from the national news outlet asking about the research of one of your faculty members. Shifting between these tasks can reduce your effectiveness. The cognitive psychology literature refers to this as “set shifting” or “task-shifting”, and research has supported the idea that there is always a cost to shift (Arrington & Logan, 2004; Monsell, 2003).  These costs will eventually affect how well you do your job and also how you deal with stress. It’s difficult to turn your full attention to helping your student with an analysis when you are also thinking about your department’s budget.

As academics, we switch and shift tasks throughout the day and throughout the week. The primary challenge in this area is to be able to work on the task at hand and to be mindful of distractions. Of course, they will occur, but through practice, it may be possible to both minimize their impact and also reduce the stress and anxiety associated with the distractions.

Challenge 2: Shared governance

One aspect of academia that sets it apart from many corporate environments is the notion of “shared governance”. Though this term is common (and has been criticized as being somewhat empty,) the general concept is that a university derives its authority from a governing board, but that faculty are also vested in the institutional decision-making process. This means that most universities have a faculty senate that sets academy policy, dean’s level committees that review budgets and programs, and departmental committees that make decisions about promotion and tenure, hiring, and course assignments.

From a leadership perspective, this can mean that as a chair or dean you are always managing personal, balancing the needs of faculty, students, budgets, senior administrators, and the public image of your university. There may not be a clear answer to the question of “who is the boss?”  Sometimes faculty are asked to assume leadership roles for a set time and will need to shift from a collegial relationship to a managerial one (then back to a collegial one) for the same people. That is, one day you are colleagues and the next you are his or her supervisor.

The challenge here is to understand that you may be manager, colleague, and friend at the same time. In this case, it’s very helpful to be mindful of how you interact with your colleagues such that your relationship aligns with the appropriate role.

Challenge 3: Finding time for research and scholarship

One of the most common complaints or concerns from faculty is that they wish they had more time for research. This is a challenge for faculty as well as leaders. Although a common workload assumes that a faculty member may spend 40% of their time on research, most faculty report spending much of their time in meetings. However, promotion and tenure is earned primarily through research productivity. Grants are awarded to research productive faculty. That is, most of those meetings are important, but do not lead to promotion and career advancement. This creates a conflict that can cause stress because although 40% is the nominal workload, it may not be enough to be research productive. Other aspects of the job, like meetings related to teaching and service, may take up more than their fair share but often feel more immediate.

In order to be effective, academic leaders also need to consider these concerns from different perspectives. For example, when I was serving as the department chair for a short period, I had to assigned teaching to our faculty. There are courses that have to be offered and teaching positions that have to be filled. And yet my colleagues still need to have time to do research and other service work. These can be competing goals and they affect different parts of the overall balance of the department. The department chair needs to balance the needs of faculty to have adequate time for research with the needs of the department to be able to offer the right amount of undergraduate teaching. So not only is it a challenge to find time to do one’s own research, a department chair also needs to consider the same for others. Being mindful of these concerns and how they come into conflict is an important aspect of university leadership.

Considering these diverse goals and trying to meet them requires a fair degree of cognitive flexibility and if you find yourself being pulled to think about teaching, about meetings, and about the workload of your colleagues, it is going to pull you away from being able to be on top of your own research and scholarship. The primary challenge in this area is to create the necessary cognitive space for thinking about research questions and working on research.

Mindfulness and Leadership

I’ve listed three challenges for leaders in an academic setting: switching, shared governance, and finding time for research. There are more, one course, but let’s stick with these. I want to now explain what mindfulness practice is and how it might be cultivated and helpful for academic leaders. That is, how can mindfulness help with these challenges?

What is mindfulness?

A good starting point for this question is a definition that comes from Jon Kabat-Zinn’s work. Mindfulness is an open and receptive attention to, and awareness of what is occurring in the present moment. For example, as I’m writing this article, I am mindful and aware of what I want to say. But I can also be aware of the sound of the office fan, aware of the time, aware that I am attending to this task and not some other task. I’m also aware that my attention will slip sometimes, and I think about some of the challenges I outlined above. Being mindful means acknowledging this wandering of attention and being aware of the slips but not being critical or judgmental about my occasional wavering. Mindfulness can be defined as a trait or a state. When described as a state, mindfulness is something that is cultivated via mindfulness practice and meditation.

How can mindfulness be practiced?

The best way to practice mindfulness is just to begin. Mindfulness can be practiced alone, at home, with a group, or on a meditation retreat. More than likely, your college or university offers drop in meditation sessions (as mine does). There are usually meditation groups that meet in local gyms and community centers. Or, if you are technologically inclined, the Canadian company Interaxon makes a small, portable EEG headband called MUSE that can help develop mindfulness practice (www.choosemuse.com). There are also excellent apps for smartphones, like Insight Timer.

The basic practice is one of developing attentional control and awareness by practicing mindfulness meditation. Many people begin with breathing-focused meditation in which you sit (in a chair or on a cushion) close your eyes, relax your shoulders and concentrate on your breath. Your breath is always there, and so you can readily notice how you breath in and out. You notice the moment where your in-breath stops and your out-breath begins. This is a basic and fundamental awareness of what is going on right now. The reason many people start with breathing-focused meditation is that when you notice that your mind begins to wander, you can pull your attention back to your breath. The pulling back is the subtle control that comes from awareness and this is at the heart of the practice. The skill you are developing with mindfulness practice is the ability to notice when your attention has wandered, not to judge that wandering, and to shift your focus back to what is happening in the present

Benefits of mindfulness to academic leaders

A primary benefit of mindfulness involves learning to be cognitively and emotionally present in the task at hand. This can help with task switching. For example, when you are meeting with a student, being mindful could mean that you bring your attention back to the topic of the meeting (rather than thinking about a paper you have been working on). When you are working on a manuscript, being mindful could mean keeping your attention on the topic of the paragraph and bringing it back from other competing interests. As a researcher and a scientist, there are also benefits as keeping an open mind about collected data and evidence which can help to avoid cognitive pitfalls. In medicine, as well as other fields, this is often taught explicitly as at the “default interventionist” approach in which the decision-maker strives to maintain awareness of her or her assessments and the available evidence in order to avoid heuristic errors. (Tversky & Kahneman, 1974) As a chair or a dean, being fully present could also manifest itself by learning to listen to ideas from many different faculty members and from students who are involved in the shared governance of academia.

Cognitive and clinical psychological research has generally supported the idea that both trait mindfulness and mindfulness meditation are associated with improved performance on several cognitive tasks that underlie the aforementioned challenges to academic leaders. For example, research studies have shown benefits to attention, working memory, cognitive flexibility, and affect. (Chambers, Lo, & Allen, 2008; Greenberg, Reiner, & Meiran, 2012; Amishi P. Jha, Stanley, Kiyonaga, Wong, & Gelfand, 2010; Amism P. Jha, Krompinger, & Baime, 2007) And there have been noted benefits to emotional well-being and behaviour in the workplace as well. This work has shown benefits like stress reduction, a reduction to emotional exhaustion, and increased job satisfaction (Hülsheger, Alberts, Feinholdt, & Lang, 2013, Nadler, Carswell, & Minda, 2020)

Given these associated benefits, mindfulness meditation has the potential to facilitate academic leadership by reducing some of what can hurt good leadership (stress, switching costs, cognitive fatigue) and facilitating what might help (improvements in attentional control and better engagement with others).

Conclusions

As I mentioned at the outset, I wrote this article from the perspective of a faculty member at large research university, but I think the ideas apply to higher education roles in general. But it’s important to remember that mindfulness is not a panacea or a secret weapon. Mindfulness will not make you a better leader, a better teacher, a better scholar, or a better scientist. Mindful leaders may not always be the best leaders.

But the practice of mindfulness and the cultivation of a mindful state has been shown to reduce stress and improve some basic cognitive tasks that contribute to effective leadership. I find mindfulness meditation to be an important part of my day and an important part of my role as a professor, a teacher, a scientist, and an academic leader.  I think it can be an important part of a person’s work and life.

References

Arrington, C. M., & Logan, G. D. (2004). The cost of a voluntary task switch. Psychological Science, 15(9), 610–615.

Chambers, R., Lo, B. C. Y., & Allen, N. B. (2008). The Impact of Intensive Mindfulness Training on Attentional Control, Cognitive Style, and Affect. Cognitive Therapy and Research, 32(3), 303–322.

Greenberg, J., Reiner, K., & Meiran, N. (2012). “Mind the Trap”: Mindfulness Practice Reduces Cognitive Rigidity. PloS One, 7(5), e36206.

Hülsheger, U. R., Alberts, H. J. E. M., Feinholdt, A., & Lang, J. W. B. (2013). Benefits of mindfulness at work: the role of mindfulness in emotion regulation, emotional exhaustion, and job satisfaction. The Journal of Applied Psychology, 98(2), 310–325.

Jha, A. P., Krompinger, J., & Baime, M. J. (2007). Mindfulness training modifies subsystems of attention. Cognitive, Affective & Behavioral Neuroscience, 7(2), 109–119.

Jha, A. P., Stanley, E. A., Kiyonaga, A., Wong, L., & Gelfand, L. (2010). Examining the protective effects of mindfulness training on working memory capacity and affective experience. Emotion , 10(1), 54–64.

Monsell, S. (2003). Task switching. Trends in Cognitive Sciences, 7(3), 134–140.

Nadler, R., Carswell, J. J., & Minda, J. P. (2020). Online Mindfulness Training Increases Well-Being, Trait Emotional Intelligence, and Workplace Competency Ratings: A Randomized Waitlist-Controlled Trial. Frontiers in Psychology, 11, 255.

Tversky, A., & Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases. Science, 185(4157), 1124–1131.

The Professor, the PI, and the Manager

5 Replies

Here’s a question that I often ask myself: How much should I be managing my lab?

I was meeting with one of my trainees the other day and this grad student mentioned that they sometimes feel like they don’t know what to do during the work day and that they sometimes feel like they are wasting a lot of their time. As a result, this student will end up going home and maybe working on a coding class, or (more often) doing non grad school things. We talked about what this student is doing and I agreed: they are wasting a lot of time, and not really working very effectively.

Before I go on, some background…

There is no shortage of direction in my lab, or at least I don’t think so. I think I have a lot of things in place. Here’s a sample:

  • I have a detailed lab manual that all my trainees have access to. I’ve sent this document to my lab members a few times, and it covers a whole range of topics about how I’d like my lab group to work.
  • We meet as a lab 2 times a week. One day is to present literature (journal club) and the other day is to discuss the current research in the lab. There are readings to prepare, discussions to lead, and I expect everyone to contribute.
  • I meet with each trainee, one-on-one, at least every other week, and we go though what each student is working on.
  • We have an active lab Slack team, every project has a channel.
  • We have a project management Google sheet with deadlines and tasks that everyone can edit, add things to, see what’s been done and what hasn’t been done.

So there is always stuff to do but I also try not to be a micromanager of my trainees. I generally assume that students will want to be learning and developing their scientific skill set. This student is someone who has been pretty set of looking for work outside of academics, and I’m a big champion of that. I am a champion of helping any of my trainees find a good path. But despite all the project management and meetings this student was feeling lost and never sure what to work on. And so they were feeling like grad school has nothing to offer in the realm of skill development for this career direction. Are my other trainees also feeling the same way?

Too much or too little?

I was kind of surprised to hear one of my students say that they don’t know what to work on, because I have been working harder than ever to make sure my lab is well structured. We’ve even dedicated several lab meetings to the topic.

The student asked what I work on during the day, and it occurred to me that I don’t always discuss my daily routine. So we met for over an hour and I showed this student what I’d been working on for the past week: an R-notebook that will accompany a manuscript I’m writing that will allow for all the analysis of an experiment to be open and transparent. We talked about how much time that’s been taking, how I spent 1-2 days optimizing the R code for a computational model. How this code will then need clear documentation. How the OSF page will also need folders for the data files, stimuli, the experimenter instructions. And how those need to be uploaded. I have been spending dozens of hours on this one small part of one component of one project within one of the several research areas in my lab, and there’s so much more to do.

Why aren’t my trainees doing the same? Why aren’t they seeing this, despite all the project management I’ve been doing?

I want to be clear, I am not trying to be critical in any way of any of my trainees. I’m not singling anyone out. They are good students, and it’s literally my job to guide and advise them. So I’m left with the feeling that they are feeling unguided, with the perception that that there’s not much to do. If I’m supposed to be the guide and they are feeling unguided, this seems like a problem with my guidance.

What can I do to help motivate?

What can I do to help them organize, feel motivated, and productive?

I expect some independence for PhD students, but am I giving them too much? I wonder if my lab would be a better training experience if I were just a bit more of a manager.

  • Should I require students to be in the lab every day?
  • Should I expect daily summaries?
  • Should I require more daily evidence that they are making progress?
  • Am I sabotaging my efforts to cultivate independence by letting them be independent?
  • Would my students be better off if I assumed more of a top down, managerial role?

I don’t know the answers to these questions. But I know that there’s a problem. I don’t want to be a boss, expecting them to punch the clock, but I also don’t want them to float without purpose.

I’d appreciate input from other PIs. How much independence is too much? Do you find that your grad students are struggling to know what to do?

If you have something to say about this, let me know in the comments.

Does This Project Bring Me Joy?

Leave a reply

 

I have too many research projects going on.

It’s great to be busy, but I’m often overwhelmed in this area. As a university professor, some of my job is well defined (e.g. teaching) but other parts not so much. My workload is divided into 40% research, 40% teaching, and 20% service. Within each of these, I have some say as to what I can take on. I can teach different classes and volunteer to serve on various committees. But the research component is mine. This is what I really do. I set the agenda. I apply for funding. This is supposed to be my passion.

So why do I feel overwhelmed in that area?

I think I have too many projects going on. And I don’t mean that I am writing too many papers. I’m most certainly not doing that. I mean I have too many different kinds of projects. There are several projects on psychology and aging, projects on the brain electrophysiology and category learning, a project on meditation and wellbeing in lawyers, a project on patient compliance, a project on distraction from smartphones, plus 4-5 other ideas in development, and at least 10 projects that are most charitably described as “half baked ideas that I had on the way home from a hockey game”.

Add to this many projects with students that may not quite be in my wheelhouse, but are close and that I’m supervising. And I’ll admit, I have difficulty keeping these things straight. I’m interested in things. But when I look at the list of things, I confess I have a tough time seeing a theme sometimes. And that’s a problem as it means I’m not really fully immersed in any one project. I cease to be an independent and curious scientist and become a mediocre project manager. And when I look at my work objectively, more often than not, it seems mediocre.

Put another way, sometimes I’m always really sure what I do anymore…

So what should I do about this, other than complain on my blog? I have to tidy up my research.

A Research Purge

There is a very popular book called “The Life Changing Magic of Tidying Up“. I have not read this book, but I have read about this book (and let’s be honest that’s sometimes the best we can do). The essence of the approach is that you should not be hanging on to things that are not bringing you joy.

Nostalgia is not joy.

Lots of stuff getting in the way is not joy. And so you go though things, one category at a time, and look at each thing and say “does this item spark joy“? If the answer is no, you discard it. I like this idea.

If this works for a home or a room…physical space…then it should work for the mental space of my research projects. So I’m going to try this. I thought about this last year, but never quite implemented it. I should go through each project and each sub project and ask “Does this project bring me joy?” or “Is there joy in trying to discover this?” Honestly, if the answer is “no” or “maybe” why should I work on it? This may mean that I give up on some things and that some possible papers will not get published. That’s OK, because I will not be compelled to carry out research and writing if it is not bringing me joy. Why should I? I suspect I would be more effective as a scientist because I will (hopefully) focus my efforts on several core areas.

This means, of course, that I have to decide what I do like. And it does not have to be what I’m doing. It does not have to be what I’ve done.

The Psychology of the Reset

Why do we like this? Why do people want to cleanse? To reset. To get back to basics? It seems to be the top theme in so many pop-psych and self help books. Getting rid of things. A detox or a “digital” detox. Starting over. Getting back to something. I really wonder about this. And although I wonder why we behave this way, I’m not sure that I would not find joy in carrying out a research study on this…I must resist the urge to start another project.

I’m going to pare down. I still need to teach, and supervise, and serve on editorial boards, etc: that’s work. I’m not complaining and I like the work. But I want to spend my research and writing time working on projects that will spark joy. Investigating and discovering things that I’m genuinely curious about…curious enough to put in the hours and time to do the research well.

I’d be curious too, to know if others have tried this. Has it worked? Have you become a better scholar and scientists by decluttering your research space?

Thanks for reading and comments are welcome.

Grade Inflation at the University Level

2 Replies

I probably give out too many As. I am aware of this, so I may be part of the problem of grade inflation. Grade inflation has been a complaint in universities probably as long as there have been grades and as long as there have been universities.

Harvard students receive mostly As.

But the issue has been in the news recently. For example, a recent story asserted that the most frequent grade (e.i. the modal grade) at Harvard was an A. That seems a bit much. If Harvard is generally regarded as of the world’s best universities, you would think they would be able to asses their students on a better range. A great Harvard undergrad should be a rare thing, and should be much better than the average Harvard undergrad. Evidently, all Harvard undergrads are great.

One long time faculty member, says that “in recent years, he himself has taken to giving students two grades: one that shows up on their transcript and one he believes they actually deserve….“I didn’t want my students to be punished by being the only ones to suffer for getting an accurate grade,”

In this way, students know what their true grade is, but they also get a Harvard grade that will be an A so that they look good and that Harvard looks good. It’s not just Harvard, of course. This website, gradeinflation.com, lays out all details. Grades are going up everywhere…But student performance may not be.

The University is business and As are what we make.

From my perspective as a university professor, I see the pressure from all sides, and I think the primary motivating force is the degree to which universities have heavily embraced a consumer-driven model. An article The Atlantic this week got me thinking about it even more. The article points out, we (university) benefit when more students are doing well and earning scholarships. One way to make sure they can earn scholarships is to keep the grades high. It is to our benefit to have more students earning awards and scholarships.

In other words, students with As bring in money. Students with Cs do not. But this suggests that real performance assessment and knowledge mastery is subservient to cash inflow. I’m probably not the only one who feels that suggestion is true.

And of course, students, realizing they are the consumer, sort of expect a good grade for what they pay for. They get the message we are sending. Grades matter more than knowledge acquisition. Money matters more than knowledge. If they pay their tuition and fees on time, they kind of expect a good grade in return. They will occasional cheat to obtain these grades. In this context, cheating is economically rational, albeit unethical.

Is there a better system?

I am not sure what to do about this. I’m pretty sure that my giving out more Cs is not the answer, unless all universities did this. I wonder if we really even need grades? Perhaps a better system would be a simple pass/fail? Or Fail/Pass/Exceed (three way). This would suggest that students have mastered the objectives in the course and we (the University) can confidently stand behind our degree programs and say that our graduates have acquired the requisite knowledge. Is that not our mission? Does it matter to an employer if a student received an A or a B in French? Can they even use that as a metric when A is the modal grade? The employer needs to know that the student mastered the objectives for a French class and can speak French. Of course, this means that it might be tricky for graduate and professional schools to determine admission. How will medical schools know who admit if they do not have a list of students with As? Though if most students are earning As, it renders moot that point.

In the end, students, faculty, and university administrators are all partially responsible for the problem, and there is no clear solution. And lurking behind it, as is so often the case, is money.