Cassandra Paul - Choosing to Attend College Forever: Life as a Professor and Physics Education Researcher

Cassandra Paul, class of 2012, is a professor
at San Jose State University

Cassandra Paul is an Associate Professor of Physics & Astronomy at San Jose State University. She came to speak to our current students in the spring of 2017, to tell them about being a professor at a Master's/Baccalaureate institution as opposed to a PhD-granting institution such as UC Davis. She also discussed the field of Physics Education Research, told us some about her current research and gave an impressive overview of her approach to balancing all the demands of teaching, research, service, and the rest of life.

Professor Paul received her doctorate in physics from UC Davis in 2012. Her current research interests include active learning techniques, assessment, instructor professional development, and supporting graduate student teaching assistants.

We start with the key points from her presentation, and follow it with a lightly edited transcript of her remarks and the Q&A session.


Key Points:

• University teaching job classification:
• Doctoral university
• High research requirements, low teaching load
• Option for faculty to lower teaching load by taking on more research or service
• Master’s/Baccalaureate university
• Lower research requirement, higher teaching load
• Option for faculty to lower teaching load by taking on more research or service
• Community College
• Little to no research requirement, highest teaching load
• Advice for new professors
• When negotiating a job offer, ask for a reduced teaching load in the first few years
• Apply for research release time both internally and externally
• Grants can supplement your salary
• Grants can 'buy you out' of your commitment to teach at your institution
• Work life balance (how Cassandra does it)
• Learn how to say no
• Choose a job that works for you as much as you work for it
• Structure your academic and career choices around your lifestyle, not the other way around
• How to prepare to be a professor while in grad school
• Get involved with extracurricular activities
• Search committees look for a well rounded person that will fit into their community, not just someone qualified for teaching
• Get involved with service
• Organizations at UC Davis include: Graduate Teaching Community, Teaching Assistant Consultants, the Graduate Student Association, Professors for the Future, Chancellor's Advisory Board, American Physical Society, American Astronomical Society, American Association of Physics Teachers, and Graduate Students’ League
• Ask to lecture during the summer
• If attending a conference, take advantage of the teaching programs that are often before or after the main events
• Educate yourself on Physics Education Research (PER)
• Spend a lot of time on your job applications
• Tailor your cover letter and CV to match the job call

Partial Transcript:

The first thing I wanted to talk about is the differences between teaching and research requirements at different universities. It’s something you may have heard a little bit about, but I want to lay it out here so you have some perspective. There are three main, really broad classifications of colleges. You have the doctoral universities like Davis or any other UC; you have the Master’s and Baccalaureate universities and colleges; all of the CSUs fit in that, and that is any university that has their terminal degree as a Master’s or Bachelor’s degree. Then there are community colleges, and those are two-year schools that don’t have any higher degrees. You can be an adjunct/lecturer or a professor in any type of place.

First let me say that this is classification is very broad, and it’s hard to make generalizations across schools, so specific schools might differ. At doctoral universities, you have very high research requirements, and you have a lower teaching load. For example, since a lot of doctoral universities have teaching loads of 1-2 courses per semester or per quarter, they have higher research requirements. Just to put a number on it, more than a paper per year. And they also have service requirements. You serve on committees, you do service for the university, maybe you volunteer to review scholarship applications; there are all different types of things that you can do that are going on behind the scenes that you don’t even know about as a graduate student, and that’s the service requirement. At Master’s and Baccalaureate colleges, they have medium research requirements. What I mean when I say medium is that research is a part of our job description, we do have to have active research programs, but the requirements as to how we display that are a little looser. For example, I can publish less than a paper per year, even less than a paper every two years, and be completely fine and up to date with the research requirement for my institution. Our teaching loads are medium to high: for example, at my university it’s a 4-4 load. That means we teach 4 required courses in the spring and 4 courses in the fall. That sounds terrifying, but I’m going to talk about how it actually isn’t what it may seem. We also have service requirements, and I actually don’t know how to compare the service requirements across universities. I think it has more to do with the individual, and what they want to take on, than it does with the university.

At community colleges, there are low or no research requirements. Some community colleges like their faculty to be involved in research, so they can get students getting into research at the associate level, but it’s more of a bonus feature than it is a need to have an active research program. Community colleges also have high teaching loads, similar to what is required at CSUs. The other main difference is that at doctoral and Master’s/Baccalaureate colleges, however high their teaching load, there’s always the option to do what’s called ‘buy-out,’ which means that you can do research or service instead of teaching a course. From what I understand, that’s not really an option at a community college: your teaching load is pretty much set.

If you’re an adjunct or a lecturer, your whole job is teaching: you do not have service requirements or research requirements. These types of jobs have a flexible course load, but that means your paycheck is similarly “flexible” as you a paid by the number of units you teach. So that is an option that some people choose if they want to have a more flexible type schedule.

For the rest of my talk, I’m going to be focusing on what it’s like teaching for a Master’s or Baccalaureate university, which is all the CSUs. The best advice that I got when I started my job was to ask for a reduced teaching load for my first few years. They had already offered me one, but then a current faculty member there told me to ask for more. I asked for a 50% reduction, I got some pretty big release time for my first few years, and that really really helped me, because that meant I had some time to start working on my research, and I could spend time preparing my courses the first time I taught them. As I became more familiar with university, I got more involved with students, I got more involved with service, and I actually had time to engage with graduate students and mentor them on projects. And these kinds of things actually count as course release time. If you teach a certain number of graduate students, they count as research units. I think you need 6 graduate students, and that counts as a full course release. That’s a nice thing to do, because you can then buy yourself out of a course for that semester.

Another thing that you can do is certain service requirements. For example, I’m a graduate coordinator, which means that I handle all the admissions and I advise students on how to get through to graduation. That also counts as a course release. So this actually starts looking pretty good: for as long as I continue to have an active research program, I anticipate not teaching any more than two classes per semester. The other thing is that large courses at my university count as double courses. Anything over 150 students means I get an extra course for release time, which is nice too.

The last thing, and this is also true at doctoral universities, is that I can apply for research release time both internally and externally. What this means is that the college, when they have these calls for proposals, you can say, ‘Hey, I would like to do this project, and can I please have some a course release time for this semester?’ For external, in the same way you might get it at another institution, you can apply to NSF, and you can say, ‘Part of my budget is buying myself out.’ That means that the grant pays the university so somebody else can teach your teaching load. That’s something you can do as well.

So what does it look like to do research at CSU? The short story is, it looks a lot like doing research at a doctoral university, except you have less time and fewer resources to do it. I have research group meetings, I go to conferences at least twice a year, I have undergraduate researchers, and I also have graduate students. The thing about graduate students at the Master’s college is that they turn over really quickly, because you only have them for two years. A lot of people say, ‘Oh, it’s just as they start to get useful that they leave.’ But I actually find that while there is some truth to that, they’re also a lot more motivated because they’re only here for a short period of time.

This is what research looks like on the whole, but what does it look like to do physics education research? Remember, the thing that I do is a little bit different than what a lot of other physics professors do. Physics Education Research is the study of how students learn in physics, methods used to teach and learn physics, the effects of different curriculum and environments on student learning in physics, the underrepresentation of women and minority students in physics, professional development for physics educators, or any other thing to do with learning or teaching in physics. You might think, ‘Okay, you’re studying all these things that have to do with education, why aren’t you over in the college of Education?’ That’s a pretty good question, and there’s a lot of great answers to that, but one of the main reasons is that in order to understand how students learn physics, you really need to understand physics at an expert level. And also, to be able to understand how to better teach it, you have to understand how you learn physics. There’s what’s called PCK, pedagogical content knowledge: this idea that if you’re an expert in physics, you have a better idea of how to help students learn these types of things. The roots of this field come from physics professors being unsatisfied with how their students were learning in the classroom and then transitioning from doing research in whatever their respective field was before into understanding how students better learn physics and can do better. And if you haven’t heard of physics education research, it’s not all that surprising, because it’s a relatively new field.

Katharine A. Anderson, Matthew Crespi, and Eleanor C. Sayre
Phys. Rev. Phys. Educ. Res. 13, 010121 – Published 1 May 2017

In fact, this plot was just published this month. This gray line here is the number of authors, and the colored lines are the numbers of papers. The graph only starts in 1980, and for the first 15-20 years, it’s the same people publishing, it’s just a few articles here and there. But around the year 2000, it jumps up. We have two new journals: one that was created in 2000 and one in 2005. Since I started as a graduate student, the number of papers and the number of authors in this field has doubled, which is crazy.

Katharine A. Anderson, Matthew Crespi, and Eleanor C. Sayre
Phys. Rev. Phys. Educ. Res. 13, 010121 – Published 1 May 2017

This is another way to represent the data, from the same paper: the different dots are the different authors, and the lines between them are co-authorships on papers. This is from 1980-1990, and there’s just a few people, there might be some collaborations, but there’s really just a lot of people doing their own thing. And then if you look at ‘91 through 2000, you can start to see these research groups start collaborating, start working together on things, but it’s still a pretty disparate group. Finally, you look at 2000 to 2010, and all of a sudden you see co-authorship across lots of different universities, different people working together. And in fact, if you plot all these together, you end up with, and you can look at the paper that shows this, an image that has pretty much all of these dots interconnected. It’s really grown to be much more of a community.

Where do you study physics education research? Turns out, all over. This (pictured below) is a map that is on the PER central website, and it shows many of the individuals in the US that are doing physics education research. Not all of these are programs where you can get a degree in physics education research, but these are all places that have faculty members who are doing this type of thing. And I also want to highlight that UC Davis is the only current physics PhD granting PER group in California. Stanford has PER researchers there, but they don’t have a way for you to earn a PhD in physics and study in PER. Berkeley has a program as well, but it’s not housed in physics, it’s housed in the department of education. Which is actually kind of sad, because I have a lot of Master’s students who are like, ‘I want a PhD in PER, where do I go?’ and I’m like, ‘You have to leave California.’ If any of you are California native in here, you know it’s really hard to get people to leave California.

What do PER grads do with their degrees? Lots! Many become academics, but there are also lots of separate options. Just to give you a snapshot of what happened to me, I applied to 9 jobs, I got 5 interviews and 3 job offers. My 3 job offers were: a physics education research postdoc; a full time lecturer position with TA training type job, and then the one I took, which was a tenure track assistant professor in physics and also in science education, which means I teach in both science education and the physics education groups. I want to stress that there are a vast number of PER postdoc jobs available around the nation, they’re having a hard time filling them. Others teach at private high schools. They also work for the national conferences, like the American Association of Physics Teachers and American Physical Society. There are also a few that have gone out on their own and become research consultants. I’m actually impressed by how many of them have been successful in that area. And a lot of them work a museums.


Dr. Knox: Who are [the research consultants’] clients?


They work for people who have a grant to do some educational type research, helping them with the assessment and that kind of thing.

Because we’re talking about physics education research, I wanted to show a couple interesting results from PER. The first study I’m going to show you is from 1998, so it’s an old study.

The Force Concept Inventory is a 30 question multiple choice exam that is just focused on Newton’s laws. There are no calculations, it’s just a ‘do you understand Newton’s laws’ kind of test. They made this exam and they thought students would be able to do really well on it after taking a physics course, but it turns out they don’t. This is what actually got a lot of physics professors interested in studying physics education research, because they could not believe that their students could perform so poorly on this test.

Hake, R., Interactive-engagement versus traditional methods:
A six-thousand-student survey of mechanics test data for introductory physics courses,
American Journal of Physics 66, 64 (1998); doi: http://dx.doi.org/10.1119/1.18809

Richard Hake surveyed across the country and he said, ‘Hey everybody who’s administered this FCI to their students, can you send me your results and also whether or not you consider your course an interactive engagement course?’ In an interactive engagement course, the instructor uses active learning methods in class and students are doing things, the professor isn’t just at the blackboard the whole time. Let me explain this plot, it’s essentially a way of assessing how much students have learned over the course of the semester. The average gains for the traditional classes were about 24%, and the averages for these interactive engagement courses are much higher, 40 or 50%. There is some overlap here, there are some interactive engagement courses that don’t perform as well, but we do see a clear distribution or distinction between these two types of courses.

If you doubt these results; it’s a long time ago, it might have changed, maybe the research methods weren’t performed as well as they could have been, this was redone again just last year with very similar results. These graphs aren’t directly comparable because the bin sizes are just a little different, but you can see that the averages for the traditional courses are around 25% and the averages for the interactive courses are somewhere between 40 and 50%. There’s quite a bit more conceptual learning happening in these interactive engagement courses.

Von Korff, J. et al. Secondary analysis of teaching methods in introductory physics:
A 50 k-student study American Journal of Physics 84, 969 (2016);
doi: http://dx.doi.org/10.1119/1.4964354

I also wanted to briefly just mention this other paper which came out just this year. This Physics Teacher did this analysis on the number of physics bachelor's degrees awarded by department, and what they found is, for physics education research programs, the average number was 19. For departments that don’t have a PER group, the average was only 12. Now you look at this, and you consider, well, big departments are more likely to have physics education research programs, so really this is maybe just measuring how big a program is. And you can look at the results, and you find that during 2004 and 2005, it was still higher for departments that had physics education groups. But these results aren’t significant. There’s an 18% chance that this happened by chance.

The Physics Teacher, Feb 2017

So this was done again in 2014-2015, and we see that PER departments have an average of 29 bachelor's degrees, where non PER departments have 19. If you look at this number by the number of faculty by department, you’ll see that PER departments still have a larger number of bachelor’s degrees PER program, and this time it’s actually significant, there's only a 3% chance that these data happened by chance. What could be happening here is that having a physics education research group in a department actually helps change the culture of what it means to make sure your students are learning in the class. You have experts now in research-based teaching, and this is now legitimizing this as a practice. This is something that other faculty members pick up: they learn things from working with their colleagues, and this ends up being better for everybody.

A little bit about my own research: I worked on this thing called RIOT, the real time instructor observing tool. A couple other researchers and I established what’s called inter-rater reliability: we measure the different kinds of interactions that happen in a classroom and make sure that the measurements between each other are within some pre-established errors. One interesting thing we did is that we looked at how the time the instructor spent listening to students or actively observing students positively correlated with the average final exam score. The reason I’m showing you this is to build off the idea that interactive engagement is useful. If you’re listening to your students talking, you have more information on how to give them feedback on what their learning is. This is one interpretation of this plot.

I’m also thinking more about TA and faculty professional development, how to get faculty and TAs to be more reflective about their teaching practices, and moving in a direction where I’m thinking more about systemic issues at the universities and how to support all students- looking to make more large scale changes at the level of the institution.

I mentioned earlier that I’m only teaching 2 classes per semester, so I have two classes: one class is 4 hours per week, the other is 8. My advising and graduate student research, that’s about 8 hours per week as well; my graduate student board, university service, and other things are 8 hours; and my research is 8 hours. If I’m being completely honest, I would say that these last two items usually end up being a little more shortchanged if I’m spending a little bit more time on teaching these hard classes, especially if I’m teaching a new course. Something that I always make sure to do is I make sure to keep one day for research, because it’s really important to me to be able to get that done. So if I can manage it every semester, and we can often do this because on Fridays at San Jose State there’s fewer courses, I try to block out a full day for research.

I’m getting into my idea here of work life balance. I don’t do weekend emailing, I don’t do night emailing, and I have a 40 hour per week boundary. My exception is conferences. If I go to conferences, it’s just all the time. And this is something that I just started doing in the past year. And it is something that has been instrumental for my mental health. Once you start getting into the habit of doing it, for me it makes me more efficient during work hours, and it also made me enjoy my off hours a lot more. I do this so I can spend more time hanging out with my son, my family, going on hikes, getting out there.

So what does it mean to have it all? What does work-life balance really mean? I’m telling you what works for me, and your work-life balance may look very different from mine. But this is what it means for me. I create professional goals that are attainable within the confines of a 40 hour work week. I don’t try to bite off more than I can chew, I know my limits and I know my boundaries, and I’m learning how to say no. I really try to emphasize that. I’m really selective about the service initiatives that I take on. I’m often asked to do things, but I try to think about it, and I tell them in the moment, ‘Let me try to find time in my schedule for that,’ but what I’m really thinking is do I want to do this? Do I care enough about this job to do it well? That’s important to me, to be able to care enough about something to do it well.

I also take my time with research. This has always been my hardest thing, because you just want to publish. You’re working so hard on this thing, you just want to get it out there, but good work takes time, and the other thing I realized is a lot of the people that are in my field are at these R1 universities, and they’ll have different types of constraints on their time. So I’ve stopped comparing myself to other researchers, I’ve stopped feeling the pressure to publish just because that’s what good researchers do. Instead, I take my time to get things right and publish when I publish.

I also try to build relationships with people that I work with and my students. I teach a class of 230 students, and I try really hard to learn their names. I don’t ever accomplish it, but I usually get around 50, which I think is pretty good for that kind of class. And when I do that, I always get comments on my evaluations like, ‘Wow, she took the time to learn our names in this giant class, she obviously really cares.’ And I do! When they come into my office and I talk with them, I hear a little bit about their life, we talk physics, it’s great.

This is like my mantra: it’s really important to choose a job that works for me as much as I work for it. I really enjoy the flexibility of the academic life, but not in the same way that I thought I would. I thought academic life was flexible and I could work when I wanted, but I found out it didn’t work for me to do it that way, so I work a traditional 9-5. If I’m starting to work outside of that 40 hours, I ask myself, ‘Is this important enough that I need to do it right now, or can it actually wait?’

Finally, I try to make the best use of my time outside of work. And this means for me: getting sleep and exercise every day. Also a lot of adventuring in home improvement. I make time for these kinds of things. It means getting up and taking that morning run every day. That’s a priority. It means spending spring break in Sedona. Also means going on vacation in Hawaii with my family.

Now that I’ve completely just bragged about my amazing life, let me just say that there are struggles. Thinking about this talk, I was thinking about my struggles in grad school, and I remember a time when I was sitting in Dr. Knox’s office, and I was crying, because I was trying so hard to understand a concept in mechanics that everybody else seemed to grasp. I remember leaving my prelim exam after 45 minutes and walking around the building for 10 minutes, because I didn’t think I could solve a single problem. And I remember during my oral exam, when Dr. Cebra asked me a particularly tricky question, completely freezing and not knowing how to answer it. There are all these struggles in grad school, and there are the same kind of struggles when you are a faculty member. I haven’t cried in my department chair’s office- yet, but I do have frustrating meetings with the administration, I’ve been denied funding sometimes, many times in fact, and I've sometimes even felt like a bad teacher. And that’s double worse for me, because that’s also what my research is about. The struggles with graduate student life and faculty life have a lot of parallels between them, and this is something that you should think about when you’re deciding whether or not you want to pursue an academic career.

Finally, if you want a job at a CSU type school, there were lots of things that I did as a graduate student that I did to prepare myself for this kind of job that I didn’t realize I was doing at the time. I was a particularly active graduate student: I did a lot of extracurricular activities, which, turned out, look really great on a CV for search committees for potential faculty, especially at CSU. Because while they want you to be able to do research, they also want to see that you’re a person and that you’re going to join a community. Faculty jobs, when they’re choosing you, they’re choosing a colleague, hopefully for life. So they want a person, they want somebody who’s going to invest in the community. And I can say from sitting on both sides of the search committee.

One thing you can do now is get involved in some service. These are the things that I know are available to get involved with on campus: there’s the Graduate Teaching Community, Teaching Assistant Consultants, the Graduate Student Association, Professors for the Future, Chancellor's Advisory Board, and Graduate Students’ League are just some of the options available to you. I think they’re all still in existence. Another thing is you can do is ask to lecture during the summer. After you earn Master's degree, you can talk to some people, and if you have a connection, somebody might bestow on you the privilege to teach a summer Physics 7 course or Physics 9. That looks really great on your application- having some teaching experience in addition to being a TA is really useful. You can also attend workshops and do certificate programs. At the Center for Educational Effectiveness, you can do these certificate programs through there. The American Association of Physics Teachers, the American Physical Society, and also AAS have workshops that are for effective teaching in the discipline, and those are also really great things to attend. Sometimes they happen before or after the conference, so you should talk with your advisor if you’re going to one of these conferences and see if this is something you’d be able to add it to your itinerary while you’re there. The other thing to do is educate yourself on what your research says about effective teaching and learning. This is so you can talk the talk and you can have a great teaching philosophy, and you can also wow them on your teaching demonstration with all your wonderful knowledge of research-based practices.


[applause]


Q: You’ve been both a candidate for and sitting on job committees, you know it’s very competitive to get positions. So, what are the three most important things you would suggest in an application? And another question, you can answer either one: what roles do gender and ethnicity play in hiring?

A: As for the first one: what are the things you can do on your job application? One thing that works for me, and I know people have other methods, is to spend a really long time, at least on that very first application, to spend a few weeks writing a teaching philosophy, sending it around different people, asking your advisor what they think about it. Same thing with your statement of research. Spend a lot of time on that: don’t just write something up and throw it out there to see if you’ve made the shot. I think I spent a month on my first application, and then the ones after that took a shorter period of time. That was one thing. I also think that it’s really helpful to tailor your cover letter to the specific program. What you don’t want to do is have a couple paragraphs about how amazing you are without talking about how your expertise is going to fit with their job call. So read the job call and use the words in the job call in your cover letter.

Dr. Knox: Can I just underscore that one? Because what we’re looking for is not just a person in a vacuum, but a person with a vision for what they’re going to do at our institution, so that’s important.

Maybe this was a little too detail oriented, but I do think it helped me: I reorganized my CV based on the job call: I sent a slightly different CV to each university, and that was because somebody would say they were interested in leadership opportunities, and someone would say they were interested in student research projects, so I would pull my CV apart and make the headings match that wording. That was one thing that I think really helped me too. I think that the number one thing to consider is that they aren’t looking for the best person, they’re looking for the best fit. So you need to sell yourself, you need to show them that you are a good fit for their institution.

The next question: how does gender and ethnicity play out in the search process? I will say that there is a lot of research that shows that if you are an underrepresented minority or a woman, you have a greater chance of being rejected from a job application. There is a lot of research that shows this. However, I will also say that, at least in my university, and this is on the other side that I can speak to, people are conscious of this bias and they try to make sure that it does not affect their decision. It’s never come down to a situation where we were like, ‘Oh, there’s this amazing person, and then there’s this other person that is the right color or gender,’ but we do discuss unconscious bias, and there are active efforts on campus to thwart it.

Q: When you arrived at Davis, did you come with the intention of doing physics education?

A: That’s a great question, and my story isn’t representative, but I’ll tell you my story. My story is that I was working on astrophysics at the University of Wyoming, I did my Master’s there. I studied quasars, I was so excited to do astronomy. I probably have a story similar to a lot of you. I loved astronomy growing up, my dad gave me a telescope when I was 3, it was our thing. But what I decided when I got there was that astronomy really looks like going through millions of spectra in a windowless lab. And that just didn't work with what I wanted to do every day. It’s not that all astronomers do that every day, but I realized that I was not really enjoying the research every day- I was enjoying the teaching. So I was thinking about what things I could do, I was thinking about applying to get my Master’s and applying to teach at community colleges, and then I just happened to have a friend who was in the graduate program in evolution and ecology here who as going through this same exact struggle. And she said, ‘You should come here, because Wendell Potter is here, and you could do your PhD in physics education research.’ So I applied.


Q: So you mentioned other things to do outside of just research and teaching that look good on job applications, like the Graduate Student Association and whatnot, so what about groups that give you those skills, but that the university may not be fond of? I’m in a graduate student union, and there's other groups that I’m in. I gain a lot of leadership skills and whatnot, but the university’s not happy with us, because there’s a lot of us who actively challenge them.

A: Wow, that’s a great question. I’ve actually never seen this played out, but if I were reviewing your application, I would see that as a plus. I can’t speak for all search committees, but I think having a role in some kind of organization where you are displaying leadership is a good thing rather than a minus. Also, you wouldn’t want to be accepted into a university that’s going to look down on something like that. Because if that’s something that you want to continue doing, you don’t actually want to be there. I think you should include it.


Q: One thing on your list was NSF review panels. So what did you want to say about that?

A: I wanted to talk about how I applied for an NSF grant, and amazingly, (I say amazingly because it was the first time I had applied for a grant, although I spent a lot of time on the application, so maybe not so amazing) but I amazingly, I got one, which is the first and only one I’ve gotten so far. Hopefully I’ll get a lot more after that. After I submitted that application, I sat on this committee, and I was in the room with this committee while we were debating all these applications and who should get funding, and the thing that I found most interesting about this process is how often we sort of had to go back and refer to what the call had said and whether or not this was actually meeting the call. So we had a lot of opinions, but when it came down to it, this is the same as the advice we were talking about for job applications: everybody has a great idea, and there’s also a lot of criticism you could give to any project. We’re not experts on judging these different proposals. Well, we are experts, but we’re not experts in your research. We don’t know your research, we don’t know all the reasons why you’re doing this. So, your job is definitely to make your case, but also, we had to often rely on what was written in the call.