5 Challenges to Collecting Data on the Psychology Workforce

Seven years ago, when I first began working at the Center for Workforce Studies (CWS), my colleagues and I were given the task of collecting, mining, analyzing, and disseminating data that would be relevant to the psychology workforce and education pipeline. I naively asked myself, “how hard could this be?”  After all, other disciplines seemed to be doing it just fine.  At the time, I had no idea just how immensely difficult this task would prove to be.

To understand our challenges, it may be helpful to provide a brief history of the Center for Workforce Studies.  The Center has been part of the APA in one form or another since at least the 1970s.  It has undergone numerous name changes and has been administratively housed in different APA units.  In the earlier years, the bulk of CWS data on the psychology workforce and education pipeline came from their own survey collection efforts.

For example, they collected several years’ worth of data on salaries for psychology faculty working at doctoral-granting departments. They also conducted numerous surveys that targeted psychology doctoral recipients.  From these surveys, they were able to gain insights into the career outcomes of psychology doctoral recipients, such as their employment status, postdoctoral training, perceptions of the job market, employment patterns, satisfaction, and debt levels.

In 2012, when I first joined a newly reorganized Center for Workforce Studies, we began our quest to address additional questions about the psychology workforce that seemed straightforward on the surface but were actually quite complicated to answer.

For example, how many students are awarded a psychology degree each year, in what subfield, and at what level?  What percentage of psychology degree holders progress to graduate school and receive an advanced degree?  How many licensed psychologists do we have in the United States?  Where are they located geographically and what populations do they serve? Do we have enough supply to meet the current demand?  Will we have enough psychologists to meet future demand (as well as unmet need)? Do we have enough of the right types of psychologists to meet the demands and needs of populations that are growing at a faster rate than the general population, such as older adults or various racial and ethnic minority groups?

As we set forth, we were faced with the following challenges:

Challenge #1:  There is no master data file for all psychologists.

By far, the biggest challenge CWS faces is that no psychology master file exists, where we can easily retrieve data about our psychology students and workforce, thus making it extremely challenging to longitudinally follow psychologists throughout their education training and subsequent careers.

In comparison, the physician workforce does have a master file.  When a student registers for the MCAT, they are given an identification number, which follows them throughout their medical career.  From this identification number, the physician workforce can track how many students move on to medical school, how many drop out, how many graduate, what they specialize in, where they do their residencies.  They can determine if there are enough doctors to meet current and future demand.  They can ascertain if there is an oversupply or shortage of certain types of doctors, such as neurologists or cardiologists.  They can easily assess the demographic makeup of this workforce. And they can look at results longitudinally.

How did we overcome this challenge?  Because a psychology master file does not currently exist, we cannot use one data source to determine any of this information.  Instead, CWS uses data that are available from multiple sources to answer these questions.  Fortunately, there are many federal statistical agencies that collect important data that CWS can use to answer questions regarding the psychology workforce and education pipeline. Although they are not perfect, federal statistical data sources are often a better and more reliable alternative to creating our own surveys. The data come from carefully designed surveys with large sample sizes that are representative of the population.  However, these data only give us snapshots of the psychology workforce at any given time.  We cannot follow a specific group of psychologists throughout their careers longitudinally.  Furthermore, these datasets don’t always give us the exact information we are looking for. As such, we must supplement these data with our own surveys, which brings us to the next challenge:

Challenge #2: Survey response rates have been dwindling over the years. 

Response rates for online and telephone surveys have seen a continuous decline in the last 20 years, making it a very time consuming and laborious process to gather data.  In addition, because we do not possess a national directory with emails of all U.S. psychologists, it is not possible to send surveys to many of these individuals.  In the past, CWS would often send surveys to the APA membership, presuming they were a representative sample of the psychologist population as a whole.  However, APA membership tends to skew more male and older than the general population of psychologists. Through mining data from the U.S. Census Bureau’s American Community Survey, we know that the psychologist population is actually slightly younger and more female.

For many years, APA maintained an email list of psychology graduate department chairs, which was used to disseminate an annual survey of faculty salaries.  This survey is one of the oldest conducted by APA, started in 1960 by the Council of Graduate Departments of Psychology (COGDOP), and after 1978 by APA in conjunction with COGDOP.  In 2012, the new CWS continued this survey on faculty salaries until it became clear that response rates were rapidly declining from one year to the next.

How did we overcome this challenge?  One solution to this problem was to retire the faculty salary survey and find an alternative data source.  In 2014, CWS began obtaining data on psychology faculty salaries from the College and University Association for Human Resources (CUPA-HR), a non-profit organization that has been collecting faculty salary data for all disciplines (including psychology) for the last 38 years.  The survey includes data for over 200,000 full-time faculty at 847 U.S. institutions.  An added benefit of using CUPA-HR data is that comparisons could now be made between psychology salaries and salaries of faculty from other disciplines.

Each year, CWS produces a report on psychology faculty salaries based on data provided by CUPA-HR.

Challenge #3: The psychology workforce is very complex.

The psychology workforce is more complex than other disciplines for a variety of reasons.  Psychology is characterized by overlapping, incomplete data sources that all have different ways of defining psychology, making it even more difficult to get a comprehensive picture of our workforce from a single data source.  As such, we must mine multiple data sources, which collect data on segments of the psychologist population.  They may overlap to some degree, but not entirely:

There are also multiple types of psychology doctorates, such as the PhD and PsyD.  Many federal data sources collect data on research doctorate recipients (e.g., PhD) but not for PsyD holders, making it impossible to gain information about our entire workforce from a single data source.  Another challenge is that psychologists can have a wide array of occupations, such as practitioners, academic faculty, researchers, and managers. They can be employed in many different work settings, such as hospitals, private practice, universities, government agencies, non-profit organizations, and corporations (just to name a few).  Although the diversity of the psychology workforce is impressive, it further complicates our research efforts.

How did we overcome this challenge?  The key to dealing with this challenge was to become very familiar with the many federal data sources that are publicly available.  These data sources provide an array of data pertaining to the psychology workforce or education pipeline.  Some examples include the National Science Foundation surveys on U.S. research doctoral degree recipients (e.g., the Survey of Doctorate Recipients; the Survey of Earned Doctorates), which provide data such as the demographics, education, career history, and postgraduate commitments of psychology research doctorate recipients.  The Census Bureau provides data on the psychology workforce through their American Community Survey.  The National Center for Education Statistics provides annual data on psychology degree recipients through its Integrated Postsecondary Education Data System (IPEDS).  Data on patient utilization of psychological services can be found in the Medical Expenditure Panel Survey, conducted by the Agency for Healthcare Research and Quality.

Challenge #4: Existing data sources can be limited and messy.

How many licensed psychologists with doctoral degrees are actively working in the U.S.? A seemingly simple question, right?  Just find a list with all licensed psychologists and start counting.  However, this is challenging to do, given there is no national database of licensed psychologists.  Instead, this information is collected by the individual states. And because there can be significant differences in licensure requirements from state to state, the information contained in these state licensure board lists can vary.

How did we overcome this challenge? To gain access to all of these lists, APA contacted the state licensure boards of all 50 states and Washington, D.C.  The result was 51 separate lists, containing different types of information and in different formats.  For example, some of the state lists contained email addresses for licensed psychologists, while many others did not.  Some state lists had a single field for first and last name, whereas other lists had them separated into multiple fields.  Some lists had both doctoral and master’s level psychologists, some had doctoral only, and with others, there was no variable that clearly distinguished between degree levels.  Given all of these inconsistencies, merging these files was difficult.  Then, of course, there was the added challenge of finding duplicate entries, such as individuals who appeared in multiple lists because they were licensed in more than one state.  To get an accurate count for the number of licensed psychologists in the U.S., these duplicates had to be noted.

So, how many psychologists are licensed in the United States?  It took almost two years to get an answer: In 2012, an estimated 106,500 psychologists possessed a license in the U.S.  This finding was published in a 2014 Datapoint of the APA Monitor and has been our most widely viewed Datapoint to date  In 2018, we were able to estimate even more accurately that the number of doctoral-level, actively employed licensed psychologists in the U.S. is approximately 95,000.

Challenge #5: With no master file, how can we determine the supply and demand of psychologists, now and into the future?

In 2018, CWS released the first-ever workforce projections for the current and projected supply and demand for psychologists.  This project was a large one, requiring the commissioning of an outside consulting group to do the analysis and an advisory group to provide professional feedback and review the results.  Because there is no master data file of psychologists, supply and demand numbers were determined using dozens of data sources, such as the American Community Survey, IPEDS, surveys from the National Science Foundation, the U.S. Centers for Disease Control and Prevention, the Center for Medicare and Medicaid Services, the Agency for Healthcare Research and Quality, and many more.  Also included in the analyses were APA membership data and our own survey of health service providers.

The final result: After years of knowing very little about the supply and demand characteristics of our workforce, the results from this project indicated that psychologist supply is projected to be insufficient to address the unmet need.  Although increasing the supply with additional graduates and fewer retirements will help, it will not entirely address the unmet need.  In fact, addressing supply alone may not address unmet need. There are many financial, social, cultural, and geographic factors that represent barriers to care (for more information on psychologist workforce projects, see https://www.apa.org/workforce/publications/supply-demand or our data tools at https://www.apa.org/workforce/data-tools/supply-demand-projections and https://www.apa.org/workforce/data-tools/interactive-state-level).

Where do we go from here?

Despite the challenges of doing research on the psychology workforce, the Center for Workforce Studies has developed new approaches to analyzing workforce data, which has allowed us to deliver research that is relevant to current and future psychology students, research psychologists, health service psychologists, public interest groups, and the general public. We provide annual or biennial reports on psychology faculty salaries, salaries for non-academic psychologists, the demographics of the psychology workforce, job advertisements in psychology, and much more.  Also available are monthly datapoints in the APA Monitor and interactive data tools that allow users to generate custom tables and answer questions regarding the psychology workforce and education pipeline.

Of course, there is still much more work to do.  In addition to updating reports and data tools with current data, new projects are always being developed.  One future project, in collaboration with APA’s Office of Graduate and Postgraduate Education and Training, will be to develop and deploy a graduate psychology alumni survey.  This survey will create a mechanism for tracking career outcomes and career pathways of psychology graduate degree holders across their lifetime.

What can you do to support CWS’s work?

Amplify CWS’s message by using our data.  Do you teach a psychology course? Use our Careers in Psychology data tool to show your students their career options. Do you need the numbers of psychology degrees awarded annually? Use our Degrees in Psychology data tool.  Are you on the job market and need reliable salary information for negotiations, or are you an employer who needs to make a fair salary offer? Use our Salaries in Psychology report for all types of psychologists, or our Psychology Faculty Salary report for academic psychologists.  Got a blog or social media following?  Share our work.

Place yourself within CWS’s research. What resonates with you?  What are you seeing on the ground or in the workplace?  How can you help psychology prepare for the future?

Give us feedback.  How can CWS serve as a resource for your work?  What workforce data do you wish we had available?  Drop us an email at cws@apa.org with your suggestions.

For more information on the Center for Workforce Studies, please visit www.apa.org/workforce.

About the Author

Peggy Christidis received her PhD in cognitive psychology from the Claremont Graduate University. She is currently a senior research officer for the Center for Workforce Studies at the American Psychological Association. Prior to coming to APA, she conducted functional magnetic resonance imaging research at the National Institute of Mental Health.