The COVID-AMP Project: Two Researchers Share Experience and Insight Tracking Local Policies

The COVID Analysis and Mapping of Policies (AMP) project is an effort of the Center for Global Health Science and Security at Georgetown. Its goal is to develop a comprehensive database of state and local policies and plans that have been implemented globally to address the COVID-19 pandemic. Housed on a website as part of the International Disease and Events Analysis (IDEA) platform, the database allows for simple searches.  Each policy or plan is categorized by the type of measure, implementation date, authorizing agency and legal authority, and by type of organization. Where available, PDFs or links to the original document or notice are included.  The project is partially supported by the Georgetown Global Cities Initiative.  

In June 2021 GGCI interviewed two staff members who are working on the COVID-AMP database. Siobhan Robinson Marshall is a research assistant at the Center for Global Health Science and Security. She is a recent graduate of Cornell, where she received her B.S. in global and public health sciences. Ariyand Aminpour is now pursuing his Masters in the GLID program. He graduated from Georgetown in 2021, majoring in the biology of global health and minoring in science, technology and international affairs.  

 

GGCI: Thanks so much for talking to GGCI about your work on the COVID-AMP database. Siobhan, let’s get started with a brief description of the project and your role. Then we’d like to hear about your role, Ariyand. 

 

Siobhan: COVID-AMP—or COVID analysis and mapping of policies—is a comprehensive database of policies in response to COVID-19. It's a fully searchable web platform with an interactive map. It's a really cool database, the results of a project that started in March 2020. At one point we had about 50 research assistants gathering and coding policies. Now we include almost 49,000 policies from all 50 states, including Washington D.C. and Puerto Rico; major cities like Los Angeles, New York, Dallas; some of the Atlanta area counties; and 66 countries. This has definitely been a collaborative effort. The entire database is in English.

Ariyand: I joined the COVID-AMP team in May 2020. I was a rising senior at Georgetown looking to get some hands-on research experience. I had just come back from study abroad, which was cut short by COVID-19. That's when I met Dr. Rebecca Katz who is director of the Center for Global Health Science and Security and learned about the research assistant position for the team. I am on the data collection side, looking at local policies from counties in Georgia to city policies in California to state policies in Massachusetts. I also had the opportunity to look at federal policies in Mexico because I have working proficiency in Spanish. I translated documents to put on our website in both Spanish and English. That was cool. 

The data collection was hands-on because you have to reach out to leaders from the local level all the way to the federal level to get access to documents or get a link that has all the policies in one place. 

Students on another project at Georgetown were working on COVID policy in Latin America. We teamed up to write research briefs on interesting trends we saw. I wrote a paper on reactionary policy choices the Mexican government was taking to handle COVID-19. For example, when it came to enacting certain business closures once hospital capacity reached 70 percent, we looked at the data to see the effect of policy during big tourist times like spring break. The data suggested that more restrictions on the front end of these holiday periods would allow the country to stay more open because they would not have a surge in cases on the back end after the tourist time ended. We had a webinar where we got to talk about our research.

 

GGCI: What was your undergraduate major? How were you prepared for the work?

 

Ariyand: I majored in the biology of global health, so I was prepared. I've been taking global health classes since I came to Georgetown with the intention to major in biology and pre-med. As time went on, I realized I was more interested in the policy side. So, I took on a biology of global health major and a science, technology, and international affairs minor, which allowed me to look at things from the level of federal organizations like HHS and other pandemic relief response organizations to the history of how we as a country have reacted to infectious disease outbreaks. That was the most interesting coursework I took at Georgetown, and it prepared me for this work. 

This current research experience is one of the most transformational experiences I have had at Georgetown. To do research and meet amazing people who want to see me succeed and give me more opportunities to show my best work is just great.

Siobhan: At Cornell I got my bachelor's in global and public health sciences. I graduated in 2020, a very interesting graduation time, and found a position as a research assistant at Dr. Katz’s center. I'm a DC native so I've always been interested in politics and policy. I can see the Capitol from my apartment, which is really cool. 

I wanted to understand policies and how they function in practice. That is the kind of urban work we do here. We try to figure out how these larger policy frameworks translate into actual policies for local leaders. Typically in a pandemic, policy comes out of the federal government. This time, however, policy came from states, counties and local areas. At one point our database had only one national policy for the US, which was crazy, and yet we had hundreds upon hundreds of state and local policies. It's been interesting to see this dynamic pan out in practice, even though it's not always positive. I have grown a lot and enjoyed working with so many different team members. 

 

GGCI: What has been the most difficult part of this project?

 

Siobhan: There are many challenges, but all are good learning opportunities. When I started, I had just graduated from undergrad and learning to manage a team of research assistants that were effectively my peers was hard. Even though the work is collaborative, at first it was a challenge to learn to convey information accurately when that information can be interpreted 10 different ways and then learn the routine of making sure we are defining things consistently. 

Another challenge was being adaptable and flexible. This project started in March 2020, and we thought it would be a quick sprint. We thought we would code policies for a couple of months until the pandemic ended. The next thing we knew, we were deep into the summer of 2020, and we could not keep up with the number of policies, the level of detail or the rapidly changing policy environment. 

A big challenge was creating a data structure that represented many different policy environments accurately. My favorite example is Palm Beach, Florida, where they used emergency orders to close services for two weeks and extended the emergency eight or nine times. We didn't expect emergency extensions to be so prevalent. 

Ariyand: For me the hardest thing is just the fact that I've been living COVID-19 non-stop for a year. Now a lot of people are coming out of the pandemic and beginning to realize that there is recovery. It has taken a toll on me to look at the numbers and the data constantly. When you’re in a pandemic, it would be nice to have a job outside of global health because at least there is a distraction. When you're in the weeds of the policies and how opposing decisions like mandating masks versus opening businesses affect case counts, it puts things into perspective. I have had my foot on the gas for an entire year. That may be the hardest part.

 

GGCI: You two could work your whole professional lives and not take on a project that has this much influence over the future. Researchers will use your database for years and years to come to ask life and death questions. The care you've taken with the information is going to make all the difference. Tell us how you clean the data, because that is a critical piece. 

 

Siobhan: It is always on my mind. We have been working on data clean-up since January. Clean-up is easier at a higher level like eliminating duplicates or taking care of other glaring problems. But we find that most policy documents contain many different policies and each one has to be defined and tracked. 

The level of detail within a policy document varies as well. A challenge arises when we break up the document, ask how to be consistent and at the same time allow for variation. You can't make a hard-and-fast rule because as soon as you do, an exemption comes up. 

Another real challenge is determining end dates for policies. Sometimes we feel that we are harassing our coders because it's really challenging to figure out when a policy ends. For example, businesses might be required to operate at 50 percent capacity for a period of time, but no end date is given. Unofficially the policy stands until another policy negates it, but nothing is official. Once the pandemic really hit in fall 2021, people stopped giving dates because reality was changing so fast. 

All the policies we have coded are from some form of government. In some ways we send the coders into Google oblivion because we define policies as anything with legal authority pertaining to COVID. We only want official policies with legal weight and backed up in practice. 

And then there are a few caveats. For example, we have a policy type called non-policy guidance for a select few policies.  Typically these exist in absence of formal policy on the local level, such as stay at home or safer at home. These are guidelines that are strongly recommended or urged and fall into a gray area from the government. They do have legal authority from a mayor's office or a governor's office, but are not necessarily enforceable. 

Ariyand: End dates are a challenge for me too. A big part of the research assistants’ position is keeping track of end dates because if you have open ended policies in your database, it looks like they go on forever. 

One of the cool parts of the COVID-AMP database is that it also has epidemiological models, which gives case counts and can show a place in a certain time. You can see when a policy was enacted, and you can see cause and effect after that policy was enacted. Start and end dates are critical—and they take a lot of collaboration. I recall one meeting when I was looking at Los Angeles County and at the the City of Los Angeles mayor’s office. Both were putting out their own policies and sometimes those policies referred to each other; for example, the City of Los Angeles used the stay-at-home order of the County of Los Angeles. It’s very confusing. There are many layers.

Another challenge is seeing how sub targets and sub policy categories have changed over time. In the year I have worked on this, I have seen just how granular these policies can get, ranging from directives on how to perform a funeral to whether you can go to a gym, and certain specifications like the difference between a restaurant, a bar and a nightclub. All of these sub-categories have different guidelines and policies. The ability to get that granular in the database is a work in progress. 

 

GGCI: I am impressed with all the collaboration, especially since the entire project has been done virtually. You and your teammates haven't been able to sit around a conference table and hash out the issues, which is another thing that's amazing. What would you like to say as we close the interview?

 

Ariyand: I want to stress that this research is opening many opportunities for me. I came into the project thinking that it would be a lot of data entry, and I'd be only a face behind the screen. The amount of opportunity the leaders have given us to analyze the data and give our own voice to what we are seeing in these policies is amazing. For me it has been life changing. 

Siobhan: I want to highlight the research assistants. This project would be nowhere without them. We've had multiple people come and go on the project. Some have volunteered and other students like Ariyand have given hundreds upon hundreds of hours tracking down policies, contacting people about end dates, dealing with changes, and being flexible, eager, and incredibly thorough and thoughtful in their work. You know, these students are truly the policy experts in their areas. They are the ones with the answers to the questions. Their effort is really cool. 

I also want to highlight the cool analyses the coders have produced. If you want to see the data in action, the coders’ voluntary research briefs are perfect. I’m especially commending the students who have been on the project for so long, living and breathing COVID all day every day, and in their spare time they produce these research briefs. The briefs are posted on our website. Please check them out.