Web-based tracking methods in longitudinal studies

Evaluation and Program Planning 45 (2014) 82–89

Contents lists available at ScienceDirect

Evaluation and Program Planning journal homepage: www.elsevier.com/locate/evalprogplan

Web-based tracking methods in longitudinal studies Izaak L. Williams a,*, Clifford R. O’Donnell b a b

University of Hawai‘i, 651 Ilalo St, John A. Burns School of Medicine, Department of Cell and Molecular Biology, BSB Building, Honolulu, HI 96813, USA University of Hawai‘i, Department of Psychology, 2530 Dole Street, Honolulu, HI 96822, USA

A R T I C L E I N F O

A B S T R A C T

Article history: Received 24 November 2013 Received in revised form 28 February 2014 Accepted 1 April 2014 Available online 13 April 2014

The use of online resources to reduce the attrition of program participants in longitudinal studies is examined. Higher-risk individuals, those involved in illegal activities, and females with last name changes are typically more difficult to locate. The effectiveness of using online resources for these participants is addressed. These resources include social networking sites, people-finder search engines, telephone and address directories, judicial records, and death records. The strengths and limitations of these resources are presented and discussed. Longitudinal studies using these resources are examined to evaluate their successful follow-up rates. The results of these studies indicate that participant characteristics are more important to successful follow-up than the length of time since participation or sample size. The use of multiple online sites increased follow-up rates, especially for those who are typically difficult to locate. The variables and websites to consider are discussed, and six lessons learned are offered. The prospective use of online participant involvement is especially important for successful longitudinal evaluation and program planning. ß 2014 Elsevier Ltd. All rights reserved.

Keywords: Longitudinal Follow-up Online resources Tracking Program planning

The purpose of this article is to examine the use of online resources in longitudinal studies of program participants. Longitudinal studies provide data for the evaluation of programs and ideas for program planning. Web-based resources are widely used, but their effectiveness in longitudinal studies has received scant attention. Only a few studies have provided useful information on the resources used, such as the name of tracking sites and search strategies, details on their advantages and disadvantages, and their success in finding information on participants (Barakat-Haddad, Elliott, Eyles, & Pengelly, 2009; Haggerty et al., 2008; Kleschinsky, Bosworth, Nelson, Walsh, & Shaffer, 2009). An important issue is the effect of attrition in longitudinal studies. Follow-up studies after 20 years without participant contact have shown that some participant groups are more difficult to contact than others (Barakat-Haddad et al., 2009; Hampson et al., 2001; Hser, Hoffman, Grella, & Anglin, 2001; Rodger, Lanigan, Hocking, & Crofts, 2001; Tehranifar, Terry, & Susser, 2002). Participants who are involved in illegal activities and women are among the most difficult groups to contact in follow-up research (Masson, Balfe, Hackett, & Philips, 2011). Family name changes among women after marriage increase the difficulty of tracking them. Individuals who are engaged

* Corresponding author. Tel.: +1 808 380 6852. E-mail addresses: [email protected] (I.L. Williams), [email protected] (C.R. O’Donnell). http://dx.doi.org/10.1016/j.evalprogplan.2014.04.001 0149-7189/ß 2014 Elsevier Ltd. All rights reserved.

in criminal behavior may conceal their identity online, assume false identities, and hide their location to evade arrest (Haggerty et al., 2008). Additional challenges presented by criminal populations are higher incidences of residential mobility, family estrangement, spontaneous relocations, lack of reliable contact information, phone disconnections, limited education, and substance abuse (Cotter, Burke, Stouthamer-Loeber, & Loeber, 2005; Haggerty et al., 2008; Kleschinsky et al., 2009; Masson et al., 2011; Wutzke, Conigrave, Kogler, Saunders, & Hall, 2000). Although attrition is always an issue in follow-up studies, the question is which online resources have empirical support and the utility to reduce attrition, especially for participants who are typically difficult to locate. Improving successful follow-up rates could provide a basis for evaluation guidelines and contribute to the creditability, validity, and generalizability of evaluation conclusions (Corsi, Hunnik, Kwiatkowski, & Booth, 2006). Several types of online resources have been used in longitudinal research, including social networking sites, people-finder search engines, telephone and address directories, judicial records, and death records. Each type is presented, and their purpose, cost, information required, and the types of information that can be obtained are discussed. The utility of each online resource is then described along with important considerations. Next, the followup success of longitudinal studies that used these methods to reduce attrition is assessed. Finally, lessons learned are offered and a conclusion reached.

I.L. Williams, C.R. O’Donnell / Evaluation and Program Planning 45 (2014) 82–89

1. Online resources 1.1. Social networking sites Social networking sites provide profile pages that contain information such as personal biographies, relationship statuses, networks, friends, ‘‘likes,’’ and ‘‘favorites.’’ All social networking sites include only people who sign up. Four sites used in longitudinal research are Classmates (classmates.com), Reunion (reunion.com), Facebook (facebook.com), and the New MySpace (Myspace.com). All memberships are free, but Classmates and Reunion also have fees for higher-level memberships. Classmates and Reunion memberships are based on school alumni and class directories. Matches of maiden and married names are particularly useful for longitudinal research with female participants. Facebook is a networking site with a large membership representing a cross-section of the United States, while New MySpace is a networking site designed for entertainers interested in promoting their music and networking with other entertainers. The information needed to join, the information obtained, and the strengths and limitations of each are presented in Table 1. For all of these sites, an important consideration is that the use of the Internet differs among participant groups by age, gender, income level, race/ethnicity, education, and geographical location. Obviously, contacting groups with lower rates of Internet use may require greater effort and may still result in lower follow-up rates. Internet use is higher among younger adults; white and Asian Americans; and those with higher incomes, with higher levels of education, with school-age children, and who live in urban areas (National Telecommunications and Information Administration (NTIA) and Economics, Statistics Administration (ESA), 2013; Zickuhr & Smith, 2012). About 80% of households in the United States use the Internet at home. The 20% who do not cite lack of interest, expense, or not having a computer (NTIA & ESA, 2013). Not surprisingly, these differences among demographic groups are reflected in the use of social networking sites (Bolanos et al., 2012; Duggan & Brenner, 2013; Rhodes & Marks, 2011). Facebook boasts over 150,000,000 users in the United States alone (Burbary, 2012). Alumni social networking sites, such as Reunion and Classmates, are popular among people over 45 (Ellison, Steinfield, & Lampe, 2007). Overall, social networking sites are increasing proportionally among people between the ages of 35 and 50 years of age, who now represent a demographic group of active users (Royal Pingdom, 2012).

83

One related issue is privacy. Access to profile information is determined by the privacy settings of individual members and their membership levels. Social networking site users have grown alarmed about privacy breaches and have significantly increased their privacy settings (Boyd & Ellison, 2007; Joinson, 2008). These users hide identifying information from ‘‘non-friends’’ and make their friends lists private so they are accessible only within their social networks (Masson et al., 2011). Users are increasingly wary about the type of personal information they reveal in order to safeguard their identities and protect against victimization or unwanted intrusions. They are concerned that personal profile information could be used for employment termination, screening job applicants, self-incrimination, corroborating evidence in criminal/civil trial proceedings, cyber-bullying, etc. In a study that began following participants using Facebook in 2010, around 83% of profiles openly revealed their friend list. Nearly one and half years later, 52.6% hid their friend list. In addition, relationship statuses, high school names, genders, graduation years, and hometowns were more likely to be excluded from profiles. Age may also make a difference. Participants over 55 were less likely to exclude information than those in their 20s (Dey, Jelveh, & Ross, 2012). When social networking sites produce many matches crosschecking names can be useful in correctly identifying participants, schools attended, and locations (Masson et al., 2011). Using multiple sites, name variations, and command search functions to sort through long lists of names are likely to increase the accuracy and effectiveness of a search. 1.2. People-finder search engines Unlike social networking sites, people-finder search engines lack self-selection bias; users do not need to sign up to be included in a database. Instead, people-finders provide information on virtually all groups across demographic dimensions. Therefore, people-finders are theoretically inclusive of anyone whether they use the Internet or not. The World Wide Web (WWW) is an expansive information grid with information divided into two layers: surface web and deep web. The surface web has information indexed by popular web search engines such as Google (google.com), Yahoo! People Search (people.yahoo.com), and Bing (bing.com). The vast amount of information is difficult to access and is inaccessible to web search engines that are traditionally used in follow-up work. The deep

Table 1 Social networking sites. Information needed

Information obtained

Strengths

Limitations

Classmates: Name, city/state of high school, year(s) attended

Maiden name, age, places lived, send messages, view profile biosketch; read and reply to messages; see if located out-of-state or in-state Age, middle name/initial, city/state locations lived in (with basic access); paid membership includes phone number, relatives/associates, education, work history, send messages Employer, job history, age, family members; biographies, comments, religious and political affiliation, status updates, hometown, personal activities, birthdate, relationships, friends list, friends in other networks, ‘‘likes’’ and ‘‘favorites’’ Short biographies, entertainment and music networks and ‘connections’, location, website address, full name

Wide collection of high school yearbooks back to 1900; preview digitized yearbooks; dynamic search filter; user-friendly; popular Includes schools in U.S. Territories, Canada, Armed Forces; nation/statewide search by name, location, job title, company, email, school

Restrictive search results without paid account; statewide name search not possible

Reunion: Name, city/state of high school, year(s) attended

Facebook: Email, name, location, school, group/membership affiliation

New MySpace: Name, city/zip code, gender, age/age range, profile type

Disorganized presentation of search results with basic access; limited capability of search filter

‘‘Surf’’ user’s ‘friend list;’ search browser facilitates finding user’s friends via multiple platforms; direct messaging; additional filters to search results; users represent a broad cross-section of the U.S. population

Common name matches produce many with similar names, some false; search filters limited to employment, location, school

Largest digital music library; can refine searches to a specific social group; links to Twitter, Pinterest, Tumblr Instagram

Emphasis on music; difficult to search; multiple similar names, some false; younger users

I.L. Williams, C.R. O’Donnell / Evaluation and Program Planning 45 (2014) 82–89

84 Table 2 Deep Web people finder search engines. Information needed

Information obtained

Strengths

Limitations

Wink: Name, city, state

Websites, work info, school, home address, photos, personal social networking profiles, age Voting records, personal profile pages, age, public records, photos, documents in online databases, home addresses, phone numbers, email addresses Photos, links, employment info, age, family info, partial home address

Can search by interests, favorite music artists, or hobby

Friend, phone, and email search by proprietary databases; limited search results Does not restrict results to specific criteria like age, gender, location

Pipl: Name, city, state

PeekYou: Name, city, state

Zabasearch: Name, city, state

Intelius: Name, city, state

Complete home addresses, phone number listings, middle initial/name; keeps a record of the ‘‘registration date’’ of addresses listed on the web (free open access); age (with ZabaPremium access) Middle name/initial, known aliases, age, address history, full name, relatives, phone numbers, birthdates, complete address (former and current), cell phone numbers, email; marriage, birth and death records, criminal record, background checks

Generates full names, residential addresses, obscure miscellaneous web documents, email addresses Results on address and age allows elimination of dated addresses; search username, interest, email, work, school; web history/‘footprint’ Details from broad variety of publicly available records; narrow results with added filters; advanced search for common names; information other sites charge for

Generates relatively more outdated addresses and telephone numbers

Most recent, information; Multidimensional web browsing, filtering; social network names search; detailed lists of addresses, name and contact information, reverse phone lookup; related people

Each email and phone lookup ranges in cost between $0.99 and $4.99; screen name $4.95 and $9.95; additional fees for more extensive information

web is also part of the WWW, but public information is estimated to be 400–550 times larger and the total quality content 1000– 2000 times greater than the surface web (Bergman, 2001). The deep web contains about one trillion content pages (Zillman, 2013). Deep web search engines allow access to websites, databases, and a wealth of content information not indexed by surface search engines. Therefore, our discussion focuses on five people-finder search engines that have been used in longitudinal research: Wink (wink.com), Pipl (pipl.com), PeekYou (peekyou.com), Zabasearch (zabasearch.com), and Intelius (intelius.com). All memberships are free, but Zabasearch and Intelius also have fees for higher-level memberships, and Intelius charges a monthly fee after a two-day trial period. Wink is principally a people-finder that combs the web for personal profiles on networking sites other than New MySpace and Facebook. Pipl is similar, but can generate results for more obscure web documents. PeekYou is essentially a hybrid of Pipl and Wink and sweeps the web with a wider search net. Zabasearch is designed to determine outdated addresses listed in web-based telephone directories and clarify the accuracy of addresses obtained from phone numbers (reverse phone look-ups). Intelius, a public records database browser, processes names across

Fewer results; often produces no matches; ZabaPremium access requires Facebook account and permission to mine personal profile data

government, proprietary, and commercial databases for any matches. Importantly, each people-finder search engine can produce a spectrum of different results and are most effective if used in combination with other sites. The information needed to join, the information obtained, and the strengths and limitations of each are presented in Table 2. 1.3. Telephone and address directories Information on three telephone and address directories that were used in longitudinal research is presented in Table 3: Switchboard (switchboard.com), Polk City Directories (polkcitydirectories.com), and Whitepages (whitepages.com). Switchboard is a free service. Polk City Directories has a menu of rates after a three-day trial period, and Whitepages relies on sponsors to provide full details for a fee. Using these directories to crosscheck phone numbers and addresses and to perform reverse searches can help eliminate invalid information. However, call-screening technologies, such as caller ID and voicemail, lowers the likelihood of making contact in follow-up studies (Faden et al., 2004). An important consideration for longitudinal studies is the type of telephone used by people of different ages (Rainie, 2010). For

Table 3 Telephone and address directories. Information needed

Information obtained

Strengths

Limitations

Switchboard: Name, city, state

Out-of-state numbers and full telephone and street addresses, and middle initial Age range, previous and current full address, known relatives, household members, estimated household income, estimated home value, years in home; consumer characteristics: lifestyle interest, registered voter status

Reverse telephone, address, Zip code

Tends to generate relatively more outdated addresses and telephone numbers Federal Trade Commission clearance needed to view telephone numbers if on Do Not Call list; high frequency of ‘U.S. consumer’ and ‘new mover’ search results provide only name and address information

Polk City Directories: Name, city, state

White Pages: Name, city, state

Age, middle initial, listing of city and state of residences; names of possible relatives and associates

Extensive list of possible name matches; data from postal service, census, voter registration, deed and assessment information, mail order buyers and responders, yellow pages and white pages; email campaign up to 20,000 contacts; ‘verified records’ 95 million consumer households; monthly updates to ‘consumer’ and ‘new movers’ databases; criteria to refine results Reverse phone and address lookup; sometimes provides full address and contact information at no cost

Unknown updates; results highly variable; relies upon sponsors to provide full details for fee

I.L. Williams, C.R. O’Donnell / Evaluation and Program Planning 45 (2014) 82–89

instance, about 75% of older adults (45–64) have landlines, while those under the age of 30 are more likely to use mobile phones and the Internet (chat, instant messaging, Skype) (Jones, 2012). Nearly 60% of households with adults 25–29 years of age have only a cell phone (Blumberg & Luke, 2012). As age increases, so does the likelihood that an individual will be listed in a telephone directory. Regional, income, and racial/ethnic differences in mobile phone usage also exist. For example, the Midwestern United States has the highest level of mobile-only households (Blumberg & Luke, 2012). Lower income adults are more likely to live in households with only cell phones. In comparison to white households with no landline and only a cell phone (30%), mobile-only households are proportionately more likely to be black or Latino (46%) (Blumberg & Luke, 2012). An additional difficulty is found in locating cell phone numbers. Currently no accessible cell phone databases are available to identity subscribers or perform reverse searches as exists in landline directories. 1.4. Judicial records Information on seven sources of judicial records that were used in longitudinal research are presented in Table 4: Federal Bureau of

85

Prisons: Inmate Locator (bop.gov/inmate_locator); InmatesPlus (inmatesplus.com); Vinelink (vinelink.com); CriminalSearches (criminalsearches.com); state criminal, civil, and traffic records; state sex-offender registries; and the National Sex Offender Registry (nsopw.gov). All are free services. Cases with a range of years can provide an indication of residency, which could be cross-referenced with social networking site data to help identify individuals of interest (Masson et al., 2011). It is important to note that public criminal record databases provide information on adults who have been arrested or convicted of an offense. However, certain demographic groups may be differentially represented in the judicial system’s enforcement (policing and arrest) and application (conviction and sentencing) of the law (Abrams, Bertrand, & Mullainathan, 2012; Anwar, Bayer, & Hjalmarsson, 2012; Beckett, Nyrop, & Pfingst, 2006; Fagan & Davies, 2000; Levin, Gettman, & Siegel, 2010a; Levin, Gettman, & Siegel, 2010b). For example, data consistently show that drug consumption rates of African Americans and Hispanics are not higher than European Americans (McCabe et al., 2007; Substance Abuse and Mental Health Services Administration (SAMHSA), 2011; Wu, Woody, Yang, Pan, & Blazer, 2011), yet they account for an overwhelming share of drug-related

Table 4 Judicial records. Information needed

Information obtained

Strengths

Limitations

Federal Bureau of Prisons: First and Last name must be an exact match or inmate ID number (one of four types) Department of Corrections (DOC) State Prison (via InmatesPlus): DOC number or last name

Age, race, sex, release date (actual or projected), middle name/initial, offender/register #

Search parameters by race, age; pretrial detainees and pre-sentenced offenders for the U.S. Marshals Service and Immigration and Customs Enforcement

DOC number, name, location of incarceration; birth date, physical description, expected release and discharge date, sentence, criminal charges, parole record/status Age, race/ethnicity, photo, aliases, inmate number, in-custody status of offender: expected release date, length of sentence

Efficient in states where VINElink is available

Name must be an exact match; search by inmate # better match than name; 4 types of inmate ID numbers; only federal offense data; records back to 1982 Data often limited to offender ID # and custody status; search state and county jails separately; criminal charges not always available; not every state has county arrest data Limited or no searches in 9 states; offenders deleted from database within two weeks of release; no data on federal inmates; does not track outstanding warrants; no historical data on offender

Department of Corrections (DOC) County Jail (via VINElink): (Partial) First and/or (partial) city, state, age range, date of birth, offender ID CriminalSearches: Name, city, zip code, state, home address, AKA/aliases, offense type, year of birth, age range

Age, middle initial/name, last known address; names of possible relatives and associates; pictures; offense code, location of crime, offense, physical description, birth year

State Criminal Records: Name, aliases, Also Known As (AKA) names

Maiden names, aliases, prison records, some county courts; full name, case numbers; relationship and names of parties or criminal accomplices

State Civil and Traffic Records: Civil: name, aliases, Also Known As (AKA) names Traffic: name, drivers license #, date of birth

Civil: temporary restraining orders (TRO’s), divorce, child support, judgments, small claims, bankruptcies, motor vehicle torts, home foreclosure, property information; Traffic: case information and status Pictures, complete address of residence; offender status (non-compliant or registered)

State Sex-Offender Registries (SORs): Name, offender type, zip code, city, radius, street name, offender type

National Sex Offender Registry: First and last name

Convictions of sexually violent offenses against adults and children and some other crimes against minors; identifying information: year of birth, picture, physical description, aliases

DOC and participating VINElink county jails; Spanish web content; userfriendly interactive map; in 47 states and Puerto Rico; data is current every 15 min Nationwide searches; email alert updates on new criminal offenses; data on sex offenders, criminal statistics, neighborhood watches, mapping of neighborhood crime statistics, offenders; user-friendly Records from local court level, state police, disposition available; records updated daily to weekly: current to within 48 h of the date a court document is filed or a court proceeding is heard Same as criminal records (above)

Sex offender and offender-againstminor database; updated daily; greater validity of search results than nationwide searches; filters allow for expansive search; shows when each individual profile was last updated Age, public sex offender registries from 50 states, DC, five U.S. territories, Indian tribes; current residences, employment, school attendance, picture; date of birth, racial/ethnic characteristics; verification of addresses: 90 days for 10-year registrant and annually for lifetime registrant

Some inaccurate name matches; data often incomplete; no record of how far back records go or how often updated

Frequent errors; some user-unfriendly records access and retrieval; some records may only go back 15 years

Civil: same as criminal records; traffic: does not provide social security #, birth dates, home addresses and home telephone numbers

Some questionable accuracy; public access restricted in some states; considerable range in registered sexual offenses; includes statutory minor offenders in consensual activity Information varies by jurisdiction; majority of registrants lack photograph; some offenders do not register or provide business or false addresses

86

I.L. Williams, C.R. O’Donnell / Evaluation and Program Planning 45 (2014) 82–89

Table 5 Overview of death records. Information needed

Information obtained

Strengths

Limitations

Social Security (DMF): Name or SSN; obtaining SSN numbers requires application for SS card of deceased and federal approval

Date of death, zip code, state or country of last residence, birthplace, lump sum payment and parent’s names (when available)

Includes all with SS number, whose death was reported (including in foreign country), updated weekly/ monthly; genealogy research tool; may request detailed income earnings; where benefit paid could be used as a proxy for where death occurred or last residence; no restriction on use of data; most accurate for older population

National Death Index: IRB/federal approval; name, gender, date of birth, SS number

Cause and circumstances of death

Consolidated state records since 1979; records based on certificates of death; detection sensitivity for death record same for all; myriad of personal identifiers accurate and reliable ‘‘gold standard;’’ names linked to all known aliases

Legacy: Name

Full name, date of birth, age, name of significant other, relatives, family members, city and state of residence before and at time of death, funeral home, degrees, date of marriage Date, place, and cause-of-death codes; occupation, residence, family members, and SS numbers when available Cause of death, description of injury; SS numbers, address, date of birth, sex, age, residence zip code, race, occupation, marital status, dispositions; relatives

Funeral home and newspaper affiliates (some international) and expedient search of social security death index

Record may take 2 months to 1 year after death is reported; may not provide complete information on cause of death; excludes those without a SS number or whose death was not reported; information varies by age, gender, and marital status; Approximately 10–25% of deaths since 1990 are not in database; About 98% percent of the people in the SSDI died after 1962; Approximately 2/3 of death records before 1988 do not contain the date of death Expensive; Extensive application process; no records before 1979; annual updates with two year delay in availability from time of death; marital status, state of residence, father’s surname, state of birth and dates of death may also be required to process search request Content can be very basic or in-depth, highly variable

Ancestry: Name Health Department (state and district level): Name, place of death, date of death

arrests (American Civil Liberties Union, 2013; Mauer & King, 2007). The importance of this issue depends on the purpose of the longitudinal study. It is highly important in studies that compare groups that differ in ethnicity/race and socioeconomic status, but much less important for studies without these comparisons. 1.5. Death records Information on five sources of death records that were used in longitudinal studies is presented in Table 5: the Social Security Administration (ssa.gov), the National Death Index (cdc.gov/nchs/ ndi.html), Legacy (legacy.com), Ancestry (ancestry.com), and the State Departments of Health (cdc.gov/mmwr/international/ relres.html). All of the sites except Legacy have a menu of fees. The Social Security Administration database contains the death notices of U.S. Social Security Program participants since 1932 (dmf.ntis.gov). The database currently contains over 92 million records of deceased persons with a social security number whose death was reported to the Social Security Administration (Hill & Rosenwaike, 2001–2002). The National Death Index is a centralized electronic database of all deaths in the United States since 1979 originating from state vital statistics offices and maintained by the National Center for Health Statistics (cdc.gov/nchs). National Death Index Plus (cause of death) and standard National Death Index (date of death) are two available search functions in this database. The National Death Index was established as a tracking site for morbidity and mortality outcomes among people diagnosed or treated for certain diseases to identify health risks and problems and for occupational health follow-up studies. It was not intended for general use by the public. Legacy is a publicly accessible database of obituaries of 75% of all people who died in the United States. Sorting people with the

State and federal death, burial, cemetery, and obituary records Includes cause of death, statistical file, death names/data by area; userfriendly application process; expedient processing of death certificates via VitalChek; monthly updates

Often difficult to sort people with same names; most public records not available online Strict confidentiality laws; timeprohibitive data files dating back to 1968; not all certificates online; application to smaller counties may require form via fax, mail, or in person

same names can be difficult. Legacy is associated with Social Security Administration records and over 900 newspapers worldwide, and provides an easy-to-search browser accompanied by an interactive map of the United States to facilitate searches. Ancestry is a public genealogy database containing 11 billion records. Most of these are from the United States, 40 million family trees, and a card catalog of 1715 sources on death, burial, cemeteries, and obituaries, including state vital records accessible from the Social Security Administration. The State Departments of Health provide links to departments of health in each state. The procedures to obtain information vary by state. For example, death certificates in the state of Washington are filed with the county’s health department or the office of vital statistics where the person died. The county can only issue certificates for people who die in that county. Washington has 39 counties, and the method of filing the death certificate varies across the state. Of those counties, 22 currently file paper certificates and 17 upload the record electronically soon after the person dies. Once the Department of Health receives the record from the county, the Center for Health Statistics can issue the death record. The Center for Health Statistics has a record of all deaths that occur in the state of Washington. New York does not maintain death records or issue copies of death certificates as is done in many other states. Instead, a request must be made to one of the five boroughs’ Office of Vital Records via the Department of Health and Mental Hygiene. Additionally, New York will only release a death certificate copy to the spouse, child, or sibling of the deceased or another person who has (1) a documented medical need, (2) a State Court order, or (3) a documented lawful right or benefit claim. In Louisiana, death records are ‘‘strictly confidential’’ until 50 years after the year of death, and access is only granted to persons

I.L. Williams, C.R. O’Donnell / Evaluation and Program Planning 45 (2014) 82–89

named on the document or their beneficiaries. In states with these restrictions, it may be possible for evaluators to be granted access to the records by the state registrar or a court injunction. Death records are not available through the Freedom of Information Act. The Centers for Disease Control and Prevention (cdc.gov/nchs/ w2w.htm) provides a menu of reference links that route users directly to state vital records departments for each state and territory. Important considerations are that multiple death record sources control for false matches and reconcile misclassification and discrepancies in age and gender. In addition, coding of race/ ethnicity in death records may differ from those used in a particular study, e.g., different codes for Hispanic participants. Social Security numbers for all death record sites greatly improve reliability and death record detection. 2. Longitudinal studies Longitudinal studies that used online resources provide important data on the success of locating participants for follow-up. In our examination of studies that used social networking, people-finder, or telephone and address sites, the success rates ranged from 8.7% (n = 551) (Bolanos et al., 2012) to 81.8% (n = 22) (Amerson, 2011). An examination of these two studies showed that both used Facebook but had important differences that could be responsible for the wide differences in outcomes: length of follow-up (eight years and one year, respectively), characteristics of the participants (adult methamphetamine users and nursing students, respectively), and sample size. These differences raise the question of which characteristics were more likely to affect the follow-up success rates. Comparisons of four other studies that also used Facebook suggest that participant characteristics may be more important than the follow-up period or sample size. Three of these studies had similar follow-up periods of two or three years and relatively large samples but different success rates. The rate was lower with at-risk, low-income families (16.2%; n = 119) (Mychasiuk & Benzies, 2012), higher with mothers of newborns who received a college savings account (32.0%; n = 919) (Rhodes & Marks, 2011), and still higher with eighth grade girls (44.6%; n = 175) (Jones, Saksvig, Grieser, & Young, 2012). The fourth study, even with a follow-up of over 40 years, located 20.0% of school children and leads on another 26.7% (n = 60) (Marsh & Bishop, 2013), which supported the greater importance of participant characteristics than the length of the follow-up period or sample size. Using multiple sites may improve success rates with harder-tolocate participants. Studies that used Facebook and MySpace reported a 23.2% rate (n = 151) for the recruitment of mothers of high-risk infants that could not be located by other means (Nwadiuko, Isbell, Zolotor, Hussey, & Kotch, 2011) and a 30.8% rate (n = 117) in a 10-year follow-up of youth with behavior problems (Masson et al., 2011). In addition, two long-term longitudinal studies included the deep web among the multiple sites they used. One of these studies, a 40-year follow-up of elementary students using one deep web site and two telephone/address directories (Merlin and DirectoryUSA, which are no longer available, and Switchboard), reported a 20.7% success rate (n = 2403) (Hampson et al., 2001). Additional online sites became available 12 years later, so 12 of them were used (the four social networking sites, five deep web sites, and three telephone/address directories discussed above) in a 35-year follow-up of high-risk youth. This follow-up resulted in an 86.1% success rate (n = 553) (O’Donnell and Williams, 2013). The large difference in the rates of these two studies suggests that using multiple sites may be especially worthwhile with participants who are typically more difficult to locate.

87

The highest success rates were found in two studies that used judicial sites (Vinelink, InmatesPlus, and Federal Bureau of Prisons: Inmate Locator) to follow up with drug abusers. These studies showed follow-up rates of 83% (n = 987) after 4–13 years (Corsi et al., 2006) and 92.8% (n = 222) after 3 years (Passetti, Godley, Scott, & Siekmann, 2000). These high rates demonstrate the effectiveness of judicial sites to track former prisoners. Judicial sites can also be useful in any study to see if participants were arrested or incarcerated. For example, state judicial records were used to obtain arrest rates in a 35-year follow-up of high-risk youth who had already been located using multiple other sites (O’Donnell and Williams, 2013). An innovative aspect of that study was the use of civil and criminal records to obtain the names of other people mentioned in these records. A criminal record search was then conducted on these names, and the results were analyzed to assess the effect of different types of relationships. Most longitudinal studies that used death records cannot be compared due to variations in the age and health status of the people traced and in the time elapsed since program participation. Obviously, studies with older participants with greater health risks will have higher death rates and studies with less time between participation and follow-up will have lower death rates. However, it is always important to know which participants are deceased and which are presumably alive and therefore possible to locate. One study directly compared Ancestry with the National Death Index to assess the mortality of college alumni (n = 750 men and 250 women) (Sesso, Paffenbarger, & Lee, 2000). The matches were excellent for men and women presumed still alive (97.5% and 98.2%, respectively) and for deceased men (94.7%) but not for deceased women (31.1%). Therefore, it was concluded that either Ancestry or the National Death Index could be used for men, but the National Death Index was much better to assess the mortality of women.

3. Lessons learned The first lesson learned is that the use of online resources reduces attrition in follow-up evaluations. All the longitudinal studies discussed located program participants, some even after 40 years (Hampson et al., 2001; Marsh & Bishop, 2013). In one study, 44.6% of eighth grade participants were located three years later after all traditional methods failed (Jones et al., 2012). The second lesson is that the use of multiple online resources is particularly effective for participants who are typically difficult to locate. If one procedure does not find a sufficient sample, others might. A special case of attrition is the death rate among participants. Longitudinal studies have different purposes. In addition to knowing how many participants have died, it may be important to know the dates and causes of their deaths. Death record databases use different procedures, have different purposes, and therefore vary in determining mortality statuses and providing information about the deaths. The third lesson is that evaluators need to select the databases that are most likely to provide useful information for their purpose. If the purpose is to learn which participants are deceased, the National Death Index is best with female participants (Sesso et al., 2000). The fourth lesson is that several variables should be considered when deciding which digital resources to use. Among those we recommend are cost, the technological development of the webpage, search capabilities, regularity of site updates, links to affiliates or content partners, whether the site offers open access or restrictions on access, user-friendliness (e.g., helpful links and a lack of pop-up advertisements), and accessibility to other users and the site developer. The number and characteristics of people in the database is particularly important because these variables will

88

I.L. Williams, C.R. O’Donnell / Evaluation and Program Planning 45 (2014) 82–89

determine the amount and type of information that might be obtained from each online source. The fifth lesson is that important issues should be considered before deciding on online sites for follow-up studies. Will some groups be difficult to contact? Are some groups less likely to use the Internet? If so, are they less likely to be included in the site database? Which online sites are more likely to reduce attrition? Did the procedures used in constructing the database create any bias among demographic groups? Will privacy restrictions reduce the effectiveness of an online search? As noted, the use of multiple sites, including deep web sites, is highly recommended to address these issues and increase the effectiveness of follow-up searches. In addition to the websites discussed, many others could be used in future longitudinal studies. Among those are Accurint (accurint.com), Been Verified (beenverified.com), and People Smart (peoplesmart.com), all of which conduct searches of public records. TLO (tlo.com) is another site that conducts searches of public and proprietary records. Some are more specialized; Google Person Finder (http://google.org/personfinder/global/home.html) assists people in finding others after natural and human disasters, LinkedIn (linkedin.com) offers a global database of 225 million professionals, and Rapleaf (rapleaf.com) matches email addresses to demographic and personal data. Of course, many more will continue to be developed and all online resources are likely to change and be updated over time. Although our examination of longitudinal studies published in English resulted in a greater US focus, there are numerous international sites. Many parallel those presented and can be located by an online search of international social media. Longitudinal studies can be conducted prospectively and retrospectively. The sixth lesson is important in program planning. Programs could construct their own social media sites to be used for participants to discuss their lives during the program, what their lives were like before program participation, and their lives after leaving the program. Prospective studies that involve their participants in social media that are selected or designed for the program are likely to maintain contact with them after they leave the program. This contact could increase the success of obtaining follow-up data. Therefore, prospective studies are highly recommended for program planning and longitudinal evaluations. 4. Conclusion Our examination of the use of online resources has provided specific information about these resources and recommendations for their use in longitudinal studies of program participants. However, in some studies, traditional follow-up methods, in addition to or instead of online resources, may be useful to reduce costs or contact specific participants. Obviously in studies with available contact information on participants, it would make sense to use that information first. Then online resources could be used to locate those with outdated contact information. Maintaining participant contact information might be especially useful for participants who do not use a computer, are likely to have a name change, or are members of groups that are typically difficult to locate. With available contact information for these individuals, traditional methods may be less costly and more effective. When resources allow, providing incentives for participants to maintain contact and be available for follow-up could reduce attrition. The incentives could range from information about study findings to payments for their time. Prospective longitudinal studies that involve participants in social media selected or designed for the program have the opportunity to create natural incentives for participants to identify with the program and care about other participants, much as successful alumni groups, sport teams, and fan bases do. Identification with the program and caring

about those who participated make successful follow-up more likely. Prospective studies also have other advantages. Any institutional review board (IRB) issues can be addressed when the study is approved, including the use of social media and follow-up procedures. Permission to contact program participants using social media can be requested when participants are recruited and be specified in IRB applications. Social media that are constructed for the study can be maintained or changed, as needs change over the course of the longitudinal period, in contrast with the online resources in this examination, which are certain to require updating of follow-up procedures over time. References Abrams, D. S., Bertrand, M., & Mullainathan, S. (2012). Do judges vary in their treatment of race? Journal of Legal Studies, 41, 347–383. American Civil Liberties Union (2013). The war on marajuana in black and white: Billions of dollars wasted on racially biased arrests. New York: American Civil Liberties Union. Amerson, R. (2011). Facebook: A tool for nursing education research. Journal of Nursing Education, 50, 414–446 http://dx.doi.org/10.3928/01484834-20110331-01 Anwar, S., Bayer, P., & Hjalmarsson, R. (2012). The impact of jury race in criminal trials. The Quarterly Journal of Economics, 127, 1017–1038 http://dx.doi.org/10.1093/qje/ qjs014 Barakat-Haddad, C. P., Elliott, S. J., Eyles, J., & Pengelly, D. (2009). Predictors of locating children participants in epidemiological studies 20 years after last contact: Internet resources and longitudinal research. European Journal of Epidemiology, 24, 397– 405 http://dx.doi.org/10.1007/s10654-009-9364-5 Beckett, K., Nyrop, K., & Pfingst, L. (2006). Race, drugs, and policing: Understanding disparities in drug arrests. Criminology, 44, 105–137 http://dx.doi.org/10.1111/ j.1745-9125.2006.00044.x Bergman, K. M. (2001). White paper: The deep web: Surfacing hidden value. Journal of Electronic Publishing, 7, 1–35 http://dx.doi.org/10.3998/3336451.0007.104 Blumberg, J. S., & Luke, V. S. (2012). Wireless substitution: Early release of estimates from the National Health Interview Survey, January–June 2012. Washington, DC: US Department of Health and Human Services Centers for Disease Control and Prevention National Center for Health Statistics. Bolanos, F., Herbeck, D., Christou, D., Lovinger, K., Pham, A., Raihan, A., & Brecht, M. L. (2012). Using Facebook to maximize follow-up response rates in a longitudinal study of adults who use methamphetamine. Substance Abuse: Research and Treatment, 6, 1–11 http://dx.doi.org/10.4137/SART.S8485 Boyd, D. M., & Ellison, N. B. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13, 210–230 http:// dx.doi.org/10.1111/j.1083-6101.2007.00393.x Burbary, K. (2012, October). Facebook demographics revisited – 2011 statistics Retrieved from http://www.kenburbary.com/2011/03/facebook-demographics-revisited2011-statistics-2. Corsi, K. F., Hunnik, B. V., Kwiatkowski, C. F., & Booth, R. E. (2006). Computerized tracking and follow-up techniques in longitudinal research with drug users. Health Services and Outcomes Research Methodology, 6, 101–110 http://dx.doi.org/ 10.1007/s10742-006-0003-2 Cotter, R. B., Burke, J. D., Stouthamer-Loeber, M., & Loeber, R. (2005). Contacting participants for follow-up: How much effort is required to retain participants in longitudinal studies? Evaluation and Program Planning, 28, 15–21 http://dx.doi.org/ 10.1016/j.evalprogplan.2004.10.002 Dey, R., Jelveh, Z., & Ross, K. (2012). Facebook users have become much more private: A large-scale study. 4th IEEE International Workshop on Security and Social Networking (SESOC) Available at http://cis.poly.edu/~ross/papers/FacebookTrends.pdf. Duggan, M., & Brenner, J. (2013, August). The demographics of social media users 2012 Pew Research Center’s Internet & American Life Project Retrieved from http:// pewinternet.org/~/media/Files/Reports/2013/PIP_SocialMediaUsers.pdf. Ellison, N. B., Steinfield, C., & Lampe, C. (2007). The benefits of Facebook ‘‘friends:’’ Social capital and college students’ use of online social network sites. Journal of Computer-Mediated Communication, 12, 1143–1168 http://dx.doi.org/10.1111/ j.1083-6101.2007.00367.x Faden, V. B., Day, N. L., Windle, M., Grube, J. W., Molina, B. S. G., Pelham, W. E., Jr., & Sher, K. J. (2004). Collecting longitudinal data through childhood, adolescence, and young adulthood: Methodological challenges. Alcoholism: Clinical and Experimental Research, 28, 330–340 http://dx.doi.org/10.1097/01.ALC.0000113411.33088.FE Fagan, J., & Davies, G. (2000). Street stops and broken windows: Terry, race and disorder in New York City. Fordham Urban Law Journal, 28, 457–504 http://dx.doi.org/ 10.2139/ssrn.257813 Haggerty, K. P., Fleming, C. B., Catalano, R. F., Petrie, R. S., Rubin, R. J., & Grassley, M. H. (2008). Ten years later: Locating and interviewing children of drug abusers. Evaluation and Program Planning, 31, 1–9 http://dx.doi.org/10.1016/j.evalprogplan.2007.10.003 Hampson, S. E., Dubanoski, J. P., Hamada, W., Marsella, A. J., Matsukawa, J., Suarez, E., & Goldberg, L. R. (2001). Where are they now? Locating former elementary-school students after nearly 40 years for a longitudinal study of personality and health. Journal of Research in Personality, 35, 375–387 http://dx.doi.org/10.1006/ jrpe.2001.2317

I.L. Williams, C.R. O’Donnell / Evaluation and Program Planning 45 (2014) 82–89 Hill, M. E., & Rosenwaike, I. (2001–2002). Social Security Administration’s Death Master File: The completeness of death reporting at older ages. Social Security Bulletin, 64, 45–51. Hser, Y. I., Hoffman, V., Grella, C. E., & Anglin, M. D. (2001). A 33-year follow-up of narcotics addicts. Archives of General Psychiatry, 58, 503–508 http://dx.doi.org/ 10.1001/archpsyc.58.5.503 Joinson, A. N. (2008). Looking at, looking up or keeping up with people? Motives and use of Facebook. Paper presented at the 26th annual SIGCHI conference on Human Factors in Computing Systems, Florence, Italy. Jones, G. (2012, November). Phone wars: More people drop landlines for cell phones 14NEWS CBS5AZ Retrieved from http://www.14news.com/story/17176473/phonewars-cell-vs landline?clienttype=printable. Jones, L., Saksvig, B. I., Grieser, M., & Young, D. R. (2012). Recruiting adolescent girls into a follow-up study: Benefits of using a social networking website. Contemporary Clinical Trials, 33, 268–272 http://dx.doi.org/10.1016/j.cct.2011.10.011 Kleschinsky, J. H., Bosworth, L. B., Nelson, S. E., Walsh, E. K., & Shaffer, H. J. (2009). Persistence pays off: Follow-up methods for difficult-to-track longitudinal samples. Journal of Studies on Alcohol and Drugs, 70, 751–761. Levin, H. G., Gettman, J. B., & Siegel, L. (2010a). Targeting blacks for marijuana: Possession arrests of African Americans in California, 2004–2008. Los Angeles, CA: Drug Policy Alliance. Levin, H. G., Gettman, J. B., & Siegel, L. (2010b). Arresting Latinos for marijuana in California: Possession arrests in 33 cities, 2006–2008. Los Angeles, CA: Drug Policy Alliance. Marsh, J., & Bishop, J. C. (2013). Challenges in the use of social networking sites to trace potential research participants. International Journal of Research and Method in Education, 36, 1–12 http://dx.doi.org/10.1080/1743727X.2013.820642 Masson, H., Balfe, M., Hackett, S., & Phillips, J. (2011). Lost without a trace? Social networking and social research with a hard-to-reach population. British Journal of Social Work, 43, 24–40 http://dx.doi.org/10.1093/bjsw/bcr168 Mauer, M., & King, R. S. (2007). Uneven justice: State rates of incarceration by race and ethnicity. Washington, DC: The Sentencing Project. McCabe, S. E., Morales, M., Cranford, J. A., Delva, J., McPherson, M. D., & Boyd, C. J. (2007). Race/ethnicity and gender differences in drug use and abuse among college students. Journal of Ethnicity in Substance Abuse, 6, 75–95 http://dx.doi.org/ 10.1300/J233v06n02_06 Mychasiuk, R., & Benzies, K. (2012). Facebook: An effective tool for participant retention in longitudinal research. Child Care, Health and Development, 38, 753–756 http://dx.doi.org/10.1111/j.1365-2214.2011.01326.x National Telecommunications and Information Administration (NTIA), & Economics and Statistics Administration (ESA), (2013). Exploring the digital nation: America’s emerging online experience. Washington, DC: National Telecommunications and Information Administration. Nwadiuko, J., Isbell, P., Zolotor, A. J., Hussey, J., & Kotch, J. B. (2011). Using social networking sites in subject tracing. Field Methods, 23, 77–85 http://dx.doi.org/ 10.1177/1525822X10384088 O’Donnell, C. R., & Williams, I. L. (2013). The Buddy System: A 35- year follow-up of criminal offenses. Clinical Psychological Science, 1(1), 54–66 http://dx.doi.org/ 10.1177/2167702612456907 Passetti, L. L., Godley, S. H., Scott, C. K., & Siekmann, M. (2000). A low-cost follow-up resource: Using the World Wide Web to maximize client location efforts. American Journal of Evaluation, 21, 195–203 http://dx.doi.org/10.1177/ 109821400002100206

89

Rainie, L. (2010). Internet, broadband, and cell phone statistics. Washington, DC: Pew Internet & American Life Project. Rhodes, B. B., & Marks, E. L. (2011). Using Facebook to locate sample members. Survey Practice, 4(5) Retrieved from http://www.surveypractice.org/index.php/SurveyPractice/article/view/83/html#r5. Rodger, A., Lanigan, A., Hocking, J., & Crofts, N. (2001). Tracing and recruiting a cohort with community acquired hepatitis 25 years later. Australian and New Zealand Journal of Public Health, 25, 489–493 http://dx.doi.org/10.1111/j.1467842X.2001.tb00309.x Royal Pingdom. (2012, September). Social network demographics in 2012 Retrieved from http://royal.pingdom.com/2012/08/21/report-social-network-demographics-in2012/. Substance Abuse and Mental Health Services Administration (SAMHSA). (2011). Results from the 2010 national survey on drug use and health: Summary of national findings (NSDUH Series H-41, HHS Publication No.11-4658). Rockville, MD: Substance Abuse and Mental Health Services Administration. Sesso, H. D., Paffenbarger, R. S., & Lee, I. M. (2000). Comparison of National Death Index and World Wide Web death searches. American Journal of Epidemiology, 152, 107– 111 http://dx.doi.org/10.1093/aje/152.2.107 Tehranifar, P., Terry, M. B., & Susser, E. (2002). #108-S tracking of a subcohort of the national collaborative perinatal project: The New York experience. Annals of Epidemiology, 12, 529–530 http://dx.doi.org/10.1016/S1047-2797(02)00396-4 Wu, L. T., Woody, G. E., Yang, C., Pan, J. J., & Blazer, D. G. (2011). Racial/ethnic variations in substance-related disorders among adolescents in the United States. Archives of General Psychiatry, 68, 1176–1185 http://dx.doi.org/10.1001/archgenpsychiatry.2011.120 Wutzke, S. E., Conigrave, K. M., Kogler, B. E., Saunders, J. B., & Hall, W. D. (2000). Longitudinal research: Methods for maximizing subject follow-up. Drug and Alcohol Review, 19, 159–163 http://dx.doi.org/10.1080/713659324 Zickuhr, K., & Smith, A. (2012). Digital differences. Washington, DC: Pew Internet & American Life Project. Zillman, P. M. (2013). White paper: Deep web research and discovery resources 2013. Washington, DC: Pew Internet & American Life Project.

Izaak L. Williams, CSAC, a former more undergraduate research fellow, is a Hawaii State Certified Substance Abuse Counselor who works in the field of treatment and prevention. He conducts research and provides group facilitation and interventions. His counseling interests include virtual reality cue reactivity, in vivo exposure and desensitization, mindfulness-based relapse prevention, critical psychology, cognitive processing and behavioral interventions. An area of general research interest is the effect of structural violence and inequality on psychological health outcomes.

Clifford R. O’Donnell, Ph.D., Emeritus Professor of Psychology, University of Hawaii, served as Founding Director of the Community and Cultural Psychology Graduate Program and as President of the Society for Community Research and Action. Professor O’Donnell is a member of the Board of Directors of the Intermountain Centers for Human Development in Arizona and the Scientific Board of The Melissa Institute for Violence Prevention and Treatment in Florida. He is a Fellow in the Society for Community Research and Action, Honorary International Fellow of the Centre for Applied Cross-Cultural Research at Victoria University in New Zealand, and has received several awards for ‘‘Outstanding Contributions.’’