Big Data and Discrimination

On May 1, the White House released a 90 day review studying the effects of big data and privacy, led by Obama's Counsel, John Podesta. The review, which can be found here, and a summary of it, here, also focuses on big data’s potential for discrimination.

Big data truly has enormous potential for social change and creative innovation. However, a key finding of the review is that big data analytics has the potential to lead to discriminatory outcomes and to evade and stymie hard-won civil rights protections in housing, employment, credit, and the consumer market.  

Fundamentally, big data creates a power imbalance between those who hold and apply the data and between those who knowingly or unknowingly supply it. The review finds that perfect personalization, which is the fusion of many different kinds of data, processed in real time, can lead to overt and covert forms of discrimination in pricing, services, and opportunities.

The review cites a study that showed that ordinary, everyday web searches involving black sounding names are more likely to display ads with the words “arrest” in them than searches with white sounding names (such as “Geoffrey”). This shows that racial biases are manifested online and shape our experiences in that arena and when we interact with big data. Furthermore, since there isn’t an obvious culprit to hold accountable, as the ads are algorithmically generated based on a variety of factors, the issue becomes more nebulous and harder to fight.

Many companies use data brokers to boost their marketing efforts. They purchase fused data profiles about a consumer (from direct, public, and spliced together sources) and target ads toward them. Data brokers use big data gathered from website activities, social media, online shopping, ad interactions, along with public or commercial records to construct profiles of individuals. They then group consumers into categories like "'Ethnic Second-City Strugglers,' 'Retiring on Empty: Singles,' 'Tough Start: Young Single Parents,' 'Credit Crunched: City Families,' and 'Rural and Barely Making It.'"

These profiles, which are sold, often times contain both factual and embellished or "modeled" information about a consumer, which is, needless to say, a huge problem that can result in factual errors and discrimination.  

Business models and big data strategies that are centered around the collection and usage of consumer data increase the risk of “encoding discrimination in automated decisions.” One disturbing example of this is the fact that big data analytics and products, score individuals beyond just credit scores, which are regulated by laws. 

These products attempt to statistically characterize everything from a consumer’s ability to pay to whether, on the basise of their social media posts, they are a ‘social influencer’ or ‘socially influenced.’’’ The scores that big data generates create a disastrous potential “to influence an individuals’ opportunities to find housing, forecast their job security, or estimate their health, outside of the protections of the Fair Credit Reporting Act or Equal Credit Opportunity Act.

There is a very real and valid concern “that such algorithmic decisions raise the specter of “redlining” in the digital economy—the potential to discriminate against the most vulnerable classes of our society under the guise of neutral algorithms.” This lack of transparency and accountability leaves individuals virtually without any means of recourse to understand or contest information collected about them, or to contest privacy invasion or discrimination.

So what is the solution? Obama’s Administration launched a series of Open Data Initiatives to unleash vast amounts of data in order to increase transparency and accountability of various governmental agencies to the public. The initiatives are in the areas of health, energy, climate, education, public safety, finance, and global development.  President Obama also signed Executive Order 13642, on May 9, 2013, which established a new principle in federal treatment of data: Agencies must now consider openness as the new standard for government information, while also taking into consideration ensuring privacy and confidentiality.

The working group that conducted the review also set forth 6 policy recommendations:

1. Advance the Consumer Privacy Bill of Rights because consumers deserve clear, understandable, reasonable standards for how their personal information is used in the big data era.
2. Pass National Data Breach Legislation that provides for a single national data breach standard, along the lines of the Administration's 2011 Cybersecurity legislative proposal.
3. Extend Privacy Protections to non-U.S. Persons because privacy is a worldwide value that should be reflected in how the federal government handles personally identifiable information from non-U.S. citizens.
4. Ensure Data Collected on Students in School is used for Educational Purposes to drive better learning outcomes while protecting students against their data being shared or used inappropriately.
5. Expand Technical Expertise to Stop Discrimination because the federal government should build the technical expertise to be able to identify practices and outcomes facilitated by big data analytics that have a discriminatory impact on protected classes.
6. Amend the Electronic Communications Privacy Act to ensure the standard of protection for online, digital content is consistent with that afforded in the physical world—including by removing archaic distinctions between email left unread or over a certain age.

President Obama, and all involved should be applauded for taking the initiative to study the effects of big data and taking steps towards creating a governmental culture that values transparency and accountability. Nevertheless, the review does not go far enough in providing solutions to the myriad ways big data can and does lead to discrimination.

The American public deserves structural protections in place, to ensure that the right to privacy and to be free from discrimination are not denied or infringed upon, when we go online and due to the rise of technology and big data. Technology and big data have countless positive benefits to society. Yet, it is imperative we ensure that regulations are in place to prevent big data from eroding our moral values and legal protections, and creating even greater burdens for communities that have been historically discriminated against and continue to be disadvantaged.