The Forum Network is a space for experts and thought leaders—from around the world and all parts of society— to discuss and develop solutions now and for the future. It aims to foster the fruitful exchange of expertise and perspectives across fields, and opinions expressed do not necessarily represent the views of the OECD.

BigTech is in the business of collecting data. The attention economy business model is well known by now: Get users to pay with attention given the bilateral exchange of information. Attention allocation generates engagement, which results in traffic on platforms; online traffic generates data; data is harvested, analyzed, and sold in targeted advertising packages to the advertisers (Figure 1). That’s the business model of many a tech platform in the information age from Meta to TikTok, from Instagram to YouTube.

Figure 1. The attention economy business model

Prediction is Profit

It gets even better than that. Data is exceptionally valuable when pooled and aggregated into Big Data, as it enables predictions of the future. Data provides patterns of past behavior that may show probable future behavior. It makes possible data-informed calculations of risk, future sales, gains and expenses, effects of marketing and communication strategies as well as on how to optimize communication, marketing and design. If you are able to calculate probable future behavior, you are able to make a profit from it:

 Will she be able to repay the loan in the future – and will she? Will he show up for work and contribute to productivity or is he just a “high cost” employee? Is she disposed for a disease, so the insurance company in case a policy is made has to cover medical expenses exceeding the income from premiums?  Which commercials will succeed persuading this person to buy the product or vote for the candidate? How many more users will push the button and provide valuable attention, if it is red? Which products and services will he desire later today? In two minutes? Ten seconds after this specific online marketing stimulus is provided through his smartphone?

All these questions come down to the same: How may future profits be generated and maximized? Data driven prediction of behavior for the sake of sales and profit is the essence of the leading surveillance capitalism business model described by Zuboff.

Profitable predictions may become chains restraining and undermining the self-determination of the citizen. In a free data market unrestrained by effective citizen protection laws, the traditional financial credit rating scores may start to look like the all-embracing social credit scores in China.

With Big Data extraction and analyzes, the market for consumer and credit information in USA has become the Wild West. The traditional credit bureaus formerly operating in regulated territory have transformed into data brokers with the will and ability to circumvent the law – at the expense of citizens’ rights. Whereas individual level information is regulated by law, the information it is possible to extract through data mining is not. With sufficient data, it’s possible to calculate information just as sensitive as individual level information. An event title in 2011 by two of former American credit bureaus turned data brokers, FICO and Equifax, is telling: “Enhancing Your Marketing Effectiveness and Decisions With Non-Regulated Data”. In the new un-regulated domain of data driven credit scores, the approach is that “all data is credit data”. With such an anything goes-approach, profile data and online social footprints, the device you use and how quickly you scroll through the sites are nowadays factors that may feed into your credit score. Facebook has taken it further and has patented a method to calculate credit scores based on one’s social network, so the average credit score of one’s friends is decisive for one’s own. In a capitalist country as USA, access to credit is a make-or-break for everyone but those in the top 1% of the wealth and income scale. To buy a house, you need a mortgage; you need a loan for buying a car and getting a college degree for yourself or your children depend on access to credit and student loans. Employers also consider credit rating scores when hiring and so do property owners evaluating potential tenants. With no effective legal restrictions on gathering and usage of data, the door is open for insurance companies gathering data on health and habits undermining the whole idea of pooling risk in insurance products and making it impossible to get insured for the ones most in need. Your credit rating score is decisive for the opportunities you have and the restrictions you face. If there is no limit to what goes into one’s credit score and how it is used by creditors and banks, landlords and car rental companies, employers and insurance companies, the difference between financial credit rating and social credit rating in China is diminishing.

Prediction is Power

If you are able to predict the future, you may also be able to influence and change it – and make a buck doing so. Predict to change is the core of targeted marketing. Marketing success is success in changing people’s behavior in a commercially or politically profitable way for the client. To be able to predict behavior makes it possible to change and modify it by providing the right stimuli at the right moment. If you can predict what people want and when they want it, it is possible to precision nudge and steer them to buy it from you. The more you know, the better you are able to predict and the better you predict, the more successfully you may influence and control. Already demographic profiling grants this kind of power. Predictions made from data such as home address, gender, ethnicity, employment, income, consumption patterns, political affiliations and social network of family and friends, makes it possible to target and tailor ads to hit the pain points where it hurts. An effective predatory method for influencing people’s behavior. However, when the profiling moves under the skin and become psychological profiles of people’s mental make-up and emotional life, it gets even more powerful and potentially oppressive.

Before the scandal, Cambridge Analytica also boasted that they are taking things a step further with profiling, integrating methods and results from scientific psychological research to create psychological profiles of users / consumers / voters / citizens. If you can categorize people according to their personality-type and mental make-up, the targeted marketing bombardment may be conducted with even more precision and effect. Psychological profiling opens up chilling possibilities of affective management and emotional control. By employing, for instance, fear-mongering messages for someone profiled as a fearful personality type, it is possible to hit the target’s pain points where it really hurts.

Even if Cambridge Analytica was the one scandalized company that got caught, it is not alone with ambitions of affective influence and emotional control made possible by psychological profiling. It is highly potent, according to Alexander Polonsky, from the French data broker company Bloom:

 ”You can do things that you would not have dreamt of before. It goes beyond sharing information. It’s sharing the thinking and the feeling behind this information, and that’s extremely powerful.”

Powerful for whom, one might ask. Not the user, costumer or citizen who is psychologically profiled to be effectively influenced and manipulated. The dream of a data broker may turn to a nightmare for the citizens. The scare of The Hidden Persuaders steering us secretly through subliminal influences in the sixties may turn out to be fully justified in the age of Big Data and psychographics. Psychological profiling takes the knowledge that is power to the next level. Most of our life and behavior is governed by fast, automatic, involuntary and unconscious mental processes evading our attention and awareness. Humans are affective beings, rather than rational agents, and more controlled by emotions than we are in control of them. If you are able to influence those processes and associations, affects and emotions going on in the dark basement of our psyche, you can more or less control us. If one is taking advantage of people’s deepest fear, makes them angry or otherwise plays emotionally on fundamental personality traits, we may not even be aware of, it may undermine individual self-determination, rational agency and autonomy. If companies – or the state for that matter – know us better than we know ourselves, an “emotional dictatorship” governing us without our knowledge or consent is right down the road. As conceived by the South-Korean born German philosopher and writer Byung-Chul Han:

““Big Data” enables prediction of human responses and the future, therefore, can be manipulated accordingly. Big Data has the ability to turn people into puppets. Big Data generates knowledge that enables ruling power. And it is Big Data that makes it possible to access and manipulate the human psyche without the affected person being aware of it. Big Data essentially spells the end of free will.”

If this extreme situation of a data driven total elimination of free will, individual sovereignty and autonomy is ever fully reached, it would be the opposite of emancipation. Total predictability makes for total control. Freedom from uncertainty is not freedom at all. On the contrary, it is the stuff totalitarianism is made of. 

Corporate totalitarianism

Corporate totalitarianism may be defined as the total interpenetration of the social and the profitable, and the identification of market and society. Everything is potentially profitable. There is no value outside the market – no value but market value. If all aspects of our life are marketized and commodified as raw material for generating profits, the market and the commercial domain of commodities has become all-embracing with nothing existing or having value outside. Quite contrary to libertarians as Rand and Ron Paul’s identification of less market regulation with increased individual freedom and self-determination,  this sort of market fundamentalism actualized in the age of Big Data may yield totalitarian results. From a citizen perspective, a multiplicity of different actors competing internally in an unregulated market of data and information may result in an information regime not radically different from the centralistic Chinese system. If everything one does, is seen, registered, evaluated and get rewarded or sanctioned accordingly, the result is a perverted proxy of an omniscient God judging and making sure everybody reap as they saw: The Big Other may be at least as powerful and oppressive as Big Brother: 

“[The Big Other] is a ubiquitous networked institutional regime that records, modifies, and commodifies everyday experience from toasters to bodies, communication to thought, all with a view to establishing new pathways to monetization and profit. Big Other is the sovereign power of a near future that annihilates the freedom achieved by the rule of law.” (Zuboff, 2015: 81).

Technological progress is not necessarily progress for humanity – it may be the opposite. “Free stuff” online may just turn out to be extremely expensive, especially if one is not aware of the full price. It may cost us our democracy, our self-determination and in the end our freedom.

For a genuinely free further read, download Reality Lost: Markets of Attention, Misinformation and Manipulation by Vincent F. Hendricks & Mads Vestergaard, Springer Nature, 2019, open access, here.