Chinese citizens must tread warily in their daily interactions. Under the country's fast-developing and Orwellian social credit system, something as ephemeral as an altercation with a restaurateur, after an undercooked meal, could saddle the complainant with an "awkward customer" rating online. Booking a table might then become as difficult as borrowing money with a poor credit history.

The reams of data now harvested in China are giving authorities more sophisticated ways to police and control its vast population. But there are parallels in the West. Anyone who has visited the Walt Disney World Resort near Orlando may be familiar with the MagicBand, a FitBit-like bracelet that tracks movement and buying behavior and then rates customers.

"They generate information from you and from all that insight are able to operationally schedule and manage the park but also tailor the experience you have," says Gillian Tomlinson, the director of data and analytics for UK mobile operator Three and a confessed admirer of the Disney scheme. "They can pick up preferences in terms of where you like to shop and what you like to buy and offer you discounts."

Figure 1: Where Data Is Mined Strap on the MagicBand and join the machine.

Disney's data capabilities are becoming even more advanced. Using cameras and neural networks in cinemas, it is starting to measure customer reactions to movies and use that information to guide future content creation, says Tomlinson. Like China, Disney is using big data to monitor individuals and as a form of control, even if the ulterior motive is to boost profits rather than quell subversive behavior.

As data-rich companies, telcos like Three are desperate to join the party. Already, Three is running predictive models to figure out if and when a customer might churn to a rival. In the future, thanks to investment in more advanced analytics and machine-learning tools, it may be able to intervene moments before a customer decides to leave, offering a new deal he cannot refuse. Analytics, says Tomlinson, could also spur the development of new services, much as Disney is relying on customer data when it makes movies.

Whether it is governments or privately owned companies that mine data, the typical concern is that data miners will start to hack away without seeking permission. Chinese citizens, unfortunately, may be unable to avoid inclusion in a state-sanctioned social credit system. Firms like Disney and Three should, in theory, be more respectful due to privacy regulations in Western democracies. "We have to maintain ethics and do everything where customers give us consent," says Tomlinson.

The data scandals surrounding Facebook show abuses do happen. One issue is whether consumers care. Facebook, which has previously had senior executives on the Disney board, did not see a mass customer walkout after last year's Cambridge Analytica scandal, despite market anxiety. Filings with the US Securities and Exchange Commission reveal at worst a dip in the number of daily active users in Europe in mid-2018, from an average of 282 million in March to about 278 million in September. But the figure was back up to 282 million by December. Facebook is more concerned about tough government oversight, and whether youngsters prefer alternative social media services.

You're invited to attend Light Reading’s Big 5G Event! Formerly the Big Communications Event and 5G North America, Big 5G is where telecom's brightest minds deliver the critical insight needed to piece together the 5G puzzle. We'll see you May 6-8 in Denver -- communications service providers get in free!

But even the privacy debate skirts around the subject of control. Is a big data world where decisions are taken for us, and machines work out our preferences before we do, a desirable outcome? Technology development seems to be racing ahead of the conversation about ethics. Critics who complain that artificial intelligence is hyped, and mock its doomsday portrayal in books and movies, do not help. While no serious observer expects a supreme artificial intelligence to appear overnight and decide humans are subordinate, technology's role in everyday life can sometimes look pernicious rather than beneficial.

Take the smartphone addiction that has struck globally. Scores of people roam the streets of the world's biggest cities with their eyes locked to their device screens, fingers tapping away furiously, oblivious to their physical surroundings. What a bizarre and perhaps troubling sight this would have seemed just 15 years ago. And yet it has rapidly become as normal a part of the cityscape as the traffic jam or bus queue. This widespread addiction to an Internet-enabled device means nearly everyone can already be fed a diet of movies, music, news and other content based on what analytics systems deem appropriate for that individual. Consumers do not have to seek out or think about anything. Their smartphones even tell them where to go. Just imagine how this might look with more immersive technology -- a set of connected glasses, for instance, or a contact lens featuring a microscopic chip.

But the issue receives little serious attention. People restrict their children's screen time as older generations once turned off the living room TV. Families go on "digital detox" vacations to re-establish temporary contact with the physical world. The absence of genuine resistance is a bonanza for the companies that can harness and extract value from big data, and explains why Disney, Three and others are so eager to invest.

The most cautionary take on this future came in 1909 with The Machine Stops, a short story by EM Forster that is surprisingly overlooked today. In the narrative, artificial intelligence effectively governs the planet, picking the music that people listen to, like some dictatorial Spotify, and ensuring their various other needs are met. Most humans have lost the ability for original thought. China's fledgling social credit system, and the services being developed by the likes of Disney and Three, all presage a similar transfer of control, responsibility and decision-making to algorithms. That should worry us more than it does.

Related posts:

— Iain Morris, International Editor, Light Reading