“Great products are no longer good enough to win in business. Creative marketing and delightful customer service too are not enough to succeed. Success and the future of business is experiential and this is the time to learn how to create and cultivate meaningful experiences.”
From X: The Experience When Business Meets Design
by Brian Solis
Depending on the day and our mood, shopping can be a necessity, a social event, a research project or an outlet for our emotions. Sometimes, we even call it “retail therapy”. On some days, we attack it head-on with a list and a single-minded focus. On other days, we might join friends at the mall and shop with complete abandon, arriving home with goods we neither needed nor wanted. Then there are days we plan for extensively, seeking recommendations, reading reviews, researching online and comparing prices. The research might lead to an immediate online purchase or it might lead us to a store for our final part of the customer journey.
While the “why” of shopping has remained constant and survived as a central part of our lives, the “how” of shopping has been rapidly evolving, driven largely by technology change. Despite the birth of Amazon, Alibaba and others, the shopping experience is about to undergo its most radical transformation yet.
Technology is already augmenting our shopping experience. You can take a virtual tour of a house you are thinking of buying. You can hang a virtual representation of that new TV you’re interested in in a virtual version of your living room created from an uploaded photo, you can take a virtual test drive in a new car or you can snap a selfie and use an app to “try on” some sunglasses. What we look at in the virtual world will soon be further enhanced by new sensory experiences spanning sound, smell and touch. Augmented shopping experiences will not only change how we buy, it will change what we buy as new experiences, products and services evolve to better meet our needs. This evolution won’t stop at shopping either; it will span travel, tourism and more.
At the heart of the movement to augment how we shop and travel is a desire to address our moments of desire, dissatisfaction and doubt (MODs). These are the moments—sometimes measured in seconds—where we experience emotions in relation to a product, event or service.
• Desire must be fulfilled immediately, often in milliseconds. Left unsatisfied, we simply seek out other services and offerings.
• Dissatisfaction drives us to seek alternatives but also opens our minds to new products and services.
• Doubt is answered with information and data, allaying concerns or anxiety and providing reassurance as to the quality of the potential services.
For instance, think about the moments Uber has created as part of its experience. We desired taxis to be available with the same convenience as making a telephone call or checking an email. We were universally dissatisfied with taxis that didn’t arrive or those that delivered poor service with impunity. The mere doubt of not knowing where our taxi was could create anxiety and stress, especially when we were in a hurry and the taxi hadn’t shown up.
Uber managed to solve all three of these moments. When we open the app, immediately we know that there are cars available. We order with a click on the “Find an Uber” button and we can observe the journey of the taxi as it makes its way to us—the stress and doubt disappear with the certainty the app delivers. Most importantly, we are able to rate the quality of the experience, affording Uber the opportunity to improve the overall service and weed out substandard drivers or vehicles.
A recent Quartz article1 identified that up to 30 per cent of Uber drivers in the United States have never had a bank account—many operated previously as taxi drivers in the cash economy. To be a driver on Uber, however, drivers need a minimum of a debit card to get paid. So Uber has had to solve this problem by allowing drivers to sign up for a bank account as part of the Uber driver application process, in real time. Unsurprisingly, this makes Uber the largest acquirer of small business bank accounts in the United States today, bigger than Wells Fargo, BofA and Chase combined.
You’ve probably never thought of Uber as an acquirer of small business bank accounts, but if you’re an Uber driver and Uber can give you a debit card that enables you to get paid—then why would you go to a bank branch to open an account instead? It also means that as an entrepreneur bank account the next obvious move is to design day-to-day banking into Uber’s app instead of standing alone as a typical bank account or mobile banking app.
For the millions of permalancers or gigging economy workers, it’s highly likely that the first time a freelancer opens a bank account will be directly in response to a new gig or job offer—if that employer (like Uber or Airbnb) offers you a bank account as part of the sign-up process, why would you stop signing up for Uber, drive to a branch and sign a piece of paper?
Uber is also offering car leases to its drivers,2 allowing drivers with no vehicle to sign up and get car financing backed by demand from Uber. This is what the new banking experience looks like for small business entrepreneurs. Uber is effectively doing all the sourcing for bank relationships, and has become an acquirer for bank accounts, leasing and insurance. An Uber driver has no reason to go to a bank branch for his needs today thanks to Uber’s commitment to experience design enabling the needs of a new driver.
Uber is a brilliant example of how technology is being used to augment our everyday journeys, and the business of journeys. It doesn’t rely on just one single technology but brings many together to transform a traditional service for its customers and drivers alike. Incumbents faced with the superior experiences offered by this new entrant often lobbied local regulators and lawmakers to outlaw Uber. When that failed, these same companies were forced to invest in similar technologies and experiences just so they could remain relevant. Most could only approximate the new experiences Uber pioneered.
At the heart of Uber is the near-term benefit of augmentation. Take an existing service and make it simple and easy. Most new technologies are focusing on this near-term innovation opportunity. Many of the examples you will see across retail and travel today reflect this. Electronic bag tags, mirrors that show you how a shirt will look on you without trying it on and bracelets that track the journey of your child on a plane or in a theme park. All harness readily available technology to make the current experience better than before.
On the near horizon are promises of far greater innovation.
Over the next five years, most innovations will seek to transform current services and address our MODs. What we see today as major shifts like Uber and Airbnb will turn out to be small incremental improvements in hindsight.
The underlying theme of an augmented world is that technologies connect to create fundamentally new experiences that aren’t possible in a manual or human process driver world. An example of what might be possible already today with a mash-up of a range of technologies is shopping for glasses. We snap a photo of ourselves on an iPhone, then select a pair of glasses from an online catalogue to see how they might look on our face (rendered via an app). Selecting the glasses we like, we then get a map of retailers with current stock of the frames we like. The closest retailer then serves an offer via text messaging to our phone but will take three days to deliver. A retailer in the same area can deliver in 24 hours but doesn’t provide us with a discount offer to purchase. The decision is ours. This scenario is highly feasible due to the technology we already carry in our pockets, the capability of the computing cloud that is integrated into the world around us and specific application of data like geolocation or GPS and mapping technologies.
Already drones are taking to the air to test personal delivery of everything from books and smartphones to pizzas. Drones may initially be required to make deliveries to centralised locations like post offices, petrol/service and train stations for safety purposes. Five to ten years from now, those same drones and/or autonomous carts and pods (small robot vehicles) will come to wherever you and your smartphone are. Lily, a self-driving robot videographer drone that automatically follows you around capturing your life on video, pulled in a whopping US$34 million in pre-orders for more than 60,000 units when it went on sale over Christmas in 2015. We can be sure that drones will play a part in experiences in many ways.
Technology will reshape products and industries dramatically the further we get away from the 20th-century norms. As Brett discussed in his earlier chapters, when computers are everywhere and our environment is smart, always connected, the limitations of existing value chains, product life cycles and campaign marketing will be vaporised in favor of engagement that is real time and moulds itself to our needs, location and preferences. The businesses of the future will be in the business of experiences, not products and services.
Retailers that no longer sell goods but deliver experiences in which products feature will be a distinct element of the Augmented Age. Whether that is a gourmet food experience, a luxury lifestyle experience or an immersive entertainment experience, goods and services will be reframed as an experience you live. Goods or products may be embedded in the service, but individually branded products themselves will likely become less of a factor in a buying decision and more a “feature” of the experience itself. Will you buy an iPhone 11s or will you buy a personal AI and healthcare system supported by a smartwatch and what we’d call a smartphone today? Would you buy a VR headset or would you buy a subscription to an immersive storytelling and gaming platform that needs a VR headset to provide you with that experience?
Travel is going to be another one of these areas where the entire experience has the potential to be impacted by technology, from what you see out of your window through to the food you eat and media you consume will be a dynamic reflection of your personal preferences learnt through past interactions and behaviour. Some of these preferences you’ll have told your personal AI (acting as a travel concierge) while others will be inferred from your data, behaviour or choices historically.
What will make a first-class experience for you? Will it be flying up front?3 Will it be a shower on board before your long-haul flight lands? Will it be an à la carte dining service, high bandwidth data connectivity or a personal greeter who can rapidly get you through immigration and home faster once you land? In the past, marketers might have been tempted to call this a market-of-one segmentation, but this is more about behaviour-based adaptation of the market and service providers to the individual in real time through experience and interaction design. It’s about technology that can learn about you, and services that can intelligently adapt to that knowledge.
In this 15- to 20-year timescale, our lives will be augmented by products and services that are designed as and when we need them. Let’s say you are planning that special night out and you’d like to give your significant other some jewellery to mark the occasion. We might no longer shop for a piece of jewellery but rather look simply for a cool design to fit the occasion. That design is downloaded and then printed on a 3D printer, but not before including some last-minute customisation that you’ve built in to personalise the piece. The Voxel8 is a 3D printer, retailing at US$9,000, that can print complex circuits using conductive ink, which hints at a future where we might even be able to print our own electronics at home.
A team of computer scientists from Germany’s Saarland University has developed a technique that could allow anyone to literally print their own custom displays, including touchscreens. Using a regular inkjet printer equipped with special ink, a DIY thin-film electroluminescence (TFEL) display can be printed out from a digital template.
The team has developed two methods using either screen-printing techniques or off-the-shelf inkjet printers that can take anywhere from several minutes to four hours for a layman to create a custom display. The team claims that this results in “relatively high resolution displays” only one tenth of a millimetre thick. They say covering a standard A4 or letter-sized printer page with a display layer would cost about €20 (US$21.69), mostly due to the cost of the special ink involved.
Within ten years, it appears that we will be able to deliver circuitry and electronics into these downloadable designs and, by 2030 to 2040, your next iPhone might, in fact, be a file you download from Apple and not a physical device you buy from a store. The only physical product we’ll absolutely have to buy in the future might be a 3D printer or fabrication unit.
To understand how we’re going to get to the future, we have to understand how we got here. The dot-com boom of the 1990s gave birth to a new generation of retailers. E-tailers, as they became known, had no physical presence save a few warehouses from where their products were shipped. Armed with the global reach of the Internet, search engines and teams of programmers, they took traditional retailers head-on. Amazon unseated booksellers and quickly became the “everything store”. Gilt, Rue La La, Alibaba and Zappos made the annual sale a daily event. Initially sceptical or even downright hostile, most incumbent retailers were dragged along kicking and screaming as these online players took market share. Some, like Blockbuster and Borders, just didn’t make it.
This first phase of augmentation was basic but necessary. We could quickly experience products before heading to the store, maybe shopping online to compare goods or prices. We could choose pick up or delivery, a recipe quickly morphed into a shopping list with the click of a mouse and we could find out if the item we wanted was in stock and/or reserve it before we got to the store. The physical world was already being augmented by an online world that enabled us to time and place-shift our desires. We no longer had to wait to visit the store to see what we wanted. Instead we browsed, researched and ordered online.
Hampered with legacy cost structures and physical debt, it took decades for traditional retailers to make headway against new entrants. As Clay Shirky put it, “‘You’re gonna miss us when we’re gone!’ has never been much of a business model.”4
Today, physical retailers compete toe to toe with online competitors, leveraging omnichannel marketing and merchandising and complimentary experiences to unify the digital and physical world. Recently, we have seen pure online stores opening physical storefronts such as Amazon and Microsoft.
The year 2015 marked another e-commerce boom after an already heady 2014, with US-based e-commerce reaching more than US$350 billion in sales. Global sales have expected to top out in excess of US$1.5 trillion. Online e-commerce over the Christmas holiday period showed an increase year-on-year of some 14 per cent,5 with US$617 billion in sales globally, and US$80 billion in the United States alone (approximately 10 per cent of all retail sales).
As Brett identified in chapter 2, the four most disruptive forces of the Augmented Age are artificial intelligence, distributed embedded experiences, smart infrastructure and gene editing and HealthTech. However, within those disruptive themes, your shopping experience is also being transformed. There are four building blocks to Augmented Retail, around which thousands of other technologies and augmented experiences are starting to cluster.
“We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten.”
Bill Gates, 1996
The cloud is perhaps the worst metaphor for computing power ever created. The name harks back to early network diagrams in which technology not on the physical site was represented with a diagram of a cloud. Today, the cloud is literally hundreds of hectares of computer and storage power operating as a massive, distributed system in service of everything from Facebook to email, photos and mapping software.
Brett discussed in chapter 3 that computers are becoming ubiquitous and essentially disappearing (by being embedded or diffused into the world around us). Today, entrepreneurs and individuals have access to computing power that was previously only available to the largest companies in the world. Any start-up with a credit or debit card can buy computing power for a few dollars to get started, and then, as the company grows, scale its spend and cloud-based platform as required.
Today, the cloud provides access to unprecedented computing power and storage, enabling any retailer to scale services. So just how big is the cloud? Unfortunately, cloud providers don’t disclose too many details about their operations. However, we know that Amazon is the biggest since it recently reported US$7 billion in annual revenue for 2014.
First, let’s look at the scale of Amazon’s cloud operations. It’s truly mind-blowing.
• Amazon Web Services (AWS) is carved up into 11 regions globally where it has datacentres.
• Each region has at least two availability zones (AZs), for a total of 28 availability zones.
• Most AZs have multiple datacentres. Amazon hasn’t revealed how many datacentres it actually has but a recent EnterpriseTech article estimates that there are around 87 AWS datacentres.
• Amazon has 50,000 to 80,000 servers in each datacentre, for a total of somewhere between 2 and 5 million servers.6
• Power requirements for each datacentre are 25 to 30 megawatts, or 2+ gigawatts for all of AWS.
• Amazon is building solar farms to make its datacentres 100 per cent renewable as well as working with Tesla to apply battery storage technology to address its power needs.
• As for network capacity, each datacentre has at least 102 Tb/sec of bandwidth coming in, connected to each other through private lines. That is 100,000 times as much bandwidth as Google Fiber. In fact, Amazon has had to develop its own networking technology just to continue its growth.
We know that both Google and Amazon have over 1.5 million servers each. Facebook has at least half a million servers for its Facebook, WhatsApp and Instagram infrastructure. Each of these installations has more storage and computing power than the entire planet had in the mid-90s. If you add IBM, Oracle and then companies like Rackspace, as well as private clouds owned by banks, governments7 and so on, we’re talking tens of millions of servers spread around the globe, zettabytes of storage space and hundreds of gigawatts of energy consumption. The size of the “cloud” is truly staggering. Maybe we should start calling it the “nebula” —cloud seems just too small.
We once worried about where our data was stored and bought physical floppy discs to do back-ups. Now our data lives in many places and flows to whatever device or application we link it to in real time. In many ways, we’re still getting started on this journey. Between 2011 and 2013, the Internet population grew 14.3 per cent to 2.4 billion people. With 80 per cent of the world likely having a web-enabled smartphone by 2020, we can expect to add another 2 billion mobile consumers to this number within the next five years. Today, no device sends and receives more personal data than that of our smartphone.
The mobile device—smartphone, iPhone, whatever your flavour—has created a new connection point with the shopper and traveller. Part tracker, wallet, camera and interface to the physical world, it sits in our pocket, always on, always listening, dynamic in its ability to interact.
According to Deloitte research, smartphones used as part of a shopping experience could impact 17 to 21 per cent of retail sales by 2016—that’s an incredible US$627–$752 billion. During the 2015 Christmas period in the United Kingdom, more than two thirds of shoppers were expected to use their mobile device to shop. Econsultancy’s “Mobile Experience Trends Briefing” report in May 2015 also found that 92 per cent of shoppers utilise mobile to make buying decisions while completing their purchase in-store. Furthermore, eMarketer estimated an increase of 35 per cent increase in mobile commerce for the 2015 Christmas shopping period.
The data is clear. Today, there is no greater influence on our shopping behaviour than the mobile phone, and its influence is growing. To be fair, growth is happening across the board in retail and in commerce, but no channel or mechanism is growing anywhere as fast as mobile is. Mobile is growing at least three to five times faster than that of both online and conventional retail commerce and sales, and faster than online grew even at the height of dot-com.
The travel industry is being greatly impacted by mobile. Nowadays, 60 per cent of all travel is booked through digital, up from 0 per cent in the mid-90s. In 2007, that accounted for about US$94 billion in online sales but that will be closer to US$300 billion by 2020. In 2016, eMarketer estimates that 51.8 per cent of travel-related digital sales will be mobile or app-based. So the biggest single channel for travel booking will be a mobile phone. Obviously as apps get better, as hotels, airlines and travel agents streamline their mobile-based booking engines, this is likely to increase significantly. Of course, this doesn’t even factor in Airbnb and the sharing economy.
Let’s face it—the retail and travel industries may be growing, but the sweet spot is mobile engagement and revenue. Essential to our retail and travel experiences will be engagement and payments. Engagement is the new “marketing” when it comes to the retail space—how can we get you into our store (physical or virtual), and buying our product, then how do we make it as easy as possible to buy? The most efficient form of payment is one that isn’t at a checkout but happens when you interact in the store, or even as you walk out of the store having picked out your goods.
In the future, your unique identity will be linked to preferential payment mechanisms coded into your smart device. As you have your new shirt, automated personal drone or new foldable tablet scanned by an in-store representative for purchase, the in-store systems will have already negotiated with your phone and be ready to process the payment. As you walk out of the store, you’ll see the receipt and transaction record (and warranty record) instantly on your device.
Figure 12.2: Microsoft and Volvo are collaborating on AR “manuals” that show you how car features work. (Credit: Volvo/Microsoft)
Augmented reality will be used for a variety of applications across the entire buying cycle from research through to purchasing and service interactions. Volvo cars and Microsoft HoloLens recently demonstrated a conceptual example of new service interactions with AR.
The rendered concept video shows how shoppers will be able to check out useful features that are often overlooked because drivers tend to no longer read car manuals. Detailed visualisations also help them understand how these features work in various scenarios. This will change support for technical products considerably. Imagine a help desk or customer support showing you a feature while you’re wearing smart glasses.
If you use a pair of smart glasses, you might see options or deals in your personal head-up display. Augmented reality will be a stronger in-store trend looking at the middle of next decade. However, to accomplish that, we will need a new technology infrastructure underpinning store experiences.
For smartphones, smart glasses, a personal AI or smart wearables to respond in the retail space, something must be listening.
Today, sensors and Bluetooth connected beacons surround us. iBeacons, NFC chips, radio frequency identification tags (RFID) and more sense our presence and communicate with our devices in real time. Most of the time, we’re unaware that this is happening. Connecting to millions of phones in millions of pockets, with their connected location-based services, they signal the opportunity to deliver extremely tightly focused, personalised messages, offers and promotions in real time.
Beacons are the fastest-growing retail technology, growing 287 per cent to 5 million beacons8 in the United States within the next four years, with most of them in use by retailers, according to BI Intelligence. By 2016, 85 per cent of the top 100 US retailers will have deployed beacons. Apple is deploying beacons in its own stores and in association with Disney for payments and in-store offers. Macy’s recently rolled out beacons to 4,000 stores.9 According to Business Insider, beacons directly influenced over US$4 billion in US retail sales in 2015 and will climb to more than ten times that in 2016.
Using a combination of virtual currency, apps and location-based services, the technology used by Macy’s (and provided by Shopkick) has the potential to significantly change the way we shop.
“Shopkick has found a way to change consumer behaviour… We can influence behaviour in a measurable way. Shopkick users spend 50 to 100 per cent more than others.”
Cyriac Roeding, CEO of Shopkick
By augmenting your shopping experience with contextual, real-time rewards, Shopkick and Macy’s can reward you for just visiting certain parts of the store. By scanning a barcode on a product, you get even more rewards. You not only get your shopping trip augmented with new experiences, you get an entirely customised experience.
“If it’s Chuck’s Macy’s, you experience a different Macy’s store,” said Roeding. “The whole experience becomes YOUR Macy’s. Just like different stores carry different products. This takes ‘My Macy’s’ to the next level. The store becomes an individual experience.”
Cyriac Roeding, CEO of Shopkick
To shoppers, the reality is that technology matters little. What matters is value delivered.
An interesting side effect of this technology will be a complete undermining of credit card reward programmes. As in-store offers emerge, rewards becomes easier and easier to redeem. As a direct result of this, there will be a decrease in the net number of customers who no longer claim rewards and, therefore, marginal profitability of current reward programmes will be hammered—quickly becoming unprofitable. If you get cashback today from your card, expect to have this disappear within the next five years. It’s unsustainable in a real-time world with decreasing interchange models.
Gen Ys are much more attuned to the notional opportunity cost of rewards built into credit card schemes in particular, where they are not convinced of the trade-off between higher interest rates, frequency of spend and the tangible value of the reward. Even for cashback offers, it doesn’t require much of a revolving balance over a couple of months for a consumer to wipe out any cashback benefits. In the era of the quantified self, the self-aware customer won’t make spending decisions based on cashback, miles or trinkets offered, they’ll make spending decisions based on whether they can afford to make a purchase, or whether this is a great deal in-store. Loyal customers will get great deals; that will be the new frequency model. Not miles or points you never spend.
Today, shoppers are surrounded by printed posters, displays and billboards illustrating these new retail experiences. We see smartphone ads encouraging the downloading of apps and connections to in-store WiFi networks. These posters and billboards are not about apps per se, but linking customers to coupons, discounts and offers in-store.
A great example is the Regent Street app promoted on double-decker buses across London.10 Download the app and you will find yourself “beaconed” into any one of the hundreds of luxury stores and brands along Regent Street with messages correlated with your predefined interests. Westfield London shopping centre is working on similar technology.
Today, what happens on our handheld screen is fairly one-dimensional. The next level of retail augmentation is when auditory, visual and olfactory experiences are added to our reality.
As Brett articulated in chapter 3, the current generation will be the last to use a physical keyboard and mouse. Soon, our so-called “phones”11 will not only take instructions from us but also sense the tone and tenor of our voice, listening for choices, instructions or preferences, and making recommendations of experiences we might enjoy.
While the humble camera and speaker are how we most commonly experience visual and audio innovations today, what lies beneath them has far more revolutionary potential.
Next-generation retail experiences will evolve into something akin to neural and bio-synchronicity. The use of biometrics (fingerprint, facial, iris and voice identification), pattern recognition (emotional, stimulus response, location-based), behavioural psychology, sensory integration and augmented reality will change the retail experience into something that is a hybrid of technology and experiential design. We’ve talked about smart health care, smart banking, etc., in the other chapters of the book, so it’s natural that retailers and hospitality will seek to merge retail, travel and tech as much as possible.
The future of big retail will be undeniably smart and augmented, but just like in other industries, smaller retailers (read “mum and pop stores”) will struggle to keep up with the changing conventions of experience and interactions.
The CEO of Whole Foods, Walter Robb, calls the future of their stores a “richer buying experience”.
“Imagine if you are picking up a head of cauliflower—and this platform allows you to actually have a video from the supplier saying ‘here this is how this cauliflower was grown.’ This is the ultimate endgame of matching up this platform with our product attributes [and] our product quality.”
Walter Robb, CEO of Whole Foods
The 2.4 million tickets sold for the 2015 Rugby World Cup featured augmented reality content to bring the experience alive for fans, both in the run-up to the competition and on the day of the games. Using an augmented reality app and a mobile phone or tablet, tickets could be scanned to reveal exclusive behind-the-scenes material hosted and delivered by Rugby World Cup 2003 winners Jonny Wilkinson, Lawrence Dallaglio and Will Greenwood.
Managing Director of England Rugby 2015 Stephen Brown hailed it as a sporting first: “using augmented reality technology as part of the ticket design enables fans to engage with the tournament through interactive content, which is a really exciting piece of activation.”12
Of course, printed tickets are not long for this world as we’re already seeing with airline boarding passes, bus tickets and so forth.
One of the world’s leading high-street retailers, John Lewis Partnership (JLP), is now trialling AR tech in its flagship stores. JLP has created a virtual showroom in-store ahead of roll-outs in its Cambridge store. Basically, any physical space can be turned into a virtual showroom today. Rather than physical models, computer-generated models such as furniture and appliances can be added to the real in-store environment. Imagine technology like Magic Leap or HoloLens being made small enough to fit into smart contact lenses over the next two decades, and the possibilities for the in-store experience are almost limitless.
Matt Hully, head of brand innovation at JLP, said, “Technology will play a key role in the new Home Department at John Lewis Oxford Street, helping customers visualise new ways to personalise and design their home.”
“Central to this is our new partnership with Imagine. It allows customers to visualise products in life like 3D, rather than just on the pages of a catalogue; helping them to virtually ‘try’ an item before they buy it. We hope it will give our customers a taste of the future of shopping and enhance their overall experience. This is just the first step in using a very exciting technology.”
Matt Hully, head of brand innovation at JLP13
The use of technology like biometrics, pattern recognition and psychology is a win-win for retailers and consumers. Not only will these technologies dramatically improve our relationship with brands and stores, they will also revolutionise security, especially in the area of payments.
Over the next five to ten years, nearly every transaction will be verified with some sort of biometric scan (facial recognition and fingerprint being the most common methods) combined with other data like heuristics and geolocation data. That same scanning technology that underpins payments security will be used to tailor offers and promotions and guide you through a retail experience. Looking ten to twenty years down the road, tourist attractions and in-store merchandising will take cues from your current “state”, or your profile, and reshape the environment to appeal.
If you use your smartphone to pay today, we can use fingerprint for confirmation (this is very difficult to replicate), we can verify that your phone is in the same geolocation as the POS terminal where your phone is being presented (or tapped), we can use heuristics or behavioural data to verify that this is a typical transaction for you and we can tie the payment app to a specific mobile device (using Device ID). In mobile payments, we also use tokens. This is when the merchant receives a one-time code for a particular transaction so that even if the POS terminal is compromised, a thief can’t reuse your card number for other transactions.
What this means in practical terms is that a phone-based payment in a store is 500 to 600 per cent safer than using your plastic card,14 and this is just with current tech. In the future, we’ll be able to use even your heart rate to uniquely identify you.
Biometrics Research Group predicted the global biometrics market to soar to US$15 billion by the end of 2015,15 up from an estimated US$7 billion just three years ago. Technology consulting firm Frost & Sullivan forecasts that nearly half a billion people will be using a smartphone equipped with biometrics by 2017.
Figure 12.4: Smart changing rooms and mirrors allow a crossover between physical and digital. (Credit: Nordstroms-ebay fitting room)
Retailers and brands like Ugg Australia, Uniqlo and Burberry are already using “magic” or “memory” mirror technologies that utilise individual profiles. RFID tags identify the product while cameras capture your image and body shape, enabling you to try on virtual outfits in different colours and styles. Once you’ve had a shopping experience in-store, that profile can then be used at home for you to make sure online goods fit you properly and see how they’ll look in a virtual fitting room experience. More sophisticated in-store displays can even change the context whereby you can try on an overcoat in the rain, a swimsuit in the Caribbean or sports gear on the track.
Bloomingdales has augmented fitting rooms with iPads to allow customers to ask for help, read reviews and see what sizes are in stock. C&A, a retailer in Brazil, has gone one step further, showing the number of Facebook Likes a particular product has, displaying this on the coat hanger in-store.
The smart dressing room is here to stay. In the future, alternatives will simply appear on the mirror in front of you if your facial expression shows uncertainty with the choice you’ve made. Further out, the dressing room vanishes. Wherever we are, merchandise appears on mirrors and screens as if we were wearing it.
Figure 12.5: C&A’s digital coat hanger shows the number of “Likes” for a product in-store, in real time. (Credit: C&A)
Online tailor InStitchu has teamed up with mPort to fuse the online and offline purchasing experience with 3D body scanning technology. 3D body scanners, similar to those used by the Transportation Security Administration (TSA) at US airports, have begun to be rolled out to shopping centres in Asia, enabling people to buy clothes tailored to their exact measurements.
“The customer simply steps into a private enclosed area to scan themselves. Their precise body measurements are then instantly captured and stored on their InStitchu profile. Once they’ve designed a suit unique to their personal preference and style—right down to the stitch colour, buttons, materials and lapel—we send their order to a team of Saville Row trained tailors who use their expertise to create the suit to the required measurements. The suit is then delivered direct to the customer, with the whole process taking just a matter of weeks.”
Robin McGowan, co-founder of InStitchu
Figure 12.6: This Lego AR display shows a virtual representation of the constructed Lego model. (Credit: Lego)
Technologies like mPort with InStitchu address the MODs that I mentioned earlier. Imagine the following scenario. Perhaps I love the look of a suit but am disappointed that I can’t find one that fits. What I’d really like is a tailored suit but I don’t know where or how or if I can afford one—the smart dressing or changing room might be able to detect this just by watching my emotional response, and then send me an offer later online to have the clothes tailored to my individual body type.
Lego augmented reality kiosks (called the Lego Digital Box) enable you to see what the finished Lego construction will look like before you make a purchase. Similarly, the United States Postal Service has a virtual box simulator that lets customers see if a package will fit into one of their flat-rate boxes, eliminating guesswork in choosing which packaging to use.
Brett referred to Amazon Echo in chapters 3 and 8. Amazon Echo looks like any ordinary speaker but it is listening. With technologies like this, you will never need a shopping list pinned to the fridge again. With a personal AI, wish list management will become a part of your life. Think of something you need, just get your PAI to tag it to your personal wish list, perhaps noting whether this is an immediate, medium-term or long-term purchase. Like many of the new generation of technologies that will augment our lives, they don’t wait to be turned on, they don’t require you to input anything on a keyboard, they are always on, waiting to adapt, learn or respond.
Human habit will be laid bare by technology shifts. Your loyalty to a particular retailer could be challenged. Augmented retail will make stickiness to brands both easier and much more difficult. What of our most emotional responses and senses?
Research by Nobel Peace Prize winners Richard Axel and Linda Buck16 reveal that our sense of smell is perhaps the most emotional sense we have. With smells, we appear to “feel first” and analyse second. Studies have shown that we remember about 10,000 distinct odours, each of which can trigger an important memory, even taking us all the way back to our childhood.
Recently, I walked into the Crown Metropole in Melbourne, Australia, and was transfixed by a warm and inviting smell. Abercrombie & Fitch has its own line of men’s fragrances called “Fierce”. Fierce assaults the senses on entering a store to give off what A&F describes as a “lifestyle...packed with confidence and a bold, masculine attitude”. When you smell like this, you inevitably start to feel and see yourself like the models depicted in-store. Over the next five years, olfactory stimulus will become a powerful brand trigger and signifier.
Scent will augment our experience and create new brand experiences. Take Johnny Cupcakes, a retail T-shirt “bakery” that started out in Boston in 2001. When you walk into the store, your sense of smell is pleasantly assaulted by the smell of cupcakes, and when you buy a T-shirt, it’s given to you in a box that looks like it came out of a bakery. The smell in a Johnny Cupcakes store is a big part of why people say they keep coming back. On 12th March 2011, the London store opened with hundreds of dedicated fans from around the world queuing up and camping out for more than 24 hours.
What does your brand smell like?
One additional area that was showcased in the science fiction film Minority Report is directed audio. In the film, protagonist Captain John Anderton is constantly identified by iris-scanning devices that direct advertising his way. This becomes problematic as these algorithms can also track him and enable the police to hone in on his location. Therefore, he gets a transplant—new eyes. After the transplant, Anderton walks into a retail environment and is identified by iris scan as a returning shopper of Japanese descent. Thus, the audio messages directed at Anderton before he boards a subway train and when he walks into a retail environment are for him specifically.
Such directed audio advertising technology is now coming of age and could be used in in-store or public space interactions in the near future. Prototypes of this technology, such as HyperSound directed audio solutions, convert audio into a highly directional, ultrasonic beam that only you would hear.
Other stores are already using subliminal audio messaging or hiding advertising prompts in high-frequency audio playing in the background of in-store muzak and soundtracks. You won’t be able to hear this high-frequency audio, for example, but an app on your phone might hear this and use it to trigger an offer on your phone. Beacons and AR are more likely to replace this tech in the near term. Nevertheless, sound remains another area where the in-store experience can be augmented.
At the 2015 Gartner CIO Symposium, Gartner principal analyst Kelsie Marian challenged her audience to imagine what the retail store would look like without humans. Brett has explored the risk of being replaced by automation or robots in chapter 2, but while this is increasingly true of environments that we would likely rather not find ourselves in—say, a bank branch—it could be assumed that retail and travel environments are places we go to for reasons beyond the transactional.
“Let’s imagine that we in this room are the last generation of retailers to have humans working in our stores…Maybe this seems a little fantastic, but the truth is that we have a new class of smart machines that can get us closer to that reality. Probably the future is something between the store we know today full of associates and the idea of an associate-less store.”
Kelsie Marian, Gartner principal analyst
The potential for technology to not just augment the sales associate but become a sales associate in and of itself is profound. Take the OSHbot being tested by Orchard Supply Hardware store in San Jose, California. The OSHbot is multilingual, capable of giving customers detailed information about a product in five different languages and can show customers on an onscreen map display where the item is located in the store, or you can just follow it physically to the shelf where the item is located.
“We have amazing store associates, but very few speak five languages fluently, right? If there’s somebody that can speak those five languages fluently, they probably don’t know the real-time location of every single object in the store.”
Kyle Nel, OSH/Lowes Innovation Labs Director17
While workers are snug in their beds at night, the OSHbot roams the aisles. “The robot actually navigates by its own overnight,” said Marco Mascorro, CEO of Fellow Robots. “So it learns when they move, for example, a shelf.”
OSHbot has an almost unassailable advantage over human store associates because it can communicate in multiple languages and knows where every product is in real time.
Technology like this doesn’t have to replace the sales associate, but it could free them up to do work of more value. For instance, rather than being locked into a checkout counter, they could roam the store, advising, informing and guiding customers. Although robots like OSHbot are showing that humans may not be the best at that for much longer.
Scott McNealy, former CEO of Sun Microsystems, liked to say, “You have no privacy. Get over it!” Privacy is increasingly something we give up with the click of a button as we join WiFi networks, share photos and other particulars with all sorts of organisations. This crucial data drives how effectively our shopping experience will be augmented. Most people will agree that the trade-off is worth it; those who don’t will suffer from a very traditional, non-augmented world. In fact, it will be increasingly impossible to not give this data to organisations like retailers.
As you enter a shopping centre or mall, the smart displays will notice who you are and match your face to a core database of previous shoppers who have visited. This data can then be shared with retailers who can start pinging you offers based on past shopping experiences or even matching a known wish list you have on Amazon, Pinterest or products you’ve talked about on your Facebook page.
These same cameras, and the machine algorithms tied to them, will notice what you did not buy. Say that you linger in front of a shop that stocks your favourite brand of jeans but you don’t enter—the camera notices. On returning to the shopping centre a few days later, you are then served offers for those jeans directly to your phone, and as you stroll past the store, a digital display changes to reinforce the offer or says, “Hey John Anderton, you’d look great in these new True Religion bootcut jeans!” Even that mannequin in-store might be tracking your movement with cameras that use facial recognition.
Figure 12.8: Smart mannequins like the EyeSee from Italy use facial recognition to track you in-store.
Worried about all this data eroding your privacy? You shouldn’t be. Rather than removing freedom or privacy, the advance of technology is putting us more in control than ever before. Rather than being confined to opening hours, we now shop when we want, often having goods delivered when and where we want. Rather than have to grapple with a European clothing size when visiting London, our personal preferences could already be uploaded to the store as we walk in (via our phone talking to beacons, and the store talking to the cloud) and we could find that the sales assistant already has the information required to help us pick out the correct size.
Data and online collaboration also means that we are concentrating our buying power. In 2014, U.S. Cyber Monday sales were US$2.3 billion—up 29 per cent from the previous year. China’s big buyer or “Singles’ Day” (11th November) dwarfs it by comparison. In 2015, Alibaba only took two hours to reach US$2 billion in sales on Singles’ Day and reported sales had surpassed US$9 billion by the end of the day.
Stores are getting smarter. They are starting to equip themselves with the technology not just to improve the in-store experience but to also learn about us like never before. The augmented world will know more about us than we even know about ourselves.
Smart retailers will learn that loyalty doesn’t come from brand marketing, tear-jerking advertisements or airline miles. It comes from the ability to know what we need before we know it, and to personalise that in real time. Shopping in the future is all about the experience, and the experience is all about the data.
____________
1 Ian Kar, “Uber is trying to lure new drivers by offering bank accounts,” Qz.com, 3 November 2015, http://qz.com/533492/exclusive-heres-how-uber-is-planning-using-banking-to-keep-drivers-from-leaving/.
2 Pavithra Mohan, “Uber to Lease Cars Directly to UberX Drivers,” Fast Company, 30 July 2015.
3 There’s a rumour that Brett King only knows how to turn left when boarding an aircraft. Can anyone confirm that?
4 Clay Shirky is a well-respected writer, consultant and professor who has been a leading voice on the social and economic effects of the Internet. He lecturers at Harvard’s John F. Kennedy School of Government, New York University, the Interactive Telecommunications Program and the Arthur L. Carter Journalism Institute.
5 http://www.practicalecommerce.com/articles/92465-4-Predictions-for-2015-Holiday-Shopping-Season.
6 Some estimates are as high as 5 million. Amazon last confirmed that it had at least 1.4 million servers, but that was in December 2014.
7 The National Security Agency’s Utah Data Center alone is said to hold more than 5,000 servers with 12 exabytes of storage space, throughout a building that spans 1.5 million square feet.
8 http://www.businessinsider.com.au/beacons-impact-billions-in-reail-sales-2015-2
9 http://www.mediapost.com/publications/article/234217/4000-beacons-coming-to-all-macys-stores.html
10 See Footmarks at http://www.footmarks.com/beacons-regent-street/.
11 Clearly, we’re talking about devices that are no longer just smartphones but supercomputers that we carry on our person, linked not only to the cloud, but to smart glasses, beacons, sensors on and inside our bodies, the devices of others and a myriad of IoT devices and sensors.
12 www.createtomorrow.co.uk/live-examples/rugby-world-cup.aspx
13 “John Lewis Adopts Augmented Reality,” Inside Retail Australia, 9 September 2015.
14 To think that some people are worried about the “security” around mobile payments!
15 “Your voice is your passport,” STORES magazine, February 2015.
16 Recipients of the 2004 Nobel Prize for Physiology or Medicine
17 ABC News Oakland, CA, December 2014
