Big data demystified

Paul Kent, VP of big data at SAS, ranks big data as a"ground changing technology" that happens every seven or 10 years."

"If you don't adapt to it, you will be irrelevant in acouple of years," Kent said at a media briefing on big data and businessanalytics at SAS's headquarters in North Carolina.

Kent offers what he says is a "solid definition" of big data

"Big data is relative, not absolute," he says. The termapplies when the amount of data you are working on "puts you out of yourcomfort zone".

"Analytics can really be done anywhere in a company," hesays. "Big data is the democratisation of data. It is empowering anybody tothink 'I can find a pattern in some data and make useful business value out ofit'."

In this interview with CIO New Zealand, Paul Kent talks about his roleand how big data is impacting organisations:

He cites the case of a grocery store where an HR staffmember analysed the employee roll and discovered a large percentage of themwould come back to work at a certain time every year. This data influenced theonboarding process for these seasonal workers who are welcomed as existingemployees and do not "have to go through and learn all the rules andregulations over and over again".

He says while improvement through the use of data does notnecessarily turn into more profit for the company, "usually this is a sideeffect of doing a better job on anything". In this case, the company saved ontraining cost and lifted employee morale.

Kent says another simple application of big data wasdemonstrated by a rental business which uses elevator logs to observe which oftheir customers might be less able to pay. This is done by aggregating the dataof the number of people stopping on each floor. He says this "unstructureddata" of elevator logs was previously unused and kept in a data warehouse. Thecompany found that when there were fewer people going to an office, thatcompany may need a smaller space, or will be unable to pay the bill.

The lessons from all this is to "think differently" and forcompanies to modernise their computer resources, says Kent.

"Think of ways to use existing data in more complex ways,"he says. "Have you kept the door open to capture new data sets and makesomething out of it?"

Big data analytics can also use existing data in morecomplex and exciting ways. A mobile phone provider in Europe, for instance,looks at the network of people a customer interacts with to aid in itsassessment of customer churn. The model takes into account calling patterns andwhether the customer's network of friends are changing mobile phone carriers."You target more marketing dollars at them or listen to them with highervolumes," he says.

Analytics in theforefront

Another SAS executive, Nelle Schantz, discusses how analyticsis helping the New Zealand Automobile Association lead a more directedmarketing campaign for its insurance group.

She says the group used to send out 400,000 solicitations ayear "with minimum intelligence". It now sends 240,000 solicitation letters, areduction of 40 percent, but the response rate has doubled and increasedcustomer satisfaction.

Analytics has also allowed Bank of America examine data todetermine loan defaults on mortgage. They take calculations around loan defaultthat used to take four days to four hours. This gives the bank ability to reactto risk more quickly, says Schantz. She says proactive analytics is criticalfor the finance sector, which could be analyzing up to a billion rows of data."You can do complex assessments more quickly," she says.

Analytics has also provided Gilt Groupe a different insightinto their customers' experiences. Gilt Groupe runs "flash sales" for designerclothing and premium home accessories.

Gilt Groupe is active in the social media space so it isinterested in mining customer comments and predicting customer churn, saysTamara Gruzbarg, senior director of analytics and research.

Customer experience is key, she says, of their core businessmodel of providing a variety of "flash sales" in different categories. Eachsale is live for a limited amount of time and for a limited inventory. She saysthe industry space for this type of sales is getting "very crowded" as morecompanies including department stores are offering similar services. "How doyou provide the best customer experience and differentiate us from everybodyelse?"

Right now, they hold 50 new sales everyday. "We need to showyou something that is relevant to you and prompt you to buy," she says.

The group also found that the customers who return the mostpurchases are also the most loyal. "It doesn't make sense,"says Gruzbarg thefirst time she reported this to her business colleagues. But she says theirindustry is so specific so when customers "start feeling comfortable [dealingwith them], they can take a gamble on a purchase and if they don't like it,they can always return it and come back and purchase some more".

She says it is important to encourage everybody to innovateand ask questions. Questions from engineers "are as important as those comingfrom the senior vice president".

Kimberly Holmes, senior vice president of strategicanalytics at XL Group, says it is important to look at data broadly. "There isa lot of data that is not in numbers, we need to think about it in new says andtake advantage of new technology," says Holmes.

She says while unstructured data will not"overwhelm" business data at XL Group, "it is just as powerful asstructured data".

She says one of the "curses of the job" is that analyticscan help with every business problem. These can range from decisions on how tohire, who to promote, how to advertise and manage distribution. "I can't thinkof a question that can't be answered using analytics."

"We have enough work lined up for us in the next two years."

Divina Paredes (@divinap) is editor of CIO New Zealand.