The weatherman

The weatherman

Managing vast and fast-changing information while ensuring innovation is on track is a daily challenge at MetService. Chief information officer Russell Turner explains how he and his team meet the growing market for ‘nearcasting'.

MetService, as New Zealand's leading weather forecaster, is in the quintessential information business. Weather forecasting is the supply of pure information in easily interpreted forms to a growing number of industry sectors, where immediacy and accuracy are priorities.

The demand is growing for real-time meteorological information, and the space and time horizons of the forecasts are shrinking.

Clients ranging from events organisers to telecommunications and road maintenance workers provide a market for "nearcasting" — weather predictions for hours rather than days ahead and in small local regions.

As the science of meteorology has progressed, more of the complex modelling of weather patterns is given over to computers. The forecasters who used to manipulate basic level data now get data that is extensively pre-analysed, so they have time to do more detailed work at a higher level.

This naturally implies massive data volumes and data flows, says Russell Turner, MetService chief information officer. "The group that does the modelling here has quite a voracious need for data, and there's always more data becoming available. It's becoming easier to get [it] across the internet. Today it would be more than 11GB a day, within a year it will be 30GB a day. Some of that comes from Washington, so there are quite significant international bandwidth requirements. More and more data is being made available from such sources.

"As their computing power increases, our needs here to crunch that become much more significant," he says. "So probably more than every three years we have to upgrade our high-performance computers to the latest technology. In fact, we're often in the position of testing equipment before it's released to the market in New Zealand. We're testing quadcore processors that are not available on the general market [yet] but we have quite specific needs."

Turner says MetService has had some "interesting conversations" with the New Zealand Supercomputing Centre — a joint venture between Weta Digital and Telecom. "We've agreed with them to carry out some trials on their supercomputer in Wellington. We have a voracious need for CPU cycles and as the data volume increases, we need somewhere to carry out the processing."

Business continuity

From a CIO point of view, that has created some interesting issues, says Turner, such as back-up of large amounts of data and assurance of continuous operation and disaster recovery.

"As a first step towards improved disaster recovery, we're having to build a hardened data centre in the Wellington head office, then design our architecture to allow us to split our systems into two physical sites, one hosted in the capital and the other in the South Island, probably Christchurch.

"We have very significant requirements in terms of business continuity and disaster recovery. When you think there's only one national weather service in the country, and that we have a very strong responsibility for the protection of public property and life, it's imperative that our systems continue to work. We have commercial customers like the airlines. If they don't have a weather forecast before they take off, they can't fly.

"Aviation is one of the key market sectors we work for. There are 15 airlines that we do forecasting for, both domestic and international. We do jetstream turbulence forecasting for airlines that don't come anywhere near New Zealand."

The geographical extent of the market is another exceptional factor about MetService's work. The company provides weather information to many overseas markets as well, "so it's a 24 x 7 operation", says Turner.

"We have responsibility for marine forecasts all the way across the Pacific to about Easter Island. So we have quite a large area that we do forecasts for, and we do a lot of work in the Pacific Islands. They don't generally have the funding necessary to run meteorological observation stations. We help them establish those and put in place the structures and procedures for running them."

In the news

The media is one sector MetService is active in. "We've got TV1 and TV3, which people are familiar with here in New Zealand and what they see in the [daily press] in terms of forecasting but we're also in the Middle Eastern and Australian markets and in Europe, for example Ireland and Finland... we have our weather forecasting being presented."

MetService is also active in the energy sector where weather prediction is crucial to demand forecasting. "Something like 45 per cent of the UK energy market gets their forecasting from New Zealand," says Turner. "We do that because we're able to compete in terms of accuracy and giving them the products and performance they want."

A review of telecommunications infra-structure was one of Turner's early priorities when he joined MetService nearly two years ago.

"In New Zealand, communication with weather radar and automatic weather stations and the acquisition of data from them is important, and that's being extended over the next couple of years. There are quite major installations in very remote locations that present some difficulty with communications and generation of reliable information, when you have copper pairs running through farmers' paddocks."

Computers are also increasingly used in presenting the forecast.

"Our graphics product Weatherscape, as most people know, is used by the BBC. What's not well understood is that a number of other stations use it. We recently added Channel 4 in Finland for that and Channel 7 in Australia."

The need for an overarching architecture definition to formalise business requirements and the way IT meets those requirements has become increasingly evident. "In a lot of organisations, that's not in place at a practical level," says Turner.

"There might be some very high-level views as to what the long-term result might be, but at a practical level there needs to be a tactical plan for how you move off legacy systems and how you consolidate systems to get towards the business goals. That's what is missing in many organisations, that tactical plan to move from A to B.

"We've introduced service management practices [according to] ITIL and done the usual things, sending everyone on foundation [ITIL] courses. We've evaluated and purchased a configuration management system and we're in the process of installing that now."

Good governance

Architecture definition implies attention to the question of governance. "Good governance needs to be established right up front, making sure the business has good visibility of all of the projects and sets the priority. You don't want IT setting the priority," says Turner. "Good governance is the way to ensure all that happens — that and getting the right people in the right job. Over the long-term, people tend to migrate by a process of osmosis into jobs which they're not necessarily the best people for." Such mismatches have to be sorted out.

"So once you've done all that, once you got your basics sorted, you can turn your attention to doing smart things to leverage IT to help the business in other areas, which you cannot do until you've got your ducks in a row."

The move from legacy systems to industry standard systems is an immediate priority. "We've got quite a range of systems. In fact, it was only a few months ago that we got rid of our last old [Digital Equipment] MicroVAX. It was running one of the radars. The legacy system is essentially an OpenVMS environment, with a lot of Fortran code over file-system-based storage, written for the state-owned enterprise that MetService was.

Getting Fortran code written today is very difficult, and it's not very amenable to being split out to multiple sites and not very amenable to leveraging common database technologies like Oracle with Java. [Such] relational databases have replication across multiple sites and those Fortran and simple file-based systems don't lend themselves to replication. So that's one reason why we want to move on."

A lot of MetService applications are web-based to give customers access to the products they use through a convenient and uniform interface and this is not friendly to pre-web technologies.

"Essentially, we are more aggressively adopting common industry systems like Oracle with Java; probably Oracle Spatial as the underlying spatial database. We have those systems in place now. The issue is that we haven't completed the move to them. So a fairly aggressive tactical program is needed in order to complete the move. If we have the business case to build a new system, we'll do that, but in this case we're migrating a customer-base across from one system to another, that's another thing altogether.

"We've got a lot of work to do on those target systems. High density computing [in terms of rack-space and power per rack] and virtualisation are also important."

Balancing innovation

Turner's background is very much in ICT rather than meteorology. He started in 1984 as a junior programmer. "I worked my way through the steps of what was then known as 'analyst programmer' and ‘systems analyst' and I worked in Australia for a few years.

"Prior to MetService, I was at TelstraClear. I was in the management team looking after their IT systems and that was interesting because it [involved] three companies. It started off with Saturn, then it became Telstra Saturn, then it became TelstraClear.

"One of the things I've learned from that is you really can't spend too long on mergers and executing change. So many people say that but so few people initiate and carry out change quickly. You can't spend too much time worrying about the exceptions. If you do you'll be lost with the 80 per cent [of comparatively straightforward work] you need to address."

So a key skill to cultivate in the team is that of "knowing what not to do". This entails balancing innovation in key areas with constraints on time and budget and figuring out what desirable but non-essential elements of innovation can safely be omitted while preserving "an acceptable level of risk".

Urgency in the development timetable is exacerbated by the need to migrate completely off the legacy systems. "We've got to reduce the fragmentation of systems that we use for support," says Turner. "Like other companies, we have a set of legacy systems and a set of target systems, new ones to move to, and while we haven't completed the move, we've got multiple systems participating in one product, [and that means] fragmentation of skills. You've got to keep skills in legacy systems as well as acquire skills in target systems; it just makes life so much harder. So I'm pushing quite strongly towards reducing the number of systems, getting them down to a much smaller number, and you can't do that without a number of curly questions about the changes you have to execute."

The key difference between TelstraClear and MetService is the size of the organisation and the budget available. TelstraClear could afford to explore major innovations in all their complexity but ambitions have to be more closely trimmed at MetService.

Nevertheless, there has to be some discretionary budget for longer term research and development, Turner emphasises. "It's good to know you can afford to spend $20,000 or so exploring an idea, then ditch it if it's not going to work."

While he is wary of IT having overly adventurous ideas about running aspects of the business outside its domain, there are areas where the skills cultivated in IT are useful in a wider area. "Often IT appears to be a centre of excellence for process development [and] for project management. These things are our bread and butter, he says. We have people who are experienced in them, we have tools and techniques that we can bring to bear at various levels in many aspects of the business. And we've been doing that in terms of product-to-market processes, making sure that the product lifecycle process is supportable.

"Other things that IT can help with include the sales process. In one of my past lives I was the product manager for optical products at Phillips. You often find that IT people are very familiar with the sales process, so they're [sensitive to] the needs of the sales force when it comes to complex processes such as technology sales.

"I guess negotiation falls into that as well. Internally, you're always negotiating SLAs and you tend to have big suppliers and large projects that require significant capital outlay; so IT people are generally quite good negotiators [and] that's a skill that can help the business.

"Fortunately, I report to the CEO, and am a member of the management committee. When we sit around the table at meetings, you put aside the portfolio that you have and talk about the needs of the business.

"So I'm in a position where I can do that now. If you're in a position where you can influence business strategy, that's good. If you're not, then you've got a bit of a hard road to try and convince people — you do tend to get tarred with a [brush] of, ‘Well you're in IT so what would you know?' and so it's much harder to do starting from that place."

Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments