lassedesignen - Fotolia

Energy company targets electricity thieves with Isilon, Pivotal Hadoop

Power company BC Hydro runs 10 Isilon nodes and the full Pivotal Hadoop software stack to perform analytics to curtail unauthorized taps of its electricity.

Canadian power company BC Hydro is using EMC storage and a Pivotal Hadoop distribution to curb electricity theft.

The Vancouver-based utility has installed nearly 2 million smart meters since 2011, triggering "a tsunami of data that we need to make actionable," said Elizabeth Fletcher, deputy director of BC Hydro's smart metering and infrastructure program.

The smart meters are part of a $939 million modernization project that BC Hydro started in 2007. The impetus for the upgrade was an estimated $100 million annual loss in revenue due to unauthorized use of its electricity generation.

"We developed a business case that we could save $20 million over seven years. From that, we developed a theory of how to do it, and then [chose] technology partners that could help us make that theory into a reality," Fletcher said during a user panel last week at EMC World 2015.

BC Hydro picked EMC for its scale-out storage and the big data analytics of its Pivotal subsidiary.

The energy company is installing a Pivotal Greenplum database, a Pivotal Hadoop distribution and EMC Isilon scale-out NAS. These products are part of the EMC Federation data lake initiative.

A Hadoop cluster that crunches the numbers runs on Isilon storage. The analytics indicate whether someone is illegally tapping into the energy grid or if data sets have been incorrectly entered in the system.

Sensor data drives storage upgrade

Fletcher fleshed out the details during an interview following a presentation at EMC World in Las Vegas. She said the IP-connected meters generate approximately 25 TB of new storage each year. The sensors measure the amount of electricity added to the grid and pinpoint how it is consumed.

She said BC Hydro chose EMC high-density storage hardware and the Pivotal software stack to help manage the dynamic information flow. The company installed three racks of Pivotal Data Computing Appliances to support a Greenplum 4.3 database that runs energy-analytics software.

The Pivotal HD 3.0 Hadoop distribution and Pivotal HAWQ parallel-processing query engine software run on Cisco UCS servers and connect to a 10-node configuration of EMC Isilon X410 nodes.

"The sensors are bringing back new types of data that we need to analyze. To be able to use it, we needed to upgrade our storage," Fletcher said.

Couple Pivotal Hadoop cluster with X410 NAS for long-term storage

Coupling the EMC X410 with a Pivotal Hadoop cluster enables complex data sets to be quickly parsed and analyzed, she said.

BC Hydro backs up to a cluster of EMC Data Domain DD990 backup appliances running Data Domain Boost deduplication software. (EMC announced at EMC World that the DD990 is being phased out in favor of the higher-capacity DD9500).

The X410s provide 160 TB of capacity and the ability to scale storage as needed, enabling BC Hydro to postpone storage upgrades for a while.

"We are looking at a seven-year horizon [before the next upgrade]. We keep most data for at least seven years and then we archive it," Fletcher said.

Next Steps

Pivotal's big data strategy rests on open source technology

Hadoop distributor Pivotal has intriguing big data viewpoint

Pivotal aligns its Hadoop-based products with Open Data Platform

Dig Deeper on NAS devices