In praise of FPGAs

Building an active data warehouse appliance


All data warehouse appliances have a massively parallel architecture in which there are multiple nodes that put processing as close as possible to the disk drives.

In Netezza's Performance Server these nodes are known as Snippet Processing Units (SPUs: pronounced, incidentally, to rhyme with Gnu rather than being a homonym for throwing up - though Teradata might think otherwise) where a snippet is executable machine code (compiled SQL).

In hardware terms an SPU consists of an FPGA (field programmable gate array) and a PowerPC processor. Now, when vendors talk about using commodity hardware neither of these are the sort of things that spring to mind.

Nevertheless, FPGAs are widely used for graphics processing, video and audio streaming and other applications, while PowerPC chips are commonly employed in small profile devices. In particular, PowerPC chips have very low power requirements compared to PC chips: for example, a Netezza rack holding 5.5Tb of user data (16.5Tb actual capacity) has a power rating of just 4,180 watts and, of course, there is commensurately low heat output. Given the current pressures on data centres this is a significant advantage that Netezza has not made as much of as it might.

The way the FPGA works in Netezza's architecture is that it effectively works like an event processing engine: data is streamed from disk and then passes through the query that is represented by the snippet (and is loaded into the FPGA). I had not previously appreciated this until a recent visit to Netezza's user conference but you can imagine (if you are a regular reader) my interest.

While I was in Boston for Netezza's conference, I also had a meeting with Kaskad Technologies, which is a new entrant into the event processing space, helping to build the new fully electronic Boston Stock Exchange, which goes live on 27 October, with its surveillance application.

Now, Kaskad has done some clever things with its event processing product but it is more or less comparable to the conventional vendors (Progress, StreamBase, Coral8 et al) in architectural terms. However, what is much more interesting is that the company is already working on its second generation of event processing architecture. This will involve the use of multiple finite state machines running in parallel, hanging off a single bus.

In other words, instead of having a single large server you have multiple small servers running in parallel. And how is Kaskad planning to implement these small servers? Using FPGAs. And bearing in mind that FPGAs are the technology of choice for streaming applications this obviously makes sense.

Now, readers may recall that I recently suggested that a better way to do what Teradata calls active data warehousing (whereby you use change data capture to trickle feed the warehouse and then run actionable queries and processes) would be to front-end an event processing engine on to a data warehouse appliance. In fact, Sybase (albeit that Sybase IQ is not an appliance) is doing exactly this with its Risk Analytics Platform, for which Sybase is talking to a number of event processing vendors about front-ending their products.

However, the use of FPGAs opens up another possibility. Why not have different types of FPGA within the same architecture? You could have some FPGAs processing data that was streamed from disk and other FPGAs that processed data that was streamed from external environments off a bus.

In other words, you could build an active data warehouse appliance. Or, looking at it another way: an integrated event processing/warehouse environment. Given that you often want to access historic information for event processing purposes this could be an architecture that makes sense.

Copyright © 2006, IT-Analysis.com


Other stories you might like

  • GPL legal battle: Vizio told by judge it will have to answer breach-of-contract claims
    Fine-print crucially deemed contractual agreement as well as copyright license in smartTV source-code case

    The Software Freedom Conservancy (SFC) has won a significant legal victory in its ongoing effort to force Vizio to publish the source code of its SmartCast TV software, which is said to contain GPLv2 and LGPLv2.1 copyleft-licensed components.

    SFC sued Vizio, claiming it was in breach of contract by failing to obey the terms of the GPLv2 and LGPLv2.1 licenses that require source code to be made public when certain conditions are met, and sought declaratory relief on behalf of Vizio TV owners. SFC wanted its breach-of-contract arguments to be heard by the Orange County Superior Court in California, though Vizio kicked the matter up to the district court level in central California where it hoped to avoid the contract issue and defend its corner using just federal copyright law.

    On Friday, Federal District Judge Josephine Staton sided with SFC and granted its motion to send its lawsuit back to superior court. To do so, Judge Staton had to decide whether or not the federal Copyright Act preempted the SFC's breach-of-contract allegations; in the end, she decided it didn't.

    Continue reading
  • US brings first-of-its-kind criminal charges of Bitcoin-based sanctions-busting
    Citizen allegedly moved $10m-plus in BTC into banned nation

    US prosecutors have accused an American citizen of illegally funneling more than $10 million in Bitcoin into an economically sanctioned country.

    It's said the resulting criminal charges of sanctions busting through the use of cryptocurrency are the first of their kind to be brought in the US.

    Under the United States' International Emergency Economic Powers Act (IEEA), it is illegal for a citizen or institution within the US to transfer funds, directly or indirectly, to a sanctioned country, such as Iran, Cuba, North Korea, or Russia. If there is evidence the IEEA was willfully violated, a criminal case should follow. If an individual or financial exchange was unwittingly involved in evading sanctions, they may be subject to civil action. 

    Continue reading
  • Meta hires network chip guru from Intel: What does this mean for future silicon?
    Why be a customer when you can develop your own custom semiconductors

    Analysis Here's something that should raise eyebrows in the datacenter world: Facebook parent company Meta has hired a veteran networking chip engineer from Intel to lead silicon design efforts in the internet giant's infrastructure hardware engineering group.

    Jon Dama started as director of silicon in May for Meta's infrastructure hardware group, a role that has him "responsible for several design teams innovating the datacenter for scale," according to his LinkedIn profile. In a blurb, Dama indicated that a team is already in place at Meta, and he hopes to "scale the next several doublings of data processing" with them.

    Though we couldn't confirm it, we think it's likely that Dama is reporting to Alexis Bjorlin, Meta's vice president of infrastructure hardware who previously worked with Dama when she was general manager of Intel's Connectivity group before serving a two-year stint at Broadcom.

    Continue reading

Biting the hand that feeds IT © 1998–2022