HPC

Terror plot spy project helps Kinetica crank up database speed

Ingest billions of records per minute with no indigestion


HPC blog Kinetica presented its in-memory database solution at the recent OpenPOWER European Summit in Barcelona, and served up some benchmark results. You can see the entire presentation in the video below, but here are a few highlights.

First, for those of you unfamiliar with Kinetica, the company came out of a US military research project that aimed to identify terrorist types before they could strike. The problem was that it could take 90 minutes or more to comb through a mass of data and provide actionable intelligence on a specific target individual.

The founder of what eventually became Kinetica solved this problem by rewriting the database to run in memory and to use the latest accelerators. The end result is a blazingly fast database that can deliver intelligence in near real time.

While the solution runs on x86, it runs particularly well on the OpenPOWER platform, due to its new NVLink GPU interface. So how fast are we talking? Here are a couple of examples:

On a BoundingBox for one billion records, Kinetica was 561 times faster than MemSQL & NoSQL

On a Min/Max with one billion records, Kinetica was 712x faster than MemSQL & NoSQL

For SQL queries using 30 Kinetica OpenPOWER nodes versus 100+ x86 nodes running SAP HANA with 150 billion rows, Kinetica turned in near sub-second results compared with 13 seconds for HANA on a SELECT, 45 seconds for a HANA JOIN, and 35 seconds for a GROUP BY query. That’s pretty sporty.

Youtube Video

Check out the video for more details on Kinetica's benchmark results, and more about its OpenPOWER Foundation (Rebel Alliance) involvement. ®


Other stories you might like

  • UK Home Secretary delays Autonomy founder extradition decision to mid-December

    Could be a Christmas surprise in store from Priti Patel

    Autonomy Trial Autonomy founder Mike Lynch's pending extradition to the US has been kicked into the long grass again by the UK Home Office.

    Lynch is wanted in the US to stand trial on 17 charges of fraud and false accounting. He is alleged to have defrauded Hewlett Packard investors over the sale of British software firm Autonomy in 2011.

    Continue reading
  • Want to buy your own piece of the Pi? No 'urgency' says Upton of the listing rumours

    A British success story... what happens next?

    Industry talk is continuing to circulate regarding a possible public listing of the UK makers of the diminutive Raspberry Pi computer.

    Over the weekend, The Telegraph reported that a spring listing could be in the offing, with a valuation of more than £370m.

    Pi boss, Eben Upton, described the newspaper's article as "interesting" in an email to The Register today, before repeating that "we're always looking at ways to fund the future growth of the business, but the $45m we raised in September has taken some of the urgency out of that."

    Continue reading
  • All change at JetBrains: Remote development now, new IDE previewed

    Security, collaboration, flexible working: Fleet does it all apparently

    JetBrains has introduced remote development for its range of IDEs as well as previewing a new IDE called Fleet, which will form the basis for fresh tools covering all major programming languages.

    JetBrains has a core IDE used for the IntelliJ IDEA Java tool as well other IDEs such as Android Studio, the official programming environment for Google Android, PyCharm for Python, Rider for C#, and so on. The IDEs run on the Java virtual machine (JVM) and are coded using Java and Kotlin, the latter being primarily a JVM language but with options for compiling to JavaScript or native code.

    Fleet is "both an IDE and a lightweight code editor," said the company in its product announcement, suggesting perhaps that it is feeling some pressure from the success of Microsoft's Visual Studio Code, which is an extensible code editor. Initial language support is for Java, Kotlin, Go, Python, Rust, and JavaScript, though other languages such as C# will follow. Again like VS Code, Fleet can run on a local machine or on a remote server. The new IDE uses technology developed for IntelliJ such as its code-processing engine for features such as code completion and refactoring.

    Continue reading

Biting the hand that feeds IT © 1998–2021