Piranha Brings Affordable Big-Data to Government

Curtis Franklin Jr., Executive Editor | 11/30/2012 | 10 comments

Curtis Franklin Jr.
Are you ready to harness the power of a piranha? New software with big-data teeth could change the way your agency looks at text.

Oak Ridge National Labs' Computation Data Analytics Group (CDA) has spent the last nine years working on a system that looks not only at text data but at the context around the text. It tries, in a sense, to "read between the lines" of the data to help with detailed analysis of the contents. Doing this with massive amounts of data is hard, and the CDA has pulled out all the tech stops in order to make the analysis happen in a reasonable amount of time. In a statement about Piranha on the CDA website, they write:

We have pioneered a software agent approach to text analysis that uses a large number of agents distributed over very large computer clusters. This method works much faster than traditional approaches and provides the capability to cluster massive amounts of textual information in relatively short amounts of time, due to the scalability of the agent architecture.

Those are fine words, but what do they really mean for IT executives in government agencies and departments? On one level, they mean that the ability to make some sense out of giant mounds of text files, whether they're in the form of email messages, tweets, or scanned and OCR'd historical documents, has just gotten a bit closer. That's going to be good news for anyone trying to figure out how to make the "wish list" a reality for departments dealing with law enforcement, public health, or natural resource management (to name three quickly obvious areas).

Piranha could also be good news for someone who's trying to figure out what to do with a room full of slightly out-of-date workstations or servers. Make no mistake, Piranha is designed to run best on a high-performance cluster of the sort that a place like, say, Oak Ridge National Labs probably has sitting around waiting for a task. If you're fresh out of high-performance clusters, though, you can run the Piranha server on a cluster made up of just about anything that will support the Java Runtime Environment (JRE), version 1.4.2 or higher. That simple requirement covers an awful lot of computing power in relatively modest chunks.

The modest hardware requirements and very rich set of functions make Piranha perfect for departments and agencies with more brainpower than hardware capex in the budget. You should understand going in that Piranha is not a simple system: It's powerful but has an initial learning curve that approaches vertical. Once scaled, though, there are few big-data problems involving text that Piranha can't help you solve in a very cost-effective manner.

Big-data presents a problem set that many government agencies must deal with. The payoff, though, can be high. In the strict-budget environment that most government entities must work in today, a tool like Piranha can be the perfect way to take a bite out of the large problem of big-data.

View Comments: Newest First | Oldest First | Threaded View
batye   Piranha Brings Affordable Big-Data to Government   12/2/2012 3:00:56 AM
Re: Piranha Brings Affordable Big-Data
I think we would see soon gov. not being able to cope with the ammount of data...
MDMConsult   Piranha Brings Affordable Big-Data to Government   12/1/2012 12:19:56 AM
Re: Piranha Brings Affordable Big-Data
CIOs should be aware of cause, confusion and work to meet the challenges the public sector. There is much more pressure for government today to developing big data strategy, resolving issues for government and being further concerned with IT able to support big data's growth. 
zerox203   Piranha Brings Affordable Big-Data to Government   11/30/2012 10:53:20 PM
Re: why those?

I think in some way this highlights an important issue with how we perceive government effeciency. Those are three great examples Curt provided, because (as he pointed out) they deal with huge volumes of text. There's another reason, though - they're three public-facing departments. We know what they're called and what they do - so they're the ones we list. Someone in the know could probably rattle off a list of dozens... That is, someone like Oak Ridge. I bet a lot of businesses gunning for gov't contracts would be thrilled to have that viel peeled back just a little more.. to know what operations look like at every level, and make just the right kind of software.
zerox203   Piranha Brings Affordable Big-Data to Government   11/30/2012 9:46:28 PM
Re: Piranha Brings Affordable Big-Data
I feel as if there's two forms the government takes on in the public's mind - the super-elite, top-secret government that runs area 51 (complete with aliens, of course), and keeps tons of shady, assasin-laden business secret from us, and the other government. The one that can't run a data center right, the one that loses all your paperwork five minutes after you file it. One is run by people who can't get careers in the private sector (detractors claim), and the other  is the highest career one can aspire to.

The existance and popularity of companies who cater their consulting and other service to the government lead me to believe there's more of the second type of government than the first. Needing a tool to parse through large volumes of text seems more on the bureacratically benign side of things to me as well. Not to say that it's a sign of incompetence... just a sign that they're struggling with the same big-data problems as the rest of us, nothing more and nothing less.
CurtisFranklin   Piranha Brings Affordable Big-Data to Government   11/30/2012 8:21:58 PM
Re: why those?
@Dave, while I haven't run tests, the sense I get is that even lower-powered clusters can provide performance that's far beyond that of "brute force" techniques using other software (or no software at all). While it might not be lightning fast, it's still dramatically faster than the alternatives.
CurtisFranklin   Piranha Brings Affordable Big-Data to Government   11/30/2012 8:20:13 PM
Re: why those?
Absolutely right, @Dave. One of the first things it does it to help provide structure to that unstructured data, and that alone can be a huge help.
CurtisFranklin   Piranha Brings Affordable Big-Data to Government   11/30/2012 8:18:45 PM
Re: why those?
@Sara, the reason I singled those out was that they typically deal with large quantities of data in text form. I suspect lots of other departments will climb on board, but these are three obvious starting points.
David Wagner   Piranha Brings Affordable Big-Data to Government   11/30/2012 7:06:01 PM
Re: why those?
Also, Curt, you mention that the system works best on high performance stacks, but it will work on less impressive stacks. When on the lower end stacks, does it do the job fast enough to make it worthwhile?
David Wagner   Piranha Brings Affordable Big-Data to Government   11/30/2012 7:05:01 PM
Re: why those?
Good question, Sara. But i'll definitely agree. It looks like a greta way to make sense out of a lot of unstructured data.
Sara Peters   Piranha Brings Affordable Big-Data to Government   11/30/2012 6:51:48 PM
why those?
Curt you say "That's going to be good news for anyone trying to figure out how to make the "wish list" a reality for departments dealing with law enforcement, public health, or natural resource management (to name three quickly obvious areas)."  What makes you single out those three in particular?

The blogs and comments posted on EnterpriseEfficiency.com do not reflect the views of TechWeb, EnterpriseEfficiency.com, or its sponsors. EnterpriseEfficiency.com, TechWeb, and its sponsors do not assume responsibility for any comments, claims, or opinions made by authors and bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.

More Blogs from Curtis Franklin Jr.
Curtis Franklin Jr.   5/30/2014   10 comments
A good community can teach you a lot. And Enterprise Efficiency has been one of the best.
Curtis Franklin Jr.   5/26/2014   41 comments
Today is Memorial Day in the US, a day for remembering those who gave, in the words of Abraham Lincoln, "the last full measure of devotion" for their country and its citizens. It is a ...
Curtis Franklin Jr.   5/22/2014   35 comments
You're about to know precisely where your customers are and what they're doing. Are you ready for Big Data Advertising Everywhere?
Curtis Franklin Jr.   5/6/2014   19 comments
PHP is a great tool for building web pages that access databases. It's pretty nifty for pwning an enterprise site, too.
Curtis Franklin Jr.   4/30/2014   36 comments
BASIC turns 50 this year. Many IT pros wrote their first line of code in the venerable language, but is the ability to write code even important at the top of the IT ladder?
Latest Archived Broadcast
We talk with Bernard Golden about accelerating application delivery in the cloud.
On-demand Video with Chat
Register for this video discussion to learn how tablets can provide true business usability and productivity.
E2 IT Migration Zones
IT Migration Zone - UK
Why PowerShell Is Important
Reduce the Windows 8 Footprint for VDI
Rethinking Storage Management
IT Migration Zone - FR
SQL Server : 240 To de mémoire flash pour votre data warehouse
Quand Office vient booster les revenus Cloud et Android de Microsoft
Windows Phone : Nokia veut davantage d'applications (et les utilisateurs aussi)
IT Migration Zone - DE
Cloud Computing: Warum Unternehmen trotz NSA auf die „private“ Wolke setzen sollten
Cloud Computing bleibt Wachstumsmarkt – Windows Azure ist Vorreiter
Like Us on Facebook
Twitter Feed
Enterprise Efficiency Twitter Feed
Site Moderators Wanted
Enterprise Efficiency is looking for engaged readers to moderate the message boards on this site. Engage in high-IQ conversations with IT industry leaders; earn kudos and perks. Interested? E-mail:
[email protected]
Dell's Efficiency Modeling Tool
The major problem facing the CIO is how to measure the effectiveness of the IT department. Learn how Dell’s Efficiency Modeling Tool gives the CIO two clear, powerful numbers: Efficiency Quotient and Impact Quotient. These numbers can be transforma¬tive not only to the department, but to the entire enterprise.

Read the full report
The State of Enterprise Efficiency in the Virtual Era: Virtualization – Smart Approaches to Maximize Gains
Virtualization is a presence in nearly all enterprise data centers. But not all companies are using it to its best effect. Learn the common characteristics of success, what barriers companies face, and how to get the most from your efforts.

Read the full report
Informed CIO: Dollars & Sense: Virtual Desktop Infrastructure
Cut through the VDI hype and get the full picture -- including ROI and the impact on your Data Center -- to make an informed decision about your virtual desktop infrastructure deployments.

Read the full report
A Video Case Study – Translational Genomics Research Institute
e2 Video

On the Case
TGen IT: Where We're Going Next

7|11|12   |   08:12   |   10 comments

Now that TGen has broken new ground in genomic research by using Dell's storage, cloud, and high-performance computing solutions, the company discusses what will come next for it and for personalized medicine.
On the Case
Better Care Through Better Communications

6|6|12   |   02:24   |   11 comments

The achievements of the TGen/Dell project could improve how all people receive healthcare, because they are creating ways to improve end-to-end communication of medical data.
On the Case
TGen IT: Where We Are Now

5|15|12   |   06:58   |   6 comments

TGen is breaking new ground in genomic research by using Dell's storage, cloud, and high-performance computing solutions.
On the Case
TGen IT: Where We Were

4|27|12   |   06:45   |   10 comments

The Translational Genomics Research Institute wanted to save lives, but its efforts were hobbled by immense computing challenges related to collecting, processing, sharing, and storing enormous amounts of data.
On the Case
1,200% Faster

4|18|12   |   02:27   |   12 comments

Through their partnership, Dell and TGen have increased the speed of TGen’s medical research by 1,200 percent.
On the Case
IT May Improve Children's Chances of Survival

4|17|12   |   02:12   |   8 comments

IT is helping medical researchers reach breakthroughs in a way and pace never seen before.
On the Case
Medical Advances in the Cloud

4|10|12   |   1:25   |   5 comments

TGen and Dell are pushing the boundaries of computing, and harnessing the power of the cloud to improve healthcare.
On the Case
TGen: Living the Mission

4|9|12   |   2:25   |   3 comments

TGen's CIO puts the organizational mission at the heart of everything the IT staff does.
On the Case
TGen Speeding Up Biomedical Research to Save More Lives

4|5|12   |   1:59   |   6 comments

The Translational Genomics Research Institute is revamping its computing to improve speed, storage, and collaboration – and, most importantly, to save lives.
On the Case
Computing Power Helping to Save Children's Lives

3|28|12   |   2:13   |   3 comments

The Translational Genomics Institute’s partnership with Dell is enabling them to treat kids with neuroblastoma more quickly and save more lives.
Tom Nolle
The Big Reason to Use Office

3|18|14   |   02:24   |   46 comments

Office and personal productivity tools come in a first-class and coach flavor set, but what makes the difference is primarily little things that most users won't encounter. What's the big issue in using something other than Office, and can you get around it?
E2 Editors
SPONSORED: Mobile Security — A Use Case

3|4|14   |   04:27   |   16 comments

New mobile security solutions can accommodate a wide array of needs, including those of a complex university environment.
Tom Nolle
Killing Net Neutrality Might Save You Money

1|16|14   |   2:13   |   16 comments

The DC Court of Appeals voided most of the Neutrality Order, and whatever it might mean for the Internet overall, it might mean better and cheaper Internet VPNs for businesses.
Tom Nolle
The Internet of Everythinguseful

1|10|14   |   2:18   |   19 comments

We really don't want an "Internet of Everything" but even building an Internet of Everythinguseful means setting some ground rules to insure there's value in the process and that costs and risks are minimized.
Tom Nolle
Maturing Google Chrome

12|30|13   |   2.18   |   25 comments

Google's Chrome OS has a lot of potential value and a lot of recent press, but it still needs something to make it more than a thin client. It needs cloud integration, it needs extended APIs via web services, and it needs to suck it up and support a hard drive.
Sara Peters
No More Cookie-Cutter IT

12|23|13   |   03.58   |   21 comments

Creating the right combination of technology, people, and processes for your IT organization is a lot like baking Christmas cookies.
Sara Peters
Smart Wigs Not a Smart Idea

12|5|13   |   3:01   |   46 comments

Sony is seeking a patent for wigs that contain computing devices.
Tom Nolle
Cloud in the Wild

12|4|13   |   02:23   |   15 comments

On a recent African trip I saw examples of the value of the cloud in developing nations, for educational and community development programs. We could build on this, but not only in developing economies, because these same programs are often under-supported even in first-world countries.
E2 Editors
SPONSORED: Is Malware Evading Your IPS?

11|18|13   |   03:16   |   4 comments

Intrusion prevention software is supposed to detect and block malware intrusions, but clever malware authors can evade your IPS in these five main ways.
Sara Peters
Where Have All the Mentors Gone?

9|27|13   |   3:15   |   38 comments

A good professional mentor can change your life for the better... but where do you find one?
Tom Nolle
SDN Wars & You Could Win

9|17|13   |   2:10   |   5 comments

VMware's debate with Cisco on SDN might finally create a fusion between an SDN view that's all about software and another that's all about network equipment. That would be good for every enterprise considering the cloud and SDN.
Ivan Schneider
The Future of the Smart Watch

9|12|13   |   3:19   |   39 comments

Wearing a bulky, oversized watch is good training for the next phase in wristwatches: the Internet-enabled, connected watch. Why the smartphone-tethered connected watch makes sense, plus Ivan demos an entirely new concept for the "smart watch."
Tom Nolle
Cutting Your Cloud Storage Costs

9|4|13   |   2:06   |   3 comments

Cloud storage costs are determined primarily by the rate at which files are changed and the possibility of concurrent access/update. If you can structure your storage use to optimize these factors you can cut costs, perhaps to zero.
Sara Peters
Do CIOs Need an IT Background?

8|29|13   |   2:11   |   23 comments

Most of the CIOs interviewed in the How to Become a CIO series did not start their careers as IT professionals. So is an IT background essential?
Ivan Schneider
The Internet Loves Birthdays

8|27|13   |   3:25   |   69 comments

The Internet has evolved into a machine for drumming up a chorus of "Happy Birthday" messages, from family, friends, friends of friends who you added on Facebook, random people that you circled on G+, and increasingly, automated bots. Enough already.