Tom Nolle: Cloud Data: Big AND Persistent!

We always hear about "Big" data, but a real issue in cloud storage is not just bigness but also persistence. A large data model is less complicated than a big application repository that somehow needs to be accessed. The Hadoop send-program-to-data model may be the answer.
3/28/2012 | 10 comments
View Comments: Threaded | Newest First | Oldest First
angelfuego   Cloud Data: Big AND Persistent!   4/13/2012 11:47:02 PM
Cloud Data: Big AND Persisitent
Thanks for the info!
Tom Nolle   Cloud Data: Big AND Persistent!   4/14/2012 7:52:42 AM
Re: Cloud Data: Big AND Persisitent
My pleasure; how are you using or planning for the combo if I may ask?

Tom
User Ranking: Blogger
mejiac   Cloud Data: Big AND Persistent!   4/20/2012 9:57:31 PM
Cloud Data Solutions
Great Video Tom,

I think that is safe to say that data will always be exponentially increasing, and thus access to that data at an acceptable speed is essential for maintain a business running.

I'm reminded of gmail, in the way that when you run a search, it can quickly retrieve an email much better than outlook, and yet maintain a high level of service.

Maybe there is something to be learn the way google manages this.
Tom Nolle   Cloud Data: Big AND Persistent!   4/21/2012 8:00:15 AM
Re: Cloud Data Solutions
Thanks, Mejiac!  I think the Google example is a good one.  They were pioneers in big data and also in distributing data and query in a hierarchy (MapReduce, now available in OpenSource as Hadoop).  One thing I think is true is that organizing data for quick access is easiest if you have a notion of what the predominate access will be.  For example, textual search can be accelerated by doing some syntactical analysis while indexing and being sure to check nouns first.  There are some divisions in big data now, between object stores that store things with structured properties (emails) and file stores that hold arbitrary data models.

Tom
User Ranking: Blogger
Technocrat   Cloud Data: Big AND Persistent!   4/21/2012 10:37:32 PM
Re: Cloud Data Solutions

Hi Tom and thanks for the information regarding Big Data persistence and repositories - the physical separation of data and the user.  If I understood correctly you mentioned, a feature of Hadoop that would separate the application and just send the "pieces of the application" to the user ?  Isn't this somewhat similar to what Amazon is doing with their recent Marketplace offering ?

Tom Nolle   Cloud Data: Big AND Persistent!   4/22/2012 7:54:21 AM
Re: Cloud Data Solutions
I think there's a little difference, Technocrat.  Hadoop is based on the presumption that big data is distributed over many sites, and so it's designed for applications that can be decomposed and sent to where the needed data is hosted.  The results can be collected and joined at the control node.  What I think Amazon is doing with marketplace is recognizing the reality of the cloud, which is that SaaS is the best answer overall for users.  So they attract software vendors to host apps on their cloud rather than try to sell IaaS services to what's going to be a small subset of the overall cloud market.  I think the non-hosted software is more a pull to get marketplace traffic up than a commitment to a broader model, though I'm sure they're happy to link software sales in a traditional sense and SaaS sales.

Tom
User Ranking: Blogger
Technocrat   Cloud Data: Big AND Persistent!   4/22/2012 1:22:11 PM
Re: Cloud Data Solutions

Thanks for the clarification Tom, I understand the difference between the two much better now !  : )

Tom Nolle   Cloud Data: Big AND Persistent!   4/22/2012 1:24:53 PM
Re: Cloud Data Solutions
Good, but we can all expect that this space is going to move forward two steps for every step that even the fastest of us can take!  We're in for a couple of years of classic catch-up ball, I'm afraid.  I know I'm scrambling.

Tom
User Ranking: Blogger
mejiac   Cloud Data: Big AND Persistent!   4/22/2012 9:28:29 AM
Re: Cloud Data Solutions
Thank you Tom for the reply,

Do you think that further down the road, will data management become more complex? and hence, will security protocols require more robust solutions?
Tom Nolle   Cloud Data: Big AND Persistent!   4/22/2012 11:03:52 AM
Re: Cloud Data Solutions
Good questions!  I think the cloud is going to redefine a lot of things in IT, one of which is data management.  The good news is that it's also going to result in a lot of automated tools that help organize information for distributed access.  For bigger businesses these tools will likely create a kind of "neutralizing" of the added complexities.  For smaller businesses the tools might actually make things easier by automating more!

Tom
User Ranking: Blogger
Latest Blogs
Charles Babcock   4/21/2017   14 comments
Eighty-four percent of large companies around the world say they are using artificial intelligence, and 62% say AI is important to remaining competitive in the year 2020. But ...
Susan Nunziata   5/28/2014   111 comments
For more than four years, EnterpriseEfficiency.com (E2) has been the best IT community on the Internet. As with all good things, soon our time together here will end.
Susan Nunziata   5/20/2014   95 comments
Is it time to ask for a raise? If you're a female IT executive, or more than 55 years old, your answer might well be a resounding "Yes!" Let's take a look at highlights of the ...
Brian Moore   5/19/2014   35 comments
I think they're lucky it still isn't a monk.
Sudha Nagaraj Bharadwaj   5/13/2014   32 comments
Internet surveillance in India is turning really fierce. A Facebook transparency report reveals that the social network removed 4,765 pieces of content originating in India in the second ...
Latest Archived Broadcast
In this episode, you'll learn how to stretch the limits of your private cloud -- and how to recognize the limits that can't be exceeded.
On-demand Video with Chat
IT has to deploy Server 2012 in a way that fits the architecture of its application delivery system.
E2 IT Migration Zones
IT Migration Zone - UK
Why PowerShell Is Important
Reduce the Windows 8 Footprint for VDI
Rethinking Storage Management
IT Migration Zone - FR
SQL Server : 240 To de mémoire flash pour votre data warehouse
Quand Office vient booster les revenus Cloud et Android de Microsoft
Windows Phone : Nokia veut davantage d'applications (et les utilisateurs aussi)
IT Migration Zone - DE
Cloud Computing: Warum Unternehmen trotz NSA auf die „private“ Wolke setzen sollten
Cloud Computing bleibt Wachstumsmarkt – Windows Azure ist Vorreiter
Like Us on Facebook
Twitter Feed
Enterprise Efficiency Twitter Feed
Site Moderators Wanted
Enterprise Efficiency is looking for engaged readers to moderate the message boards on this site. Engage in high-IQ conversations with IT industry leaders; earn kudos and perks. Interested? E-mail:
moderators@enterpriseefficiency.com
Informed CIO: Dollars & Sense: Virtual Desktop Infrastructure
Cut through the VDI hype and get the full picture -- including ROI and the impact on your Data Center -- to make an informed decision about your virtual desktop infrastructure deployments.

Read the full report
Virtualization Management: Time To Get Serious
Welcome to the backside of the virtualization wave. Discover the state of virtualization management and where analysts are predicting it is heading

Read the full report
PUBLIC SECTOR RESOURCES
WHITE PAPERS
A Video Case Study – Translational Genomics Research Institute
e2 Storage Video


On the Case
TGen IT: Where We're Going Next

7|11|12   |   08:12   |   10 comments


Now that TGen has broken new ground in genomic research by using Dell's storage, cloud, and high-performance computing solutions, the company discusses what will come next for it and for personalized medicine.
On the Case
Better Care Through Better Communications

6|6|12   |   02:24   |   11 comments


The achievements of the TGen/Dell project could improve how all people receive healthcare, because they are creating ways to improve end-to-end communication of medical data.
On the Case
TGen IT: Where We Are Now

5|15|12   |   06:58   |   6 comments


TGen is breaking new ground in genomic research by using Dell's storage, cloud, and high-performance computing solutions.
On the Case
TGen IT: Where We Were

4|27|12   |   06:45   |   10 comments


The Translational Genomics Research Institute wanted to save lives, but its efforts were hobbled by immense computing challenges related to collecting, processing, sharing, and storing enormous amounts of data.
On the Case
1,200% Faster

4|18|12   |   02:27   |   12 comments


Through their partnership, Dell and TGen have increased the speed of TGen’s medical research by 1,200 percent.
On the Case
IT May Improve Children's Chances of Survival

4|17|12   |   02:12   |   8 comments


IT is helping medical researchers reach breakthroughs in a way and pace never seen before.
On the Case
Medical Advances in the Cloud

4|10|12   |   1:25   |   5 comments


TGen and Dell are pushing the boundaries of computing, and harnessing the power of the cloud to improve healthcare.
On the Case
TGen: Living the Mission

4|9|12   |   2:25   |   3 comments


TGen's CIO puts the organizational mission at the heart of everything the IT staff does.
On the Case
TGen Speeding Up Biomedical Research to Save More Lives

4|5|12   |   1:59   |   6 comments


The Translational Genomics Research Institute is revamping its computing to improve speed, storage, and collaboration – and, most importantly, to save lives.
On the Case
Computing Power Helping to Save Children's Lives

3|28|12   |   2:13   |   3 comments


The Translational Genomics Institute’s partnership with Dell is enabling them to treat kids with neuroblastoma more quickly and save more lives.
Tom Nolle
How Deep Is My Storage Hierarchy?

7|3|12   |   2:13   |   5 comments


At the GigaOM Structure conference, a startup announced a cloud and virtualization storage optimizing approach that shows there's still a lot of thinking to be done on the way storage joins the virtual world.
E2 Interview
What Other Industries Can Learn From Financial Services

6|13|12   |   02:08   |   3 comments


We asked CIO Steve Rubinow what CIOs in other industries can learn from the financial services industry about datacenter efficiency, security, and green computing.
E2 Interview
Removing Big-Data Flow Bottlenecks

6|12|12   |   02:55   |   No comments


We ask CIO Steve Rubinow what pieces of financial services infrastructure need to perform better to get traders info faster.
E2 Interview
Getting Traders the Data They Need

6|11|12   |   02:04   |   1 comment


We ask CIO Steve Rubinow: What do stock market traders need to know, how fast do they need it, and how can CIOs get it to them?
E2 Interview
Can IT Help Fix the Global Economy?

6|8|12   |   02:32   |   2 comments


We ask CIO Steve Rubinow whether today's IT can help repair the global economy (and if IT played any role in the economy's collapse).
E2 Interview
More Competitive Business via Datacenter Strategy

5|4|12   |   2:46   |   1 comment


Businesses need to be competitive, yet efficient, and both goals affect datacenter design.
E2 Interview
The Recipe for Greater Efficiency

5|3|12   |   3:14   |   2 comments


Intel supplies the best ingredients to drive greater datacenter efficiency and support new compute, storage, and networking needs.
E2 Interview
Datacenters Enabling Business Transformation

5|1|12   |   06:37   |   1 comment


Dell’s Gaurav Chand says that for the first time ever datacenter technology is truly enabling all kinds of organizations to transform their business and achieve new objectives.
Tom Nolle
Project Lightning Streamlines Storage

2|16|12   |   2:09   |   2 comments


EMC's Project Lightning has matured into a product set, and it's important, less because it has new features or capabilities in storage technology and management, than because it may package the state of the art in a way more businesses can deploy.
Tom Nolle
Big Data Appliance Is Big News

1|12|12   |   2:18   |   No comments


Oracle's release of a Hadoop appliance for Big Data may be a signal that we're shifting to database appliances.
Tom Nolle
Myopia Can Hurt Storage Policy

12|22|11   |   2:08   |   No comments


We're at the beginning of a cloud-driven revolution in storage, but Oracle's quarter shows that enterprises are hunkering down on old concepts because they're afraid of the costs in the near term.
Sara Peters
An Untrained User & a Mobile Medical Device

12|19|11   |   2:43   |   11 comments


Untrained end users, clueless central IT staff, and expensive mobile devices are a worrisome combination for healthcare CIOs.
Tom Nolle
Too Many Labels on 'Big Data'?

12|9|11   |   2:12   |   3 comments


However you label it, structured and unstructured information are different and will likely always require different tools.
Sara Peters
E2 Debuts New Storage Section

12|8|11   |   1:51   |   1 comment


Need strategic guidance on everything from SSDs to 100 percent virtualized datacenters? Look no further.