Hadoop Summit 2010 - What a Ride!

Hadoop Summit 20010 - What a ride... finally I have a few minutes to recap on the past few crazy weeks.


Attending dozens of big conferences over the past few years, I never really knew what is behind the nice colors, music and streamline sessions. Well behind it there is huge amount of work, planning and obviously a significant "expect the unexpected".

It is all about team work and trusting others to do their part the same way they trust you.

It was great to be "sold out" 10 days ahead, no pressure on people not showing up. Although I have to admit that until I saw +1,000 heads in the huge main ballroom, I was not at-ease.

The most rewarding thing for me is that you can see at the end of one single day the fruits of your efforts. You can almost hear the BigData community moving forward.

I've did a little recap with photos and videos on the Yahoo! Developers Network Blog. Yahoo! NewsBytes team also did a cool video clip.

I think the most fun part is to scroll through the twitter feeds.

Enjoy and see you next year

Dekel

Hadoop Bay Area User Group

Hi All,

Come join us this week for the monthly Bay Area Hadoop User Group (HUG).

Whether you are an active submitter, building/planing to build hadoop-based applications or completely new to Hadoop -- we'd love to see you.

We have an exciting agenda planned covering features updates from the massive usage of Hadoop at Yahoo! as well as technologies/solutions developed by the growing hadoop eco-system.

Below are the planned sessions:

• Katta, Solr, Lucene and Hadoop - Searching at scale
  Jason Rutherglen and Jason Venner
• Walking through the New File system API
  Sanjay Radia, Yahoo!
• Keep your data in Jute but still use it in python
  Paul Tarjan, Yahoo!

It's happening on Wednesday, November 18, 6PM-8PM at the Yahoo! Sunnyvale Campus.

Please sign-up on the Bay Area HUG Meetup page (see Meetup page for location specifics).

This is the last HUG for 2009, as the December HUG is canceled due to the holidays.

Hope to see you there and if you can't make it person, we are posting the presentations on the Yahoo! Developer Network Hadoop blog

Dekel

Yahoo! at Cloud Computing Expo

Hi
I had lot's of fun this week at cloud computing expo in santa clara.
It was really impressive to see how people from the cloud industry see Yahoo's massive scale cloud technology and how our open source strategy resonates.
I got a lot of comments around "it's great to see a massive enterprise adopting cloud for it's core business".

I think our message of cloud is about faster innovation and less around saving costs hit the spot in this event.

I blogged about the event in the Yahoo Developer Network (YDN)
http://developer.yahoo.net/blog/archives/2009/11/cloud_expo.html


And yes.. it was great to meet many of you - friends and colleagues.

Cheers
Dekel

Happy Holidays

I am off to a new cloud-related adventure with the break of 2009. Looking forward to post soon from my new role.

Cheers and happy holidays
Dekel

A joint Webinar with CohesiveFT - Making Cloud Portability a practical reality

You are welcome to join us next week – Tuesday, November 18th, 1PM EST,  to a joint Webinar with our good partners – CohesiveFT.  

During this 1 hour Webinar we will provide a live demo illustrating how you can deploy, scale and control a complex transactional application between two clouds – EC2 and Flexiscale 

You will see with your own eyes that “cloudstorming” (connecting multiple clouds) Cross-cloud failover and hybrid solutions between private data center and clouds are more than just words, it is really available today.

We hope to convince you that combining CohesiveFT’s VPN Cubed solution  and GigaSpaces XAP  with its Cloud framework  will allow you to capitalize on the immediate cost and efficiency benefits of cloud-enabling your IT infrastructure as well as make the cloud relevant for a variety of data-intensive transactional applications.

Space is limited, so please sign-up as soon as possible.

November 18, 2008, 1:00 PM EST.

Register at: https://www1.gotomeeting.com/register/593992249 

We hope to see you there.

Dekel

Feedback on the GigaSpaces-EC2 Framework

Our UK partners  NTE  posted on their a blog a detailed feedback  for the EC2 Framework private beta . Ian thanks for your feedback, we really appreciate it. 

I have to say I’m quite excited by the amount of traction our EC2 and cloud solution is receiving these days.

I encourage all of our users to provide feedback and suggest things we should do differently or better.  

With that in mind, let me comment on Ian’s great feedback.

Issue 1 - Adding ports to security groups

You need to open the following ports on the “default” security group in order to use the framework 

TCP Port 22 - used for SSH communication to the AMIs    

TCP Port 80 - used for deploying the GigaSpaces Web Management Center and the Ganglia monitoring tool

TCP Port 443 - used for secured http. (we fixed the typo, thanks!)

You can do the above very easily using the elastic fox plugin (no need to install the Amazon API). Click the “security groups” tab and then click the “Grant Permission” green check mark. We will add this to our docs.

The private beta currently supports the default security group. The next release will have support for custom security groups, and then it will be recommended to create your own group and not to use the default one. 

Issue 2 - start-ui = true

“true” and “admin application” is the same thing, in the next release we've changed the default in the data-example to be “admin-application”. 

Issue 3 - Starting the GigaSpaces Management Tool. 

In the current version we still have some issues with the UI connectivity, It has been fixed for the next release. Adding the nxclient local installation as a better workaround, is a good idea, we will updated the docs accordingly.

Step-by-step Instructions

The cloud framework can save some of the trouble of doing this, but obviously if you want to use the EC2 API directly, your post  contains a very helpful description. Thank you. We will work on adding a “step-by-step” guide to our WIKI as well.  

A few notes I would like to add:

1. As mentioned above I recommend using the FireFox Plugin  to open the ports.

2. Please make sure the AWS Account ID you are using is dedicated to GigaSpaces (i.e. not used to run AMIs which are not GigaSpaces related), we don’t want to bill you for non-GigaSpaces usage :)

3. Port 22 is used for SSH communication to the AMIs

4. Monitoring – we have ganglia built-in to the cloud framework AMIs, you can point to the UI machine with /ganglia at the end of the public IP and view the entire cluster.

Cheers

Dekel

Not a user yet? Interested in access to the private beta? Click here.

GigaSpaces EC2 Framework - Private Beta release

I am excited to bring to your attention that we have released today a private beta of the GigaSpaces EC2 Framework.

This framework “clouds” the GigaSpaces Application Server to enable easy development and deployment in the cloud. 

You can deploy applications on EC2 using the "One Click Cloud" approach - develop the application code (i.e. processing units) on your desktop and then use this framework to deploy, run and monitor it on EC2 with a few simple scripts.

This framework is designed specifically for cloud portability. EC2 is only the exciting beginning and expect CohesiveFT, Flexiscale, GoGrid, Joyent, RightScale and more to follow soon.

The private beta is currently available to our signed customers and partners. A public beta will be available very soon. If you want access to this exciting framework, email us at cloud@gigaspaces.com.

Shana Tova