Happy Holidays

I am off to a new cloud-related adventure with the break of 2009. Looking forward to post soon from my new role.

Cheers and happy holidays
Dekel

A joint Webinar with CohesiveFT - Making Cloud Portability a practical reality

You are welcome to join us next week – Tuesday, November 18th, 1PM EST,  to a joint Webinar with our good partners – CohesiveFT.  

During this 1 hour Webinar we will provide a live demo illustrating how you can deploy, scale and control a complex transactional application between two clouds – EC2 and Flexiscale 

You will see with your own eyes that “cloudstorming” (connecting multiple clouds) Cross-cloud failover and hybrid solutions between private data center and clouds are more than just words, it is really available today.

We hope to convince you that combining CohesiveFT’s VPN Cubed solution  and GigaSpaces XAP  with its Cloud framework  will allow you to capitalize on the immediate cost and efficiency benefits of cloud-enabling your IT infrastructure as well as make the cloud relevant for a variety of data-intensive transactional applications.

Space is limited, so please sign-up as soon as possible.

November 18, 2008, 1:00 PM EST.

Register at: https://www1.gotomeeting.com/register/593992249 

We hope to see you there.

Dekel

Feedback on the GigaSpaces-EC2 Framework

Our UK partners  NTE  posted on their a blog a detailed feedback  for the EC2 Framework private beta . Ian thanks for your feedback, we really appreciate it. 

I have to say I’m quite excited by the amount of traction our EC2 and cloud solution is receiving these days.

I encourage all of our users to provide feedback and suggest things we should do differently or better.  

With that in mind, let me comment on Ian’s great feedback.

Issue 1 - Adding ports to security groups

You need to open the following ports on the “default” security group in order to use the framework 

TCP Port 22 - used for SSH communication to the AMIs    

TCP Port 80 - used for deploying the GigaSpaces Web Management Center and the Ganglia monitoring tool

TCP Port 443 - used for secured http. (we fixed the typo, thanks!)

You can do the above very easily using the elastic fox plugin (no need to install the Amazon API). Click the “security groups” tab and then click the “Grant Permission” green check mark. We will add this to our docs.

The private beta currently supports the default security group. The next release will have support for custom security groups, and then it will be recommended to create your own group and not to use the default one. 

Issue 2 - start-ui = true

“true” and “admin application” is the same thing, in the next release we've changed the default in the data-example to be “admin-application”. 

Issue 3 - Starting the GigaSpaces Management Tool. 

In the current version we still have some issues with the UI connectivity, It has been fixed for the next release. Adding the nxclient local installation as a better workaround, is a good idea, we will updated the docs accordingly.

Step-by-step Instructions

The cloud framework can save some of the trouble of doing this, but obviously if you want to use the EC2 API directly, your post  contains a very helpful description. Thank you. We will work on adding a “step-by-step” guide to our WIKI as well.  

A few notes I would like to add:

1. As mentioned above I recommend using the FireFox Plugin  to open the ports.

2. Please make sure the AWS Account ID you are using is dedicated to GigaSpaces (i.e. not used to run AMIs which are not GigaSpaces related), we don’t want to bill you for non-GigaSpaces usage :)

3. Port 22 is used for SSH communication to the AMIs

4. Monitoring – we have ganglia built-in to the cloud framework AMIs, you can point to the UI machine with /ganglia at the end of the public IP and view the entire cluster.

Cheers

Dekel

Not a user yet? Interested in access to the private beta? Click here.

GigaSpaces EC2 Framework - Private Beta release

I am excited to bring to your attention that we have released today a private beta of the GigaSpaces EC2 Framework.

This framework “clouds” the GigaSpaces Application Server to enable easy development and deployment in the cloud. 

You can deploy applications on EC2 using the "One Click Cloud" approach - develop the application code (i.e. processing units) on your desktop and then use this framework to deploy, run and monitor it on EC2 with a few simple scripts.

This framework is designed specifically for cloud portability. EC2 is only the exciting beginning and expect CohesiveFT, Flexiscale, GoGrid, Joyent, RightScale and more to follow soon.

The private beta is currently available to our signed customers and partners. A public beta will be available very soon. If you want access to this exciting framework, email us at cloud@gigaspaces.com.

Shana Tova

Go out there and start clouding!

Over the past few weeks I’ve been asked what’s my view on adoptions barriers and challenges when moving application to the cloud (either public or private), specifically as it relates to large organizations. I’ve shared those views with several of our customers and prospects on EC2 (big as well as small) and I thought it would make sense to blog about this as well.

Security

This is probably the first question people ask. Can I put my application on a public cloud? My first answer is why do you think your data center is more secured than Amazon's, GoGrid's, Flexiscale, Joyent, etc.. (same analogy of money under the mattress and banks a few decades ago). My second is answer is that you should go with the Cloud provider that satisfies your concerns (e.g. a cloud vendor called AppNexus is providing advanced security features designed specifically for hedge funds requirements), while having the flexisbility to move from your own data-center (or “internal/private” cloud as it’s called today) to the cloud of your choice. Obviously this is not a trivial thing to achieve without a framework like GigaSpaces.

It is also why we strongly believe in cloud portability. Last week’s announcements from VMware’s vCloud and Citrix’s Cloud Center certainly proves we are not alone in this approach. 

When drilling down to this question I find it interesting that in some cases security comes down to – what happens if I’m running the business logic outside the cloud and the data inside (is the client-server traffic secured?) and what happens when I need to persist the data (e.g. flash to S3 or EBS in the case of Amazon) to maintain a runtime state. If this is your security concerns then GigaSpaces can help reduce both – with Space Based Architecture, logic and data are running together in the same in-memory process (in the same Amazon Machine Image - AMI  in the case of EC2).  As for latter, you can keep your runtime data partitioned in-memory across any number of AMIs (while having one or more in-memory backups for each partition) without the need to persist it to S3.  (note also the performance implication of this later in the post)

In addition partitioining is a great soltution for "not keeping all the eggs in one basket", or in this case all the data in one AMI. This can serve as a secutirty mechanism as well as a way to "control and contain" a pontetail damage (see the Audit section below).

Please read the recently released Amazon Security White Paper, I think this long-awaited paper helps address some of the common security concerns as well.

Reliability, SLAs and data loss

The second most common concern is “Amazon does not provide SLA”, “What happens if my AMI fails”. 

The whole idea behind sharing recourses in a virtual environment is that you shouldn’t care about the availability of a specific resource. In other words, nodes in the cloud can and will break. Amazon will undoubtedly provide some sort of SLA very soon, other vendors like GoGrid or Joyent offers it today, but none of these SLAs will truly cover zero data loss and data reliability. The reason for that is simple (and accurate in my view), this is the responsibility of the application architecture and not the cloud provider. The question one is asking, does this mean I need to develop my own reliable data-layer, the answer in most cases is yes. 

However, XAP Cloud Application Server  solves this problem out-of-box. So if you are using GigaSpaces on EC2 for example, you really don’t care if you’re AMI fails, your application will fail-over , at in-memory speeds to another AMI that is running a “hot” backup. The result - zero data loss and zero downtime.

Persistency is another painful issue with regards to reliability. I believe all cloud vendors will have issues in making cloud persistency available in a high-performance reliable manner (our customers are having a lot of issues with S3, and thank you Amazon for releasing EBS). Ironically, the the cloud vendor is not at fault, it is the tier-based architecture itself. Cloud environments simply “shines” the need to persist a-synchroniously to the “cloud disk”, while having the run-time data partitioned in-memory across many nodes. This is exactly why our cloud framework includes support for GigaSpaces Mirror Service.

The risk of complexity or the “hidden costs of the cloud”

I agree with Nikita that deploying and developing to the cloud involves complexity and architectural challenges. It is somewhat ironic that the cloud offers enormous potential in terms of cost saving and efficiency but at the same time can introduce considerable hidden costs (thank you John Willis for this coining the term). I believe cloud vendors and solutions like RightScale will make it simpler to use the cloud as we move forward, but the fundamental fact will not change – moving a complex application (specifically “enterprise grade” transactional and data intensive) from a “static” environment to the cloud is not a simple and trivial task. In fact, although I seldom hear this as a primary concern, I believe this is the most significant barrier of moving to the cloud. 

The good thing is I also believe XAP Cloud Application Server considerably reduce this risk and basically “cloud enable” and “future proof” your application. In other words, by running your application on GigaSpaces you are making sure to avoid these “hidden costs” and that your application will not break when you move it from the single developer machine to a cloud of hundreds of AMIs. 

Development and “cloud lock-in”

I often hear the argument that cloud is not platform agnostic, forces you to use specific software stacks, etc… Well as I mention above, I believe cloud portability combined with a flexible/"open" development environment is the answer. There will not be one cloud for everyone (as we see de-facto today), so you have to “enable” your application to run on multiple clouds. In addition your developers will want to use their familiar IDEs, debugging and testing tools, or basically “develop on their desktops” and use the cloud as the deployment and runtime environment.

Gigaspaces Cloud Framework approach is exactly that – develop on your desktop with Java/.NET or the scripting language of your choice and then deploy it to a cloud like EC2 with “one click”. The big advantage of using GigaSpaces in this context is twofold: 

1. Due to the “Write once Scale anywhere” capability, you CAN develop and test on a single desktop and then seamlessly deploy to a large cloud

2. As your application logic bundled with the data it needs, it becomes an atomic unit running as a service on the GigaSpaces java-based  containers. This means you can move your application, almost seamlessly, between clouds.

Audit, logging and compliance 

A concern I hear often, is “I have no idea where my data runs”, “How can I audit, track and isolate an issue with my application, it can run anywhere”. Specifically I heard this concern in the context of Google App Engine and the BigTable implementation. 

This is a valid concern that Amazon partially address in their security white paper. Currently I think vendors like GoGrid, Joyent and Flexiscale offer a better solution to this issue.

As for GigaSpaces, the fact that your data is coupled with the logic in several Processing Units that runs on specific nodes, plus the fact that you can easily decide how your data (and logic) are partitioned across the cluster, enables you to have a much higher control of the   “application whereabouts” in the cloud.

Recently I did the cloud framework demo  to a Telco company. They were very concerned about knowing at any given moment where a specific application service is running, after seeing the demo they commented “all I have to do is watch the GigaSpaces UI and track a specific Processing Unit and I can know instantly on which AMI it is running”.

Additional advantage of this approach, that may be helpful to our friends at Singapore is that if something goes wrong in a specific service, the problem is “contained” to this specific node and its backup.

Another concern I encounter in this context is data segmentation, people wants to achieve a locking and SQL access isolation in a granularity of a specific row or a specific column in a table (you can think of it as a security concern as well). This can be somewhat complex to achieve with databases today in the cloud, however simplier to achieve with GigaSpaces. In the "space approach" everything is object-based - a "row" is an instance of an object, a "column" is a value of a specific attribute within this object. GigaSpaces enables roles and permissions hadnling in the granularity of an attribute within an object.

 

Performance “Penalty” and Scalability – will my application run faster and scale?

Obviously people are concern about what will be the performance implications of moving their applications to the cloud. From my experience this revolves mostly around latency and not limited to public clouds (i.e. the same concern applies to organization who are thinking about implementing clouds in their own data centers).

If you plan to run your business logic and data on different nodes, or logic outside the cloud and data inside, on top of the security issue you are in for an unpleasant surprise in terms of high latency. The appropriate solution is to run the data and logic together on the same cloud node. This is not trivial to achieve on your own, but literally out-of-the-box with Gigaspaces.

As for linear scalability or the “holy grail” of cloud computing, the answer can be quite complicated, as traditional tier-based middlware will not scale out-of-the-box. Fortunately, I truly believe that with GigaSpaces scalability CAN be made seamless. In my personal view, this one of the most compelling reasons to use GigaSpaces on the cloud.  Or in other words, by using Gigaspaces on the cloud you can truly make the “Cloud Computer”  works 'as advertised' (credits to Kent from Joyent of coining this term). Please read a nice post by Nati Shalom in that regards 

To conclude…

I assume many of you read a few months ago the enterprises-aren’t-ready-to-trust-the-cloud post on The Server Side. 

I personally think they are, big time! Not without serious concerns, but they are gradually moving in this direction, mainly due to the fundamental fact that it makes perfect economic sense. 

I believe the biggest advantage about implementing your application on top of GigaSpaces is that it allows you to take “baby steps to the cloud” while significantly reducing the risk of joining this emerging, but challenging, technology.

Go out there and start clouding…

GigaSpaces’ upcoming cloud framework

Recently I had the pleasure of doing a demo to one of our customers using the new “cloud framework”.

This framework brings forth 3 key concepts, which we think are essential to successfully develop, test and deploy any scalable transaction applications on the cloud:

“Desktop on the cloud”
View the cloud environment, as if it was your own desktop. You can develop and unit-test using any familiar IDE and then deploy it to the cloud for testing with zero code-changes. In addition the distributed environment in which you deploy and run your code does not affect your business logic. At GigaSpaces, we call it “write once, scale anywhere”

“One-Click cloud”
Once your application is unit-tested and running on your desktop, you can seamlessly transition and scale it on a large number of Amazon Machine Images (AMIs) without any configuration, administration or application changes. This is a key concept behind the idea of “pay as you go” in the cloud. As I wrote in a previous post it’s great that Amazon charges $0.40 an hour for a large machine, but if it takes 4 weeks to deploy my application to this environment, it is no longer that attractive. This framework really set the above as a practical reality.

Scale on demand
I have discussed the missing link between on-demand Hardware (like EC2) and on-demand scalability in a recent post. In this demo I deploy a market-data application with one click, then with another click from my laptop, I add more AMIs and the application simply scales! (by increasing the amount of stock symbols being processed..)

I have to remove my hat (I bought one especially for that…) to our amazing development team – with our new cloud framework, you can take your “mini application” (we call it "processing unit") and literally deploy it to any number of AMIs in one click – behind the scenes, the framework launches the AMIs, start the GigaSpaces containers (GSCs) and deploys your code to the Gigaspaces-EC2 cluster.
It even starts the GigaSpaces Graphical Management Center in the cloud and automatically connect your desktop to it from any internet browser.
All of this magic requires one simple “unzip” to a few Megs of the cloud package and an AWS user credentials. All the GigaSpaces runtime is pre-configured on our AMIs.

The above is the slide I’m using for the demo, which I hope, help clarify what I wrote... a picture worth a thousand words, right?

This cool staff is already available as a private beta and until our formal release we plan other exciting features like support for the recently announced Elastic Block Storage (EBS) using our Persistency as a Service model, support for MySQL, monitoring the AMIs status and more...

If you want access to the private beta, please contact us at ec2@gigaspaces.com

Cheers
Dekel

RightScale and EBS

At this post for a change, I don’t want to write about GigaSpaces...

Today I've attended RightScale's Webinar regarding their support for the recently announced Amazon’s Elastic Block Store (EBS).

It was an impressive broadcast, accompanied by a live demo on EC2.

One of the things I found impressive is the ability to mount an "EBS volume" in the RightScale dashboard, and interact with it almost as if it was your local C: or D: drives.
I'm also excited by the way RightScale combined EBS with MySQL, explained in their recent EBS post.
This is especially important to me as I can say by now, after speaking to many GigaSpaces' customers on EC2, that MySQL is the most popular database in the cloud.

I'm looking forward to exploring how RightScale EBS support can work with GigaSpaces persistency as a service and EC2 support. I believe this can provide our joint customers with the best experience out-there today when deploying and scaling data-intensive, transactional applications on the cloud.

I agree with Thorsten that EBS really brings a new era to the Amazon cloud, especially after our own somewhat questionable experience with S3.

Kudos to the RightScale folks for pulling this event together only a few days after Amazon announced EBS.