Digital Walkabout

Intel and EMC Atmos for High Performance Storage Clouds

Intel announced that it's working with EMC to offer a very low power hardware-and-software building block for those looking to float Private Storage Clouds, slated for debut in the second half of next year. The product will be based on EMC's Atmos software platform, but Intel VP of End User Platform Integration Prasad Rampalli says that with certain hardware and software optimizations, they've dropped power levels by 15 per cent "at the rack level" - without losing performance.

"How do we ensure that when you run a specific application on a specific server, it's using just the amount of resources needed to consume the lowest amount of power? We believe that it requires software like what EMC has with Atmos that works harmoniously with what the hardware provides," Intel's Prasad said. "What we're doing is giving EMC's software more knobs and dials for them to use."

According to Prasad, these knobs and dials can save you $1.5m across a 100,000-server data center. Over the next six months, Intel and EMC will pilot their hardware-and-software combo with select users.

Amazon Relational Database Service (Amazon RDS)

Amazon Web Services is releasing Relational Database as a Service (RDS) - whilst AWS has a storage cloud (S3), Compute Cloud (EC2) and a non relational Database Cloud (SimpleDB), until now, they have not offered Relational Database capabilities as a service. This has been one of the greatest demands by AWS users - given this and competition from Microsoft's SSDS, they have delivered. 

Amazon RDS is but a MySQL 5.1 database instance that is exclusively for a particular user and can be accessed via a single API call. The user gets all the capabilities of MySQL database with an additional ability to scale up based on the needs. It rids the customers of any need for time consuming database administration tasks. The patches are applied automatically with the database backed up automatically with a possibility for the user to set the retention period. 

Amazon RDS is priced similar to Amazon EC2. There are costs involved with usage of database instances, persistent storage up to 1 TB, costs for I/O requests, costs for backup storage and data transfer costs. The smallest instance costs 11 cents per hour and one can scale up to quadruple extra large instance of 68 GB of memory, 26 ECUs (8 virtual cores with 3.25 ECUs each), 64-bit platform. 

After resisting the demands for a relational database for a long time, Amazon has suddenly jumped into the game. It is suspected that Amazon RDS seeks to preempt Microsoft's announcement about the public release of the Microsoft Azure SSDS later this month at PDC '09. For some time, Amazon has not been keen to releasing Windows based instances as a part of their EC2 offering; when it became clear that Microsoft will announce their Public Cloud "OS" at PDC '08, Amazon pre-empted the announcement with their own Windows based EC2 instances.

The announcement will also pressure the Y-Combinator startup FathomDB, which offers Database as a Service on Amazon EC2. Amazon RDS should also serve as a klaxon for companies building their entire business upon gaps in the Amazon ecosystem. It is interesting to note that this is not unique to the Amazon ecosystem alone, but can happen to any vendor whose offerings rely entirely on one vendor's ecosystem.

Moire on Amazon RDS:

 http://aws.amazon.com/rds/

“Amazon RDS gives you access to the full capabilities of a familiar MySQL database. This means the code, applications, and tools you already use today with your existing MySQL databases work seamlessly with Amazon RDS. Amazon RDS automatically patches the database software and backs up your database, storing the backups for a user-defined retention period. You also benefit from the flexibility of being able to scale the compute resources or storage capacity associated with your relational database instance via a single API call. As with all Amazon Web Services, there are no up-front investments required, and you pay only for the resources you use.”

http://aws.typepad.com/aws/2009/10/introducing-rds-the-amazon-relational-database-service-.html

“RDS usage is charged by the DB Instance hour. As noted above, there are five instance sizes and corresponding hourly rates. You'll also pay 10 cents per GB per month for your provisioned storage and 10 cents for every million I/O requests. You get backup space to store 100% of your provisioned storage at no additional charge, with additional space priced at 15 cents per GB per month. The usual AWS charges for data transferred in and out of the cloud also apply.”

http://blog.rightscale.com/2009/10/26/amazon-relational-database-service/

“What AWS does is keep your RDS instance backed up and running, plus give you automation to up-size (and down-size). You can create snapshot backups on-demand from which you can launch other RDS instances and AWS automatically performs a nightly backup and keeps transaction logs that allow you to do a point-in-time restore.

The way I think of an RDS instance is as a virtual appliance or a special-purpose server. You really get an EC2 instance with an EBS volume running a specific version of MySQL plus automation for backups and resizing the storage volume. The API is designed such that additional versions of MySQL and other databases can easily be added in the future. Just like a regular server, each RDS instance lives within an availability zone and access is controlled through a security group (plus the mysql authentication). I haven’t had the opportunity to run some performance tests, but I would surmise that it’s not too different from DIY running mysql on a regular instance.”

Gartner Identifies the Top 10 Strategic Technologies for 2010

Analysts Examine Latest Industry Trends During Gartner Symposium/ITxpo, October 18-22, in Orlando

ORLANDO, Fla., October 20, 2009 — 

Gartner, Inc. analysts today highlighted the top 10 technologies and trends that will be strategic for most organizations in 2010. The analysts presented their findings during Gartner Symposium/ITxpo, being held here through October 22.

Gartner defines a strategic technology as one with the potential for significant impact on the enterprise in the next three years. Factors that denote significant impact include a high potential for disruption to IT or the business, the need for a major dollar investment, or the risk of being late to adopt.

These technologies impact the organization's long-term plans, programs and initiatives. They may be strategic because they have matured to broad market use or because they enable strategic advantage from early adoption.

“Companies should factor the top 10 technologies into their strategic planning process by asking key questions and making deliberate decisions about them during the next two years,” said David Cearley, vice president and distinguished analyst at Gartner. “However, this does not necessarily mean adoption and investment in all of the technologies. They should determine which technologies will help and transform their individual business initiatives.”

The top 10 strategic technologies for 2010 include:

Cloud Computing. Cloud computing is a style of computing that characterizes a model in which providers deliver a variety of IT-enabled capabilities to consumers. Cloud-based services can be exploited in a variety of ways to develop an application or a solution. Using cloud resources does not eliminate the costs of IT solutions, but does re-arrange some and reduce others. In addition, consuming cloud services enterprises will increasingly act as cloud providers and deliver application, information or business process services to customers and business partners.

Advanced Analytics. Optimization and simulation is using analytical tools and models to maximize business process and decision effectiveness by examining alternative outcomes and scenarios, before, during and after process implementation and execution. This can be viewed as a third step in supporting operational business decisions. Fixed rules and prepared policies gave way to more informed decisions powered by the right information delivered at the right time, whether through customer relationship management (CRM) or enterprise resource planning (ERP) or other applications. The new step is to provide simulation, prediction, optimization and other analytics, not simply information, to empower even more decision flexibility at the time and place of every business process action. The new step looks into the future, predicting what can or will happen.

Client Computing. Virtualization is bringing new ways of packaging client computing applications and capabilities. As a result, the choice of a particular PC hardware platform, and eventually the OS platform, becomes less critical. Enterprises should proactively build a five to eight year strategic client computing roadmap outlining an approach to device standards, ownership and support; operating system and application selection, deployment and update; and management and security plans to manage diversity.

IT for Green. IT can enable many green initiatives. The use of IT, particularly among the white collar staff, can greatly enhance an enterprise’s green credentials. Common green initiatives include the use of e-documents, reducing travel and teleworking. IT can also provide the analytic tools that others in the enterprise may use to reduce energy consumption in the transportation of goods or other carbon management activities.

Reshaping the Data Center. In the past, design principles for data centers were simple: Figure out what you have, estimate growth for 15 to 20 years, then build to suit. Newly-built data centers often opened with huge areas of white floor space, fully powered and backed by a uninterruptible power supply (UPS), water-and air-cooled and mostly empty. However, costs are actually lower if enterprises adopt a pod-based approach to data center construction and expansion. If 9,000 square feet is expected to be needed during the life of a data center, then design the site to support it, but only build what’s needed for five to seven years. Cutting operating expenses, which are a nontrivial part of the overall IT spend for most clients, frees up money to apply to other projects or investments either in IT or in the business itself.

Social Computing. Workers do not want two distinct environments to support their work – one for their own work products (whether personal or group) and another for accessing “external” information. Enterprises must focus both on use of social software and social media in the enterprise and participation and integration with externally facing enterprise-sponsored and public communities. Do not ignore the role of the social profile to bring communities together.

Security – Activity Monitoring. Traditionally, security has focused on putting up a perimeter fence to keep others out, but it has evolved to monitoring activities and identifying patterns that would have been missed before. Information security professionals face the challenge of detecting malicious activity in a constant stream of discrete events that are usually associated with an authorized user and are generated from multiple network, system and application sources. At the same time, security departments are facing increasing demands for ever-greater log analysis and reporting to support audit requirements. A variety of complimentary (and sometimes overlapping) monitoring and analysis tools help enterprises better detect and investigate suspicious activity – often with real-time alerting or transaction intervention. By understanding the strengths and weaknesses of these tools, enterprises can better understand how to use them to defend the enterprise and meet audit requirements.

Flash Memory. Flash memory is not new, but it is moving up to a new tier in the storage echelon. Flash memory is a semiconductor memory device, familiar from its use in USB memory sticks and digital camera cards. It is much faster than rotating disk, but considerably more expensive, however this differential is shrinking. At the rate of price declines, the technology will enjoy more than a 100 percent compound annual growth rate during the new few years and become strategic in many IT areas including consumer devices, entertainment equipment and other embedded IT systems. In addition, it offers a new layer of the storage hierarchy in servers and client computers that has key advantages including space, heat, performance and ruggedness.

Virtualization for Availability. Virtualization has been on the list of top strategic technologies in previous years. It is on the list this year because Gartner emphases new elements such as live migration for availability that have longer term implications. Live migration is the movement of a running virtual machine (VM), while its operating system and other software continue to execute as if they remained on the original physical server. This takes place by replicating the state of physical memory between the source and destination VMs, then, at some instant in time, one instruction finishes execution on the source machine and the next instruction begins on the destination machine.

However, if replication of memory continues indefinitely, but execution of instructions remains on the source VM, and then the source VM fails the next instruction would now place on the destination machine. If the destination VM were to fail, just pick a new destination to start the indefinite migration, thus making very high availability possible. 

The key value proposition is to displace a variety of separate mechanisms with a single “dial” that can be set to any level of availability from baseline to fault tolerance, all using a common mechanism and permitting the settings to be changed rapidly as needed. Expensive high-reliability hardware, with fail-over cluster software and perhaps even fault-tolerant hardware could be dispensed with, but still meet availability needs. This is key to cutting costs, lowering complexity, as well as increasing agility as needs shift.

Mobile Applications. By year-end 2010, 1.2 billion people will carry handsets capable of rich, mobile commerce providing a rich environment for the convergence of mobility and the Web. There are already many thousands of applications for platforms such as the Apple iPhone, in spite of the limited market and need for unique coding. It may take a newer version that is designed to flexibly operate on both full PC and miniature systems, but if the operating system interface and processor architecture were identical, that enabling factor would create a huge turn upwards in mobile application availability.

“This list should be used as a starting point and companies should adjust their list based on their industry, unique business needs and technology adoption mode,” said Carl Claunch, vice president and distinguished analyst at Gartner. “When determining what may be right for each company, the decision may not have anything to do with a particular technology. In other cases, it will be to continue investing in the technology at the current rate. In still other cases, the decision may be to test/pilot or more aggressively adopt/deploy the technology.”

Why Microsoft sees its future in [Rich Services] Cloud

From health care systems to cellphones, Steve Ballmer wants Microsoft "to invent everything that's important on the planet."
http://www.nytimes.com/2009/10/18/business/18msft.html?th&emc=th

Microsoft is very much aware that they are facing a present where the traditional desktop OS and traditional thick desktop apps (i.e. Office) are being squeezed and a future where these lines will have less relevance. Without a path forward -  MS Online (the first full Cloud app suite for SMB/Enterprise - essentially MS Office in the Cloud with Rich Services clients) and to make "Cloud run better on Win7" - Microsoft is looking at a bleak future with diminishing revenue streams. 

The greatest threat the Cloud poses is not to rich client hardware - there will continue to be a solid place for notebooks and high-powered mobiles (e.g, ultraportables, which differ from netbooks in that they are designed to provide top-end-quality richness in a netbook sized form factor). Yet whilst the new world of Cloud is opening new opportunities for HW manufacturers and OEMs alike, it is making the game harder to play for the old school OS and desktop apps. Users want the agility that Rich Services Cloud provides - any device, any where - rather than the binding client-server model that traditional desktop apps provide. Likewise, IT wants to leverage the centralised management and administration of Rich Services Cloud; continued cost pressure is forcing them to abandon the high-end maintenance model associated with traditional OS and desktop apps.  

Microsoft, if it wants to maintain shareholder value and secure a place in this new world, has to adapt to the reality of the Rich Services Cloud, and both Steve Ballmer and Ray Ozzie have recognised this. They have bet Microsoft's future on the relevance of Rich Services Cloud. Whilst Microsoft will look at a diminution of their revenue streams from the "cash cow" days of Windows OS and Microsoft Office, their role in a world that includes KVM-type platforms (such as the WinXP KVM in Win7) and Rich Services Cloud apps (such as MS Online) will continue to have relevance. In embracing further value-added apps that utilise the Rich Services Cloud model, Microsoft may find its next "cash cow" in vertical suites. Whether this works for the depends upon whether they can make a splash early on with MS Online, and show that it can engender innovation in office productivity (i.e. compete very favourably with Google Apps), whilst providing the better usage models and user experiences. 

Why Microsoft/Danger represents Traditional Browser-based Cloud #FAIL

The very public and spectacular Microsoft/Danger/Sidekick failure has been a rallying cry for those who detract as well as over-simplify Cloud Computing. A large part of the discussion has focussed on the fragility of CC services and storage, and the over-hying of CC. However, this reaction is itself anti-CC hype, failing to truly understand the implications of the failure. A recent blog entry from Dan Dilger provides a reasoned analysis that should provide a cautionary tale against:

• Assuming "if you build it, they will come" (referring to wishful-thinking RAS), 
• Shrug-and-grin, fluff SLAs (referring to "SLAs" which essentially state "caveat emptor" and "we are not responsible [for anything]")
• Traditional Browser-based CC is sufficient (with no viable service redundancy, which would increase DC costs considerably, expecting "always connected" modes for basic functionality makes little sense. 

The T-Mobile Sidekick interacts with the Danger cloud services Microsoft operates in the same manner as a Traditional Browser-based Cloud app. It expects to always be connected to Danger's Cloud, so the device does not durably cache or otherwise store any of your data locally. When the device is power-cycled, the database is with an empty database and reloads all data from the service when it is able to reconnect.

Dilger uses iPhone as a foil for the Sidekick. The iPhone represents the opposite extreme - the device that utilises a Client-Installed, Network-Accessed Cloud application - ITunes - and the MobileMe Cloud service to sync data between the MID, a netbook/notebook/desktop device, and the Cloud. Data is durably cached and stored on the iPhone, which together with the local storage repo (the netbook/notebook/desktop device storing a copy of your content) and the Cloud, provides not only the ability to use the iPhone disconnected, but an additional level of redundancy. 

I see a better path which lies between these two extremes. This path involves more efficient utilisation of client resources and SLAs that can be taken seriously. The former would not have every install of iTunes redundantly storing all music files on the clients, rather targeting each netbook, notebook, or desktop for a level of storage, or allowing one to treat a series of their personal clients as an on-premise cloud. Likewise, content can be streamed from one's nearest storage rather than locally stored. Synchronising every file across each device is not the most efficient model. Yet some local storage and processing is necessary to make the content useful, since devices are never always connected, and the quality of that connection and location vary widely. 

The latter would provide for a level of redundancy and data protection that users have come to expect from CC. Although there are a number who would attest that they "trust" providers with 2 9s reliability, or that they are comfortable with providers taking liberties with their data as they are providing them with a "service", the same people react more viscerally to outages and failures, painting these as a CC #FAIL. The both of these extremes are daft: 2 9s reliability leaves one's data RAS to the winds of fortune, and the failure of 2 9s Cloud services does not equate CC #FAIL. Users of CC need to take it upon themselves to understand their providers' SLAs and, rather than wait for a failure to occur, either demand greater rigour or move to services which do. The continued exodus from Flickr is an example of this. If the "service" has no responsibility for what's stored therein, and anyone can delete users' data, what is the use of the "service"? Sharing must enjoy a degree of reliability, otherwise, the "service" offers nothing substantial that provides an advantage over traditional sharing forms. 

Rather than ignoring the Microsoft/Danger #FAIL, or using it to detract from CC as a whole, I see the failure as exposing the fallacy of Traditional Browser-based thin architectures, and calling for greater balance between devices and the DC for CC. 

Cisco Teaches Routers to Act Like Servers

An interesting development for branch offices. In addition to UCS for the DC, Cisco is enabling creation of general Linux/Java apps to run in an embedded server in their routers.

Here are the details on the development environment (Eclipse based).

http://www.cisco.com/en/US/prod/collateral/routers/ps9701/data_sheet_c02_459075.html

Here is one of the prize winners:

"The second-place contestant, Rajesh Kotagiri of India, won $30,000 for devising a system for sending ads to electronic displays in stores. The idea there is that a store already has networking equipment to support things like computers, cash registers and phones. So, why not turn one of the routers into an ad-serving dynamo that can retrieve information from advertisers, handle some limited billing operations and send the ads out to displays on a rotating basis?"

 

Cisco Teaches Routers to Act Like Servers

By Ashlee Vance

"Cisco Systems has found a way to accelerate its push into the server market. The company is simply turning many of the millions of routers it has already sold into servers.

Typically, a router sits in a data center or communications closet somewhere just pushing around data packets. It’s sort of like a cop being punished with traffic duty. Meanwhile, the servers — the detectives, if you will — are back in the office working on grander things.

Well, Cisco has been hammering away on something it calls the Application Extension Platform, or AXP, that it thinks can change this relationship between routers and servers.

At its core, the AXP gives software developers a way to write applications that run on an additional bit of hardware that sits alongside Cisco’s routers. In many cases, the applications written for the router can take on jobs typically handled by a small server sold by companies like I.B.M., Hewlett-Packard and Dell.

To drum up interest in AXP, Cisco held a contest for developers who could write the most compelling, router-friendly software. And, this week, Cisco officially began doling out cash prizes to the three big winners.

The third-place contestant, Bernard Beckmann of Germany, who won $20,000, found a way to turn VoIP phones connected to Cisco’s networking gear into office surveillance systems. The speakers on the phones were essentially made to act like microphones that would try to sense unusual noises during off-peak office hours. If the phone picked up on something strange, it would send an alert to security personnel.

The second-place contestant, Rajesh Kotagiri of India, won $30,000 for devising a system for sending ads to electronic displays in stores. The idea there is that a store already has networking equipment to support things like computers, cash registers and phones. So, why not turn one of the routers into an ad-serving dynamo that can retrieve information from advertisers, handle some limited billing operations and send the ads out to displays on a rotating basis?

The first-place winner, David Perez of Spain, picked up $50,000 for his efforts. He concocted a way to pull data from a building’s plumbing, air-conditioning and lighting systems and funnel all of it through a router. Basically, this would let maintenance people locate problems in a building and also provide a way to do things like lowering shades automatically during certain parts of the day or turning off signs to save energy.

The breadth of work done by the flood of contestants who entered Cisco’s program was fairly impressive given some of the AXP’s limitations. For one, Cisco’s hardware runs on low-power PC chips instead of proper server chips. And the software development platform has been limited to Linux this far.

But upgrades are coming soon. Cisco plans to begin slotting standard x86 chips into its routers and to open AXP to Microsoft’s Windows, said Shashi Kiran, a senior manager at Cisco. At that point, the line between a router and a server will really start to blur.

“It is a logical progression for us,” Mr. Kiran said.

Uh-huh.

While routers will not come close to replacing servers en masse anytime soon, Mr. Kiran argues that the developer contest provided a nice insight into what can be done. Cisco plans to keep investing in its developer efforts and has basically given H.P., I.B.M. and Dell the wink-and-nod that it means business.

Cisco, of course, began selling a true server earlier this year, putting it in direct competition with a host of companies that had long been its partners in the data center." 

WWhat are the hardware specifications of the AXP service module?

A. Table 2 lists hardware specifications.

Table 2. Hardware Specifications

Service Module	CPU	RAM	Storage
AIM-102	300-Mhz Intel Celeron	256 MB	1-GB Flash
NME-302	1.0-GHz Intel Pentium	512 MB	80-GB hard drive
NME-522	1.4-GHz Intel Pentium	2 GB	160-GB hard drive

http://www.cisco.com/en/US/prod/collateral/routers/ps9701/qa_c67_463943.html

US CIA endorses Private Cloud

http://www.computerworld.com/s/article/9139016/CIA_endorses_cloud_computing_but_only_internally

 

One of the U.S. government's strongest advocates of cloud computing is also one of its most secretive operations: the Central Intelligence Agency. The CIA has adopted cloud computing in a big way, and the agency believes that the cloud approach makes IT environments more flexible and secure.

 

Jill Tummler Singer, the CIA's deputy CIO, said that she sees enormous benefits to a cloud approach. And while the CIA has been moving steadily to build a cloud-friendly infrastructure -- it has adopted virtualization, among other things -- cloud computing is still a relatively new idea among federal agencies.

 

"Cloud computing as a term really didn't hit our vocabulary until a year ago," said Singer.

 

But now that the CIA is building an internal cloud, Singer sees numerous benefits. For example, a cloud approach could bolster security, in part, because it entails the use of a standards-based environment that reduces complexity and allows faster deployment of patches.

 

"By keeping the cloud inside your firewalls, you can focus your strongest intrusion-detection and -prevention sensors on your perimeter, thus gaining significant advantage over the most common attack vector, the Internet," said Singer.

 

Moreover, everything in a cloud environment is built on common approaches. That includes security, meaning there's a "consistent approach to assuring the identity, the access and the audit of individuals and systems," said Singer. But there are limits. The agency isn't using a Google model and "striking" data across all its servers; instead, data is kept in private enclaves protected by encryption, security and audits.

 

The CIA uses mostly Web-based applications and thin clients, reducing the need to administer and secure individual workstations. And it has virtualized storage, protecting itself "against a physical intruder that might be intent on taking your server or your equipment out of the data center," said Singer.

 

Speaking at Sys-Con Media's GovIT Expo conference today, Singer not only provided a rare glimpse into the IT approaches used by the agency, but also talked about one of its greatest challenges: the cultural change cloud environments bring to IT. A move to cloud environments "does engender and produce very real human fear that 'I'm going to lose my job,'" she said.

 

In practice, highly virtualized environments reduce the need for hardware administration and, consequently, for system administrators. Barry Lynn, the chairman and CEO of cloud computing provider 3tera Inc. in Aliso Viejo, Calif., said a typical environment may have one systems administrator for every 75 physical servers. In contrast, a cloud-based environment may have just one administrator for every 500 servers or more.

 

The CIA has "seen a significant amount of pushback, slow-rolling [and] big-process engineering efforts to try to build another human-intensive process on top of enterprise cloud computing," said Singer. "It will take us a good long while to break that."

 

One thing the agency will do to address resistance will be to base contract competitions on performance, not head count, "where it's to [a service provider's] benefit to do the work with fewer bodies and make more profit for their company," said Singer.

 

Federal CIO Vivek Kundra is encouraging agencies to adopt cloud computing, and he recently opened an online apps store that enables federal agencies to buy cloud-based services from Google, Salesforce.com and other vendors. That's something the CIA will not do; its data will remain within the agency's firewalls, said Singer.

 

Government market research firm Input has revised its forecast for federal cloud-related spending upward; it now expects the government's cloud expenditures to grow from $363 million this year to $1.2 billion by 2014. "I think this is probably a conservative estimate, considering the push from the administration," said Deniece Peterson, an analyst at Reston, Va.-based Input.

 

Obstacles to the adoption of cloud computing, including concerns about security and loss of data control, may slow momentum, but "I think we'll see broader adoption and higher spending after the administration makes progress in some of the pilot programs it has planned," said Peterson.

 

Singer said the CIA's IT department was moving in the direction of cloud computing, even if it wasn't using that term, when it widely deployed virtualization technology. Abstracting the operating system and software from the hardware "is the foundation of the cloud," Singer said. "We were headed to an enterprise cloud all along."

Cloud Security issues get real

http://www.theregister.co.uk/2009/10/05/amazon_bitbucket_outage/

DDoS attack rains down on Amazon cloud
Code haven tumbles from sky

"Updated Web-based code hosting service Bitbucket experienced more than 19 hours of downtime over the weekend after an apparent DDoS attack on the sky-high compute infrastructure it rents from Amazon.com.

This in turn left many developers without access to code projects hosted on Bitbucket, a GitHub-like service based on the Mercurial version control system.

"Looks like all (my and a large number of fellow nerds) bitbucket projects have evaporated in a temporary, cloudy way. This is a major pissoff," said one Reg reader and Bitbucket user, as others vented via Twitter.

But on another level, the news is sure to fuel fears over the security of Amazon's Elastic Compute Cloud (EC2) and similar "infrastructure clouds," online services that provide grid-like access to scalable processing, storage, and networking resources.

"The lesson here is: 'Don’t bet the farm on a single cloud provider,'" says Craig Balding, founder of cloudsecurity.org and a security practitioner at a Fortune 500 company. "It's common sense really. But people get lulled into thinking they site is always going to be available [when they host with a single provider]."

According to a blog post from Jesper Nøhr, the Danish developer who runs Bitbucket.org, the site's Amazon-hosted network storage became "virtually unavailable" beginning Friday evening, and the outage persisted well into Saturday before Amazon pinpointed the problem.

Nøhr says Amazon advised him not to divulge the cause of the outage. But he divulged anyway. "We were attacked. Bigtime. We had a massive flood of UDP [User Datagram Protocol] packets coming in to our IP, basically eating away all bandwidth to the box," he wrote. "So, basically a massive-scale DDOS. That’s nice."

Speaking with The Reg, Nøhr said that Amazon urged him not to reveal the attack because it might help attackers develop new ways of DDoSing the site.

After uncovering the problem - at least 16 hours after it was first reported - Amazon blocked the offending traffic, and service returned to normal. But by the next morning (Sunday), the problem returned, and another two hours passed before this second outage was reversed. According to Nøhr, Amazon told him the second attack used a flood of TCP SYN connection requests, rather than UDP packets.

Then, it seems, a third attack arrived. Nøhr tells The Reg that an attack on an Amazon edge router took out service for some but not all Bitbucket customers for close to one and a half hours earlier today.

For Nøhr and other Bitbucket devotees, it seems odd that traffic from the net at large could bring down what should be "internal" storage resources. Nøhr speculates that Bitbucket's storage sits on the same network interface that connects the site to the outside world. He asks why the storage isn't on a separate channel - and why Amazon doesn't have methods in place to rapidly detect and combat such DDoS attacks.

"I do think they could’ve taken precautions to at least be warned if one of their routers started pumping through millions of bogus UDP packets to one IP," he wrote, "and I also think that 16+ hours is too long to discover the root of the problem."

Cloudsecurity.org's Balding is equally surprised that an outside attack could somehow "get between" EC2 and EBS. But since Amazon treats its service as a black box, he says, it's difficult to tell what actually occurred. He says it's possible that the attack came from inside EC2 - i.e. from another EC2 customer - but this is unlikely. "You'd think that Amazon could have shut down that sort of thing pretty quickly," he tells The Reg.

Amazon did not immediately respond to a request for comment, but we made contact before Pacific Coast office hours. We will update this story when the company responds.

In a security white paper (pdf) dated September 2008, the company says it uses standard DDoS-fighting techniques such as syn cookies and connection limits. It also says: "To further mitigate the effect of potential DDoS attacks, Amazon maintains internal bandwidth which exceeds its provider-supplied Internet bandwidth."

Bitbucket runs its entire site on Amazon's Elastic Compute Cloud, using the company's Elastic Block Store (EBS) for storing its database, log files, user data, and more. EBS provides persistent storage for EC2 server instances.

On Friday evening, Jesper Nøhr and Bitbucket told Amazon that there appeared to be a serious slowdown in the transfer of data to and from its Elastic Block Store. Nøhr said the site was "getting less throughput than you can pull off of a 1.44MB floppy".

But when Nøhr first reported the problem, an Amazon support rep blamed it on that fact that EBS is a shared resource, saying that performance could vary. But after Nøhr made another call, a second rep acknowledged that the problem went beyond the usual performance hiccups.

"We had been extremely frustrated up until this point, because 1) we couldn’t actually *do* anything about it, and 2) we were being told that everything should be fine," Nøhr wrote. "It felt like there was an elephant right in front of us, and a person next to us was insisting that there wasn’t."

Eventually, Nøhr said, Amazon gave the problem the attention it deserved. "After a bit of stalling with their first rep, our case received absolutely stellar attention. They were very professional in their correspondence, and in communicating things to us along the way."

Nøhr says that Amazon wants to work with him to ensure this sort of attack doesn't happen in the future, but he's now considering a switch to another hosting provider.

"We’re seriously considering moving to a different setup now," he said. "Not because Amazon isn’t providing us with decent service, which they are, most of the time. While we were down, several large hosting companies took direct contact with us, pitching their solutions. I won’t mention names, but some of the offerings are quite tempting, and have several advantages over what we get with Amazon.

"One thing’s for sure, we’re investing a lot of man-hours into making sure this won’t happen again. If this means moving to a different host, so be it. We haven’t decided yet."

According to Nøhr, the DDoS traffic was spoofed, so it's unclear where it originated. Nøhr speculates that attackers were targeting a Bitbucket-hosted project "related to" World of Warcraft, but he declined to name the service."

Amazon's Virtual Private Cloud - Will it be Private enough?

Amazon just announced the limited beta of Amazon Virtual Private Cloud (Amazon VPC), a secure and seamless bridge between existing IT infrastructures and the AWS cloud. Amazon VPC enables us to connect our existing infrastructure to a set of isolated AWS compute resources via a Virtual Private Network (VPN) connection.

“Amazon VPC enables you to use your own isolated resources within the AWS cloud, and then connect those resources directly to your own datacenter using industry-standard encrypted IPsec VPN connections. With Amazon VPC, you can:

• Create a Virtual Private Cloud on AWS’s scalable infrastructure, and specify its private IP address range from any block you choose.
• Divide your VPC’s private IP address range into one or more subnets in a manner convenient for managing applications and services you run in your VPC.
• Bridge together your VPC and your IT infrastructure via an encrypted VPN connection.
• Add AWS resources, such as Amazon EC2 instances, to your VPC.
• Route traffic between your VPC and the Internet over the VPN connection so that it can be examined by your existing security and networking assets before heading to the public Internet.
• Extend your existing security and management policies within your IT infrastructure to your VPC as if they were running within your infrastructure.”

Besides the standard EC2 rates, you will need to cover the VPN connection ($0.05 per VPN Connection-hour) and data transfer through the VPN tunnel ($0.10 per GB IN – and starting with $0.17 per GB OUT).

Amazon is trying to get a jump on it's competitors in the enterprise space. While many enterprises have used Amazon Web Services, most perceive it as being insufficiently secure for important or confidential data, and too nuts-and-bolts to provide the economies promised by Platform As A Service. Microsoft, IBM and Rackspace are trying to find the right mix of scale and security for enterprise clients: With Azure, Microsoft is building its own platform and infrastructure-as-a-service offering called Azure; IBM is creating several gradations of a hybrid cloud, from private cloud infrastructure deployed inside a corporation’s own data center, to services delivered from Big Blue’s DCs; and Rackspace is hoping security-minded customers use its dedicated hosting that can scale up to the Rackspace cloud.

Michael Crandell, CEO of Rightscale tried to explain a bit more what Amazon is trying to do with VPC:

"Something that initially puzzled me is what the benefits of a VPC are when all the marketing fluff dissipates. Here is what I’ve learned. First, instances in the VPC are separated from non-VPC instances at a deeper network level than instances in different security groups or belonging to different users. As is typical, Amazon doesn’t say anything of substance about the nature of this isolation. Let’s see how soon that will have to change to actually attract enterprises…Second, instances in the VPC can seamlessly integrate into a company’s internal network routing. This is significant because it means that tools used to inventory, secure, audit, manage and access all servers in the IT infrastructure can now be brought to bear on instances in the cloud as well."

The Amazon offering is different from IBM and Microsoft’s efforts in that it provides access to the raw infrastructure, rather than PaaS, which both Microsoft and IBM are betting heavily upon. In the next few years, as more enterprise compute will shift to some of these companies, Amazon apparently believes it will remain relevant and can get a slice of that enterprise pie through VPC. The question will be whether or not businesses find Amazon's VPC private enough.

In any case, this is an interesting direction for sharing resources between the DC and the Public Cloud. I will be rather interested in what Eucalyptus, Nagius/Ganglia do with this to better enable the Private Cloud, which would enable AWS in a sense to compete more directly with Microsoft and IBM.