Digital Walkabout

Gold-Bernstein on SOA & ITIL

I found Gold-Bernstein's blog entry on SOA & ITIL very insightful and a good read:

"Tony Baer presented his research on the connection between SOA and ITIL to the SOA Consortium. The inspiration for Tony's research was a concern over the growing numbers of governance silos that were becoming apparent. After being briefed by a vendor who announced an SOA life cycle management platform Tony immediately recognized the overlap across numerous other tools. Tony concluded that there was significant similarities between an SOA Service life cycle and the ITIL Service Management Lifecycle and considered how exchange of information between different groups at specific points in the life cycle could help integrate the silos. 
 
Tony focused on run-time SOA governance, because he said that is where you see all the issues. He gave an example for the interaction between development and IT operations where the development group monitors and manages SOA at runtime and IT operations monitors and manages the underlying infrastructure. When an SLA issue arises with the SOA service, development analyzes the problem as the monitoring system automatically submits a trouble ticket to the service desk. IT operations then applies incident, problem, and change management processes (all defined in ITIL) if the problem is attributed to infrastructure.

Todd Biske was on the call, and talked about the role of a Service Manager. Tony enthusiastically embraced the concept as a way of assigning clear responsibility for life cycle management of each service.

I was very interested to hear what Tony and others on the call would have to say about ITIL and SOA because I believe that governance is key to SOA success. One of the most oft touted benefits of SOA is that it creates business alignment. However, there is nothing in the SOA concepts and precepts, including loose coupling, layers of abstraction through composite services, reuse, that actually delivers IT and business alignment. Governance does that.

While Tony focused on run time governance, I feel that the governance process must start at the very beginning. In fact, ITIL has some excellent processes for determine which IT projects to invest in, and track the value of IT investments to the business. ITIL also includes the definition of roles and responsibilities. IMHO this is crucial to SOA governance. A single service may be used by different parts of the organization. Policies governing that service may be defined by different people in the organization. While it all hits the fan at run time, defining this much earlier in the process using ITIL processes and best practices, will make run time much more manageable. Once you know who is responsible for what (which I'm afraid is a far more complex issue than just having a service manager for each service) then the technology is available to send alerts to the right person at the right level, or to trigger an automated process to remediate the problem.

Another big issue, not currently part of ITIL but which can be part of the Enterprise Architecture framework, is a data architecture for defining the policies and procedures for repository integration. It is clear that multiple repositories will remain. There is the LDAP repository for security information, the CMDB defined in ITIL for configuration management, the SOA registry/repository, and the runtime repository, and probably other home grown metadata repositories. Each of these repositories is the authoritative source for some information, and probably also duplicates information in other repositories. The problem with redundant data is when it is out of synch. Integration technologies and techniques make it possible to automate the synchronization of information across repositories. But unfortunately different parts of the organization own the different repositories and do not usually work together to ensure consistency.

IMHO what is needed is not a handoff of information during runtime, but an overall methodology, including processes, roles and responsibilities, best practices, metrics, and governors, to ensure the business maximizes the value of IT investments, and IT can manage the software and infrastructure in a cost effective and agile manner. ITIL also has frameworks for defining who gets value of out of services, which can help identify how service development is paid for.

I view ITIL as a good overall framework for finding best practices on managing IT assets to maximize business value. This is what will create IT and business alignment. Organizations generally choose a subset of ITIL that will work within their organization. These ITIL processes should be full integrated with enterprise architecture processes. SOA should be considered a subset of the overall enterprise architecture and not a separate new effort. In other words, rather than an interaction between ITIL processes and SOA processes, I am suggesting a full integration of portfolio management, enterprise architecture and SOA within the organization, through an overall governance framework. In fact,I would go as far as to say that governance silos are a recipe for failure, and think this is what we are seeing with the current state of SOA."

You can read this entry in full here.

Major Shift to Cloud "Inevitable"

Per IDC, the shift to CC is "inevitable", driven by the promise of reduced TCO in a recession and "the ease and speed of deployment". Anyone involved in CC has realised this by now, but the use of "inevitable" is a first from the analysts, to my experience. 

What is also remarkable about this entry is the confused definition of CC. Indeed, CC overlaps some of the concepts of distributed, grid computing and virtualization, and whilst there is an ongoing convergence of CC and virtualization capabilities, the two are quite different. CC is a broader concept than virtualization and relates to the underlying architecture in which services are designed. 

When one really thinks about it, CC is In essence distributed computing. An application is built using the resource from multiple services potentially from multiple locations - accessing resources and services needed to perform functions with dynamically changing needs. An application or service developer requests access from the cloud rather than a specific endpoint or named resource. The underlying hardware and software of networking is of course still there but there are now higher level service capabilities available used to build applications. Behind the services are data and compute resources. A user of the service doesn’t necessarily care about how it is implemented, what technologies are used or how it’s managed - only that there is access to it and has a level of reliability necessary to meet the application requirements. 

CC manages multiple infrastructures across multiple organizations and consists of one or more frameworks overlaid on top of the infrastructures tying them together. This is an abstraction of resources that maintains and manages itself - of course, there are traditional IT resources to keep hardware, operation systems and networking in proper order - from the perspective of a user or application developer, though, only the Cloud is referenced.

 

Microsoft's Future in the Cloud

http://bits.blogs.nytimes.com/2009/03/20/steve-ballmer-maps-microsofts-cloud
-y-future/

It is interesting what Ballmer states around the local applications emphasizing that even Microsoft sees the need for Universal Client computing - 1:1 User/App experience regardless of location, connection or hardware.  His comments about it not being about just Apps in the browser is spot on and reiterated by some of the comments to this blog.

The following, based on the BMC/Cisco announcement, is interesting as well:
http://cloudstoragestrategy.com/2009/03/taming-the-cloud-cisco-bmc-software.html

Notice the emphasis on the missing piece - management.  The common thread that many of the comments had was compliance, privacy and security.  There are growing concerns and rallys in the market on how things like HIPAA - or Personal Health Information Acts in EMEA/ASIA, SOX, Federal Desktop Core Configuration Guidelines, and new statements about oversight are going to both drive and hinder the Cloud.

In order to do it right - more around virtualisation and dynamic provisioning needs to be addressed across the various layers (Cloud and use of virtualization - both server and desktop -  actually adds another layer of complexity for compliance by tying EVERY desktop with the server - even offline..).  In order for the Cloud to become more mainstream the auditors need more education and visibility to bake things like dynamic provisioning of desktops, apps and servers when loads are being reprovisioning in the cloud for capacity. This undoubtedly will have to be part of the Cloud providers SAS70 audit for the ISP - but what happens when it moves offline for the desktop? 

There are a few things there that need to happen from a standards perspective:
1) Technology that plugs into existing discovery tools
2) Pushing W3C to expand SMASH, DASH, and CIM to update their structure to accomodate for complexity and diversity between virtual and physical environments
3) Using FDCC as the guideline for hardening the "virtual desktop environment"
4) Expanding ITIL - to update procedures/COBIT controls beyond the HOST OS to the multiple GUEST OSes (if VDI/Type 1 Hypervisor) or single Guest OS if alternative model (like InstallFree Shell integration)

Fielding: It is OK to use POST

A good one from Roy Fielding on REST and what it really implies: 

“Tim Bray’s article on RESTful Casuistry revisits an odd meme in the REST debates that I’ve been meaning to discredit for a while.

Some people think that REST suggests not to use POST for updates.  Search my dissertation and you won’t find any mention of CRUD or POST. The only mention of PUT is in regard to HTTP’s lack of write-back caching.  The main reason for my lack of specificity is because the methods defined by HTTP are part of the Web’s architecture definition, not the REST architectural style. Specific method definitions (aside from the retrieval:resource duality of GET) simply don’t matter to the REST architectural style, so it is difficult to have a style discussion about them. The only thing REST requires of methods is that they be uniformly defined for all resources (i.e., so that intermediaries don’t have to know the resource type in order to understand the meaning of the request). As long as the method is being used according to its own definition, REST doesn’t have much to say about it.

For example, it isn’t RESTful to use GET to perform unsafe operations because that would violate the definition of the GET method in HTTP, which would in turn mislead intermediaries and spiders.  It isn’t RESTful to use POST for information retrieval when that information corresponds to a potential resource, because that usage prevents safe reusability and the network-effect of having a URI. But why shouldn’t you use POST to perform an update? Hypertext can tell the client which method to use when the action being taken is unsafe. PUT is necessary when there is no hypertext telling the client what to do, but lacking hypertext isn’t particularly RESTful.

POST only becomes an issue when it is used in a situation for which some other method is ideally suited: e.g., retrieval of information that should be a representation of some resource (GET), complete replacement of a representation (PUT), or any of the other standardized methods that tell intermediaries something more valuable than “this may change something.” The other methods are more valuable to intermediaries because they say something about how failures can be automatically handled and how intermediate caches can optimize their behavior. POST does not have those characteristics, but that doesn’t mean we can live without it. POST serves many useful purposes in HTTP, including the general purpose of “this action isn’t worth standardizing.”

I think the anti-POST meme got started because of all the arguments against tunneling other protocols via HTTP’s POST (e.g., SOAP, RSS, IPP, etc.). Somewhere along the line people started equating the REST arguments of “don’t violate HTTP’s method definitions” and “always use GET for retrieval because that forces the resource to have a URI” with the paper tiger of “POST is bad.”

Please, let’s move on. We don’t need to use PUT for every state change in HTTP. REST has never said that we should.

What matters is that every important resource have a URI, therein allowing representations of that resource to be obtained using GET.  If the deployment state is an important resource, then I would expect it to have states for undeployed, deployment requested, deployed, and undeployment requested. The advantage of those states is that other clients looking at the resource at the same time would be properly informed, which is just good design for UI feedback. However, I doubt that Tim’s application would consider that an important resource on its own, since the deployment state in isolation (separate from the thing being deployed) is not a very interesting or reusable resource.

Personally, I would just use POST for that button. The API can compensate for the use of POST by responding with the statement that the client should refresh its representation of the larger resource state. In other words, I would return a 303 response that redirected back to the VM status, so that the client would know that the state has changed.”

You can read Roy Fielding's blog here.

Amazon revises pricing structure signalling shift in CC momentum

Amazon recently revised their pricing structure, a significant signal for EC2 and S3:

http://blogs.zdnet.com/BTL/?p=14420

http://coderslike.us/2009/03/12/you-can-now-by-options-for-amazon-ec2-instances/ 

The short of it is that Amazon won't be able to compete over the long-term (>5 yrs), principally because Microsoft and Google are locking in long-term contracts for well-priced power, and because Microsoft is off-loading power consumption to client with their P2P+Rich Internet Services strategy. Unless Amazon starts building mega data centres around the globe, they cannot come close in their pricing model from a power perspective, since they can't get cost leverage for one of the most expensive components in providing hosted services CC. The key, un-advertised benefit of renewable energy is that, if one is a big enough consumer of power, as Microsoft and Google are today, one can lock in their price per kwh through a long term power purchase agreement (PPA).

Then there is the broader part of Microsoft’s strategy – lower power consumption and energy costs by offloading compute to clients. Microsoft has recognized how client compute provides a better power consumption model, given how client users/owners are already conditioned to maximize power efficiency with their client hardware. This is how Microsoft sees themselves edging out Google. 

The hidden cost that Amazon and other existing data centres will face is carbon taxation, once sustainability is officially legislated. Microsoft and Google won't pay that tax, since the long-term electricity they are locking in is of the renewable variety.

Omtzigt's "Comparing Cloud Web Services"

I found "Comparing Cloud Services" by Theodore Omtzigt to be an excellent read. An excerpt:

"In my continued quest to build an operational model that properly accounts for the costs of different cloud web services, I have reached back to the visual vocabulary of operational analysis. If it was good enough to build BMC Software I figured it would be good enough for this task.

The following figure captures the typical resources in a modern data center. In the vocabulary of operational analysis we have servers and transactions, and the diagram depicts the read and write transactions going into different services such as filers or Internet, and read responses coming out. If you would build your own data center these servers and services would reflect all your capital and operational expenditures.

Different data centers select different resources to monetize. This makes the comparison between different providers so difficult: they are all selling something different. 

Let's start with Amazon as the baseline since AWS tries to monetize all the resources in its data center, except for the internal routers. The next diagram shows the resource costs that Amazon charges you when running an application on their data centers."

Read the full article on http://smartdatacollective.com/Home/16828