IT in the Enterprise

ITIL ver.3 Continual Service Improvement

2009-01-21T02:11:00.000-08:00

Purpose

Continual Service Improvement (CSI) is concerned with maintaining value for customers through the continual evaluation and improvement of the quality of services and the overall maturity of the ITSM service lifecycle and underlying processes.

CSI combines principles, practices and methods from quality management, Change Management and capability improvement, working to improve each stage in the service lifecycle, as well as the current services, processes, and related activities and technology.

CSI is not a new concept, but for most organizations the concept has not moved beyond the discussion stage. For many organizations, CSI becomes a project when something has failed and severely impacted the business. When the issue is resolved the concept is promptly forgotten until the next major failure occurs. Discrete time-bound projects are still required, but to be successful CSI must be embedded within the organizational culture and become a routine activity.

The CSI Model shown in Figure 5 provides a way for an organization to identify and manage appropriate improvements by contrasting their current position and the value they are providing to the business, with their long-term goals and objectives, and identifying any gaps that exist. This is done on a continual basis to address changes in business requirements, technology, and to ensure high quality is maintained.

Key processes and activities

CSI defines three key processes for the effective implementation of continual improvement, the 7-Step Improvement Process, Service Measurement, and Service Reporting.

7-Step Improvement Process

The 7-step improvement process covers the steps required to collect meaningful data, analyze this data to identify trends and issues, present the information to management for their prioritization and agreement, and implement improvements.

Each step is driven by the strategic, tactical and operational goals defined during Service Strategy and Service Design:

Step 1 - Define what you should measure

A set of measurements should be defined that fully support the goals of the organization. The focus should be on identifying what is needed to satisfy the goals fully, without considering whether the data is currently available.

Step 2 - Define what you can measure

Organizations may find that they have limitations on what can actually be measured, but it is useful to recognize that such gaps exist and what risks may be involved as a result.

A gap analysis should be conducted between what is or can be measured today and what is ideally required. The gaps and implications can then be reported to the business, the customers and IT management. It is possible that new tools or customization will be required at some stage.

Step 3 - Gather the data

This covers monitoring and data collection. A combination of monitoring tools and manual processes should be put in place to collect the data needed for the measurements that have been defined.

Quality is the key objective of monitoring for CSI. Therefore monitoring focuses on the effectiveness of a service, process, tool, organization or CI. The emphasis is on identifying where improvements can be made to the existing level of service, or IT performance, typically by detecting exceptions and resolutions.

CSI is not only interested in exceptions. If a Service Level Agreement is consistently met over time, CSI is also interested in determining whether that level of performance can be sustained at a lower cost or whether it needs to be upgraded to an even better level of performance.

Step 4 - Process the data

Raw data is processed into the required format, typically providing an end-to-end perspective on the performance of services and/or processes. Processing the data is an important CSI activity that is often overlooked. While monitoring and collecting data on a single infrastructure component is important, it is key to understand that component’s impact on the larger infrastructure and IT service.

Step 5 - Analyze the data

Data analysis transforms the information into knowledge of the events that are affecting the organization.

Once the data is processed into information, the results can be analyzed to answer questions such as:

Are we meeting targets?
Are there any clear trends?
Are corrective actions required? What is the cost?

Step 6 - Present and use the Information

The knowledge gained can now be presented in a format that is easy to understand and allows those receiving the information to make strategic, tactical and operational decisions. The information needs to be provided at the right level and in the right way for the intended audience. It should provide value, note exceptions to service, and highlight any benefits that have been identified during the time period.

Now more than ever, IT must invest the time to understand specific business goals and translate IT metrics to reflect an impact against these business goals. Often there is a gap between what IT reports and what is of interest to the business.

Although most reports tend to concentrate on areas of poor performance, good news should be reported as well. A report showing improvement trends is IT services’ best marketing vehicle.

Step 7 - Implement corrective action

The knowledge gained is used to optimize, improve and correct services, processes, and all other supporting activities and technology. The corrective actions required to improve the service should be identified and communicated to the organization.

CSI will identify many opportunities for improvement and an organization will need to determine priorities based on their goals, and the resources and funding available.

The 7-Step Improvement Process is continual and loops back to the beginning.

Service Measurement

There are four basic reasons to monitor and measure, to:

validate previous decisions that have been made
direct activities in order to meet set targets - this is the most prevalent reason for monitoring and measuring
justify that a course of action is required, with factual evidence or proof
intervene at the appropriate point and take corrective action.

Monitoring and measurement underpins CSI and the 7-Step Improvement Process, and is an essential part of being able to manage services and processes, and report value to the business.

Many organizations today measure at the component level, and although this is necessary and valuable, service measurement must go up a level to provide a view of the true customer experience of services being delivered.

There are three types of metrics that an organization needs to collect to support CSI activities as well as other process activities.

Technology metrics: often associated with component and application based metrics such as performance, availability.
Process metrics: captured in the form of Critical Success Factors (CSFs), Key Performance Indicators (KPIs) and activity metrics.
Service metrics: the results of the end-to-end service. Component/technology metrics are used to compute the service metrics.

An integrated Service Measurement Framework needs to be put in place that defines and collects the required metrics and raw data, and supports the reporting and interpretation of that data.

Service Reporting

A significant amount of data is collated and monitored by IT in the daily delivery of quality service to the business, but only a small subset is of real interest and importance to the business. The business likes to see a historical representation of the past period’s performance that portrays their experience, but it is more concerned with those historical events that continue to be a threat going forward, and how IT intends to mitigate against such threats.

It is not enough to present reports depicting adherence or otherwise to SLAs. IT needs to build an actionable approach to reporting, i.e. what happened, what IT did, how IT will ensure it doesn’t impact again and how IT are working to improve service delivery generally.

A reporting ethos which focuses on the future as strongly as it focuses on the past also provides the means for IT to market its offerings directly aligned to the positive or negative experiences of the business.

Key Roles and Responsibilities

Whilst a CSI Manager is responsible for the overall CSI activities within an organization, the majority of the detailed improvement related work is carried out within each of the lifecycle stages, processes and activities.

ITIL ver3 Service Operation

2009-01-21T00:21:00.000-08:00

Purpose

The purpose of Service Operation is to deliver agreed levels of service to users and customers, and to manage the applications, technology and infrastructure that support delivery of the services.

It is only during this stage of the lifecycle that services actually deliver value to the business, and it is the responsibility of Service Operation staff to ensure that this value is delivered.

It is important for Service Operation to balance conflicting goals:

internal IT view versus external business view
stability versus responsiveness
quality of service versus cost of service
reactive versus proactive activities.

For each of these conflicts, staff must maintain an even balance, as excessive focus on one side of any of these will result in poor service.

Many organizations find it helpful to consider the “operational health” of services. This identifies “vital signs” that are critical for execution of Vital Business Functions. If these are within normal ranges, the system or service is healthy. This leads to a reduction in the cost of monitoring, and enables staff to focus on areas that will lead to service success.

Key processes and activities

Event Management Process

An event is a change of state that has significance for the management of a configuration item or IT service.

An event may indicate that something is not functioning correctly, leading to an incident being logged. Events may also indicate normal activity, or a need for routine intervention such as changing a tape.

Event management depends on monitoring, but it is different. Event management generates and detects notifications, whilst monitoring checks the status of components even when no events are occurring.

Events may be detected by a CI sending a message, or by a management tool polling the CI. After an event has been detected it may lead to an Incident, Problem or Change, or it may simply be logged in case the information is needed.

Response to an event may be automated or may require manual intervention. If actions are needed then a trigger, such as an SMS message or an incident being automatically logged, can alert support staff.

Incident Management Process

An incident is an unplanned interruption to an IT service, or a reduction in the quality of an IT service. Failure of a configuration item that has not yet impacted service is also an incident.

The purpose of Incident Management is to restore normal service as quickly as possible, and to minimize the adverse impact on business operations.

Incidents are often detected by event management, or by users contacting the service desk. Incidents are categorized to identify who should work on them and for trend analysis, and they are prioritized according to urgency and business impact.

If an incident cannot be resolved quickly, it may be escalated. Functional escalation passes the incident to a technical support team with appropriate skills; hierarchical escalation engages appropriate levels of management.

After the incident has been investigated and diagnosed, and the resolution has been tested, the Service Desk should ensure that the user is satisfied before the incident is closed.

An Incident Management tool is essential for recording and managing incident information.

Request Fulfillment Process

A service request is a request from a user for information or advice, or for a standard change, or for access to an IT service.

The purpose of Request Fulfillment is to enable users to request and receive standard services; to source and deliver these services; to provide information to users and customers about services and procedures for obtaining them; and to assist with general information, complaints and comments.

All requests should be logged and tracked. The process should include appropriate approval before fulfilling the request.

Access Management Process

The purpose of the Access Management process is to provide the rights for users to be able to access a service or group of services, while preventing access to non-authorized users.

Access Management helps to manage confidentiality, availability and integrity of data and intellectual property.

Access Management is concerned with identity (unique information that distinguishes an individual) and rights (settings that provide access to data and services). The process includes verifying identity and entitlement, granting access to services, logging and tracking access, and removing or modifying rights when status or roles change.

Problem Management Process

A problem is a cause of one or more incidents. The cause is not usually known at the time a problem record is created, and the problem management process is responsible for furtherinvestigation.

The key objectives of Problem Management are to prevent problems and resulting incidents from happening, to eliminate recurring incidents and to minimize the impact of incidents that cannot be prevented. Problem Management includes diagnosing causes of incidents, determining the resolution, and ensuring that the resolution is implemented. Problem Management also maintains information about problems and the appropriate workarounds and resolutions.

Problems are categorized in a similar way to incidents, but the goal is to understand causes, document workarounds and request changes to permanently resolve the problems. Workarounds are documented in a Known Error Database, which improves the efficiency and effectiveness of Incident Management.

Common Service Operation Activities

Service Operation includes a number of activities that are not part of the five processes described. These include:

monitoring and control: to detect the status of services and CIs and take appropriate corrective action

console management/operations bridge: a central coordination point for monitoring and managing services

management of the infrastructure: storage, databases, middleware, directory services, facilities/data centre etc.

operational aspects of processes from other lifecycle stages: Change, Configuration, Release and Deployment, Availability, Capacity, Knowledge, Service Continuity Management etc.

Key Functions

Service Desk Function

The Service Desk provides a single central point of contact for all users of IT. The Service Desk usually logs and manages all incidents, service requests and access requests and provides an interface for all other Service Operation processes and activities.

Specific Service Desk responsibilities include:

logging all incidents and requests, categorizing and prioritizing them
first-line investigation and diagnosis
managing the lifecycle of incidents and requests, escalating as appropriate and closing them when the user is satisfied
keeping users informed of the status of services, incidents and requests.

There are many ways of structuring and organizing service desks, including:

local service desk: physically close to the users
centralized service desk: allows fewer staff to deal with a higher volume of calls
virtual service desk: staff are in many locations but appear to the users to be a single team
follow the sun: Service Desks in different time zones give 24-hour coverage by passing calls to a location where staff are working.

Technical Management Function

Technical Management includes all the people who provide technical expertise and management of the IT infrastructure.

Technical Management helps to plan, implement and maintain a stable technical infrastructure and ensure that required resources and expertise are in place to design, build, transition, operate and improve the IT services and supporting technology.

Activities carried out by Technical Management include:

identifying knowledge and expertise requirements
defining architecture standards
involvement in the design and build of new services and operational practices
contributing to service design, service transition or continual service improvement projects
assistance with service management processes, helping to define standards and tools, and undertaking activities such as the evaluation of change requests
assistance with the management of contracts and vendors.

Technical Management is usually organized based on the infrastructure that each team supports.

Application Management Function

Application Management includes all the people who provide technical expertise and management of applications. As such they carry out a very similar role to Technical Management, but with a focus on software applications rather than infrastructure.

It is common in many organizations to refer to applications as services, but applications are just one component needed to provide a service. Each application may support more than one service, and each service may make use of many applications. This is especially true for modern service providers who create shared services based on service- oriented architectures.

Application Management works closely with Development, but is a distinct function with different roles. Activities carried out by Application Management are similar to those described above for Technical Management.

Application Management is usually organized by the lines of business that each team supports.

IT Operations Management Function

IT Operations Management is responsible for the management and maintenance of the IT infrastructure required to deliver the agreed level of IT services to the business. It includes two functions:

IT Operations Control is usually staffed by shifts of operators who carry out routine operational tasks. They provide centralized monitoring and control, usually from an operations bridge or network operations centre.

Facilities Management is responsible for management of data centres, computer rooms and recovery sites. Facilities Management also coordinates large-scale projects, such as data centre consolidation or server consolidation.

ITIL ver3 Service Transition

2009-01-21T00:14:00.000-08:00

Purpose

The role of Service Transition is to deliver services that are required by the business into operational use. Service Transition delivers this by receiving the Service Design Package from the Service Design stage and delivering into the Operational stage every necessary element required for ongoing operation and support of that service. If business circumstances, assumptions or requirements have changed since design, then modifications may well be required during the

Service Transition stage in order to deliver the required service. Service Transition focuses on implementing all aspects of the service, not just the application and how it is used in ‘normal’ circumstances. It needs to ensure that the service can operate in foreseeable extreme or abnormal circumstances, and that support for failure or errors is available. This requires sufficient understanding of:

potential business value and who it is delivered to/judged by
identification of all stakeholders within supplier, customer and other areas
application and adaptation of service design, including arranging for modification of the design, where the need is detected during transition.

Key Principles

Service Transition is supported by underlying principles that facilitate effective and efficient use of new/changed services. Key principles include:

Understanding all services, their utility and warranties - to transition a service effectively it is essential to know its nature and purpose in terms of the outcomes and/or removed business constraints (utilities) and the assurances that the utilities will be delivered (warranties).

Establishing a formal policy and common framework for implementation of all required changes - consistency and comprehensiveness ensure that no services, stakeholders, occasions etc. are missed out and so cause service failures.

Supporting knowledge transfer, decision support and re-use of processes, systems and other elements – effective Service Transition is delivered by involving all relevant parties, ensuring appropriate knowledge is available and that work done is reusable in future similar circumstances.

Anticipating and managing ‘course corrections’ – being proactive and determining likely course correction requirements, and when elements of a service do need to be adjusted, this is undertaken logically and is fully documented.

Ensuring involvement of Service Transition and Service Transition requirements throughout the service lifecycle.

Key Processes and Activities

Within the Service Transition process set, some of the processes most important to Service Transition are whole lifecycle processes and have impact, input and monitoring and control considerations across all lifecycle stages. The whole lifecycle processes are:

Change Management
Service Asset and Configuration Management
Knowledge Management.

Processes focused on Service Transition, but not exclusive to the stage, are:

Transition Planning and Support
Release and Deployment Management
Service Validation and Testing
Evaluation.

Change Management

Change Management ensures that changes are recorded, evaluated, authorized, prioritized, planned, tested, implemented, documented and reviewed in a controlled manner.

The purpose of the Change Management process is to ensure that standardized methods are used for the efficient and prompt handling of all changes, that all changes are recorded in the Configuration Management System and that overall business risk is optimized.

The process addresses all service change.

A Service Change is the addition, modification or removal of an authorised, planned or supported service or service component and its associated documentation.

Therefore change management is relevant across the whole lifecycle, applying to all levels of service management – strategic, tactical and operational.

Change management delivers, to the business, reduced errors in new or changed services and faster, more accurate implementation of changes; it allows restricted funds and resources to be focused on those changes to achieve greatest benefit to the business.

Service Asset and Configuration Management (SACM)

SACM supports the business by providing accurate information and control across all assets and relationships that make up an organization’s infrastructure.

The purpose of SACM is to identify, control and account for service assets and configuration items (CI), protecting and ensuring their integrity across the service lifecycle.

The scope of SACM also extends to non-IT assets and to internal and external service providers, where shared assets need to be controlled.

To manage large and complex IT services and infrastructures, SACM requires the use of a supporting system known as the Configuration Management System (CMS).

Knowledge Management

The purpose of Knowledge Management is to ensure that the right person has the right knowledge, at the right time to deliver and support the services required by the business. This delivers:

more efficient services with improved quality
clear and common understanding of the value provided by services
relevant information that is always available.

At the heart of Knowledge Management is the Data-Information-Knowledge- Wisdom structure, condensing raw – and unusable – data into valuable assets. This is illustrated by the Service Knowledge Management System, holding relevant information and wisdom derived from Asset and Configuration Data.

Transition Planning and Support

The goals of Transition Planning and Support are to:

plan and coordinate resources to ensure that the requirements of Service Strategy encoded in Service Design are effectively realized in Service Operations

identify, manage and control the risks of failure and disruption across transition activities.

Effective Transition Planning and Support can significantly improve a service
provider’s ability to handle high volumes of change and releases across its
customer base.

Release and Deployment Management

The goal of the Release and Deployment Management process is to assemble and position all aspects of services into production and establish effective use of new or changed services.

Effective release and deployment delivers significant business value by delivering changes at optimized speed, risk and cost, and offering a consistent, appropriate and auditable implementation of usable and useful business services.

Release and Deployment Management covers the whole assembly and implementation of new/changed services for operational use, from release planning through to early life support.

Service Validation and Testing

Successful testing depends on understanding the service holistically – how it will be used and the way it is constructed. All services – whether in-house or bought-in – will need to be tested appropriately, providing validation that business requirements can be met in the full range of expected situations, to the extent of agreed business risk.

The key purpose of service validation and testing is to provide objective evidence that the new/changed service supports the business requirements, including the agreed SLAs.

The service is tested explicitly against the utilities and warranties set out in the service design package, including business functionality, availability, continuity, security, usability and regression testing.

Evaluation

Ensuring that the service will be useful to the business is central to successful Service Transition and this extends into ensuring that the service will continue to be relevant by establishing appropriate metrics and measurement techniques.

Evaluation considers the input to Service Transition, addressing the relevance of the service design, the transition approach itself, and the suitability of the new or changed service for the actual operational and business environments encountered and expected.

Service Transition Stage Operational Activities

Service Transition is also the focus for some operational activities. These have wider applicability than Service Transition and comprise:

managing communications and commitment across IT Service Management
managing organizational and stakeholder change
stakeholder management
organization of Service Transition and key roles.

Key Roles and Responsibilities

The staff delivering Service Transition within an organization must be organized for effectiveness and efficiency, and various options exist to deliver this. It is not anticipated that a typical organization would consider a separate group of people for this role, rather there is a flow of experience and skills – meaning the same people may well be involved in multiple lifecycle stages.

ITIL ver3 Service Design

2009-01-20T23:19:00.000-08:00

Purpose

Service Design is a stage within the overall service life cycle and an important element within the business change process. The role of Service Design within the business change process can be defined as:

The design of appropriate and innovative IT services, including their architectures, processes, policies and documentation, to meet current and future agreed business requirements.

The main goals and objectives of Service Design are to:

- design services to meet agreed business outcomes
- design processes to support the service life cycle
- identify and manage risks
- design secure and resilient IT infrastructures, environments, applications and data/information resources and capability
- design measurement methods and metrics
- produce and maintain plans, processes, policies, standards, architectures, frameworks and documents to support the design of quality IT solutions
- develop skills and capability within IT
- contribute to the overall improvement in IT service quality.

Key Principles

Service Design starts with a set of business requirements, and ends with the development of a service solution designed to meet documented business requirements and outcomes and to provide a Service Design Package (SDP) for handover into Service Transition.

There are 5 individual aspects of Service Design:

- new or changed service solutions
- service management systems and tools, especially the Service Portfolio
- technology architectures and management systems
- processes, roles and capabilities
- measurement methods and metrics.

A holistic approach should be adopted in Service Design to ensure consistency and integration in all IT activities and processes, providing end-to-end business-related functionality and quality. Good service design is dependent upon the effective and efficient use of the Four Ps of Design:

- people: the people, skills and competencies involved in the provision of IT services
- products: the technology and management systems used in the delivery of IT services
- processes: the processes, roles and activities involved in the provision of IT services
- partners: the vendors, manufacturers and suppliers used to assist and support IT service provision.

Service Design Package (SDP): defines all aspects of an IT service and its requirements through each stage of its lifecycle. An SDP is produced for each new IT service, major change, or IT service retirement.

Key Processes and Activities

Service Catalogue Management (SCM)

The Service Catalogue provides a central source of information on the IT services delivered to the business by the service provider organization, ensuring that business areas can view an accurate, consistent picture of the IT services available, their details and status.

The purpose of Service Catalogue Management (SCM) is to provide a single, consistent source of information on all of the agreed services, and ensure that it is widely available to those who are approved to access it.

The key information within the SCM process is that contained within the Service Catalogue. The main input for this information comes from the Service Portfolio and the business via either the Business Relationship Management or the Service Level Management processes.

Service Level Management (SLM)

SLM negotiates, agrees and documents appropriate IT service targets with the business, and then monitors and produces reports on delivery against the agreed level of service.

The purpose of the SLM process is to ensure that all operational services and their performance are measured in a consistent, professional manner throughout the IT organization, and that the services and the reports produced meet the needs of the business and customers.

The main information provided by the SLM process includes Service Level Agreements (SLA), Operational Level Agreements (OLA) and other support agreements, and the production of the Service Improvement Plan (SIP) and the Service Quality Plan.

Capacity Management

Capacity Management includes business, service and component capacity management across the service lifecycle. A key success factor in managing capacity is ensuring that it is considered during the design stage.

The purpose of Capacity Management is to provide a point of focus and management for all capacity and performance-related issues, relating to both services and resources, and to match the capacity of IT to the agreed business demands.

The Capacity Management Information System (CMIS) is the cornerstone of a successful Capacity Management process. Information contained within the CMIS is stored and analyzed by all the sub-processes of Capacity Management for the provision of technical and management reports, including the Capacity Plan.

Availability Management

The purpose of Availability Management is to provide a point of focus and management for all availability-related issues, relating to services, components and resources, ensuring that availability targets in all areas are measured and achieved, and that they match or exceed the current and future agreed needs of the business in a cost-effective manner.

Availability Management should take place at two inter-connected levels and aim to continually optimize and proactively improve the availability of IT services and their supporting organization. There are two key aspects:

- reactive activities: monitoring, measuring, analysis and management of events, incidents and problems involving service unavailability

- proactive activities: proactive planning, design, recommendation and improvement of availability.

Availability Management activities should consider the availability, reliability, maintainability and serviceability at both service and component level, particularly those supporting Vital Business Functions (VBFs).

The Availability Management process should be based around an Information System (AMIS) that contains all of the measurements and information required to provide the appropriate information to the business on service levels. The AMIS also assists in the production of the Availability Plan.

IT Service Continuity Management (ITSCM)

As technology is a core component of most business processes, continued or high availability of IT is critical to the survival of the business as a whole. This is achieved by introducing risk reduction measures and recovery options. On-going maintenance of the recovery capability is essential if it is to remain effective.

The purpose of ITSCM is to maintain the appropriate on-going recovery capability within IT services to match the agreed needs, requirements and timescales of the business.

ITSCM includes a series of activities throughout the lifecycle to ensure that, once service continuity and recovery plans have been developed, they are kept aligned with Business Continuity Plans and business priorities.

The maintenance of appropriate ITSCM policy strategies and ITSCM plans aligned with business plans is key to the success of an ITSCM process. This can be accomplished by the regular completion of Business Impact Analysis and Risk Management exercises.

Information Security Management (ISM)

ISM needs to be considered within the overall corporate governance framework. Corporate governance is the set of responsibilities and practices exercised by the board and executive management with the goal of providing strategic direction, ensuring that the objectives are achieved, ascertaining that the risks are being managed appropriately, and verifying that the enterprise’s resources are used effectively.

The purpose of the ISM process is to align IT security with business security and ensure that information security is effectively managed in all service and Service Management activities, such that:

- information is available and usable when required (availability)
- information is observed by or disclosed to only those who have a right to know (confidentiality)
- information is complete, accurate and protected against unauthorized modification (integrity)
- business transactions, as well as information exchanges, can be trusted (authenticity and non-repudiation).

ISM should maintain and enforce an overall policy, together with a set of supporting controls within an integrated Security Management Information System (SMIS), aligned with business security policies and strategies.

Supplier Management

The Supplier Management process ensures that suppliers and the services they provide are managed to support IT service targets and business expectations.

The purpose of the Supplier Management process is to obtain value for money from suppliers and to ensure that suppliers perform to the targets contained within their contracts and agreements, while conforming to all of the terms and conditions.

The Supplier and Contract Database (SCD) is a vital source of information on suppliers and contracts and should contain all of the information necessary for the management of suppliers, contracts and their associated services.

Key Service Design stage activities

- Business requirements collection, analysis and engineering to ensure they are clearly documented.
- Design and development of appropriate service solutions, technology, processes, information and measurements.
- Production and revision of all design processes and documents involved in Service Design.
- Liaison with all other design and planning activities and roles.
- Production and maintenance of policies and design documents.
- Risk management of all services and design processes.
- Alignment with all corporate and IT strategies and policies.

Key Roles and Responsibilities

The key roles involved within the Service Design activities and processes are:

- Service Design Manager: responsible for the overall coordination and deployment of quality solution designs for services and processes

- IT Designer/Architect: responsible for the overall coordination and design of the required technologies, architectures, strategies, designs and plans

- Service Catalogue Manager: responsible for producing and maintaining an accurate Service Catalogue

- Service Level Manager: responsible for ensuring that the service quality levels are agreed and met

- Availability Manager: responsible for ensuring that all services meet their agreed availability targets

- IT Service Continuity Manager: responsible for ensuring that all services can be recovered in line with their agreed business needs, requirements and timescales

- Capacity Manager: responsible for ensuring that IT capacity is matched to agreed current and future business demands

- Security Manager: responsible for ensuring that IT security is aligned with agreed business security policy risks, impacts and requirements

- Supplier Manager: responsible for ensuring that value for money is obtained from all IT suppliers and contracts, and that underpinning contracts and agreements are aligned with the needs of the business.

ITIL ver.3 Service Strategy

2009-01-20T22:30:00.000-08:00

Purpose

The service strategy of any service provider must be grounded upon a fundamental acknowledgment that its customers do not buy products, they buy the satisfaction of particular needs. Therefore, to be successful, the services provided must be perceived by the customer to deliver sufficient value in the form of outcomes that the customer wants to achieve.

Achieving a deep understanding of customer needs, in terms of what these needs are, and when and why they occur, also requires a clear understanding of exactly who is an existing or potential customer of that service provider. This, in turn, requires the service provider to understand the wider context of the current and potential market places that the service provider operates in, or may wish to operate in.

A service strategy can not be created or exist in isolation of the over-arching strategy and culture of the organization that the service provider belongs to. The service provider may exist within an organization solely to deliver service to one specific business unit, to service multiple business units, or may operate as an external service provider serving multiple external businesses. The strategy adopted must provide sufficient value to the customers and all of the service provider’s stakeholders – it must fulfill the service provider’s strategic purpose.

Irrespective of the context in which the service provider operates, its service strategy must also be based upon a clear recognition of the existence of competition, an awareness that each side has choices, and a view of how that service provider will differentiate itself from the competition. All providers need a service strategy.

Hence, the Service Strategy publication sits at the core of the ITIL V3 lifecycle. It sets out guidance to all IT service providers and their customers, to help them operate and thrive in the long term by building a clear service strategy, i.e. a precise understanding of:

- what services should be offered
- who the services should be offered to
- how the internal and external market places for their services should be developed
- the existing and potential competition in these marketplaces, and the objectives that will differentiate the value of what you do or how you do it
- how the customer(s) and stakeholders will perceive and measure value, and how this value will be created
- how customers will make service sourcing decisions with respect to use of different types of service providers
- how visibility and control over value creation will be achieved through financial management
- how robust business cases will be created to secure strategic investment in service assets and service management capabilities
- how the allocation of available resources will be tuned to optimal effect across the portfolio of services
- how service performance will be measured.

Key Concepts

The Service Strategy publication defines some key ITIL concepts.

The four Ps of Strategy:
- perspective: the distinctive vision and direction
- position: the basis on which the provider will compete
- plan: how the provider will achieve their vision
- pattern: the fundamental way of doing things – distinctive patterns in decisions and actions over time.

Competition and Market Space:
- every service provider is subject to competitive forces
- all service providers and customers operate in one or more internal or external market spaces. The service provider must strive to achieve a better understanding than its competitors of the dynamics of the market space, its customers within it, and the combination of critical success factors that are unique to that market space.

Service Value: defined in terms of the customer’s perceived business outcomes, and described in terms of the combination of two components:

- Service Utility: what the customer gets in terms of outcomes supported and/or constraints removed
- Service Warranty: how the service is delivered and its fitness for use, in terms of availability, capacity, continuity and security.

Service Value also includes the associated concepts of services as Assets, Value Networks, Value Creation and Value Capture.

Service Provider Types:

- Type I: exists within an organization solely to deliver service to one specific business unit
- Type II: services multiple business units in the same organization
- Type III: operates as an external service provider serving multiple external customers.

Service Management as a Strategic Asset: the use of ITIL to transform service management capabilities into strategic assets, by using Service Management to provide the basis for core competency, distinctive performance and durable advantage, and increase the service provider’s potential from their:

- capabilities: the provider’s ability (in terms of management, organization, processes, knowledge and people) to coordinate, control and deploy resources
- resources: the direct inputs for the production of services, e.g. financial, capital, infrastructure, applications, information and people.

Critical Success Factors (CSFs): the identification, measurement and periodic review of CSFs to determine the service assets required to successfully implement the desired service strategy.

Service Oriented Accounting: using financial management to understand services in terms of consumption and provisioning, and achieve translation between corporate financial systems and service management.

Service Provisioning Models: categorization and analysis of the various models that may be selected by customers and used by service providers to source and deliver services, and the financial management impacts of on-shore, off-shore or near-shore variants:

- Managed Service: where a business unit requiring a service fully funds the provision of that service for itself
- Shared Service: the provisioning of multiple services to one or more business units through shared infrastructure and resources
- Utility: services are provided on the basis of how much is required by each customer, how often, and at what times the customer needs them.

Organization Design and Development: achieving an ongoing shape and structure to the service provider’s organization that enables the service strategy. Considerations include:

- Organizational Development Stages: delivering services through network, direction, delegation, coordination or collaboration depending on the evolutionary state of the organization.

- Sourcing Strategy: making informed decisions on service sourcing in terms of internal services, shared services, full service outsourcing, prime consortium or selective outsourcing.

- Service Analytics: using technology to help achieve an understanding of the performance of a service through analysis.

- Service Interfaces: the mechanisms by which users and other processes interact with each service.

- Risk Management: mapping and managing the portfolio of risks underlying a service portfolio.

Key Processes and Activities

In addition to Strategy Generation, Service Strategy also includes the following key processes.

Financial Management

Financial Management covers the function and processes responsible for managing an IT service provider’s budgeting, accounting and charging requirements. It provides the business and IT with the quantification, in financial terms, of the value of IT services, the value of the assets underlying the
provisioning of those services, and the qualification of operational forecasting.

IT Financial Management responsibilities and activities do not exist solely within the IT finance and accounting domain. Many parts of the organization interact to generate and use IT financial information; aggregating, sharing and maintaining the financial data they need, enabling the dissemination of information to feed critical decisions and activities.

Service Portfolio Management (SPM)

SPM involves proactive management of the investment across the service lifecycle, including those services in the concept, design and transition pipeline, as well as live services defined in the various service catalogues and retired services.

SPM is an ongoing process, which includes the following:

- Define: inventory services, ensure business cases and validate portfolio data
- Analyze: maximize portfolio value, align and prioritize and balance supply and demand
- Approve: finalize proposed portfolio, authorize services and resources
- Charter: communicate decisions, allocate resources and charter services.

Demand Management

Demand management is a critical aspect of service management. Poorly managed demand is a source of risk for service providers because of uncertainty in demand. Excess capacity generates cost without creating value that provides a basis for cost recovery.

The purpose of Demand Management is to understand and influence customer demand for services and the provision of capacity to meet these demands. At a strategic level this can involve analysis of patterns of business activity and user profiles. At a tactical level it can involve use of differential charging to encourage customers to use IT services at less busy times.

A Service Level Package (SLP) defines the level of utility and warranty for a Service Package and is designed to meet the needs of a pattern of business activity.

Key Roles and Responsibilities

The Service Strategy publication defines some specific roles and responsibilities associated with the execution of a successful service strategy, including:

- Business Relationship Manager (BRM): BRMs establish a strong business relationship with the customer by understanding the customer's business and their customer outcomes. BRMs work closely with the Product Managers to negotiate productive capacity on behalf of customers.

- Product Manager (PM): PMs take responsibility for developing and managing services across the life-cycle, and have responsibilities for productive capacity, service pipeline, and the services, solutions and packages that are presented in the service catalogues.

- Chief Sourcing Officer (CSO): the CSO is the champion of the sourcing strategy within the organization, responsible for leading and directing the sourcing office and development of the sourcing strategy in close conjunction with the CIO.

64 Seriously Cool iPhone Applications

2008-09-05T10:42:00.000-07:00

OK, you've bought your iPhone, showed it to all your friends, and learned to love it. The thing's cool. But now you want more. You want your iPhone to sing and dance and play with you. You want more functionality – and more fun – and you want tons of freebies.

With that in mind, peruse this list of iPhone apps. Lots of cool tools. Weather, sports, news, games, music, an eBook reader, even time management apps: they're all here. Check 'em out...

Personal Productivity

1) Befree 4 iPhone (http://www.ewe-software.de/en/download.html)
Enables remote control of your PC from your mobile device. It's freeware, but the paid pro version allows you remote access to all the files stored on your PC hard drive.

2) iBay (http://iphone.computerfaq.be/ibay/)
Search eBay on your iPhone. (You bought that hand held to help you shop, didn't you?)

3) iWebSaver (http://iwebsaver.com/)
Lets you save your favorite Web sites and application to your iPhone (or iPod). It's simple: type in an URL, click "Save it now," and chose "Add bookmark." Also available as the iWebSaverFavlet.

4) 911 Help (http://www.usa-links.com/911help/)
Turn your iPhone into a safety device: this app displays a big bold red 911 sign visible dozens of feet away from the device. It flashes to help others realize you need assistance.

5) Can I Drive Yet? (http://www.canidriveyet.com/)
Helps you calculate if you're sober enough to operate a motor vehicle; enter your relevant data (how many drinks, your body weight, etc.) and it spits out an answer, using national blood alcohol standards.

6) BarCheck (http://barcheck.net/)
When you're shopping in your neighborhood store, enter the numbers on an item's bar code to get review and prices from Amazon, Google and Yahoo. Yup, it's smart shopping.

7) Widgetop (http://www.widgetop.com/mobile.html)
Helps you manage your collection of widgets. (A widget is a software app that lets you do something. An example is a menu bar or a toggle switch, but there are zillions.)

8) Dictionary (http://dictionary.aapl.org.uk/#_0)
Not just a dictionary, but also a thesaurus. Comes in handy when you're typing that business e-mail and you want to seem suave.

9) Calorie Counter (http://iphone.trebitowski.com/Calculate/ DailyCalorieIntake/)
Calculate your daily Calorie intake based on age, weight and exercise level. If you're feeling disciplined, it also gives you a number to shoot for to lose weight. (Cupcake, anyone?)

10) Currency Exchange (http://rates.speedymarks.com/)
Keep track of the relative value of the world's major currencies. Watch the dollar's precipitous fall right there on your hand held. You might also try eSignal's Currency Exchange Calculator. In theory the two apps should produce the same numbers.

11) Mortgage Payment Calculator (http://calcnexus.com/mortgage-payment-calculator.php)
Calculate your monthly payment on that monstrously expensive house that you can't afford. (I haven't yet found an iPhone app that calculates foreclosure costs.)

12) ModGuitars.com Tuner (http://modguitars.com/tuner/)
Helps you tune your guitar. Very useful for those lunchtime gigs that help you pay for your pricey gadgets.

Fun and Enjoyment

13) iDoodle (http://www.idoodleapp.com/)
Let's face it: you're too productive. But with iDoodle, you can wile away the hours drawing on your iPhone –- circles, polygons, many colors. Designed specifically for the iPhone.

14) xRay (http://www.usa-links.com/ixray/)
No, it's not real, but it might fool some of the people some of the time. Tap the screen, hold it over a friend's hand (someone who scoffs at the iPhone, for example) and tell them you'll X-ray their hand. A stock photo of an X-ray will show up. Very juvenile, and very cool.

15) Free-Tunes.net (http://www.free-tunes.net/us/)
A collection of the free songs on iTunes. Sure, they're little-known bands now, but the price is right, and you might discover a real gem.

16) Podcaster (http://podcaster.soprotech.com/login.aspx?ReturnUrl=%2fDefault.aspx)
An incredibly useful program to stream audio and video podcasts, using an iTunes-styled control panel.

17) Stripr (for Comics Fans) (http://dbelement.com/index.php?site=st)
Helps you manage and view all those online comics you enjoy. Plus: give feedback on comics. Thumbs up or thumbs down on Marmaduke?

18) iPhones Wallpaper (http://iphone.interfacelift.com/#_Samples)
Give your iPhone a forest background, or urban, or sunset, or psychedelic. Those pretty flowers are nice, but then there's also that mountain view…

19) MacLight (http://www.gundersondesign.com/i-phone/i-light/flashlight.html)
Turn your iPhone into a flashlight, or give it a "campfire" flickering look, or a strobe light. Might be useful the next time the power grid goes down.

20) The Illusionist (http://iphone.computerfaq.be/Illusionist/)
Strange, trippy visuals on your iPhone: rolling cylinders, brain conflicts, Escher-like optical mind-twisters. Don't look at it after your daily double espresso.

Games

21) Castle Feud (http://www.underclouds.com/iphone/castlefeud/)
Slaughter your opponents and leave them dismembered on the field of battle. Cool! A multiplayer game.

22) iSudoku (http://isudoku.janjanousek.cz/)
Having this app means never having to be away from your Sodoku addiction. Hey, you can quit whenever you want to, you just don't want to…

23) Speed Type (http://mac-gratuit.fr/iphone/speedtype/)
Measures your typing speed, with the top 200 scores saved. This game is not only fun but also improves your iPhone e-mailing skills.

24) iConnect4 (http://38i.biz/iconnect4/)
If you like Connect, you'll be happy to see iConnect. Can you connect four chips in a row before the iPhone does?

25) mobileLife (http://mobilelife.cinnamonthoughts.org/)
A cellular automation game, "Game of Life," has been developed specifically for the iPhone and iPod touch.

26) Madelinette (http://38i.biz/madelinette/)
In this game (originally European) you attempt to block your foe from moving. Who will goof up first?

27) Mines (http://imines.janjanousek.cz/)
Find all the mines that are hidden underneath the boxes – or else.

28) Gumball Bingo (http://www.fatfreegames.com:2112/gameServer/Bingo)
A multiplayer game that lets you chat with others while you wait for your winning combo. Plus: play three bingo cards at the same time.

29) iget4 (http://freenet-homepage.de/lars.nordmeyer/iget4/)
Play against your iPhone – and your computer opponent will always shift moves. This game forces you to think.

30) Olympic Sports (http://www.dseffects.com/iphone/games/OlympicSports/OlympicSports.php)
Compete virtually against the Olympic greats to see if you can capture the gold. Earn glory without the sweat (except for a few cramped fingers).

Information/Updates

31) iNews (http://www.ceneda.net/inews/)
Scan and read the news from high-profile publications, read RSS feeds, and take a Quick Look at the most critical news. In English, German, French, Italian, Spanish, and other languages.

32) PressDisplay (http://www.pressdisplay.com/pressdisplay/viewer.aspx)
Access dozens and dozens of magazines and newspapers from across the globe. Zoom in or zoom out on images.

33) Politicker (http://38i.biz/politicker/)
Scandal, polls, results, breaking news – all the developments about politics.

34) Sticher Radio (http://www.stitcher.com/iphone/)
Pick a radio station and start streaming: comedy, sports, news, finance, and more.

Manage Your Life

35) TouchMail (http://www.tm.comvalid.com/index/)
Manage and read your e-mail from any of your accounts, including Gmail, Yahoo, or MSN. Lets you read your e-mails even if you're stuck behind your company's firewall.

36) Goal Tracker (http://limeade.com/iphone/)
Monitor your progress toward your personal goals, focusing on using your time more effectively.

37) BudgetBuster (http://www.tippytops.net/iphone/budgetbuster/)
Track your daily-weekly-monthly expenses. How deep in debt are you today?

38) ezMemorize (http://www.ezmemorize.com/)
Helps you remember all those pesky little factoids that enable you to navigate through life.

39) Noter (http://dbelement.com/index.php?site=nt)
A to-do list application that works offline or online. In theory, it helps you list all those things you need to do. But do you ever get past item No. 4?

Information Retrieval/Search

40) Phone Number Trace (http://www.phone-number-trace.com/iphone.php)
Learn the identity of who just called you by entering their phone number into this search box. The free info includes location and type of line (landline, cell phone); actually finding the name will cost you $15.

41) Free Stuff Times (http://www.freestufftimes.com/)
A list of free stuff on the Internet that's constantly updated. Yeah, a lot of it's junk (c'mon, it's free) but you never know what you might find. How about a free Journal of Accountancy Calculator Calendar (thrills!) or a free Clear Wood Finish shirt? (snazzy!)

42) Song Lyric Search (http://www.motorcyclelife.net/lyricsearch/)
You just can't get that tune out of your head – but what the heck are the words to the verse?

43) About People Search (http://people.aapl.org.uk/#_0)
Use your iPhone or iPod Touch to look up anyone in America by phone number or by name. A useful little app.

44) Pinpoint Location Search (http://iphone.toughturtle.com/pinpoint/)
Quickly find a gas station, restaurant, Wi-Fi hotspot, or a residence. Plus: get traffic updates to help you avoid the rush.

45) Background Check (http://www.spontaneous-insight.com/)
Check out an individual's background, based on public records. The summary result is free, the real dirt costs.

Social Networking

46) Facebook for iPhone (http://iphone.facebook.com/login.php?next=http%3A%2F%2Fiphone.facebook.com%2F)
Allows you to access the version of Facebook that's optimized for the iPhone and iPod Touch.

47) Floort – We Know What You're Thinking (http://www.floort.com/)
A site where you can share your opinions with the world – on any topic – and start a dialogue. "Think of it as your own personal brain blog," says Floort.

48) webChattr – Live Chat Rooms (http://webchattr.com/go/thelounge)
Easy and free, webChattr is your own personal chat room, whenever you need one. Chat with your whole crew instead of texting each person. The webChattr app is on Bebo and Facebook.

49) PhoneMeeter (http://www.phonemeeter.com/all/index.html)
Share photos, text chat with friends, find a location on a Google map, check out events posted by bars and nightclubs. Think of PhoneMeeter as a mobile network community builder.

50) Fluther (http://www.fluther.com/)
Got a question you need answered? Post it to Fluther and get an answer from the crowd. Recent examples: What's a good place to take pictures in Chicago? What should I get for my birthday?

Sports

51) ScoreMobile iPhone Edition (http://iphone.thescore.com/)
All the sports scores, instantly. NBA, NCAAB, NFL, MLB, NHL. The moment the ball/puck/birdie goes in the hole/hoop/goal, you'll know about it. Or, you can also use Live Scores (http://www.streamingscores.com/iphone/Scores.htm), which is updated every 30 seconds.

52) iphodmeter (http://iphodometer.com/)
Find out how many calories your workout is burning, and send all your stats by e-mail. Plus: measure walking mileage.

53) iTeeMaster (http://www.teemaster.com/iteemaster/itm.html)
Where's the nearest golf course? iTeeMaster will clue you in, and with one touch you can call the proshop.

54) NASCAR Schedule (http://sprintcup08.dustinmcgrew.com/)
Keep track of all the auto racing events, as well as results. Vroom, vroom!

55) Madness2Go Basketball Bracket (http://madness2go.com/)
A men's basketball bracket game specially formatted for the iPhone. Choose and follow your bracket right from your handheld. Play for free, and if you're lucky (and talented) there's a chance to win prizes.

Traveling to New and Interesting Places

56) Taxi Please, for the iPhone (http://72.47.215.64/taxiplease/)
Kind of funny little app, Taxi Please flashes the word TAXI in big bold black-on-yellow letters, to help you attract a cabbie on a busy city street.

57) Learnit Lists (http://widget.learnitlists.com/iphone)
Teach yourself a new language, in a manageable 10 words at a time. Each day the app offers a list built from the 1,000 most common English words. (Is "iPhone" one of those words yet?)

58) iTinerary (http://www.mdmalin.com/webapps/itinerary/)
Manage all your flight information, including arrival gate, weather, and maps to the airport.

59) WORLDview for iPhone (http://www.longfingers.com/)
Provides all kinds of pertinent factoids for countries all across the globe: history, essential phone numbers, capital city, and more. Don't be an Ugly American, learn about the culture ahead of time. (Then go visit the McDonald's like you always do…)

60) Pocket Express Travel Edition (http://express.handmark.com/index.php?r_id=ggl_download)
A cornucopia of travel aids: airline schedules, flight status, hotel information, reservation services – it's like have a personal assistant built into your phone.

Weather

61) Weather Forecast with Live Camera (http://iphone.alice.gs/WEATHER/)
Because it covers only Japan, this is a niche app. But it still earns a spot on a list of noteworthy apps – its combination of forecast with an updated photo is so cool, I'm hoping the idea spreads to all locations. Just a matter of time, surely.

62) Weather Made Simple (http://www.briansutton.com/wx/)
Enter a city or a zip code and get current conditions plus a 10-day forecast. Quite useful.

63) The Weather Channel (http://www.weather.com/iphone/)
The classic powerhouse of weather information, right on your iPhone. Tons of current information, from any location, in an interactive format. An iPhone Reader

64) iReading for iPhone and iPod Touch (http://iphone.norbsoft.com/iReading/#_home)
A free online eBook reader for your favorite handheld. Search for book by author or title, then add it to your favorites list. Bonus: the software remembers where you last stopped reading. (Wait, didn't Steve Jobs say that people don't read anymore? Yikes, he needs to check out this nifty app.)

Get iPhone Widgets to Use on Any Phone

2008-09-05T10:28:00.000-07:00

The Apple iPhone is a breathtaking departure from the stale smartphone market. Everyone loves the iPhone's huge screen and eye-candy user interface. But many expressed disappointment when Steve Jobs announced that the iPhone applications environment would not follow the PC model of software installed on the phone, but instead a "thin-client"model of Web-based applications launched through the phone's wireless connection to the Internet.

I think the focus on Web-based applications and widgets, rather than directly installed on the phone, is one of the best things -- possibly THE best thing -- about the iPhone.

The reason people were lukewarm about the idea is that most smartphone users hate using the Internet on their phones. It's slow, the content is hard to read, navigation is a pain and typing URLs is frustrating.

What iPhone users will learn over the next year as new Web-based widgets come on-line is the Internet content and application experience can be really great -- far better than either phone browsing or phone-based applications.

I know this because I've been using iPhone-like widgets -- first on my Treo, then on my BlackBerry Pearl -- for a year now.

My source for widgets is a service called Plusmo. I use Plusmo not only for instant access to Web content (without browsing), but also to "mobilize" my personal blog, The Raw Feed.

Here's how it works: go to plusmo.com with your phone's browser to install the application on your phone, then visit the site with your Web browser to choose from among Plusmo's 20,000 widgets -- yeah, that's right: 20,000 -- and they just show up on your phone. (You can also choose widgets using just the phone.) You can pick stuff like Dilbert cartoons; a cheap-gas locator; games galore; video sites like YouTube; big-name news sources like CNN or BBC; many thousands of blogs; and a lot more widgets. For most phones, these widgets can update themselves all at once. Later, when you've got some time to kill or need some information, the information is already there.

It takes you about three minutes of using Plusmo to realize that getting content and using widgets like this is far superior to "browsing" on a cell phone. Best of all, Plusmo works on the major "smartphones" - and most "dumb" phones, too. And if you're an iPhone user, you're in luck, too. Plusmo has a new service for the iPhone, and the icons even look like Apple designed them.

It's time to stop wasting your time slogging from site to site on your cell phone's browser, and embrace the new widget movement.

Everything You Need to Know About Your iPhone Part 3

2008-09-05T10:20:00.000-07:00

Generate Custom iPhone Ring tones from iTunes Song Samples

With this hack, you'll be able to use almost any of Apple's free 30-second song previews in iTunes, which happen to be available for every track in the vast music store, to create custom ring tones for your iPhone.

Here's a brief outline of the steps involved:

The first step is to create a new playlist of un-purchased songs, export that playlist, and then save it as text. When you open it up, locate the URLs for the audio files you want as ring tones; using those Web addresses, download the 30-second snippets of the songs.

Once the audio samples are downloaded, rename the tracks — giving them a name (while keeping the .m4p file extensions) that's more meaningful to you than Apple's obscure file designations. That way the tracks will be more easily recognizable for the songs they represent when you go to use them as a ringtone.

You'll be able to do that last bit after loading the files onto the iPhone. Once they're on the device, you simply pick the file you want to use as a ringtone just like you would with the ringtones Apple pre-loaded. The 30-second song previews you just added are now listed in Settings --> Sound --> Ringtone along with these other tones.

MegaPhone Brings Disc Mode to iPhones

MegaPhone, formerly called iPhoneDrive ($9.95), fromEcammNetwork, allows you to use your iPhone for file storage, something you can't do with Apple's iPod/smartphone combo out-of-the-box, as there's no diskmode for iPhones like there is for iPods.

Launching MegaPhone brings up the utility's browser window and toolbar. It is from there you can transfer files and folders back and forth between your Mac OS X computer and iPhone.

There are two ways to perform transfers to an iPhone: Drag and drop content from the Mac Desktop or a Finder window into the MegaPhone browser or click the "Copy To iPhone" button on the toolbar.

To do the reverse, you can either click the "Copy From iPhone" button on the toolbar to move highlighted files or folders to a location of your choosing on the computer; drag content directly from the MegaPhone browser window into a Finder window or onto the Desktop; or simply double-click a file to download it to your Documents folder.

With MegaPhone, you can also create folders on and delete files or folders from your iPhone.

You can't view content you've transferred from your computer on the iPhone with MegaPhone, however, as the software is for storage purposes only.

Access 160 (And Counting)iPhone Applications

iPhone users should check out www.iphoneappr.com through the SafariWeb browser on their device. They'll find a site that lists and delivers access to 160 (and counting) applications that are iPhone compatible.

The interface is easy to navigate and software is divided by categories, including Business & Finance, Email & Chat, Games, Miscellaneous, Multimedia, News & Sports, School, Search Apps, Shopping, Social Networking and Tools. It also compiles a list of the Top 10 Apps and allows users to search the entire application database. Right now, the only third-party applications that are available for the iPhone run through Safari and are accessible when connected to the Internet—either through Wi-Fi or AT&T's EDGE network.

They do not run on the iPhone itself. Because of this, most of the software listed at www.iphoneappr.com should run on most any smartphone or desktop computer. They've been specifically formatted for the iPhone's display, however.

Typically, a listing offers a short description of the application, a live demo, user ratings, and a button to launch the title on your iPhone. Launching the application means you'll be led to the Web site where the software is being hosted.

Troubleshooting Wi-Fi Connection Problems

Apple has posted a series of tips to help you manage Wi-Fi connections with an iPhone or iPod Touch, which is essentially the same as the former, but without phone features and some other functions.

The first tip offers advice on what to do when you're having trouble connecting with a wireless connection to a paid commercial hotspot, such as the ones available at Borders and Starbucks. It suggests renewing the hotspots DHCP lease. To do that:

1. Tap Settings > Wi-Fi Networks, then pick the network you are trying to connect to.

2. In the DHCP panel, select the Renew Lease button.

If that doesn't work, see iPhone Basic Troubleshooting at http://docs.info.apple.com/article.html?artnum=305740

It then leads into a couple of tips that involve the iPhone's cellular-data EDGE capability.

For example, Apple explains what to do if your Wi-Fi connection keeps reverting to EDGE. That's when your iPhone looks like it is connected to the Internet over Wi-Fi, but goes to EDGE when you try to access a Web page. This can happen when the wireless router is using MAC Address Filtering and the iPhone's MAC address hasn't been entered into the filter list or when you've entered a WEP password wrong.

Here's what Apple suggests:

If MAC Address Filtering is enabled on the wireless router, make sure iPhone's Wi-Fi address (in Settings > General > About) is entered into the router's filter. See the documentation that came with your wireless router for additional information.

If you experience this and use a WEP Password, on the iPhone tap Settings > Wi-Fi. Then tap More Info ( > ) next to the Wi-Fi network name and tap Forget this Network. Then try accessing the Wi-Fi network again. Alternatively, turn off WEP encryption on the wireless router.

Additional tips include what to do if there's a weak iPhone Wi-Fi signal, you receive the "unable to join Network Failure (error -3)," and when there's no Internet access when switching networks.

Everything You Need to Know About Your iPhone Part 2

2008-09-05T09:59:00.000-07:00

Keyboard Tips & Tricks

1. A letter isn't entered until you lift your finger off a key. So if you touch a wrong key by accident when typing, simply slide your finger to the one you meant to type. Let go when the correct letter or symbol appears.

2. While Apple's auto-correction feature is useful, many iPhone users often find themselves accidentally accepting a suggested word when they didn't mean to. There are two things to keep in mind to prevent this from happening:

a) To accept a suggested word you either type a space, punctuation mark, or hit return.

b) To reject the keyboard's correction finish typing the word your want and then tap on the word itself. I know that's a little inconvenient, but it works. Do that twice and the iPhone adds the word to its dictionary.

If you want to start from scratch with the dictionary, if for example it has accepted and is now suggesting a number of wrongly spelled words, you can reset it. To do this go into settings from the home page, hit general, and tap reset. Once there, tap Reset Keyboard Dictionary. This will erase all the words you've added.

3) It's easy enough to capitalize a letter, right? Simply hit the up-arrow (Shift) key next to the letter Z before you tap a letter. But what if you want to capitalize a whole word?

a) To do that you must first go into Settings on the home page, hit General, and then Keyboard.

b) Turn Enable Caps Lock on. Now when you double tap the Shift key before typing, all letters you type will be uppercase. The Shift turns blue in this mode. A single tap will still only cause a single letter to be capitalized.

4) This is also where you can turn off Auto-Capitalization, which is on by default to capitalize the first word of every sentence, off.

5) The keyboard automatically appears when you enter text into a form on a Web page in the iPhone's Safari Web browsers. Conveniently, you can hit the blue Go or Search key, whichever appears depending on the type of form, when you're finished entering the text to submit what you've entered.

Sometimes you may want to close the keyboard before submitting the information in the form. To do this, simply tap the Done key just above the keyboard on the upper-right hand side.

6) If you hold down the E, Y, U, I, O, A, S, L, Z, C, or N key for more than a couple of seconds, the keyboard will offer up anywhere from two to nine additional characters for you to choose from. These additional characters mostly include these letters with various forms of accents.

Everything You Need to Know About Your iPhone Part 1

2008-09-05T09:48:00.000-07:00

Discover the iPhone’s Hidden Goodness
By Damon Brown

In August, Apple started holding classes in its AppleMusic Stores to help people operate the iPhone. Asmany of you know, the iPhone doesn't come with an instruction booklet. Instead, there is a little pamphlet, 12 or so pages, with drawings like something fromIKEA. A National Public Radio program interviewed me about the phenomenon—Apple teaching classes instead of just including instructions like everyone else—but I was never able to attend the classes
myself. Itmay have been for the best, the author of the first book on the iPhone sitting in.

In all honesty, even so-called experts are learning more about the iPhone every day. Apple updates the iTunes and/or iPhone software as often as twice a month, so the capabilities are literally evolving as they are learned. There is plenty of hidden goodness within the iPhone, including tips and tricks that are undocumented and often undiscovered.

Some of these you may already know. All of them are simple.

Silence is Golden

Like many cell devices, the iPhone has a silent/buzz function. Unlike other companies, Apple doesn't actually tell you about it. Take your iPhone and look on the left side. There is a small, all-black switch above the volume buttons.

Flip the switch and a red dot will appear. The iPhone screen will show a bell with a line through it a la Ghostbusters.

Now all calls will buzz, not ring. Flip the switch again and the screen will flash a regular bell, indicating that rings are normal now.

Battery Power

Another area that has not been discussed is battery power.

Some users err on the paranoid side, wary from all those short battery life concerns six months ago, and connect their iPhone to a nearby power outlet as soon as the "20 percent power" warning flashes. (I would be in this category.) The iPhone will warn you twice, at 20 percent and again at 10 percent. It will then shut down. No beep. It will cut off a phone call or any other activity immediately.

The phone will not function at all until it is charges for roughly 15 minutes.

Press any button and a Mac-like set of icons will appear on the screen: the current battery power followed by a power plug and an arrow pointing to a lightning bolt. Expect to see this screen for about an hour.

Call it poor design, but once the iPhone juice runs out, it requires a long time to power up again. (Connecting it to the computer doesn't help, either.) Consider investing in a car charger if you're planning a long road trip.

Music @Home

There is "Home", the square button located at the bottom of the iPhone's face. Its main purpose is to take you back to the default menu screen, but it can also make controlling your music easier.

Let's say you are listening to one of your favorite songs on the iPhone, for instance, James Brown's classic "Funky Drummer." Press the sleep button at the top of the iPhone to make the screen lock. It will say, "slide to unlock" at the bottom, along with the current time and song. Now double tap the "Home" button. The music controls will appear immediately under the time. Press rewind, fast-forward or pause as you would normally do, or use your finger to adjust the volume by sliding the silver ball along the blue line below. A similar "Home" technique can be used on the main menu screen. Press the "Home" button to get to the main screen. Now tap "Home" again. Music info will pop up, including performer, song and album, as well as the volume ball. Unlike the previous mode, you remove the information — by hitting the onscreen Close icon — or move to the actual iPod menu -- by tapping the onscreen iPod button.

iPod Mode

A simple, often neglected music control is actually in the iPod mode. Press the iPod icon on the main menu, find a song and press play. The album art will appear as the music starts. (If it has no art, a grey music note will appear instead.) Now tap the center of the album art. A nice list of options will pop up at the top of the art. The first icon is a loop. Press the loop once and the current playlist or album will repeat. The icon will turn blue. Tap it again and a small "1" will appear on the bottom-left corner of the loop. Now the current song will repeat.

In the middle is the current song number within the current album or playlist (four out of 14, for instance). Above the song number is the song time elapsed, song time remaining and a silver-and-blue line identical to the other menus. However, the line here represents the song track. Move the ball with your finger to rewind or fast-forward the song.

The last icon is two arrows twisted together. Tap it once to randomize the current album or playlist.

Last But Not least

As a final hidden goody, look in the upper-right hand corner when in iPod mode. You'll see a series of three lines. Tap it.

The icon will "flip," turning into a miniature version of the current album art, and the large album art will turn into a list of the current album songs. (It will always list the current album, even if you are playing from a playlist.)

Click on another song to hear it. The list icon is always available in iPod mode.

Managing the Complexity of SOAs for Success

2008-09-05T09:16:00.000-07:00

Imagine you recently completed the first successful phase of your "Enterprise SOA Master Plan." Services are distributed throughout the enterprise, linking legacy applications with your ERP and CRM applications, and are presented to your business users via a new portal interface.

The users love the ease with which they can access all this information and conduct their daily tasks. Now that they know what is possible, they want more and you are ready to give them what they want. After all, you have just proven that SOA is more than a mere buzzword.

However, no one has yet told you there is a problem.

This initial, highly successful project has introduced a new level of complexity into your IT organization. Monolithic and heterogeneous applications running in isolation for many years have suddenly become crucial components in a more complex organism held together by the principles of service-oriented architectures (SOAs).

The IT architects have very detailed architectural diagrams to describe it all, the developers wrote thousands of lines of code to make it all work, and the integration team has configured a complex web of message infrastructure to ensure that every message is guaranteed to be delivered to the right application. What could potentially be amiss in this picture?

Let's pose a few questions:

• How many services are currently deployed within the organization?

• Of these services, how many are currently up and running and how many are experiencing problems?

• How well does every service perform its given task? Are your end-users waiting longer to achieve a given task?

• How stable are these services? How are some of these newly created business processes and applications affected when a crucial service (or services) malfunctions?

• Can you conduct an impact analysis to measure the potential impact of changes to an existing service or set of services?

By using the principles of SOA you have been able to solve a complex problem and by doing so introduced additional complexity into your IT infrastructure. The challenge that you now have to face is how to manage that complexity. SOA Governance Approach SOA Governance is a discipline that, in an ideal world, should be established early on in the SOA life cycle. Yet, it is never too late to implement good policies, procedures, and technology to assist with the management of a freshly implemented SOA project.

Consider governance a "best practice" whereby one transforms lessons learned into sound policies, guidelines, and procedures with the aim of establishing a mature SOA ecosystem.

Here are five guidelines considered essential to good governance:

1. Establish an SOA Competency Center

This is a cross-functional team of key people that represent different disciplines within IT. By establishing such a group you facilitate communication between the various groups, ensuring that everyone is represented and that every aspect of the SOA architecture gets proper consideration.

Furthermore this group can be instrumental in establishing organizational standards, setting guidelines and creating blueprints. This core group can also disseminate knowledge to new groups that need to adopt the SOA approach.

2. Experience is a Good Teacher

Blueprints and best practices can be very good teaching mechanisms. They are a tangible way in which everyone involved in the SOA implementation can benefit from the work that has been done before.

Too many times development teams have to "reinvent the wheel" in order to solve a particular problem. Identify and document those solutions that work well and turn them into patterns that can be used repeatedly.

By making a particular approach repeatable you are able to reduce risk, shorten development time and also decrease the cost associated with the overall project. Likewise, it may be just as important to document those approaches that don't work well or which may have failed. By documenting these so-called anti-patterns one is able to prevent mistakes from being repeated.

3. Use an SOA Repository

Services are by definition reusable and self-describing, but don't be deceived by this oversimplified view.

If services are not properly documented and made accessible in a consistent manner, they could become just as unusable as some of those legacy applications.

Accomplishing such simple tasks as establishing the exact nature of a service, obtaining the latest service description, or determining the owner of a particular service may become quite a daunting task if services management infrastructure is not put into place. This is where an SOA repository can play an important role. It can become the focal point where all services can be documented, searched, and accessed.

4. Monitor Quality of Service (QoS)

Bring predictability to your SOA by implementing service management facilities. What you need is the capability to maintain end-to-end visibility into your services infrastructure via the ability to define SLAs (service level agreements), measure services operation and response levels, analyze service usage and resource utilization patterns, and pro actively be alerted of potential and imminent service failures.

Stability is the hallmark of a mature system and through the effective management of your deployed services (and related infrastructure) you can bring a great measure of stability to your SOA.

5. Manage the SOA Life cycle

Create a disciplined and well-managed SOA life cycle. In all other areas of IT this kind of discipline is well established and maintained. SOA should be no different.

Implement tools and procedures that will help manage the services development, testing, quality control, deployment, and maintenance processes.

As part of the life cycle, create a services approval process, ensuring that services are tested for standards compliance and services are put through a comprehensive vulnerability, interoperability, and regression testing process. Implementing these measures will assist in elevating the overall maturity and robustness of your SOA implementation.

Implementing an SOA entails more than just implementing a few services or using an ESB (enterprise service bus). SOAs may start out simple, but they inherently introduce complexity into the IT organization. This is mainly due to the nature of the problems that we are attempting to solve.

Acknowledging the complex nature of SOA and taking the appropriate steps to manage this complexity is a crucial step toward the successful development, implementation, and maintenance of an SOA. The best advice is to plan for SOA success. Plan to be hugely successful and in preparation, consider how you will govern the next generation of business applications.

This content was adapted from EarthWeb's Developer.com and CIO Update Web sites. Contributors: Theo Beack, Allen Bernard, Enrique Castro-Leon, Arulazi Dhesiaseelan, Kishore Channabasavaiah, Kerrie Holley, Edward M. Tuggle, Jr.

Staffing for SOA Success

2008-09-05T08:59:00.000-07:00

Just as SOA is changing the way business and IT interrelate, it is also about to transform the types of people who work in IT.

This is because SOAs basically transform the IT department from a transaction-oriented cost-center to a change agent. SOAs do this by making business leaner and more agile whilst simultaneously cutting costs and wringing ever more value from existing infrastructure through the reuse of past investments.

But making this happen requires looking at IT and the business it serves in more holistic ways. And this, although changing from days past, is not exactly IT's long suit.

"To make SOA really successful it can't be about just building technology, it's got to be about solving business problems," says BEA Systems' Paul Patrick, who is vice president and chief architect of its SOA infrastructure product, AquaLogic.

From a technology perspective, SOA is comparatively straightforward. If you have a mature IT department with some depth of skills, then the technological portion of your SOA implementation - wrapping data in XML, setting up SOAP calls for Web services, setting up your registry and repository, etc. - should come pretty natural.

The hard part is seeing things from a big-picture perspective and, unfortunately for those who will have to deal with it, the politics of moving your line-of-business users used to consuming dedicated IT services towards a model that, at its heart, is all about sharing.

"The most important skill, which is going to be new to IT professionals, is they're going to have to shift their thinking and their ability from being very technology and product-focused to becoming much more (business) process-centric," says Marianne Hedin, a programme manager at IDC.

This is where new blood may be needed to get your SOA initiative off the ground and help make it successful.

Key to this effort is having a systems' architect on an SOA team who is specifically challenged with tackling the larger, enterprise-wide issues that will inevitably come up, including:

• Who is going to have priority if shared services get bottlenecked?

• Whose budget pays for new hardware and software if the services are going to be rolled out company wide?

• How will the SOA affect the business as a whole?

• Who's responsible for re-organizing IT into a service-delivery business?

"It's in many ways a micro view of the problems sitting at Homeland Security where you've got 17 distinct agencies all trying to act as if they are one big thing, but their culture has been, 'I've got my world, you've got yours. Why should we share?' That's the hidden problem," says BEA's Patrick.

The ideal solution, is to set up a team led by the CIO that includes a business analyst who understands both IT and the business processes it supports (not any easy person to find by all accounts); a system's architect that can turn business concepts into IT-speak and ride herd over the infrastructure in order to minimize unintended consequences; and representatives from each of the company's lines of business.

This last piece of the puzzle is particularly important to a successful SOA transition, claims Chris Warner, Software AG's director of Technical Marketing and a member of SAG's SOA competency center. Without executive buy-in and sponsorship of the changes wrought by SOA, it will most likely, at least in part, fail.

"It's kind of important that you have an executive sponsor for any new initiative," he says. "In the case of SOA, the way that works best is if you can tie that initiative to something the executives already care about. In other words, not SOA for SOA's sake."

Other team members can come and go, such as programmers to explain different concepts, but the core members should be charged with smoothing the feathers sure to be ruffled by the wholesale changes SOA can bring and making sure that whatever individual processes are served up by your SOA do not implode the entire system.

"A lot of people think SOA and they just think just technology," says BEA's Patrick. "Our experience has been that if all you do is take a technology approach to this problem … you can do a really great job and quick hit a wall."

SOAs to Lower Legacy Costs and Free Up Manpower

2008-09-05T08:53:00.000-07:00

Why adopt service SOA in the first place? The main reasons are eminently practical and driven by economics because SOA adoption represents a clear path for IT organizations to reduce cost and to improve operational efficiency whilst bringing in agility and competitiveness for their companies.

Let's look at the fundamental problem today. Let's take as an example a typical Fortune 500 corporation. This company might gross £20.5 billion a year and have an IT organization with a budget of about £563 million and about 6,000 employees.

Out of that budget, a figure of merit is the breakdown between dollars spent sustaining existing services ("legacy" costs) versus dollars spent in new services. The dollars in the second category are the ones associated with growth and business innovation.

Studies from Gartner and Aberdeen put the fraction dedicated to legacy anywhere between 70 and 80 per cent, depending on the company with the lower numbers associated with the more progressive companies. Because of competitive pressures, and in spite of the economic recovery that's going into the fifth year, typical IT budgets have remained flat or growing slowly, whilst the cost of legacy tends to grow with the company. Left unchecked, the cost of legacy will overwhelm the IT budget.

One strategy to lower the cost of legacy is through outsourcing/off shoring and the ensuing cost arbitrage. The arbitrage can be geographic and taking advantage of lower salary scales in some countries, or through division of labor, where an efficient, specialized firm delivers
a service such as payroll at a lower cost than the in sourced cost.

Off shoring entails a learning curve and cost bump where the transition negotiations take place and consultants from the service provider are brought in. There is an initial cost decrease after the transition due to staffing reductions.

Although lower initially, legacy cost follows a steeper slope because salaries in Bangalore are growing much faster than salaries in the United States and Europe. Eventually, a new stasis is reached when the cost of the outsourced service plus overhead reaches parity with the in-sourced cost.

And not every service can be outsourced. Outsourcing a company's core activities might lead to loss of intellectual assets and collective knowledge that will limit an organization's growth potential. Outsourcing could also impact customer satisfaction, quality-of-service and employee morale that might hurt the organization in the long term.

Technology refreshes provide an alternative. These refreshes have a deflationary effect. A strategic emphasis to aggressively adopt emerging technologies will lead to reductions in capital outlays and cost of labor. Every refresh slides the cost line down.

These transitions are not without risk and must be managed carefully. However, the rewards can be very attractive. For instance, investing in server consolidation increases a system's capability to handle bigger workloads with minimal data centre new investment.

SOA & Legacy

SOA adoption brings more fundamental change, lowering the legacy curve and its growth. Moreover, SOA does not exclude outsourcing and technology adoption. In fact, it might provide a more rational framework for their application, with clean interfaces to mix and match in-house and outsourced composable services with feedback based on business outcomes.

The adoption of SOA becomes attractive if it can be shown that it leads to a structural and permanent reduction of the current 70% legacy cost by 10 to 20 per cent. Through elimination of redundancy and breaking silos, SOAs should bring increased operational simplicity allowing dialing cost increases to a sustainable rate.

The elimination of redundancy and a 15% increase in efficiency would be equivalent to hiring 1,000 staff without actually increasing headcount, accomplished through the use of standardized platforms, simplified operating environments.

The adoption of SOA won't be without pain. Many job descriptions will change. An overall strategy requires an EA governance structure, consolidation of projects into fewer and deeper activities, and more consumption of reusable components. This transition will take time.

The cost curve will continue running its course for a time, and eventually tapers off as an increasing proportion of applications are brought up under a SOA framework.

When the processes have been institutionalized, this 1,000 headcount truly represents the resource that gets freed up from legacy work to contribute to an organization's growth.

Not Just Web Services

2008-09-05T08:48:00.000-07:00

The success of many Web services projects has shown that the technology does in fact exist whereby you can implement a true service-oriented architecture. It lets you take another step back and not just examine your application architecture, but the basic business problems you are trying to solve. From a business perspective, it's no longer a technology problem; it is a matter of developing an application architecture and framework within which business problems can be defined and solutions can be implemented in a coherent, repeatable way.

First, though, it must be understood that Web services and a service-oriented architecture are not the same thing. Web services are a collection of technologies -- including XML, SOAP, WSDL, and UDDI -- which let you build programming solutions for specific messaging and application integration problems. Over time, you can reasonably expect these technologies to mature, and eventually be replaced with better, more efficient, or more robust ones, but for the moment, they will do. They are, at the very least, a proof of concept that SOAs can finally be implemented. So what actually does constitute a service-oriented architecture?

SOA is just that, an architecture. It is more than any particular set of technologies, such as Web services; it transcends them, and, in a perfect world, is totally independent of them. Within a business environment, a pure architectural definition of a SOA might be something like "an application architecture within which all functions are defined as independent services with well-defined invokable interfaces which can be called in defined sequences to form business processes." Note what is being said here:

1. All functions are defined as services. This includes purely business functions, business transactions composed of lower-level functions, and system service functions.

2. All services are independent. They operate as "black boxes." External components neither know nor care how they perform their function, merely that they return the expected result.

3. In the most general sense, the interfaces are invokable; that is, at an architectural level, it is irrelevant whether they are local (within the system) or remote (external to the immediate system), what interconnect scheme or protocol is used to effect the invocation, or what infrastructure components are required to make the connection. The service may be within the same application, or in a different address space within an asymmetric multiprocessor, on a completely different system within the corporate intranet, or within an application in a partner's system used in a B2B configuration.

In all this, the interface is the key, and is the focus of the calling application. It defines the required parameters and the nature of the result; thus, it defines the nature of the service, not the technology used to implement it. It is the system's responsibility to effect and manage the invocation of the service, not the calling application. This allows two critical characteristics to be realized: first, that the services are truly independent, and second, that they can be managed. Management includes many functions, including:

1. Security: authorization of the request, encryption and decryption as required, validation, and so forth.

2. Deployment: allowing the service to be redeployed (moved) around the network for performance, redundancy for availability, or other reasons.

3. Logging: for auditing, metering, and so on.

4. Dynamic rerouting: for fail over or load balancing

5. Maintenance: management of new versions of the service

Requirements for a Service-Oriented Architecture

2008-09-05T08:46:00.000-07:00

1. First and foremost, leverage existing assets. Existing systems can rarely be thrown away, and often contain within them great value to the enterprise. Strategically, the objective is to build a new architecture that will yield all the value hoped for, but tactically, the existing systems must be integrated such that, over time, they can be componentised or replaced in manageable, incremental projects.

2. Support all required types or "styles" of integration. This includes:

• User Interaction: being able to provide a single, interactive user experience

• Application Connectivity: communications layer that underlies all of the architecture

• Process Integration: choreographs applications and services

• Information Integration: federates and moves the enterprise data

• Build to Integrate: builds and deploys new applications and services.

3. Allow for incremental implementations and migration of assets. This will enable one of the most critical aspects of developing the architecture: the ability to produce incremental ROI. Countless integration projects have failed due to their complexity, cost, and unworkable implementation schedules.

4. Include a development environment that will be built around a standard component framework, promote better reuse of modules and systems, allow legacy assets to be migrated to the framework, and allow for the timely implementation of new technologies.

5. Allow implementation of new computing models -- specifically, new portal-based client models, grid computing, and on-demand computing.

Planning a Service-Oriented Architecture

2008-09-05T08:44:00.000-07:00

Over the last four decades, software architectures have attempted to deal with increasing levels of software complexity. But the level of complexity continues to increase and traditional architectures seem to be reaching the limit of their ability to deal with the problem.

At the same time, the traditional needs of IT organizations persist, such as the need to respond quickly to new requirements of the business, the need to continually reduce the cost of IT to the business, and the ability to absorb and integrate new business partners and new customer sets, to name a few. As an industry, we have gone through multiple computing architectures designed to allow fully distributed processing; programming languages designed to run on any platform, greatly reducing implementation schedules; and a myriad of connectivity products designed to allow better and faster integration of applications. However, the complete solution continues to elude us.

Service Oriented Architecture (SOA) is seen as the next evolutionary step in software architecture to help IT organizations meet their ever more complex set of challenges. According to IBM, the SOA market nearly doubled in 2005 to more than £2 billion and could top £7 billion by 2009.

The W3C Web Services Architecture Working Group defines SOA as a form of distributed systems architecture that is typically characterized by the following properties:

• Logical view: The service is an abstracted, logical view of actual programmes, databases,business processes, and the like, defined in terms of what it does, typically carrying out a business level operation.

• Message orientation: The service is formally defined in terms of the messages exchanged between provider agents and requester agents, and not the properties of the agents themselves. The internal structure of an agent -- including features such as its implementation language, process structure, and even database structure -- are deliberately abstracted away in the SOA. By using the SOA discipline, one does not and should not need to know how an agent implementing a service is constructed. A key benefit of this concerns so-called legacy systems. By avoiding any knowledge of the internal structure of an agent, one can incorporate any software component or application that can be "wrapped" in message handling code that allows it to adhere to the formal service definition.

• Description orientation: A service is described by machine-processable meta data. The description supports the public nature of the SOA: Only those details that are exposed to the public and important for the use of the service should be included in the description. The semantics of a service should be documented, either directly or indirectly, by its description.

• Granularity: Services tend to use a small number of operations with relatively large and complex messages.

• Network orientation: Services tend to be oriented toward use over a network, though this is not an absolute requirement.

• Platform neutral: Messages are sent in a platform neutral, standardized format delivered through the interfaces. XML is the most obvious format that meets this constraint.

A service-oriented architecture allows for software systems to be designed that provide services to other applications through published and discoverable interfaces, and where the services can be invoked over a network. When you implement a service-oriented architecture using Web services technologies, you create a new way of building applications within a more powerful and flexible programming model. Development and ownership costs as well as implementation risks are reduced. SOA is an architecture and a programming model, a way of thinking about building software.

Virtualization Enters the SMB World

2008-08-29T07:47:00.000-07:00

Virtualization is the latest buzzword steamrolling across the IT landscape, influencing every computer roomin its path. Everyone is doing it, according to analysts. Everybody needs it, say the vendors. But does it really add any value to the small business? As usual the answer varies fromcompany to company.

If you own or manage a small shop with a handful of servers and a bookkeeper that doubles as your IT guy two hours per day, don't even bother reading the rest of this article. For those of you with more substantial computing needs, who seem to have way too many servers or skyrocketing IT costs, virtualization may offer some relief.

"Companies with IT teams of one to four people or firms with 60 or fewer employees often don't have the level of sophistication required to make virtualization pay," said Chip Nickolett, president of Comprehensive Consulting Solutions that has helped several small businesses take their first steps into the virtual world. "In these environments, the benefit of virtualization would be marginal at best."

These are not hard-and-fast numbers, of course. Tiny firms in financial services, law and other sectors sometimes have both significant IT requirements, the budget and the staff required to make virtualization pay big dividends.

Understanding the V Word

But let's start with a statement to clarify what virtualization is in simple terms — not as easy a task as it sounds. You have to speak to the geeks in order to get a definition and it's not in their DNA to use laymen's terms. So what does the V word mean?

"Virtualization enables one server or computer to act as many," said Dan Chu, vice president of emerging products and markets at VMware. "Instead of keeping your important programs on separate servers so that if one application or server fails, the other applications aren't affected, virtualization software lets you run many applications on the same server."

In such a scenario, you actually have one server sitting on the floor, but it acts as though it were several servers. Virtualization software enables that server to be split up into different partitions.

For example, one server could act as three virtual servers with each virtual server running an application (e.g., file server, Web server, and e-mail server). Each virtual server acts completely independently from one another, so if one crashes, the others are not affected. The net result is that you have to buy only one server and pay for its power consumption. You get the benefit of three servers for the cost of one.

"The way that I explain it to people is that virtualization is a way to make an environment portable," said Nickolett.

This means software can be easily relocated to a larger or smaller machine or even moved from one operating system to another. This is accomplished by splitting one physical server into numerous virtual servers or virtual machines (VM). Each VM hosts a specific application or set of software. As everything is virtual, it is easy to move the VMs around and make changes in the IT environment.

Some small businesses embark on the virtualization journey as a way to simplify disaster recovery. Typically, they just went through their first disaster and recovering their systems was a nightmare. Finding the right back up tapes, hooking them up to new hardware, and finding all sorts problems — like not having a record of their software licenses and not being able to find the original CDs for their operating systems and programs. They have to go through the laborious task of reinstalling and re-configuring everything and making it work on the new hardware and then figuring out how to get the data from the tapes back into the systems.
This process can take many days if you're not familiar with it.

Virtualization, on the other hand, makes it relatively easy to capture everything onto a single system image, which makes recovery a snap. "A virtual image makes recovery or failover faster, easier and more foolproof," said Nickolett. "Like anything, it requires planning and
testing, but it can be an attractive alternative for some businesses."

Blossoming Trend

There is no doubt that virtualization is catching on like wildfire. Just about every large and midsize firm is already doing it extensively, and now it is percolating down into the small business strata.

"Virtualization is exploding in popularity," said Jim Smith, a performance specialist at TeamQuest Corp. "Virtual machine deployments are expected to grow from 540,000 in 2006 to more than four million 2009."

He cautions, though, that although the benefits are widely advertised, the complexities have not been comprehensively discussed. VMs add a whole new layer of administration to IT. If you're already well schooled in IT complexity, fine. But for companies still coming to terms with internal networking or hooking up servers to storage arrays, virtualization is going to mean the addition of a highly paid specialist into the fold. So it's by no means a must-have technology for many smaller organizations.

Smith makes the point that a good reason to use virtualization is to improve the utilization rate of hardware — i.e., how much processing power your server uses to run the application. Many companies, for example, buy a server for every application they run. But you can end up with dozens of servers on the floor, most of which are very poorly utilized.

What this adds up to is that you have a hefty power and cooling bill but aren't getting much return on the money. Low utilization means computers aren't being used to their limits, and that represents an awful lot of inefficiency.

"When people look, they are often shocked to find that many servers are running at utilization levels of less than 12 percent," said Smith. "Since 9/11, however, the tide has been turning and the ongoing trend is to maximize utilization rates. And server virtualization certainly plays a big part in solving this problem."

This brings many other advantages to the IT world. Servers can now be deployed faster. Instead of hours or days, it can be done with a virtual machine within the hour. Other benefits include a reduction in the amount of space required for computers. That in turn leads to lower costs for ventilation, electricity and cooling.

Each vendor, of course, advocates its own virtualization schemes, and the various approaches can be quite confusing. For the purposes of this article, we will discuss only the options that small businesses would likely encounter.

VMware

VMware is the darling of the marketplace. Just about every company of any size engages some form of VMware deployment. VMware ESX Server is softwarebased virtualization solution that facilitates hardware sharing. It makes it possible to have a powerful processor shared by multiple virtual machines behaving as though they were completely separate servers.

Microsoft Windows Virtual Server

Windows Virtual Server (WVS) is also software-based, and like VMware, it lets you share hardware resources such as memory and CPUs.

Let's move this over into one possible scenario. HP offers a product for small businesses named the HP c3000 (also known by its nickname, Shorty). This is, in essence, just a chassis or enclosure to hold blade servers (thin, streamlined servers).

You can buy a c3000 enclosure for with power supplies, fans, and management software for less than $5,000. It can hold two-to-eight server blades, which range between $2,100 and $5,000 each, depending on processor, memory and configuration. There are some additional costs for storage and networking. Depending on the mix of devices, a Shorty enclosure may cost between $7,000 and $45,000.

This hardware from HP supports VMware, WVS and other virtualization solutions. By consolidating all of IT into a couple of these boxes, it is possible to establish
a powerful virtual world composed of scores of virtual servers. Now set up another such box at a remote location and disaster recovery and you simplify backup tremendously.

"We have customers using the c3000 for virtualization projects," said Barry Sinclair, product manager for HP c3000. "One small business customer has four enclosures (two in each of two sites in a virtualized environment, and it is handling disaster recovery scenarios between sites."

Let's end by looking at how one small business benefits from virtualization. The Los Angeles Universal Preschool (LAUP) is non-profit with a goal to make voluntary, highquality preschool available to every four year old in Los Angeles County. It has several physical servers running VMware. Each physical server represents 15 to 20 virtual machines.

"The cost of purchasing physical servers would have easily run over $100,000," said Robert Lazo, director of systems and operations at LAUP. "VMware technology allowed us to avoid that expense."

Other benefits reported by Lazo include being able to set up a new server in less than five minutes. Such a task would have taken many hours previously.

LAUP, however, has a well-established IT staff of seven to service 150 employees. And that's probably the make-break point of virtualization — it's great if you have clued-in people who are coping with IT headaches on a daily basis. But if your business is coping fine without high-level computing expertise, it's probably safe to give virtualization a pass.

Virtual Servers Update: VMware vs. Microsoft vs. Xen

2008-08-29T07:26:00.000-07:00

What a difference a year makes with the whirlwind of the server virtualization world. New cross-platform management tools, embedded hypervisors, wider acceptance of open source methods, protocols, standards, and simplified pricing have all made virtualization much more popular with IT managers.

While the market is growing, it still represents a minor portion of the entire server marketplace – less than 10 percent, according to Microsoft representatives. What is new is that virtual machine (VM) server technology is now available and more attractive to mid-tier users for four reasons:

• The free versions are more capable.

• Prices are coming down.

• Ease of setup and management is increasing.

• The technology can help reduce power and cooling requirements just as being green is gaining traction.

Nevertheless, virtual servers are just one part of the entire virtualization market, which is growing to include all kinds of computing, from storage virtualization to streaming applications installation, to virtual desktops. But in the past year, four trends are obvious:

1) Growth of the hypervisor: The hypervisor is now found in more places, both exploited in the latest processor chips from Intel's Virtualization Technology vPro and AMD-V, and as a standard package with most of the popular Linux distributions and soon for Solaris too. The hypervisors, or virtual machine control programs, for the three major vendors (Microsoft, Citrix and VMware) now support this embedded hardware, which makes for simplified installation and nearly onebutton booting of virtual servers. And VMware has begun selling ESXi, a specialized embedded version that will begin shipping on servers imminently. HP's ProLiant servers now offer built-in support for Citrix' XenServer; older ProLiants can be upgraded too.

2) Interoperability: Interoperability has taken root, and we have seen in the past year a series of initiatives to make managing multiple VM vendors more palatable. Novell's ZenWorks VM Manager and Orchestrator products are just from one of many products that will offer a way to manage more than one vendor's hypervisor. Microsoft's SystemCenter, CA and others have announced plans to support both Microsoft's and VMware hypervisors, and Novell will also support Citrix's solution too. VMware announced several management tools that enable automation of the entire lifecycle of a VM, including staging the migration from a development/test environment into production, according to Bogomil Balkansky, the Senior Director of Product Marketing for the company. "Our customers tend to want to do more with virtual servers once they get it into their shops."

Another dimension to the interoperability story is a standards effort called the open VM format that is first expected to be finished sometime early summer. "With this format, organizations can use a standard set of VM management metadata to manage VMs running on different hypervisors. This architecture is fully extensible, allowing VMs to advertise custom configuration information, such as a virtual barcode, security requirements, or service level requirements," says Chris Wolf, a senior analyst with the Burton Group.

"While work remains, the eventual goal of these standards is to provide hypervisor interoperability, such as by taking a VM image built on the Microsoft Hyper-V hypervisor and running it on a Citrix XenServer hypervisor without having to modify the VM's configuration."

And as another example of increased manageability, inventory and asset management vendors such as BDNA have tools that can account for individual VMs that are hosted on virtual servers when they discover server resources across an enterprise. "This is a strong sign that the market is maturing and that customers have a choice," says Simon Crosby, the co-founder of XenSource and now the CTO of the division for Citrix.

3) Falling Prices and Improved Functionality: Prices are coming down and functionality for even the free versions is improving. The free products – and indeed, all of Microsoft's virtual server line – continue to be a great way for enterprises to become familiar with VM technology and to do any evaluations before deploying them into production. Most noticeably Microsoft has announced they will expand their product line with Hyper-V, which will be included in all 64-bit versions of its Windows Server 2008, expected in August. Hyper-V ups the ante considerably, with support for symmetrical multi-processors and larger memory support.

On the paid products, XenServer continues to be the lower-priced spread, offering single-CPU versions and better value when compared to VMware. The latter's prices are now almost comprehendible, an improvement from their obscure complexity of last year. VMware also introduced support for 10 gigabit Ethernet networks and larger memory and disk support with its latest version, and now has more than 700 pre-built virtual "appliances" or virtual disk images that are available as well.

Citrix hasn't stood still either, and boosted the performance of XenServer since acquiring the company last year, especially when it comes to XenServer working with the company's flagship Presentation Server product line. "We recognize that our customers want to run both products to solve dynamic data center problems," says Crosby.

4) Widening Channels: The virtual server channel continues to widen, with more partnerships, agreements, and expertise than ever before. As smaller, specialty companies enter this market, they are looking to cement relationships, expand distribution, and make just about every component in the data center virtualized. "All of the services that do hardware and applications failover, disaster recovery, chargeback, and security will be built into hypervisors and run on VMs," says Susan Davis, the VP of marketing for Egenera, one of the newer specialty virtual software vendors.

"This year is shaping up to be one of the most interesting years ever in enterprise IT infrastructure," says Crosby.

Storage Virtualization Plays Catch Up

2008-08-29T07:21:00.000-07:00

While server virtualization rolls onward with seemingly unstoppable momentum, storage virtualization lags behind. That may be changing, as storage virtualization moves forward on two distinct fronts — block-level and file-level virtualization.

The demand to eliminate disruption from IT operations is driving block-level virtualization. Taking the SAN down on weekends to perform maintenance or move data is no longer acceptable.

"Some might be surprised that a key driver of this is planned downtime," said Doc D'Errico, vice president and general manager of the infrastructure software group at EMC. "While most people understand unplanned events such as natural and man-made disasters, planed downtime — like schedule maintenance, data migrations, lease roll-overs, or technology refreshes — account for 60 percent to 75 percent of all downtime."

To complicate this, most enterprise-class infrastructures typically include multi-vendor server environments, diverse connectivity technologies, and multi-While server virtualization rolls onward with seemingly unstoppable momentum, storage virtualization lags behind. That may be changing, as storage virtualization moves forward on two distinct fronts — block-level and file-level virtualization. The demand to eliminate disruption from IT operations is driving block-level virtualization. Taking the SAN down on weekends to perform maintenance or move data is no longer acceptable. "Some might be surprised that a key driver of this is planned downtime," said Doc D'Errico, vice president and general manager of the infrastructure software group at EMC. "While most people understand unplanned events such as natural and man-made disasters, planed downtime — like schedule maintenance, data migrations, lease roll-overs, or technology refreshes — account for 60 percent to 75 percent of all downtime." To complicate this, most enterprise-class infrastructures typically include multi-vendor server environments, diverse connectivity technologies, and multivendor tiered storage environments. Organizations must be able to allocate any storage to any application based on the needs of the business, and they must be able to do so non-disruptively. Enter storage
virtualization — to deliver the right information at the right performance level with the right functionality to the business at the lowest total cost.

"Without storage virtualization, host servers must be individually mapped to physical arrays in a many-to-many or server-to-array configuration," said D'Errico. "Administrators are forced to tier their infrastructure manually."

If a company's financial data, for example, must be kept on Tier 1 storage and e-mail on Tier 2, then the servers running these applications must be manually mapped to an appropriate physical array. With storage virtualization, administrators can instead map all servers to a single endpoint like the EMC Invista virtualization application for SANs; no need to manually touch each physical array. The result is much simpler SAN administration and easier interoperability between the multiple data center components.

Other vendors offering block-level virtualization include IBM SVC from IBM, TagmaStore USP from Hitachi Data Systems, and SANmelody from DataCore Software.

DataCore has developed a series of what it calls Feature-Packaged Virtual Storage Solutions. These run on virtualization platforms, such as VMware, Microsoft VS, Oracle VM, Sun xVM, Virtual Iron and Citrix XenServer. They support anywhere from 2 TB to 32 TB. Pricing starts at $4,500.

"They can thin-provision storage capacity, migrate data, accelerate storage performance and create high-speed disk copies for fast disk backup and recovery," said George Teixeira, president and CEO of DataCore.

File Virtualization Gains More FANs

File virtualization deals mainly with the virtualization of the files stored on NAS boxes, storage servers, and file servers. File virtualization is also known under the term "file-area network" or FAN. A FAN is a way to aggregate file systems so they can be moved easier and managed centrally via a logical layer known as a global namespace. The benefits are easier server administration, file reorganization, and consolidation. Files can be moved without the user being aware that they may now physically reside in a completely different location.

"A file area network consists of a collection of network-attached storage appliances and file servers that are virtualized and for which the data on them can be managed under a single file system," said Deni Connor, an analyst with Storage Strategies Now. "As NAS appliances proliferate, the management of them becomes more complex. Rather than managing each NAS device by itself, combining them into a FAN, allows them to be managed collectively."

Some vendors, such as Acopia (acquired by F5 Networks), implement FANs as hardware. Others take a software approach (although they are sometimes delivered within an appliance), such as StorageX by Brocade Communications Systems. StorageX works in conjunction with the Microsoft Distributed File System (DFS) protocol, which maps logical physical devices to logical storage and replicates data across WAN links.

StorageX extends the functionality of DFS by adding global namespace capabilities and simplifying data management at remote locations.

"Without virtualization, you can have one server being fully utilized while others sit idle, and it is cumbersome to move files, reconfigure file systems or make them available elsewhere in the event of a disaster," said Philippe Nicolas, technology evangelist for FAN solutions at Brocade. "Such issues are solved by adding a logical layer (via a global namespace) between clients and file systems."

According to a recent study by New York City-based analyst firm TheInfoPro, however, EMC Rainfinity is the No. 1 file virtualization technology. Rainfinity virtualizes unstructured data environments and moves data (including active, open files) without disruption. It can be deployed as either software or an appliance.

"Rainfinity continues to accelerate its penetration into enterprises globally on the strength of its file virtualization capabilities for multi-vendor NAS environments and is currently being used to virtualize petabytes of customer information in a wide range of industries and operating environments," said D'Errico.

Virtual Problems

Like everything else, however, you can get too much of a good thing. With so much server virtualization happening, and now two different categories of storage virtualization being deployed in storage environments, it stands to reason that integration must be addressed.

"Virtualization will become increasingly important both for storage and servers," said Mike Karp, an analyst at Enterprise Management Associates. "The challenge here will be managing across the virtual interfaces — the abstraction layers that separate the physical devices from the management. It is necessary to manage storage within the context of its connection with application servers."

Interestingly, there is a developing field that addresses the management of multiple virtualization technologies. Known as enterprise virtualization, it encompasses servers and storage.

"IT managers are increasingly considering the prospect of a fully virtualized data center infrastructure," said Emulex's McIntyre. "There is a high degree of affinity between SANs and server virtualization because the connectivity offered by a SAN simplifies the deployment and migration of virtual machines."

Hardware for Virtualization: Do’s and Don’ts

2008-08-29T07:03:00.000-07:00

Virtualization is catching on like never before. Just about every server vendor is advocating it heavily, and IT departments worldwide are buying into the technology in ever-increasing numbers.

"The use of virtualization in the mainstream is now relatively commonplace, rather than just in development and test," said Clive Longbottom, an analyst at U.K.- based Quocirca. "In addition, business continuity based on long-distance virtualization is being seen more often."

As a result, the time has come to more closely align hardware purchasing with virtualization deployment. So what are some of the important do's and don'ts of buying servers and other hardware for a virtual data center infrastructure? What questions should IT managers ask before they make selection decisions on servers? And how should storage virtualization gear be integrated into the data center?

Do’s and Don’ts

There are, of course, plenty of ways to virtualize, depending on the applications being addressed. This article will focus on a typical case where infrastructure and business logic applications are the main targets.

With that in mind, one obvious target is memory. It is a smart policy to buy larger servers that hold more memory to get the best return on investment. While single- and dual-processor systems can host multiple applications under normal circumstances, problems arise when two or more hit peak usage periods.

"Our field experience has shown that you can host more VMs [virtual machines] per processor and drive higher overall utilization on the server if there are more resources within the physical system," said Jay Bretzmann, worldwide marketing manager, System x at IBM. "VMware's code permits dynamic load balancing across the unused processor resources allocated to separate virtual machines."

He advised buying servers with more reliability features, especially those that predict pending failures and send alerts to move the workloads before the system experiences a hard failure. Despite the added cost, organizations should bear in mind that such servers are the cornerstone of any virtualization solution. Therefore, they deserve the lion's share of investment.

"Businesses will lose significant productivity if the consolidation server fails," said Bretzmann. "A hard crash can lead to hours of downtime depending upon what failed."

Longbottom, however, made the point that an organization need not spend an arm and a leg for virtualization hardware — as long as it doesn't go too low end.

"Cost of items should be low — these items may need swapping in and out as time goes on," said Longbottom. "But don't just go for cheapest kit around — make sure that you get what is needed."

This is best achieved by looking for highly dense systems. Think either stackable within a 19-inch rack or usable as a blade chassis system. By focusing on such systems, overall cooling and power budgets can be better contained. Remember, too, not every server is capable of being managed in a virtual environment. Therefore, all assets should be recognizable by standard
systems management tools.

Just as there are things you must do, several key don'ts should be observed as well. One that is often violated is that servers should not be configured with lots of internal storage.

"Servers that load VMs from local storage don't have the ability to use technologies like VMotion to move workloads from one server to another," cautioned Bretzmann.

What about virtualizing everything? That's a no-no, too. Although many applications benefit from this technology, in some cases, it actually makes things worse. For example, database servers should not be virtualized for performance reasons.

Support is another important issue to consider. "Find out if the adoption of virtualization will cause any application support problems," said Bretzmann. "Not all ISVs have tested their applications with VMware."

Storage Virtualization

Most of the provisos covered above also apply to purchasing gear for storage virtualization.

"Most of the same rules for classic physical environments still apply to virtual environments — it's really a question of providing a robust environment for the application and its data," said John Lallier, vice president of technology at FalconStor Software.

While virtual environments can shield users from hardware specific dependencies, they can also introduce other issues. One concern when consolidating applications on a single virtualization server, for example, is that you may be over-consolidating to the detriment of performance and re-introducing a single-point-of-failure. When one physical server fails, multiple virtual application servers are affected.

"Customers should look for systems that can provide the same level of data protection that they already enjoy in their physical environments," said Lallier.

He believes, therefore, that storage purchasers should opt for resilient and highly available gear that will keep vital services active no matter what hardware problems arise. In addition, Lallier suggests investing in several layers of protection for large distributed applications that may span multiple application servers. This should include disaster recovery (DR) technology so operations can quickly resume at remote sites. To keep costs down, he said users should select DR solutions that do not require an enormous investment in bandwidth.

As a cost-cutting measure, Lallier advocates doubling up virtual environments. If the user is deploying a virtual environment to better manage application servers, for example, why not use the same virtualization environment to better manage the data protection servers? As an example, FalconStor has created virtual appliances for VMware Virtual Infrastructure that enable users to make use of its continuous data protection (CDP) or virtual tape library (VTL) systems that can be installed and managed as easily as application servers in this environment.

Of course, every vendor has a different take. Network Appliance, aka NettApp, provides an alternative to FalconStor using the snapshot technology available in its StoreVault S500. This storage array handles instant backups and restores without disrupting the established IT environment.

"Useful products are able to host VMs over multiple protocols, and the StoreVault can do it via NFS, iSCSI or FCP — whatever your environment needs," said Andrew Meyer StoreVault Product Marketing Manager at NetApp.

"Don't get trapped into buying numerous products for each individual solution. One product that is flexible with multiple options (can handle VMs, create a SAN, handle NAS needs, provide snapshots and replication) may be a smarter investment as a piece of infrastructure."

Virtualization Craze Brings the Bad With the Good

2008-08-29T06:53:00.000-07:00

Although the technology required to virtualize data centers has been around for more than 40 years, 2007 will go down in history as the end of the beginning — the year the technology vaulted into mainstream consciousness and hurtled to the top of every CIO's "must-do" list.

Now that everyone has caught on to the benefits this technology can deliver, chiefly, the ability to reduce an organization's data center footprint by a factor of six, eight or 12, the focus is turning to making sure all these efficiencies and cost savings don't come at the expense of data availability and security.

"We've jumped over the chasm without even looking down below," Don Norbeck, director of product development at SunGard, told InternetNews.com. "Luckily, we'll probably land on the other side. But there's still plenty of risk."

These risks, from an organizational standpoint, start long before the first virtualization application is downloaded. Just like any other software installation, committing to a virtualization project requires not only an appreciation for the technological vulnerabilities inherent in any operating system — like bugs, malware, and access control — but also a fundamental understanding of exactly which applications and systems are used the most, which are the most critical to operations, when they're used, and how to orchestrate the workload of all these applications running on both physical and virtual servers.

For those charged with the responsibility of managing and maintaining one data center or multiple data centers, the temptation to simply initiate a straightline consolidation — take the workloads running on 100 servers and cram them on to 10 or 15 servers — is alluring. Every company wants to get greener, lower energy consumption, reduce the size of their data centers and have the ability to shift workloads with a simple click of a button.

"Now that everything is in the pool, people just have to push a button," Norbeck said. "And they'll keep pushing the button until the button doesn't work anymore. Without proper planning and provisioning, you're back to where you were before. How do you audit it? You push a button and then wait to see who starts complaining."

In other words, just as all servers are not created equal, neither are the applications running in a corporate data center.

And the stakes are increasing.

In December, SAP announced it will now begin supporting its virtualized enterprise resource planning (ERP) software running on VMware and 64-bit Windows, Linux, and Solaris platforms. It's a sign of the virtualized times.

After all, it's one thing to have the corporate e-mail system or some back-end storage system running on a virtualized machine, but companies are now virtualizing their most critical applications. Workloads can spike unpredictably. Power outages occur. One malicious bit of code has the potential to infiltrate multiple applications. Patching, more than ever, becomes a preoccupation.

And no data center is an island unto itself. It's dependent on multiple vendors to make it all work: security, storage, applications, operating systems, and networking equipment.

"Virtualization, as with any emerging technology, will be the target of new security threats," Neil MacDonald, an analyst at Gartner, said in a research report published earlier this year. "Many organizations mistakenly assume that their approach for securing virtual machines will be the same as securing any OS and thus plan to apply their existing configuration guidelines, standards and tools. While this is start, simply applying the technologies and best practices for securing physical servers won't provide sufficient protections for VMs."

MacDonald said that through 2009, 60 percent of production virtual machines will be less secure than their physical counterparts.

At the architecture level, it starts with the hypervisor, which is basically a stripped down version of the Windows or Linux or Solaris operating system.

You have 10 workloads and you merge them onto one," MacDonald said. "That's a very attractive target for a bad guy. Now, if I compromise just one thin layer, I get all 10 machines."

Nand Mulchandani, senior director of security product management and marketing at VMware, deals with the security implications of his company's industryleading software all day long. Not surprisingly, he thinks most of the security concerns raised by the media and some security experts are overblown.

"Virtualization in some sense looks like a titanic shift in computing," he said in an interview with InternetNews.com. "But frankly, from a security and technology standpoint, it's not as radical as it's been portrayed.

"It's a temporal issue," he said. "The thinking is that anything new and different is going to have problems. Everyone is looking for a big Achilles' heel that no one is talking about. We have to roll with the punches. Tomorrow there will be another new thing in the industry that everyone will call insecure. That's life."

For VMware, Microsoft, Virtual Iron, XenSource and now Oracle, the focus will shift from functionality to security as the virtualization software industry matures.

Misconfiguration and mismanagement — the propensity to set up default passwords insecurely — has been the scourge of operating systems since their inception. And while virtualization vendors continue to strip down the core operating system in the hypervisor, there's no such thing as a foolproof virtualization project.

"From a threat profile, the most important thing customers should worry about is hardening their platforms," Mulchandini said. "Locking down your platform is something most people in the Windows or Linux world are used to doing. Securing the system and the code is purely and primarily on us."

Because all large corporations — and most small- and midsize firms — didn't have the benefit of a crystal ball, applications and the operating systems running those applications grew in a staggered, chaotic fashion and can't always be configured, provisioned, or moved around in a tidy, virtualization-friendly box.

But that probably won't dissuade companies from eventually embracing virtualization in their data centers. "Where money is involved and efficiency is involved, people tend to overlook whatever minor queasiness they might have," Mulchandini said.

Seven Virtualization Trends to Watch

2008-08-29T06:34:00.000-07:00

It would be hard to argue that 2007 was a watershed for virtualization technology. From chipmakers to ISVs, every IT demographic felt a virtual tug as the technology made its way into the mainstream.

Virtualization is even starting to impact which servers are leaving the factory. According to research firm IDC, third-quarter factory revenue in the worldwide server market grew 0.5 percent year-over-year, while the number of server units shipped grew 1.5 percent less than it did for the previous period in 2006. Is this an indicator of a bigger trend, as companies soup-up hardware to scale out an go virtual?

Perhaps. But there's little doubt that going virtual will bring sweeping changes to IT organizations and the data centers they support. Here are some trends to keep an eye on:

1. Security Will Loom Large

Despite data breaches, spam, and viruses under pretty much any rock you pick up, virtualization remained largely immune so far. This will most likely change. In 2007, Gartner began singing a tune of caution for enterprises in the process of virtualizing their infrastructures, noting "that security issues are endemic to virtualization. They begin at the architecture level. Even the hypervisor itself represents a threat that malicious hackers will target."

This is because, as vice president and Gartner fellow, Neil MacDonald said, "You have 10 workloads and you merge them onto one, that's a very attractive target for a bad guy. Now, if I compromise just one thin layer I get all 10 machines… And this layer will be targeted."

Such a breach did not occur in 2007. However, deployments are increasing, and enterprises remain complacent about security. The advice here is obvious: Be proactive and tighten up your virtual machines because a breach is inevitable.

MacDonald noted that managing the security in a virtualized environment is a lot like managing yet another operating system. Therefore be sure to keep the hypervisor patched, correctly configured and up to date.

2. Easy Money Won't Be Quite So Easy Anymore

In 2007, any vendor that put "virtual" in front of its name or its product's name was able to pretty much write its own ticket. This will most likely change. With the economy tightening and virtualization offerings exceeding deployment and deployment plans, it's doubtful the venture capital firms that have been funding this whole exercise will continue to be so generous. Ideas will need to be more solid to receive funding, and venture capital firms may be ready to cash out, whether through a sale or IPO.

3. Value-Add Will Be Critical

What the virtualization vendors offer beyond the hypervisor will be the deciding factor for their success.

In early 2007, "it's all about the management," was the mantra. By now, user enterprises and ISVs have drunk the Kool-Aid. Management options are out there — whether they're being used properly is unclear, but they are available. In the latter half of 2007, attention moved to performance, integration and usability. In some cases enterprises are growing their own options. In other cases they are partnering. VMware, which traditionally has been a go-it-alone type of company, announced a partnership with SAP. Expect to see much more of such deal between the virtualization vendors and ISVs, as its less expensive and more efficient for all parties to do it this way.

4. Vendor Consolidation Will Continue

EMC may have given VMware a longer leash, but other vendors that played in the virtual playground were swallowed up. HP acquired Opsware; Citrix swallowed Xen. Virtual Iron and SWsoft (recently rechristened Parallels) remain independent. With Microsoft coming on the scene, the competition will tighten. Virtual Iron plays the partnership card well, and where that will take it is anyone's guess.

As for who might be acquiring — watch the hardware side carefully. Theories about Intel, Sun Microsystems, and IBM going on shopping sprees have surfaced in the blogosphere in recent months.

5. Microsoft Will Make Waves

Microsoft may be easy to mock, and the oft-described Redmond behemoth hardly ever comes out with anything new, technology-wise. It didn't become what it did by being stupid, however, and it has acknowledged that virtualization is here and it's here to stay. Therefore, a cannon bigger than Microsoft Virtual Server will be needed. Enter Hyper-V, the hypervisor formerly known as Viridian, which Microsoft introduced with Windows Server 2008.

Hyper-V will have an enormous and direct impact on the virtualization landscape once it ships. But does that even matter in user communities that have largely accepted beta to be nearly as good as gold — especially from Microsoft, known for its constant patching and release packs?

More organizations deploy Microsoft than any other operating system. With Hyper-V a standard component, enterprises still on the virtual fence will be able to dip a toe in the water, easily and inexpensively. This will make VMware and its ilk a tougher sell, especially if Microsoft truly plays nice with other operating systems. The value VMware and other competitors add will need to be enough to justify their price.

6. Storage and Servers Will Converge

Convergence has been a buzzword in the dot-com era. Sometimes it holds true, sometimes not. The storage and server infrastructures have been skirting each other for years, and storage has a legacy of its own with virtualization. RAID arrays really aren't that different from virtual machines, after all.

VMware unwrapped a new version of Virtual Infrastructure, and Virtual Iron took a new version of its software out of the bag. Both vendors emphasized storage, and the evidence of melding is pretty clear.

7. I/O and Automation Will Become

Increasingly Critical Not that they weren't before. When you're moving workloads around and planning out deployments, it's important to have an understanding of your needs and limitations. As more workloads are moved around virtually, it's even more paramount. A recent survey from Xsigo Systems highlighted I/O problems that have been surfacing. As for automation, consider the price HP paid for Opsware and how quickly it integrated it into the organization.

10 Coolest Features in Windows Server 2008

2008-08-27T11:18:00.000-07:00

There's still plenty of mileage left in Microsoft Windows Server 2003, but it doesn't hurt to look ahead. You won't find any killer features in Windows Server 2008, but that's not to say there's nothing to get excited about. There's a great deal that's new, and depending on the set up of your organization, it's almost certain you'll find some or all of it extremely valuable.

Any ranking is bound to be subjective, and bearing that in mind, here are what we believe to be the 10 most interesting new features in Windows Server 2008.

1. Virtualization

Microsoft's Hyper-V hypervisor-based virtualization technology promises to be a star attraction of Server 2008 for many organizations.

Although some 75 percent of large businesses have started using virtualization, only an estimated 10 percent of servers are running virtual machines. This means the market is still immature. For Windows shops, virtualization using Server 2008 will be a relatively low-cost and low-risk way to dip a toe in the water.

At the moment, Hyper-V lacks the virtualized infrastructure support virtualization market leader VMware can provide. Roy Illsley, senior research analyst at U.K.-based Butler Group, noted that Microsoft is not as far behind as many people seem to think. "Don't forget Microsoft's System Center, which is a fully integrated management suite and which includes VM Manager. Obviously it only works in a Wintel environment, but if you have Server 2008 and System Center, you have a pretty compelling proposition.

"What Microsoft is doing by embedding virtualization technology in Server 2008 is a bit like embedding Internet Explorer into Windows," said Illsley. "This is an obvious attempt to get a foothold into the virtualization market."

At launch, Microsoft is unlikely to have a similar product to VMware's highly popular VMotion (which enables administrators to move virtual machines from one physical server to another while they are running), but such a product is bound to available soon after.

2. Server Core

Many server administrators, especially those used to working in a Linux environment, instinctively dislike having to install a large, feature-packed operating system to run a particular specialized server. Server 2008 offers a Server Core installation, which provides the minimum installation required to carry out a specific server role, such as for a DHCP, DNS, or print server. From a security standpoint, this is attractive. Fewer applications and services on the server make for a smaller attack surface. In theory, there should also be less maintenance and management with fewer patches to install, and the whole server could take up as little as 3Gb of disk space according to Microsoft. This comes at a price - there's no upgrade path back to a "normal" version of Server 2008 short of a reinstall. In fact there is no GUI at all - everything is done from the command line.

3. IIS

IIS 7, the Web server bundled with Server 2008, is a big upgrade from the previous version. "There are significant changes in terms of security and the overall implementation, which make this version very attractive," said Barb Goldworm, president and chief analyst at Boulder, Colo.-based Focus Consulting. One new feature getting a lot of attention is the ability to delegate administration of servers (and sites) to site admins while restricting their privileges.

4. Role-Based Installation

Role-based installation is a less extreme version of Server Core. Although it was included in 2003, it is far more comprehensive in this version. The concept is that rather than configuring a full server install for a particular role by uninstalling unnecessary components (and installing needed extras), you simply specify the role the server is to play, and Windows will install what's necessary - nothing more. This makes it easy for anyone to provision a particular server without increasing the attack surface by including unwanted components that will not do anything except present a security risk.

5. Read Only Domain Controllers (RODC)

It's hardly news that branch offices often lack skilled IT staff to administer their servers, but they also face another, less talked about problem. While corporate data centers are often physically secured, servers at branch offices rarely have the same physical security protecting them. This makes them a convenient launch pad for attacks back to the main corporate servers. RODC provides a way to make an Active Directory database read-only. Thus, any mischief carried out at the branch office cannot propagate its way back to poison the Active Directory system as a whole. It also reduces traffic on WAN links.

6. Enhanced Terminal Services

Terminal services has been beefed up in Server 2008 in a number of ways. TS RemoteApp enables remote users to access a centralized application (rather than an entire desktop) that appears to be running on the local computer's hard drive. These apps can be accessed via a Web portal or directly by double-clicking on a correctly configured icon on the local machine. TS Gateway secures sessions, which are then tunnelled over https, so users don't need to use a VPN to use RemoteApps securely over the Internet. Local printing has also been made significantly easier.

7. Network Access Protection

Microsoft's system for ensuring that clients connecting to Server 2008 are patched, running a firewall and in compliance with corporate security policies - and that those that are not can be remediated - is useful. However, similar functionality has been and remains available from third parties.

8. Bitlocker

System drive encryption can be a sensible security measure for servers located in remote branch offices or anywhere where the physical security of the server is sub-optimal. Bitlocker encryption protects data if the server is physically removed or booted from removable media into a different operating system that might otherwise give an intruder access to data that is protected in a Windows environment. Again, similar functionality is available from third-party vendors.

9. Windows PowerShell

Microsoft's new(ish) command line shell and scripting language has proved popular with some server administrators, especially those used to working in Linux environments. Included in Server 2008, PowerShell can make some jobs quicker and easier to perform than going through the GUI. Although it might seem like a step backward in terms of user friendly operation, it's one of those features that once you've gotten used to it, you'll never want to give up.

10. Better Security

We've already mentioned various security features built into Server 2008, such as the ability to reduce attack surfaces by running minimal installations, and specific features like BitLocker and NAP. Numerous little touches make Server 2008 more secure than its predecessors. An example is Address Space Load Randomization - a feature also present in Vista - which makes it more difficult for attackers to carry out buffer overflow attacks on a system by changing the location of various system services each time a system is run. Since many attacks rely on the ability to call particular services by jumping to particular locations, address space randomization can make these attacks much less likely to succeed.

It's clear that with Server 2008 Microsoft is treading the familiar path of adding features to the operating system that third parties have previously been providing as separate products. As far as the core server product is concerned, much is new. Just because some technologies have been available elsewhere doesn't mean they've actually been implemented. Having them as part of the operating system can be very convenient indeed.

If you're running Server 2003 then, now is the time to start making plans to test Server 2008 - you're almost bound to find something you like. Whether you decide to implement it, and when, is up to you.

Clustering and Storage with Windows Server 2003 Part 4

2008-08-27T11:04:00.000-07:00

iSCSI Storage

So far, we discussed the two most common choices - direct-attached SCSI (with popularity resulting from its long-lasting, widespread commercial presence and low pricing) and Fibre Channel storage-area networks (FC SANs), which are frequently chosen because of their superior performance and reliability.

Unfortunately, the cost associated with FC SAN deployments is prohibitive for most smaller or less critical environment, whose requirements cannot be satisfied with parallel SCSI because of its performance and scalability limitations. The introduction of iSCSI resolves this dilemma by combining the benefits of both technologies and at the same time avoiding their biggest drawbacks.

iSCSI is an acronym derived from the term Internet SCSI, which succinctly summarizes its basic premise. iSCSI uses IP packets to carry SCSI commands, status signals, and data between storage devices and hosts over standard networks. This approach offers tremendous advantage by leveraging existing hardware and cabling (as well as expertise). Although iSCSI frequently uses Gigabit Ethernet, with enterprise class switches and specialized network adapters (containing firmware that processes iSCSI-related traffic, offloading it from host CPUs), its overall cost is lower than equivalent Fibre Channel deployments. At the same rate, however, features, such as addressing or automatic device discovery built into FC SAN infrastructure, must be incorporated into iSCSI specifications and implemented in its components.

iSCSI communication is carried over a TCP session between an iSCSI initiator (for which functionality is provided in Windows 2003 in the form of software or a mix of HBA firmware and Storport miniport driver) and an iSCSI target (such as a storage device), established following a logon sequence, during which session security and transport parameters are negotiated. These sessions can be made persistent so they are automatically restored after host reboots.

On the network level, both initiator and target get assigned unique IP addresses, which allow for node identification. With node identification, the target is actually accessed by a combination of IP address and port number, which is referred to as portal. In the iSCSI protocol, addressing is typically handled with iSCSI Qualified Name (IQN) convention. Its format consists of the type identifier (i.e., "iqn."); registration date field (in the month-year notation); followed by the period and domain in which the name is registered (in reversed sequence); the semicolon; and the host (or device) name, which can be either autogenerated (as is the case with Microsoft implementation, where it is derived from the computer name), preassigned, or chosen arbitrarily, serving as a descriptor providing such information as device model, location, purpose, or LUN.

Targets are located either by statically configuring software initiator, by specifying target portal parameters (and corresponding logon credentials), by leveraging functionality built into HBAs on the host, or discovered automatically, using information stored on an Internet Storage Name Server (iSNS). This server offers a centralized database of iSCSI resources, where iSCSI storage devices are able to register parameters and status, which subsequently can be referenced by initiators. Access to individual records can be restricted based on discovery domains, serving a purpose similar to FC SAN zoning.

In a typical Microsoft iSCSI implementation, the initiator software running on a Windows host server (with a compatible NIC or an HBA that supports Microsoft iSCSI driver interface), is used to mount storage volumes located on iSCSI targets and registered with iSNS server.

Installation of the initiator includes iSNS client and administrative features, in the form of the iSCSI Initiator applet in the Control Panel and Windows Management Instrumentation and iSCSI Command Line interface (iSCSICLI). The software-based initiator lacks some of the functionality that might be available with hardwarebased solutions (such as support for dynamic volumes or booting from iSCSI disks).

To provide a sufficient level of security and segregation, consider isolating iSCSI infrastructure to a dedicated storage network (or separating the shared environment with VLANs), as well as applying authentication and encryption methods. With Microsoft implementation, authentication (as well as segregation of storage) is handled with Challenge Handshake Authentication Protocol (CHAP), relying on a password shared between an initiator and a target, providing that the latter supports it. Communication can be encrypted directly on end devices, using built-in features of high end iSCSI HBAs, third-party encryption methods, or Microsoft's version of IPSec.

Although network teaming is not supported on iSCSI interfaces, it is possible to enable communication between an initiator and a target via redundant network paths that accommodate setup with multiple local NICs or HBAs and separate interconnects for each. Implementing multiple connections per session (MCS), which leverage a single iSCSI session, can do this. It can also be done with Microsoft Multipath I/O (MPIO), which creates multiple sessions. The distribution of I/O across connections (applied to all LUNs involved in the same session) or sessions (referencing individual LUNs), for MSC and MPIO, depends on Load Balance Policies configured by assigning Active or Passive type to each of network paths. This results in one of the following arrangements:

• Fail Over Only uses a single active path as the primary and treats all others as secondaries, which are attempted in round-robin fashion in case the primary fails. The first available one found becomes the primary.

• Round Robin distributes iSCSI communication evenly to all paths in round-robin fashion.

• Round Robin with Subset functions with one set of paths in the Active mode and the other remaining Passive. The traffic is distributed according to the round robin algorithm across all active paths.

• Weighted Path selects a single active path by picking the lowest value of arbitrarily assigned weight parameter.

• Least Queue Depth, available only with MCS, sends traffic to the path with the fewest number of
requests.

The multi-pathing solution selected depends on a number of factors, such as support on the target side, required level of granularity of Load Balance Policy (individual LUN or session level), and hardware components (MCS is recommended in cases where a software-based initiator - without presence of specialized HBAs on the host side - is used). Regardless of your decision, take advantage of this functionality as part of your clustering deployment to increase the level of redundancy. When incorporating iSCSI storage into your Windows 2003 Server cluster implementation (note that Microsoft does not support it on Windows 2000), also ensures that components on the host side fully comply with iSCSI device logo program specifications and basic clustering principles. Take domain and network dependencies into account. Also bear in mind that besides SCSI RESERVE and RELEASE commands (which provide basic functionality), iSCSI targets must support SCSI PERSISTENT RESERVE and PERSISTENT RELEASE to allow for all of the Load Balance policies and persistent logons.

The latter requires a persistent reservation key be configured on all cluster nodes. Choosing an arbitrary 8-byte value, with the first 6 bytes unique to each cluster and the remaining 2 bytes varying between its nodes, does this. Data is entered in the PersistentReservationKey REG_BINARY entry of the HKLM/System/CurrentControlSet/Services/MSiSCDSMPersistentReservation registry key on each cluster member. In addition, the UsePersistentReservation entry of REG_DWORD type is set to 1 in the same registry location. You should also enable Bind Volumes Initiator Setting (in the Properties dialog box of the iSCSI Initiator Control Panel applet), which ensures all iSCSI hosted volumes are mounted before the Cluster Service attempts to bring them online.

To avoid network congestion-related issues, consider setting up dedicated Gigabit Ethernet network or implementing VLANs with non-blocking switches supporting Quality of Service. Optimize bandwidth utilization, by implementing Jumbo frames and increasing value of Maximum Transmission Unit.

Clustering and Storage with Windows Server 2003 Part 3

2008-08-27T10:43:00.000-07:00

SAN-Based Storage

Fibre Channel storage area network (FC SANs) represent a considerable shift from the directly attached storage paradigm. They offer significant functionality and performance improvements. The basic idea is to use a network infrastructure for connecting servers to their disks, allowing physical separation of the two by far greater distances than was previously possible. But there are also other, equally important, advantages of this separation. Managing storage in larger environments no longer requires dealing with each individual system, as was the case with directly attached models. Disks are grouped together, simplifying their administration (e.g., monitoring, backups, restores, provisioning and expansion) and making it more efficient, through such inventions as LAN-free or server-free backups and restores, or booting from a SAN.

In addition, since large number of servers and storage devices can participate in the same SAN, it is possible to attach new ones as needed, making allocation of additional space a fairly easy task. This is further simplified by the DISKPART.EXE Windows 2003 Server utility, which is capable of dynamically extending basic and dynamic volumes, as explained in Microsoft Knowledge Base Article Q325590. This is especially true when comparing the SAN with a SCSI-based setup, where the limited amount of internal or external connectors and adjacent physical space available must be taken into account.

Fibre Channel SAN technology leverages SCSI-3 specifications for communication between hosts and target devices, since its implementation is based on the SCSI command set. Their transmission, however, is handled using FC transport protocol. This is done in a serial manner, typically over fiber optic cabling (although copper-based media are allowed), which eliminates distance limitations inherent to parallel SCSI.

Note, however, that the term "network" should not be interpreted in the traditional sense, since SANs do not offer routing capabilities, primarily because they are intended for high-speed, low-latency communication. SANs also use a distinct end node identification mechanism, which does not rely on Media Access Control (MAC) addresses associated with each network adapter, but instead employs 64-bit (expressed usually in the form of eight pairs of hexadecimal characters) World Wide Names (WWN), burned into fibre host bus adapters (HBAs) by their manufacturers. FC interconnecting devices handle dynamic address allocation on the fabric level. In addition, unlike majority of IP-based networks, FC SANs have primarily asymmetric characters, with active servers on one end connecting mostly to passive devices, such as disks arrays or tape drives on the other, arranged in one of the following topologies:

• Point-to-point: Links a single host and its storage device directly via a fiber connection. This type of configuration is the simplest and least expensive to deploy and manage, but it lacks the flexibility and expandability of the other two, since it is conceptually equivalent to SCSI-based directly attached disks. It is, therefore, rarely implemented.

• Shared, also known as Fibre Channel Arbitrated Loop (FC-AL): Takes the shape of a logical ring (but physically forming a star), with an FC hub or a loop switch serving as the interconnecting device. The design is similar to Token Ring architecture. This similarity is also apparent when it comes to arbitration of loop usage.

Since FA-CL devices share the same media, whenever one of them needs to communicate with another, it is must send an arbitration packet around the loop, which once returned back to the sender, signals exclusive loop access can be granted. Should conflicts occur when multiple devices attempt to communicate at the same time, the one with the lowest address wins. Addresses, which differentiate among all nodes participating in the loop, can be hard coded or assigned dynamically. The majority of loop switches provide this capability. Although dynamic allocation simplifies configuration in multi-node scenarios, it might also cause instability when devices are restarted or new ones added, since such events trigger loop reinitialization and node readdressing.

Although considerably less expensive than their switch-based counterparts, FC-ALs are not as efficient. Access to fabric is shared across all interconnected devices, which allows only two of them communicate at any given time. They are also not as scalable and support fewer nodes - the maximum is 126. As with SCSI-based shared storage, FC-AL-based Windows 2003 Server clusters are limited to two nodes. In addition, Microsoft recommends using arbitrated loops for individual cluster implementations, rather than sharing them with other clusters or non clustered devices. Larger or shared implementations require switched configuration.

• Switched, referred to as Switched Fibre Channel Fabric (FC-SW): These networks use FC switches functioning as interconnecting devices. This topology addresses the efficiency limitations of the loop configuration by allowing simultaneous, dedicated paths at the full wire speed between any two Fibre attached nodes. This is based on the same principle as traditional LAN switching. Scalability is greatly increased due to hierarchical, fully redundant architecture. It consists of up to three layers with core employing highest speed and port density switches, distribution relying on midrange hardware, and access characterized by low-end switches, arbitrated loops, and point-to-point connections.

Switches keep track of all fabric-attached devices, including other switches, in federated and cascaded configurations, using 3-byte identifiers. This sets the theoretical limit of roughly 16 million unique addresses. Stability is improved as well, since restarts and new connections are handled gracefully, without changes to an already established addressing scheme or having a negative impact on the status of the fabric. This is partially because of the introduction of less disruptive, targeted LUN and SCSI ID resets, which are attempted first before resorting to the bus wide SCSI Reset command. Previously, this was the only available option in Windows 2000 Server cluster implementations. Keep in mind, however, that the availability of this feature depends on the vendor developed HBA specific mini-port driver, which must be written specifically to interact with the Microsoft provided StorPort port driver. This is a new feature in Windows 2003 Server. It is designed specifically to take advantage of performance enhancing capabilities of FC adapter, rather than legacy SCSI Port.

Increased performance, flexibility, and the reliability of switched implementations come with their own set of drawbacks. Besides considerably higher cost (compared to arbitrated loops) and interoperability issues across components from different vendors, one of the most significant ones is the increased complexity of configuration and management. In particular, it is frequently necessary to provide an appropriate degree of isolation across multiple hosts connected to the same fabric and shared devices with which they are supposed to interact.

As mentioned earlier, this exclusive access is required to avoid data corruption, which is bound to happen with unarbitrated, simultaneous writes to the same disk volume. In general, three mechanisms deliver this functionality - zoning, LUN masking (known also as selective presentation), and multi path configurations.

Zoning can be compared to Virtual LANs (VLANs) in traditional networks, since it defines logical boundaries (known in SAN terminology as zones) that encompass arbitrarily designated switch ports.

Zone definitions in clustered deployments are typically stored and enforced by the switch port ASIC (Application-Specific Integrated Circuits) firmware, with communication permitted only between nodes attached to the switch ports that belong to the same zone. They can also be implemented by referencing WWN of host bus adapters. In addition to preventing accidental data corruption, zoning offers also an additional level of security. It protects the server from unauthorized access. In clustered configurations, cluster nodes, along with the shared disks that constitute clustered resources, should belong to the same zone. LUN (an acronym for Logical Unit Number, describing a logical disk defined in a FC SAN) masking makes it possible to limit access to individual, arbitrarily selected

LUNs within a shared storage device. Such functionality is typically required in configurations involving large multi disk systems, where port-level zoning does not offer sufficient granularity. LUN masking provides necessary isolation in cases of overlapping zones, where hosts or storage devices belong to more than one zone. The relevant configuration is performed and stored on the storage controller level.

Multi path technology is the direct result of the strive for full redundancy in SAN environment. Such redundancy is available on the storage side (through fault-tolerant disk configurations, dual controllers with their own dedicated battery-backed caches and power supplies) and on the server side (through server clustering, with each of the member servers featuring dual, hot-swappable components). It is reasonable to expect the same when it comes to SAN connectivity.

Unfortunately, the solution is not as simple as installing two FC host bus adapters (HBAs) and connecting them to two redundant switches, each of which in turn, attaches to separate FC connections on the storage controller. This is because without additional provisions, Windows would detect two distinct I/O buses and separately enumerate devices connected to each (resulting in a duplicate set of drives presented to the operating system), which could potentially lead to data corruption. To resolve this issue, Microsoft Windows 2003 Server includes native support for Multi path I/O, which makes it possible to connect dual HBAs to the same target storage device with support for fail over, fail back, and load balancing functionality.

Each implementation of a Windows 2003 Server cluster must belong to a dedicated zone, to eliminate potential adverse effect of the disk access protection mechanism included in the clustering software on other devices. This does not apply, however, to storage controllers, which can be shared across multiple zones, as long as they are included on the Cluster/Multi-Cluster Device HCL. In addition, you should avoid collocating disk and tape devices in the same zone, as the SCSI bus reset commands can interfere with normal tape operations.

Remember, the rule regarding consistent hardware and software setup across all cluster nodes extends to SAN connections - including host bus adapter models, their firmware revision levels, and driver versions. You should also ensure that automatic basic disk volume mounting feature is disabled. This does not apply to volumes residing on dynamic disks or removable media, which are always automatically mounted. Earlier versions of Windows would spontaneously mount every newly detected volume. In a SAN environment, this could create a problem if zoning or LUN masking was
misconfigured or if prospective cluster nodes had access to the shared LUNs prior to installation of the clustering software. This feature is configurable, and disabled by default, in Windows 2003 Server. Running the MOUNTVOL command or using AUTOMOUNT option of the DISKPART utility can control it.

Clustering and Storage with Windows Server 2003 Part 2

2008-08-27T10:39:00.000-07:00

Disk Storage (SCSI)

So far, we've provided a brief overview of server clustering, and described its three basic types, categorized according to characteristics of the Quorum resource (i.e., Single Shared, Single Local, and Majority Node Set). Next up, we will examine one of the most important of clustering components - disk storage.

The importance of disk storage stems from its role in the server clustering architecture. As you might recall from earlier discussions, the Quorum resource must be implemented as an NTFS volume, hosting Quorum log and Checkpoint files (for more details, refer to our earlier article). Just as relevant is the ability to implement the Physical Disk resource (separate from the Quorum resource), which is required in the overwhelming majority of typical clustered applications.

To comply with server clustering principles, storage must have certain characteristics. More specifically, the volumes it hosts must be accessible to all cluster nodes; a critical requirement for the Single Shared cluster category is concerned. This applies to many deployments but not Single Local or Majority Node Set types.

Cluster Service Communication

The storage must also be able to communicate with the Cluster Service, an instance of which runs on every node, via SCSI protocol. This does not limit hardware choices to SCSI disks, channels, and controllers; however, disks and controllers must be capable of properly processing (and sharing a channel for proper transmitting) such SCSI commands as Reserve (used by individual cluster nodes to obtain and maintain exclusive ownership of a device), Release (which relinquish reservation of a device, allowing another cluster node to take ownership of it), and Reset (forcibly removing existing reservation of an individual device or all devices on the bus).

These commands serve a very important purpose - they prevent a situation where two hosts would be permitted to write simultaneously to the same disk device. This is likely to happen otherwise, considering both hosts share a physical connection to it. When the first cluster node is brought online, its Cluster Service (with help of the Cluster Disk Driver Clusdisk.sys) scans the devices of the shared storage bus and attempts to bring them online. It issues the Reserve command to claim ownership. The same command gets re-sent in subsequent, regular intervals (every three seconds). This ensures ownership is maintained, and the owning node has exclusive access to all volumes on the target disk.

Reserve also plays a critical role in cases when network communication between nodes fails. As mentioned earlier, such a situation is handled by first establishing which node is the owner of the Quorum (potentially triggering new election, if the previous owner is no longer operational) and transferring all cluster resources to it. A successful outcome of this process relies on two settings that control SCSI commands issued by cluster nodes. The first one forces the Quorum owner to renew its reservation every three seconds. The second one, inter-node communication failure, causes non-Quorum owners to initiate bus-wide Reset, followed by a sevensecond waiting period.

If the Quorum remains available after the wait period is over (which indicates the previous Quorum owner failed), the challenging node takes over ownership of the Quorum (by sending Reserve signal) as well as all remaining resources. Another purpose of Reset command is to periodically terminate reservations to detect situations in which a node becomes unresponsive (without failing completely). Providing that this is not the case, reservations are subsequently re-established.

Now that we have established functional requirements of storage in Single Shared Quorum clusters, let's review technologies that satisfy criteria outlined above. Regardless of your choice, the actual hardware selected must be Microsoft-certified, which can be verified by referencing Microsoft Windows Server catalog). In general, storage clustering solutions belong to one of four categories:

• Shared SCSI
• Fibre Channel Storage Area Networks (SANs)
• NAS (Network Attached Storage)
• iSCSI

SCSI

SCSI (Small Computer System Interface) is the best known and most popular storage technology for multidisk configurations. The term SCSI also refers to the communication protocol, providing reliable block-level data transport between a host (known as the initiator) and storage (known as the target), which is independent of the way data is stored. Its architecture consists of a parallel I/O bus shared between multiple (frequently daisy-chained) devices (including controllers), and enclosed on both ends with terminators, which prevent electrical signals from bouncing back (terminators are frequently built directly into SCSI devices).

A SCSI controller is typically installed in a host system as the host adapter, but it can also reside in an external storage subsystem. Each device on the bus is assigned a unique identifier referred to as SCSI ID that is numbered from 0 to 7 or from 0 to 15, for narrow and wide SCSI bus types, respectively. In addition to providing addressing capabilities, the SCSI ID determines priority level (with an ID 7 being the highest and assigned typically to the controller, ensuring proper bus arbitration).

A limited range of SCSI IDs (which restrict the number of devices on the bus to 15) is extended through the assignment of Logical Unit Numbers (LUNs), associated with each individual storage entity, which is able to process individual SCSI commands. Typically, they represent individual disks within a storage subsystem, connected to the main SCSI bus via an external SCSI controller. In addition to LUN and SCSI ID, the full address of such Logical Unit also contains a bus identifier, which commonly corresponds to a specific SCSI interface card. A server can have several such cards installed. The total number of available LUNs ranges from 8 to 254, depending on the hardware support for Large LUNs. For more information on this subject, refer to Microsoft Knowledge Base article 310072.

Implementing SCSI technology for the purpose of shared clustered storage adds an extra layer of complexity to its configuration. Since the bus must be accessible by clustered nodes, install a SCSI controller card in each (and disable their BIOS). Furthermore, since these controllers will be connected to the same bus, they cannot have identical SCSI IDs. Typically, this dilemma is resolved by setting one to 7 and the other to 6, which grants the latter the next-highest priority level. To ensure the failure of a single component (such as a device, controller, or host) does not affect the entire cluster, use external (rather than device's built-in) terminators. Keep in mind that number of nodes in a SCSI storage-based clustered implementation cannot exceed two.

As part of your design, you should ensure sufficient level of storage redundancy by implementing RAID, which enables individual disk failures to not affect overall data accessibility. Although Windows 2000 and 2003 Server products support software-based fault tolerant RAID configurations (RAID 1 and 5, known also as mirroring and striping with parity, respectively), this requires setting up target disks as dynamic, which in turn are not permitted - at least not without installing third-party products (e.g., Symantec Storage Foundation for Windows add-in) as shared clustered storage This restriction does not apply to local cluster node drives.

Although this means that you must resort to more expensive, external storage arrays, which implement hardware-based RAID, you can benefit not only from significantly better performance but also from improved functionality, including such features as redundant hot swappable fans, power supplies, extra disk cache memory, and more complex and resilient RAID configurations (such as RAID 10 or 50, which combine disk mirroring with striping or striping with parity, protecting from losing data access even in cases of multiple disk failures).

Unfortunately, the SCSI technology, despite its relatively low cost, widespread popularity, and significant transfer speeds of up to 320 MBps with SCSI-3 Ultra320 standard is subject to several limitations. They result mainly from its parallel nature, which introduces a skew phenomenon (where individual signals sent in parallel arrive at a target at slightly different times), restricting the maximum length of the SCSI bus (in most implementations, remaining within 25 meters range, requiring physical proximity of clustered components, which makes them unsuitable for disaster recovery scenarios). A recently introduced serial version of SCSI (Serial Attached SCSI, or SAS) addresses the drawbacks of its parallel counterpart, but it is unlikely to become a meaningful competitor to Fibre Channel or iSCSI. The SCSI bus is also vulnerable to contention issues, where a device with higher priority dominates communication. Finally, storage is closely tied to the hosts, which increases the complexity of consolidation and expansion efforts.

Although shared SCSI technology is a viable option for lower-end server clustering implementations on the Windows 2003 Server platform, other types of storage solutions offer considerable advantages in terms of performance, scalability, and stability.

Clustering and Storage with Windows Server 2003 Part 1

2008-08-27T10:31:00.000-07:00

In this article we're going to take a look at clustering and storage with Windows Server 2003. You can use the information we present here as a foundation for creating high-availability solutions.

Although some of the technologies we will be describing have been available in earlier version of Windows (as inherent components, add on programs, or third-party offerings that made their way into Microsoft's portfolio through acquisitions) their latest incarnations are superior from functionality, stability, and manageability perspectives.

Server Clustering

Two basic approaches to reaching high availability have been built into the Windows Server 2003 operating system. The first, known as Server Clustering, requires Windows Server 2003 Enterprise and Data center Editions. The second one, known as Network Load Balancing (NLB), was incorporated into all Windows Server 2003 versions (including Standard and Web).

Each represents a unique approach to eliminating "a single point of failure" in computer

system design. They also share one important underlying feature that serves as the basis for their redundancy: Both increase availability by relying on multiple physical servers, hosting identically configured instances of a particular resource (such as a service or application). The main difference lies in the way these instances are defined and implemented.

In case of NLB, each instance is permanently tied to the hosting of its physical server, and it remains active as long as this server is functional. In other words, all of them operate simultaneously during cluster uptime. With Server Clustering, on the other hand, there is only a single active instance for each highly available resource, regardless of the total number of servers that are members of the entire cluster. The server that currently hosts this resource becomes its owner and is responsible for processing all requests for its services.

These underlying architectural principles introduce a number of challenges. Since the NLB cluster consists of up to 32 instances running in parallel servers, there is a need for additional mechanisms that enable them to decide which is responsible for handling the processing of client requests targeting any of the highly available resources at any given time. This determination must be made for every new incoming request and, depending on the configuration, might have to be performed independently for each of them.

With Server Clustering, the equivalent process is trivial since there is only one instance of each highly available resource. The cost, however, is increased complexity of the logic that dictates which member server hosts this resource, especially following the failures of its previous owner.

To function as a unit, servers participating in a cluster (also referred to as nodes) must be able to interact with each other. This is accomplished by setting up redundant network connections so as to minimize the possibility of failure. Thus, each node should have at least two network adapters. The connections are organized into two groups, private and public, also referred to as "Internal Cluster communications only" and "All Communications," respectively. They are identified and configured during cluster installation on each member server.

The first one contains links dedicated to inter-node, intra-cluster traffic. Although the primary purpose of the second one is to carry service requests and responses between clients and the cluster, it also serves as a backup to the first one. Depending on the number of nodes in a cluster (and your budget), you can employ different technologies to implement node interconnects. In the simplest case (limited to two nodes), this is possible with a crossover cable. When a larger number of servers participate in a cluster (up to a total of eight supported by Windows Server 2003 Enterprise and Data center Editions) a preferably dedicated hub or a switch is needed.

To optimize inter-node communication, which is critical for a cluster to operate properly, we recommended eliminating any unnecessary network traffic on the private network interfaces. This is accomplished by:

• Disabling NetBIOS over TCP/IP. Relevant options are listed in the NetBIOS section on the WINS tab of the Advanced TCP/IP settings dialog box of the interface properties

• Removing file and printer sharing for Microsoft Networks. Configurable on the General tab of the interface properties dialog box

• Setting appropriate speed and duplexity mode. Rather than relying on Autodetect option - Done from the Advanced tab of the network adapter Properties dialog box

• Ensure that statically assigned IP addresses are used. Instead of using Dynamic Host Configuration Protocol or Automatic Private IP Addressing.

• There should be no default gateway. Entries should be cleared for the "Use the following DNS server addresses" options, present on the Internet Protocol Properties dialog box for the connection

It is no longer necessary to disable the Media Sensing feature on Windows Server 2003. This was accomplished by registry modification on Windows 2000-based cluster members.

Despite these extra measures, communication between nodes can still fail. This makes it necessary to provide an additional safety mechanism that would prevent a so-called "split-brain" scenario, where individual nodes, unable to determine status of clustered resources, attempt to activate them at the same time. This would violate the principles of server clustering described above and result in potentially serious implications, such as data corruption in the case of disk-based resources.

Quorum Designations

To prevent this, every cluster contains one designated resource, called Quorum, implemented as a dedicated disk volume. Most frequently, this volume consists of a pair of mirrored disks, which increases the level of its fault tolerance. Its optimum size is 500 MB (due to NTFS characteristics), although its use typically constitutes only a fraction of this capacity. Like with other resources, only one server owns the Quorum at any given time. The Quorum owner has the ultimate responsibility for making decisions regarding ownership of all other resources.

More specifically, nodes exchange "heartbeat" signals, formatted as User Datagram Protocol (UDP) packets at pre-configured intervals (every 1.2 seconds) to confirm their network interfaces are operational. The absence of two consecutive packets triggers a reaction that is supposed to address potential cluster problems. In Windows 2000 Server-based implementations, this consisted of activating all resources on the current owner of the Quorum and, simultaneously, deactivating them on all other nodes. This effectively ensured only a single instance of each resource remained online. However, under certain circumstances, it could lead to an undesirable outcome.

Although a rather rare occurrence, it is possible for the Quorum owner to lose connectivity on all of its interfaces and, at the same time, the remaining nodes remain able to communicate with the client's network. As the result, user requests will not be able to reach cluster resources, which are still active but reside on the node that is no longer accessible. Remaining nodes, however, would be fully capable of handling these requests, if they can take ownership of the Quorum and all other resources.

The introduction of additional logic in the way Windows Server 2003-based clusters handle the absence of heartbeat traffic resolved this issue. Rather than following the legacy procedure when missing heartbeat signals are detected, nodes first check whether any of their network interfaces designated as public are operational and, if so, whether client networks are still reachable. This is accomplished by sending ICMP (Internet Control Message Protocol) echo requests (i.e., executing PING) to external systems - typically the default gateway configured for these interfaces. If the node hosting the Quorum fails any of these tests, it will voluntarily deactivate all its resources, including the Quorum. If the remaining nodes discover their network links are still working, they will have no problem establishing a new Quorum owner and transfer control of all cluster resources to it.

Besides assisting with resource arbitration following communication failure, Quorum serves another important function - providing storage for up-to-date cluster configuration. This configuration resides in two files in the MSCS folder on the quorum volume - the cluster hive checkpoint file (Chkxxx.tmp) and Quorum log (Quolog.log). The first one stores a copy of configuration database, which mirrors the content of the Cluster registry hive on the server hosting the Quorum resource and stored in the %SystemRoot%ClusterCLUSDB file on that server. This database is replicated to all remaining nodes and loaded into their Registry (maintaining a single "master" copy of this information ensures its consistency). Replication takes place for every new cluster configuration change, as long as all nodes are operational. If this is not the case, timestamped changes are recorded in the Quorum log file and applied to configuration database once the offline nodes are brought back online. Being familiar with these facts is important when troubleshooting some of the most severe cluster problems.

As already mentioned, Quorum is implemented as a volume on a physical disk. However, details of this implementation vary depending on a number of factors, such as number of nodes, server cluster type, or storage technology.

Maintaining a single instance of each clustered resource (ensuring at the same time its fault tolerance and preventing "split-brain" scenarios) is accomplished through two basic mechanisms, resource virtualization and inter-node communication.

Resource virtualization requires each clustered service or application be represented by a number of related software and hardware components, such as disks, IP addresses, network names, and file shares, which can be assigned to any server participating in the cluster and easily transferred between them, if necessary. This is made possible by setting up these servers in a very specific manner, where they can access the same set of shared storage devices, reside on the same subnet, and are part of the same domain. For example, to create a highly available network file share, you would identify a shared disk drive hosting the share, an IP address (with corresponding network name) from which the share can be accessed remotely, and target file share, with its name and access permissions.

Although this sounds complicated and time consuming, all necessary resources are pre-defined, making this procedure fairly straightforward. Once resources are identified and configured (by specifying disk drive letters, assigning unique IP addresses, network names, or file share characteristics), they can be assigned to any server participating in the cluster (as long as each one is capable of supporting them). Resources can then be easily moved between nodes in case the one currently hosting them fails.

The Importance of Quorum

Inter-node communication is facilitated through heartbeat signals carried over redundant network connections between cluster members and through Quorum's presence, which determines how resource ownership should be handled. As we pointed out, Quorum has the additional important function of storing the most up-to-date cluster configuration, copied subsequently to a dedicated registry hive on each node. Local copies are referenced when nodes join the cluster during startup. Because of its significance in clustering architecture, Quorum also serves as the basis for three main server clustering models:

• Single Shared Quorum: Quorum is implemented as the Physical Disk clustered resource.

• Single Local Quorum: Quorum is implemented as the Local Quorum clustered resource.

• Majority Node Set Quorum: Quorum is implemented as the Majority Node Set clustered resource.

Single Shared Quorum clusters are by far most popular among server cluster implementations. They most closely match the traditional clustering design (which is reflected by continuing support for this model since introduction of Microsoft Cluster Server in Windows NT 4.0 Server Enterprise Edition), offering high-availability of resources representing wide variety of services and applications as well as simplicity of installation and configuration.

As their name indicates, Single Shared Quorum clusters use storage design, which enables them to access the same set of disks from every cluster member. While underlying hardware varies widely (and might involve such types of technologies as SCSI, SANs, NAS, or iSCSI, which we will review more closely later), the basic premise remains the same.

Only one instance of any specific resource is permitted at any given time within the cluster. The same applies to Quorum, located on a highly available disk volume, physically connected via a SCSI bus, Fibre Channel links, or network infrastructure to all servers participating in the cluster. Ownership of the shared volume is arbitrated to ensure it is granted only to a single node, thus preventing other nodes from accessing it at the same time (such situation would likely result in data corruption).

This arbitration is typically handled using internal SCSI commands (such as SCSI reserve and SCSI release) as well as bus, Target, or Logical Unit Number (LUN) resets. The specifics depend on the type of storage technology implemented. Note that support for a clustering installation is contingent on strict compliance with the Hardware Compatibility List (which is part of the Windows Server Catalog, containing all clustering solutions certified by Microsoft). Therefore it is critical that you verify which system you intend to purchase and deploy. Quorum, in this case, is implemented as the Physical Disk resource, which requires having a separate volume accessible to all cluster nodes (clustering setup determines automatically whether the volume you selected satisfies necessary criteria).

Unfortunately, the majority of hardware required to set up clustered servers is relatively expensive (although prices of such systems are considerably lower than they were a few years ago), especially if the intention is to ensure redundancy for every infrastructure component, including Fibre Channel and network devices, such as adapters and switches, or disk arrays and their controllers. The cost might be prohibitive, especially for programmers whose sole goal is developing clusteraware software or exploring the possibility of migrating existing applications into clustered environment.

To remediate this issue, Microsoft made such functionality available without specialized hardware setup, by allowing the installation of a cluster on a single server with local storage only (also known as a single node cluster). Obviously, such configuration lacks any degree of high availability, but it has all features necessary for application development and testing. Since local disks are not represented as Physical Disk resources, this clustering model requires using a distinct resource type called Local Quorum when running New Server Cluster Wizard during initial setup, which we will review in details later.

Despite the benefits mentioned earlier (such as a significant level of high availability and compatibility with a variety of hardware platforms, applications, and services), Single Shared Quorum has limitations. The first one is inherent to the technologies used to implement it. For example, configurations relying on SCSI-based shared storage are restricted by the maximum length of the SCSI bus connecting all cluster nodes to the same disk array (which typically forces you to place them in the same or adjacent data center cabinets). This distance can be increased considerably by switching to a Fibre Channel infrastructure, but not without significant impact on hardware cost. Introducing iSCSI and NAS into the arsenal of available shared storage choices provides the same capability at lower prices, but there are still some caveats that restrict their widespread use (e.g., NAS devices are not supported as the Quorum resource). The second limitation is that despite redundancy on the disk level (which can be accomplished through RAID sets or duplexing, with fault-tolerant disks and controllers), Single Shared Quorum still constitutes a single point of failure.

There are third-party solutions designed to address both of these limitations, and with release of Windows 2003 Server-based clustering, Microsoft introduced its own remedy in the form of Majority Node Set (MNS) Quorum. Like Local Quorum, MNS is defined as a separate resource that must be selected during cluster setup with New Server Cluster Wizard. Also like Local Quorum model, dependency on the shared storage hosting Quorum resource is eliminated, without having a negative impact on high availability.

The level of redundancy is increased by introducing additional copies of Quorum stored locally on each node (in the %SystemRoot%ClusterMNS.%Resource GUID%$%ResourceGUID%$MSCS folder, where %ResourceGUID% designates a 128-bit unique identifier assigned to the cluster at its creation). As you can expect, having more than one Quorum instance requires a different approach to preventing "splitbrain" scenario. This is handled by defining a different rule that determines when the cluster is considered operational (which, in turn, is necessary to make its resources available for client access). For this to happen, more than the half of cluster nodes must be functioning properly and able to communicate with each other. The formula used to calculate this number is:

[(total number of nodes in MNS cluster)/2] + 1

where the square brackets denote Ceiling function, returning smallest integer equal to or larger than the result of dividing total number of nodes by two. For example, for a five-node cluster, three nodes would need to be running and communicating for its resources to be available (the same would apply to a four-node cluster). Clearly, setting up a two-node MNS cluster, although technically possible, does not make much sense from availability perspective (since one node's failure would force the other one to shut down all of its resources). For an MNS cluster to function, at least two servers (in a three-node cluster) must be operational (note that with a Single Shared Quorum, a cluster might be capable of supporting its resources even with one remaining node).

Effectively, the rule guarantees that at any given point there will be no more than a single instance of every cluster resource. Clustering service on each node is configured to launch at boot time and to try to establish communication with majority of other nodes. This process is repeated every minute if the initial attempt fails.

This solution introduces additional requirements, since its architecture implies existence of multiple copies of the clustered data (unlike with the Single Shared Quorum model), which must be consistently maintained. Although the clustering software itself is responsible for replication of Quorum configuration across all nodes, this does not apply to services and application specific data. In general, there are two ways of handling this task. The first one relies on mechanisms built into the application (e.g., log shipping in SQL Server 2000/2005 deployments). The second one involves setting up replication on file system or disk block level. This can be handled through software or hardware. In addition, since clustered resources are virtualized, some of the restrictions placed on the Single Shared Quorum model still apply. In particular, for resource fail-over to take place, nodes must be able to detect failure of others through the absence of heartbeat signals. This requires round trip latency between nodes be no longer than 500 ms -- affecting, in turn, the maximum allowed distance between them. They also must be members of the same domain and their public and private network interfaces have to reside on the same subnets (which can be accomplished through setting up two VLANs spanning multiple physical locations hosting cluster nodes).

Furthermore, since Quorum updates are handled via network file shares called %ResourceGUID%$ (associated with the Quorum location listed earlier), both Server and Workstation services (LanManServer and LanManWorkstation, respectively) must be running on all cluster nodes and File and Printer Sharing for Microsoft Networks must be enabled for both private and public network connections.

Thus, when designing architecture it is important to keep in mind the impact the architectural design will have on availability of the MNS cluster. For example, setting up two sites separated by a network link with an equal number of nodes in each will cause both to fail if communication between them is severed (since neither one contains majority of nodes). It might be beneficial in such situation to set up a third site with a single cluster node in it (and dedicated network links to the other two sites), dedicated exclusively to establishing majority node count when needed. Alternatively, you can also force some of the cluster nodes to host resources, although this requires manual intervention.

SCSI Inflection Point: The New Era of Serial Attached SCSI

2008-08-18T00:22:00.000-07:00

Introduction

Necessity can indeed be the mother of invention, and some twenty years ago it gave birth to an innovative interface that went on to achieve worldwide success: parallel SCSI. To be sure, the authors of the original parallel SCSI standards well understood the theoretical superiority of serial interfaces, with their inherently simpler and more robust architecture. Unfortunately, the serial technologies extant two decades ago were woefully slow, adequate for the pedestrian needs of peripheral devices (for example, keyboards, mice) but far too sluggish to efficiently move the multi-megabyte files increasingly found on primary storage devices.

And so parallel SCSI (and its desktop counterpart, parallel ATA) came into being; not because parallel interfaces are fundamentally preferable (quite the contrary), but simply because they were the best option at the time, given the limitations of available technology. The premise, and promise, of parallel architecture was certainly compelling: To bypass the performance bottleneck of a single signal path (sending bytes serially, or one bit after another), just add more signal conductors to enable a byte’s multiple bits to be sent concurrently (or in parallel), each on its own separate data path. The simple logic seemed unassailable—an eight-lane highway can certainly move far more traffic than a single-lane one.

To be sure, practical implementation of parallel interface theory proved decidedly complex, with numerous technological hurdles to overcome. Nevertheless, the fact remains that for its time and purpose, parallel SCSI was a profoundly important advancement in storage interface technology. But, of course, times change....

2004: The SCSI Inflection Point

Flash forward twenty years to the present, where the technological landscape has been radically transformed. Dramatic advances in processor speed (2000x faster), RAM size (1000x greater) and RAM speed (2000x faster) have combined to accelerate system performance to levels unthinkable just a few years ago, let alone two decades past. Yet one aspect of system capability has conspicuously lagged behind: parallel SCSI drive performance. Of course, remarkable advances in areal density have yielded exponential growth in drive capacity (3600x greater), but parallel SCSI transfer rates have achieved only modest gains (32x). And those gains have come grudgingly, as the inherent limitations of the parallel, shared bus architecture have made each speed enhancement increasingly problematic and costly.

Yet while parallel SCSI is clearly approaching its practical performance limits, serial interfaces have gained a new lease on life due to recent breakthroughs in Very Large Scale Integration (VLSI) technology and high-speed serial transceivers. SCSI technology has now reached a fundamental inflection point, where the constraints of parallel interfaces have discouraged further development, while the burgeoning potential of serial interfaces has already delivered remarkable performance and scalability benefits. To better understand the significance of this inflection point, as well as the specific challenges presented by parallel SCSI and the corresponding advantages of Serial Attached SCSI (SAS), a summary table and detailed analyses are provided below.

Parallel SCSI: The Art of Compromise

The very elements that underpinned parallel SCSI’s initial appeal (multiple data paths for greater throughput, a shared bus to enable easy connection of multiple SCSI devices) gradually became stubborn obstacles on the road to improved performance and scalability. Over the years juggling design parameters (for example, increase the clock rate but shorten cable length) to circumvent SCSI’s inherent limitations has wrought hard-won advances, but in recent times the shortcomings of its parallel, shared-bus architecture have become increasingly apparent.

• Skew

Simply put, skew can be defined as the time lag between the first and last bits’ arrival at the receiver. Parallel SCSI transmits data as a series of parallel bits, thus any skew would disrupt that parallel relationship. As clock speeds have increased in response to market demand for faster throughput, bit cell times (that is, time frame in which bits will retain proper synchronization) have steadily shrunk (see Figure 1 below). Prevention of skewing errors thus becomes increasingly difficult. Seemingly innocuous details such as unequal conductor lengths (even variations in how connectors are attached to those conductors) can adversely affect the ability to maintain the bits’ proper arrival times at the receiver.

• Crosstalk

Crosstalk occurs when the signal traveling down a conductor produces high-frequency electromagnetic interference (EMI) that is radiated into nearby conductors, distorting the signals they carry. The faster the signal is moving, the more EMI it radiates. Parallel SCSI cables are particularly susceptible to crosstalk, given the close proximity of so many conductors (each of which both radiates EMI and receives it from its neighbors).

Significant distortion and consequent loss of data integrity can occur if EMI is left untreated. Effective solutions include slowing the speed of the signal and reducing cable length, both obviously undesirable trade-offs. Twisted-pair cable designs have proven effective at reducing crosstalk in external cables, though at the cost of added bulk, loss of flexibility and greater expense. Internal ribbon cables (with their multitude of parallel conductors) are particularly vulnerable to crosstalk—minimizing their length is an effective solution.

• Bus Arbitration and Latency

Parallel SCSI’s shared bus allows no more than two devices (for example, the SCSI controller and a SCSI drive) to access the bus and communicate at any given time. Thus all connected drives must compete for bus access via a relatively complex and timeconsuming bus arbitration procedure that grants access based primarily on a device’s SCSI ID number (fair arbitration modes use additional criteria). Although bus arbitration ensures greater reliability and data integrity, it also degrades performance; the more drives present on the shared bus, the greater the latency for each drive as it waits for the arbitration process to grant it access to the bus. And of course the more arbitration that occurs on the bus, the less the bus is available to actually transmit data, and thus overall performance suffers. Ultra320 SCSI employs packetization and Quick Arbitration and Selection (QAS) to reduce (but not eliminate) the impact of bus arbitration overhead.

• Limited Scalability

In theory, parallel SCSI allows a maximum of 15 devices per 16-bit bus and a maximum of seven devices per 8-bit bus. (In both cases, the SCSI controller itself occupies the remaining SCSI address on the bus.) In practice, parallel SCSI’s shared bus architecture results in progressively less available bus access per device as more devices are added to the bus (see Bus arbitration and latency, above). As such, the actual number of devices that can be deployed on a SCSI bus while maintaining adequate performance may fall well below 15.

• Incompatible with Other Interfaces

When the parallel SCSI and parallel ATA standards were written, compatibility was simply not an issue. The roles of the two parallel interfaces were seen in black and white terms, with no expectation they’d ever be called upon to interoperate in common applications. At one end of the spectrum was SCSI, clearly intended for enterprise duty; at the other was ATA, optimized for use in desktop computers.

Two decades later, deploying desktop-class drives (first parallel ATA and now Serial ATA) for near-line and other light-duty storage has become common practice in the enterprise. Parallel SCSI’s incompatibility with these popular drive types exacerbates the logistical (and financial) headache of supporting multiple, mutually exclusive interfaces.

• Cable Expense

Early SCSI standards allowed only moderate cable lengths (six meters for SCSI-1, dropping to three meters with several subsequent increases in bus speed). The introduction of LVD signaling (see Point-to-Point Architecture, below) raised the maximum cable length to 12 meters. This came at a price, however, as the sheer number of conductors required (and the substantial shielding necessary to protect them from RFI) contributed to the high cost of premium SCSI cables. Less expensive cables may employ marginal materials and/or construction techniques, with deleterious (and often maddeningly intermittent) effects on bus performance and stability. Internal ribbon cables are highly susceptible to crosstalk (see above), and thus should be kept as short as possible.

• Bulky Cables/Connectors

Parallel SCSI’s bulky cabling and connectors clutter enclosures, inhibit airflow/cooling and preclude use with small form factor (such as 2.5-inch) drives in dense computing environments.

• Not Hot Pluggable

No activity can be present on the bus when SCSI devices are added or removed, effectively dictating that the system be powered down. As should be obvious, this can have severely negative effects on system uptime and availability. As a result, SCSI drive changes are often deferred to periods of lowest system activity, thus delaying timely deployment of needed resources.

• Manual Device ID

In order for the SCSI controller to recognize and communicate with the SCSI devices on the bus, a unique SCSI ID address must be manually assigned to each device. SCSI ID numbers range from 0 to 7 on an 8-bit bus and 0 to 15 on a 16-bit bus. The address need not correlate with the device’s physical position on the bus. Adding drives to a bus without ascertaining the ID numbers already in use can lead to SCSI address conflicts (wherein two devices have the same ID number on the same bus). Such conflicts can manifest themselves as instability and erratic performance, up to and including data corruption.

• Termination

The first and last physical devices on the SCSI daisy chain (irrespective of their SCSI ID numbers) must be terminated in order to absorb signals when they reach the ends of the bus, thus minimizing reflections back onto the bus. Passive terminators are typically employed, but can prove problematic, in which case active terminators are indicated. When SCSI drives are added to or moved on the bus, it’s not uncommon for the necessary termination changes to be overlooked. Incorrect/incomplete termination can contribute to significant performance degradation (including data corruption).

Serial Attached SCSI: The Direct Approach

Serial Attached SCSI takes a decidedly straightforward and direct approach to achieving outstanding performance, scalability and flexibility. Its point-to-point, serial architecture is far simpler and more robust than that of its parallel predecessor, yet offers significantly higher throughput (3.0 Gbits/sec, with a clear roadmap to 12.0 Gbits/sec) as well as vastly superior scalability. In response to the growing demand for more cost-effective, rationalized storage solutions, SS incorporates seamless compatibility
with Serial ATA (SATA). IT managers can easily deploy a mix of SAS and SATA drives to most efficiently meet their needs.

• Point-to-Point Architecture

The elegant simplicity of Serial Attached SCSI’s point-to-point architecture pays numerous dividends. Of course, serial data transfer means there are no skew or timing issues to consider. Bits are simply sent one after the other—always sent and received in proper order, always departing and arriving when expected. In addition, point-to-point cabling ensures there is a discrete, dedicated signal path for every SAS device attached. Without a shared bus, SAS arbitration is a straightforward switching process (each attached device is always immediately available on its dedicated bus, minimizing latency), consuming vastly less overhead than parallel SCSI’s bus arbitration and thus ensuring substantially higher throughput.

But SAS’s point-to-point architecture offers other, less obvious benefits. For example, Low Voltage Differential (LVD) signaling (see Figure 2, below) has established itself as a highly effective means to drive high-speed signals over long cables while minimizing vulnerability to environmental noise (for example, RFI). Utilizing the differential signal between a pair of conductors promotes immunity to common-mode noise, while the low voltages made possible by this approach enable increased signal speeds that would be unattainable at higher voltage levels.

While Ultra320 SCSI requires an imposing 32 signal conductors (two for each of the 16 data paths) to implement LVD signaling, SAS needs only four. Fewer conductors require less power to drive the signals, are far less susceptible to crosstalk, and result in much smaller and less intrusive cabling.

• Full-Duplex, Dual-Port Design

Full-duplex operation doubles effective throughput by enabling simultaneous signal transfers in both directions. To provide for high availability and greater uptime, dual data ports ensure that if one SAS host controller fails, the extra data port can maintain uninterrupted communication with a second controller. In addition, these two ports can be combined into a single wide port for even higher throughput.

• Enhanced Scalability

SAS was specifically designed to maximize the ease with which drives can be added to boost both capacity and throughput. In concert with SAS’s point-to-point architecture (see above), high-speed switches known as expanders enable quick aggregation of many drives, allowing a single SAS domain to contain up to 16,384 devices (128 maximum SAS devices per edge expander x 128 maximum edge expanders per fan-out expander) without performance degradation. And multiple SCSI domains can easily be interconnected to achieve exceptional levels of data availability.

• Compatibility with SATA

By ensuring Serial Attached SCSI cables/connectors, backplanes, expanders and host bus adapters (HBAs) are fully compatible with Serial ATA drives, SAS ensures the freedom to seamlessly adapt as storage needs inevitably change and evolve. SAS disc drives are clearly the best choice for mission-critical enterprise use where transactional/online performance and reliability are crucial, while SATA disc drives are cost-effective for lighter-duty use, such as near-line and backup/restore storage. When storage priorities shift, it’s a simple matter to alter the drive mix by plugging in additional SAS or SATA drives.

Furthermore, SAS/SATA compatibility will significantly reduce the cost and complexity of the data center by minimizing the number of individual components that must be qualified, inventoried and maintained. Such component rationalization also places fewer demands on management resources and support personnel.

• Longer Effective Cable Length

A maximum cable length of eight meters (approximately 25 feet) facilitates connections to both direct-attached storage and discrete storage arrays deployed near the server. SAS’s point-to-point architecture enables this maximum cable length to be used for each dedicated connection between two devices, thus allowing thousands of feet of cabling in a SAS domain. The maximum length allowed is not additive of all connections, as it is on a parallel SCSI bus.

• Compact Cabling/Connectors

SAS connectors and cables are far smaller than comparable parallel SCSI pieces; this simplifies cable routing, saves space and improves airflow/cooling in system cabinets, and ensures SAS connectors easily fit on small form factor devices.

• Hot Pluggable/Hot Swappable

SAS’s hot plug capability enables drive swapping without system shutdown, thus ensuring uninterrupted data availability. Blind mate connectors (designed to positively lock in place, eliminating the need to visually verify a connection is properly seated) ease connection in cramped or inaccessible installations.

• Worldwide Unique Device ID

Every SAS port and expander has a worldwide unique 64-bit SAS address (derived from the same namespace as the Fibre Channel Port Name), burned into the device’s firmware at the time of manufacture. This eliminates the need to manually set SCSI ID numbers via jumpers or switches, as well as any possibility of device ID conflicts when installing/moving SCSI devices or expanders.

• Termination Built In

SAS’s point-to-point architecture provides a discrete signal path between any two devices in a SAS domain, as well as clearly identifying both ends of that dedicated bus. Without the need to accommodate a variable number of devices on a shared bus, all SAS devices are terminated by default—no user intervention required.

SCSI Commands Retained

While Serial Attached SCSI overcomes many of parallel SCSI’s interface limitations through the use of modern serial-based technologies, it also maintains the core strengths of its parallel predecessor by integrating existing SCSI commands. SCSI’s robust, mature command set employs sophisticated features such as Cyclic Redundancy Check (CRC) to ensure rock-solid data integrity, even in the most rigorous enterprise environments. Furthermore, the SCSI commands interface with a broad variety of storage management and enterprise application software. By incorporating this command set, SAS also protects the enterprise’s current investment in SCSI software, middleware and drivers.

In addition, SAS also preserves the value of the enterprise’s vast SCSI intellectual capital garnered over many years of configuring, deploying and maintaining SCSI storage solutions. IT professionals can continue to tap this wellspring of SCSI knowledge and experience, helping to drive innovative and effective SAS deployments throughout the enterprise.

Conclusion

Serial Attached SCSI was conceived as a best of breed storage solution, retaining the time-tested SCSI command set of its forebearer and melding it with the most up-to-date serial technologies available. The result is an enterprise-class interface that blends four compelling attributes (performance, scalability, compatibility and SCSI integration) to form a cohesive, comprehensive solution. Serial Attached SCSI’s forward-thinking design brief ensures SAS will meet the enterprise storage needs of today…and tomorrow.

Three Steps Forward, No Steps Back: Investment Protection With Serial Attached SCSI

2008-08-18T00:19:00.000-07:00

Introduction

More than a few veteran IT professionals have been known to instinctively flinch at the phrase "technology breakthrough." Why? Not because they eschew technical progress (far from it), but simply because hard-earned experience has shown them that deploying fundamentally unfamiliar solutions can be a "two steps forward, one step back" proposition. For example, many of the gains offered by a new hardware or software storage solution can be initially offset by the time and expense needed to integrate it into the current storage environment-including modification or replacement of existing infrastructure, and retraining IT personnel to deploy, maintain and troubleshoot the unfamiliar solution.

Furthermore, it's clear that storage demands will continue to grow exponentially in the years to come. New solutions that fail to meet these demands may quickly fall into costly obsolescence. Is there a clear roadmap for increased performance? Can the solution seamlessly scale to address ever-growing throughput and capacity needs? Is it compatible with other storage devices used in the enterprise? Solutions that fail this simple test may well entail a technological (and fiscal) step backward.

SAS Takes Three Steps Forward, No Steps Back

Serial Attached SCSI (SAS) was specifically developed to reject the "two steps forward, one step back" phenomenon described above. By design it incorporates four complementary attributes that ensure investment in SAS storage solutions remains protected, both today and long thereafter. Specifically, SAS makes three significant strides forward in performance, scalability and compatibility, while standing firm on the proven strengths of SCSI (enterprise-class reliability, mature and robust command set, enormous installed base of SCSI applications). This "three steps forward, no steps back" approach defines a new paradigm for value in enterprise storage solutions, particularly welcome given today's tightly constrained IT budgets. To better understand SAS's unprecedented level of storage investment protection, it's instructive to examine the four elements that combine to form it: Performance, Scalability, Compatibility and SCSI Integration.

SAS Performance (Step One)

It's clear that explosive growth in transactional enterprise data will continue unabated, posing significant performance challenges for any storage solution. With a transfer rate of 3.0 Gbit/sec (and a clear roadmap to 12.0 Gbit/sec), Serial Attached SCSI utilizes full-duplex, point-to-point architecture to ensure superior performance now and in the foreseeable future. Full-duplex operation doubles effective throughput by enabling simultaneous signal transfers in both directions. To provide for high availability and greater uptime, dual data ports ensure that if one SAS host controller fails, the extra data port can maintain uninterrupted communication with a second controller. In addition, these two ports can be combined into a single "wide port" for even higher throughput.

SAS Performance Feature

1. 3.0 Gbit/sec transfer rate, roadmap to 12.0 Gbit/sec
2. Point-to-point architecture
3. Full-duplex operation

Investment Protection Benefit

1. Sets new standards for enterprise-class performance today, ensures increased throughput in the future
2. Delivers maximum bandwidth by providing dedicated signal path for each device,banishes parallel SCSI shared-bus slowdowns
3. Doubles effective throughput by enabling signal path to handle data transfers in both directions at once

SAS's point-to-point cabling maximizes bandwidth by providing a dedicated signal path for each device, eliminating the shared-bus bandwidth slowdowns and bottlenecks of parallel SCSI. This point-to-point architecture also enables vastly greater scalability (see below) than its daisy-chained predecessor. These technologies, complemented by the advanced SCSI command queuing discussed earlier, work together to ensure SAS performance will keep pace with high-demand enterprise traffic well into the future.

SAS Scalability (Step Two)

As the volume of enterprise data continues its relentless expansion, any viable long-term storage strategy must include simple, cost-effective scalability as an essential component. SAS was specifically designed to maximize the ease with which drives can be added to boost both capacity and throughput. In concert with SAS's point-to-point architecture, inexpensive switches known as expanders enable quick aggregation of many drives, allowing a single SAS domain to contain up to 16,384 devices (128 maximum SAS devices per edge expander x 128 maximum edge expanders per fan-out expander) without performance degradation. And multiple SCSI domains can easily be interconnected to achieve remarkable levels of data availability.

SAS Scalability Feature

1. Expanders enable aggregation of over 16,000 devices vs. 16 devices per parallel SCSI bus
2. Improved device addressability
3. Longer cabling distances

Investment Protection Benefit

1. Inexpensive switches enable seamless scalability, preserve performance in high-demand environments ( such as frequent, concurrent access by multiple initiators or host)
2. Worldwide unique device IDs eliminate parallel SCSI address conflicts and termination problems
3. Enables connections to both direct-attached storage and discrete storage arrays deployed near server

Furthermore, the specifications for SAS expanders also include the ability to accept both SAS drives and Serial ATA (SATA) drives as end devices. This compatibility (see below) enables a single SAS domain to contain a blend of SAS and SATA drives, in order to achieve the optimal balance of performance and economy in a specific application.

SAS Compatibility (Step Three)

The freedom to select the most appropriate disc drive for a given task is key to maximizing storage efficiency and cost-effectiveness. Yet until now IT managers have had few options in that regard. SAS provides much-needed flexibility by leveraging existing SATA standards to enable compatibility with its serial cousin. By ensuring Serial Attached SCSI cables/connectors, backplanes, expanders and host bus adapters (HBAs) are fully compatible with Serial ATA drives, SAS gives unprecedented freedom to easily adapt as storage needs inevitably change and evolve.

Now IT managers can quickly deploy storage solutions tailored to their specific needs: SAS disc drives are clearly the best choice for mission-critical enterprise use where transactional/online performance and reliability are crucial, while SATA disc drives are cost-effective for lighter-duty use such as near-line and backup/restore storage. Should storage priorities shift, it's a simple matter to alter the drive mix by plugging in additional SAS or SATA drives.

SAS Compatibility Feature

1. SAS backplanes, HBAs and expanders compatible w/SATA
2. SAS drives available in both standard 3.5-inch and small (2.5-inch) form factors
3. SAS platform can accommodate both SAS and SATA devices

Investment Protection Benefit

1. Connectivity for multiple device types lowers TCO, gives flexibility to specify best drive for intended use: SAS for transactional/online performance and reliability, Serial ATA for nearline and backup/restore storage

2. 3.5-inch drives ensure compatibility with standard enclosures, 2.5-inch drives enable greater drive count in dense computing environments for higher IOPS/U, reduced power consumption

3. Standardization on one platform reduces number of components to qualify and inventory

Beyond the obvious efficiency of specifying the optimal solution for a given application, standardizing on the SAS platform also protects IT investment by employing a single storage system to accommodate both SAS and SATA devices. This will significantly reduce the cost and complexity of the data center by minimizing the number of individual components that must be qualified, inventoried and maintained. Such component rationalization also places fewer demands on management resources and support personnel.

SCSI Integration (No Steps Back)

Boasting over two decades of rigorous development and refinement, it's no surprise that SCSI has firmly established itself as the overwhelming I/O technology choice for high-availability enterprise storage solutions. SCSI's robust, mature command set interfaces with a broad variety of storage management and enterprise application software. By building upon this proven SCSI foundation, SAS protects current investment in these software solutions. In addition, SCSI integration ensures migration to SAS is a straightforward process, with no need for the system-level workarounds required by ATA.

But SAS doesn't just protect the enterprise's existing investment in SCSI software, middleware and drivers-it also preserves the value of the enterprise's vast SCSI intellectual capital, garnered over many years of configuring, deploying and maintaining SCSI storage solutions. IT professionals can continue to tap this wellspring of SCSI knowledge and experience, helping to drive innovative and effective SAS deployments throughout the enterprise.

SAS SCSI Integration Feature

1. Utilizes existing SCSI command sets
2. Advanced SCSI command queuing
3. Leverages SCSI's proven reliability

Investment Protection Benefit

1. Protects enterprise's investment in SCSI software (and accumulated SCSI knowledge and expertise), eases migration to SAS
2. Will maintain high performance as demanding enterprise traffic conditions (such as frequent, concurrent access by multiple initiators or host) escalate
3. Ensures enterprise-class reliability and durability for mission-critical use

The specific details regarding SAS utilization of the SCSI command set are straightforward: SAS employs three different protocols to transport information over its serial interface-Serial SCSI Protocol (SSP), SCSI Management Protocol (SMP) and Serial ATA Tunneling Protocol (STP). SSP communicates with SAS devices and existing SCSI software, SMP manages SAS's point-to-point topology, and STP enables SAS controllers to identify and communicate with Serial ATA devices. Current SCSI software and middleware merely require minor SMP and STP modifications to function in a SAS environment.

Conclusion

As constrained budgets put growing pressure on IT departments to justify expenditures, storage solutions that protect investment in both the short- and long-term are increasingly compelling. SAS takes a holistic approach to achieving this goal, blending four compelling attributes (performance, scalability, compatibility and SCSI integration) to form a cohesive, comprehensive solution. As a result Serial Attached SCSI sets a new standard for efficient enterprise storage solutions, both today…and tomorrow.

Rationalizing the Data Center

2008-08-17T23:00:00.000-07:00

Executive Summary

Several years ago the enterprise IT infrastructure, already growing at an impressive pace, kicked into high gear. Spurred by the red-hot Internet economy and widespread concern over the Y2K crisis, infrastructure investment leapt to unprecedented levels. In the rush to deployment, little time was allotted to ensure proper integration and coordination of these systems. When the economic bubble burst, many enterprises found themselves awash in expensive, complex systems that were both inefficient and maintenance-intensive.

In today’s more thoughtful economic climate, rationalizing the data center is crucial to reducing costs and improving productivity in the short term, while laying the groundwork for more strategic, measured growth in the long term. Reducing the number and size of data centers, consolidating the plethora of systems in those centers into fewer, more efficient configurations, and deploying more targeted, cost-effective storage solutions in those systems are key elements of rationalization.

Four distinct attributes of Serial Attached SCSI (SAS) make it uniquely suitable for achieving optimal enterprise storage rationalization: performance, compatibility, scalability and flexibility. The greater the flexibility in choosing and deploying storage systems, the more likely enterprise IT managers can achieve optimal performance and cost-effectiveness. SAS offers an unparalleled degree of freedom to create high-performance storage solutions that can easily scale and adapt as needs change.

Infrastructure in Overdrive

Just a few short years ago the enterprise was at the forefront of stupendous economic growth, scrambling to keep up with a data explosion unprecedented in the Information Age. The Internet’s meteoric rise was continuing unabated, e-business and e-commerce had quickly established themselves as essential business models, and every day seemed to bring another flurry of mergers and acquisitions. Harried IT managers found themselves in reactive mode, adding new data storage solutions at breakneck speed with little time (or fiscal incentive) to calculate the long-term financial consequences. IT budgets were plump and business opportunities apparently unlimited. Woe indeed to any IT department that jeopardized potentially lucrative new ventures by failing to ensure sufficient storage capacity and bandwidth.

And in the midst of this economic euphoria, the specter of catastrophic data loss due to the Y2K bug provided yet another justification for costly infrastructure upgrades. With business revenues running at fever pitch, almost any expenditure necessary to safeguard those revenues was deemed acceptable.

Not surprisingly, enterprise IT investment soared to then-record levels as firms around the globe poured money into state-of-the-art storage solutions (as well as brawny servers and powerful business applications) to accommodate their burgeoning businesses. Of course, it was also necessary to hire legions of additional IT personnel to deploy and maintain those sophisticated assets.

And then the economic bubble burst…

Cost and Complexity

In the aftermath, characterized by rapidly dwindling revenues, the enterprise was forced to critically examine every aspect of operations for possible cost-cutting opportunities, and found itself drowning in a plethora of expensive, complex IT infrastructure. It soon became apparent that in the rush to deploy these cutting-edge solutions, insufficient time had been taken to integrate them into existing infrastructures or even ensure proper coordination between newly-deployed systems. Constant mergers and acquisitions exacerbated the problem by demanding that dissimilar systems, often at widely remote locations, be jury-rigged together.

The result was a hodgepodge of redundant and often incompatible infrastructure, with mediocre efficiency and vexing reliability. When massive budget cuts made the grim reality of IT staff layoffs an almost daily ritual, the ability to effectively monitor and troubleshoot these problematic systems quickly diminished.

Of course technological progress marches on, even in sluggish economies, and soon much of the enterprise’s costly infrastructure was rendered obsolete. Beyond the expected advancements (faster and higher-capacity disc drives, speedier processors, improved I/O, and so on), the latest generation of enterprise systems also marked a newfound sensitivity to tightly-constrained IT budgets. A new breed of economical, minimalist systems (for example, ATA-based NAS and blade servers) quickly found favor with cost-conscious IT managers keeping a close eye on new expenditures.

Unfortunately, though many businesses were long overdue for infrastructure upgrades, the stagnant marketplace ensured there were simply no funds available for replacing outdated systems with newer, more efficient ones. And as noted above, razor-thin budgets even precluded adequate IT staff to maintain those obsolete systems. For many companies it was simply a question of making do with the existing infrastructure, of hunkering down and weathering the fiscal storm.

Lessons Learned

Happily the storm has now passed and stoic resolve is gradually being replaced by cautious optimism for sustained growth over the long term. As difficult as the economic downturn was, it provided a valuable lesson on the importance of a fundamental enterprise tenet: IT infrastructure investment must emphasize long-term goals rather than short-term prospects, must be thoughtfully proactive rather than hurriedly reactive.

As IT budgets are slowly increased and pent-up demand for new systems is gradually addressed, the first task for savvy IT managers is to evaluate their current infrastructure and determine how today’s more targeted solutions can help lower costs and improve efficiency now, while still remaining viable in the long term. The most direct and effective means to achieve this goal is rationalization of the data center: simply put, paring down the multitude of decentralized data centers (and servers they house) to fewer, smaller and more centralized centers while simultaneously consolidating the storage systems within to ensure optimal performance and efficiency.

The cost savings that can be realized from data center rationalization are significant: less hardware and smaller physical spaces yield obvious economies, but they also enable fewer IT personnel to administer and maintain the systems, an important factor when considering data center overhead.

How to Rationalize Enterprise Storage With Serial Attached SCSI

Four distinct attributes of Serial Attached SCSI (SAS) make it uniquely suitable for achieving optimal enterprise storage rationalization:

• Performance
• Compatibility
• Scalability
• Flexibility

Storage Rationalization Through SAS Performance

With a transfer rate of 3.0 Gbits/sec (and a clear roadmap to 12.0 Gbits/sec), Serial Attached SCSI accelerates storage to remarkable speeds by utilizing full-duplex, point-to-point architecture. Full-duplex operation enables signal transfers to take place in both directions simultaneously, doubling effective throughput. Dual data ports ensure high availability—in the event one SAS host controller fails, the extra data port maintains uninterrupted communication with a second controller. In addition, these two ports can be combined into a single “wide port” for higher throughput. Point-to-point cabling ensures maximum bandwidth by providing a dedicated signal path for each device, banishing the shared bandwidth slowdowns and bottlenecks of its parallel predecessor. This also means the end of daisy-chain address conflicts and termination headaches.

SAS performance is further enhanced by integrating the same SCSI command sets that support today’s enterprise storage environment, ensuring superior throughput under high-demand enterprise traffic conditions, such as e-commerce applications, which entail frequent, concurrent access by multiple initiators or hosts.

Storage Rationalization Through SAS Performance—Benefit: SAS performance makes it a viable, highly cost-effective alternative to Fibre Channel when remote servers are consolidated in centralized data centers (that is, local-attach environments).

Storage Rationalization Through SAS Compatibility

Serial Attached SCSI cables/connectors, backplanes and host bus adapters (HBAs) are fully compatible with Serial ATA (SATA) drives. Furthermore, SAS employs three different protocols to transport information over its serial interface, one of which (Serial ATA Tunneling Protocol, or STP) enables SAS controllers to identify and communicate with Serial ATA devices.

SAS provides the path through which deployment of SATA in the enterprise offers the greatest strategic benefit—the freedom to select the most appropriate disc drive for a given task. SAS disc drives are clearly the best choice for mission-critical enterprise use where transactional/online performance and reliability are crucial, while SATA disc drives are cost-effective for lighter-duty use such as near-line and backup/restore storage.

SAS compatibility also ensures maximum value for your IT investment by enabling a common enclosure to house both SAS and SATA devices on the same SAS backplane. Enterprise users can seamlessly deploy a mix of SAS and SATA drives to most efficiently meet their needs, saving both space and money. Smaller businesses can initially purchase SAS backplanes for use with SATA drives, knowing that as business increases and storage demands grow they can simply add SAS drives without needing to upgrade their enclosures.

Storage Rationalization Through SAS Compatibility—Benefit: SAS and SATA drives can be consolidated on a common backplane and housed in a single enclosure, enabling one SAS-based subsystem to economically handle the full gamut of enterprise storage duties. Consolidating multiple remote servers in a centralized data center can yield even greater savings if SAS drives are then deployed in place of Fibre Channel SANs. SAS drives can be deployed for mission-critical duty in place of parallel SCSI while being housed in a common cabinet with SATA drives used for near-line storage.

Storage Rationalization Through SAS Scalability

Scalability was never among parallel SCSI’s strengths, and a key goal of Serial Attached SCSI was to vastly improve the ease with which drives could be added to increase capacity and throughput. In concert with SAS’s point-to-point architecture, inexpensive switches known as expanders enable quick aggregation of many drives, allowing a single SAS domain to contain up to 16,384 devices (128 maximum SAS devices per edge expander x 128 maximum edge expanders per fan-out expander) while preserving performance.

Furthermore, the specifications for SAS expanders also include the ability to accept
both SAS drives and SATA drives as end devices, thus enabling a single SAS domain
to contain a hybrid SAS/SATA deployment of enormous scale.

Storage Rationalization Through SAS Scalability—Benefit: SAS expanders enable quick, inexpensive scalability far beyond that of parallel SCSI, and vast heterogeneous SAS/SATA configurations are possible.

Storage Rationalization Through SAS Flexibility

SAS drives will be available in the industry-standard, 3.5-inch form factor for storage systems utilizing a common backplane. A single storage subsystem will thus be able to house a low-cost, 7200-RPM SATA drive in the same enclosure as the preferred online enterprise solution, the industry-standard, 15,000-RPM SCSI drive.

SAS drives will also be offered in the new 2.5-inch small form factor, an ideal pairing of complementary technologies. SAS’s compact connectors and seamless scalability are the perfect match for these slim drives and the space-efficient applications in which they excel. For denser computing environments in which raw capacity takes a back seat to higher throughput (IOPS/U), 2.5-inch SAS drives will play an increasingly prominent role.

Note that choosing 2.5-inch SAS drives for transactional applications (for example, database storage for ERP and CRM software) doesn’t preclude use of 3.5-inch SATA drives for periodic backup and restore. For example, servers and storage subsystems of varying sizes (for example, 1U, 2U, 4U) stacked on top of one another in a cabinet
can transfer data interchangeably between SAS and SATA drives. Depending on the
cabinet’s configuration, Serial Tunneling Protocol (STP) could be accomplished at an
HBA or expander, thus eliminating the need for SATA and SAS drives to share the same backplane.

Storage Rationalization Through SAS Flexibility—Benefit: SAS offers a choice of standard 3.5-inch form factor drives or new 2.5-inch drives, which boast lower power consumption, easier cooling, 70 percent smaller size and greater IOPS/U in high-density computing environments.

SAS expanders enable 3.5-inch SAS, 3.5-inch SATA and the new 2.5-inch SAS drives to share a single rack. For dense computing environments 2.5-inch SAS drives ensure energy savings and superior IOPS/U.

Conclusion

As IT budgets continue to undergo intense scrutiny, the pressure to rationalize enterprise storage systems has never been greater. That said, the ultimate goal of any IT manager has always been to employ the best, most cost-effective storage solutions for the applications at hand. The greater the flexibility in choosing and deploying those solutions, the more likely optimal performance and cost-effectiveness will be achieved. Serial Attached SCSI offers an unprecedented degree of freedom to create high-performance storage solutions that can easily scale and adapt as needs change. As such, SAS truly heralds an exciting new era in enterprise storage.

Change is Good

2008-08-17T22:52:00.000-07:00

Managing Outsourcing Transition to Maximize Value

Outsourcing changes the way an organization is managed. A successful relationship requires both parties to recognize and accept a different role. Generally speaking, in the new environment the vendor’s role is to deliver efficiency—skills, people, processes, measurement, and accountability. The client’s responsibility is effectiveness—understanding and communicating the right things to do for the organization.This requires business knowledge, awareness of corporate culture, relationships, and internal networks.

The transition to outsourcing requires a shift in responsibility whereby the client becomes a facilitator, a negotiator, and a general manager, while the vendor becomes the process expert and implementer.While these new roles can allow each party to focus on its core areas of expertise and enhance its contribution, the introduction of change can also foment resistance and resentment, thereby threatening the relationship.

This Compass white paper examines the challenges of successfully managing the changes that characterize outsourcing, and focuses on strategies and tactics that can defuse conflict and ensure cooperation toward achieving mutually defined and shared goals. Specifically, the author analyzes the role of trust in outsourcing relationships and its importance in facilitating a positive transition.

Adapting to a New Role

One of the central risks of outsourcing is that the client organization will be unprepared for the new role it must assume. Rather than “doing IT”—developing applications, maintaining and upgrading systems, and troubleshooting—the retained function’s primary responsibility suddenly becomes to manage the outsourcer, and to apply business knowledge and communication and negotiation skills.The problem is, pure technical expertise doesn’t necessarily translate into business or managerial acumen. Put more bluntly, the guy running your data center may not necessarily be the guy who should be managing your outsourcer.

This disconnect in skills and experience is a major contributor to outsourcing dysfunction in general, but is especially toxic to deals aiming to achieve strategic value. If retained IT staff are unprepared to tackle the new and challenging management and business-oriented tasks the outsourcing relationship demands, there’s a high risk they’ll revert to their “comfort zone” by carving out duties more in line with traditional areas of expertise.The result is duplicated effort, the re-creation of silos of IT activity, and a poorly managed relationship.

The challenge for the retained function is to let go of the “doing” to focus on the “steering.” In many instances this requires learning new skills—through training sessions, workshops, interaction with counterparts, and colleagues. Again, sounds simple, but the time and cost required for this training must be addressed in the outsourcing planning, negotiation, and governance process. Roles and responsibilities must not only be clearly understood, they must be embraced by both sides. And then both sides must deliver.

Success requires a collaborative approach to define roles and to allow each organization to hold the other accountable for meeting its commitments.

Since measures are often used to gauge the outsourcer’s success at meeting commitments, the outsourcer should also be able to measure how well the client organization is meeting its commitments.An annual process that establishes shared goals and objectives, defines measurement mechanisms, and aligns incentive programs to attain these goals and objectives could encourage a collaborative and successful approach to management.

Micromanagement vs. Abdication

Client organizations must balance between micromanagement and abdication of management responsibility. Micromanagement happens when an organization has to do things its way. So rather than telling the outsourcer what needs to be done, they tell the outsourcer how to do the job.This negates the outsourcer’s operational expertise and creativity and results in duplicated work. The outsourcing team becomes nothing more than a set of extra hands.

Vendors working for a micromanaging client, meanwhile, face a dilemma. One option is to push back and risk a serious client confrontation.The other is to try to manage the untenable situation by telling themselves “the client’s always right”—in which case everyone loses.

In such a situation, the level of trust that defines the relationship is key. A client that trusts the outsourcer won’t feel compelled to police every task. And a secure outsourcer who recognizes a problem won’t hesitate to speak up.

If the client has to micromanage the outsourcer, then perhaps they have the wrong outsourcer.However, if the client simply prefers to micromanage the outsourcer because this makes them more comfortable and secure, then perhaps the client has the wrong person handling the relationship.That said, micromanagement should not be confused with holding the outsourcer accountable. Accountability is key to success.

Equally destructive is abdication of management responsibility—not taking any ownership of decisions, commitments, or outcomes.The client cannot watch from the sidelines and then blame the outsourcer if something goes wrong.“If you are not part of the solution, then you are part of the problem.” The adage applies to client organizations as well.

Cultural Fit

Can outsourcing relationships survive cultural incompatibility? Let’s say, for instance, that the client organization’s culture is buttoned-down and conservative, while the outsourcer’s style leans toward t-shirts with logos and nerf ball battles during meeting breaks. Or vice-versa. Can two such organizations work effectively together?

Superficial issues of style can and should be overcome if compelling reasons exist for two organizations to work together in terms of capabilities and requirements. That said, these differences should be recognized and addressed. Don’t dismiss them as trivial.

Differences in style can lead to miscommunication—which in turn can fuel a breakdown in trust that threatens the relationship. Compass has observed situations where a detail-oriented client peppered an outsourcing vendor with a barrage of specific questions.The outsourcer felt the client was hostile and became defensive, while the client believed the outsourcer was unprepared. In this instance, a better understanding by each party of the other’s approach could have defused the perceived conflict.

Defusing the “Victim” Mentality

In planning a transition to outsourcing, how much turnover within the client organization is desirable and realistic? On the one hand, leaving the client organization completely intact might be good for morale, but would be cost-prohibitive and essentially defeat the whole purpose of outsourcing. At the other extreme, a lack of careful planning and management of people issues could lead to a mass exodus of client staff and result in a crippling knowledge drain.

Whatever approach is taken, fostering a sense of involvement, ownership, and choice is essential. Resentment festers when change is imposed on staff in a “take it or leave it” fashion. In such situations, employees easily succumb to a victim mentality—and victims are not effective change agents. If, however, the transition is presented through a process of evaluation and feedback, those most affected by outsourcing will be more likely to trust that the change will be positive, and to develop a vested interest in making the process work.

Service Levels vs. Service

Many clients today are clearly unhappy with their outsourcing vendors, convinced that they’re not getting the service they’re paying for and entitled to. But in most instances, vendors can demonstrate and document that they’re meeting all service targets specified in the contract. Under the circumstances, they argue, the client has no legitimate complaint.

This scenario illustrates a curious disconnect that often characterizes service level agreements in outsourcing deals. Vendors use their negotiating skills to establish service levels that pose little risk. Subsequently, clients are dumbfounded by vendor reports that service level targets for a critical system have been met—even though that system wasn’t available when needed. By playing a numbers game based on averages and minimum service level targets, vendors are often able to spin a story of achievement, which only fuels
client resentment and cynicism.

Service level agreements and performance targets are obviously essential to an effective outsourcing agreement. However, simply meeting a minimum service level to fulfill a contractual obligation—and then telling the client they should be satisfied with that—ultimately becomes counter-productive.

Similarly, penalties and incentives can be useful tools in managing an outsourcer’s performance, particularly when applied to performance that impacts critical business goals. In practice, however, they require careful management. For example, focusing irrelevant penalties on every little miss only encourages a risk-averse approach by vendors and does nothing to promote quality service. Incentive clauses, meanwhile, often give the vendor a bonus simply for doing their job, and add no value to the client.

Careful development of penalty and incentive programs is a worthwhile investment, as long as both organizations are subject to the penalties and incentives. Such an exercise clearly communicates priorities and defines how each organization will be measured.

Trust and Strategic Value

One of the most widely discussed shortcomings of outsourcing relationships has been the inability of vendors to deliver strategic added value to the business. There’s plenty of blame to go around: Clients can’t define a value proposition and won’t manage the relationship. Vendors focus on their bottom line and on meeting their operational requirements, ignoring strategic concerns.

That may all be true, but trust and personal relationships play a critical role. Consider: Business value as defined in most outsourcing arrangements requires the vendor to invest time and effort in understanding the client’s business.There’s no guaranteed return on that investment because it can’t be specifically written into an agreement upfront.What’s the incentive for the outsourcer to make that investment in what is, in effect, a new sale that involves risk?

The quality of the client/vendor relationship goes a long way toward answering that question. In a firmly grounded relationship built on trust, the outsourcer will be more likely to invest in value initiatives, because the outsourcer trusts the client to take the investment seriously and in good faith. If trust is lacking, then the outsourcer has little incentive to make the effort, and the client has little reason to value the effort.

The Desire to Trust (Faith)

Trust clearly is essential if organizational change is to be welcomed and seen in a positive light. But trust does not happen magically. It must be earned (a reason to trust must be demonstrated) and it must be retained (don’t betray the trust).Yet the single most important element in establishing trust is the desire to trust. This requires room for forgiveness, but leaves no room for cynicism.

The most successful and effective outsourcing relationships exist where clients readily understand and acknowledge that they have the desire to trust and to make it work, that they have confidence and respect for their outsourcer, and that they believe in the possibilities of what can be achieved by working together.

Conclusions

• A transition to outsourcing requires the retained function to assume a fundamentally new role involving new skills. Implementing and managing this change is risky.

• Clients must find the right balance between micromanagement and abdication. Micromanagement chips away at creativity and effectiveness, while abdication neutralizes the synergies of collaboration and teamwork.

• Differences in style and corporate culture don’t preclude effective outsourcing relationships. However, such differences must be recognized and addressed, lest they lead to serious conflict.

• Effective service level agreements and incentive and penalty clauses must reflect business goals. Incentives and penalties should apply to both the vendor and the client.

• Client and vendor must work together to build effective bridges to facilitate escalation and resolution of issues.

• The client and the outsourcer equally share the responsibility for the success of the outsourced environment.The client sets the tone for the relationship, while the outsourcer must be responsive.

• Don’t create victims.Wherever possible, invite input from those affected by outsourcing decisions. At the least, communicate decisions clearly and unambiguously.Wherever possible, offer people choices.

• The client and outsourcer should collaborate in setting goals and then work toward reaching them. Consistent communication of these goals up and down every level of both organizations is essential.

• Trust and communication are needed to effectively align the client and outsourcing organizations, and to provide meaningful incentives for the vendor to invest in value initiatives.

• Client and outsourcer must listen to each other, share results, and collaborate in the development of continuous improvement plans. Remember that statistics tell only one part of the story. Perception is the only reality. Manage perceptions. Be responsive and be seen to be responsive.

ITIL:What It Is and Why You Should Care

2008-08-17T22:43:00.000-07:00

Introduction
Information Technology Infrastructure Library (ITIL) has been around for 20 years, but interest within the United States has only increased in the last five years. ITIL is becoming the next big thing in Information Technology. It is the new industry buzz-word, the new certification, the new conference, and the new idea that the IT world feels it needs. This paper will describe the origin of ITIL, who controls the ITIL contents, who are
the biggest users of ITIL, and why you should care about it. Throughout this paper, it will be important for you to remember that ITIL describes a framework of processes for the management of IT. Because it is a framework, ITIL does not describe in great detail how any particular process should be implemented.

Where did ITIL come from?

ITIL started in the late 1980s when the British Central Computer and Telecommunication Agency (CCTA), now called the Office of Government Commerce (OGC), made a decision that there should be a better way for Information Technology to function. The CCTA commissioned a study group to develop a new approach to managing Information Technology. From this group came Version 1 of ITIL, which was called GITIM, Government Information Technology Infrastructure Management. Version 1 of ITIL was a great deal different from the present-day version. Part of this difference is due to the gradual maturing of ITIL and changes in the Information Technology industry.

Between the development of Version 1 and the year 2001, the number of documents (books) used within ITIL grew to more than 32. In the year 2000, Microsoft used ITIL as the basis for development of their proprietary Microsoft Operations Framework (MOF). The year 2000 also saw the CCTA merge into the Office of Government Commence (OGC).

Version 2 of ITIL was released in 2001. The present day version contains just 8 books:

• Service Support
• Service Delivery
• Business Perspective
• ICT Infrastructure Management
• Applications Management
• Security Management
• Planning and Implementation
• Software Asset Management

Two of these books, Service Support and Service Deliver, are the heart of ITIL and the focus of the present drive for ITIL adoption.

Because ITIL had its start in the British government, ITIL adoption began there; It then quickly spread to nongovernment organizations within Britain. From Britain, ITIL moved to Europe and Canada, where it has seen heavy adoption. From Canada, ITIL finally made its way to the United States. The adoption of ITIL, by both government and non-government organizations within the US, is gaining momentum. ITIL truly is the next big thing. If ITIL is new to you, then now is the time to learn more.

Who are the players in ITIL?

The following is a list of organizations and companies that have successfully implemented ITIL.(A list of organizations that have implemented ITIL is several sections down. The groups listed in this section are the ones that control ITIL.)

Office of Government Commerce (OGC)

The Office of Government Commence, whose web site can be located at http://www.ogc.gov.uk/, is the owner of ITIL. The mission of the OGC is to work with the public sector as a catalyst to achieve greater efficiency, increase value in commercial activities, and improved success in the delivery of programs and projects. When you look at the OGC, you will see that the scope of their concern for standards is much broader than just the improvement of IT, extending to other diverse areas (even including plumbing standards).

The Stationary Office (TSO)

The Stationary Office (http://www.tso.co.uk) is the largest publisher by volume in the UK, publishing over 15,000 titles a year and providing a comprehensive range of document and publishing services. TSO is the official publisher of the ITIL documentation. The downloadable PDF versions of the old ITIL books can be obtained through TSO’s website. EXIN and ISEB Within ITIL there are a number of certifications for individuals. The owner of the certification and the certification testing is the Dutch foundation Exameninstituut voor Informatica (EXIN) (www.exin-exams.com) and Britain’s Information Systems Examination Board (ISEB) (www.iseb.org.uk). EXIN and ISEB jointly developed the professional certification system for ITIL. This was done in close cooperation with the OGC and Information Technology System Management Forum (itSMF described below). There are three recognized individual certifications: Foundation Certificate in IT Service Management, Practitioner Certificate in IT Service Management, and Manager Certificate in IT Service Management. In addition to certification of individuals working in the industry, there is an organizational certification, BS15000, which is the world's first standard for IT service management. This standard specifies a set of inter-related management processes and is based heavily upon the ITIL framework. BS15000 consists of two parts: BS15000-1 consists of 10 sections:

• Scope
• Terms and Definitions
• Requirements for a Management System
• Planning and Implementing Service Management
• Planning and Implementing New or Changed Services
• Service Delivery Process
• Relationship Processes
• Resolution Processes
• Control Processes
• Release Process

BS15000-2 provides assistance to organizations that are to be audited against BS15000-1 or are planning service improvements.

The Information Technology System Management Forum (itSMF)

ItSMF (www.itsmf.net) is an industry consortium dedicated to managing the cost and quality of IT service management. Members include IT organizations and software and services companies augmented by advisory and review boards of industry leaders, analysts, and customers. Collectively, itSMF USA represents those with a stake in IT service management. The itSMF is involved in two areas. First, the organization provides a forum to address technical and business issues that will enhance the benefits of IT management applications and services. Second, it educates the market about IT service management and its value—in effect, marketing IT service management. Note that www.itsmf.com will take you the international itSMF web site.

Loyalist College

Loyalist College(www.itilexams.com ) is a Canadian college that administers the certification test for individuals for the Americas.

Companies that have implemented ITIL

Hewlett-Packard, Microsoft, and IBM have used ITIL as a base for their own proprietary IT management frame work and created the following tools.

• Microsoft offers Microsoft Operations Framework (MOF)
• HP offers IT Service Management Reference Model
• IBM offers IT Process Model

Many of the chapters within the ITIL’s eight books were written by individuals from one of these companies.

What makes ITIL different?

Over the years many of you have probably been involved in projects related to the improvement of Information Technology such as:

Quality Enablers

• Project Management (http://www.pmi.org/)
• Balanced Scorecards (http://www.balancedscorecard.org/)
• Six Sigma (http://www.isixsigma.com/library/content/c010204a.asp)

Quality Methodologies

• ISO-9000 (http://www.iso.org/)
• TQM / Deming (http://www.deming.org/)
• Capability Maturity Model (www.sei.cmu.edu/) (http://www.itservicecmm.org/)

All of these programs provide methodologies that can be used to improve the processes that you have in place. However, these methodologies provide little or no guidance about which processes are required for IT to function well. ITIL provides a guide to the framework of processes required to run IT as a Business - for the Business and the relationship between those processes. ITIL states that you need to have one function/department and ten processes. For example:

The Service Desk (function)

• Incident Management
• Problem Management
• Change Management
• Release Management
• Configuration Management
• Service Level Management
• Financial Management
• IT Continuity Management
• Availability Management
• Capacity Management

Service Support, the blue book, covers the one function and the first five processes. Service Delivery, the red book, covers the last five processes.

It is possible to adopt ITIL while still using any of the process improvement methodologies listed above. It is not a case of using one or the other.

What is the expected ROI?

Implementation of ITIL can be costly, so where can an organization expect to recover those costs? Here is a partial list the benefits:

• ITIL has become the de facto best practice for running IT. The wide spread adoption of ITIL within an industry will provide guides to what works and what doesn’t.
• ITIL brings with it a common dictionary, an item that has been lacking in the present IT world.
• Improved financial management of IT and a better matching of the services of IT to the needs of the overall organization.
• Improved relationship between IT and the organization for which it provide services.
• Improved utilization of the IT infrastructure.
• Improved utilization of IT personnel.
• Improved reputation of IT within the organization that IT services.

Who are some of the organizations that are adopting ITIL?

The adoption of ITIL is gaining momentum within the US. This is evident in the fact that as more of the IT tools have added the tag line “ITIL compliant” into their advertisements. Here is a short list of state, federal and private organizations that are implementing the ITIL framework.

• Microsoft—by providing Microsoft Operations Framework (MOF)
• HP—by providing IT Service Management Reference Model
• IBM—by providing IT Process Model
• US Army
• State of California
• State of North Carolina
• Blue Cross – Blue Shield of Florida
• Blue Cross – Blue Shield of Texas
• LG&E Energy LLC
• United Health Group in Minneapolis

Microsoft, HP and IBM each have their own version of ITIL—which is focused on their own hardware and software—but each is based on ITIL Version 2. Each of these organizations are somewhere on the path of ITIL adoption. Most people with a lot of ITIL experience state that adoption of ITIL has no end point, only a starting point and milestones along the way to measure your level of continuing success.

What are some of the difficulties of ITIL implementation?

The adoption of ITIL may not be easy or short or cheap.

One reason is that IT has not often run itself like a business. IT has a habit of operating as an indispensable group that can use resources and provide services as it sees fit. How many times have you heard the phase, “we know what our customer needs and that is what we are providing”?

IT has not been in the practice of running itself based on documented needs of the business. One place where this is clearly evident is that many organizations lack a strong change management process. In these organizations, system changes frequently have an adverse effect on the productivity of the entire organization.

The ITIL framework is about process, not organization. Most IT organizations have spent years dividing up IT resources into towers of responsibility, hardware, software, and staff. Breaking down or redefining these towers to expedite the processes within the ITIL framework and making sure that the necessary information is passed between processes may be the biggest and most difficult challenge. Because this is such a large stumbling block, it should be understood that the requirement for ITIL adoption must come from the very top of the IT organization.

The implementation of ITIL cannot be accomplished in six months. But most IT projects are expected to show major positive results or even be completed within six months. ITIL will not meet either of those milestones. ITIL milestones are measured in years rather than months.

Implementing ITIL may also require new resources. Your present technology may not be able to support the processes or inter-process communication required by ITIL. Even though the ITIL framework is about processes, that framework can only work efficiently when you have the right tools to support both the processes and the inter-process data exchange.

Another difficulty is determining where to start. Two questions often asked are:

• Do we try to do everything at once?
• It is such a major undertaking, where do we start?

The answer to the first question is “no.” Trying to do everything at once is too difficult and will only increase the chance of failure. The answer to the second question depends on each individual organization. ITIL provides flexibility in approach, not requiring a specific methodology. Common advice is to look at your present processes and see which of those processes comes the closest to matching the processes describe in either the Service Support or the Service Delivery books. Begin with those processes. Try to pick one that will show a positive result quickly, so others will stay on board.

The ITIL framework will cause a major change in the way your business is being done. Changes in the business processes are always difficult.

Even though ITIL implementation can be difficult, the true value is the long-term adoption of best practices that have shown a positive return on investment. ITIL builds a stronger organization that matches the service provided to the present and future needs of the business and provides that service in a cost-effective manner. Both of these actions will improve the standing of IT within the business and improve both the competitive advantage and bottom line of the business.

What are the steps to getting started?

For most organizations, education or training is the first step. ITIL brings with it a common dictionary of IT terminology. Many companies begin by requiring most or all of their IT staff to take an ITIL Foundations Certification class. This class can be 2 – 3 days in length. Many companies require that the class include a certification test. This training provides a common understanding of the ITIL framework and a common language for a more accurate discussion during the implementation.

A second step is to designate someone as manager of the ITIL adoption project. The adoption of ITIL rises to the level of a major project that requires formal oversight. The project manager should be an ITIL Certified Service Manager or have at least one advisor who is so certified. This is a long-term project and every effort should be made to have the same Certified Service Manager to provide continuity to the ITIL implementation.

A third step is more training. As progress is made toward the implementation of each of the 10 ITIL processes, the manager of each process may be assigned to take a special ITIL Practitioner Certificate class that covers the process for which they have responsibility.

An ongoing step is to consider is the need for new technology. However, new technology should not be considered until your processes are better defined. It is important to remember that technology is only an enabler and does not improve weak processes.

Summary

ITIL is becoming the industry standard for best practices for the management of IT. If you are an IT company or an IT division within a company then you can expect to hear, “Where are you in your ITIL implementation?” If you work in the IT industry, and you work for a medium-to-large organization or plan to in your future, then you should think about adding one or more ITIL certifications to your resume—because ITIL will be in your future.

For those of you who are about to begin the ITIL implementation journey, remember to think and talk about the long term, because this journey is measured in years not in months. ITIL is a new approach to continuous improvement. Part of ITIL is the constant improvement of each of the 10 ITIL processes and the inter-process communication. This constant improvement has as its goal to provide IT services in a more cost-effective manner and to better match those services to the present and future needs of the business.

ITIL is a framework of how to manage IT like a business for the business. As such, the ITIL framework does not describe in absolute terms how any of the ITIL processes should be implemented. Those details are left up to the implementer. The goals and key performance indicators for each of the processes is well defined, so that there is a clearly defined road map to measure your success.

Total Knowledge Management

2008-08-17T22:39:00.001-07:00

Tacit knowledge is exchanged. It’s about people sharing “know-how” in ways that help organizations foster innovation, streamline inward and outward-facing response times, get products and services to market faster, reduce costs, increase productivity, and increase revenues.

Tacit knowledge, you see, is made up of things that we know, but cannot explain. Things like driving a car or recognizing someone’s face. Tacit knowledge—truly tacit knowledge—is transferred through human-to-human contact: storytelling, apprenticeship, conversations around the water cooler. Tacit knowledge is transferred by people with ideas talking to other people with ideas, by people with experiences talking to other people with experiences. It is synthesized through social effort.

In this model, tacit knowledge isn’t captured—it’s exchanged.

Knowledge by Half

The concept of knowledge exchange isn’t new. You can go back thousands of years and find master/apprentice relationships in tight-knit guilds where arcane knowledge was passed carefully from one generation to the next. If you think of each guild as a kind of “corporation,” it is clear that the way in which knowledge was “managed” in these “corporations” was through a system of continuous tacit knowledge exchange.

Sure, there were also endless hours of study where apprentices would peer closely at hand-lettered texts, many carefully guarded so that guild secrets would never be revealed outside the elite membership. But the primary model was not oriented around study. Study only provided the basic “how-to” knowledge these young guild members needed. Without the tacit knowledge exchanged through apprenticeship, apprentices would never learn enough to become masters themselves.

True knowledge management, then, was a blend of learning that combined both the carefully wrought explicit texts and the learned tacit experience of the most senior guild members.

If the barely literate people of centuries past could figure out that effective knowledge management consisted of both tacit and explicit knowledge transfer, why have today’s companies forgotten this lesson? Why do so many KM projects today focus on the capture and dissemination of purely explicit knowledge? Simple. We can see it. It’s easier. And that’s a dangerous, one-sided view.

The One-Sided Knowledge Project

There’s been a lot written about why so many KM projects fail, and there is certainly no single reason. We know, for example, that in many cases KM is a solution in search of a problem and that companies have often failed to look at the business purpose for implementing a KM strategy. We know, too, that end user acceptance has been a factor, as companies too often ignore the cultural implications that accompany the creation of a knowledge-sharing culture. We have also seen more subtle problems including fading corporate interest, organizational structures that hamper support, and more.

One of the overlooked areas is the emphasis of many KM projects on the organization’s explicit knowledge while ignoring the vast value of the tacit knowledge that also exists. The approach is akin to paying attention only to the part of the iceberg you can see.

Effective KM can only occur when a systemic approach to all knowledge sources is adopted…an approach that includes both tacit and explicit knowledge. As Nonaka and Takeuichi point out in The Knowledge Creating Company, “the tacit—explicit dichotomy is false.” Any system that aspires to truly manage the intellectual capital within an organization needs to approach both as two sides of the same coin. It’s all knowledge after all. Why would anyone want to manage only half of it? As Alee has summarized, Nonaka and Takeuichi “see tacit and explicit knowledge working in both directions, in continual flux and movement” leading to “knowledge conversion.”

There’s an argument that managing tacit knowledge may be the more vital activity. Stephen Denning writes that, “explicit knowledge is the only knowledge that is visible and so it is tempting to focus on it. And yet we know that most of our real knowledge is tacit.” Denning would argue that the communication between individuals is a fundamental KM principle, and that “in the end [it] provides a vehicle for conveying unseen tacit knowledge.”

Reaching for Knowledge

Think about the last time you needed help doing something, or needed an answer to a question. What did you do? If you’re like most people, you probably did one of two things: checked a reference of some kind (like a web site or instruction manual, for example), or asked someone whom you believed had the knowledge you needed. Let’s look at those two processes a bit more closely.

When you check a reference of some kind, you are touching explicit knowledge—knowledge that someone has physically instantiated. We do it all the time when we check the television listings in the newspaper, when we watch the sports scores scroll across the bottom of an ESPN broadcast, when we try to put together the furniture we just bought from Ikea. The knowledge we access may be simple or complex, easy or difficult to understand, but it’s still explicit. It’s written down. (Whether it’s written down well or not is a different issue: I think we’ve all had the stereotypical experience of trying to follow instructions that were translated from another language, or worse, were purely pictorial.)

When we ask someone for help, we are doing something very different. We are asking someone to share what’s in his or her head, and perhaps to assist us in understanding meaning and application. We engage in an interchange—an interchange that is more likely to teach us something more than merely referring us to something written.

The simple truth is we reach for help both ways: diving through manuals and web sites, newspapers and books, and diving through the experience and knowledge of those around us. (Unfortunately, what often happens is that those who are asked to provide answers aren’t necessarily the right people. They are just those that are convenient.)

With respect to KM, what should be obvious is the need to manage both types of knowledge sources: tacit and explicit. We reach for both all the time. Yet too often companies ignore tacit knowledge management in favor of explicit knowledge management.

As I mentioned earlier, explicit knowledge is easier to see. It’s almost always physical and is sometimes already stored electronically. So it’s easy to understand why corporations are willing to spend money on managing explicit knowledge resources. Explicit resources have a physical presence and seem more controllable. They seem somehow more “real.” Conversely, tacit knowledge isn’t visible. It’s hard to get a handle on.

Another significant difference is that with tacit knowledge, it’s hard to know when people are going to reach for it. Again, it’s much simpler with explicit knowledge. Basic processes can be mapped, people can be observed doing their jobs, and the when-and-where of explicit knowledge access can itself be made explicit, resulting in a set of business rules that control the content, rights and availability associated with the explicit knowledge sets.

With tacit knowledge, the mapping isn’t so clean. Often people don’t even realize they’re “accessing” anything when they ask someone for help. And since tacit-knowledge needs are more amorphous than the kinds of explicit knowledge so often captured in current KM systems. It’s also harder to determine where in a work process that knowledge will be needed. So people just get up and walk down the hall when they have a moment. They schedule “face time” with people they think might help. They pick up the phone or drop an e-mail on someone. Others will desperately search for anyone who can help them with a customer’s issue NOW! Whatever mechanism is used, and whatever urgency is shown, the reach for tacit knowledge is done with the hope of getting some guidance and advice from someone who has “been there” and “done that.”

Wikström and Normann, in Knowledge and Value, refer to such knowledge as “know-how” and argue that it is, “learnt by watching other people do.” The result is often newly acquired skills, and when combined with thought and review can yield “understanding” which “arises when we recognize principles and connections.” A good deal of understanding, they say, “can never be codified.” It requires an “open problem-solving style” where experts share their knowledge with others.

With all the value inherent in tacit knowledge, it makes sense to plan a systematic and simultaneous approach to managing both. Knowledge workers need both.

Combining Tacit and Explicit Strategies: The Whole KM View

I can’t think of a single work activity that isn’t going to need ongoing access to both explicit and tacit knowledge resources. Think about sales, customer service, and marketing. Think about pharmaceutical regulations, insurance, and taxation. Think about manufacturing, distribution, travel and tourism. Slice it anyway you want, the result is the same: there are going to be policies and procedures—ways of doing things—that are easily and efficiently made explicit and that are ripe for KM principles and practice.

Now look at that list again and you’ll see something rather obvious. Every one of those functions and industries relies heavily on the creative “know how” of the individuals performing those tasks. There are always “masters” and “apprentices” who share knowledge. They share it in meetings and they share it around the coffee machine. They create new knowledge through socialization and externalization, and synthesize new ideas and new ways of doing things.

The explicit knowledge can easily be made available through the myriad “knowledge delivery systems” that are currently in the marketplace. They can integrate with other systems, attach themselves to data warehouses for analytical purposes, and contribute to the efficient ongoing operations of organizations across changes in staff, management and even strategy.

It’s the “know-how” that rarely gets approximated in any explicit way. “Human beings may be expensive and cantankerous,” Tom Davenport writes in Some Principles of Knowledge Management, “but they are quite accomplished at certain knowledge skills,” such as when there is a need to “combine [knowledge] with other types of information, or to synthesize various unstructured forms.” He continues, “Computers [are best] for the capture, transformation and distribution of highly structured knowledge.” Tacit knowledge needs something more.

Quite simply, if you’re going to manage your organization’s knowledge, you need to simultaneously look at both the explicit and the tacit domains. It’s the two together that represent your organization’s intellectual capital.

The Risk of the One-Sided Approach

Let me say it again. I can’t think of a single work activity that isn’t going to need ongoing access to both explicit and tacit knowledge resources. Yet time and again I see corporations making decisions based on a false dichotomy—tacit vs. explicit knowledge—and then going ahead with plans to implement a KM program that manages only one or the other. Taking such an approach means ignoring the systemic and changing nature of knowledge.

Explicit knowledge is fundamental to all work. There’s absolutely no question about that. But explicit tends towards the simplistic or rote, and doesn’t often require the kind of thinking and learning that increases the value of knowledge resources—including the people in the organization.

Tacit knowledge, on the other hand, is the “know how” that Wikström and Normann talk about. It’s the type of knowledge that truly adds increasing value to an organization, to the individuals who make up that organization,and to the work activities those individuals perform.

What happens if you look at only one side of the coin? While there have been no specific studies on “explicit-only” or “tacit-only” KM projects, it’s not hard to imagine some nearly-certain pitfalls.

First, it’s likely that user-acceptance of an “explicit-only” approach would be weak. Much of the “easy” knowledge that can be captured and made explicit is knowledge that many people already have in their heads, the habits developed through performing the same basic procedural functions for a long time. There would certainly be an advantage for new employees but that’s only a small part of the role KM should play in an organization. Most employees—particularly the experienced ones who have all that “know-how”—aren’t likely to be motivated to use such a system.

Second, there would likely be employee morale issues. Individuals who have “know-how” would feel pressure to somehow “capture” what they know into a document form but would run up against the constant fact that there are some things they “know but cannot describe.” Even if they do manage to approximate some of what they know in a way that can be captured, they can’t be expected to actively support such a restrictive KM approach without ongoing acknowledgement and respect for their tacit skills.

Where does that leave a company that has just spent a six- or seven-figure budget on an incomplete KM approach? Not much further ahead than before they even started. Yet companies are still spending their KM dollars on the explicit approach first when tacit knowledge management is such a critical piece of the puzzle.

Managing Tacit Knowledge

Through “Enterprise Expertise Management” One of the points raised earlier is worth repeating: explicit knowledge is often the first to be “managed” because it’s easier to do. The knowledge itself exists physically and is (somewhat) easy to locate. Technologies for delivering explicit knowledge in a myriad of ways are available from a variety of vendors using a variety of techniques. Until recently, however, there were few effective approaches to managing tacit knowledge. Pioneers in the field had done it—at places like Buckman Labs, Hewlett-Packard, SAIC and Johnson & Johnson, to name a few—but there wasn’t a clear technology and industry niche until Knowledge Exchanges came along.

The early knowledge exchanges weren’t very good. They modeled themselves on internet auction companies hoping that people would pay for answers to questions. They didn’t. People stayed away in droves, and the result was a complete rethinking of the knowledge exchange approach. What emerged was recognition that the real value in knowledge exchange technology was within corporations, creating a virtual social network where the driving forces behind “who got asked what” wasn’t convenience, but expertise. The result is Enterprise Expertise Management which is a knowledge exchange technology focused on balancing the KM equation, thereby allowing companies to intelligently pursue both explicit and tacit KM programs simultaneously (and to escalating effect).

Enterprise Expertise Management (EEM) is based on a simple premise: when people have questions that require “know-how,” the system should match a questioner with someone who can answer the question AND share that know-how, and do it in a way that maps to work activities and takes into account business rules, procedures and ad hoc events (like someone being on vacation, for example). At the same time, the communication should be stored so that any advice that is of the explicit type can be captured, and so that any tacit knowledge that is shared is rated for value.

Matching available experts and resources to needs, and tracking the resulting workflow and information exchange are fundamental business processes in all enterprises. EEM technology matches inquiries to the right expert, facilitates the right interaction process through its action-oriented workflow, captures the questions, answers and behavior, and provides measurement tools for enhancing the question & answer process. It’s a technically sophisticated approach to knowledge-enabling the tacit elements of an organization.

Conclusion

In the working world, true intellectual capital comes from a balanced combination of tacit and explicit knowledge. David Skyrme writes that one of the keys to a successful knowledge strategy is, “a well-developed knowledge infrastructure” that includes “people and information” that is “readily accessible through your computer and communications network.”
It’s not just information. Its’ People and information. Until now, too much emphasis has been placed on the latter. The emergence of Enterprise Expertise Management provides the balanced approach required for real Knowledge Management success.

Quality Governance for Software Organizations

2008-08-08T09:53:00.000-07:00

Governance in a software context

Nowhere is effective governance more critical than in the organizations that deliver software and software-intensive systems. Increasingly, companies depend on software to run their business processes; many also incorporate software into their products, whether they produce it themselves, outsource its production, or buy it from another supplier.

For a software-dependent company to succeed, it must have applications that consistently perform well in a runtime environment -- that are high-quality and thoroughly tested before deployment. This requires agile business processes for software delivery that are adaptable to many different types of projects and speed time to market, rather than slowing it down. In turn, such a process must be governed by an adaptable framework consisting of rules, best practices, and continuous management visibility into project progress and alignment with business requirements.

Good governance is different from good management, which is often tactical and reactive rather than strategic and proactive. Governance structures are repeatable touchstones that ensure that both providers and clients will set and live up to expectations. As IBM Distinguished Engineer Kurt Bittner points out, governance establishes and enforces accountability. It should measure things that matter and hold team members accountable for things that will affect over-all team results.

The IBM Rational quality governance vision

Of course, quality is one of the things that matters most for today's software and software-intensive systems. Quality has an enormous impact on team results and customer satisfaction, and the IBM Rational organization has long recognized the need to make quality an organization-wide concern. It has a long, proven record of providing automated capabilities to oversee testing, measurement, change management, error correction, and other quality-related activities.

Quality governance is a discipline within a larger governance environment for software and systems delivery. One component of quality governance is organizational and structural, providing chains of responsibility, authority, and communication that enable a workflow for applying quality measures. The second component relates to measurement: rules, policies, and control mechanisms that enable people to assess product quality and progress. This is where IBM Rational has always focused its attention.

For more than 25 years, a desire to help clients create a comprehensive quality management environment for software development has driven Rational's product strategy. One of the best practices embodied in the Rational Unified Process®, or RUP®, is iterative development, and quality management is an underlying principle of this approach. Iterative development emphasizes continuous alignment with changing business requirements, early testing, and deep client involvement in prototyping and beyond.

Through internal development, strategic acquisitions, and harvesting best practices from client engagements, Rational has continuously enhanced its clients' quality management capabilities. Via integrations with IBM WebSphere and Tivoli technology, the Rational Software Delivery Platform now provides automated capabilities for infusing quality into every phase of software and systems delivery -- from component testing by developers through build and actual production environments. These capabilities represent a flexible framework that fosters agility, not rigidity. In the words of IBM Distinguished Engineer Grady Booch, good governance "...promotes predictability and repeatability but still allows cre-
ativity to flourish."

With the Rational Software Delivery Platform, creating an effective structure for governing quality management does not require a total organizational transformation. Tight product integrations and an open source ecosystem make it relatively easy and affordable to introduce Rational products into a workplace that employs other quality management products and processes -- and protect previous IT investments. As the quality governance framework evolves, individual technologies may keep their identity, but will naturally begin assuming organizational characteristics.

Solutions to ensure software quality

Built on the IBM Rational Software Delivery Platform, our testing solutions enable tighter management, better planning, and improved data sharing for all team members. To help you make confident go-live decisions and build high-quality enterprise applications, we offer solutions for performance testing, functional and regression testing, manual testing, developer testing, and test management. The relationship of these testing roles and associated IBM Rational products is illustrated in the figure next page.

With our solutions, quality assurance teams can easily manage and address issues with application functionality, usability, reliability, scalability, and performance. For more information on these testing products, see
http://www-306.ibm.com/software/rational/offerings/testing.html

Benefits of quality governance

Introducing quality governance into a software organization -- establishing chains of responsibility as well as instruments and processes for quality measurement and control -- can produce significant business results. Let's consider the key benefits.

Faster time to market/value

Although some organizations fear that quality measures will slow their software delivery schedule, a quality governance framework that produces continuous insight and data throughout the delivery lifecycle actually helps to accelerate production readiness and sustain high performance after the software is deployed.

• Component development teams can test and eliminate defects before build. This may indicate a need for feature alterations early on, when changes have a relatively small impact and require less time and effort to execute.

• By discovering, resolving, and documenting problems early, teams can navigate more quickly through later development phases. For example, the production team can use quality data from testing performed earlier in the delivery lifecycle to quickly locate and correct root causes of problems. In addition, teams can avoid wasting deployment and post-deployment testing time on bad builds -- and apply their efforts to more productive and satisfying work.

• Development teams can use post-production data to redefine test cases, detect more pre-production defects, and guide feature modifications for the next release. In other words, the organization can control quality process es across and between software and systems delivery cycles to achieve continuous quality improvement.

Quality governance is the way to counter the prevailing attitude in many software organizations, summed up in Meskimen's law of quality: "There's never time to do it right but always time to do it over." In fact, it is better quality that lets organizations deliver on time, at lower cost, and with more features.

Greater levels of productivity and innovation across software and systems delivery teams

If governance measures are in place to ensure that your organization delivers only high-quality products, then once those systems are in production, teams can focus their attention on new projects rather than bug fixes. According to the National Institute of Standards and Technology, software organizations currently spend 80 percent of their resources on error correction and maintenance. Quality software allows teams to spend more time thinking about ways to add value and create useful details rather than making corrections.

Greater customer satisfaction

With a well-governed, automated approach to quality, software managers can constantly monitor coordination between requirements and execution. Meta Group reports that most customer dissatisfaction stems from poor understanding of requirements. Rational offers strong support for keeping software products aligned with requirements, including automated tools for defining and tracking requirements throughout the development and delivery lifecycle, as both requirements and features evolve and change. Other products in the Rational Software Delivery Platform help organizations define a requirements-based business process for application development and then gather real-time information from multiple environments to analyze process performance.

Risk reduction

Controlling the quality processes across and between software and systems delivery cycles gives companies many ways to lower risk.

• By conducting quality progress checks throughout every delivery phase, managers can ensure that whatever deliverable a team is working on will meet exit criteria and be ready for primetime production.

• With continuous, integrated automation, each activity team (e.g., test) can leverage artifacts from previous activities or phases (e.g., build). This improves efficiency, productivity, and workflow; it also creates auditable trails, provides process validation, and reduces the risks of error and failure.

• With metrics that enable them to compare apples to apples, managers can more easily direct their teams to pin point and eliminate sources of risk. For example, by comparing code churn from phase to phase, they might spot an underlying problem if the pace picks up in one phase. Likewise, by comparing performance data from release to release, they can assess whether the quality measures they applied resulted in real improvements.

• Upcoming releases of Rational products will provide common log and tracing capability to help accelerate the location and repair of problems that occur during the application development process. Using these products across the various software and systems delivery phases and activities will help teams identify root causes via the systematic consolidation, correlation, and analysis of log events, using a common "language."

Market advantage

The software marketplace is plagued by low-quality standards and project failures. Companies can realize substantial competitive advantage by consistently producing high-quality software in a timely way. According to the Cutter Consortium, nearly a third of organizations say they release software with too many defects; an even greater proportion say they lack an adequate software quality assurance program. Against such a landscape, companies with effective quality measures can establish a reputation for having quality internal systems and producing high-quality, reliable software. Based on this, they can build and strengthen brand loyalty -- even in markets that are increasingly commoditized and brand agnostic.

Cost efficiency

Employing a governance structure that cultivates an internal culture of quality can help software organizations realize substantially better returns on their investments. Instead of using up 80 percent of development time on identifying and correcting defects (as noted above), organizations can use that time to come up with innovative ideas and products. As industry expert Philip Crosby has noted, all of this is "free" once you do the organizational transformation necessary to put a governance structure in place, along with a well-defined process that gets unanimous buy-in from team members. When you can integrate and automate processes that establish and encourage quality from beginning to end of the delivery cycle (i.e., from requirements gathering through delivery), you already have a foundation in place to support your project schedule and cost restraints. Ultimately, you have an organization in which developers can focus on value-added features and quality across the lifecycle -- and managers can constantly evaluate end-user feedback to invigorate the product line and satisfy repeat customers.

Better quality of life for managers

Among all the measures companies take to retain good employees, none is more effective than allowing them to do their job without experiencing high stress and having to agonize over details. Managers who work within a software organization with quality measures in place do not have to constantly feel that their job is on the line -- as they weigh the consequences of shipping products with known defects. In addition, the consistency and predictability that a quality framework provides allows them to continuously move their projects forward instead of stopping to re-scope and make corrections. They are able to meet deadlines and use project milestones for their intended purpose: to assess the project's value and progress, ensure it is aligned with business strategy, and decide whether it is worth the investment in time and money required to move forward toward completion.

Accommodating technological change

Rational has long recognized that governing software quality is not just a managerial concern; it has a strong impact on the daily lives of everyone within a software delivery organization, as well as on business results. Today and in the future, Rational's mission is to help clients support a quality governance framework that is broad and flexible enough to accommodate the organizational transformation that inevitably comes with technological advances. Along with this mission comes the recognition that such transformations are typically evolutionary; Rational's Software Delivery Platform enables companies to achieve significant levels of success by implementing products and processes in a selective, iterative fashion.

Most recently, the Rational organization has developed and assembled the tools and service capability required to govern quality in organizations that have adopted service-oriented architecure (SOA). These architectures necessitate changes in command chains as well as software development practices that many organizations are still trying to sort out and evaluate. The IBM Rational integrated platform coupled with deep process expertise can help companies build an automated SOA quality governance structure that clarifies responsibilities, ensures application integrity as well as alignment with business strategy, and protects investments -- both now and in the long term. These capabilities will serve as a model for future quality offerings that enable software delivery organizations to take full advantage of powerful new technologies.

For more information on the latest release of IBM Rational tools that can help software delivery teams establish a quality governance initiative, see the Rational Software Delivery Platform V7 announcement page at

http://www-306.ibm.com/software/rational/announce/dt/

And to learn more about the IBM Rational strategy for quality management, see the overview at

http://www-306.ibm.com/software/rational/offerings/testing.html

Start Focusing on Architecture--Your Career and Future Depend On It

2008-08-07T20:13:00.000-07:00

Increasingly, companies report that the globalization of distributed software development is growing more complex, making software projects harder to manage. Even when software development is still performed in house, most development operations are typically globally distributed. Concurrently, business processes and technology are growing more and more intertwined, further increasing complexity in both distributed and offshore projects.

To control costs and decrease development time as they try to cope with all these changes, companies are driving their domestic and outsourced developers to produce high quality sophisticated software applications faster than ever using development environments that have become exceedingly complex. Consequently, it has become virtually impossible for developers to keep pace with today's development challenges and escalating sophistication without resorting to a higher level of abstraction.

At its most basic level, abstraction is achieved using a higher level programming language. However , the specificity and detailed nature of code (Java, C/C++, etc.) doesn't convey the bigger picture of large, sophisticated development projects. When this type of software project is spread out across different time zones, languages, and cultures, it becomes so difficult to manage that the only practical solution is to logically break up the development effort and assign different parts of the system to specific development teams in various geographical regions. To do this successfully requires an even more abstract view of the software design, dependencies, and all the things that comprise the project--the software architecture view.

Architectural Skills Increases Your Value to Your Company

Why should you care about architecture? The answer is simple--your career and your future depend on it. Architecture is growing in importance affecting what you do, how you work, and where you work, relentlessly influencing your career and opportunities for future employment. But there are many reasons to focus more attention on architecture apart from saving your career .

Outsourcing and offshoring deliver good programming skills at a lower cost using cheaper labor. Developers certainly recognize the serious impact this has had on the availability of mainstream, domestic, programming positions. Although most companies have not completely eliminated all programming jobs, there will continue to be fewer jobs available. One of the ways senior managers decide what developers they are going to keep and which ones they are going to lay off is determined by the additional skills a developer may possess.

Do you feel threatened by job shifting? You certainly can increase your value to your employer if you can relate software design and development to the intrinsic needs of your business and customers--something most outsourced personnel can't do. One way to do that is to become more focused on architecture than on implementation.

For starters, any attempt to offshore and outsource any project without considering the architecture is a potential recipe for disaster due to the cultural and language barriers which lead to communications problems and misinterpretation of the specifications. In addition, the logistics of managing a project across different time zones and countries can be quite difficult, particularly when your outsourcing company is subcontracting part of the work to another company in another country. Working at the architectural level provides a language of discourse that is closer to that used by business analysts and others. In other words, it is easier for a business analyst to see how their requirements map to architecture than to source-level code.

The best part is that there are a variety of tools and technologies available to help you focus more on architecture and requirements and less on implementation. The tools are structured with functionality to make this transition and learning curve as painless as possible while keeping you within your comfort level. Some tools contain the languages of architecture such as business process modeling language or unified modeling language (UML). Even rudimentary tools such as Visio or PowerPoint let you understand your project from a more abstract, architectural context.

Today's powerful, highly automated design and construction tools go far beyond basic IDE to support development using high level architectural languages instead of programming languages. These are especially well suited for those companies who can't outsource for one reason or another, but can not keep staffing using expensive domestic programmers. Productivity gains are achieved because a fewer number of people can work more efficiently than a team of software engineers by using these multi-function and highly automated tools for development.

How to Start Focusing on Architecture

The first thing you can do is learn more about architecture and prove to your management that you are ready, willing, and able to leverage new technology skills that enable you to be more productive and valuable to your organization. A good place to start is to review these materials:

• Evolution of Developer Skills Is Essential for Job Survival
• eBook: The IDE as a Development Platform
• Find the Right IBM Rational Analysis Design and Construction Tools for Your Project
• IBM Rational UML Resource Center

Tools to Assist You Each Step of the Way

You probably are already familiar with IDEs containing some level of design capability and/or visual modeling. You certainly have used wizards to automate some of your development tasks. You might even be familiar with IBM Rational Application Developer (RAD) which has built-in code-level architecture features. RAD is built on Eclipse and extends the Eclipse framework. It is positioned primarily for both visual and code-oriented developers but won't alienate you with lots of UML model driven capability. However, if you want to think like an architect and improve your skills by working in a more abstract (and architectural) view of your code, you should explore RAD's UML visualization capability.

RAD allows existing code to be rendered in UML for developers who just don't know the UML language. Many developers find it easy to transition from code to code level modeling using RAD. RAD utilizes class and sequence diagrams and also includes code analysis functions and automated test and deployment tools.

RAD lets you traverse source files using the pictorial view and if you make changes to the code, the diagrams update automatically. You can also edit the diagram to update the code because RAD maintains an isomorphic or 1:1 relationship between the UML and code. RAD is a good first step toward thinking and working at an architectural level. However, if you are using code rendered into UML you are just looking at a pictorial view of the code which
means you are still editing at the code level

Taking the Next Steps on the Road to Architecture

Architects have to think more abstractly--at levels that don't immediately have any relationship to code. Architects work between the business requirements that drive the business and the implementation levels that are a reflection of the programming decisions that are executed into code. Achieving these skills requires you to learn how to work with basic UML modeling.

Since the industry is also becoming more service oriented in its design and implementation of architecture and software, you should learn as much as you can about service oriented architectures (SOA) because SOA combines reuse thinking with business-level abstraction to support the creation of reusable components. This enables software to be built using components in a manner similar to the way engineers build hardware.

There are design and construction tools that support your efforts to work at a more abstract level and also facilitate the creation of SOA solutions. One example is IBM Rational Software Architect (RSA) which leverages model-driven development with UML, letting you create well-architected applications and services.

While RAD eases the transition between the architecture, it only serves the code-level of modeling and the model-to-code and code-to-model transformations. RSA, on the other hand, includes all the features of RAD to give you an integrated design and development experience and adds model-to-model transforms, as well as a powerful patterns development capability and an architectural analysis capability.

Architects and SOA developers use patterns to accomplish many of their development tasks, since the essence of SOA is reuse and the mechanism of reuse is design patterns. Patterns let you capture domain knowledge, best practices, and design expertise to improve productivity, efficiency, and quality while automating and minimizing the mundane and mechanical aspects of software development. Patterns can also represent known problems and situations in business domains.

Pattern implementations are one of the most powerful software engineering tools you can use to improve development and productivity in your organization. Their use eliminates tedious, error-prone tasks, minimizes complexity and confusion, improves governance, and increases the overall quality of any type of software project. They allow you to respond more quickly and cost-effectively to changing business conditions.

As any software architecture evolves, an architect takes the revised requirements and creates an analysis level model that reflects the requirements but doesn't yet reflect the implementation. Using RSA, an architect first creates an analysis-level model. A subject matter expert like an analyst validates the representations captured in analysis-level model diagrams to ensure that the architecture meets their requirements.

There are patterns that specify how you transform an analysis-level model into a design-level model (known as a model-to-model transform). Once you provide sufficient detail in your design model, you can apply model-to-code transforms automating much of the implementation work.

RSA lets you create and use pattern implementations and model transforms. RSA enables you to leverage abstraction using multiple models to capture the relevant aspects of a solution. Transformations allow you to automate how you move between the levels of abstraction. RSA also enables you to specify high-level aspects of a system created in a model and generate a larger percentage of the solution automatically.

If you need to model, but don't want all the specificity required for code generation, consider using IBM Rational Software Modeler (RSM). RSM is a modeling-only subset of RSA and extends Eclipse to enable architects, systems analysts, designers and others to specify and communicate development project information. It supports the same level of modeling with UML version 2.1 as RSA, supports patterns and model-to-model transforms, and has the same flexible model management for parallel development and architectural refactoring. RSM is useful for situations where automated modeling is needed but the model-to-code transformations are done more manually.

Hopefully this article has been able to pave a professional roadmap you can follow. It will take you from the abyss of job loss due to outsourcing to the wealth of opportunities that await you by focusing on architecture. As you follow this roadmap keep in mind that there are developers who are working at levels of abstraction even higher than those of a software architect. They are in the business community working with databases, operation deployment, hardware and process servers, focusing their efforts on enterprise architecture, which is a subject for another paper entirely.

Model-driven and Pattern-based Development Using Rational Software Architect, Part 1

2008-08-07T14:31:00.000-07:00

Overview of the Model-driven Development Paradigm with Patterns

Introduction

Model-driven development (MDD) is a new software development paradigm supported and driven by the Modeldriven Architecture (MDA) methodology, a software design approach released by the Object Management Group (OMG). MDA provides a set of guidelines for structuring specifications expressed as models, starting from a platform-independent model (PIM), continuing through an appropriate domain-specific language, and then tranforming into one or more platform-specific models (PSMs) for the actual implementation platform. This could be any number of platforms, such as Java™ 2 Platform, Enterprise Edition (J2EE™), CORBA, or .Net, implemented in a general programming language such as Java™, C#, and Python. MDA is normally performed using automated tools, like IBM® Rational® Software Architect. Driven by MDA, MDD focuses more on model transformation and code generation.

However, the code generation-based approach used by MDD has its own downside, due to things like constraints in generated code, insufficiently skilled developers, and tight coupling to the model. When companies commit themselves 100% to code generation, there is little room for tweaking, especially when developers always have to go through their model.

A pattern-based development approach helps solve this problem. A pattern is a solution to a recurring problem within a given context. Patterns encapsulate a designer's time, skill, and knowledge to solve a software problem. In addition, when it is used repeatedly in a number of different projects, a pattern becomes established as a best practice. By designing around a particular design pattern, developers have greater flexibility to control the generated code, which is not constrained simply based on an abstract model.

Furthermore, MDD can automate implementation patterns with transforms, which eliminate repetitive low-level development work and encode technical expertise to improve consistency and maintainability. A modified transformation is rapidly reapplied in order to generate solution artifacts that reflect a change to the implementation architecture.

This article focuses on how to optimize MDD with asset-based development as an integrated development approach. Using this approach, developers first build the object model using the Unified Modeling Language (UML), and then generate the code from that UML model using a code generation tool that utilizes a pattern repository.

UML is an open standard and the de-facto standard for software modeling. UML is a language for specifying, visualizing, and documenting software systems. UML provides a visual notation and underlying semantics for software models. UML also has a standard machine-readable serialization format, thereby enabling automation.

Applying MDD with Rational Software Architect

Reading this information, you have probably realized that applying this development approach requires an Integrated Development Environment (IDE) that can support the following:

• Modeling using UML
• Patterns infrastructure
• Model transformation and code generation
• Platform-specific design and development tool and unit test environment

Rational Software Architect is the tool that provides all those capabilities. Rational Software Architect is an integrated design and development tool that leverages model-driven development with UML to create well-architected applications and services. With Rational Software Architect, you can:

• Leverage an open and extensible modeling platform
• Accelerate software modeling and design
• Automate development processes and maximize asset reuse
• Develop application and Web services more productively

The series of these articles is organized as follows:

Part 1: This article, which focuses on an overview of MDD and Pattern-based development approach

Part 2 (to come): The MDD and Pattern-based development approach with Rational Software Architect

Part 3 (to come): A case study

UML model

One of the main characteristics of MDD is using models as the key artifacts. A model is a description of a system from a particular perspective, omitting irrelevant detail so that the characteristics of interest are seen more clearly. In MDD, a model must be machine-readable, so that you can access the content of the model in an automated manner. Models must be machine-readable in order for you to be able to generate artifacts. A diagram on a white board may meet the other criteria for being a model, but until you can capture it in a machine-readable manner, you cannot use it within an MDD tool chain.

Software models are typically expressed in UML. UML models hide technical implementation details, so that a system can be designed using concepts from the application domain. Application design is typically carried out using a UML modeling tool such as Rational Software Architect, and concepts relevant to the application domain.

The use of UML models to design software was a well-established practice even before MDD. In most cases, models are used in two ways:

• As sketches that informally convey some aspect of a system

• As blueprints that describe a detailed design that you manually implement.

In MDD, models are used not just as sketches or blueprints, but as primary artifacts from which efficient implementations are generated by applying transformations. In MDD, precisely described domain-oriented application models are the primary focus when developing new software components. Code and other target domain artifacts are generated using transformations designed with input from both modeling experts and target domain experts.

Transformation

UML modeling is a valuable technique by itself, but manually synchronizing models and code is error-prone and time consuming. Transformation is the main characteristic that distinguishes MDD from other approaches that use modeling. MDD is concerned with automating the development process so that any artifact, which can be derived from information in a model, is generated. Besides code, the generated artifacts from transformation include but are not limited to the following:

Documentation: In organizations that follow a formal development approach, producing documentation takes a significant amount of development effort. Keeping documentation in line with the implementation is notoriously difficult. When using MDD, documents are generated from models. This both ensures consistency and makes information available within the models that developers are working with on a daily basis, rather than in documents that are difficult to navigate. Documentation can either be generated by tools such as Rational Software Architect's Report Generator, or by a transformation.

Test artifacts: It is possible to generate basic test harnesses (for example, using JUnit) from the information contained in software models.

Build and deployment scripts: Using their expertise, build and deployment architects can create transformations to generate build and deployment scripts.

Other models: A system involves many interdependent models at different levels of abstraction (analysis, design, implementation), representing different parts of the system (UI, database, business logic, system administration), different concerns (security, performance, and resilience), or different tasks (testing, deployment modeling). In many cases, it is possible to partially generate one model from another (for example, moving from an analysis model to a design model, or from an application model to a test model).

Patterns

Another main characteristic of MDD is capturing expertise. Patterns capture best practice solutions to recurring problems. In MDD, patterns can occur at all levels of abstraction in modeling. For example:

• Architecture patterns
• Design patterns
• Implementation patterns

Patterns can be composed to produce pattern recipes for solving larger problems and cover best practices for a domain area.

Patterns specify both characteristics of model elements and the relationships between those elements. You can automate patterns so that new elements are created, and existing elements are modified in transformation to conform to the pattern when the pattern is applied to a model.

Patterns improve the consistency and quality of solutions. They do this by automating implementation patterns with transforms in MDD, which eliminates repetitive low-level development work. Rather than repeatedly and manually applying technical expertise when building solution artifacts, the expertise is encoded directly in transformations. This has the double advantages of consistency and maintainability. A modified transformation is rapidly reapplied to generate solution artifacts that reflect a change to the implementation architecture.

From business problems to software solutions

Figure 1 shows how a business problem is translated into a software solution with MDD. The business problem is reviewed, and some common business patterns are applied. This partially populates a design model and fills in details of the specific business function under construction. Then, platform independent design patterns are applied to transform the design model into runtime-independent components. Following that, a runtime platform is selected, and runtime-specific implementation patterns are used to generate the artifacts.

Figure 1. Using MDD to translate a business problem into a software solution

Benefits of using MDD with Patterns

The software development industry has realized quite a lot of benefits by adopting MDD with Patterns. Some significant ones include:

Increased productivity: MDD reduces the cost of software development by generating code and artifacts from models, which increases developer productivity.

Increased reuse: MDD is especially powerful when applied at a program or organization level. This is because the return on investment from developing the transformations increases each time they are reused. The use of tried and tested transformations also increases the predictability of developing new functions, and reduces the project risk (since the architectural and technical issues have already been resolved).

Improved Maintainability: Technological evolution leads to solution components becoming legacies of previous platform technologies. MDD helps to solve this problem by leading to a maintainable architecture, one in which changes are made rapidly and consistently, enabling you to migrate components more efficiently onto new technologies.

More Flexibility: High-level models are kept free of irrelevant implementation detail. Keeping the models free of implementation detail makes it easier to handle changes in the underlying platform technology and its technical architecture. Changing the technical architecture of the implementation is accomplished by updating a transformation. The transformation is reapplied to the original models to produce implementation artifacts following the new approach.

Consistency: Manually applying coding practices and architectural decisions is an error-prone activity. MDD ensures that artifacts are generated consistently

Improved communication: Models omit implementation detail that is not relevant to understanding the logical behavior of a system. Models are therefore much closer to the problem domain, reducing the semantic gap between the concepts that are understood by stakeholders and the language in which the solution is expressed. Models also facilitate understanding and reasoning about systems at the design level. This leads to improved communication about a system.

Retain expertise: Projects or organizations often depend on key experts who repeatedly make best practice decisions. With their expertise captured in patterns and transformations, they do not need to be present for other members of a project to apply (and benefit from) it.

Lower risk: When using an MDD approach, early application development is focused on modeling activities. This means that it is possible to delay the choice of a specific technology platform or product version until a later point (when further information is available, thus reducing the risk).

Lessons learned

MDD is a sound development methodology. However, industry feedback indicates that not all the projects using MDD were successful. This was generally due to several factors:

• Poor project selection
• Insufficient planning
• Lack of necessary experience and skills

To help you gain the most benefits from MDD, the following are some lessons we have learned from our past projects.

Select the right candidate projects: MDD is especially powerful when applied at a program or organization level, because the return on investment (ROI) from developing or buying the transformations increases each time they are reused. A good candidate project should also have well-defined architecture and repeatable patterns of behavior, so that those patterns can be abstracted to capture the expertise and be used in the transforms.

Consider the additional cost: You may need to consider the cost of developing or buying transformations, but careful cost planning will ensure that there is an overall cost reduction in the long run.

Make small wins: In many organizations, a new approach is greeted with some skepticism until it is proven to work. If this is your first project that uses the MDD approach, it is advisable to split the development of the MDD tooling into a number of iterations. Subsequent iterations build on the experience you gain, allowing you to support a broader and broader set of scenarios for the business application.

Develop the right skills: MDD requires architects and developers to be familiar with UML and the IDE for MDD. Building these skills within the team may take some time at the beginning, which leads to project delay. However, with the same project team, subsequent projects will be much smoother. If this is your first project that uses MDD, bringing in experts to jump-start the team is a good choice.

Always think about reuse: The MDD tooling built for your project will have been used many times by the application developers while they are generating artifacts for the business application. It is also possible that this tooling could be reused on subsequent projects. Always thinking about reuse in a bigger context will benefit your organization significantly in a long run.

Select the right development tool: Having the right weapon with you will ensure that you are getting the maximum value from MDD. The project team (including the project manager, business analysts, solution architect, developers, and testers) should evaluate and validate the development tool being used.

Summary

Part 1 of this series lays out the picture of the model-driven and pattern-based development paradigm. In summary, MDD unlocks the potential of patterns to create well designed solutions, and patterns provide the content for MDD.

Resources

Learn

• Model-driven and pattern-based development using Rational Software Architect, Part 2- Model-driven development tooling support in IBM Rational Software Architect: Part 2 of this series..

• Learn about reusable assets, recipes and patterns in this developerWorks article.

• Learn about SOA at the New to SOA and Web services developerWorks page.

• Stay current with developerWorks technical events and Webcasts.

• IBM Rational Software Architect product page: Find technical documentation, how-to articles, education, downloads, and product information about Rational Software Architect.

• Visit IBM's Pattern Solutions and find out what IBM is doing around patterns and reusable assets.

• In the Architecture area on developerWorks, get the resources you need to advance your skills in the architecture arena.

• Browse the technology bookstore for books on these and other technical topics.

Get products and technologies

• Download a free trial version of Rational Software Architect v7.

Discuss

• Rational Software Architect, Data Architect, Software Modeler, Application Developer and Web Developer forum: Ask questions about Rational Software Architect.

• Check out developerWorks blogs and get involved in the developerWorks community.

Flexible, Architecture Management: Good for Business, Easier than Ever

2008-08-07T14:29:00.000-07:00

Reusability has been the holy grail of software development for decades. The growing popularity of Service Oriented Architecture has not only created a renewed interest in software reuse, but in many companies a mandate to reuse code, models, tests, and a host of other software assets and artifacts across the organization and throughout the development lifecycle.

Thanks to today's tools, it is possible to efficiently create, locate, assess, and draw on reusable assets thanks in part to the creation of variable asset repositories that can address the needs of different developer roles and communities and a federation of information among these repositories that makes content available in the right form to the appropriate community.

Employing reusable assets as part of development activities is beneficial to both businesses and developers in a number of ways. First, it facilitates a reduction of redundancy particularly in the area of having to maintain and manage multiple, similar software assets. This works to eliminate the task of maintaining (or even creating) multiple baselines of code, or other artifacts, such as tests or designs to name but a few.

Second, it improves the overall quality and reduces time spent maintaining the software by virtue of the fact that the development team is using the same piece of code or other artifact multiple times, thereby continuously reworking and enhancing it. With this investment in the code, its quality and value increases which means developers are spending less time creating new (and often redundant) code or maintaining the code

Third, it improves productivity by saving developers the effort involved in recoding redundant routines or rebuilding the same artifacts. If they can quickly find assets they can use and easily determine whether or not what they have found meets their requirements, then developers don't have to spend time creating the same things over and over again. This frees them to work on more important and often more interesting aspects of the project.

Fourth, business managers have fiduciary responsibility as well as a vested interest in controlling the reusable assets in an organization and mapping existing assets to the needs of the business. The ability to understand what assets exist, effectively manage their creation and use, and to establish consistent terminology and verbiage when describing them controls costly mistakes, ensures architectural integrity, and enhances governance across the organization

Now it is easier than ever to handle all these tasks. IBM Rational Asset Manager (RAM) lets you create, modify, govern, find, and reuse any type of development assets, including SOA and systems development assets. It delivers these capabilities through a development-time asset repository that manages assets relevant to development roles, such as technical managers, analysts, architects, developers, and testers. Through classification schemas and asset types it also standardizes and enforces the definition and consistency of terms for any size project.

RAM governs assets as they are submitted. It lets you browse and categorize, assets, provides access control, manages and versions assets, and measures metrics such as activity level in terms of their usage. More importantly, it comes as an easy to install plug-in for your Eclipse development environment and leverages and builds upon the skills you already have.

The RAM plug-in inserts new tasks into Eclipse and lets you create, use, or reuse code. You don't have to discard any tools your organization is already using. Assets can be versioned through integration with tools you already use- -Rational ClearCase (CC), and CVS. The Rational ClearQuest integration provides asset defect tracking and asset review customization. The WebSphere Service Registry and Repository integration provides traceability of services from development into runtime.

RAM wraps a collection of files (either from the file system, CVS, or CC) with metadata, identifies it as an asset, and then manages it. When an asset needs to be changed or updated, RAM notifies the appropriate person through these tools and provides the information required to retrieve the asset, fix or update it and check it back into the repository using the tools with which you are already familiar. RAM is targeted to specific kinds of users, namely in the development space. Before these users can start using

RAM, a sequence of events must be completed by an administrator to set up the repository. In brief, an administrator configures one or more development communities in RAM. Each RAM development community is targeted for a specific audience with users, roles, and their permissions. Other items to be configured include asset types and classification schemas that are relevant to the target users.

A RAM development community can also be configured with a connection to one or more WebSphere Service Registry and Repository service registries. WebSphere Service Registry and Repository is targeted towards users who want to use deployed service interfaces. The RAM software synchronizes with the WebSphere Service Registry and Repository software to coordinate the development of service assets (code, models, test, etc.) and related metadata with the interfaces of deployed services.

For example, the RAM administrator may configure one connection to a WebSphere Service Registry and Repository test registry, and another connection to a WebSphere Service Registry and Repository SOA registry. Each connection gets information such as the address to the host, which port to use, the WebSphere Service Registry and Repository login credentials, as well as the name of the owner of the assets. RAM software will allow service asset artifacts to be published to the WebSphere Service Registry and Repository system when the appropriate or qualifying conditions are met.

With the repository configuration prepared, the administrator and developers populate the repository with reusable software assets. The repository can then be used to show you what it contains and to reuse the assets. While RAM is the primary source of information about many kinds of development assets, you can track various references to WebSphere Service Registry and Repository service interfaces for more detailed information about services and other related runtime information.

Simple to Install and Use

RAM is installed into Eclipse by pointing to the location of the Eclipse RAM Update Site and clicking on it to select the plug in and start the download to Eclipse. After the download is complete and you restart Eclipse, RAM is ready to be used. At that point Eclipse will identify the different perspectives you prefer to use, select the appropriate views, and add them to where you normally work. You never have to leave your favorite perspective to access the assets in RAM.

Suppose, for instance that you are working in a Java perspective and have one view called 'search results'. Let's further suppose that you need to access a logging component. You would do a keyword search on the asset metadata or inside the files of the asset by entering the word, ”logging”. This will return a result list back to you containing all the assets containing that keyword.

Double click on any asset you are interested in learning more about to find the one that is best suited to your needs. This brings in all the metadata about the asset so you can view its relevant content. You can find out the asset's contents, read the discussion thread about the asset, see how other developers ranked it, and view the statistics showing how often it was downloaded. This level of detailed information helps you decide whether or not you want to download the asset.

RAM knows where the asset is stored so you only need to tell RAM the target where the asset needs to be downloaded. RAM relies on your source control system (Rational ClearCase, CVS, etc.) to manage the versions of the source files of the assets. It checks out and downloads into your workspace all the appropriate files that comprise the “logging” asset you selected.

RAM recognizes when any change was made or new asset was created so when you submit that asset back to the repository, it will alert you that a new version has to be created. Once you approve the version update, RAM will submit the file(s) into the repository where the asset is now subject to all the governance activities that are required before the asset can be approved for use and viewed by other developers

The governance activities will depend on the review process established by the RAM community. Some communities may have many levels of reviewers from business analysts and project managers to the senior programmers on a particular project. Others may just need to be approved by the project manager. Since this information was originally configured into RAM, emails are automatically sent to the appropriate people who need to be involved in the review process.

All the activities that have to be performed on the asset can be viewed in a web-based user interface. When the asset is finally accepted (or rejected) the developer who worked on it is notified. If the asset is approved it is placed into the repository where it is available to the people who are allowed to access and use it.

When a new asset is created for the repository, RAM guides you through the step by step process of defining all the attributes and variables established and required by the community.

RAM provides numerous other benefits to businesses and developers including the ability to reduce development cycles and reduce costs. Secure asset tracking, asset management, enforcement of architectural standards and models, as well as storing and providing quick access to information ensures architectural integrity and better communication for globally distributed and outsourced development projects. Governance challenges are improved with RAM's sophisticated discovery and federated search capability across the development lifecycle and secure intellectual property sharing.

It's time to look more seriously at governed asset reuse now that IBM Rational has delivered the combination of a unifying platform and collaborative tools to simplify the process of creating, discovering, understanding, and using reusable assets. After all the decades of promises, the reality of software reuse is finally possible.

Network Management Evolves Along with Networks

2008-07-31T06:25:00.000-07:00

If one thing is clear fromour discussion of networks in this site, it's that we ask a lot of our networks and we'll be asking evenmore fromthemin the future. The proliferation of IP-based communications andWeb applications began in recent years, and it won't be easing anytime soon.

The end result is that many enterprises have vast, complex networks that have a life of their own. It's a huge challenge to ensure that every cable, switch, router , and configuration is performing correctly, and the constant changes we make to our networks to accommodate new applications and business objectives make managing the network even more difficult.

In a white paper on Network Lifecycle Management written for HP , Enterprise Management Associates found one IT environment where eight out of 10 changes to the network required other changes to be made, and 10 percent of the time the result was catastrophic failure.

The technical operation of the network is only part of the issue, however . As IT becomes more involved in supporting business objectives, it's not enough to make sure the network hardware is operating smoothly. Compliance and security concerns mean the network operations center (NOC) is called upon to help the business minimize liability and reduce risk.

Networks Have a Role in Corporate Compliance

The last decade has seen an increase in corporate compliance initiatives by governments, industries, and international organizations, and network management is an important part of maintaining compliance and avoiding penalties.

The most famous compliance law in the United States is Sarbanes-Oxley, passed in the wake of the Enron accounting scandal. SOX, as it is known, took effect in 2005. It's a broad piece of legislation that covers almost every aspect of financial accounting in publicly traded companies, but for network operators the most important parts of the law concern the security of financial data. Strong network security measures are required under SOX to ensure financial data cannot be tampered with and financial results altered.

Access to sensitive information is also at the heart of compliance with the Health Insurance Portability and Accountability Act (HIPAA). Any organization involved in the electronic storage and transport of medical records and healthcare information must have security measures in place to protect such data.

The Patriot Act, passed soon after the Sept. 11, 2001 terrorist attacks in the United States, also contains requirements that affect network operators, mostly concerning financial transactions and the ability to monitor payments that may be financing illegal activity. The Gramm-Leach-Bliley Act, passed in 1999, has provisions to protect personal financial information. Outside the United States, the European Union passed a data protection directive in 1995.

What all of these regulations have in common (as well as many more regulations not mentioned here) is the need to maintain the security of data, limit and monitor access to the data, make the data available to those authorized to receive it, and monitor events that happen on the network.

For many companies, the path to complying with these regulations includes improving efficiencies in the network operations center . Network management software that automates manual tasks can help the NOC with compliance because it provides a quick, efficient way to view everything that happens across the network and proactively alerts the operators when something goes wrong. By automating network monitoring and reporting, the NOC can also more quickly respond to inquiries concerning compliance because the historical data is often right at the operators' fingertips.

Automating network tasks like configuration also helps eliminate human error , which helps immensely when trying to comply with sophisticated regulations. By creating a process once and automatically repeating it, it's far less likely something will go wrong that will affect compliance. Automation also helps by reducing the possibility of an inside threat because humans are less involved in the day-to-day management of the network and the data that lives on it.

Companies with less exposure to compliance initiatives are also finding ways to use network management software to increase efficiency and get control over growing networks. Revenue-generating applications, such as e-commerce sites and online self-service, require the NOC to maximize availability and keep network performance at acceptable levels. Especially important as the economy slows is getting the most from network assets and people. Network management plays an important part in each of these challenges.

It's rare for a modern network to be built on hardware and software from a single vendor thanks to years of mergers and acquisitions, siloed IT operations, and a proliferation of branch offices, home offices, and a dispersed workforce.

The network management software used to manage today's networks needs to scale to cover the size of the network, and it must also support equipment and products from multiple vendors. Custom approaches, such as scripts implemented by the IT staff, have little potential for automation in a large, complex enterprise network.

So far in this post we've discussed how networks are growing and carrying even more information because of the growth in IP communications, and how the market for network management software is changing to help network operators understand what is happening across their networks and quickly address problems. We've looked at how corporate compliance initiatives are driving the need for network management and automation, and how the need to make network resources available to customers, employees, and partners is crucial to competing in today's economy.

Network Discovery: New Category of Essential Products

2008-07-31T06:24:00.000-07:00

Enterprise networks have gotten so complex that it is rare that any single person knows exactly what is connected to them. That could become an issue, particularly if someone brings an infected PC or if disaster strikes and a portion of the network goes south.

The solution is a variety of network discovery tools and techniques, some simple and cheap, others less so, to keep up with your knowledge of what's on the network.

A combination of security threats, legal compliance issues, and general troubleshooting complexity have motivated a growing number of security consulting firms to look more closely at network discovery as a bona fide practice area. But before you rush out and hire someone, take stock of the skill set you have in your existing IT organization, figure out a budget for the activity, and realize that network discovery has multiple dimensions (this is security, after all) and not just a onestop shopping experience.

Larry Dietz, research director for The Sageza Group, in Union City, Calif., thinks there are several things to consider .

"First, there is a basic hardware and software inventory of what the client thinks he has out there. If you discover things that the client doesn't know about, then the client will think you are a genius. Second, you need to find unauthorized hardware, such as servers, wireless access points, and endpoints that users have brought into the building and running on the network. Again, whatever you can dig up is gravy."

The Basics, And Beyond

The key takeaway here is that you need to get started, and there are a wide variety of asset-tracking tools available. Microsoft's System Center , Landesk Asset Manager , and the products Symantec acquired from Altiris are all enterprise-wide tools that can capture a wide variety of hardware and software types and be useful for IT managers who want to ensure that they have sufficient software licenses for the number of users, or that their corporate-owned PCs are accountable.

But these tools just evaluate the basic elements, and don't really provide information on things like what is happening on the network, who is bringing in personal laptops from home, and staffers who are connecting to rogue wireless access points either by design or mistake. For these situations, you need one or more network analysis tools to be able to see your traffic patterns.

WildPackets.com's OmniPeek and NetScout's Sniffer and Visualizer product lines and are great tools for doing this, but require a significant investment in training to operate them properly.

"Ideally, you would like to gather this data once and reuse it for a variety of IT purposes," says Dennis Drogseth, an analyst with Enterprise Management Associates.

Such purposes go beyond mere discovery and could include optimizing applications performance, network troubleshooting, and handling compliance issues. Part of any solid understanding of what is happening on your network is knowing when something has changed, and being able to react to these changes when error messages pop up or users start calling with connection problems.

A good place to learn more about this is a site called NetPerformance.com. The site also has materials on using the analysis tools and offers training classes as well in their use.

Another great source of tools for network analysis is SolarWinds. The site has a product called Engineers Toolset that sells at the low end of the price range for network analysis tools.

The final dimension is to examine your Web presence, including looking for unauthorized but viable Web sites that IT doesn't know about, or potentially harmful, hostile or adversarial sites such as those that may be run by ex-employees or those of competitors that provide links to questionable external sites, or blogs that mention privileged corporate information.

"This could lead to a whole series of services, such as vulnerability assessments, patch management, and data forensics," says Dietz.

What tools are available? A good place to start is to look for 30-day free licenses to try out scanning tools, along with more extensive training classes for using the paid versions.

Another place is the self-training materials that can be found at the Open Web Application Security Project. It has samples for how to discover and harden Web servers, and very detailed examples of typical Web exploits too. It is a great place to learn more about overall Web security, as well as what you need to do to track down other kinds of Web problems. And sometimes just doing Google searches can be an effective means of finding a particular site of a disgruntled exemployee.

One tactic is to educate your C-level executives, by looking for workshops or passing along articles and Web sites of interest.

Brian Cohen, who was SPIdynamics' CEO before the company was acquired by HP , suggests hiring established security firms that are doing traditional vulnerability assessments of operating systems and networks and looking to expand their offerings into the Web presence area. The key is having a solid grounding in Internet security, and being able to do regular scans to ensure that changes to a Web site haven't opened up new vulnerabilities.

"Business managers have lots of problems they need to investigate -- compliance, security, and just general network operations. They need to be able to analyze what's happening on their network and collect the evidence for taking action, regardless of which application (e-mail, IM, Web mail, etc.) is involved," says John Bennett, VP of Marketing for WildPackets Inc.

As you can see, doing network discovery has many different dimensions, tools, and cuts across a variety of skills. But as Bennett says, "IT forensics itself is simply a new category of must-have technology that is appropriate for any business manager today."

This article was adapted from Internet.com's CIO Update Web site.

Opportunities Abound in Transitioning Network Management Marketplace

2008-07-31T06:22:00.000-07:00

The network operations center (NOC) is no longer just about managing the network, it's now a part of a broader operations and ultimately business service vision.

Now, to be clear , I realize this suggests one of two things: either viewing network operations as an organization used to evangelize and bring IT processes into the mainstream; and/or leveraging the existing capabilities in some NOCs to manage across network/system/application interdependencies to integrate other more siloed IT organizations from the data center .

For instance, while ITIL best practices typically don't begin with the NOC, but more often with the service desk and then the data center . In fact, the trend can sometimes be the reverse. I would say that while in most organizations the NOC is the last group on board with configuration management database (CMDB) system implementations, in probably 25 percent of the cases I've worked with it's the first group to implement an early phase CMDB; typically oriented at service impact management. And I've often heard NOC directors complain about the lack of network support for many CMDB system offerings today — proving that the NOC is ahead, not behind, market.

The reasons for this somewhat schizophrenic role vis-à-vis more holistic management shouldn't be too much of a surprise. On the one hand, the NOC, and network engineers in particular , are famous for being stubborn, independent, and "misunderstood." By image, at least, they are on the high end of fitting in with being classically noncommunicative "Dilberts."

On the other hand, many network operations organizations are already established in managing across interdependencies so that problems can be diagnosed across the network, or isolated to the application, or the server , or the database. In accordance with this, many good network management solutions are capable of identifying application design issues, such as chatty applications, or informing on server performance.

As a result, in many IT organizations, it's the network team, particularly the network engineers, who are best prepared to coordinate troubleshooting across silos, or plan for overarching requirements in infrastructure optimization, or manage remote locations including systems and application access/responsiveness.

The Vendors & The Marketplace

Recent data shows that an astonishing 51 percent of purchases involving network change and configuration management solutions were made in conjunction with a CMDB initiative. Standalone management purchases came in a distant second at about 16 percent.

Purchases made in conjunction with systems management configuration and other software came in next at 13 percent, beating out purchases of network device hardware needing configuration tools at 12 percent.

This data is radically different from what we would have seen five or even three years ago. It's one indicator that planning network management strategies is becoming a much more holistic endeavor . Another striking data point is that 64 percent of our respondents from Q4 2006 indicated that their organization had done, or was about to make, some organizational change to facilitate better collaboration between the NOC and the data center .

Investing in management solutions is no longer just about buying siloed tools to manage just the network. And much of the push and shove in the network management market is consciously or not driven by this very fact. Vendors selling network management solutions know they have to change their business model to support a broader set of roles; from engineering to operations to service assurance across all domains.

Given that, and getting back to technology, I'd like to wrap up with Enterprise Management Associates' recommendations for assessing the value of management investments for what we call "next generation service assurance." These requirements were evolved to address cross-domain requirements that impact all of IT operations, but it turns out that every one of them are relevant to making investments on the network management front.

They include:

• Discovery that supports not only network, but (ideally systems and application dependencies for performance management and asset and inventory

• Analytic capabilities that can triage across network, systems, and application issues

• Leveraging modeling technology to capture relationships to drive more focused approaches to automating diagnostics, or even reconfiguring devices

• Using application flow and route analytics in monitoring application services themselves

• Integrated support for configuration so that when changes are made to the infrastructure or its services, performance management capabilities are proactively aware of anything that's non-policy compliant

• Support for Web Services and SOA application componentry across a distributed networked environment

• Integration between fault and performance management

• Integration between network and security management (a No.1 priority in many shops)

• Active control to change network configurations, or optimize bandwidth, or in some cases actually do dynamic server rebalancing in conjunction with network performance

• Support for lifecycle management so that understanding, for instance, an application's robustness in a highly distributed networked environment prior to deployment can help to ensure that promised SLAs are actually deliverable.

These are, admittedly, only some pointers targeted at evaluating solutions specifically in support of service assurance. Other disciplines such as capacity planning, asset management and financial planning, and configuration management in and of itself have their own lists of design attributes. But this list, in combination with a good plan for evolving organization and process, can serve as a good departure point for planning strategic service management technology adoption.

These capabilities not only answer technical needs, but they reinforce and enable the kind of cultural and process change that many IT organizations are just beginning to make, often with strikingly positive results.

This article was adapted from Internet.com's CIO Update Web site.

Why Is Networking Growing So Fast?

2008-07-31T06:20:00.000-07:00

Did you know that the growth of networking spending is outpacing all other sectors of
IT? Abner Germanow, director of enterprise networking at IDC, gave his reasons why to a standing-room only audience at Interop in May 2007.

According to Germanow's IDC data, enterprise networking growth was 17 percent in 2006. In contrast, servers grew by 6 percent, packaged software grew by 8 percent, and storage grew by 6.2 percent.

When he reported the figures to his management at IDC, they asked him to double check since they couldn't believe that networking could be growing that fast because it's a mature market.

So what is propelling networking? According to Germanow it boils down to five driving factors. The No. 1 reason is voice and video. Germanow noted that analysts and vendors have been talking about voice and video for years, mostly as a future-proofing topic. But in 2006, voice became a reality.

IDC is projecting an 11 percent growth in compound annual growth rate for IP-PBXs through 2011 and a 22 percent CAGR for IP phone gear. That growth will fuel additional demand on the network.

Video is also a massive networking bandwidth hog and is placing large demands on IT infrastructure. It's not just YouTube users that are driving networking video demand, either. Germanow noted that there is a solid case also for IP surveillance and video training, and that there is a lot of excitement around conferencing and telepresence.

The No. 2 reason for the growth of networking is the rise of network-based businesses. Because the network is the business, Germanow said, networking-based business is all businesses and not just Amazon, Google, and eBay.

"The way that companies succeed is the applications that they use to reach their customers," Germanow said.

There is also a shift toward an appliance-based model across the networking space where both hardware and software services co-exist in one form factor.

Originally appliances were just for security devices but now the trend is more wide spread. "The new context for hardware appliances is as a simple service delivery component," Germanow said.

The third reason for the growth of networking, according to Germanow, is the fact that intelligent networks are winning. With all the applications and users coming onto the network, the need for intelligent networking gear is a key enterprise requirement.

The fourth reason for the growth of networking is end-point growth. Germanow noted that their has been an explosion of network touch points, which means that more items are connecting to the network.

Virtualization is the No. 5 reason on Germanow's list of factors driving networking growth.

"The first thing that happens is you consolidate servers, which reduces your port count, and as a network guy that's scary," Germanow said. "You lose low utilization connections and move to highly utilized connections where the importance of each port is increased."

While new application demands are helping to drive adoption, Germanow urged the audience to remember performance.

"While the market is shifting from speeds and feeds to features and functions, performance is still critical," he said. "Performance has to be there."

This article was adapted from Internet.com's InternetNews.com Web site.

Understanding a Network Management Marketplace in Transition

2008-07-31T06:18:00.000-07:00

Just a casual look atmedia headlines would suggest that the networkmarketplace is in transition. But the question remains, a transition to what?

Probably the most compelling evidence is that many major network management vendors have been acquired.

A few examples:

• Event correlation and root cause vendor SMARTS by EMC in February of 2005.

• Concord/Aprisma, a combination of network performance management and root cause
analysis, by CA in June of 2005.

• Micromuse, a leader in service level management with strong service provider roots
by IBM, announced in December of 2005.

• Quest acquired Magnum Technologies, with strengths in root cause, performance and service management in May of 2007.

The drumbeat goes on. HP acquired Opsware, which includes strong network configuration management capabilities. On a less major scale, BMC acquired Real Ops with strong roots in network process automation, and BMC also formed a partnership with Entuity for root cause diagnostics; all three occurring in July of 2007

Nevertheless, there are a significant number of free-standing network management vendors, a number of which have made acquisitions of their own. Among these, Fluke has acquired Crannog and Visual Networks for application service performance management, OPNET acquired Altaworks for Web-based, transaction-driven, application management, Network General acquired Fidelia for a variety of service and event management features, NetScout then acquired Network General and then Quantiva for application analytics, and NetQoS acquired RedPoint Systems for SNMP-based polling.

All this activity suggests the network management marketplace is consolidating and reshaping itself at a rapid rate. Vendors are struggling not only to gobble up new real estate, but perhaps even more importantly, to reposition who they are within a broader management marketplace that’ s similarly in a rapid state of flux.

Planning & Strategies

What does this mean for you when it comes to planning IT management strategies that increasingly depend on network efficiencies in reaching distributed environments? It at least implies that traditional siloed approaches to buying niche tools in isolation may not be the right strategy for very much longer . Along with all these acquisitions, in fact, the network management market is deconstructing and reconstructing itself to support more cross-domain requirements, better analytics, and more modular approaches to deployment and functional packaging.

Let's take a look at some of the hot spots in the new network management marketplace.

Deconstructing and reconstructing: Platforms such as CA, EMC, and IBM are not just acquiring domain-specific management capabilities when they acquire network management vendors. They are also looking for extensible and reusable parts.

For instance, CA is evaluating how far it can extend SPECTRUM’ s inferencing engine as a source of root cause diagnostics across the broader infrastructure, including servers and application services, just as EMC has extended SMARTS analytics to support storage and application flows. IBM will be leveraging Micromuse discovery and has already integrated its dashboard into a more cohesive service management capability.

Network management in support of application delivery: If the network is an instrumented ocean across which all applications must flow, then it makes sense to exploit the network as a resource in monitoring application traffic and diagnosing application problems — and not only when the network is at fault.

At minimum, good network management today can accurately isolate where a problem is occurring in large distributed environments, often pinpointing specific servers or exposing chatty application design.

The real-time rise of real-time: Application flow management across the network is increasingly being done in real-time, or near real-time. This is because unlike traditional, component-centric network performance management, it doesn't require polling.

Some capabilities, such as route analytics, expose the actual path of application traffic in a fully real-time context. This type of visibility will become yet more important with the advent of service-oriented architectures (SOA) that exploit distributed networked access beyond single data center implementations.

The rise of configuration management: In part because of the IT Infrastructure Library (ITIL) and its emphasis on configuration management (meaning visibility into configuration and topological change) as an enabler for all management disciplines, network configuration capabilities are becoming increasingly strategic for IT buyers. The multi-purpose benefits of good configuration management tools to help automate change, as well as to support more effective diagnostics, compliance, security, asset management, and other disciplines represents a breakthrough in both network management and in the industry at large.

OSS to IT: Operation support systems (OSS) as used by classic telecommunications providers are beginning to look at IT best practices such as ITIL, and more conscious support for application delivery services.

In parallel, IT organizations are increasingly beginning to adopt a more service provider-like posture with a focus on measurable accountability and quality. These commonalities are also causing cross currents in the market that are allowing vendors to repurpose innovations targeted at one set of buyers to serve the other—typically at different levels of scalability and different price points.

The famous American novelist, Nathaniel West, when confronted with a poetry magazine called Transition populated by the ill-conceived writing of pampered starlets was asked to comment on the quality of the contents. His reply, arguably the single fastest uptake in the history of American literary discourse was, "All I can say is – 'Transition' spelled backwards is 'no it isn' art.'"

But the network management marketplace is transitioning to something better , difficult though it is to see through all of its complexity. Perhaps the core to appreciating this brave new world is to focus on the deconstruction and reconstruction along the lines of correlation and analytics, or more advanced discovery now visible within some of the more enlightened platform architectural strategies.

Investing in network management is no longer just about managing networking hardware. It's about investing in extensible technologies that may be applied in application and service management, change and configuration management, discovery and asset management, security and compliance, just to mention a few areas.

This article was adapted from Internet.com's CIO Update Web site.

The New Networking Landscape

2008-07-31T06:17:00.000-07:00

Like two heavyweight prizefighters slugging it out in a titlematch, networking colossus Cisco and challenger Juniper Networks tradedmonumental announcements in early February in their battle for themultibillion-dollar switchmarket.

Make no mistake about it, the new product announcements from Cisco (for its NX-7000) and from Juniper (for its EX-series switches) are big news in the networking world. There is no question that as demand continues to increase for Internet bandwidth, Cisco and Juniper each want to be the vendor of choice to help meet it.

To add to the stakes, it's important to remember that the networking business is one that has typically experienced long product cycles. As a result, the new products from Cisco and Juniper may well help to shape the face of networking for the next five to ten years, or longer .

For Cisco, the NX-7000 represents a new platform shift toward a fully 10-Gigabit Ethernet (10 GbE)-optimized switching infrastructure. It also marks the first major new switching platform from Cisco since the first Catalyst debuted more than a decade ago.

For Juniper , the EX switch portfolio marks the entry of Juniper into the switch business itself. At long last, Juniper can now claim that it has an end-to-end portfolio -- from big routers to big switches to security and everything in between.

In the switch market, particularly, much is up for grabs. A recent market forecast from research firm Dell'Oro Group reported that the Ethernet switch market was worth $18.1 billion in 2007 and is expected to hit $21.8 billion in 2012.

A key driver of Ethernet switch growth is the need for higher speeds, such as 10 GbE and greater . Both vendors also claim that their respective platforms will be ready for 100 GbE, which is the next major shift for the networking market. The technology is expected to begin rolling out in late 2009 or early 2010 as the standard is ratified.

While the networking space is rife with vendors who claim to rival Cisco, the reality is that few have the scale to compete successfully across an entire networking infrastructure play.

In my experience when briefing with Cisco, I'll ask about competitors (or there is already a competitive slide in their deck.) More often than not, Cisco will name Juniper .

Certainly Foundry, HP Procurve, and Nortel are all viable competitors in the networking space as well. Yet none of them to date have really challenged Cisco in the core routing business, which is where Juniper continues to gain share.

Juniper's T1600 multi-terabit router offers some real competition to Cisco's flagship CRS-1, AKA "Huge Fast Router ." The reality is that big routers need big switches, and I'd suspect that, typically, many networking buyers will choose to acquire both routers and switches from the same vendor .

The day that Juniper made its announcement, I also got some comments from Nortel, who said it welcomed Juniper's entry into the switch business. A Nortel spokesperson noted that the announcement shows that customers are demanding choices other than Cisco, which Juniper and Nortel can offer .

The spokesperson added that the Juniper announcement also validates Nortel's position that the market is moving toward converged solutions. Yet according to Nortel, enterprise data is a two-horse race -- Nortel and Cisco are it, while Juniper is late out of the gate.

While Juniper is likely to disagree strongly with Nortel's assessment, the simple reality is that Juniper is doing well financially and is growing overall share at a rapid rate. As a result of this and its move into the switch business, along with Cisco, it's poised to be one of the two companies setting the pace.

Fundamentally, success in the new networking environments is about end-to-end speed with a complete infrastructure portfolio. The move toward 10GbE and more importantly, 100GbE-capable networking infrastructure, is critical to enterprises' continued growth and stability, since they need speed to compete in the global economy.

Regardless of who ultimately triumphs, the high-profile, big-dollar announcements from Cisco and Juniper no doubt will serve to expedite the move to increased speeds. They will also raise awareness in networking groups around the globe that a faster world for data is coming soon.

This article was adapted from Internet.com's InternetNews.com Web site.

In 2008, Your Network Will Know Who You Are, What You Want

2008-07-31T06:13:00.000-07:00

In 1949, GeorgeOrwell published hismasterpiece novel 1984. Even if they haven't read it,most people remember the book's key takeaway: Big Brother is watching.

Fast forward to 2008 and Big Brother really is watching. The vehicle that makes Big Brother's omniscience possible is the network.

The network: that mass of boxes, interface cards, cables, and antennae that when combined become a platform forevery type of human interaction and collaboration on the planet Earth (and beyond).Let's take a look at how the networking world of 2008 will become all knowing and all seeing -- a silicon- and fiber-based Big Brother , if you will.

Network Smarts

A theme I heard time and again from vendors big and small last year was that the network was getting "smarter ." More intelligence is being embedded into the network layer than ever before, with 2008 now poised to be a watershed year for the smart network.

Today, networks at a basic level are no longer "dumb" pipes that transport information. The networks of 2008 will build on recent and coming innovations to become application- and user-aware -- they'll know who you are and what you are allowed to do (or what you're prohibited from doing).

Intelligence in 2008 will arrive in the form of more Ethernet standards that provide increasing amounts of information about data types. Intelligence will also come in the form of smarter quality-of-service (QoS) and bandwidth-management offerings that intelligently provision the right bandwidth at the right QoS for users and their applications.

Network Security

The smarter network of 2008 will rely strongly on Network Access Control (NAC). While NAC has been a buzzword for several years, NAC will go mainstream in the coming year thanks to Microsoft. A key component of Microsoft Windows Server 2008 is what it terms Network Access Protection, or NAP . The cornerstone of the technology is pre-admission control: A NAP server will first validate the health of an endpoint (a user or machine, for instance) before allowing admission to the network.

Microsoft's Windows XP Service Pack 3, as well as Windows Vista, are both ready to serve as NAP end-points. Due to Windows' massive installed base,Microsoft's NAP will be something that enterprises can activate out of the box to begin to secure their networks.

The power of access control for the smart network of 2008 cannot be understated. If hundreds of millions of Windows users are using NAP , it may end up being the single most important security innovation since the invention of the firewall.

Imagine: a world where insecure endpoints aren't granted access to do their dirty deeds. What a wonderful world it would be.

Network Identity

The smart network of 2008 isn't just more secure, it also knows who you are and what you need access to. While directories such as Microsoft's Active Directory have been used for identity for years, they're not enough. The 2008 network will have identity built into the framework of the network itself.

The big push for network identity in 2008 will come from Cisco, with its TrustSec initiative. Instead of a user needing to enter multiple passwords for each and every application they need to visit, a TrustSec-powered network essentially will know who they are, what their business function is and where they're allowed to go.

From a Big Brother-auditing point of view, TrustSec, and its various competitive implementations from vendors other than Cisco, also will offer a full audit trail of
a user's activities at both a network level and the application level.

By embedding identity into the network layer , the network will have better understanding and control over what users are doing.

Network Speed

With greater intelligence in the network, efficiency is likely to improve, though it still won't be enough for the bandwidth-intensive demands that networks will face in 2008.

It's possible 2008 could well be a breakout year for 10-gigabit Ethernet, or GbE, which so far has lagged in adoption. With costs falling for 10 GbE equipment, coupled with rising bandwidth demands, 10 GbE will likely be on the list of many IT admins' requisition forms.

While enterprises and data centers start full adoption of 10 GbE, standards bodies such as the IEEE will be hard at work finalizing even faster speeds.

At some point in 2008, the 100 GbE standard is likely to be published as a draft, providing a ten-fold increase in Ethernet connection speeds. The 100 GbE standard will also include specifications for a 40 GbE standard,which will likely set the stage for a final showdown between SONET and Ethernet.

The fastest connection possible in 2007 was the venerable 0C-768 at 40 gigabits per second. Once 40 GbE and 100 GbE come into play, it may well only be a matter of time before OC-768 loses share to the Ethernet upstarts.

IPv6: Everything Gets an Address

Orwell's Big Brother was all knowing because he was everywhere. The networks of today aren't quite there yet, but with IPv6, they get much closer .

The current version of the Internet Protocol (version four , or IPv4) relies on address space that's near exhaustion. IPv6, with its billions upon billions of possible addresses, will be its successor -- and 2008 will be the year it finally takes off.

Why 2008? Because Uncle Sam says so.

In June, there is a Federal government mandate for the U.S. government's IT to switch to IPv6. It's a move that will spur tens of billions of dollars in capital and software upgrades. It will also force all those business that deal with the government to strongly consider IPv6 as well.

With IPv6's massive address space, anything can have an IP address. When anything -- be it a server , a phone or even just a refrigerator -- has an IP address, the network becomes pervasive.

Vendors

So who's the power behind "Big Brother"? Who is the face behind the network? Well, for 2007, it was Cisco. In 2008, Cisco is still likely to be the chief mastermind behind the network's growing capabilities, though competitive challenges will continue to emerge.

In the core routing space, Juniper Networks will make still more inroads, chipping away at Cisco's dominance. Expect either someone to buy Juniper this year , or Juniper to make its own purchase of a switch vendor to bolster its bid for network dominance.

HP , Nortel, Alcatel-Lucent and others will also ratchet up their competitive offerings -- as well as the marketing hyperbole -- as each takes on Cisco.

The bottom line, though, is that all the vendors are pushing the same goal: faster , more aware and smarter networks.

So remember , Big Brother isn't just a literary fabrication anymore. In 2008, Big Brother is the network.

This article was adapted from Internet.com's InternetNews.com Web site