Tuesday, December 13, 2011

Rommana/HP Quality Center Comparison



The comparison below have be conducted by individuals who have used the functionalities of the tools and information has been stated as accurately as possible and to the best of knowledge of these individuals.

Some information has also been verified by HP and Rommana Software.

Anyone can submit corrections to any statement made in this document by writing on this blog all comments you have, as long as the correction is verifiable.

As you know all software tools are always under updates and adding new features and components, I will try to keep this blog up to date, with the information about this tools so on this way you can link to this document for future reference.





Rommana

Quality Center

General

Platform

Server: Windows 2003, 2008, Vista, XP, Windows 7

Client: IE6 or higher

Server: AIX, Linux, Sun Solaris, Windows (2000, 2003)

Client: IE6, not possible to work with IE9 or higher

Architecture

Rommana does not require any specific architecture other than the server must be running IIS.

HP recommends a 3-tiered approach:

Tier 1 - Load Balancer (Depending on your organization's network structure, the Load Balancer may require purchasing additional licenses).

Tier 2 - Application - Supported platforms are JBoss (which requires subscription and maintenance costs), Websphere (license cost associated) and Weblogic (license cost associated).

Tier 3 - Database - Supported platforms are Oracle or SQL Server, both of which require licenses.

There is also a 4th tier that normally required but is not mentioned in the documentation, and that's the File Repository.

Data Base Used

Rommana does not require a license for a database engine. It uses SQL Express, which is available free of charge for Microsoft. It also works with SQL server if available.

Quality Center requires a licensed copy of SQL server or Oracle database engine.

Integration

Rommana is a fully integrated system that uses one single repository (SQL Server) and one Common User interface to support all 6 components listed under features

Quality Center uses separate repositories for its main user client, its Administration module and its Dashboard module.

Methodology Supported

Rommana supports the following methodologies:

- Scenario Bases Development and Testing (SBDT™)

- Requirement Based Project Management (RBPM™)

No specific methodology support. Quality Center needs to be customized to work as you need.

Requirement Management Features

Custom Attributes

Yes. Rommana allows defining custom attributes of different data types including multiple selection and single selection attributes.

Yes. Quality Center supports field-level customization in its modules.

Associating Models, files, and artifacts to requirements

Yes. Rommana allows the use of models and artifacts such as decision tables, state models, process flows, user interface and screen prototypes, etc to provide more details for a requirement or a group of requirements. The use can define any type of model as appropriate.

Rommana by default sort the requirements by functional and non-functional requirements.

Quality Center allows file attachments to Requirements, Test Cases, Test Runs and Defects. It supports artifacts as attachments, but not links to stand-alone artifacts.

You need to know how to set this link among these objects, otherwise when you try to get metrics from dashboard your results will not be trustable.

Requirement Versioning

Yes, Rommana maintains different versions of the same requirement as requirements are changed based on change requests.

Quality Center began supporting requirements versioning in version 10.0. Previous versions of QC require integration with 3rd party software to support requirements versioning.

Associating Use Cases with Requirements

Yes. Rommana allow defining use cases for a requirement to provide more details. Use cases can be changed any time to reflect changes in the requirement. The Use Case Management component of Rommana supports all details of a Use Case Based requirement process.

Quality center does not support Use Cases. Quality Center only support requirements and then you link them directly to the test cases. There is a gap, because scenarios (use cases) are not documented.

Associating Tasks with Requirements

Yes. Rommana allows associating project tasks to a requirement or a group of requirements. These tasks can be almost any thing that the project team needs to do in order to deliver the requirement. Tasks are defined by the user. Based on the specific methodology they are using. The purpose is for the Project Management component of Rommana to provide effort and cost estimation as well as progress for each requirement or a group of requirements.

There is no specific method to track tasks.

Associating Project Risks with Requirements

Yes. Rommana allows associating certain project risks to a requirement or a group of requirements. This allows test leads, test managers, and development mangers to assess the feasibility of delivering certain features in a given release.

Quality Center began supporting Risks to Requirements in version 10.0, which allows risk assessment based on several factors. It is not so clear how it works, some companies have this features but do not use it.

Associating Actions with Requirements

Yes. Rommana allows associating action items to a requirement or a group of requirements. This allows test leads, test managers, and development mangers to keep track of different action items that are needed to deliver the requirement.

There is no specific method to track action items.

Importing requirements from Word documents

Yes. Rommana allows importing requirements from a word documents. Rommana also maintains the link between the requirement and the Word document.

Yes. Quality Center supports a Microsoft Word template and Excel template for importing Requirements.

Breaking requirements down to scenarios

Yes. Rommana allows breaking down requirements to scenarios to provide more details about the requirement.

There is no support of the concept of scenarios. You need to do a detailed information in the same module “Requirements”. And you can get a mess of trees.

Planning requirements for different releases

Yes. Rommana allows planning different requirements for different releases.

Quality Center introduced Release Management in version 10.0. You can now in Quality Center have order of the releases, and link them to the requirement. The link is directly from releases to the requirement.

Associating Change Requests to requirements

Yes. Rommana allows associating a change request to a requirement to create a new version of the requirement.

Quality Center supports versioning of requirements. However, versions cannot be linked to an event or artifact.

Test Management Features

Methodology

Rommana uses a scenario-based test design methodology

No specific methodology support. You need to configure one, do codification on it.

Defining Test Case based on Scenarios

Rommana supports creating and grouping test cases by scenarios

No support of scenario based test design. Quality Center has a module “Test Plan” dedicated to create test cases. Not scenarios.

Defining Test Cases based on Use Cases

Yes. Rommana allows for inheriting pre-conditions and post conditions from Use Case Scenario to test cases.

No support of Use Cases

Separating Test Cases from Test Scripts

Yes. Rommana enforces data-driven test design by separating the Test Case, which consists of input, Pre-Conditions, Output, and Post-conditions, from the Test Script, which is merely the set of steps to be followed to run a group of test cases (a Test Set).

A Test Case in Quality Center represents a test procedure. Each combination of data elements for a particular Test Case must be created as a separate Test Case.

Grouping Test Cases into a Test Set that uses the same Script

Yes. To minimize the number of test scripts, Rommana allows grouping test cases into a test set. A test script can then be defined for the test set.

Quality Center allows grouping of Test Cases into Test Sets. However, each instance of a Test Case (test procedure) must be represented separately in the test set.

Allowing a Test Case to be moved to a different Test Set.

Yes. User can move a Test Case from on Test Set to another any time. The Case will then use the script defined for the new Test Set.

Yes. A user can move a Test Case to another Test Set. However, the steps and all other elements in the Test Case remain the same.

Defining Pre-conditions and Post-conditions for each Test Case

Yes. Rommana allows defining pre-conditions and post-conditions for test cases.

With customization, Quality Center can define pre-conditions and post-conditions. You need to create these fields in the feature “Customize”, it takes a couple of minutes to do this.

Maintaining the relationship from a Test Case back to Scenarios, Use Cases, and Requirements.

Yes. Test Case Traceability in Rommana allows for tracing Test cases to requirements, scenarios, use cases, and issue reports.

No support of Scenarios or Use Case. Quality Center only keeps traceability from Requirements to the Test Cases.

Allowing a test Case to be moved to a different Requirement, a different Scenario, or a different Use Case.

Yes. When moving a test case to a different requirement, the user will need to specify the Use Case and/or the Scenario from those defined for the new requirement.

Links between entities can be defined and deleted at any time in Quality Center.


Providing statistics for each Test Set regarding number of Test Cases passed, failed, or not executed yet as well as % of completion and % of readiness for release

Yes. Rommana provides statistics on the status of test case execution for each test set.

Yes. Quality Center provides statistics on the status of test case execution for each test set.

Providing statistics for each requirement regarding number of Test Cases passed, failed, or not executed yet as well as % of completion and % of readiness for release

Yes. Rommana provides statistics on the status of test case execution for each requirement.

Yes. Quality Center provides statistics on the status of test case execution for each requirement.

Providing the option to open an Issue Report when a Test Case fails

Yes. When a test case is marked as Fail, Rommana gives the option to open an Issue Report and it fills in some info from the failing test case.

Yes. Quality Center supports defect creation from failed Test Cases. Quality Center will also fill in some defect information from the test case being executed.

Enforcing the association between Test Cases and Issue Reports as the status of either changes

Yes. When a test case is marked as pass, if there is any issue report associated with the test case still open, Rommana displays a warning and give the option to close the issue report.

Quality Center supports optional email notification to defect assignees when an associated test case is passed.

Ad hoc Test

Yes. Rommana allows creating a test on the fly without following the Rommana Scenario Based methodology. However, an Ad hoc test may optionally have a related requirement, a related use case, or a related scenario

Quality Center allows Test Case creation through its standard interface. You only need to create a special folder in the tree menu for this purpose.

Defect/Issue Management

Enforcing the association between Test Cases and Issue Reports as the status of either changes

Yes. When a test case is marked as pass, if there is any issue report associated with the test case still open, Rommana displays a warning and give the option to close the issue report.

Quality Center supports optional email notification to defect assignees when an associated test case is passed.

Enforcing Impact analysis

Yes. Rommana requires developers to provide impact analysis information when an issue is marker as fixed. This impact analysis is use by the test team for regression testing.

Comments are an optional field in the Defects module in Quality Center. There is no separate input field for a developer’s fix report.

Issue Report Lifecycle

Yes. Rommana supports an Issue Report Lifecycle from creation by the test team, handling by the development team and closure by the test team. Rommana maintains the history of all changes.

Yes. Quality Center supports a Defect Life-Cycle between testers and developers. Each defect stores the history of its changes.

Ad hoc Issue Reports

Yes. Rommana allows creating an issue report that is not based on a specific test case. However, an Ad hoc issue report may optionally have a related requirement, a related use case, a related scenario, or a related Ad hoc test. (Release 1.1)

Through customization, Quality Center provides a report of Defects that are not linked to a Test Case. Reports cannot be linked to other entities in Quality Center.

Use Case Management

· Define use cases to provide more details for a requirement

· Define use case scenarios for both development and testing

· Auto-validate use cases for completeness

· Maintains the traceability between Use Cases, Scenarios and Test Cases

Not Supported

Project Management

· Allocate and manage project resources, roles, responsibilities and access rights.

· Create and manage project task for both development and testing on the project level as well as on the requirement level.

· Create and maintain action item lists for both development and testing effort

· Manage risks throughout the project for both development and testing on the project level as well as on the requirement level.

· Create and maintain project timeline for all project tasks for both development and testing.

· Create and maintain cost and time estimates for both development and test tasks either on the projects level or the requirement level.

· Manage product release and features included in each release

Not Supported as 100%.

But:

· Maintain certain traceability on the test components you create, but not linked automatically.

· You can manage releases in “Release” module.

· You can execute test cases based on risks.

· You cannot do estimation on.

· You can allocate and define roles, but only developers and testers. You need to modify code if you desire more roles.

Change Management

· Create change requests either based on an existing requirements or just as a new feature

· Associate change requests to existing requirements to create new versions of the requirement

· Allow developers to provide impact analysis information for each change request for the test team to use for effective regression testing

Not Supported

Collaborative Inspections and Reviews

The Collaboration Management component of Rommana provides very strong support for collaboration between all project team. It allows for collaborative reviews of all project entities such as requirements, test cases, scenarios, change requests, etc.

Not supported

Test Execution

Supported through integration with Quick Test Pro

Supported through integration with Quick Test Pro

Prices

Free Trial

20 floating users for 6 months either installed or hosted

3o days free trial

Fixed License

$5,000 for first 5 users
$800 for each additional user

Not available

Floating License

$10,000 for First 5 users
$ 1,600 for each additional user

There are three different versions of Quality Center supporting three different Architectures: Enterprise, and Premier. The only version that supports multiple project teams at multiple sites the same way Rommana does is the Premier version.

It must be noted that all three versions of Quality Center support only 3 of the 6 components listed above in the feature comparison. These are Requirement Management, Test management, and Bug Tracking. None of the version support Use Cases Management, Project Management or Change Management.

Although HP does not publish Quality Center Prices, the prices for the premier version as provided by sales staff is as follows:

$37,000 for the first 5 users

$ 5,400 for each additional user

Subscription (SAAS) Rommana is running on hosted environment

$100 per user for 1-10 users

$90 per user for 11-20 users

$80 per user for 20-30 users

$70 per user for 31-40 users

$60 per user for 41-50 users

$50 per user for 51 or more users

Not available

Tech Support

Annually at 20% of invoice

Annually at 22% of invoice

No comments: