What is SUMI?

http://sumi.ucc.ie/whatis.html

SUMI_sample.pdf

What is SUMI?

[SUMI Homepage]

---

The Software Usability Measurement Inventory is a rigorously tested and proven method of measuring software quality from the end user's point of view.

SUMI is a consistent method for assessing the quality of use of a software product or prototype, and can assist with the detection of usability flaws before a product is shipped.

It is backed by an extensive reference database embedded in an effective analysis and report generation tool.

Who should use SUMI?

SUMI is recommended to any organisation which wishes to measure the perceived quality of use of software, either as a developer, a consumer of software, or as a purchaser/consultant. SUMI is increasingly being used to set quality of use requirements by software procurers.

SUMI also assists the manager in identifying the most appropriate software for their organisation. It has been well documented that if staff have quality tools to work with, this contributes to overall efficiency of staff and the quality of their work output.

Our customers have used SUMI effectively to:

• assess new products during product evaluation
• make comparisons between products or versions of products
• set targets for future application developments.

SUMI has been used specifically within development environments to:

• set verifiable goals for quality of use attainment
• track achievement of targets during product development
• highlight good and bad aspects of an interface.

SUMI is the de facto industry standard questionnaire for analysing users' responses to desktop software or software applications provided through the internet.

Why use SUMI?

SUMI is the only commercially available questionnaire for the assessment of the usability of software which has been developed, validated, and standardised on an international basis.

There is a large range of languages in which SUMI is available. Each language version is carefully translated and validated by native speakers of the language.

SUMI ennables measurement of some of the user-orientated requirements expressed in the European Directive on Minimum Health and Safety Requirements for Work with Display Screen Equipment (90/270/EEC).

SUMI is mentioned in the ISO 9241 standard as a recognised method of testing user satisfaction.

What do I get when I buy SUMI?

We now have a variety of methods in which we deliver the SUMI service.

Professional consultancy service

This is a service housed at University College Cork, the url of which is sumi.ucc.ie/en/ (for the English language version - other language versions are coming onstream with demand.) You can see the questionnaire on this link but you'll need a password to be able to send data.

This service provides a detailed report giving the SUMI output including content analysis of some additional questions.

Basic online service

As above, this service is housed at University College Cork, the url of which is sumi.ucc.ie/en/ (for the English language version - other language versions are coming onstream with demand.) You can see this questionnaire on this link but you'll need a password to be able to send data.

This service provides a CSV file of scored data and the additional questions but little else. It is intended for studies of between 12 to 100 users and clients who prefer to do their own analyses.

Offline SUMI service

This service is primarily intended for labs who want to do a quick in-house test, or academic users who cannot purchase the online or the standalone service (for student work towards a major thesis, for example.) You will get a questionnaire in pdf form and a licence for the period of your data collection. You make as many copies of the questionnaire as you need. When you have gathered your data, you send a file of data to HFRG who will then produce a standard SUMI output and send it back to you by e-mail.

Dates for submitting and sending data back must be agreed upon beforehand. The usual turn-around time can be very short (half a working day) if you have arranged this with HFRG.

If you are a student and want to apply for a free use of SUMI you will have to get your academic supervisor to write a letter on headed stationery stating that the work you are carrying out is academic and not for profit or part of a consultancy arrangement (see the suggested wording, below.)

Customised service

For a large scale survey, or a survey with additional questions, you will have to contact Dr Jurek Kirakowski at jzk@ucc.ie to discuss your survey and get a quotation for the cost of it.

SUMISCO is based on profiles of about two thousand responses to commercially available software. It will tell you how your product compares to this standardisation base: whether you are about average for the market, below, or above. Because of the background statistics in our database, you can find out very precisely how you compare to the market.

What does SUMI look like?

SUMI consists of 50 statements to which the user has to reply that they either Agree, Don't Know, or Disagree.

Here are some example statements:

Item No. Item Wording

1.       This software responds too slowly to inputs.

3.       The instructions and prompts are helpful.

13.      The way that system information is presented is clear and understandable.

22.      I would not like to use this software every day.

You may also take a look at a sample questionnaire (UK wording) in pdf format,

How do I administer SUMI and how long does it take?

It takes a user about 3 minutes to fill out the questionnaire, perhaps a few minutes longer on the internet version.

One way of administering it is on paper: print out the SUMI form and get your user to make marks on the page. It takes an analyst about one minute to type each user's responses into a file for scoring by SUMISCO or to send to HFRG for scoring.

On the other hand you may decide to go for the internet, online option.

You can also do a hybrid: concoct your own HTML pages to be served on your intranet and either send the results to HFRG for analysis, or purchase SUMISCO and analyse them yourself.

How many users do I need?

Online SUMI might require sample sizes with a minimum of about 30 unless your respondents are well targetted.

However, we know that paper SUMI will give you reliable results with as few as 12 users. This is because you are able to control the quality of your user sample directly when administering SUMI on paper.

You can use fewer users if you wish, but beware that your results may not be as representative of the true user population. In fact, SUMI has yielded useful information with user sample sizes of four or five.

However, this question is always like 'how long is a piece of string.' You should try to get as many users as you can within your timeframe.

How much does it cost?

It all depends on the kind of service you want. See the Pricing and Ordering Information page.

Are there educational discounts?

We don't do educational discounts on SUMI.

However, if you are a student pursuing a full-time course of study and wish to use SUMI for your project or term assessment, eMail Dr Jurek Kirakowski and discuss the use you would like to make of SUMI. This year's students are, after all, next year's professionals...

Because of some bad experiences with students from a University which shall not be named, for a student licence we will need information from the permanent staff member supervising the project involving SUMI, who must supply the following undertaking on headed stationery:

My student [student name] of the [department name] is carrying out research involving the use of the SUMI questionnaire during the period [start month] to [end month] involving the evaluation of the computer system [name of system]. This is to guarantee that [student name] is not being paid directly for the evaluation work nor that [department name] is in direct receipt of any funds or consideration for this work. [name of supervisor] [position in department] [postal address]

This will assure us that students are not carrying out commercial activities in the guise of student projects and will therefore not be under-cutting commercial SUMI clients.

---

[SUMI Homepage] [Top of page]