Client Satisfaction Research - Final Report 2010-2011

Other formats:

September 2011

Submitted to: Canadian Transportation Agency

Prepared by: Ipsos Reid Corporation

Executive Summary

Overview

This section details the key findings and recommendations which follow from our 2010-2011 research among clients of the Agency. Two hundred and thirty surveys were conducted among six different types of client, differentiated by their interaction with the Agency, including: those who made an inquiry to the Agency, those who went through an inspection or new licensing process, those involved in a travel-related dispute adjudication, those involved in a non-travel dispute adjudication or multi-party determination, those whose disputes were facilitated by the Agency, and those involved in a mediation.

Among the findings of the research we note that:

  • two in three clients (65%) report satisfaction with the overall service from the agency;
  • satisfaction is closely linked to outcome with nearly all (95%) whose objectives were met saying they are satisfied, in contrast with 21% among those who's objectives were not met (given its mandate to adjudicate, mediate, and facilitate the resolution of disputes, the Agency will always have clients who feel that their objectives are not being met and will rate their satisfaction with the Agency lower as a result); and
  • the Agency's level of client satisfaction is consistent with other government departments-we believe a goal of 70% overall satisfaction over the next three years is realistic.

Key Findings

To what extent are clients satisfied with their interactions with the Agency?

  • Two in three respondents (65%) report satisfaction with the overall service provided by the Agency, while one in five (21%) report that they are dissatisfied, and 13% report that they are neither satisfied nor dissatisfiedNote 1.
  • Levels of satisfaction vary by the type of client, ranging from a high of 84% among inspections and new licensing respondents to a low of 33% among travel-related dispute adjudication respondents (although the base of 12 among this group is too low to offer statistical reliability).
  • Respondents satisfied with the Agency most often say that they are satisfied because the staff were professional or courteous (27%), helpful or informative (27%), or because their issue had been addressed or resolved in a timely manner (25%). An additional one in six (16%) say that it was because their issue had a successful outcome or was resolved.
  • Respondents dissatisfied with the Agency most often say that they are dissatisfied because of the outcome of their issue or because the issue was not resolved (46%), because they received poor service (21%), or because the service was slow (15%).
  • Respondents were asked to rate their satisfaction with respect to several aspects of service from the Agency. Elements of service such as courtesy, being easy to deal with, or being knowledgeable and competent garner higher levels of satisfaction, while respondents are less satisfied with the time it took to resolve the issue (49% express satisfaction, 33% express dissatisfaction).
  • An area of lower satisfaction noted was with respect to the sense that the Agency accommodates those with disabilities. When asked whether they were satisfied or dissatisfied that the Agency "interacted with me in a way that accommodated my disability," the level of satisfaction among the 58 respondents to whom the statement applies (50%) was nearly as low as satisfaction with the timely resolution of issues (49%).

What drives satisfaction with the Agency?

  • Satisfaction with overall quality of service is linked to the perception that the respondents' objectives were met:
    • Satisfaction is nearly unanimous (95%) among those who say that the process they were involved with met their objectives (fully/largely), and is 52% among those who say it met their objectives to an acceptable degree.
    • In contrast, only one in five (21%) are satisfied among those who say the process did not meet their objectives (not at all/a bit).
  • Satisfaction with overall quality of service is also strongly linked to the timeliness of service. Those satisfied with the time to acknowledge or resolve their process/complaint are three times more likely to be satisfied overall than are those dissatisfied with timeliness.
  • Satisfaction sits at nine in ten (87%) among those satisfied with the time it took to resolve their process/complaint, and at three in four (74%) among those satisfied with the time it took to acknowledge; this is compared to 32% and 25% respectively among those dissatisfied in these same areas.
  • Those dissatisfied with timeliness or who feel their objectives were not met are less satisfied with every attribute of service and aspect of the service experience.

To what extent were client expectations met during their interactions with the Agency?

  • When asked to rate the extent to which they felt their interaction with the Agency met their objectives, over two in five respondents (44%) say that the process met their objectives either fully or largely, while two in five (39%) say that the process met their objectives only a bit (11%) or not at all (28%).
  • Travel-related dispute adjudication respondents most often say that the process only met their objectives a bit or not at all. Facilitation respondents are split between those who say that the process fully or largely met their objectives (44%) and those who say the process only met their objectives a bit or not at all (40%).
  • Mediation respondents were also more likely to say their objectives were met fully or largely than to say they were met only a bit or not at all.

Why do clients interact with the Agency and how do they do so?

  • Respondents tend to interact with the Agency on matters related to air transportation, except for non-travel dispute adjudication and multi-party determination clients and mediation clients, who are more likely to have dealt with the Agency on matters related to rail transportation. The specific matters they deal with range widely depending upon the audience, although most deal with air travel-related matters, ranging from baggage complaints to accessibility issues.
  • Web searches are the top source of awareness of the Agency; however, email (72%) followed by telephone (63%) are the top sources of information about Agency processes and services. As we have found in other research on government services, the website is often an entry point and source for basic information, while interpersonal communications (such as email and telephone) are where clients obtain information specific to their needs.
  • When respondents were asked to rate their satisfaction with a variety of aspects of service from the Agency, two in three respondents (68%) express satisfaction with respect to the variety of means of contact.
  • Majorities also expressed satisfaction with respect to different aspects of the Agency's website; however, these levels of satisfaction are lower than satisfaction with other attributes of service. For example, 63% express satisfaction that information on the website was easy to understand, 57% that the website provided necessary information, and 52% that it was easy to find information on the website. This compares to 69% expressing satisfaction that information from the Agency, in general, was clear and easy to understand.

How do the results of this research compare with other satisfaction research?

  • Given its mandate to adjudicate, mediate, and facilitate the resolution of disputes, the Agency will always have clients who feel that their objectives are not being met and will rate their satisfaction with the Agency lower as a result. Indeed, in this research, we find that those who say their objectives were met only a bit or not at all are much less likely to express satisfaction with the Agency overall and with respect to each of the many attributes of service tested.
  • As a result of its uniqueness, comparison to the client satisfaction results of other government organizations should be considered carefully:
    • Bodies with similar regulatory and dispute resolution mandates, including the Competition Bureau, the Canadian Human Rights Commission and the Canadian Radio and Television Commission, do not at this time conducted client satisfaction research.
    • The Institute for Citizen-Centred Service (ICCS), which publishes the common measurements tool (CMT) and compiles normative results on satisfaction with government services, does not have data on an organization with regulatory and dispute resolution mandates similar to the Agency.
  • At 65%, satisfaction with Agency service overall is consistent with the rating obtained by the Government of Canada in a 2006 Ipsos Syndicated Study, Government Service and Satisfaction, during which 62% expressed satisfaction with service from the Government of Canada overall.
  • In the same study, the Canada Revenue Agency obtained a higher top two box score (70%).
  • The 2008 ICCS study, Citizens First, identifies an overall level of satisfaction of 72% across the various government services tested in that research.
  • In terms of other service attributes tested in this research, the findings are consistent with what we have found on behalf of specific government departments and agencies, and with respect to government services as a whole.
  • In particular, our past research has found that government tends to score highly with respect to the fairness and respectfulness of staff, the accuracy of information, and the knowledge of staff. The Agency likewise scores well in this research against comparable measures.
  • Government tends to fare poorly when it comes to timeliness, flexibility, and the ease of access to service. In comparison, while the Agency also obtains low scores with respect to timeliness (49%), its scores for ease of access (68%) and the variety of means of contact (63%) are on the high side.

Conclusions and Recommendations

Issues and Observations Stemming from this Research

  • The Agency's dispute resolution mandate means that many of its clients will not feel that their objectives have been met, and will attribute a lower level of satisfaction to the Agency as a result.
  • Those who expressed dissatisfaction with the Agency point to the accuracy of information, the time to resolve their issue, a quick response time by staff, providing information on what can and can't be done in dealing with complaints, and being treated fairly as key areas for improvement.
  • It may prove difficult for the Agency to influence perceptions of fairness when the client has not received a favourable outcome to their complaint, and perceptions with respect to the accuracy of information and what can or cannot be done may also relate more closely to the outcome than the service provided.
  • Elsewhere in the research, we find lower levels of satisfaction with respect to the Agency's Website. This represents an area of particular concern, given that our surveys were conducted among those engaged enough to have contacted and provided information to the Agency and not among those who may have wished to interact with the Agency but became discouraged because they found the website difficult to access.
  • The Agency may also wish to consider how to ensure enhanced accessibility of its services. When asked whether they were satisfied or dissatisfied that the Agency "interacted with me in a way that accommodated my disability", the level of satisfaction (50%) was nearly as low as satisfaction with the timely resolution of issues (49%). However, views on the accommodation of those with disabilities are linked as much to the outcome of their process as to issues related to accommodation in dealing with the Agency itself: those who say that the process met their objectives were much more likely to express satisfaction that the Agency accommodated their disability, while those who say the process did not meet their objectives were less likely to express satisfaction.

Recommendations Stemming from this Research

  1. Ipsos Reid believes that a 70% level of satisfaction with Agency service overall may be a realistic goal over the next three years (representing a five percentage point increase from the 65% measured this year).
  2. Whether respondents feel that their objectives were met colours their views of the Agency in a variety of ways, but timeliness represents a concrete area where the Agency might improve. Doing so does not necessarily require greater speed as much it requires clear communication and well-managed expectations regarding service standards.
  3. The website should be a particular area of focus given its front-line role in communicating about the Agency and its services.

Recommendations for Future Research

The 2009-2010 and 2010-2011 iterations of research have provided a solid benchmark against which to proceed with future iterations. This research provides a basis upon which to track results over time, and because it was developed to be compatible with other public sector client satisfaction research, in particular the CMT, it also affords the ability to compare results with other public sector institutions where it is possible to do so.

With that said, there are a variety of steps that can be taken to improve the research and make it an even more robust measurement of client satisfaction with the Agency. The points below represent our major recommendations in this area. We have already begun working with the Agency to implement some of these recommendations.

  1. After a thorough review of the 2010-2011 questionnaires, we have several recommendations to improve the questionnaire. In general, we recommend asking questions to all groups using consistent language whenever possible in order to ensure that there is a robust base size for analysis across a maximum number of indicators, and to facilitate the comparison of results across groups.
  2. While the questionnaire was developed to be consistent with other public sector client satisfaction research, most notably the CMT, we have reviewed the CMT again in light of the 2010-2011 questionnaires and have identified several additional lines of inquiry which may be considered.
  3. We recommend measuring the importance with which respondents view the various attributes of service from the Agency in addition to their satisfaction with these attributes. Doing so can enable a grid analysis, which can provide a way to identify areas for improvement that are most likely to increase overall client satisfaction.
  4. We also recommend changing the structure of the survey, so as to bring all six surveys into a single survey with branching logic. This would complete the transition of the survey from its paper-based roots to an online format, and allow it to benefit fully from the scripting advantages offered by online research. It will also address important data consistency issues caused by the dual methodology: online scripting ensures that respondents consistently follow directions (e.g., 'select two…'), where paper cannot; as well, an online format forces a response, where a paper format can allow respondents to overlook a section, meaning a much higher number of 'don't know/refusal' answers for the paper-based surveys. Making this change will also serve to considerably simplify the data deliverables and make it more straightforward for the Agency to make its own comparisons across audiences.
  5. In this same vein, we strongly recommend moving the entire satisfaction research program online (i.e. removing the option for paper-based surveys). This will contribute to increased data quality and may help to increase the response rate among inspection and new licensing clients.
  6. Given the small universe of mediation clients and the importance of this audience, in the future the Agency may wish to consider a mixed methodology approach to gauging the satisfaction of this audience. First, we would suggest that quantitative research with this audience be limited to the crosscutting summary measures, so that their views are still represented in the overall client satisfaction survey results. Then, because the number of potential and actual mediation clients is too small to support statistical analysis on its own, we would propose to use a qualitative approach to better understand the client experiences of the mediation process and desired improvements, enabling these clients to describe and explain their experience more fully than a survey among such a limited number of participants can allow. We would accomplish this through in-depth interviews conducted among a smaller sub-sample of mediation clients. This same recommendation could apply equally well to the travel-related dispute adjudication and non-travel dispute adjudication and multi-party determination clients, for whom the available sample sizes are comparably small.

Background and Objectives

Ipsos Reid was commissioned by the Canadian Transportation Agency (hereafter referred to as the Agency) to conduct client satisfaction research among clients dealing with the Agency in different ways. This research has several objectives:

  • establishing baseline levels of satisfaction with various aspects of service and measuring improvement over time;
  • identifying elements of service in need of improvement in order to be client-centred and responsive; and,
  • connecting the research to government-wide efforts to deliver client-centred services.

Ultimately, this research provides a clear picture of the current state of satisfaction with service delivery by the Agency to its various clients, measures changes year over year and compares the results to other research to the extent possible.

The first iteration took place during the 2009-2010 fiscal year and was conducted among two types of clients (those whose dispute was facilitated by the Agency and those involved in mediation). The 2009-2010 research also included a number of qualitative in-depth interviews with major service providers and senior officials.

This year's research, conducted during the 2010-2011 fiscal year, expanded the pool of respondents to include six types of clients (adding inspection and new licensing, travel-related dispute adjudication, non-travel dispute adjudication and multi-party determination, and inquiries to the types of clients interviewed).

The Agency's research on client satisfaction is directly related to the Government of Canada's management accountability framework on results for Canadians. Most significantly, it will support the Government's initiative on client-centred service delivery, and also lead to better quality information for parliamentarians about the Agency's programs and results.

In undertaking this research, the Agency is responding to the Treasury Board's recommendation in its assessment of the Agency's Management Accountability Framework, which encouraged the Agency to measure client satisfaction using the Common Measurement Tool (CMT), to set service standards and to measure its performance relative to them. The establishment of baseline data on client satisfaction is an important element of the Treasury Board Secretariat's Service Improvement Initiative.

Specifically, this research employs a range of core measures from the CMT, including overall client satisfaction, and satisfaction with timeliness, accessibility of service, fairness, competence, and ease of access to information.

Methodology

Ipsos Reid conducted quantitative survey research among six different target audiences served by the Agency during the 2010-2011 fiscal year. The surveys were conducted primarily online, although a limited number of surveys were administered by paper (and returned to Ipsos Reid for manual data entry).

The six audiences included in this year's research are described in the table below.

Survey TypeTarget Audience
Facilitation Passengers with travel-related complaints processed through facilitation (including persons with disabilities)
Mediation Those who have been involved in mediations
Travel-related dispute adjudication Those with travel-related adjudicated disputes and determinations involving only one party
Non-travel dispute adjudication and multi-party determination Those with both non-travel-related disputes and determinations involving more than one party
Inspection and new licensing New licensing and inspection activity
Inquiries Inquiries and requests for information that are identified as meaningful enough to survey.

Agency clients within each of the six audience types were asked to complete a survey. The table below provides a breakdown of the survey universe (representing the total number of clients within each audience type), the number of online and paper-based surveys administered, and the resulting number of completed surveys and participation rates.

Survey TypeSurvey UniverseOnline Surveys AdministeredPaper Surveys AdministeredTotal Completes (Online/
Paper)
Participation Rate
Facilitation 240 240 0 108 58%
Mediation 22 15 7 10 45%
Travel-related dispute adjudication 25 17 8 12 50%
Non-travel dispute adjudication and multi-party determination 19 14 5 8 42%
Inspection and new licensing 242 153 89 45 19%
Inquiries 248 248 0 47 19%
TOTAL 796 687 109 230 29%

Among the six audiences interviewed, two audiences, inspection and new licensing and inquiries, garnered significantly lower participation rates (both 19%, compared to the overall response rate of 29%). With respect to inquiries respondents, we feel that this has to do with their relatively lower level of engagement with the Agency as people seeking information (as opposed to those engaged in an adjudication or mediation). With respect to inspection and new licensing respondents, we feel that this has to do with the greater number of paper-based surveys administered by this audience. Paper-based surveys tend to be more time-consuming to complete and return, and therefore generate lower rates of response.

Note on Reading this Report

Among the six audiences interviewed, three audiences generated completed surveys in numbers high enough for independent statistical analysis: inspection and new licensing (n=45, when taken together), inquiries (n=45), and facilitation (n=108). The other audiences, drawing from much smaller universes, did not generate large enough samples for independent statistical analysis. As a result, the data from these smaller audiences should be interpreted as directional in nature.

Please note that while respondents answered different questionnaires depending upon the specific circumstances of their interaction with the Agency, several areas of inquiry, based on the Common Measurement Tool, were common to all audiences. This report provides the results of those questions asked in a common fashion across multiple audiences.

Please also note that categories do not always sum to 100% due to rounding.

Detailed Findings

Interacting with the Agency

Respondents were asked several questions pertaining to the nature of their interactions with the Agency; in particular, what type of transportation service was involved, the nature of their interaction with the Agency, and the source of their awareness of the Agency. Please note that the participation rates of many client groups were low as a result of the infrequency of these types of interactions; all reported values must be taken in the context of the number of respondents.

Type of Transportation Service Involved

Respondents were asked what type of transportation service their interaction with the Agency dealt with (or, in the case of those in the inspection and new licensing stream, what type of organization or agency they represented).

On this basis, respondents from travel-related dispute adjudication (100%)Note 2, the facilitation stream (99%), inspections and new licensing (80%) and inquiries (60%) say that they interacted with the Agency as the result of an issue dealing with air transportation or on behalf of an air carrier.

Those interacting with the Agency in a non-travel dispute adjudication and multi-party determination (50%) or those interacting with the Agency in mediation (40%) were more likely to say that their matters dealt with rail transportation.

Nature of Interaction with the Agency

Respondents were asked to describe the nature of their interaction with the Agency. Responses varied widely across the different streams of respondents. Among inspection and new licensing clients, 53% contacted the Agency regarding a new license, and 42% interacted with the Agency with regard to an inspection. Travel-related dispute adjudication and facilitation clients were primarily concerned with baggage (42% and 37%, respectively) and flight disruption (25%, 34%). Non-travel dispute adjudication and multi-party determination respondents were issuing complaints (38%), dealing with the determination of salvage value (25%) or with the entry of foreign vessels (25%). Inquiries customers were concerned with making complaints about a transportation service provider (34%), obtaining information about the Agency's services (23%), or requesting publications (11%). Reported reasons for mediation respondents' complaints include noise or vibration (20%) and the rejection of a one-person-one-fare application (20%).

Source of Awareness of the Agency

Respondents were asked how they became aware of the Agency. Travel-related dispute adjudication (33%), inquiries (38%), and facilitation respondents (59%) most often mention a web search as their source of awareness. High proportions of non-travel dispute adjudication and multi-party determination respondents (40%), inquiries respondents (32%) and mediation respondents (29%, two respondents) say that they already knew of the Agency. Mediation respondents also reported hearing of the Agency from another government department (29%).

Source of Information about the Agency

Overall, the method most often used to obtain information about the Agency is email (72%), followed by the telephone (63%) and the Agency website (56%). Other methods of obtaining information are much less frequently used, including regular mail (18%), fax (9%) and courier (6%).

Fig. 1 - Source of Awareness of the Agency, text version available via the link below.

Figure 1 - Text version: E-mail, telephone and the Agency website are most often used to obtain information about the Agency's processes and services

Inspection and new licensing and mediation respondents are more likely to obtain their information about the Agency by telephone (82% and 71% for telephone, compared to 64% and 57% by email, respectively). Inquiries respondents are as likely to use the Agency website as email (both 88%).

Satisfaction with the Agency

To determine the survey participants' level of satisfaction with their Agency interaction, respondents were asked to rate their satisfaction with the overall quality of service provided by the Agency, and were also asked to comment upon individual attributes of customer service. These attributes address the responsiveness of the process, the accessibility of information about the Agency, and the experience of communicating with Agency staff.

Overall Satisfaction with the Agency

Respondents were asked how satisfied they were with the overall level of service they were provided by the Agency on a scale of one to five, where one means very dissatisfied and five means very satisfied. On this basis, two thirds of respondents (65%) report satisfaction with the service that they received from the Agency (rating as a four or five on the scale). One in five (21%) report dissatisfaction with the quality of service that they received (rating as a one or two on the scale). Just over one in ten (13%) express a neutral view (rating as a 3 on the scale).

Fig. 2 - Overall Satisfaction with the Agency, text version available via the link below.

Figure 2 - Text version: Regardless of the nature of their interaction, two in three clients are satisfied with the overall quality of service provided by the Agency

Levels of satisfaction vary by the particular interaction with the Agency, with the highest level of satisfaction found among inspection and new licensing respondents (84%) and the lowest level found among travel-related dispute adjudication respondents (33%).

Please note that samples of the travel-related dispute adjudication, non-travel dispute adjudication and multi-party determination, and mediation streams are too small to afford statistical reliability, and as such, these results should be interpreted as directional in nature.

Reasons for Satisfaction with the Agency

Respondents satisfied with the Agency were asked to identify the main reason for their satisfaction with the service they received. These respondents most often say that they are satisfied because the staff were professional or courteous (27%), helpful or informative (27%), or because their issue had been addressed or resolved in a timely manner (25%). An additional one in six (16%) say it was because their issue had a successful outcome or was resolved.

Fig. 3 - Reasons for Satisfaction, text version available via the link below.

Figure 3 - Text version: Reasons for satisfaction with the services provided by the Agency

Reasons for Dissatisfaction with the Agency

Those who said that they were dissatisfied with the service they received most often say that they are dissatisfied because of the outcome of their issue or because the issue was not resolved (46%), because they received poor service (21%), or because the service was slow (15%). Others say that they are dissatisfied because they feel the Agency needs more power or should be able to help with more issues (16%). One in ten say that they are dissatisfied because they feel that the Agency is biased towards the transportation companies (10%), or because they feel that the Agency makes unfair decisions (8%).

Fig. 4 - Reasons for Dissatisfaction, text version available via the link below.

Figure 4 - Text version: Reasons for dissatisfaction with the services provided by the Agency

Perception of Agency Process Meeting Respondents' Objectives

Respondents were asked to rate the extent to which they felt that their interaction with the Agency met their objectives. Just over two in five respondents (44%) say that the process met their objectives either fully (28%) or largely (16%). An additional 14% say the process met their objectives to an acceptable degree. Two in five (39%) say that the process met their objectives only a bit (11%) or not at all (28%). Of interest, the proportion saying that their objectives were met fully (28%) is identical to the proportion saying that their objectives were not met at all (28%).

Fig. 5 - Process Meeting Respondents' Objectives, text version available via the link below.

Figure 5 - Text version: As many respondents report that the process fully met their objectives as say it did not at all

Travel-related dispute adjudication respondents (50%) most often say that the process only met their objectives a bit or not at all. Facilitation respondents are split between those who say that the process fully or largely met their objectives (44%) and those who say that the process only met their objectives a bit or not at all (40%). Inquiries respondents were more likely to say that their interaction met their objectives fully or largely (47%) than to say that their objectives were met only a bit or not at all (34%). Mediation respondents were also more likely to say that their objectives were met fully or largely (50%) than to say that they were met only a bit or not at all (30%).

Please note: inspection and new licensing respondents were not asked this question and that samples of the travel-related dispute adjudication, non-travel dispute adjudication and multi-party determination, and mediation streams are too small to afford statistical reliability, and as such, these results should be interpreted as directional in nature.

Factors Affecting Satisfaction with Agency Interaction

Satisfaction with the overall quality of service provided by the Agency is closely tied to whether or not the objectives of the respondents' interaction were met. Satisfaction among those who say that their objectives were fully or largely met is near total (95%), while satisfaction among those whose objectives were met to an acceptable degree declines to 52%. Among those who say that their objectives were met only a bit or not at all, satisfaction with the service they received declines to 21%.

Fig. 6 - 'Level of satisfaction / objectives being met', text version available via the link below.

Figure 6 - Text version: Satisfaction with overall quality of service is closely tied to the perception that their objectives were met

This trend is consistent among all Agency audiences; those who felt that their objectives had been met were more likely to report satisfaction with the Agency overall.

Satisfaction with the overall quality of service experienced by respondents is also connected with perceptions about the time it took for the Agency to either acknowledge or to resolve their issue. Three in four respondents (74%) who were satisfied with the time it took to acknowledge their issue were also satisfied with the overall service they received from the Agency. By comparison, 25% of those who were dissatisfied with the time it took to acknowledge their issue say that they were satisfied with the overall service provided by the Agency. Similarly, 87% of those satisfied with the time it took to resolve their complaint were also satisfied with the overall service provided by the Agency. Among those dissatisfied with the time it took to resolve their issue, overall satisfaction with the service they received from the Agency declines to 32%.

Fig. 7 - 'Level of satisfaction / timeliness', text version available via the link below.

Figure 7 - Text version: Clients who are satisfied with timeliness are much more likely than those who are not to be satisfied with overall quality of service

This trend is consistent among those Agency audiences for which statistical analyses can be made; for both inquiries and facilitation audiences, those respondents who reported satisfaction with the time to acknowledge and the time to resolve their interaction were more likely to report satisfaction with the Agency overall.

Satisfaction with Attributes of Agency Service

Respondents were asked to indicate their level of satisfaction with various aspects of service from the Agency on a scale of one to five, where one means very dissatisfied and five means very satisfied. On this basis, majorities expressed satisfaction with each of the indicators tested but one: the time it took to resolve the issue (49% expressed satisfaction on this item, rating as 4 or 5 on the scale, while 33% expressed dissatisfaction, rating as a 1 or 2). The highest level of satisfaction was attributed to courteousness (79%), while all the other indicators tested, apart from the time it took to resolve the issue, fell within a narrow band of overall satisfaction (ratings of 4 or 5 on the scale), ranging from 60% for the time it took to acknowledge the issue to 69% for how easy it was to deal with the Agency.

Fig. 8 - Satisfaction with Attributes of Agency Service, text version available via the link below.

Figure 8 - Text version: Most aspects of service receive at least two thirds satisfaction; the variety of means of contact, the time to acknowledge, and especially resolution time, are less satisfactory

Satisfaction among facilitation respondents tends to be higher across the tested indicators, with the exception of the time it took to acknowledge and resolve the complaint (for which the results among facilitation respondents were close to the average). Satisfaction among inquiries respondents tends to be lower across the tested indicators.

The proportions who were very satisfied (rating as a 5 on the scale) varied more dramatically, with notably lower levels saying they were very satisfied with respect to the time to resolve issues (27%), the accuracy of information provided (35%), the clarity and ease of understanding information provided by the Agency (38%), and the time required to acknowledge issues (39%).

Fig. 9 - 'Very Satisfied' with Agency Service, text version available via the link below.

Figure 9 - Text version: While the levels of overall satisfaction are fairly consistent, the proportions who are very satisfied differ widely

Suggestions for Improvement of Agency Interactions

When asked to identify two aspects of service upon which the Agency could improve, they identify the clarity and ease of understanding of information as the area most in need of improvement among those who report overall satisfaction with the Agency's service (48%). Those reporting overall satisfaction with the service experienced with the Agency's staff also identify resolution time as an area in need of improvement (45%).

Fig. 10 - Needs Improvement (Interactions) - Satisfied, text version available via the link below.

Figure 10 - Text version: Those satisfied with overall service of Agency identify the clarity of the information and resolution time as areas in need of improvement

Those who report overall dissatisfaction with the Agency's service identify the accuracy of information as the area most in need of improvement (56%), followed by resolution time (40%) and acknowledgement time (38%).

Fig. 11 - Needs Improvement (Interactions) - Dissatisfied, text version available via the link below

Figure 11 - Text version: Those dissatisfied with overall service of Agency identify the accuracy of the information as the major area in need of improvement

Satisfaction with Aspects of Experience with Agency

Respondents were asked to rate their satisfaction with several specific aspects of their experience with service from the Agency. Satisfaction with the aspects of experience tested range from a high of 89% with respect to communications in the official language of the respondent's choice to a low of 50% with respect to service that accommodated respondent's disability (excluding respondents to whom this question did not apply).

Between these, we find that seven in ten respondents believe that they were treated fairly (70%) and that the Agency's forms were easy to complete (69%). Two thirds of respondents are satisfied with the variety of means offered to interact with the Agency staff (68%), and the clarity and transparency of the Agency's processes (67%). Lower levels of satisfaction were attributed to the Agency's website; in particular, the ease of finding information on the website (52%) and the perception that the website contained the information respondents' needed (57%).

Fig. 12 - Satisfaction with Aspects of Experience, text version available via the link below.

Figure 12 - Text version: Clients are broadly satisfied with aspects of their experience with the Agency; the Agency's accommodation of those with disabilities and the website are areas of lower satisfaction, as is staff response time

It is worth noting that the views on the accommodation of those with disabilities may be linked as much to the outcome of their process as to issues related to accommodation in dealing with the Agency itself: those who say that the process met their objectives were much more likely to express satisfaction that the Agency accommodated their disability, while those who say that the process did not meet their objectives were less likely to express satisfaction.

Suggestions for Improvement of Agency Service Attributes

When asked to identify two aspects of service upon which the Agency could improve, both the ease of finding information on the Agency's website (36%) and the ease of understanding that information (30%) are identified as areas most in need of improvement among those who report overall satisfaction with the Agency's service. Those reporting overall satisfaction with the service experienced with the Agency's staff also identify a lack of required information on the Agency website as an area in need of improvement (27%).

Fig. 13 - Needs Improvement (Service) - Satisfied, text version available via the link below.

Figure 13 - Text version: Those satisfied with overall service identify the functionality of website, clarity of information, and response time as areas in need of improvement

Those who report overall dissatisfaction with the Agency's service identify response time (42%) and the transparency of information (40%) as areas most in need of improvement, followed by the fairness of the process (36%).

Fig. 14 - Needs Improvement (Service) - Dissatisfied, text version available via the link below.

Figure 14 - Text version: Those dissatisfied with overall service identify the response time, transparency of process, and fairness as areas in need of improvement

Timeliness of Acknowledgement from the Agency

Respondents were asked how long it took to receive an acknowledgement from the Agency. A third of respondents received an acknowledgement within four days (35%). A quarter of respondents (28%) report that their acknowledgement was received between five and nine days after their initial contact. When asked what they considered an acceptable time to receive an acknowledgement from the Agency, nearly two in five (37%) expected an acknowledgement within four days, and another two in five (39%) expected the acknowledgement within five to nine days.

Fig. 15 - Timeliness of Acknowledgement from the Agency, text version available via the link below.

Figure 15 - Text version: Timeliness of acknowledgement from the Agency

With respect to the time it took to receive an acknowledgement from the Agency, we find a spike in the proportion who says it took 30 days or more (17%). This could suggest a different understanding of what it means to "acknowledge" an issue: perhaps these respondents interpret it to indicate the receipt of a written notice from the Agency. As such, it may be possible to address this in the future with specific wording to define the meaning of "acknowledge" in this context.

Additional comments

When asked to provide additional comments related to their level of satisfaction with Agency services, those satisfied with the service they received mention gratitude for the help they have received (15%), and that their contact at the Agency was very helpful or effective (7%). Half of satisfied respondents did not add further comment (49%).

Fig. 16 - Additional Comments - Satisfied, text version available via the link below.

Figure 16: Additional comments from those satisfied with the services provided by the Agency

Those who report overall dissatisfaction with the Agency's service comment on their dissatisfaction with or dislike of the Agency (13%), as well as mention that their issue was not resolved (10%). Others add that the Agency requires a larger scope of action (10%), and 8% call for the dissolution of the Agency. One in five dissatisfied respondents (19%) provided no further comment.

Fig. 17 - Additional Comments - Dissatisfied, text version available via the link below.

Figure 17: Additional comments from those dissatisfied with the services provided by the Agency

Appendices to this report can be obtained by contacting us.

Date modified: