2015-2016 Client Satisfaction Survey Summary
Table of Contents
This report details the key findings and conclusions which follow from the 2015-2016 client satisfaction research among clients of the Canadian Transportation Agency. Separate phases of quantitative and qualitative research were conducted.
The quantitative research consisted of 271 surveys conducted among five client types: those who went through an inspection or new licensing process, those involved in a travel-related dispute adjudication, those involved in a non-travel related dispute adjudication and multi-party determination, those whose disputes were facilitated by the Agency and those involved in a mediation.Footnote 1
Respondents to the online survey from adjudication, mediation, and non-travel dispute adjudication were asked at the end of the survey if they would be willing to be contacted by EKOS for a follow-up interview. Eleven of these respondents indicated they would be willing to be interviewed, and a total of eight qualitative interviews were conducted by telephone with these respondents.
In addition, a total of 22 qualitative interviews were conducted with stakeholders representing the railway, air, and marine industries, as well as shipping associations and associations representing persons with accessibility issues.
This summary presents the key findings ensuing from all research components.
The contract amount of this research is $60,247.92 (including HST).
Satisfaction with the Agency
Clients were asked how satisfied they were with the overall quality of service provided by the Agency, regardless of the outcome of their interaction with the Agency. Two-thirds of respondents (69 per cent) report satisfaction with the service that they received from the Agency (up three per cent since 2014/2015).
Clients were also asked to rate the extent to which they felt that their interaction with the Agency met their objectives. Six in ten (61 per cent) say that the process met their objectives either fully (44 per cent) or largely (17 per cent). These findings represent a net improvement of nine points over 2014/2015, including a ten-point jump in in the percentage of respondents who say their objectives were fully met.
As in previous years, satisfaction with the overall quality of service provided by the Agency is closely tied to whether or not the objectives of the respondents’ interaction were met. Among those who say that their objectives were met, nearly all report satisfaction with the overall quality of service (97 per cent). Among those who say that their objectives were met only a bit or not at all, satisfaction with the service they received declines to just 13 per cent.
Clients were asked how long it took to receive an acknowledgement from the Agency, and what timeframe would be acceptable to receive an acknowledgement. Results reveal that about four in ten respondents (37 per cent) say they received an acknowledgement from the Agency within five days, and an additional 16 per cent indicated they received an acknowledgement within five to nine days. Results further reveal that one in five clients indicated they had to wait more than 30 days for an acknowledgement (17 per cent), and very few (two per cent) find it acceptable to wait this long.
Respondents were also asked how long it took from the time they submitted their issue to the Agency to the resolution of the issue, and also what they consider to be an acceptable time to resolve their issue. One in five (19 per cent) said it took between one and thirty days for their issue to be resolved, however, nearly half (46 per cent) expect the Agency to resolve their issue in 30 days or less. Tracking reveals a significant decline in the proportion who indicated it took the Agency more than 121 days to resolve their issue (from 43 per cent in 2014/2015 to 24 per cent currently).
Views on Agency Interactions
Results reveal that all of the aspects of service examined are seen as important by a majority of clients, although the knowledge and competence of staff and the accuracy of the information provided are seen as particularly important (80 per cent and 78 per cent of respondents, respectively, feel these aspects are very important). The time it takes to resolve the matter (57 per cent), and the time it takes to acknowledge their issue (56 per cent) are seen as relatively less important issues (although most clients still see these as important service aspects).
Respondents were also asked to rate their satisfaction with these various aspects of service. Results reveal that the majority of clients are at least somewhat satisfied across virtually all of the aspects examined. Respondents are most satisfied with the courtesy of staff (79 per cent very or somewhat satisfied), the ease of dealing with staff (76 per cent), and the knowledge and competence of staff (74 per cent). The time required to receive acknowledgment (58 per cent) and the time it took to resolve the matter (54 per cent) garnered the lowest levels of satisfaction. Tracking reveals that satisfaction levels have remained largely stable over the past year, however, satisfaction with the Agency’s two weakest areas – the time required to receive acknowledgement and the time it took to resolve the matter – is up six points in each case.
We conducted a gap analysis for each of these service attributes to help identify the areas most in need of attention. In conducting this analysis, we subtracted the proportion who indicated very satisfied across each of the attributes from the proportion who indicated the attribute was very important. The resulting number reflects areas where gaps exist between the priority of the issue relative to the satisfaction with this issue. Results from this analysis reveal that gaps are most pronounced in terms of accuracy of information, the clarity of information, and the knowledge and competence of staff.
Respondents were also asked to indicate how important they found several other aspects of their experience with service from the Agency. Results reveal that all aspects are seen as at least somewhat important by roughly eight in ten or more clients. Focusing on the “very important” responses only, results reveal that clients place the most importance on the fairness of treatment they receive (81 per cent see this as very important), having all questions answered (78 per cent), and being informed of everything they need to do (77 per cent).
Clients were also asked whether or not they agreed with a number of statements about these same aspects of service. Across all of the statements, at least six in ten respondents strongly or somewhat agreed (i.e., expressed satisfaction). Respondents most often agree that the contact they had with the Agency was in the official language of their choice (88 per cent agree with this statement). Three in four agree that they were informed of everything they had to do in order for their matter to be dealt with by the Agency (78 per cent) and that the Agency let them know what they could do in dealing with their matter (73 per cent).
As with the previous service attributes questions, we conducted a gap analysis for these service aspects to help identify the areas most in need of attention. Results from this analysis reveal that gaps are most pronounced in terms having all questions answered, fairness of treatment, being informed of everything the Agency could do, and being informed of everything the client had to do - suggesting that these are the areas most in need of attention from the Agency.
Views on Agency Process
Respondents were asked whether or not they agreed with a number of statements about Agency processes. Three in four respondents agree that the Agency made clear the information required for submission (74 per cent), while two-thirds felt that the process was carried out in a professional manner (66 per cent) and that the Agency’s process was clearly explained (65 per cent). Only about half, however, feel that the process was impartial (51 per cent), or that they had enough opportunity to respond to the other party’s point of view (48 per cent). Comparing these results to those found in previous years reveals a steady and continuing decrease in the proportion of clients who feel that the process was conducted in professional manner (from a high of 84 per cent in 2011/2012 to 66 per cent today).
Interacting with the Agency
Respondents were also asked which method they used to get information about the Agency. E-mail (76 per cent) is identified most often as the method used to obtain information, followed by the Agency’s website (62 per cent). Tracking reveals that use of the Agency website has decreased significantly as a source of information about the Agency since 2014/2015.
Clients were then asked which method they would most prefer to use to get information about the Agency. Just under half indicated e-mail (46 per cent, up four points since 2014/2015), while four in ten prefer the Agency website (37 per cent, down seven points since 2014/2015). Only about one in seven indicate a preference for telephone (15 per cent, although this is up six points since 2014/2015).
Clients were also asked a number of questions about the Agency’s website. Respondents were asked if they had ever visited the Agency’s website. Over nine in ten (93 per cent) indicate that they have visited the Agency’s website, identical to last year’s results. Among those who visited this website, one in four (26 per cent) said they had visited the Agency’s website within the past three months, and one in three (33 per cent) visited the website between three and six months ago. Comparing these figures to last year’s results reveals a 13-point decrease in the percentage of clients who have visited the website in the six months prior to the survey.
The survey also included a series of questions examining satisfaction with the Agency’s website. Roughly six in ten indicate that the information on the website was easy to understand (57 per cent), and that the website had the information they were looking for (59 per cent). About half indicated that it was easy to find information they needed on the website (52 per cent). These figures have remained largely stable since 2014/2015. Results further reveal that two-thirds feel the Agency’s website is accessible (65 per cent – unchanged over the past year), and that the information on the website is useful (59 per cent – down four percentage points over the past year). A little over half feel that the Agency’s website is easy to navigate (56 per cent, although this is up four percentage points since last year).
Recommendations – Quantitative Research
Overall, satisfaction levels are good and generally improving, but the survey suggests some areas in need of attention.
- Provision of clear, accurate information and knowledge of staff should be given high priority (identified as areas in need of attention).
- Website receives fairly positive ratings and there has been improvement in terms of ease of navigation; however, current and preferred use of the website is down over the past year, and perceived usefulness of information on the website is down as well.
Key Findings from Small Bases Interviews
We conducted follow-up interviews with select respondents to add texture to the survey findings. Many of those that felt that the process was not fair generally cited procedural issues. Participants cited a lack of instructions for proceeding with more complex cases, a lack of opportunity to revise or modify submissions, and a feeling that the process was overwhelming. Others felt that the process lacked fairness because it was too complex for an “average person”, with no legal training, to argue against one or more “high priced” corporate lawyers.
Most interviewees felt that the process was not very efficient because they thought that it took longer than it should have, in some cases more than a year. Some felt that while they understood the “outcome” of their case, they were not able to clearly understand on what basis the ruling was made.
Generally, participants felt that the process required a lot of time and energy. Proposed enhancements to the process include:
- Simplifying processes and making it easier for people with limited time to access;
- Where possible, avoiding legal jargon and complex language;
- Greater transparency throughout the process; and
- Clarifying procedures and documentation requirements, particularly for complex complaints.
Overall assessment of service quality was fairly positive. Agency case officers were praised for being “nice”, and in some cases for their ability to answer questions and provide some assistance. However, others felt that there should be more communication from staff.
Recommendations – Small Bases Interviews
Results from these small bases interviews suggest a number of areas to improve service, including:
- Better communication with complainants about the processes and outcomes;
- Focusing processes on the needs of citizens instead of companies;
- Reducing the length of the process and lightening the burden on complainants;
- Providing more resources and explaining processes; and
- Ensuring language and documentation is simple and comprehensible by everyone.
Key Findings from Stakeholder Interviews
Satisfaction with Agency
As discovered during previous consultations, stakeholders are generally satisfied with the opportunities for dialogue offered by the Agency. Many stakeholders describe the Agency as responsive and transparent in its communications with them. The one common critique in this area is that the Agency is not always proactive in communicating with stakeholders (those representing persons with accessibility issues were particularly likely to feel this way).
As in previous consultations, stakeholders view Agency staff in largely positive terms (e.g., “easy rapport,” “professional,” “responsive”). A few stakeholders attributed improved responsiveness of the Agency to an increased staffing level. However, some also felt that the new staff did result in a learning curve for both staff and stakeholders.
Dispute Resolution Process
Facilitation is most commonly relied on by airline stakeholders. The process is seen as fair, reasonable, and efficient.
Mediation is seen in positive terms by many stakeholders as an alternative when facilitation fails. As was found during previous consultations, some stakeholders expressed the view that mediators were not always active or involved enough. Others, however, mentioned that this had improved of late, with mediators becoming more engaged in the process.
Arbitration was seen as an important remedy, particularly by shipping stakeholders. While the process is seen as fair and effective overall, as was seen in previous consultations, some stakeholders felt that the arbitrators sometimes did not seem to know the issues well enough to be effective.
The Adjudication process is generally seen as fairly administered by the Agency. However, it is also seen as the most drawn out and expensive of the dispute resolution options. A few expressed frustration with the relatively short time frame to respond; however, these stakeholders also felt that as long as the short timeframes are adhered to by both parties, the timelines would be considered fair.
Agency’s Role in the Accessibility of the Federal Transportation System
Stakeholders representing persons with accessibility issues view the role of the Agency in dealing with their issues as limited and “complaint-driven”. They feel the Agency should have a wider scope to investigate and enforce accessibility regulations.
As mentioned earlier, these stakeholders were least satisfied with communication from the Agency and indicated they would like more proactive communication from the Agency (a few indicated that the Accessibility Advisory Committee has not met in some years).
Recommendations – Stakeholder Interviews
- Consistent with earlier consultations, stakeholder interviews suggest that efforts by the Agency to be more proactive in its communications would be well received, particularly among those representing persons with accessibility issues.
- Consistent with quantitative findings, some stakeholders pointed to a lack of knowledge and understanding among some Agency staff (perhaps due to new staff).