Scale Development and Inferential Analysis of Service Quality in Higher Education:  A Sequential Exploratory Mixed-Method Study

Scale Development and Inferential Analysis of Service Quality in Higher Education:  A Sequential Exploratory Mixed-Method Study

Gersha Pierre PhD

The University of the Southern Caribbean

DOI: https://dx.doi.org/10.47772/IJRISS.2024.803130

Received: 14 February 2024; Revised: 07 March 2024; Accepted: 12 March 2024; Published: 14 April 2024

ABSTRACT

In response to the consumer-oriented shift in higher education, institutions are embracing technology and flexible delivery to enhance services. This is critical for student attraction, retention, and academic success. Consequently, the focus on service quality and student satisfaction has grown. This study investigates service quality expectations and perceptions of students in a local Caribbean -based tertiary institution. A novel scale, the “Service Quality Scale for Higher Education (SQSHE),” is developed using a sequential-exploratory mixed-method approach. The SERVQUAL-driven Disconfirmation framework informs this scale, measuring the gap between expected and perceived service encounters. Three focus group interviews generate a 53-item pool, refined to a 23-item scale with four dimensions: student well-being, service efficiency, social interaction, and value co-creation. The SQSHE demonstrates reliability, stability, and internal consistency. Data from 179 participants via an online questionnaire reveal four service quality dimensions, with two displaying significant negative gap scores. This culturally relevant four-factor scale contributes significantly to higher education research. By addressing student expectations, institutions can enhance strategies to elevate satisfaction and success rates.

Background

Higher education is a fast-growing and competitive service industry. Whelan (2016) notes the zeal and desperation on the part of some public tertiary institutions to make sure they maintain their student numbers given that state funding is often closely tied to enrolments.

Higher education students are exhibiting more of a consumer- oriented approach to their education. (Bunce et al., 2017). They are looking for access to quality services and education across new technologies.The data suggests that when students feel satisfied with their tertiary experiences, satisfied students whose expectations have been met, may “bring continuous advantages for their institution through word-of-mouth communication” (Farahmandian et al., 2013, p.65). How can administrators measure students’ satisfaction with the services they receive?  What are the key indicators of service quality and which scale is most valid and reliable as well as culturally relevant? This study seeks to describe and highlight the methodology utilised to explore the dimensions of students’ service quality expectations and perceptions at a local tertiary institution and develop a contextually relevant, valid and reliable scale to measure these key dimensions of service quality.

Service Quality

When the concepts of quality,excellence  and service are combined, the concept of service quality emerges as the gap between customers’ expectations and perceptions (Zeithaml, Berry, & Parasuraman, 1990). It can be defined as ‘an attitude developed over all previous encounters with a service firm’ (Clow, Kurtz, Ozment & Ong, 1997, p. 232). Lewis and Booms (1983), cited in Lewis and Mitchell, (1990) believe that service quality is a measure of how well the service level delivered matches customers’ expectations.  As Nadiri et al. (2009, p. 525, cited by Teeroovengadum et al., 2016) posits, “if service quality is to be improved, it must be reliably assessed and measured. Therefore, institutions must employ reliable and valid instruments to measure traditional indicators of excellence but must also utilize effective instruments to measure service quality from the students’ perspective.”

There are several well-established models that have been developed and utilized to measure service quality. Among them is the SERVQUAL instrument which measures the difference between what is expected from a service encounter and the perception of the actual service encounter (Parasuraman et al., 1988). The authors named this the disconfirmation paradigm and operationalized it as:

Service Quality (Q) = Perception (P) – Expectation (E) (Tan & Kek, 2004).

Administrators of higher educational institutions realize that when making the major decision of selecting a university, students look for indications that the institution offers excellent service quality (Donaldson & McNicholas, 2004), and the positive perceptions of service quality can provide student satisfaction. Satisfied students may then attract new students by engaging in positive word-of-mouth communication However, based on the examination of current research, there is a lack of enquiry into defining the dimensions of service quality as it pertains to the Caribbean context. In addition, there is a lack of culturally valid and reliable instruments that assesses service quality among tertiary education students.

METHODOLOGY

Design

Anexploratory sequential design that was quantitative-driven (Morse, 1991) was utilized in this study to explore the concept of service quality among the students at a local university, and develop a new scale. For this study, the goal was to explore the key dimensions of service quality and then allow the dimensions discovered to guide the development of a new instrument based in part on the SERVQUAL instrument, which would measure the phenomenon of service quality on a much larger scale. This would require a sequential approach, collecting qualitative data from a small group of participants through focus group interviews in the first phase and using the findings generated from their responses to produce a quantitative instrument that was applied to a larger sample. In this study the following research questions were addressed:

Research Question #1: How do students at a local university define the determinants of service quality expectations and perceptions?

Research Question #2: What are the distinct factors of service quality and which scale items best represent these factors?

Figure 1 shows the key aspects and stages of this mixed-method research design.

Figure 1. Exploratory Sequential Mixed Method Diagrammatic Illustration

Development of the Instrument

In the first phase of the study, based on the qualitative research objective a list of interview questions was prepared to guide each or three focus group interview sessions. As far as was possible, identical questions were used for each interview session. The development of the questions was based on several sources in the literature, including other qualitative studies that also explored service quality in higher education. Generally, the questions focused on participants’ expectations and perceptions with academic and non-academic aspects of their tertiary experience at the institution.

Appendix A provides a copy of the Interview protocol for each Focus Group Meeting including the list of questions used to guide the discussions.

Qualitative Sampling Method

The homogeneous purposive sampling technique was primarily used in the qualitative (QUAL) phase of this study. In other words, the sample of 18 students shared similar characteristics to the population of the institution, with the required selection criteria: male or female, over 18 years, and English as their first language.

For this study there were three focus groups. Guided by the work of Asquith (1997) the first focus group consisted of eight participants, while the second and third consisted of four and five participants each.

The first focus group meeting was held with a mixed group of eight students comprising   four local Trinidad and Tobago students and four overseas students. To deepen the rigor of the qualitative component of the study, seeking data saturation, two additional focus group interviews were conducted over a period of one year. For the second and third focus group interviews, the students were grouped homogeneously.  Focus group 2 consisted of four oversees students, and the third focus group consisted of five local students.

The use of multiple focus groups went a long way towards data saturation and the development of a typology of motives for the study of students from within the Caribbean region.  It was felt that Caribbean students’ views varied significantly from those highlighted by students based in Europe, Asia and the more developed countries from which previous research on service quality emanated.

Qualitative Data Analysis

Table 1 presentsa summary of the qualitative data analysis procedures and products that were implemented for this study.

Table 1

Following the transcribing of the focus group meetings, the NVIVO version 8.2 software was utilized to conduct qualitative data analysis through coding, thematic development, and thematic analysis.

Qualitative Content Validity and Reliability Measures

This study utilized appropriate strategies to ensure validity and reliability of the research, during both the qualitative and the quantitative stages of the study. For the qualitative stage, descriptive validity was enhanced during each focus group interview first by the recording of interviews with two different digital audio recorders and secondly by conducting verbatim transcription of each session. To enhance reliability, the researcher followed identical interview protocols for each session, using the same open-ended questions, member checks at multiple points throughout study, triangulation with several types of data and peer and colleague examination of the data during analysis to help confirm findings. At the end of each focus group interview, protocols to ensure interpretive validity (Hayashi et al., 2019) included a discussion meeting between the researcher and the rapporteurs.  Also, reflective journaling of non-verbal responses, body language and other useful observations was done by the researcher.

Transitioning from the Qualitative to the Quantitative Data Collection Phase

In this sequential-exploratory mixed-method study, the transition from the qualitative phase of the research to the quantitative phase occurred as the focus group interview responses by students, along with extensive literature review by the researcher, were utilized to develop items for the new scale. Initial open coding on analyses of the data items resulted in 8 coded themes which were divided into three categories as follows:

• Category 1: Themes from the SERVQUAL dimensions for measuring service quality (the five themes of tangibility, reliability, responsiveness, assurance and empathy.
• Category 2 – Themes gleaned from the literature.
• Category 3 –New themes generated by this researcher based on student feedback

Quantitative Phase

Figure 2 presents the steps followed for the scale development, data collection and preliminary construct validation phases of the study to address the first quantitative research question.

Figure 2. Steps followed for Scale Development and Validation

Quantitative Instrument (Scale) Development

The scale development process was guided by the three-phase, nine-step approach advocated by Boateng et al. (2018). Phase 1, called Item Development, consisted of two steps:  identification of the domain and item generation and secondly, consideration of content validation. The second phase, Scale Development, focused on turning individual items into a harmonious and measuring construct, and consisted of four steps:  pre-testing questions, sampling and survey administration, extraction of latent factors and item reduction. The third and final phase, Scale Evaluation, consisted of three steps: tests of dimensionality, tests of reliability, and tests of validity.

Identification of the Domain and Item Generation

First, the emerging themes that were generated during the qualitative phase were used to develop new survey items. The five themes of the SERVQUAL instrument were used as a conceptual underpinning or starting point for identification of the domains. Utilizing a combined deductive and inductive approach, new items were added to each of the original five SERVQUAL themes, along with items drawn from the emerging themes, based on the analysis of the focus group interviews and supported by a review of the current literature on dimensions of service quality in higher education (Pereda et al., 2007).

During the item generation stage, at least two items were created and added to each of the original five SERVQUAL dimensions, along with three additional dimensions. The three new dimensions added to the original five from SERVQUAL were Ethics, Social Benefits and Cocreation.

Following the format used by Cerri (2012), the original 22 item SERVQUAL scale was significantly altered in terms of names of factors, number of items, and reformulation of item wording. Appendix B presents a Listing of the new scale (with newly created items highlighted in bold). In addition,Appendix C – Joint Display #1, displays a completed table showing all the new items developed inductively through analysis of focus group student response data.  Table 2 presents a breakdown of the eight dimensions of service quality that comprised the new scale that was administered to the sample.

Table 2. Summary of Structure of New Scale with Eight Dimensions of Service Quality

Scale Reliability and Content validity

During the item development phase, five expert judges were engaged in the process of “theoretical analysis” (Morgado et al., 2017 p. 2) of the items. Cognitive interviews were conducted utilizing a small group of three students and three faculty as an additional content validity step to collect verbal information about the response process. Additionally, the criterion of a minimum of three items to establish a subscale was followed in the item development phase (Anderson & Rubin, 1956, Comrey,1988 cited in Hassad, 2007)

Quantitative Sampling Method and Questionnaire Administration

Purposive sampling was used to select participants for the quantitative data collection phase, following a similar procedure to that used at the qualitative phase. The survey remained on the Survey Monkey platform for three weeks. Three reminder prompts (one per week) were sent to the sample group to maximize their awareness of the survey’s existence and increase participation rates. A total of 279 responses were received, representing a percentage response rate of twenty-three percent.

Quantitative Data Analysis

Item Reduction

Using the SPSS software, item-total correlation was applied to determine the correlations between scale items as well as identify potential factors for the new scale. Initial inspection of the correlation results helped ensure that only parsimonious, functional, and internally consistent items were ultimately included in the new scale (Boateng et al., 2018). For this research, the estimation of item total correlation greater that 0.3 was used as the rule of thumb for item reduction (Nunnally & Bernstein, 1994).

Variable Reduction Technique Selected

Following the initial inspection, the factorability of the scale items was examined, utilizing the Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett’s test of sphericity. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy was used to indicate the proportion of variance in the variables that might be caused by underlying factors. Then the process of variable reduction was implemented.

The literature identifies Principal Components Analysis (PCA) and Exploratory Factor Analysis (EFA)as variable reduction techniques. For this study the goal was to explain the variance and determine the optimal number of factors, or domains, which fit a set of closely related items. Therefore, the EFA method of variable reduction was selected for this study and Factor Extraction was implemented.

Extraction of Latent Factors

Exploratory Factor Analysis was conducted using the Mplus Version 8.3 software, with the maximum likelihood as its default factor extraction method.  Fabrigar et al. (1999) highlight maximum likelihood as the best choice for factor extraction when the data is normally distributed as was the case for this study, in addition to its compatibility with the goodness of fit techniques of deriving the factors.

Determining the Number of Factors to Retain

Goodness-of-fit measures used for factor selection and their suggested cutoffs were based on the “EFA model fit, evaluated by the standards proposed by Hu and Bentler (1999) and Brown (2015) : RMSEA (≤0.06, 90% CI ≤0.06), SRMR (≤0.08), CFI (≥0.95), TLI (≥0.95), and the chi-square/df ratio less than 3 (Kline, 2016 )”. (Kyriazos et al., 2018, p. 1154).

Next, an initial extraction was performed to identify any irregularities in the data. Items were dropped if a loading score of less than   0.5 was reached and constructs contained less than three items.

Factor Rotation

According to Fabrigar et al. (1999) for EFA models with more than one factor, there is no single unique solution but rather there could possibly be an infinite number of different orientations of the factors that may explain the data equally well. The researcher must therefore seek to select one solution by applying the principle of simple structure (Thurstone, 1947) with each item loading highly on only one factor.  For this research, by default, the factors were rotated with the oblique GEOMIN factor rotation on Mplus. The researcher defined the number of factors (from-to) in the modelling step and Mplus computed all necessary factor models in parallel. After factor rotation, additional factor extraction, and goodness -of -fit measures were applied for final factor selection, the data analysis protocols resulted in the most appropriate number of latent constructs (common factors), comprising the domain of service quality in this context.

Interpretation and Naming of the Factors (Subscales)

At the conclusion of the oblique factor rotation stage the remaining groups of variables were carefully examined with the aim of understanding the underlying construct that bound the items together. Each factor was renamed to more appropriately reflect the nature of the items grouped within, with the application of the factor pattern matrix coefficients (Field, 2000; Hair et al., 1998; Russell, 2002; Stevens, 1992, as cited in Hassan (2007), previously identified marker items, and the general standard of at least three variables per factor (Anderson & Rubin, 1956; Comrey, 1988, as cited in Hassan, 2007)).

Scale Evaluation

The third and final phase in developing the new scale, scale evaluation, consisted of three steps: tests of dimensionality, tests of reliability, and tests of validity.

Tests of Dimensionality

For this study, the degree of internal consistency of the new scale was conducted using Cronbach alpha reliability testing of the final scale items and factors. The factors (subscales) were also examined for possible multidimensionality. Due to the small sample size, splitting of the data into two samples with the use of Confirmatory Factor Analysis would not have been acceptable. However, the value of Fabrigar’s recommendation “that a confirmatory approach might be adopted later in the research program after exploratory analyses helped to more fully develop the researcher’s hypotheses” (Fabrigar, 2012 p. 12) is fully supported as the way forward for future research in this area. As a final step in the scale development process the percentage of variance explained by the factor analysis was computed and reported.

Ethical Issues

The appropriate steps were taken during the conduct of this study to ensure that the highest ethical standards were upheld, that all participants’ rights were respected, and privacy and confidentiality of the participants’ data were maintained:

• Informed Consent – An informed consent form was sent to all prospective student participants. Students received clear information on the purpose of the study and all measures taken to protect their rights as participants. Those who agreed to participate clicked ‘yes’ for their consent and they then gained access to the online survey. Those who declined and clicked ‘no’, ended their participation at this point.
• Institutional Review Board (IRB) approval: Data for this study was collected on the main campus of the university Since the study involved the use of human participants, the researcher was required to first obtain approval from the University’s IRB.

RESULTS

The focus of this study was on describing the methodology followed in identifying the dimensions of service quality that are relevant to domestic and overseas students at a local university and incorporating those dimensions into the development of a valid and reliable scale that can be used to measure service quality. In phase 1 of this mixed method study, research question #1, the  qualitative research question for this study was as follows:

• How do students at a local university in Trinidad and Tobago define the determinants of service quality expectations and perceptions?

The most important requirement for a quality tertiary institution, based on frequency of student participants’ responses, was categorized as academic concerns, which encapsulated such responses as: the ability of the lecturers to deliver content, the university’s reputation in the region as a top tertiary institution, and opportunities for students’ self-directed learning. The second most frequent response from the students was categorized as ethical concerns. The third most frequently cited aspect of service quality was categorized as social concerns and encompassed such responses as the availability of extra-curricular activities, opportunities for students to interact socially, and participation in sporting facilities and clubs. In the fourth ranked position was the student support concept and the fifth was categorized as self-development concerns. The realization was made during the NVIVO data analysis that the students’ preferences paralleled to some extent the SERVQUAL themes and three additional themes (social benefits, ethics and co-creation) so the decision was taken for their inclusion as factors in the modified survey instrument along with the traditional SERVQUAL themes. These ranked students’ responses, matched to the eight themes used to create items for the quantitative survey are presented in Table 3

Summary of Qualitative Data Analysis Findings – Response to Research Question#2

Research Question #2: What are the distinct factors of service quality and which items best represent these factors?

Students’ top determinants of service quality closely matched the themes of Responsiveness and Assurance from the SERVQUAL instrument along with Cocreation, a new theme was deemed most important. However, the other determinants of service quality followed closely behind in rank order with ethical standards and social benefits rounding out the top three choices in participants’ view. The students focused the least on the physical aspects of their tertiary environment (tangibles).

Item Development

Table 4 shares a brief sample of how integration occurred between the qualitative and quantitative phases as responses made by students to focus group questions were used as the basis for the items developed for the new service quality instrument.

Content validation of newly developed items

Following this first step of item development, the second step – content validation of the newly developed items, took place using five expert judges.  For content validation, Fleiss’ Kappa was run to determine if there was agreement between the expert judges on whether the items were appropriately placed within each of the eight different dimensions of service quality (Fleiss, 1971). Fleiss’ kappa showed Overall Percentage agreement of 89.453. The guidelines adapted from Landis and Koch (1977) on assessing how good the strength of agreement is when based on the value of Cohen’s kappa coefficient, indicate that 0.81 -1.00 is rated as a very good strength of agreement.

Results of Scale Development

The new instrument consisted of matched pairs of items: 55 expectation items and 55 perceptions items. Following the scale adjustment based on the review of the items by the expert judges, cognitive interviews occurred and this led to a re-examination and re-wording of two of the items to increase clarity. The 116-item survey was submitted electronically yielding a response rate of approximately 14%.

Extraction of Factors using Factor Analysis

Initially, the factor ability of the 55 items in the expectation section of the survey was examined for factor ability using Pearson’s Correlation, Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett’s test of spheri city. Firstly, it was observed that all the 55 items correlated at .3 or above with at least one other item, suggesting reasonable factor ability (see Appendix D, Correlation of survey items results). Next, the Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett’s test of sphericity were used.

The KMO test result of .892, above the commonly recommended value of .6, indicated that a factor analysis should be useful with this data.

The Bartlett’s test of sphericity was significant (χ2 (1485) = 6137.619 p < .05). Both these results are displayed in Table 5. Given these overall indicators, factor analysis was deemed to be possible and suitable for use with all 55 items.

According to Boateng et al. (2108), factor extraction is the next step after item reduction to be conducted in the scale development process. This was performed by extracting latent factors which represented the shared variance in responses among the multiple items (Mc Coach et al., 2013). The focus was on the number of factors, the salience of factor loading estimates, and the comparative magnitude of residual variances (Boateng et al., 2018). Exploratory Factor Analysis was conducted using the M plus Version 8.3 software, with the maximum likelihood as its default factor extraction method.

For the next step in factor extraction, an exploratory factor analysis (EFA) was performed on the fifty-five items of the service quality scale.

In this study, Exploratory EFA, was applied using the Mplus Version 8.3 software. The criteria selected for factor extraction were the scree test as a reference, along with factor loadings guided by basic parameters. An initial extraction was performed to identify any irregularities in the data. Items were dropped based on the following criteria: first, if a loading score of less than   0.5 was reached and secondly, constructs containing less than three items were identified.

The results of the factor loadings are presented in Supplementary Table 1. Note that the items that were retained for further Exploratory Factor Analysis (EFA) appear in bold:

The factors were rotated with GEOMIN factor rotation in the standard EFA model.  From this initial GEOMIN Factor loading, cross loading items were extracted. Supplementary Table 2 presents a sample, listing four cross loading items that were ultimately removed: Though they did not load at .5 and above on any factor, they all loaded above .30 on more than one factor and were removed from the scale.

The scale now consisted of 32 items down from 55, contained within five factors. The fifth factor formerly known as ‘CoCreation’, was significant that all seven items achieved a loading score above 0.5, and seven items, all developed by the researcher, based on student focus group discussions and literature review, were all retained. Supplementary Table 3 presents the results of the GEOMIN Factor Rotation in EFA, unacceptable items being removed, and five factors retained. Those items that were retained are identified in bold.

Several items showed evidence of loading on a factor other than the ones that were used in the scale development process. Of the eight original factors only five were retained at the end of the first stage of the EFA process and three factors  were eliminated with none of their items retained. Overall, the data reduction process resulted in 32 items retained in the new scale and 23 that were discarded, not having met the criteria for retention. Fourteen items did not have an adequate loading (less than 0.5) on any factor.

Factor Loading and Goodness-of-fit measures

The factors were rotated with the oblique GEOMIN factor rotation on M plus. The researcher defined the number of factors (from-to) in the modelling step and M plus computed all necessary factor models in parallel. After factor rotation, additional factor extraction, and goodness-of-fit measures were applied for final factor selection, the data analysis protocols resulted in the most appropriate number of latent constructs (common factors), comprising the domain of service quality in this context. The fit indices were used to report the results in the format set out by Kyriazos:

“EFA model fit was evaluated by the standards proposed by Hu & Bentler (1999) and Brown (2015): RMSEA (≤0.06, 90% CI ≤0.06), SRMR (≤0.08), CFI (≥0.95), TLI (≥0.95), and the chi-square/df ratio less than 3 (Kline, 2016)” (Kyriazos et al., 2018b: p. 1154).

Utilizing the Mplus software for statistical analysis, the researcher began with a one-factor model and obtained the fit statistics for this model. The researcher then fit the two-factor model and computed the test statistic for this model. Because an EFA model with fewer factors is a special case of a model with more factors (i.e., the first model is nested within the second), it was possible to conduct statistical tests to see if the addition of each factor led to a statistically significant improvement in fit. For example, this was done by computing the likelihood ratio χ 2 difference test between models where:

  χ 2 = χ 2 (of model with fewer factors) −

             χ 2 (of model with more factors)

df= df (of model with fewer factors) − df (of model with more factors)

The difference between the χ 2 values of the two models also follows a χ 2 distribution with a df equivalent to the difference between the df of the two models. Thus, this new χ 2 was examined to determine if the test was significant. If the test showed a lack of significance, the model with one less factor was retained. If it was significant, the model with one less factor was rejected in favour of the more complex model. This more complex model was the new comparison point against which a model with one additional factor was compared. The process terminated when a nonsignificant test was obtained. Thus, factors were only included if they provided a significant improvement over a model with one fewer factor. For my study, the four-factor model emerged as the most acceptable with the most appropriate fit statistics. Supplementary Table 4 presents a comparative summary of the one, two, three and four factor EFA models showing fit statistics results.

The pattern matrix indicated a four-component structure, unlike the initial factor loading which indicated a five-factor structure. Component loadings which are measures of the unique relationship between the latent variable (i.e., component), and the item, i.e., observed variable; (Tabachnick & Fidell, 2007), were examined for evidence of cross-loading.

A sample of the correlations among components of the remaining items, as well as the Cronbach alpha levels are displayed in Appendix D.

At this stage more meaningful names for the extracted factors were established using the technique suggested by Neill (2007).

Service Quality Scale for Higher Education (SQSHE)

Factor 1 – Student physical and mental well-being, shortened in tables to “Spm Wel”

Factor 2 – Efficiency of services provided, which is shortened in tables to “Eff Ser”

Factor 3 – Social interaction, which is shortened in tables to “Soc Int”

Factor 4 – Value cocreation, which is shortened in tables to “Val Cocre”

Testing for dimensionality and validity

Scale evaluation began with testing for scale dimension ality. This analysis was performed to assess the internal consistency of the scale. The small sample size that was utilized for this study and the limited time frame available for data collection and analysis meant that the reapplication of the four factor scale on a new sample, or, the same sample being tested at a different point in time (Boateng et al, 2018) were beyond the scope of this study. However preliminary construct validation was seen as a reasonable achievement for this phase of the research and the researcher looks forward to the future research opportunities when validation of the SQSHE can be continued with Confirmatory Factor Analysis (CFA)

Tests for Reliability of New Scale

To determine the reliability of the data collected, Cronbach’s Alpha was calculated for each subscale or service quality dimension in the new survey. Supplementary Table 5 highlights the results the Cronbach Alpha test.

Cronbach’s alpha showed the questionnaire to reach acceptable reliability, α = 0.847 – 0.883. Most items appeared to be worthy of retention, resulting in a decrease in the alpha if deleted. The one exception to this was item 5 in Factor 4 which would increase the alpha to α = 0.904. As such, removal of this item was considered for future implementation of the scale.

Summary of results for Research Question # 2:

Research Question# 2: What are the distinct factors of service quality and which items best represent these factors?

In response to this research question, it has been confirmed by EFA analysis that there are four dimensions/factors of service quality. These four factors form an underlying structure based on the variables that have undergone preliminary construct validation.

For the EFA, using factor rotation and goodness-of-fit measures with GEOMIN rotation method, resulted in the retention of four factors with 23 items. The analysis showed that Kaiser-Meyer-Olkin measure of sampling adequacy was .892, which indicates that the data were appropriate for this analysis. Bartlett’s test of sphericity was significant χ2 (1485) = 6137.619 p < .05) indicating that correlations exist among the service quality dimensions. Thirty-two items were removed from the scale because the value of the loading was < 0.5 and there were fewer than three loaded items for each factor.

 The final dimensions were identified as: student physical and mental well-being, efficiency of services provided, social interaction and value cocreation. The result of EFA showed that the four-factor model was satisfactory with all measures within acceptable bounds. For the four-factor model, fit measures all achieved the prerequisite limits with Chi-square = 281.923, Chi-square/df = 149.The root mean square error of approximation (RMSEA) index was 0.063, which is within the recommended critical limit of 0.08 indicating moderate/ acceptable fit. The comparative fit index (CFI) 0.949, and TLI Tucker Lewis Index was 0.998. These values represent good fit compared with the acceptable value >0.90 for these two fit indices. SRMR = 0.031, factor loadings 0.504 – 0.940 and factor correlations ranged from 0.663 – 0.883.

The primary purpose of the present research was to extend the existing service quality literature by providing evidence that a newly developed scale to measure service quality expectations verses perceptions has preliminary construct validity. Factor analysis indicated that the newly developed service quality scale consisted of four factors, rather than the five to eight factors suggested by the initial qualitatively driven, deductive and inductive item development processes and existing literature on the topic. Exploratory factor analysis supported a four-factor structure; however, the four factors were different than the proposed structure. The new scale has been named the Service Quality Scale for Higher Education (SQSHE).

Study Strength

This newly developed Service Quality Scale for Higher Education (SQSHE) represents the first scale that assesses students’ experiences of service quality in any local university in the Caribbean environment.  It includes factors other than those five measured by SERVQUAL, (tangibles, reliability, responsiveness, assurance and empathy) which have received extensive attention in the literature on service quality in higher education.

Implications of the Scale Development Process

Taken holistically, the results of the scale development process revealed that the students were concerned with both their professional and personal development at the university. They gave priority to administrative customer services, self-directed learning support and most especially social interaction opportunities. These findings support to some extent the outcomes from a project conducted here in the Caribbean region by Ali, (2012), the Caribbean Higher Education Service Quality Project, which sought to gain a better understanding of how students prioritized various service categories in higher education. In Ali’s project, students in his sample were primarily concerned with their professional and personal development, and felt their higher education institutions should concentrate on services related to providing learning support, instruction and student development experiences. (Ali, 2015)

Study Limitations

The current study was limited by the lower-than-expected student participation rate.  Based in this result, it is emphasized that the results generated from this study may only apply to the context of medium sized private universities in the Caribbean region, and the generalization of findings requires further testing in a larger sample size and among other educational cultures.

DISCUSSION OF FINDINGS

There are major gaps in the literature on the dimensions that comprise service quality in the Caribbean, and the measurement of service quality using a contextually relevant instrument. Previous research in this area has emphasized the use of the SERVQUAL instrument in international countries and the need for modification to this instrument to address the unique experiences of students whenever service quality is measured in higher education. To extend the literature, this study integrated both qualitative and quantitative measures to develop a new instrument that captures student experiences, through an EFA factor analysis.The SQSHE scale consists of 23 items and four subscales (Student wellbeing, Efficiency of Services, Social Interaction and Value Cocreation).   The result of EFA showed that the four-factor model was satisfactory. The root mean square error of approximation (RMSEA) index was 0.063, which is within the recommended critical limit of 0.08 indicating moderate/ acceptable fit. The comparative fit index (CFI) 0.949, and TLI Tucker Lewis Index was 0.998. These values represent good fit compared with the acceptable value >0.90 for these two fit indices. SRMR = 0.031, factor loadings were 0.504 – 0.940 and factor correlations ranged from 0.663 – 0.883. Preliminary construct validity was established.

For additional validation, it is recommended that the SQSHE be administered to a larger sample above 300 participants and Confirmatory Factor Analysis (CFA) conducted on this larger sample in order to confirm the a priori hypothesis about the relationship between the initially validated scale factor items.

This study lends support to the notion that wholesale use of the SERVQUAL instrument to measure service quality in higher education may not be appropriate. Instead, significant modification of the instrument to create a contextually robust scale is the more valid approach.

One of the most noteworthy findings of the research was that the newly developed scale differed widely from the original five-dimension SERVQUAL structure by Parasuraman (1988). Neither did it reflect the structures proposed by Lehtinen and Lehtinen (1991). Instead, of the eight original dimensions of service quality, four dimensions are identified as valid for this study, and of these, Factors 3 and 4 were by far the most important as evidenced by them displaying the highest factor loadings during the factor analysis process. The findings of EFA support the application of a four-dimension model for measuring service quality.At the start of this study the original scale developed to measure service quality expectations and perceptions contained all five SERVQUAL factors, tangibles, reliability, responsiveness, assurance and empathy. The students’ responses during the focus group interviews and the NVIVO data analysis led to the initial inclusion of all five traditional SERVQUAL dimensions on the scale. However, a surprising result of this study was that following the item reduction and factor analysis phases only two of those original SERVQUAL themes were retained as factors of service quality. Elements of the tangibles dimension (five items on physical characteristics and two items from the reliability dimension (courtesy and caring by administrative staff) displayed moderate factor loadings to form the new scale factor now called Student physical and mental well-being or Stuwel.

Parasuramam et al. (1988) defined tangibles as “the appearance of the servicescape and the physical facilities and materials.” (as cited in Chavan et al., p. 163). For most international research studies, students had much to say regarding those common tangible aspects of university life that shaped their perceptions of a high-quality tertiary experience. However, it was interesting to note that for this study, during the three focus group interviews there was minimal reference by both overseas and domestic students to the physical facilities of the university in shaping their expectations and perceptions of service quality. When asked at the end of each focus group meeting to list in order of importance the top three qualities they consider when evaluating service quality at a tertiary institution, the accumulated data indicated that tangibles, identified by students as Safe, clean and attractive living conditions /environment, ranked as number 6 out of 7, quite low on their list of priority concerns. Though further research would be required in this area to form conclusions, it may be reasonable to suggest that in the present sample of students their evaluation of the service quality of their tertiary experience had less to do with the tangible aspects and more to do with the interrelationships and non-academic experiences whether involving lecturers, peers or support staff. Studies by Chavan et al. (2014) and others based on international and domestic students service quality experiences in countries like Australia indicated differences in results when compared with this study. Students there, especially international students were strongly influenced by tangible factors (such as an attractive campus environment; a modern library; fast, reliable Internet services; convenient access to public transport and well-developed student well-being services, among other similar factors) in their evaluations of a quality tertiary experience. In contrast, however, this present study reflects to some extent the findings of research done by Ongo (2019) in the US. His study found that international students showed less concern about the appearance and neatness of the institution’s physical facilities, equipment, personnel and other tangibles, once their institutions had higher levels of reliability and empathy (Ongo, 2019)

RECOMMENDATIONS FOR HIGHER EDUCATION INSTITUTIONS IN THE CARIBBEAN

Students are considered the key stakeholders of any higher education institution, consequently a clear understanding of their needs and wants would be vital to the success and development of the institution in the longer term (Khattab 2019). The findings of this study have indicated that in this local Caribbean university at least, “service quality is a strategic issue that required continuous measurement of service expectations” (Jusoh et al., 2004, p. 261, as cited by Ali, 2015), because “the students’ perceptions of SQ elements change over a period of study” (Kontic, 2014 p. 651). So, it is imperative that institutions of higher learning keep up with a continuous service quality improvement system. Rather than engage in piecemeal, isolated and uncoordinated improvements, it has been suggested that a holistic approach within the context of continuous quality improvement should be adopted. Bourne (2011) highlights the paradigm shift in managing quality that must occur to foster an institution-wide quality culture transformation that is cost-efficient and  sustainable rather than a series of disconnected project initiatives.  continuous improvement or continuous quality improvement (CQI) may be preferable. (Ali, 2015; Grigoroudis& Siskos, 2010; Schneider & White, 2004; Ali, 2021). Such an approach would ensure that tertiary institutions remain poised to deliver high service quality to students that are continuously responsive to the constantly monitored but changing needs of the student population, which is an imperative for all tertiary institutions in the competitive higher education environment that currently exists in the Caribbean.  This would be an effective strategy to develop student loyalty towards the institution and promote the spread of positive word-of-mouth feedback, which is an influential “promotional conduit” (Asim & Kumar, 2018 p. 80) especially in the close-knit societies of the Caribbean.  In order to establish a CQI framework for continuous improvement in response to the service quality deficits that this research revealed, an institution wide improvement strategy is imperative, incorporating administrative commitment and support as evidenced by strategic goal setting and allocation of resources by the institution’s governing body. In addition, the institution may benefit from the implementing a continuous improvement system that is modeled after international Quality Management Systems while incorporating the institutions local needs and external regulatory agency requirements.

REFERENCES

1. Ali, E. (2015). Transforming higher education in the Caribbean: The total quality management/service quality model. In A. K. Perkins (Ed.), Quality in higher education in the Caribbean, UWI, pp. 255-268.
2. Ali, E. (2021). A critical review of strategic quality improvement practices in Caribbean higher education institutions. In S. I. Gift (Ed.), Caribbean quality culture: Persistent commitment to improving higher education (pp. 260-273). UWI Press.
3. Asquith, J.A. (1997). The effects of group size on the outcome of focus group sessions. Management Research News, Vol. 20 No.12, pp.1-15.  https://doi.org/10.1108/eb028582
4. Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, Vol. 6, p.149. https://doi.org/10.3389/fpubh.2018.00149
5. Bunce, L., Baird, A., & Jones, S. (2017) The student-as-consumer approach in higher education and its effects on academic performance, Studies in Higher Education, Vol.42 No. 11, pp.1958-1978. https://doi.org/10.1080/03075079.2015.1127908
6. Chavan, M., Bowden-Everson, J., Lundmark, E. & Zwar, J. (2014). Exploring the drivers of service quality perceptions in the tertiary education sector: Comparing domestic Australian and international Asian students. Journal of International Education in Business,Vol. 7 No.2, pp.150-180. https://doi.org/10.1108/JIEB-02-2014-0004
7. Çerri, Shpetim. (2012). Assessing the quality of higher education services using a modified SERVQUAL scale. Annales Universitatis. https://doi.org/:10.29302/oeconomica.2012.14.2.32.
8. Creswell, J. W. (2014). Research design: Qualitative, quantitative and mixed methods approaches (4th ed.). Sage.
9. Donaldson, B. & McNicholas, C. (2004). Understanding the postgraduate education market for UK‐based students: a review and empirical study. Int. J. Nonprofit Volunt. Sect. Mark., Vol.9, pp. 346-360. https://doi.org/10.1002/nvsm.259
10. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, Vol.4No.3, pp. 272–299. https://doi.org/10.1037/1082-989X.4.3.272
11. Farahmandian, S., Minavand, H., &Afshardost, M. (2013). Perceived service quality and student satisfaction in higher education. IOSR Journal of Business and Management, Vol. 12, pp. 65-74. https://doi.org/10.9790/487X-1246574
12. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, Vol.76 No. 5, pp. 378-382. https://doi.org/10.1037/h0031619
13. Hassad, R. (2007). Development and Validation of a Scale for Measuring Instructors’ Attitudes toward Concept-Based or Reform-Oriented Teaching of Introductory Statistics in the Health and Behavioral Sciences [Unpublished doctoral dissertation]. Touro University International.
14. Hayashi, P., Abib, G., & Hoppen, N. (2019). Validity in qualitative research: A processual approach. The Qualitative Report, Vol.24 No.1, pp. 98-112. https://nsuworks.nova.edu/tqr/vol24/iss1/
15. Hu, L. T., & Bentler, P. M. (1999). Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives. Structural Equation Modeling, Vol.6, pp.1-55. http://dx.doi.org/10.1080/10705519909540118
16. Kyriazos, T. A. (2018). Applied Psychometrics: Writing-up a factor analysis construct validation study with examples. Psychology, Vol.9 No.11, pp. 2503-2530. https://doi.org/ 10.4236/psych.2018.911144
17. Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, Vol. 33No. 1, pp. 159-174. https://doi.org/10.2307/2529310
18. McCoach, D. B., Gable, R. K., & Madura, J. P. (2013).  Instrument development in the affective domain. school and corporate applications, (3rd ed.). Springer
19. Morgado, F. F., Meireles, J. F., Neves, C. M., Amaral, A. C., & Ferreira, M. E. (2017). Scale development: Ten main limitations and recommendations to improve future research practices. Psicologia: Reflexão e Crítica, Vol. 30 No. 3, pp.1-20. https://doi.org/10.1186/s41155-016-0057-1
20. Morse, J. M. (2017). Essentials of qualitatively driven mixed-method designs. Routledge.
21. Neill, J. (2007). Writing up a factor analysis.  Centre for Applied PsychologyUniversity of Canberra. Creative Commons Attribution 2.5 Australia http://creativecommons.org/licenses/by/2.5/au/
22. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw Hill.
23. Ongo, M. O. (2019). Examining perceptions of service quality of student services and satisfaction among international students at universities in Indiana and Michigan.Dissertation. Andrews University. https://digitalcommons.andrews.edu/dissertations/1675
24. Parasuraman, A., Zeithaml, V.A. & Berry, L.L. (1988). SERVQUAL: a multiple-item scale for measuring consumer perceptions of service quality, Journal of Retailing, Vol.64 No.1, pp.12-40.
25. Parasuraman, A., Zeithaml, V.A. & Berry, L.L. (1991). Refinement and reassessment of the SERVQUAL scale. Journal of Retailing, Vol.67 No.4, pp. 420-50.
26. Pereda, M., Airey, D., & Bennett, M. (2007). Service quality in overseas education: The experience of overseas students. Journal of Hospitality, Leisure, Sport and Tourism Education, Vol. 6 No.2, pp. 55 – 67.
27. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. (5th ed.). Allyn & Bacon/Pearson Education.
28. Thurstone, L. L. (1947). Multiple factor analysis. University of Chicago Press.
29. Whelan, C. (2016). Organisational culture and cultural change: A network perspective. Australian & New Zealand Journal of Criminology, Vol.49 No.4, pp. 583–599. https://doi.org/10.1177/0004865815604196

APPENDIX

APPENDIX A

Questions for Focus Group Interviews

Section 1: Motives for pursuing higher Education

1. Why did you decide to pursue tertiary /higher education? Prompts: what motivated you?

2. Why did you decide to pursue higher education here at USC, a private Christian University?
   1. Prompts for international students: Why did you decide to leave your home country to seek tertiary education?

Section 2:  Expectations for the tertiary experience

3. Did you have any expectations for your physical environment at the university? What were they?
4. What were your major academic expectations regarding this tertiary education experience? E.g. the degree of difficulty courses and classes would entail.
5. What were some of your expectations regarding campus life?
6. What were some of your expectations regarding the quality of services?
7. What were some tangible aspects of university life that you considered to be relevant to a quality university experience in terms of faculty provision?

Section 3: Social Benefits (a new theme not present in SERVQUAL)

8. The prospect of making friends at USC was that an important expectation to you? Did you expect to make friends from outside your home territory?
9. Did you feel lonely, miss your former friends, and find it difficult to fit in on this campus?
10. Did you expect to form social networks, join clubs?
11. Do you think cultural barriers added or contributed in any way to your ability to make friends?
12. International students: Do you think your cultural background in terms of your traditional ways of living your life influenced your expectations in coming to study here?

Section 4: Co- Creation (a new theme from “Exploring the Drivers…2014)

13. How were you involved in co-creation?
14. Was this one of your expectations that you would be required to become involved in the co-creation of your educational experiences?
15. Was active participation one of your expectations as well as self-directed learning?

Final Request

Each of you has been given a small card. On this card please write your top three requirements for a quality tertiary experience, ranking them from one to three with number one as the most important and three as the third most important. Please also write on the card whether you are a local or overseas student.

APPENDIX B

Listing of new scale (with newly created items highlighted in bold)

Dimension #1 – Tangibles

201. The lecturers have access to modern or the latest equipment
202. The appearance of the physical facilities of the university is attractive
203. Lecturers and Administrative staff are well dressed and neat in appearance
204. The Library has the latest resources in my area of interest
205. The classrooms are well lit, and well ventilated
206. The campus provides adequate spaces for students to relax
207. The campus has clean and comfortable dormitory facilities
208. The institution’s offices display convenient operating hours

Dimension #2 – Reliability

301. When something is promised by a certain time, it is always provided by the Lecturers or Administrative staff

0302 When students have problems, Administrative staff members are courteous, even if not able to help

0303 Administrative Staff personnel show a sincere interest in solving students’ problems

0304 There is a high quality of service offered by Administrative staff

0305 Lecturers provide clearly stated requirements for course and programme success

0306 The university maintains accurate student records

0307 Courses are taught by highly knowledgeable lecturers

Dimension #3 – Responsiveness   

02201 Students are informed of schedules and changes in schedules in advance

2202. Service hours of the university’s learning facilities accommodate all students
2203. 02203 Administrative staff are never too busy to respond to student requests

2204. Administrative staff respond promptly to student queries
2205. Lecturers give prompt feedback to students on assignments and assessments

Dimension #4 – Assurance

401. The behavior of lecturers and administrative staff instill confidence
402. Students can trust lecturers and administrative staff
403. Lecturers and administrative staff are friendly and polite
404. Lecturers are dependable
405. The students feel safe on campus
406. Student complaints are treated promptly and fairly
407. Opportunities exist for student to give feedback on their level of satisfaction with key university experiences

Dimension #5 – Empathy                                             

501. Lecturers provide personal attention to every student
502. Lecturers allocate sufficient office-hours to cater to students’ inquiries
503. Administrative staff members give students individual attention
504. Lecturers understand the specific needs of students
505. University policies are student-centered and fair
506. Needy students have access to financial and other forms of assistance
507. Support for at risk students is provided through university policies and practices

Dimension #6 – Social Benefits                                                                                

601. The University provides opportunities for students to interact socially outside of the classroom
602. Social activities for students are scheduled at convenient times
603. It is easy to become involved in campus social organizations
604. The university has a friendly and welcoming atmosphere
605. Students can choose to participate in a wide range of social activities provided by the university
606. The university organizes social activities that students from various ethnic and cultural backgrounds can enjoy
607. The university provides opportunities for students to showcase and share their non-academic talents and abilities

Dimension #7 – Ethics

0701.The university upholds high standards of integrity

702. The university’s reputation as a quality tertiary institution is promoted
703. A degree from this university is well recognized locally and abroad
704. The university’s ethical standards are practiced by members of faculty and staff in their interactions with students
705. Students can participate in a wide range of scheduled activities that promote the moral values of the University
706. Students are treated with impartiality by lecturers in the classroom
707. Students are treated with impartiality by administrative staff when requesting assistance

Dimension #8 – Cocreation

801. Students receive an educational experience that goes beyond the classroom

0802 Real-world work experiences are integrated into each student’s’ programme of study

803. Internships/ practicums are integrated into the structure of each university programme.
804. Leadership skills are nurtured through the student government and other activities
805. Community involvement and volunteerism are promoted and facilitated by the university
806. Lecturers provide opportunities for students to engage in self- directed learning
807. Lecturers encourage students to “think outside the box”

APPENDIX C

Joint Display #1

Integration of focus group interview comments into identification of domains and item generation:

APPENDIX D

Correlations

Domain #1 – Tangibles

Correlation result – All items acceptable, all items above .300. Reliability = .846 Acceptable

Supplementary Table 1. EFA Factor loadings

Supplementary Table 3

Pattern Matrix Obtained after GEOMIN Rotation – Cross Loading Items Removed

Supplementary Table 4. EFA Goodness-of-fit indices for various EFA models of SQSHE 

Supplementary Material Table 5

Cronbach’s Alpha Test for Reliability of New Scale