Online self-exclusion from multiple gambling venues: Stakeholder co-design of a usable and acceptable self-directed website

Australian National Library of Medicine – Dec 18, 2021

D. Pickering,^⁎ A. Serafimovska, S.J. Cho, A. Blaszczynski, and S.M. Gainsbury

Author information Article notes Copyright and License information PMC Disclaimer

Gambling self-exclusion programs are under-utilised and barriers to entry include shame and embarrassment with face-to-face registration, and complex and effortful procedures. The current study aimed to facilitate self-exclusion from gambling venues via an online self-directed website. A co-design approach was used to elicit key stakeholders' perspectives on required website features, functionality, and to identify variables potentially impacting on development and implementation. Semi-structured focus groups and interviews were conducted across four stakeholders (N = 25): self-exclusion end users (consumers, n = 5), gambling counsellors (n = 7), venue staff (n = 6), and policy makers (n = 7).

Overall, stakeholder perspectives were consistent with content analysis indicating the importance of website user-friendliness, flexibility, supportiveness, and trustworthiness. Importantly, these attributes were linked to target end users': perceived vulnerabilities, diverse backgrounds and individual expectations. Participants believed that the entire self-exclusion process should be conducted online, including identity verification, whilst expecting high-level data security measures to protect their personal privacy. A separate webpage was also suggested containing relevant information and links to additional help services, such as counselling.

This study describes an adaptable co-design framework for developing a usable and acceptable self-exclusion website. Future studies should empirically test system usability and acceptability to refine and maximise system uptake upon implementation. Findings may have broader implications for digital health intervention design.

Keywords: Gambling disorder, Online self-exclusion, Harm minimization, eHealth, Co-design, Self-directed intervention

·        Self-directed online registration will remove structural barriers and improve access to gambling self-exclusion.

·        We interviewed gambling employees, policymakers, counsellors, and individuals with lived experience of Gambling Disorder.

·        Results suggest the self-exclusion website should optimally balance system usability with information security.

·        Customizability will empower users to personalise self-exclusion to their needs and preferences.

·        Government and industry support and targeted promotional efforts were identified as crucial to implementation success.

Legalised gambling is a popular recreational activity in many international jurisdictions. However, a minority of people who meet criteria for a Gambling Disorder repeatedly gamble to excess by spending more time and money than is personally affordable, which leads to significant impairment and distress (Productivity Commission, 2010). Gambling Disorder is classified as an addictive disorder in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association [APA], 2013) and International Classification of Diseases (ICD-11; World Health Organization, 2019). Epidemiological studies estimate the past-year prevalence of Gambling Disorder ranged between 0.1 and 5.8% internationally (Calado et al., 2016). However, around two-to-three times as many people who do not meet the clinical threshold for a Gambling Disorder are estimated to experience less serious levels of gambling harm (Abbott, 2017). Among multiple types of gambling, electronic gaming machines (EGMs) are commonly associated with the greatest risk of harm (Delfabbro et al., 2020; Productivity Commission, 2010; Gainsbury et al., 2019).

A range of formal treatment options exist for Gambling Disorder, including counselling and psychotherapy (Petry, 2005), pharmacological treatments (Bullock and Potenza, 2013), support groups (Schuler et al., 2016), and stimulus control interventions such as self-exclusion (Tanner et al., 2017). However, few individuals with Gambling Disorder (≈10% or less) seek formal help despite experiencing significant harm (Delfabbro, 2011; Kessler et al., 2008; Slutske, 2006). The most frequently reported barriers to seeking help include stigma and the associated embarrassment or shame of seeking treatment, denial of problem severity or desire to self-manage problems, and issues with specific interventions (e.g., practical barriers to access, or perceived effectiveness; Gainsbury et al., 2014, Gainsbury et al., 2020; Suurvali et al., 2009). Internet-based interventions have been introduced to overcome some of these barriers by improving access to cost-effective evidence-based gambling treatment (Gainsbury and Blaszczynski, 2011).

Self-exclusion interventions are mandated in many international jurisdictions where regulated land-based and online gambling opportunities exist (Gainsbury, 2014). Land-based gambling requires in-person participation (e.g., physically playing EGMs, casino table games; purchasing a Keno/Lottery ticket in a licensed venue), whereas online gambling activities are conducted via the Internet (e.g., sports/race betting websites; online casinos and poker sites). Self-exclusion interventions are available for both land-based and online gambling forms, and are mostly provided by industry operators, or can be administered by government departments and private companies depending on the jurisdiction (Australasian Gaming Council, n.d.). Self-exclusion represents a type of stimulus control strategy involving a formalised process for individuals to voluntarily ban themselves from entering/accessing nominated gambling venues/sites. Staff in land-based venues are authorized to remove self-excluded individuals from the premises if detected breaching agreements (Ladouceur et al., 2017); whereas online operators suspend a self-excluded user's wagering account until the exclusion period expires (Gainsbury et al., 2020). Individuals involved in self-exclusion interventions self-report reduced gambling urges and behavior and increased psychological and social functioning among other benefits (Hayer and Meyer, 2011; Kotter et al., 2018; Ladouceur et al., 2007; Nelson et al., 2018; Pickering et al., 2018).

Self-exclusion has several weaknesses that impair intervention uptake and efficacy despite positive reported outcomes (Gainsbury, 2014), including structural barriers that deter individuals from enrollment (Motka et al., 2018). Many jurisdictions still require individuals to physically attend separate face-to-face meetings at gaming venue sites for each-and-every venue from which they wish to self-exclude (Thomas et al., 2016). Subsequent exposure to gambling-related stimuli is a risk factor for further gambling episodes as these have been shown to trigger strong urges to gamble in clinical populations (Potenza et al., 2003; Smith et al., 2013). Individuals typically report high psychological distress leading up to self-exclusion (Hing et al., 2014; Ladouceur et al., 2007; Pickering et al., 2019). Situational factors, such as stress, can impair inhibitory control functions that support behavior self-regulation, leading individuals to prioritise immediate desires over long-term interests (Brand et al., 2019). Once self-exclusion is enacted, rates of individual noncompliance range between 8 and 59% (Kotter et al., 2018), with venues detecting between 23 and 52% of breaches (Pickering et al., 2018; Schrans et al., 2004; Verlik, 2008).

Recommendations have been advanced, tested, and in some instances implemented to structurally enhance self-exclusion interventions, with new and emerging technology as the primary source of innovation (Håkansson and Henzel, 2020; Ipsos MORI Public Affairs, 2020; Pickering et al., 2018; Turner et al., 2021). Certain contemporary programs allow individuals to self-exclude from multiple gambling venues under a single registration process that uses a centralised online database to collect and store client information (Hing and Nuske, 2012; Pickering et al., 2018). Additionally, individuals can self-exclude outside the gambling venue with an authorized counsellor or trained officer, although the process still requires they attend a face-to-face meeting (Hing and Nuske, 2012; Pickering et al., 2018; Tremblay et al., 2008). Recent qualitative evidence suggests that end users would prefer an online self-exclusion registration pathway, indicating that it would streamline the process, enhance convenience and privacy, whilst encouraging personal ownership of help-seeking behavior (Pickering et al., 2019). Despite being readily available for online gambling forms (Motka et al., 2018), online self-exclusion registration for land-based gambling is rare. To the best of our knowledge, the only purely online system exists in Sweden (Spelpaus - ‘Game Break’) and according to industry figures, has attracted over 65,000 sign-ups since inception which is correct as of November 2021 (Swedish Gambling Authority, n.d.).

Systematic research to guide the development of contemporary self-exclusion interventions and evaluate their outcomes is scarce, thus making it difficult to establish best practices in this domain. In particular, there are few studies aimed at understanding the specific needs and expectations of self-exclusion end users, and tailoring interventions to meet these requirements.

1.1. Study context

This study was conducted in the geographical context of New South Wales (NSW) Australia. EGMs in Australia are located in clubs (not-for-profit community centres), hotels (pubs), and casinos. All gambling operators in Australia are mandated by state law to offer self-exclusion schemes upon request (Australasian Gaming Council, n.d.). Multiple self-exclusion interventions exist which vary considerably between states, especially in their level of sophistication. For this reason, and to enhance the ease of entry for consumers, we have previously advocated for a single unified national self-exclusion system (Pickering et al., 2018). Whilst this analysis is intended to inform specifications of a NSW based website, it provides a useful knowledge base for the development of similar services in other jurisdictions.

1.2. Study aims and approach

This qualitative study represents the requirements analysis component of a larger program of research to develop and evaluate a self-directed online registration portal for individuals in NSW Australia to self-exclude from land-based gambling venues. Interviews and focus groups were conducted with relevant stakeholder groups: self-exclusion end users (consumers), gambling counsellors, venue employees, and government policy makers, in order to investigate their perspectives of the functional and contextual requirements for an effective self-exclusion website. Specifically, this study had a primary and secondary aim: (1) to elicit key stakeholders' ideal expectations of a self-exclusion website in terms of its design features and functioning; (2) to identify practical issues that could potentially impact the website development and implementation. A co-design research framework was applied given its suitability for developing new electronic health (eHealth) systems (Crosby et al., 2017; Donetto et al., 2015; Eyles et al., 2016). A self-exclusion website is consistent with the broadly defined ‘eHealth’ construct: “an emerging field in the intersection of medical informatics, public health and business, referring to health services and information delivered or enhanced through the Internet and related technologies” (Eysenbach, 2001, e20). The operational definition of ‘co-design’ is highly variable (Slattery et al., 2020); our prior research demonstrates end users' involvement in developing the research question (Pickering et al., 2019); whereas the current study empowered end users' to guide important design and implementation decisions, thus shaping how this eHealth service will be disseminated to the public (Slattery et al., 2020). Co-design has been shown to lead to more efficient decision-making, reduced development time and costs, and greater stakeholder investment and cooperation (Dawda and Knight, 2017; Steen et al., 2011). The qualitative design was appropriate, relative to a quantitative design, because it can provide a more in-depth and contextualised understanding of service gaps and their potential solutions (Van Gemert-Pijnen et al., 2018; Van Velsen et al., 2013).

The study was approved by the institution's Human Research Ethics Committee (HREC; Project #2020/111), and pre-registered on the Open Science Framework (OSF; https://osf.io/6d54v/) using a template tailored to qualitative research designs (Hartman et al., 2019). Any deviations from pre-registered protocol are described in the Methods as footnotes.

2.1. Participants

The total sample consisted of twenty-five eligible participants with either: 1) lived experience of a gambling problem and past or current involvement in land-based self-exclusion, or 2) current employment in a profession that has direct relevance to self-exclusion, gambling, and problem gambling. Purposive sampling was applied to gather data from four key stakeholder categories: consumers (self-exclusion end users; n = 5), mental health (gambling counsellors, n = 7), industry (gambling venue staff, n = 6), and government (liquor and gaming policy makers¹; n = 7), all of whom fulfilled either the first or second above-specified inclusion criteria. The latter three groups (i.e., ‘professional participants’) had been working in their field for 9.2 years on average (range = 9 months to 27 years). Consumer participants were recruited from an industry self-exclusion program database, and professional participants were recruited via existing contacts and with the assistance of the research funding body. Study information materials informed eligible participants of the study purpose: to “gain their unique opinions and perspectives” on the “design of a self-directed website for convenient self-exclusion from land-based gaming machine venues” to explore its “key functions and features” and anticipate “practical challenges to implementation”. The sample size requirement of 5–10 participants per group was met (minimum required to reach data saturation, Boddy, 2016), which is consistent with other similar eHealth studies (Krog et al., 2018; Ly et al., 2015; Ospina-Pinillos et al., 2019; Peek et al., 2016).

Table 1 presents a breakdown of key participant characteristics by subgroup. Average age of participants across the full sample was 37.7 years old (SD = 12.4), approximately half were female (52.0%) and most were well-educated (64% held a University degree). Almost the entire sample accessed the internet daily (96.0%), spent a mean of 6.1 h per day (SD = 3.4) using the internet, primarily for work or study (76.0%), social networking (52.0%), and news media consumption (52.0%). Just under half of the participants (48.0%) indicated they were extremely or somewhat likely to trial new technology before others, whereas the remaining 52.0% were either neutral or believed they were somewhat less likely. Consumer participants, specifically, reported having dealt with gambling problems for an average of 9.4 years (SD = 4.5), mostly due to playing EGMs (80.0%), and four out of five had sought assistance from a gambling counsellor in the past.

Table 1

Common participant characteristics (variables) per stakeholder category (N = 25).

Open in a separate window

2.2. Procedure

Semi-structured focus groups and interviews² were conducted between April and May 2020, separately for each stakeholder group and via Internet-based video conferencing software (Zoom Video Communications Inc., 2016).³ Prior to scheduled sessions, participants completed and returned a digitally signed consent form and anonymised online demographics survey. Focus groups and interviews began with an overview of relevant information and study purpose. Participants were asked to treat each focus group or interview as if it were a “brainstorming session”. They were then asked to briefly outline their knowledge and experiences of self-exclusion, and respond to open ended questions around optimal features, processes and contextual considerations for a self-exclusion website. Off-script probe questions were asked sparingly to clarify or elaborate on participants' responses (Berg, 2004). The proceedings took an average of 55 min to complete, verbal responses were audio recorded and subsequently transcribed and anonymised for analysis. Consumer participants received a $50 retail gift voucher to compensate them for their time. Participation among the other groups was not reimbursed as it was considered within the scope of their professional responsibilities, i.e., to actively engage in gambling harm minimization initiatives.

2.3. Measures

Online demographics and screening questionnaire: age, gender, education, employment status (consumers only), occupational history (professionals only), gambling history (consumers only), general Internet use, and likelihood of trialling new technology.

Core semi-structured focus group/interview questions were:

In your opinions, what will people look for when deciding to use a website to self-exclude? That is, what might their ideal expectations of this website be?

What might deter people from using a website to self-exclude?

In addition to facilitating self-exclusion requests, what other features or functionality do you think the website should incorporate to provide users with assistance for their gambling problem?

Can you think of any practical issues or possible challenges that could arise during the development or implementation of this self-exclusion website?

2.4. Qualitative data analysis

Schreier, 2014 eight steps of content analysis were performed on the qualitative dataset using NVivo (Version 11.0.0; QSR International Pty Ltd., 2015). To begin, two authors, DP and AS, adopted a concept-driven approach (guided by study aims and interview questions; see Schreier, 2012, Ch. 5) to formulate initial categories and sub-categories based on four transcripts selected to represent each stakeholder group. They subsequently applied a data-driven method to further develop and refine the preliminary coding framework, ensuring that all transcript material was accounted for (Schreier, 2014). A codebook was compiled detailing all categories, subcategories, coding frequencies, operational definitions, rules of application, and illustrative examples (see Supplementary Materials for the full coding frame). To assess reliability of the coding framework, SJ double coded the four representative transcripts coded by DP and AS. Inter-rater reliability coefficients indicated a discrepancy between high (98%) agreement, and comparatively low Cohen's Kappa value of 0.56 (some to moderate agreement; Cohen, 1988) (see ‘Kappa paradox’ for an in-depth explanation of this finding; Zec et al., 2017). All authors deliberated on remaining coding discrepancies or ambiguities until consensus was achieved (Bengtsson, 2016). The final coding frame was applied to all transcripts, with total coding frequencies per subcategory calculated. Relationships between emergent subcategories were indicated by high frequency instances of cross-coding (i.e., the same text is coded multiple times across different subcategories; Saldaña, 2009).

Content analysis produced a final coding frame encompassing a three-level hierarchical structure with four main categories, 13 level one sub-categories and 10 level two subcategories. Fig. 1 provides a visual representation of the coding frame structure, coding frequencies, and thematic relationships between subcategories. A more detailed account of the results is provided underneath.

Fig. 1

A visual map of the coding frame structure derived from all stakeholders. Circles = main categories; Rectangles =1st level subcategories; Text-only = 2nd level subcategories. Coding frequencies are shown in brackets and relationships across sub-categories are indicated using coloured dotted lines.

3.1. Contextual factors

Any variables (i.e., personal, environmental) expected to either contribute to success or failure of the system implementation, that encompassed by specific features and functioning.

3.1.1. Perceived end-user characteristics 

Participants indicated that the system design should be tailored to meet the specific requirements of target end users. End users were expected to be younger adults who use computer devices and the Internet for a variety of tasks.

“You find a lot of young people self-exclude [who] come in here [casino]… They are students. They are here on a visa. And these are the people that you really want to capture. Especially students. Why? Because they are usually computer savvy, they wouldn't mind doing it if they [can] do it online.”(Venue staff, 59yrs, Male)

Despite several common characteristics, participants also acknowledged the diverse backgrounds of potential end users, especially across life experiences and readiness to change. This subcategory was associated with system flexibility (Section 3.2.1) and availability of process supports (Section 3.2), suggesting that these features are important to meet the individual needs of end users. One professional participant in gambling policy commented: “a couple of things we'd like to do better is [to] have a good system for culturally and linguistically diverse people using our website.” (Policy maker, 45 yrs., Female).

Individuals self-excluding were perceived to be experiencing significant negative emotional states that would likely impact on completing self-exclusion registrations (i.e., psychological distress, ambivalence, impatience, embarrassment, shame). Speaking to their mental state at the time, a service end user (Male, 25 yrs) with prior self-exclusion experience highlighted that “You're handling a lot, and, in the midst of that, any barriers just make it more difficult, you know?” Notably, this subcategory related to encouragement (subcategory 3.2.3) and process supports (subcategory 3.3.2), highlighting the importance of factoring in end user's emotional state to decisions regarding website design, content, and workflow.

3.1.2. Socio-legal environment 

Many potentially contributing environmental factors were identified that could impact the uptake and utility of this web-based self-exclusion across stakeholder groups. These included; different geographical regions, social (e.g., stigma around problem gambling), political (e.g., government priorities and relationships with industry), and legal factors (e.g., regulated self-exclusion requirements). This subcategory was linked to credibility and trustworthiness (Section 3.2.2) and ID verification (Section 3.3.1), possibly reflecting legal issues that could arise if the website was unreliable in detecting fraudulent self-exclusion attempts, like those attempted on behalf of others without their knowledge or consent.

“Their [gambling venue operators] only concern is compliance. That would be, as a manager of a club, if you came to me with your proposal, I could [not] care less what it is, just how do I how ensure that I don't get sued? I'm not signing up to it if don't have to.” (Counsellor, 48yrs, Male)

3.1.3. Advertising and promotion 

Participants emphasised the importance of actively promoting the system's availability and unique benefits to increase awareness and stimulate uptake among end users. The online platform was viewed as particularly advantageous by facilitating both digital marketing strategies and more traditional advertising methods. This importance of this subcategory was underscored in one self-exclusion end user focus group:

“Hopefully just having the digital platform makes it easier to promote as well. Because when [participant] said he'd never heard of self-exclusion before, that to me is a bit of a worry… I reckon it's definitely one of those things that's sort of out of sight out of mind unless someone asked for it [specifically]. Yeah, making sure that everybody knows this is an option and it's not a difficult option. I think it's super important.” (Consumer, 32yrs, Male)

3.2. Website attributes

This main category comprises latent qualities that strategically interweave throughout website elements to convey its core functions, content, design, and aesthetics. These qualities are tailored to match the end users' specific needs and expectations.

3.2.1. User-friendly 

As the highest coded subcategory in the analysis, participants emphatically asserted that usability was crucially important, particularly given the self-directed nature of this system. First, end users should easily be able to locate and interpret relevant information on the website, then seamlessly navigate the online self-exclusion process unassisted.

“Easy, easy-to-use. Step 1-2-3. For the layman, who doesn't understand, you know, a lot of people, even if it was an app or anything like that, a lot of people struggle to download apps. So, it's got to be extremely easy.” (Venue staff, 49yrs, Male)

Second, the website should accommodate end users' personal needs and preferences, ensuring that it is usable for all target population members – from individuals initially seeking information about gambling support options to those with extensive involvement in self-exclusion and other specialised help services. This includes customizable features allowing end users to control how they access and use the website. Smartphone compatibility was one commonly suggested feature under this subcategory:

“For me, I do a lot of stuff on my phone. And I know some websites don't work properly on your phone. I think you've got to design it so that you can use all the prompts from your phone… Yeah, I would say an app is better for me personally… I think that I would want to check out the website to see what's going on and what I'm actually downloading. But then fill in my form on the app… Because I find that easier.” (Venue staff, 26yrs, Female)

3.2.2. Credible and trustworthy 

The website should convey to end users that the service is legitimate, safe, and secure to use. Participants provided several suggestions to possibly achieve this: a professionally designed ‘government-style’ theme, display of data security badges and certifications and official Government and University logos, and featuring previous end users' feedback.

“I think legitimacy... if it doesn't look official and schmick and professional, regardless of how the security and privacy is, people won't trust it. Part of it, I think, is about the quality of the design and the kind of the finishing there is in the graphic design rather than the functional design. That it looks neat. You know, there can't be mistakes. It can't look… 20 years old or like maybe someone it did at home.” (Policy maker, 35yrs, Female)

3.2.3. Encouraging 

Given their vulnerable psychological state upon self-excluding (subcategory 3.1.1), participants felt it important for the website to validate end users' decisions to take action and address their gambling problem, motivate them to overcome ambivalence and see the self-exclusion process through to completion. Specifically, suggestions included incorporating positive imagery (i.e., happy people) and warm colours into aesthetic elements of the web design, and personal testimonies of those who successfully self-excluded using the service. For example, a self-exclusion end user participant suggested:

“I think some kind of positive reinforcement statements, you know, congratulating the person for taking the first step in going on to the website, and just encouragement. I think that can be important sometimes because it can be a bit scary going through the whole process. Especially initially when you're thinking of barring yourself from the venue. It can be a bit negative, [a] bit of a downer. So, having some positive reinforcement, I think would help.” (Consumer, 25yrs, Male)

3.3. Core self-exclusion function

These are the main processes and technologies required to operate a web-based self-exclusion system, including online registration procedures, features built in to support registration, and specific data security and privacy measures.

3.3.1. Key process elements 

The content analysis revealed five distinct chronological steps involved in online self-directed self-exclusion. In the first step, end users must provide basic information about themselves (i.e., first and last name, date of birth, residential address) and have a passport-style image taken for identification purposes. Second, end users must be able to prove that is indeed them registering for the program. A venue staff member described a hypothetical scenario to reinforce the importance of verifying end user identity:

“The biggest issue I see is if I think that my mother's got a problem, I might log in and exclude her from all the clubs she regularly goes to. It's usually the elderly who their children are ringing saying they're spending all our inheritance and really want them barred. So there has to be a way of them proving and signing off on their identity…. Probably a 100-point check style verification… I think if you make it loud and clear that if you are, you know, a third party and you're trying to say that you're the person is excluding the ramifications of that are quite serious.” (Venue staff, 49yrs, Male)

Third, end users' are able to customise certain aspects of their personal self-exclusion agreement, i.e., selecting different venues and venue areas, as well as the self-exclusion timeframe. Fourth, a review section will allow end users to edit their information and rectify mistakes. Additionally, they will be required to review all terms and conditions of a formal self-exclusion agreement and digitally sign it to indicate their consent. The fifth and last step involves confirming the submission and acceptance of an agreement and notifying all relevant parties selected at the end users' own discretion or court-ordered. These parties mostly referred to nominated venues, also concerned family, counsellors, and legal representatives. Referring to the final step in the process, one gambling policy representative indicated:

“I'd expect some kind of confirmation of what I've done, you know, something that I can take away or have after the fact for whatever purpose to show that I have done this self-exclusion… maybe down the track, I want to be able to show my counsellor or family, ‘yeah don't worry, I've done it’… And… Having a record of actually what rights [I] signed away, you know, under what conditions.” (Policy makers, 35yrs, Female)

3.3.2. Process support elements 

These refer to ancillary features designed to assist end users to navigate the online self-exclusion process. Therefore, they contribute to but do not constitute key process elements (i.e., subcategory 3.3.1). This subcategory encompassed passive design features (e.g., a progress bar) through to more complex interactive elements (e.g., live chat support).

“So maybe the first page of your website is sort of like the informational stuff. And then, as you go into describing or writing down your details, if you're not sure about the process… If you needed help, the support was there… phone or email or you know, these days, you can have like a little chat down the bottom, that would be handy.” (Venue staff, 26yrs, Female)

3.3.3. Data security measures 

Related to trustworthiness, the website must incorporate highly secure data protection measures given the sensitivity of gambling issues and importance of maintaining end user privacy. Such measures include high-level information security systems and software (e.g., encrypting data transmitted between client and server) and robust data management practices (e.g., limiting who has access to the data). One gambling counsellor spoke about their discussions with clients around privacy and confidentiality concerns relating to existing self-exclusion programs:

“A couple of my clients. We've discussed the whole self-exclusion programme thing and they've been very wary about confidentiality. Who can access their information once it goes to the venues? These often are people who have worked in hospitality. So they have concerns about where that information stored who can access it, how's it going to influence, you know.” (Counsellor, 57yrs, Female)

3.4. Additional website features

The final main category comprised of supplemental features and functionality that add value to the website in terms of its usefulness and user experience, though they may not necessarily be directly relevant to self-exclusion registration itself.

3.4.1. Additional help resources 

All participant groups indicated that the website should include relevant psychoeducational material and links to additional gambling help resources. To illustrate, a self-exclusion end user commented: “Definitely have links to other problem gambling websites, treatment facilities or counselling services… Or even just little articles or blogs or podcasts, you know, just stuff around gambling and treatment and education. I think that would be handy.” (Consumer, 25 yrs., Male). Furthermore, these resources should be holistic and target the multiple areas of life affected by gambling.

Although not required to complete a self-exclusion registration, participants also believed this process as an opportune time to request end users' consent to be contacted by a gambling counsellor.

“I like the suggestion of having a follow up call, I think that should be an option. Like once they're done with the exclusion, maybe tick a box, do they want a phone call? You know, in a week or whatever time period… [to] gain further information about individual therapy or you know, so they know there is someone that they can call or that can call them, but I think as an option for them rather than [it being] mandatory.” (Counsellor, 28yrs, Female)

3.4.2. User information dashboard 

Mostly consumer participants suggested incorporating a user profile dashboard feature allowing end users to login, review, and update details of an active self-exclusion agreement. This feature may be useful for self-reporting of breaches, extending the timeframe and/or geographical scope of an agreement, being notified of and renewing self-exclusion as the termination date approaches. One participant with prior self-exclusion experience commented: “For me, I think having some sort of portal where I can log-in and see where I am self-excluded from… when [the self-exclusion] runs out and all that kind of stuff.” (Consumer, 25 yrs., Male).

3.4.3. Non-self-exclusion user functions 

Professional stakeholders believed it would be useful to include included a login platform for professional users (similar to the end user dashboard) that supported breach reports submitted by counsellors or venues operators, relevant usage data collection, analysis and insights, and provision of pertinent information relating to self-exclusion program operations. Overall, these functions should be directed toward improving self-exclusion efficacy as reflected by the following participant quote:

“I think access to data would be really important. Trying to understand uptake, those sorts of things. How it links into a venue… so breaches can be reported… And I think having access to that information, not as a compliance tool, but as an information tool to see what's going on, where there might be issues… a venue might need some extra training in dealing with people that [are] breaching. I think that'd give us a really good idea if we've got particular hotspots in the state that we need to be mindful of.” (Policy maker, 45yrs, Female)

Self-directed online registration is a logical empirically supported next step to advancing accessible self-exclusion interventions intended to enhance convenience and remove shame and embarrassment for end users. This study sought to identify key design and functional requirements with important contextual considerations for a co-designed self-exclusion website involving multiple relevant stakeholders. Although participants in this study expressed alternative viewpoints in relation to specific design issues, many of the findings were consistent between stakeholder groups.

Desirable latent website attributes were strongly emphasised across all groups, particularly, that self-directed online self-exclusion should be an easy and simple process for end users, and the platform should accommodate individual needs and expectations. Multi-platform compatibility (i.e., functionality across smartphone, tablet, laptop, and desktop devices) was frequently referenced as an example of the latter requirement of ‘flexibility’. Simplicity and flexibility are universal attributes important to the success of any web-based application (Chisholm and May, 2008); however, they take on increasing importance with systems designed for vulnerable patient groups (Arsenijevic et al., 2020; Miller et al., 2017; Ben-Zeev et al., 2013). In support, ‘usability’ or “the ease with which a system can be learned and used” (Dabbs et al., 2009, p.176), is a key predetermining factor for the level of acceptance and effectiveness of digital health technology in target populations (Maramba et al., 2019). Conversely, overly long and complex registration procedures have been identified as a key barrier for entry into existing self-exclusion programs (Pickering et al., 2018; Motka et al., 2018). Participants in the current study emphasised the importance of a user-friendly website so as not to exacerbate end users' feelings of emotional distress and ambivalence to change. The current findings highlighted the value of a website that encourages end users by positively reinforcing their decisions to self-exclude. Examples provided by study participants mostly related to aesthetic features and positive user feedback or testimonials to enhance self-efficacy; however, the application of game design principles (i.e., ‘gamification’) to incentivise self-exclusion completion may also warrant consideration in this context (see Griffiths, 2019; Sardi et al., 2017).

Participants drew upon their understanding of existing self-exclusion processes to describe the core sequential functions of a web-based self-exclusion system. These included the collection of strictly relevant personal information and preferences, in addition to administering information regarding program terms and conditions and gaining end users' consent prior to enacting the agreement. For participants, integrating processes currently performed face-to-face into an online environment presented several challenges. These mostly related to the absence of an informed professional to verify end users' identity and to guide them through each step of the process. However, identity verification software has been successfully applied to comparable online tasks that were previously completed face-to-face, and many end users are already experienced in using this technology (e.g., online banking; Goode, 2018). Furthermore, self-exclusion consumers' experiences of unhelpfulness and insensitivity with some venue staff has been documented as a self-exclusion deterrent (Hing et al., 2014; Pickering et al., 2019). However, process support features (e.g., clear instructional content, telephone or live chat support hotline) were identified in the current analysis as a viable offset regarding the lack of in-person support.

All participants recognised the importance of data security measures to protect end users' confidentiality, especially given the pervasive social stigma that is associated with gambling problems (Hing et al., 2015). However, they simultaneously identified that highly rigorous security may hinder the required simplicity and ease of use in the system. Therefore, it is likely that an optimal balance must be struck between such attributes. It was important to incorporate security measures and to promote the existence of these to end users via website content, thus reassuring them that the system is safe to use. Our findings are consistent with e-commerce and behavioral economics studies indicating higher consumer confidence and engagement with websites displaying security or ‘trust’ badges (Johnson et al., 2020, Nong and Gainsbury, 2019). Beyond the current website, these findings may be extended to inform efforts to improve the uptake of eHealth programs that require sensitive end user data; specifically, by highlighting the importance of transparent communication of security information to end users in a format that is easy to interpret (Van Gemert-Pijnen et al., 2011).

All participants agreed that information and links to a range of additional help services would be a useful auxiliary website feature. This finding adds support to the current perspective that self-exclusion can serve as a gateway into more intensive treatment for gambling problems (Blaszczynski et al., 2007). Suggestions to incorporate both gambling and non-gambling specific services are consistent with a contemporary, multidimensional model of gambling recovery (Pickering et al., 2019). Stakeholder groups considerably differed in proposing other auxiliary website features; consumers focused on ideas to enhance end user experience (i.e., user information dashboard), whereas professionals focused on ideas relevant to administrative users (e.g., a reporting mechanism for exclusion breaches). Incorporating features relevant to different stakeholders into the website design and functioning may be expected to enhance acceptability or ‘buy-in’ across all groups thus increasing the likelihood of service adoption, penetration and sustainability upon implementation (Proctor et al., 2011; Steen et al., 2011).

Gaining government and industry support for the system was emphasised, particularly by professional participants, as a key practical consideration for implementation. Government support may be contingent on compliance with current gaming machine regulations and data privacy law; whereas industry support is affected by the associated costs, impacts on business operations, and disruption to recreational gamblers (Blaszczynski, 2018; Chóliz, 2018; Ladouceur et al., 2016). The importance of open collaboration between these stakeholders in efforts to minimise gambling harm is detailed in a prominent framework of shared responsibility, coined The Reno Model (see Blaszczynski et al., 2004). Other concerns related to integrating a self-exclusion website into an already complex environment of competing self-exclusion schemes (see Australasian Gaming Council, n.d.). Ideally, jurisdictions would work toward establishing unified state or national self-exclusion programs that encompass both land-based and online gambling forms (depending on legal availability). Notably, almost all study participants resided in NSW, Australia. Although similar political forces may be expected to operate across jurisdictions internationally, specific features of the gambling environment will differ, thus warranting careful consideration in the implementation process.

Lastly, a good marketing strategy was perceived as important to promote the availability of the self-exclusion website to potential end users. In previous studies, self-exclusion consumers have reported lack of awareness and difficulty accessing information about self-exclusion options (Pickering et al., 2019; Hing and Nuske, 2012; Hing et al., 2014). One UK study found that fewer than 1-in-5 participants in a sample comprised mostly of gamblers were aware of any one of several nationally available multi-operator self-exclusion schemes (including a self-exclusion website for online gambling operators; Ipsos MORI Public Affairs, 2020). Most participants learned about these schemes through traditional communication mediums including television/radio and word-of-mouth (Ipsos MORI Public Affairs, 2020). Current findings highlight the need to utilise the online format of self-exclusion websites and apply digital marketing strategies (e.g., via social media), potentially attracting a younger population prior to or at the initial onset of their problem gambling (Black and Shaw, 2019).

Overall, it is difficult to compare our results to the limited experiences of other jurisdictions that have implemented gambling self-exclusion websites due to a general lack of public reporting on design and implementation processes; and before now, there has been no standard framework of stakeholder requirements to guide relevant decisions. Existing self-exclusion website operators are encouraged to retrospectively review their services in the context of current findings to identify areas for improvement. With respect to eHhealth interventions more broadly, the findings demonstrate overlap with several design features that are proposed to mediate design effects on outcomes (e.g., tailoring, aesthetics, credibility, usability, information architecture; Morrison et al., 2012). However, the requirements advanced in this study extend beyond established frameworks by detailing how such features and new ones (e.g., linking to additional help services) can be integrated into self-exclusion website design, and by outlining practical considerations for effective implementation.

The strength of qualitative research is in its capacity to facilitate detailed, in-depth investigations of research questions, including the flexibility to explore new and emerging topics (Queirós et al., 2017). Although this multi-stakeholder, semi-structured interview design was appropriate, it was simultaneously vulnerable to the limitations of qualitative research. Particularly, self-selection bias may have arisen due to a purposive sampling strategy and the use of a small non-representative sample. Furthermore, it is possible unequal stakeholder groups may have contributed to over- or under-representation of certain stakeholder perspectives. All self-exclusion consumers involved in this analysis had strong formal help-seeking histories for gambling problems, thus were not representative of many gamblers who never seek assistance. Despite difficulties in accessing this non-help seeking subgroup for research, their perspectives are especially important in the context of increasing engagement with available help services.

The current findings provide a qualitative data-driven guiding framework to inform the development of self-directed web-based systems for gambling self-exclusion.

Although this study was targeted toward self-exclusion, several of the findings are relevant to eHealth interventions on a broader level. In particular, the website attributes (easy and simple, flexible, credible and trustworthy, and encouraging) could be used as a basic model for developing self-directed online interventions to maximise their likelihood of patient uptake.

Requirements that are specific to self-exclusion programs, such as collecting and verifying personal information and program preferences, must be considered in the unique context of each jurisdiction, including its social norms, legal and geographical environment.

Practical constraints will likely prevent the inclusion of all requirements in the website design; thus, it is useful to prioritise these by feasibility (i.e., cost, time, difficulty) and importance (as perceived by stakeholders).

The next steps involve a systematic design and evaluation process, including iterations between prototype development, and testing for acceptability, usability, and effectiveness (Van Gemert-Pijnen et al., 2018; Van Velsen et al., 2013).

Lastly, registration is only one aspect of the self-exclusion process. Future research should investigate technology-based strategies to enhance detection rates of non-compliant self-excluded individuals.

This research was funded by the NSW Office of Responsible Gambling under the 2019 Responsible Gambling Grants Program.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

We would like to thank ClubsNSW for their support in providing access to research participants, as well as the participants themselves for all their efforts and valuable insights.

¹Policy makers were added as a key stakeholder group after preregistration of the study due to their professional contact with self-exclusion interventions in the domain of compliance and legislation.

²The addition of one-on-one interviews deviates from the pre-registered study protocol which includes focus groups only, however was necessary due to participant non-attendance and scheduling clashes.

³Whilst face-to-face methods were originally intended for data collection, COVID-19 physical distancing regulations necessitated this alternative online method.

^{Appendix A}Supplementary data to this article can be found online at https://doi.org/10.1016/j.invent.2021.100491.

Final coding frame

Click here to view.^{(37K, docx)}Image 1

1.      Abbott M. The epidemiology and impact of gambling disorder and other gambling-related harm. World Health Organisation. 2017. https://www.who.int/docs/default-source/substance-use/the-epidemiology-and-impact-of-gambling-disorder-and-other-gambling-relate-harm.pdf?sfvrsn=5901c849_2

2.      American Psychiatric Association . Arlington, VA; American Psychiatric Association: 2013. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. [Google Scholar]

3.      Australasian Gaming Council, n.d.Australasian Gaming Council (n.d.). Self-exclusion programs. https://austgamingcouncil.org.au/seeking-help/self-exclusion-programs.

4.      Arsenijevic J., Tummers L., Bosma N. Adherence to electronic health tools among vulnerable groups: systematic literature review and meta-analysis. J. Med. Internet Res. 2020;22(2) [PMC free article] [PubMed] [Google Scholar]

5.      Bengtsson M. How to plan and perform a qualitative study using content analysis. NursingPlus Open. 2016;2:8–14. [Google Scholar]

6.      Ben-Zeev D., Kaiser S.M., Brenner C.J., Begale M., Duffecy J., Mohr D.C. Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatric Rehab. J. 2013;36(4):289–296. [PMC free article] [PubMed] [Google Scholar]

7.      Berg B.L. 5th ed. Pearson; Boston, MA: 2004. Qualitative Research Methods for the Social Sciences. [Google Scholar]

8.      Black D.W., Shaw M. In: Gambling Disorder. Heinz A., Romanczuk-Seiferth N., Potenza M., editors. Springer; Switzerland: 2019. The epidemiology of gambling disorder; pp. 29–48. [Google Scholar]

9.      Blaszczynski A. Responsible gambling: the need for collaborative government, industry, community and consumer involvement. SUCHT. 2018;64(5–6):307–315. [Google Scholar]

10. Blaszczynski A., Ladouceur R., Nower L. Self-exclusion: a proposed gateway to treatment model. Int. Gambl. Stud. 2007;7:59–71. [Google Scholar]

11. Blaszczynski A., Ladouceur R., Shaffer H.J. A science-based framework for responsible gambling: the Reno model. J. Gambl. Stud. 2004;20(3):301–317. [PubMed] [Google Scholar]

12. Boddy C.R. Sample size for qualitative research. Qual. Mark. Res. Int. J. 2016;19(4):426–432. [Google Scholar]

13. Brand M., Wegmann E., Stark R., Muller A., Wolfling K., Robbins T.W., Potenza M.N. The interaction of person-affect-cognition-execution (I-PACE) model for addictive behaviors: update, generalization to addictive behaviors beyond internet-use disorders, and specification of the process character of addictive behaviors. Neurosci. Biobehav. Rev. 2019;104:1–10. [PubMed] [Google Scholar]

14. Bullock S., Potenza M. Update on the pharmacological treatment of pathological gambling. Curr. Psychopharmacol. 2013;2(3):204–211. [PMC free article] [PubMed] [Google Scholar]

15. Calado F., Griffiths M., Calado F. Problem gambling worldwide: an update systematic review of empirical research. Journal of Behavioural Addiction. 2016;5(4):592–613. [PMC free article] [PubMed] [Google Scholar]

16. Chisholm W., May M. (1st edition) O’Reilly Media; 2008. Universal Design for Web Applications: Web Applications that Reach Everyone. [Google Scholar]

17. Chóliz M. Ethical gambling: a necessary new point of view of gambling in public health policies. Front. Public Health. 2018;6:12. [PMC free article] [PubMed] [Google Scholar]

18. Cohen J. Routledge Academic; New York, NY: 1988. Statistical Power Analysis for the Behavioral Sciences. [Google Scholar]

19. Crosby L.E., Ware R.E., Goldstein A., Walton A., Joffe N.E., Vogel C., Britto M.T. Development and evaluation of iManage: a self-management app co-designed by adolescents with sickle cell disease. Pediatr. Blood Cancer. 2017;64(1):139–145. [PMC free article] [PubMed] [Google Scholar]

20. Dabbs A., Myers B.A., Mc Curry K.R., Dunbar-Jacob J., Hawkins R.P., Begey A., Dew M.A. User-centered design and interactive health technologies for patients. Comput. Inform. Nurs. 2009;27(3):175. [PMC free article] [PubMed] [Google Scholar]

21. Dawda P., Knight A. Australian Healthcare and Hospital Association (AHHA) and Consumers Forum of Australia (CHF); 2017. Experience based co-design: A tool for Australia. [Google Scholar]

22. Delfabbro P. Australasian gambling review. Independent Gambling Authority. 2011. https://www.cbs.sa.gov.au/sites/default/files/resource-files/australasian-gambling-review-5th-edn-web.pdf?timestamp=1549253806331

23. Delfabbro P., King D., Browne M., Dowling N., Delfabbro P. Do EGMs have a stronger association with problem gambling than racing and casino table games? Evidence from a decade of australian prevalence studies. J. Gambl. Stud. 2020;36(2):499–511. [PubMed] [Google Scholar]

24. Donetto S., Pierri P., Tsianakas V., Robert G. Experience-based co-design and healthcare improvement: realizing participatory design in the public sector. Des. J. 2015;18(2):227–248. [Google Scholar]

25. Eyles H., Jull A., Dobson R., Firestone R., Whittaker R., Te Morenga L., Goodwin D., Mhurchu C. Co-design of mHealth delivered interventions: a systematic review to assess key methods and processes. Curr. Nutr. Rep. 2016;5(3):160–167. [Google Scholar]

26. Eysenbach G. What is e-health? J. Med. Internet Res. 2001;3(2) [PMC free article] [PubMed] [Google Scholar]

27. Gainsbury S. Review of self-exclusion from gambling venues as an intervention for problem gambling. J. Gambl. Stud. 2014;30(2):229–251. [PMC free article] [PubMed] [Google Scholar]

28. Gainsbury S., Blaszczynski A. Online self-guided interventions for the treatment of problem gambling. Int. Gambl. Stud. 2011;11(3):289–308. [Google Scholar]

29. Gainsbury S., Angus D., Blaszczynski A. Isolating the impact of specific gambling activities and modes on problem gambling and psychological distress in internet gamblers. BMC Public Health. 2019;19(1):1372. [PMC free article] [PubMed] [Google Scholar]

30. Gainsbury S.M., Angus D.J., Procter L., Blaszczynski A. Use of consumer protection tools on internet gambling sites: customer perceptions, motivators, and barriers to use. J. Gambl. Stud. 2020;36(1):259–276. [PubMed] [Google Scholar]

31. Gainsbury S., Hing N., Suhonen N. Professional help-seeking for gambling problems: awareness, barriers and motivators for treatment. J. Gambl. Stud. 2014;30(2):503–519. [PubMed] [Google Scholar]

32. Goode A. Biometrics for banking: best practices and barriers to adoption. Biom. Technol. Today. 2018;2018(10):5–7. [Google Scholar]

33. Griffiths M. The use of gamification in facilitating the use of responsible gambling tools. Gaming Law Rev. 2019;23(1):19–22. [Google Scholar]

34. Hartman A., Kern F., Mellor D.T. Preregistration for qualitative research template. 2019. https://osf.io/j7ghv/wiki/home/

35. Hayer T., Meyer G. Internet self-exclusion: characteristics of self-excluded gamblers and preliminary evidence for its effectiveness. Int. J. Ment. Heal. Addict. 2011;9(3):296–307. doi: 10.1007/s11469-010-9288-z. [CrossRef] [Google Scholar]

36. Håkansson A., Henzel V. Who chooses to enroll in a new national gambling self-exclusion system? A general population survey in Sweden. Harm Reduct. J. 2020;17(1):82. [PMC free article] [PubMed] [Google Scholar]

37. Hing N., Nuske E. The self-exclusion experience for problem gamblers in South Australia. Aust. Soc. Work. 2012;65(4):457–473. [Google Scholar]

38. Hing N., Elaine N., Gainsbury S.M., Russell A. Perceived stigma and self-stigma of problem gambling: perspectives of people with gambling problems. Int. Gambl. Stud. 2015;16(1):31–48. [Google Scholar]

39. Hing N., Holdsworth L., Tiyce M., Breen H. Stigma and problem gambling: current knowledge and future research directions. Int. Gambl. Stud. 2014;14(1):64–81. [Google Scholar]

40. Hing N., Tolchard B., Nuske E., Holdsworth L., Tiyce M. A process evaluation of a self-exclusion program: a qualitative investigation from the perspective of excluders and non-excluders. Int. J. Ment. Heal. Addict. 2014;12(4):509–523. [Google Scholar]

41. Ipsos MORI Public Affairs Process and impact evaluation of the multi-operator self-exclusion schemes: baseline report (evaluation phase 1) 2020. https://about.gambleaware.org/media/2247/18-038856-01-moses-combined-evaluation-report-final-for-publication.pdf

42. Johnson S.D., Blythe J.M., Manning M., Wong G.T.W. The impact of IoT security labelling on consumer product choice and willingness to pay. PLoS ONE. 2020;15(1) [PMC free article] [PubMed] [Google Scholar]

43. Kessler R.C., Hwang I., LaBrie R., Petukhova M., Sampson N.A., Winters K.C., Shaffer H.J. DSM-IV pathological gambling in the National Comorbidity Survey Replication. Psychol. Med. 2008;38(9):1351–1360. [PMC free article] [PubMed] [Google Scholar]

44. Kotter R., Kräplin A., Bühringer G. Casino self- and forced excluders’ gambling behavior before and after exclusion. J. Gambl. Stud. 2018;34(2):597–615. [PubMed] [Google Scholar]

45. Krog M.D., Nielsen M.G., Le J.V., Bro F., Christensen K.S., Mygind A. Barriers and facilitators to using a web-based tool for diagnosis and monitoring of patients with depression: a qualitative study among danish general practitioners. BMC Health Serv. Res. 2018;18(1):503. [PMC free article] [PubMed] [Google Scholar]

46. Ladouceur R., Sylvain C., Gosselin P. Self-exclusion program: a longitudinal evaluation study. J. Gambl. Stud. 2007;23:85–94. [PubMed] [Google Scholar]

47. Ladouceur R., Shaffer P., Blaszczynski A., Shaffer H.J. Responsible gambling: a synthesis of the empirical evidence. Addict. Res. Theory. 2017;25(3):225–235. [Google Scholar]

48. Ladouceur R., Blaszczynski A., Shaffer H.J., Fong D. Extending the Reno model: responsible gambling evaluation guidelines for gambling operators, public policymakers, and regulators. Gaming Law Rev. Econ. 2016;20(7):580–586. [Google Scholar]

49. Ly K.H., Janni E., Wrede R., Sedem M., Donker T., Carlbring P., Andersson G. Experiences of a guided smartphone-based behavioral activation therapy for depression: a qualitative study. Internet Interv. 2015;2(1):60–68. [Google Scholar]

50. Maramba I., Chatterjee A., Newman C. Methods of usability testing in the development of eHealth applications: a scoping review. Int. J. Med. Inform. 2019;126:95–104. [PubMed] [Google Scholar]

51. Miller D.P., Weaver K.E., Case L.D., Babcock D., Lawler D., Denizard-Thompson N., Pignone M.P., Spangler J.G. Usability of a novel mobile health iPad app by vulnerable populations. JMIR MHealth UHealth. 2017;5(4) [PMC free article] [PubMed] [Google Scholar]

52. Morrison L.G., Yardley L., Powell J., Michie S. What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed. e-Health. 2012;18(2):137–144. [PubMed] [Google Scholar]

53. Motka F., Grune B., Sleczka P., Braun B., Ornberg J., Kraus L. Who uses self-exclusion to regulate problem gambling? A systematic literature review. J. Behav. Addict. 2018;7(4):903–916. [PMC free article] [PubMed] [Google Scholar]

54. Nelson S.E., Kleschinsky J.H., LaPlante D.A., Shaffer H.J. Evaluation of the Massachusetts voluntary self-exclusion program: June 24, 2015 – November 30, 2017. Division on Addiction, Cambridge Health Alliance, Harvard Medical School; 2018. p. 215.https://massgaming.com/wp-content/uploads/VSEeval.pdf [Google Scholar]

55. Nong Z., Gainsbury S.M. Website design features: exploring how social cues present in the online environment may impact risk taking. Hum. Behav. Emerg. Technol. 2019;2(1):39–49. [Google Scholar]

56. Ospina-Pinillos L., Davenport T., Diaz A.M., Navarro-Mancilla A., Scott E.M., Hickie I.B. Using participatory design methodologies to co-design and culturally adapt the spanish version of the mental health eClinic: qualitative study. J. Med. Internet Res. 2019;21(8) [PMC free article] [PubMed] [Google Scholar]

57. Peek S.T.M., Luijkx K.G., Rijnaard M.D., Nieboer M.E., van der Voort C.S., Aarts S., van Hoof J., Vrijhoef H.J.M., Wouters E.J.M. Older adults’ reasons for using technology while aging in place. Gerontology. 2016;62(2):226–237. [PubMed] [Google Scholar]

58. Petry N.M. American Psychological Association; Washington, DC: 2005. Pathological Gambling: Etiology, Comorbidity, and Treatment. [Google Scholar]

59. Pickering D., Blaszczynski A., Gainsbury S.M. Multi-venue self-exclusion for gambling disorders: a retrospective process investigation. J. Gambl. Issues. 2018;38:127–151. [Google Scholar]

60. Pickering D., Nong Z., Gainsbury S.M., Blaszczynski A. Consumer perspectives of a multi-venue gambling self-exclusion program: a qualitative process analysis. J. Gambl. Issues. 2019;41:20–39. [Google Scholar]

61. Proctor E., Silmere H., Raghavan R., Hovmand P., Aarons G., Bunger A., Griffey R., Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. Policy Ment. Health. 2011;38(2):65–76. [PMC free article] [PubMed] [Google Scholar]

62. Productivity Commission . Vol. 1. 2010. Gambling: Productivity Commission inquiry report.http://www.pc.gov.au/inquiries/completed/gambling-2009/report No. 50. [Google Scholar]

63. Potenza M., Steinberg M., Skudlarski P., Fulbright R., Lacadie C., Wilber M., Rounsaville B., Gore J., Wexler B. Gambling urges in pathological gambling: a functional magnetic resonance imaging study. Arch. Gen. Psychiatry. 2003;60(8):828–836. [PubMed] [Google Scholar]

64. QSR International Pty Ltd. QSR International Pty Ltd.; 2015. NVivo qualitative data analysis version 11.0.https://www.qsrinternational.com/nvivo/home [Google Scholar]

65. Queirós A., Faria D., Almeida F. Strengths and limitation of qualitative and quantitative research methods. Eur. J. Educ. Stud. 2017;3(9):369–387. [Google Scholar]

66. Saldaña J. Sage Publications Ltd.; 2009. The coding manual for qualitative researchers. [Google Scholar]

67. Sardi L., Idri A., Fernández-Alemán J.L. A systematic review of gamification in e-health. J. Biomed. Inform. 2017;71:31–48. [PubMed] [Google Scholar]

68. Schrans T., Schellinck T., Grace J. Report submitted to the Nova Scotia Gaming Corporation. 2004. 2004 NS VL self-exclusion program process test: Final report. [Google Scholar]

69. Schreier M. SAGE Publishing; Thousand Oaks, California, US: 2014. Qualitative Content Analysis. [Google Scholar]

70. Schreier M. Sage; Los Angeles: 2012. Qualitative Content Analysis in Practice. [Google Scholar]

71. Schuler A., Ferentzy P., Turner N.E., Skinner W., Mcisaac K.E., Ziegler C.P., Matheson F.I. Gamblers anonymous as a recovery pathway: a scoping review. J. Gambl. Stud. 2016;32(4):1261–1278. [PMC free article] [PubMed] [Google Scholar]

72. Slattery P., Saeri A.K., Bragge P. Research co-design in health: a rapid overview of reviews. Health Res. Policy Syst. 2020;18:17. [PMC free article] [PubMed] [Google Scholar]

73. Slutske W.S. Natural recovery and treatment-seeking in pathological gambling: results of two U.S. national surveys. Am. J. Psychiatry. 2006;163(2):297–302. [PubMed] [Google Scholar]

74. Smith D., Pols R., Battersby M., Harvey P. The Gambling Urge Scale: Reliability and validity in a clinical population. Addiction Research & Theory. 2013;21(2):113–122. [Google Scholar]

75. Steen M.S., Manschot M.M., Koning N.D. Benefits of co-design in service design projects. Int. J. Des. 2011;5(2):53–60. [Google Scholar]

76. Suurvali H., Cordingley J., Hodgins D.C., Cunningham J. Barriers to seeking help for gambling problems: a review of the empirical literature. J. Gambl. Stud. 2009;25:407–424. [PubMed] [Google Scholar]

77. Swedish Swedish Gambling Authority (n.d.). Spelpaus Statistik [Game Break Statistics]. https://www.spelinspektionen.se/spelproblem1/spelpaus/spelpaus-statistik.

78. Tanner J., Drawson A., Mushquash C., Mushquash A., Mazmanian D. Harm reduction in gambling: a systematic review of industry strategies. Addict. Res. Theory Gambl. Policies Pract. 2017;25(6):485–494. [Google Scholar]

79. Thomas A., Carson R., Deblaquiere J., Armstrong A., Moore S., Christensen D., Rintoul A. Australian Institute of Family Studies; Melbourne: 2016. Review of Electronic Gaming Machine Pre-Commitment Features: Self-exclusion.https://aifs.gov.au/agrc/sites/default/files/publication-documents/agrc-precommitment-self-exclusion.pdf [Google Scholar]

80. Tremblay N., Boutin C., Ladouceur R. Improved self-exclusion program: preliminary results. J. Gambl. Stud. 2008;24(4):505–518. [PubMed] [Google Scholar]

81. Turner N.E., Shi J., Robinson J., McAvoy S., Sanchez S. Efficacy of a voluntary self-exclusion reinstatement tutorial for problem gamblers. J. Gambl. Stud. 2021 doi: 10.1007/s10899-021-09998-x. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

82. Van Gemert-Pijnen L., Kelders S.M., Kip H., Sanderman R. Routledge; Abbington: 2018. eHealth research, theory and development: A multi-disciplinary approach. [CrossRef] [Google Scholar]

83. Van Gemert-Pijnen J.E., Nijland N., van Limburg M., Ossebaard H.C., Kelders S.M., Eysenbach G., Seydel E.R. A holistic framework to improve the uptake and impact of eHealth technologies. J. Med. Internet Res. 2011;13 [PMC free article] [PubMed] [Google Scholar]

84. Van Velsen L., Wentzel J., Van Gemert-Pijnen J.E. Designing eHealth that matters via a multidisciplinary requirements development approach. JMIR Res. Protocols. 2013;2(1) [PMC free article] [PubMed] [Google Scholar]

85. Verlik K. Paper presented at the 7th European Conference on Gambling Studies and Policy Issues, Nova Gorica, Slovenia. 2008. Casino and racing entertainment centre voluntary self-exclusion program evaluation. July. [Google Scholar]

86. World Health Organization . 11th ed. 2019. International statistical classification of diseases and related health problems.https://icd.who.int/ New York, NY. [Google Scholar]

87. Zoom Video Communications Inc. Zoom meetings & Chat. Zoom Video Communications Inc. 2016. https://zoom.us/meetings

88. Zec S., Soriani N., Comoretto R., Baldi I. High agreement and high prevalence: the paradox of Cohen’s kappa. 

[bottom.htm]