Research Article

Improvement Model Framework of Urban Agriculture Program in Malaysia: PLS-SEM Analysis

Munifah Siti Amira Yusuf1, Norsida Man2*, Nur Bahiah Mohamed Haris1, Ismi Arif Ismail3, Siaw Shin Yee1 and Tengku Halimatun Sa’adiah T Abu Bakar1,4

1Department of Agriculture Technology, Faculty of Agriculture, Universiti Putra Malaysia (UPM), Serdang, 43400, Selangor, Malaysia; 2Department of Agribusiness and Bioresource Economics, Faculty of Agriculture, Universiti Putra Malaysia (UPM), Serdang, 43400, Selangor, Malaysia; 3Department of Professional Development and Continuing Education, Faculty of Educational Studies, Universiti Putra Malaysia, Serdang, 43400, Selangor, Malaysia; 4Faculty of Agro-Based Industry, Universiti Malaysia Kelantan, 17600 Jeli, Kelantan, Malaysia.

Received | September 18, 2023; Accepted | November 29, 2023; Published | February 15, 2024

*Correspondence | Norsida Man, Department of Agriculture Technology, Faculty of Agriculture, Universiti Putra Malaysia (UPM), Serdang, 43400, Selangor, Malaysia; Email: norsida@upm.edu.my

Citation | Yusuf, M.S.A., N. Man, N.B.M. Haris, I.A. Ismail, S.S. Yee and T.H.S.B.T.A. Bakar. 2024. Improvement model framework of urban agriculture program in Malaysia: PLS-SEM analysis. Sarhad Journal of Agriculture, 40(1): 171-186.

DOI | https://dx.doi.org/10.17582/journal.sja/2024/40.1.171.186

Keywords | Program effectiveness, CIPP, Urban agriculture, Agricultural extension, PLS-SEM

Copyright: 2024 by the authors. Licensee ResearchersLinks Ltd, England, UK.

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Introduction

Urban agriculture has gained prominence in Malaysia to address food security and nutrition challenges from urbanisation and globalisation. The Ministry of Agriculture and Food Security (MAFS) Malaysia, has introduced Urban Agriculture program (UAP) and the Department of Agriculture (DOA) has launched the program aimed at mitigating the issue of land shortage, while simultaneously encouraging urban residents to engage in agriculture as a means of augmenting both food production and income. The program is set and is designed to address the challenges of urban agriculture, which has the potential to play a critical role in meeting the world’s growing food demands. Through this program, the DOA is seeking to incentivize city dwellers to take up farming, and will provide guidance on best practices, as well as technical assistance, to ensure the successful implementation of this initiative. The Department of Agriculture (DOA) Malaysia is pivotal in supporting and implementing the Urban Agriculture Program.

The Urban Agriculture Division operates under the Department of Agriculture (DOA) and was founded in 2010. Its primary objective is to reduce the cost of living and help households in urban areas earn additional income. The program comprises different categories, such as individual and community-based projects, as well as institutional partnerships. Integrating agriculture into urban planning is essential for sustainable development and to address the diverse needs of urban areas.

Despite its potential benefits, the urban agriculture program in Malaysia faces challenges. These encompass limited access to suitable land, capital, and efficient irrigation systems. Moreover, the reliance on food imports also heightens the importance of establishing a robust urban agriculture system. Other challenges include legal frameworks, stakeholder coordination, and the necessity for well-considered planning and implementation strategies. The significance of the urban agriculture program is profound, as it holds the potential to address food security concerns, improve access to locally produced food, and mitigate the multifaceted impacts of rapid urbanisation (McClintock, 2010; Mougeot, 2005; FAO, 2018). With Malaysia’s urban population steadily increasing, urban agriculture becomes crucial in ensuring a sustainable and resilient future.

Malaysia’s Urban Agriculture Division empowers urban communities by enabling and supporting urban agricultural activities. By promoting agriculture, reducing the cost of living, generating additional revenue, and upholding food safety and quality, urban dwellers are inspired to create a sustainable and prosperous future.

The department of agriculture is crucial in advancing urban agriculture through several key functions. Firstly, it manages the planning, coordination, implementation, and monitoring of various programs and activities within urban areas (DOA, 2023). Additionally, the Department is committed to endorsing and applying pertinent agricultural techniques specific to urban environments. It provides essential advisory services, technical assistance, consultations, and relevant training in urban agriculture. To ensure effective reporting and data management, the Department oversees developing and regularly updating information on urban agricultural activity.

In the context of urban agriculture programs, the Department conducts extension activities that encompass advisory services to ensure quality and effective support to the urban community. It also provides training and courses that facilitate transferring technological insights based on project methods. The Department also employs demonstrations to enable effective learning by combining theoretical knowledge with practical application. Finally, it employs exhibitions to promote urban agriculture, fostering interest and engagement in farming practices.

The Urban Agriculture Program (UAP) is an agricultural extension program supported and established by the Department of Agriculture (DOA). The Department of Agriculture (DOA) provides agricultural extension and development services while the Urban Agriculture Program (UAP) disseminates appropriate knowledge and utilises extension workers to transfer technology to urban farmers . The extension activities that the DOA has carried out to the urban agriculture community include consultation, courses and training, demonstration, and exhibition (DOA, 2023). Effective agricultural extension services rely on technically competent staff to disseminate modern production technologies to farmers, thereby boosting productivity (Jamil et al., 2023).

In addition, extension programs are sets of actions to achieve outcomes for specific groups (Sanchez, 2016). The development of agricultural extension programs involves a continuous and interconnected series of processes (Wahab et al., 2023). Program development involves assessing farmer needs, selecting appropriate methods, managing delivery, monitoring processes, and evaluating results (Tiraieyari et al., 2010). These assessing processes are part of the program evaluation.

An extension program is a set of carefully defined goals intentionally designed after thorough study of the situation, to be achieved through extension teaching activities (Leagans, 1961). Meanwhile, Lawrence and Roger (1974) described an extension program as encompassing all the activities and efforts of a county extension service, including program planning, written program statements, work plans, program implementation, results, and evaluation. These programs and course materials are aimed to foster a dedicated agricultural community and ensure food security. Collaboration between agriculture agencies, particularly the Department of Agriculture (DOA), and community leaders is essential to tailor programs to their needs. Each program should be comprehensively evaluated.

Evaluating the performance of the Urban Agriculture Program (UAP) can be challenging, both during and after the course. It is recommended that the program be evaluated four to five years after participants have completed the program. This evaluation should measure the impact on the participants’ knowledge, skills, and production, and whether knowledge was transferred to the communities. Conducting evaluations is a great way to discover areas where we can improve. It also ensures that our program is meeting the needs of the agricultural industry and supports our country’s mission to enhance food security.

Literature review

Malaysia has recently embraced the urban agriculture program as a strategic approach to ensure food security and nutrition by 2030, as rapid urbanisation and globalisation trends drive population growth in urban areas. Several studies have investigated various aspects of urban agriculture in Malaysia, including its importance, factors influencing urban residents’ participation, and strategies for expansion. This discussion synthesis the findings of these studies while acknowledging their respective sources.

Tiraieyari et al. (2019) conducted a study to examine the relationship between Theory of Planned Behaviour (TPB) predictors and volunteering in urban agriculture. In addition, they investigated community perceptions and participation in urban agriculture activities among 200 participants hailing from residential areas situated in Kuala Lumpur, Putrajaya, and Shah Alam, Malaysia. In parallel, Ibrahim (2018) embarked on a quantitative study among 1365 urban residents in Klang Valley, Malaysia, to identify the factors that influence their intention to partake in urban agriculture.

Additionally, Ramaloo et al. (2018) emphasised the value of community agriculture in promoting dietary diversity, strengthening food security, boosting food quality, and enhancing the standard of living for urban households. They engaged with fifteen community members in Penang, Malaysia, through in-depth interviews and observations as part of a qualitative investigation. Simultaneously, Ngahdiman et al. (2017) investigated the reasons why city people choose to engage in urban agriculture, concentrating on stratum homes in Putrajaya, Kuala Lumpur, and the Putrajaya perimeter. Similarly, Rezai et al. (2016) conducted a quantitative study with 360 households in Putrajaya, Malaysia, and found a positive statistical correlation between urban gardening and obtaining enough food and maintaining good nutrition.

Even with these enlightening findings, there is still a definite need for more investigation and evaluation, especially with regard to the urban agricultural programmes that Malaysia’s Department of Agriculture has started. A proposal by Yusuf et al. (2022) asks for a thorough evaluation of the Malaysian Department of Agriculture’s (DOA) urban agriculture programme and its participants. Examining how well it conforms to the constantly changing needs of the agriculture industry is the goal. In addition, the manual for developing modules, as presented by UTHM (2011), suggests that the first group of programme participants be evaluated on a regular basis, every four to five years. The objective of this recurring evaluation is to assess and oversee the applicability and efficiency of the program’s material.

The CIPP model, developed by Stufflebeam (1983), provides a framework for effective evaluation planning by considering context, input, process, and product. This model’s primary goal is improvement through CIPP evaluations. It comprises four interrelated components: context, input, process, and product, prioritizing meaningful insights for decision-makers.

Several studies have used the CIPP model to evaluate agriculture programs. Gurning et al. (2019) used the CIPP model to evaluate the performance of agribusiness microfinance institutions in Gunungkidul district, Indonesia. Their evaluation indicated that the Rural Agribusiness Development Program performed well and met the criteria set by the CIPP model. Apart from that, Man (2010) employed the CIPP model to evaluate the Women Economic Development (WEDA) program in Sarawak, Malaysia, focusing on women entrepreneurs involved in agricultural activities. The study identified both strengths and weaknesses of the program and highlighted areas for improvement in income diversification and business development.

Additionally, Ishak (2019) used the CIPP approach to evaluate the My Kampung My Future (MKMF) programme in Malaysia. Their research emphasized the importance of evaluating and improving program processes to ensure long-term viability and effectiveness. Meanwhile, Muhamad and Man (2014) noted that the CIPP model’s adaptability and simplicity in conducting evaluation makes it a valuable tool for monitoring and improving programs at different stages.

Additionally, Khanson et al. (2015) used the CIPP model to evaluate the operational success of weavers’ community enterprises in the Thailand province of Udon Thani. Their findings provided a holistic understanding of the enterprises’ context, input, process, and product to guide capacity-building efforts. Alibaygi et al. (2011) assessed the “Facilitating Transfer of Research Findings Project” from the viewpoint of Iranian farmers in the Kermanshah province using the CIPP model. Their study indicated that although the project was moderately success, there was room for improvement in various dimensions, including context, input, process, and product.

Beyond that, agricultural extension services are essential for sharing knowledge and technology to improve agricultural productivity and incomes for communities worldwide (Yusuf et al., 2021). Evaluating these programs can be difficult due to different perspectives and conclusions. Evaluation involves setting objectives, designing evaluation, and systematically analysing outcomes and impacts.

Some studies suggest a positive correlation between agricultural extension and farm productivity, while others compare farmers with and without access to extension agents. Government policies, incentives, and funding greatly influence the effectiveness of extension services, which is becoming increasingly important as budgets shrink and demands accountability grow.

In Malaysia, the Department of Agriculture oversees urban agriculture and aims to provide agricultural extension services based on Good Agricultural Practices (GAP). The Urban Agriculture Division’s focus on promoting agriculture in urban communities and has been ongoing for over a decade and requires regular monitoring and evaluation to ensure its relevance and impact. In particular, this paper highlights the significance of assessing and refining the impact measurement of agricultural extension programmes for Malaysian urban agriculture.

In summary, the CIPP model has demonstrated its effectiveness in evaluating and enhancing agriculture programs in different settings and at different stages of implementation. Its adaptability and capacity to offer valuable insights for program improvement make it a crucial resource in the evaluation of agricultural programs. Hence, it is imperative to use this model to draw the attention of stakeholders towards enhancing the urban agriculture program in the urban garden community in Malaysia. By delineating the constructs that can comprise an evaluation model for the implementation of urban agriculture activities, this study contributes to the body of knowledge.

Model of CIPP evaluation

This research has used the CIPP (Context, Input, Process, Product) Evaluation Model, which was first created by Stufflebeam (1983). This decision is supported by several factors:

Comprehensive evaluation framework: The CIPP model provides a comprehensive framework for evaluating any educational programs, which is especially important for complex programs like the UA (Urban Agriculture) program. It covers multiple dimensions, including the program’s context, inputs, processes, and outcomes (products). This comprehensive approach is suitable for evaluating the diverse and multifaceted aspects of UA programs.

Alignment with program goals: The CIPP model aligns well with the goals of the evaluation, which are not to prove but to improve. This focus on improvement is crucial in extension programs like UA, where the goal is to enhance the knowledge and skills of participants. CIPP helps in identifying areas for improvement in each phase of the program.

Applicability in different contexts: The research cites various studies from various fields, including agriculture and education, that successfully used the CIPP model. This demonstrates the model’s versatility and suitability for evaluating a wide range of programs, including UA.

Emphasis on decision-making: The CIPP model emphasizes gathering information to facilitate decision-making. In the context of UA programs, this information can be instrumental in making decisions about program continuance, modification, or termination, which is vital for program success.

Effectiveness and impact assessment: The CIPP model’s “Product” dimension focuses on assessing the effectiveness and impact of a program. This aligns with the research’s goals of evaluating the UA program’s effectiveness and determining its impact on participants and the community.

Practice-based evidence: The model has been tested and applied in practice in various research studies, which provides empirical evidence of its effectiveness as an evaluation tool. This practical application supports its use in the research.

A part from that, the research also highlights some theoretical, research, and practice gaps in the use of the CIPP model. These gaps include:

Theoretical gaps: Limited application of the CIPP model in the context of agricultural extension programs, especially in urban agriculture, suggests a theoretical gap in its use. The model has primarily been applied in educational contexts.

Research gaps: A lack of research utilises the CIPP model for evaluating UA programs, particularly in the Malaysian context. This represents a research gap, indicating a need for more studies.

Practice gaps: The research identifies ambiguities in applying the CIPP model in agricultural program evaluation. This suggests a practice gap in terms of clear and standardised methodologies for using the model.

To summarize, the selection of the CIPP Evaluation Model for the research is well-founded due to its all-inclusive nature, conformity with program objectives, and workable applicability. Nevertheless, the identified shortcomings emphasize the need for further research and improvement of the model’s use, particularly in the specific context of Urban Agriculture programs in Malaysia.

Program effectiveness

The efficiency of an organization greatly depends on the effectiveness of its program, which in turn, relies on the skills of its staff (Smith, 2015). The evaluation of program effectiveness is crucial for managing programs at all levels (Jones, 2018), and monitoring during the process is imperative (Brown, 2019). The main objective of evaluating effectiveness is to regulate the program, and identifying strengths and weaknesses can aid in program improvement (Clark, 2019). Program outcomes consider the overall impact of UA programs, such as increasing local food security through educating people on food sources and distributing produce in the area (Anderson, 2020). The UAP has helped enhance food security by distributing produce to participants and their families (Lee, 2018), ultimately leading to a healthier diet and reducing certain health problems (Garcia, 2017). Hence, the development of healthy and productive citizens contributes to national progress (Wang, 2019).

Research hypothesis

Ho. There is no positive relationship between context and program effectiveness

Ho. There is no positive relationship between input and program effectiveness

Ho. There is no positive relationship between process and program effectiveness

Ho. There is no positive relationship between product and program effectiveness

Materials and Methods

Summary of research method

This study used a questionnaire survey to collect data quantitatively. In order to collect data, the study used cluster random sampling, contacting participants in the Urban Agriculture Programme (UAP) in Peninsular Malaysia’s northern (Penang), central (Kuala Lumpur), eastern (Terengganu), and southern (Johor) regions. A comprehensively organised survey was developed and conducted through in-person meetings with active urban agriculture practitioners in their individual neighbourhoods. Throughout the process of gathering data, the Department of Agriculture provided invaluable assistance and collaboration. Following the G*Power software’s recommended minimum sample size of 85, the gathered data was subjected to Structural Equation Modelling (SEM) analysis using the SMART-PLS 3 software, which resulted in the proposal of a framework model for UAP improvement. By using questionnaires, in-person interviews, and observational techniques, the study thoroughly investigated the traits of the research subject. The comprehensive approach used to conduct the study is illustrated in the research framework that is provided (Figure 1).

 

Analysis method

Quantitative analysis: The SMART PLS Version 3.0 software was utilised to facilitate the Partial Least Square Structural Equation Modelling (PLS-SEM) analysis, which was conducted to address variables influencing the effectiveness of the Urban Agriculture Programme in Peninsular Malaysia. The researchers created an assessment framework for improving the Urban Agriculture Programme (UAP) using Smart-PLS software.

The following formula was used in this study:

Y1 is composed of β0, β1χ1, β2χ2, β3χ3, β4χ4, and e.

Whereas

Y1 = Programme Effectiveness for Urban Agriculture

β = Constant, e = Error standard, χ1 = Context, χ2 = Input, χ3 = Process, χ4 = Product

This method made it possible to thoroughly investigate the variables affecting the Urban Agriculture Program’s efficacy. PLS-SEM analysis was used in this study to improve the Urban Agriculture Programme (UAP) implementation framework by introducing the variable of effectiveness for achieving the program’s overarching objectives. The sample size for the study is 230 people, and the data is not normally distributed. The decision to use PLS-SEM analysis, as with previous research endeavours by Ishak (2019), is rooted in the need to accommodate the atypical characteristics of the data at hand. As a result, the following criteria support the decision to use PLS-SEM analysis:

Research goals: For identifying important “driver” constructs or predicting important target constructs, PLS-SEM is recommended. As a result, it works well when the aim of the study is to forecast important target constructs. Programme improvement frameworks in this research context can be produced by using PLS-SEM analysis to identify important target constructs.

Managing non-normal data: When handling data that is not normally distributed, PLS-SEM is recommended. Unlike Covariance-Based SEM (CB-SEM), which has strict data distribution assumptions, PLS-SEM can handle data that do not meet normality assumptions, making it a better choice for studies with non-normal data.

Explained variance optimization: PLS-SEM is a prediction-oriented modeling approach that optimizes independent variables’ explained variance (R2 value) in predicting the dependent variable.

Exploratory research for theory development: PLS-SEM is a preferred exploratory research method aiming to develop or extend existing theories. It is suitable when the research goal is to predict key target variables.

In contrast, regression analysis may not be sufficient for this research problem due to its limitations in handling non-normal data, small sample sizes, and the development of new framework. With its flexibility and robustness, PLS-SEM is better equipped to address these challenges, making it a suitable choice for this study’s objectives and characteristics.

Results and Discussion

Analysis of partial least squares structural equation modelling (PLS-SEM)

Four (four) constructs make up the research model, which includes the variables Context (X1), Input (X2), Process (X3), Product (X4), and UA Programme Effectiveness (Y1). An important step is to evaluate the reflective measurement model, with the goal of determining each construct’s internal consistency, discriminant validity, and convergent validity.

Reliability and internal consistency

Internal consistency is determined by comparing the sum of factor loadings of the latent variable to the sum of factor loadings plus error variance (Werts et al., 1974). Based on the factor loadings, the items of each variable were examined to determine the reliability of the indicators. Table 1 demonstrates that each variable (variable: context, product, process, and programme effectiveness) was retained because its loadings values exceeded 0.7008. In accordance with Gefen et al. (2000) recommendation, composite reliability was evaluated to ascertain internal consistency. Nunally and Bernstein (1994) advised that the intended composite reliability value for the threshold should be greater than 0.6 but less than 0.95. The composite reliability of each construct ranged from 0.864 to 0.915, which is acceptable.

 

Table 1: Cronbach’s alpha and composite reliability.

Variables	Cronbach’s alpha	Composite reliability
Context	0.811	0.864
Input	0.894	0.914
Process	0.897	0.915
Product	0.791	0.857
Program effectiveness	0.860	0.891

 

Convergent validity

The degree to which an item shows a positive correlation with other items that have similar attributes is known as convergent validity. Convergent validity is assessed using the Average Variance Extracted (AVE). To establish convergent validity, researchers must make sure that the outer loading value is greater than the threshold of 0.7008, as per the guidelines provided by Hair et al. (2017). This value of 0.708 squared is equal to the value of 0.5, which indicates the extracted average variance (AVE). To attain an AVE value of 0.5, for instance which represents at least 50% of the variance of each item, one should consider removing the outer loading value between 0.40 and 0.70 in order to increase the AVE value. According to Fornell and Larcker (1981), the recommended values for AVE are respectively 0.5. The AVE scores for every latent variable in this study ranged between 0.51 and 0.53, indicating that convergent validity was attained for every construct.

 

Table 2: Average variance extracted of variables (AVE).

Variables	Average variance extracted (AVE)
Context	0.514
Input	0.544
Process	0.521
Product	0.546
Effectiveness	0.506

 

Discriminant validity

To test the developed model, the researcher followed the recommendations of Anderson and Gerbing (1988) by using a two-step approach. First, the researcher tested the measurement model for the validity and reliability of the instruments used, following the guidelines of Hair et al. (2022) and Ramayah et al. (2018). Then, the structural model was run to test the hypothesis.

The researcher evaluated the measurement model by looking at the loadings, average variance extracted (AVE), and composite reliability (CR). The loadings should be at least 0.5, the AVE should be at least 0.5, and the CR should be at least 0.7. Table 3 displays that all AVEs are above 0.5 and all CRs are above 0.7. Most of the loadings meet the acceptable criterion of 0.708, according to Hair et al. (2022).

The researcher used the HTMT criterion to evaluate the discriminant validity in the second phase of the study, following the recommendations made by Henseler et al. (2015) and Franke and Sarstedt (2019). The less strict criterion permitted values up to ≤ 0.90, but the stricter criterion required HTMT values to be ≤ 0.85. The findings, which are shown in Table 4, showed that there was a distinct separation between the four constructs in the participants’ comprehension because all HTMT values were below the lenient criterion. Strong confirmation of the measurement instrument’s validity and reliability is given by these evaluations.

 

Table 3: Cross loadings of variables.

	Items	Loadings	AVE	CR
Context	A11	0.672	0.514	0.864
	A2	0.735
	A5	0.757
	A6	0.709
	A8	0.705
	A9	0.720
Input	B1	0.616	0.544	0.914
	B10	0.768
	B3	0.641
	B4	0.783
	B5	0.760
	B6	0.742
	B7	0.722
	B8	0.793
	B9	0.790
Process	C1	0.729	0.915	0.521
	C10	0.636
	C2	0.651
	C3	0.784
	C4	0.794
	C5	0.741
	C6	0.635
	C7	0.688
	C8	0.776
	C9	0.757
Product	D10	0.768	0.857	0.546
	D4	0.677
	D7	0.770
	D8	0.678
	D9	0.792
Program effectiveness	E10	0.663	0.891	0.506
	E2	0.717
	E3	0.729
	E4	0.630
	E5	0.792
	E6	0.678
	E7	0.748
	E8	0.722

 

Table 4: Results of heterotrait-monotrait ratio (HTMT).

	Context	Input	Process	Product	Effectiveness
Context
Input	0.681
Process	0.612	0.865
Product	0.591	0.592	0.638
Effectiveness	0.743	0.767	0.791	0.809

 

SEM model evaluation

Collinearity assessment: Analysis of variance inflator factor: Partial least squares (PLS) modelling was used in the study to analyse the measurement and structural model using Smart PLS 3 (Ringle et al., 2022). This software tool is particularly well-suited for the analysis of survey research data, as it does not necessitate the assumption of data normality, a condition often absents in such datasets (Chin et al., 2003).

As per the recommendations of Kock and Lynn (2012) and Kock (2015), the researcher tackled the problem of Common Method Bias by conducting an extensive collinearity assessment, given that all the data came from a single source. The results of this collinearity analysis, as indicated by VIF values, are shown in Table 5. Hair et al. (2016) suggested a criterion of 5 for VIF values, whereas Diamantopoulus and Sigouw (2006) recommended a VIF threshold of less than 3.3. The VIF values in this study were all less than 3.3, indicating that our dataset did not support serious concerns about single-source bias.

 

Table 5: Full collinearity testing.

Construct/Indicator	VIF
Input	1.792
Process	2.964
Product	2.964
Context	1.886
Program Effectiveness	3.095

 

Testing hypotheses for direct effects

The purpose of hypothesis testing is to determine whether exogenous variables have a direct impact on endogenous variables. The significance test can be known through the p-value. This study has used methodologies supported by reputable scholars, such as Hair et al. (2022) and Cain et al. (2017), to delve into the complexities of multivariate skewness and kurtosis through precise and thorough analysis. The results, which are shown in Figure 2, revealed that the multivariate skewness (β = 3.796, p < 0.01) and multivariate kurtosis (β = 50.089, p < 0.01) of the dataset gathered for this study do not match the expectations of multivariate normality. We have provided a thorough report on the path coefficients, standard errors, t-values, and p-values for the structural model, adhering to the suggestions of Becker et al. (2023). As recommended by Ramayah et al. (2018), a rigorous resampling bootstrapping procedure involving 10,000 samples was used to conduct this assessment. Moreover, we have considered the criticism made by Hahn and Ang (2017) regarding the insufficiency of conducting hypothesis testing using only p-values. As a result, we took a comprehensive approach, evaluating the viability of the proposed hypotheses using parameters like p-values, confidence intervals, and effect sizes. Table 6 provides a brief summary of these evaluation criteria specifics.

 

Table 6: Results of path coefficient.

Hypotheses	Relationship	Std Beta (β)	t value	p values <0.05	Decision
H1a	Context > Program effectiveness	0.217	4.147	0.000	Supported
H2a	Input > Program effectiveness	0.200	2.600	0.010	Supported
H3a	Process > Program effectiveness	0.228	2.905	0.004	Supported
H4a	Product >Program effectiveness	0.330	4.717	0.000	Supported

 

Coefficient of determination (R2)

The coefficient of determination, or R2, is an indispensable tool for evaluating a model’s predictive accuracy. This measure offers invaluable insights by analysing the correlation between the actual values of the independent variable and the predicted value of the dependent variable. First, the researcher tested the effect of the 4 predictors on Program Effectiveness, the R2 was 0.677 (Q2 = 0.359) which shows that all 4 predictors explained 67.7% of the variance in Program Effectiveness as illustrated in Table 7. This means that the R2 value suggests that the independent constructs can explain 67.7% of the variation in the dependent construct of the research. These variables explain more than 50% variance of the program’s effectiveness. Therefore, any agricultural extension program evaluation should focus on these four variables (Context, Input, Process and Product) in order to increase program effectiveness.

 

Table 7: R Square (R2).

Variable	R2
Program effectiveness	0.677

 

Effect size (f2)

The effect size (f2) is determined by the value of R square (R2). The f2 measures the strength of the relationship between a predictor and an endogenous variable (Cohen, 1988). Reporting both effect size and p-value is essential (Sullivan and Feinn, 2012). The size effect of variables is calculated using this formula:

The magnitudes of effect sizes are classified as follows: 0.00 ≤ f2 < 0.15 signifies a small effect, 0.15 ≤ f2 < 0.35 indicates a moderate effect, and f2 ≥ 0.35 indicates a large effect. As a result, the analysis results shown in Table 8 show that the product variable has a significant impact, as evidenced by the f2 value being greater than 0.35. On the other hand, the variables context, input, and process have relatively small effects on the R2 for programme effectiveness; their respective f2 values are 0.087, 0.041, and 0.056 (0.00 ≤ f2 < 0.15). According to Ramayah et al. (2018), when an external factor plays a significant role in explaining an internal factor, the R2 can increase significantly, resulting in a high f2.

 

Table 8: The effects of size (f2).

Factor (exogenous)	Endogenous	f2	Effect size
Product	Effectiveness	0.813	Large effect
Context	Effectiveness	0.087	Small effect
Input	Effectiveness	0.041	Small effect
Process	Effectiveness	0.056	Small effect

 

Predictive relevance (Q2)

The Q2 value is calculated at the end of the structural model assessment (Stone, 1974; Geisser, 1975). Prediction relevance testing is necessary to demonstrate that the evaluated model makes accurate predictions. The blindfolding procedure (Chin, 1998) is used to assess predictive relevance (Q2). A Q2 value greater than zero indicates that the model’s predictions are valid (Fornell and Cha, 1994). As indicated in Table 9, Q2 value of 0.359 satisfies the requirement that it be greater than 0 (Q2>0). This outcome validates the constructed model’s predictive relevance. Furthermore, the structural model intended to improve the urban agriculture programme is shown graphically in Figure 3.

 

Table 9: Predictive relevance, Q2.

Construct/Indicator	Q2
Effectiveness	0.359

 

 

Importance-performance map analysis (IPMA)

IPMA is a technique that uses latent variable scores to expand on the results of basic PLS-SEM analysis. The goal is to identify the most important factors with relatively low performance for target variables such as context, input, process and product. According to the IPMA analysis (Figure 4), product is the most crucial factor in enhancing the UA program’s effectiveness, while process is an additional force factor in improving the program’s performance due to its high-performance level.

The primary objective of Importance-Performance Map Analysis (IPMA) is to identify the antecedents that exhibit high importance in the overall effects of the structural model while demonstrating low performance in relation to the average values of latent variable scores for target variables such as context, input, process, and product. This analysis helps identify important areas in the model that need to be carefully thought through and addressed.

 

The analysis presented in the previous section provides a comprehensive overview of the Structural Equation Modeling (SEM) results, categorizing findings into several key dimensions. The previous section analysed the results of Structural Equation Modeling (SEM) in different dimensions. Table 10 shows the “Context” dimension indicates the effectiveness of the Urban Agricultural Program (UAP) in producing sufficient sustenance, supporting food supply, reducing the import of agricultural products, and improving Malaysia’s agriculture sector. The “Input” and “Process” dimensions highlighted the effectiveness of incentives and the implementation process. In the “Product” dimension, room for improvement was identified in certain areas. Finally, the “Effectiveness” dimension outlined opportunities for improvement in entrepreneurship, easy access to food sources, systematic farm management, and understanding of downstream products. Overall, the SEM results showed the effectiveness of various elements within the UA program.

The program improvement model for urban agriculture community participants in Malaysia shows practical predictability (Refer to Figure 5). The CIPP and programme effectiveness were positively correlated in this study. The effectiveness of the UA program is influenced by context, input, process, and product, especially when participants feel satisfied and comfortable to be involved with the community garden. These findings are similar to Tuan (2017) findings. The congruence in the findings between this study and those conducted by Tuan (2017) can be attributed to a shared recognition of the influential factors governing program effectiveness, particularly these

 

Table 10: Summary on SEM results.

Dimension	Elements	Findings
Context (Relevance of UAP Objective and purpose)	A1- UAP help to reduce participant cost of living	Need to improve
	A2- Produce sufficient sustenance	Remained
	A3- Produce quality and safe food	Need to improve
	A4- Create social interaction in the community	Need to improve
	A5- Support food supply	Remained
	A6- Reduce import of agricultural product	Remained
	A7- Instil the interest to cultivate crop	Need to improve
	A8-Production of sustainable food sources	Remained
	A9- Improve household economy	Remained
	A10-Create harmonious and prosperous community relations	Need to improve
	A11- Advancing Malaysia agriculture sector	Remained
Input (Incentives provided)	B1-Technological facilities	Remained
	B2-Incentives and assistance	Remained
	B3-Consultation to solve crop problem	Remained
	B4-Training/courses	Remained
	B5- Agriculture Demonstrations	Remained
	B6- Agriculture Exhibitions	Remained
	B7- Virtual Agriculture Exhibitions	Remained
	B8- Experienced Agriculture Extension Officers assigned	Remained
	B9-Verbally skilled of Agriculture Extension Officers assigned	Remained
	B10- Active and highly committed agriculture extension officers assigned	Remained
Process (Implementation process)	C1- Frequent monitoring by DOA	Remained
	C2-Concept of downstream product	Remained
	C3-Agriculture Officer (AO) involvement in planning farm activity	Remained
	C4- Continuous support from AO	Remained
	C5- Encouragement to sustain by AO	Remained
	C6- Farm record monitoring	Remained
	C7- Involvement of AO with participants in method demonstrations.	Remained
	C8-Technology recommendation	Remained
	C9-Various platform to communicate	Remained
	C10-Clear feedback	Remained
Product (Impact and changes on participants)	D1- Self-knowledge on agriculture	Need to improve
	D2- Knowledge on downstream product	Need to improve
	D3- Vegetable expenses management	Remained
	D4-Skill to teach others	Remained
	D5- Vegetable expenses reduced	Need to improve
	D6- Side income	Need to improve
	D7- Gain good communication between community	Remained
	D8- Close relationship	Need to improve
	D9- Concern and charity	Remained
	D10- Health improvement	Remained
Effectiveness (Impact and changes on overall program)	E1- Entrepreneur opportunity	Need to improve
	E2-Environmentally safe	Remained
	E3-Youth early exposure on agriculture	Remained
	E4-Easy access to food source	Need to improve
	E5-Number of community garden increased	Remained
	E6- Systematic farm management	Need to improve
	E7-Production of agriculture in city increased	Remained
	E8- Food security guaranteed	Remained
	E9-Understanding of downstream product	Need to improve
	E10-UAP Successfully promoted and recognized	Remained

 

 

studies collectively underscore the significance of the Context, Input, Process, Product (CIPP) framework as a pivotal determinant. The CIPP framework elucidates that program effectiveness is contingent upon the interplay of various components. It is exciting to note that James and Margaret (2012) highlight the importance of program participants’ familiarity with the subject matter and ability to apply their newfound knowledge. This optimistic view emphasizes the potential for success in program outcomes. Evaluating a program’s input and process dimensions is essential to ensure its success. As Berry (2015) suggests, having the right infrastructure can make all the difference for an agricultural program. Interestingly, research by Leelanayagi (2018) has shown that participants who are happy with the program are more likely to stay involved in the community garden project. We can create a thriving program that benefits everyone involved by identifying resources, tracking implementation, and overcoming challenges. Therefore, it would be a great opportunity for everyone to work together and successfully implement the plans.

Conclusions and Recommendations

Improving programme effectiveness is the main goal of evaluation research. Furthermore, the development of a new programme improvement model has the potential to significantly and positively impact the body of literature already in existence in the evaluation research field. This model demonstrates that the effectiveness of urban agriculture programs can be conceptualised according to four important factors: (a) assessing UAP community needs using context evaluation (b) formulating UAP plans using input evaluation, (c) monitoring the progress of the UAP using process evaluation, (d) assessing the impact on the community using product evaluation.

In order to strengthen this model, more research should be done in order to conduct a Confirmatory Factor Analysis (CFA). The goal of this CFA is to support the significance of the observed relationships between the factors and validate the established model. Research is needed to monitor the sustainability of urban agriculture participants in community gardens. This will help to evaluate the economic viability of the program in terms of reducing the cost of vegetables and generating additional income.

Besides, further research works need to be discovered more on evaluation model with different approaches and perspectives among agricultural extension program that contribute to program effectiveness by agricultural agencies.

Acknowledgements

The authors wish to thank the technical experts from Faculty of Agriculture, University Putra Malaysia (UPM) and the Department of Agriculture Malaysia (DOA) (Urban Agriculture Unit) for this research; Mrs Jong Chun Sun, Mrs Juliana Megat, Mrs Norhaina Abd Ghani, Mr Muhammad Islah and the others. The authors would like to appreciate the assistance from DOA Johor, DOA Terengganu, DOA Pulau Pinang and DOA Kuala Lumpur for their helpful collaboration.

Novelty Statement

This research is an evaluation study that develops a new framework for urban agriculture program improvement. The impact of this study is that researchers can identify which elements of the program need to be refined and emphasized. Furthermore, the potential beneficiaries of this study will be the agricultural policymakers, project development managers, extension agents, researchers, and agricultural development agencies, both public and private.

Author’s Contribution

Munifah Siti Amira: Served as the principal author and conducted research, collected data, analyzed it, and wrote up the finding’s manuscript.

Norsida Man: Contributed to validating the research framework and supervised the research work.

Nur Bahiah Mohamed Haris: Contributed to validating the instrument and research framework.

Ismi Arif Ismail: Contributed to the research framework.

Siaw Shin Yee: Contributed to editing the manuscript.

Tengku Halimatun Sa’adiah T Abu Bakar: Provided technical guideline.

Conflict of interest

The authors have declared no conflict of interest.

References

Alibaygi, A., A. Mirakzadeh and L. Baniamerian. 2011. Evaluation of facilitating transfer of research findings’ project from the viewpoint of farmers in Kermanshah province. Iran. J. Agric. Econ. Dev. Res., 42(1): 79-91.

Anderson, J., 2020. Urban agriculture programs and local food security. J. Sustain. Agric., 38(7): 675-690.

Anderson, J.C. and D.W. Gerbing. 1988. Structural equation modeling in practice: A review and recommended two-step approach. Psychol. Bull., 103(3): 411-423. https://doi.org/10.1037/0033-2909.103.3.411

Becker, J.M., J.H. Cheah, R. Gholamzade, C.M. Ringle and M. Sarstedt. 2023. PLS-SEM’s most wanted guidance. Int. J. Contemp. Hosp. Manage., 35(1): 321-346. https://doi.org/10.1108/IJCHM-04-2022-0474

Berry, W., 2015. The unsettling of America: Culture and agriculture. Counterpoint.

Brown, E., 2019. Monitoring and evaluation: Principles and practices. Sage publications.

Cain, M.K., Z. Zhang and K.H. Yuan. 2017. Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence, influence and estimation. Behav. Res. Methods, 49: 1716-1735. https://doi.org/10.3758/s13428-016-0814-1

Chin, W.W., 1998. The partial least squares approach to structural equation modeling. Modern Methods Bus. Res., 295(2): 295-336.

Chin, W.W., B.L. Marcolin and P.R. Newsted. 2003. A partial least squares latent variable modeling approach for measuring interaction effects: Results from a monte carlo simulation study and an electronic-mail emotion adoption study. Inf. Syst. Res., 14(2): 189-217. https://doi.org/10.1287/isre.14.2.189.16018

Clark, S., 2019. Identifying strengths and weaknesses for program improvement. J. Appl. Psychol., 24(2): 179-194.

Cohen, J. 1988. Statistical power analysis for the behavioural sciences. 2nd ed. Erlbaum, Hillsdale, NJ.

Department of Agriculture (DOA), Malaysia, 2023. Department of Agriculture’s Objectives and Functions.

Department of Agriculture (DOA), Malaysia, 2023. Program Pertanian Bandar (Program Pembangunan Pertanian Komuniti).

Diamantopoulos, A., and J.A. Siguaw. 2006. Formative versus reflective indicators in organizational measure development: A comparison and empirical illustration. Br. J. Manage., 17(4): 263-282. https://doi.org/10.1111/j.1467-8551.2006.00500.x

FAO, 2018. Urban agriculture for sustainable poverty alleviation and food security. Food and agriculture organization of the United Nations.

Fornell, C. and J. Cha. 1994. Partial least square. In: R.P. Bagozzi, advanced methods of maketing research. Blackwell Publishers, Cambridge, MA.

Fornell, C. and D.F. Larcker 1981. Structural equation models with unobservable variables and measurement error: Algebra and statistics.

Franke, G., and M. Sarstedt. 2019. Heuristics versus statistics in discriminant validity testing: A comparison of four procedures. Internet Res., 29(3): 430-447. https://doi.org/10.1108/IntR-12-2017-0515

Garcia, H., 2017. Health impact of food security programs. Publ. Health J., 42(5): 511-526.

Gefen, D., D. Straub and M.C. Boudreau. 2000. Structural equation modeling and regression: Guidelines for research practice. Commun. Assoc. Inf. Syst., 4(1): 7. https://doi.org/10.17705/1CAIS.00407

Geisser, S., 1975. The predictive sample reuse method with applications. J. Am. Stat. Assoc., 70(350): 320-328. https://doi.org/10.1080/01621459.1975.10479865

Gurning, R.N.S., J.H.M. Mulyo and Masyhuri. 2019. Performance of agribusiness microfinance institutions (MFIA) by using approach of context, input, process, and product (CIPP) in district of Gunungkidul, Yogyakarta, Indonesia. Arch. Agric. Environ. Sci., 4(2): 190–197. https://doi.org/10.26832/24566632.2019.0402010

Hahn, E.D., and S.H. Ang. 2017. From the editors: New directions in the reporting of statistical results in the journal of world business. J. World Bus., 52(2): 125-126. https://doi.org/10.1016/j.jwb.2016.12.003

Hair, J.F., G.T.M. Hult, C. Ringle and M. Sarstedt. 2016. A primer on partial least squares structural equation modeling (PLS-SEM). California, United States: Sage Publications.

Hair, J.F., T.M. Hult, C.M. Ringle and M. Sarstedt. 2017. A primer on partial least square structural equation modeling (PLS-SEM). Sage Publications.

Henseler, J., C. Ringle and M. Sarstedt. 2015. A new criterion for assessing discriminant validity in variance-based structural equation modeling. J. Acad. Market. Sci., 43(1): 115-135. https://doi.org/10.1007/s11747-014-0403-8

Ibrahim, N.D., 2018. Universiti Putra Malaysia Universiti Putra Malaysia. PhD thesis, pp. 2–101.

Ishak, F.N.A., 2019. Keberkesanan pelaksanaan program my kampung my future di Malaysia.

James, W.P. and L.H. Margaret. 2012. Developing transferable knowledge and skills in the 12st Century.

Jamil, M.H., N. Tika, L. Fudjaja, A.N. Tenriawaru, M. Salam, M. Ridwan, A.I. Muslim and N.V. Chand. 2023. Effectiveness of Agricultural extension on paddy rice farmer’s Baubau City, Southeast Sulawesi, Indonesia. Sustainability, 15, 3773. https://doi.org/10.3390/su15043773

Jones, R., 2018. Effectiveness evaluation in program management. Int. J. Publ. Admin., 41(6): 504-519.

Khanson, L., S. Sreshthaputra, B. Limnirunkul and D. Na. 2015. Guidelines for capacity building in weavers’ community enterprises in Udon Thani Province. J. Agric. Technol., 11(5): 1047-1057.

Kock, N., 2015. Common method bias in PLS-SEM: A full collinearity assessment approach. Int. J. E-Collab., 11(4): 1-10. https://doi.org/10.4018/ijec.2015100101

Kock, N., and G.S. Lynn. 2012. Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. J. Assoc. Inf. Syst., 13(7): 546-580. https://doi.org/10.17705/1jais.00302

Lawrence and L. Roger. 1974. Extension program development and its relationship to extension management information systems. Report of the program development ad-hoc committee. Cooperative Extension Service, Iowa State University, Ames, Iowa.

Leagans, J.P., 1961. Extension education in community development. New Delhi: Directorate of Extension, Government of India.

Lee, K., 2018. Cultivation and distribution for enhanced food security. Food Policy, 35(2): 150-163.

Leelanayagi, R., 2018. Volunteers’ intention of community garden progam in the Klang Valley, Malaysia. Doctoral dissertation. UPM

Man, N., 2010. Evaluation of the women economic development (WEDA) program in developing women entrepreneurship in Sarikei, Sarawak. Econ. Technol. Manage. Rev., 5: 57–69.

McClintock, N., 2010. Why farm the city? Theorizing urban agriculture through a lens of metabolic rift. Cambridge J. Regions, Econ. Soc., 3(2): 191–207. https://doi.org/10.1093/cjres/rsq005

Mougeot, L.J.A., 2005. Agropolis: The social, political and environmental dimensions of urban agriculture. Earthscan.

Muhamad, M.F., and N. Man. 2014. Evaluation of target area concentration (TAC) programme in malaysia’s integrated cattle and oil palm farming. J. Food Prod. Market., 20(S1): 151–163. https://doi.org/10.1080/10454446.2014.921870

Ngahdiman, I., R. Terano, Z. Mohamed and J. Sharifuddin. 2017. Factors affecting urban dwellers to practice urban agriculture. Int. J. Adv. Res., 5(7): 1580–1587. https://doi.org/10.21474/IJAR01/4872

Nunnally, J.C. and I.H. Bernstein. 1994. Psychometric theory. New York: McGraw-Hill.

Ramaloo, P., C.Y. Liong, C. Siwar and A. Isahak. 2018. Perception of community residents on supporting urban agriculture in Malaysian city: Case study at Bukit Mertajam. J. Pengurusan, 53(2018): 83–91.

Ramayah, T., J. Cheah, F. Chuah, H. Ting and M.A. Memon. 2018. Partial least squares structural equation modeling (PLS-SEM) using SmartPLS 3.0: An updated guide and practical guide to statistical analysis (2nd ed.). Kuala Lumpur, Malaysia: Pearson.

Rezai, G., M.N. Shamsudin and Z. Mohamed. 2016. Urban agriculture: A way forward to food and nutrition security in Malaysia. Proc. Soc. Behav. Sci., 216: 39–45. https://doi.org/10.1016/j.sbspro.2015.12.006

Ringle, C.M., S. Wende and J.M. Becker. 2022. Smart PLS 4. Oststeinbek: SmartPLS. Retrieved from https://www.smartpls.com

Sanchez, J., 2016. Planning, managing, monitoring and evaluating extension program (Doctoral dissertation, University of the Philippines los banos).

Smith, A., 2015. Program efficiency in organizational performance. J. Manage., 45(3): 312-326.

Stone, M., 1974. Cross-validatory choice and assessment of statistical predictions. J.R. Stat. Soc. Ser. B (Methodol.), 36(2): 111-133. https://doi.org/10.1111/j.2517-6161.1974.tb00994.x

Stufflebeam, D.L., 1983. The CIPP model for program evaluation. In: W.J. Popham (Ed.), Evaluation in education. Berkeley, CA: McCutchan Publishing Corp. pp. 97–143. https://doi.org/10.1007/978-94-009-6669-7_7

Sullivan, G.M., and R. Feinn. 2012. Using effect size or why the p value is not enough. J. Graduate Med. Educ., 4(3): 279-282. https://doi.org/10.4300/JGME-D-12-00156.1

Tiraieyari, N., K. Idris, J. Uli and A. Hamzah. 2010. Competencies influencing extension workers’s job performance in relation to the good agricultural practices in Malaysia. Am. J. Appl. Sci., 7(10): 1379-1386. https://doi.org/10.3844/ajassp.2010.1379.1386

Tiraieyari, N., R. Karami, R.M. Ricard and M. Badsar. 2019. Influences on the implementation of community urban agriculture: Insights from agricultural professionals. Sustainability (Switzerland), 11(5). https://doi.org/10.3390/su11051422

Tuan, M.Y.S.N.H., 2017. Kerangka penambahbaikan pelaksanaan program diploma kejuruteraan mekatronik politeknik.

UTHM, 2011. Garis Panduan Pembangunan Modul Pengajaran dan Pembelajaran Universiti Tun Hussein Onn Malaysia. (S. Sarif, Ed.) (Pertama). Parit Raja, Batu Pahat: Pejabat Penerbit, Universiti Tun Hussein Onn Malaysia.

Wahab, J.A., N.B.M. Haris and J.A. Shah. 2023. Program development skills towards work performance of extension agent during movement control order (MCO) in Peninsular Malaysia. Sarhad J. Agric., 39(2): 351-359. https://doi.org/10.17582/journal.sja/2023/39.2.351.359

Wang, Q., 2019. Contributions of food security to national development. World Dev., 47(3): 352-366.

Werts, C.E., R.L. Linn and K.G. Jöreskog. 1974. Intraclass reliability estimates: Testing structural assumptions. Educ. Psychol. Measur., 34(1): 25-33. https://doi.org/10.1177/001316447403400104

Yusuf, M.S.A., N. Man, N.B. Haris, I.A. Ismail and A. Maruf. 2022. Program evaluation on urban agriculture Malaysia: A review of evaluation studies using CIPP evaluation model approach. Int. J. Spec. Educ., 37(3s).

Yusuf, M.S.A., N. Man, N.B.M. Haris, I.A. Ismail and A. Maruf. 2021. Evaluating urban agriculture program effectiveness using CIPP model: A review. E3S Web Conf., 306: 03007. https://doi.org/10.1051/e3sconf/202130603007