Executive Summary - Canadian Heritage

Archived Content

Information identified as archived on the Web is for reference, research or record keeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you may request alternative formats by contacting the Department of Canadian Heritage.

Appendix A – Evaluation Questions

Evaluation Issues

Evaluation Questions

Evaluation Sub-questions

Relevance

R1:  Do the components and programming elements of the APP continue to be consistent with departmental and government-wide priorities?

R1.1:  Is APP clearly linked to GOC Budget or other priorities?

R1.2:  Is the APP aligned with PCH’s mandate?

R1.3:  Is the APP aligned with the Canadian identity objective?

R1.4:  How do the components and elements of APP fit in the Government of Canada’s urban Aboriginal strategy, as it exists today?

R1.5:  Is APP clearly linked to PCH mandate and priorities?

R1.6:  Does APP belong at PCH?

R2:  Do the components and programming elements of the APP continue to be relevant to urban Aboriginal Peoples?

R2.1:  Is the APP relevant to the needs identified by urban Aboriginal Peoples?

R2.2:  Is the APP relevant to the needs identified based on demographics trends in Aboriginal populations in urban and other areas off-reserve?

R2.3:  To what extent is the APP aligned with the objectives and activities of the recipient organizations?

R2.4: Will the APP be relevant in the future considering demographic trends?

R2.5:  Are the community, language and culture needs of Aboriginal Peoples living in reserves, hamlets and communities different from those urban Aboriginal Peoples who have moved away from their First Nations reserves, Inuit hamlets and Métis communities?

R2.6:  Are the community, language and culture needs of urban Aboriginal Peoples changing?

R3:  Do the Territorial Languages Accords programming elements of the APP continue to be relevant to the federal and territorial governments and urban Aboriginal Peoples?

R3.1:  What is the current state of Aboriginal languages in the territories?

R3.2:  What are the trends in the use of Aboriginal languages?

R3.3:  What is the rationale and extent of PCH’s role with respect to Aboriginal languages in the three Territories?

R3.4:  To what extent are the APP’s objectives and activities consistent with Federal and Territorial objectives under the Territorial Language Agreements?

R3.5:  To what extent are APP’s objectives and activities under the Territorial Language Agreements consistent with stated objectives of Aboriginal organizations and communities?

R3.6:  Is there a clear link between the objectives of the Territorial Languages Accords programming elements of the APP and Federal and Territorial priorities and activities?

Design and Delivery

D1:  To what extent have the APP programming elements, under the APP Implementation Strategy, supported the effectiveness of the program?

D1.1:  What were the objectives behind the consolidation of the APP programs?

D1.2:  What improvements in program effectiveness were sought when consolidating the APP programs?

D1.3:  To what extent are there clear linkages among the APP programming elements?

D1.4:  Do federal officials perceive that consolidation of the APP has increased linkages among the programming elements?

D1.5:  Do federal officials perceive that consolidation of the APP has improved the effectiveness of the APP?

D2:  How effective has the program transition been under the APP Implementation Strategy?

D2.1:  At what levels (national, regional, program delivery, projects) was the transition implemented?

D2.2:  To what extent do Federal officials perceive that the program transition has been effective?

D2.3:  Has the quality of program administration, information and reporting increased or improved since consolidation of APP?

D2.4:  Do funded recipient organizations perceive that the program transition has been effective?

D2.5:  Are there opportunities to further improve the effectiveness of program transition?

D2.6:  What are the major barriers to further improving the effectiveness of program transition?

D2.7:  To what extent is the information on the APP reported consistently (by time periods, level of detail, types of activities and organizations)?

D2.8:  What are the opportunities to improve the consistency of information reported?

D2.9:  What are the major barriers to improving the consistency of information reported?

D3:  Is the level of program access appropriate?

D3.1:  Are program information and tools readily available and in what formats to existing and potential recipients organizations involved in the APP programming?

D3.2:  To what extent are funding allocations aligned with the distribution of need by geographic areas, target populations groups, and funded organizations?

D3.3:  What are the existing methods of dissemination of program information and tools?

D3.4:  Do the identification, selection and contracting processes used to contract recipient organizations encourage and allow for new organizations to receive APP funding?

D4:  Are the current program delivery mechanisms/ /structures appropriate and effective?

D4.1:  What are the existing methods used by PCH to support delivery of APP elements by Third-party Delivery organizations and Direct Delivery in the W6?

D4.2:  Do funded recipients regularly and consistently report the information necessary for effective delivery and management of the programming?

D4.3:  To what extent do the Aboriginal stakeholders and PCH staff believe that the current program delivery mechanisms and structure are appropriate and effective?

D4.4:  What do the Aboriginal stakeholders and PCH staff believe are the strengths of the APP elements, and what areas needing improvement?

D4.5:  To what extent do the Aboriginal stakeholders and PCH staff believe that current delivery mechanisms (Third-party & direct delivery) are appropriate for the APP programming elements?

D4.6:  Are current delivery mechanisms (Third-party & direct delivery) achieving the desired reach identified for the APP programming elements?

D4.7:  Are the funding mechanisms and approaches used with Third-party delivery organizations and direct delivery in the W6 appropriate and effective?

D4.8: With respect to UMAYC program delivery, what are the strengths and the possible improvements within a delivery involving a Third-party?

D4.9: Regarding UMAYC program delivery, what are the strengths and the possible improvements within the process of Direct Delivery?

D5:  Does the current reporting system allow for funded organizations to adequately communicate their results?

D5.1:  What are current reporting requirements and formats for APP programming elements and their scheduled implementation dates?

D5.2:  To what extent are funded organizations regularly completing and submitting reports to fulfill their reporting requirements?

D5.3:  Do funded organizations believe that the reporting system allows them to adequately communicate their results?

D5.4:  Do PCH officials believe that the reporting system allows the funded delivery organizations to adequately communicate their results?

D5.5:  Does PCH review, summarize and/or otherwise use the information produced within these reports?

Performance

P1:  To what degree has the Aboriginal Communities program component within the Aboriginal Peoples’ Program achieved its immediate and intermediate outcomes?

P1.1:  Are the Aboriginal Communities program components contributing to the achievement of the immediate and intermediate outcomes identified within the logic model?

P1.2:  Is there direct or indirect evidence of improvement in the skills, knowledge and leadership of urban Aboriginal youth and women from this program?

P2:  To what degree has the Aboriginal Living Cultures program component within the Aboriginal Peoples’ Program achieved its immediate and intermediate outcomes?

P2.1:  Are the Aboriginal Living Cultures program components contributing to the achievement of the immediate and intermediate outcomes identified within the logic model?

P2.2:  To what extent does this program increase visibility and recognition of Aboriginal achievements throughout Canada?

P2.3:  Does APP make a difference in Canadians perception of Aboriginals?

P3:  Have any unintended impacts, positive or negative, been observed as a result of the activities conducted under APP funding?

P3.1:  Have there been any unintended (positive or negative) impacts as a result of APP activities?

P3.2:  Have there been any unintended (positive or negative) impacts as a result of the APP Implementation Strategy?

P4:  What are PCH’s expenditures to deliver APP?

P4.1:  What are the annual budget allocations and actual expenditures for each of the fiscal years being evaluated?

P4.2:  What is the budget breakdown by APP programming element and PCH program management and administration activities?

P4.3:  What are the overhead costs as percentage of program costs?

P4.4:  How many FTEs are employed to deliver the program?

P5:  To what extent is the APP efficient (overall and by programming element)?

P5.1:  What are the APP administrative and overhead costs as a percentage (admin cost ratio) of total APP grants and contributions distributed by PCH; for APP as a whole and each of its programming elements?

P5.2:  How does APP’s admin cost ratio compare to (benchmark against) PCH as a whole, and for each of the larger program areas within PCH?

P6:  Are there any PCH internal management or administrative overlaps or internal duplication between the components or programming elements of APP?

P6.1:  Have clear administrative practices and standard operating procedures been established and implemented?

P6.2:  Are there administrative and operational inefficiencies that are unnecessarily increasing the workload of APP staff?

Cost-effectiveness and Alternatives

C1:  Do the components and programming elements of the APP duplicate or complement any other programming? To what extent?

C1.1:  Do elements of the APP complement or support other elements of APP?

C1.2:  Do elements of the APP duplicate portions of other elements of APP?

C1.3:  Are there gaps between APP elements and PCH’s stated mandate for APP?

C2:  Is the APP the most effective way to achieve the expected results given the level of resources?

C2.1:  How are the APP and comparative programs delivered?

C2.2:  Have the program management structures been adapted to the delivery of an integrated program?

C3:  Are any changes required for the new APP structure to operate more cost-effectively?

C3.1:  What are the streamlining opportunities within PCH to improve cost effectiveness?

C3.2:  What are the barriers in program delivery that impact upon cost effectiveness, both internal and external to PCH?

C4:  Could certain aspects/components of the program be transferred to other orders of governments or other organizations to be more cost-effective?

C4.1:  What elements, if any, of the APP could be transferred to other orders of government?

C4.2:  What operational barriers and/or advantages would be achieved in transferring elements of the APP to other orders of government?

C4.3:  If elements of the APP were to be transferred, to which organizations and/or orders of government should they be transferred to?

 

[ Previous Page | Table of Contents | Next Page ]