|
APPLICATION FOR PILOT PARTICIPATION |
|
|
|
Provided is the five-part application form for providers to use to submit proposals for pilots associated with the Options for People Through Services (OPTS) initiative. The parts of the application are: Identifying Information, Executive Summary, Program Narrative, Evaluation, and Budget and Roster. In preparing the application, please include all pertinent explanatory data and exclude any extraneous information. EMAIL NYSACRA at
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
for a hard copy.
State of New York
Office of Mental Retardation and Developmental Disabilities
OPTIONS FOR PEOPLE THROUGH SERVICES (NYS OPTS)
APPLICATION FOR PILOT PARTICIPATION
Provided is the five-part application form for providers to use to submit proposals for pilots associated with the Options for People Through Services (OPTS) initiative. The parts of the application are: Identifying Information, Executive Summary, Program Narrative, Evaluation, and Budget and Roster. In preparing the application, please include all pertinent explanatory data and exclude any extraneous information.
OPTS Proposals should be submitted through the OMRDD Secure Messaging Center to the following e-mail account:
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
The complete proposal will include:
• The Application, Parts I through IV submitted in the format provided in the attached with Part V being a completed Budget and Roster (EXCEL separately provided document) with instructions for completion provided in the attached, *please note “Additional Information Which May Need to be Provided” at the end of the budget instructions
• A cover letter (addressed to Eugenia Haneman, Interim Director, OPTS Administration Unit) signed by the Executive Director and a member of your governing body, and
• At least 2 signed Letters of Support.
Letters of Support may come from individuals, advocates, family members or others who participated in the planning of the proposal. Service Coordinators who also advocate for consumers may also submit letters of support.
For more information, please contact:
OPTS Administration Unit (518) 408-2744
Questions via E-Mail
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
Visit the OMRDD website at www.nysopts.com for OPTS information, including evaluation tips and a copy of the most current Budget and Roster.
Documents that require signatures should be scanned and then e-mailed.
The attached Application form must be used for all proposals submitted after November 1, 2006.
NYS OMRDD
OPTIONS FOR PEOPLE THROUGH SERVICES (NYS-OPTS)
APPLICATION FOR PARTICIPATION
PART I: IDENTIFYING INFORMATION
Please provide the following information:
Agency Name: _________________________________________________________
AgencyAddress: ________________________________________________________
Title of Proposal: ________________________________________________________
Agency Contact Person: __________________________________________________
Agency Contact Phone #: __________________Email:__________________________
Agency Evaluation Contact Person: _________________________________________
Evaluation Contact Phone #:__________________Email: _______________________
Agency Financial/Budget Contact Person ____________________________________
Financial/Budget Contact Phone #_________________Email:____________________
Agency CFR Code: ____________________ DDSO: ___________________________
FOR RESIDENTIAL PROPOSALS ONLY:
Does this Proposal serve individuals on the NYS-CARES Waitlist?
_____ Yes _____ No
Will participants in this Proposal create backfill opportunities to be filled by individuals from the NYS-CARES Waitlist?
_____ Yes _____ No (If yes, provide backfill information on the Budget and Roster)
_____ # of NYS-CARES Waitlist individuals to be served directly and through backfill opportunities.
General Instructions:
The Executive Summary, Program Narrative and Evaluation Sections must be submitted using this Application format, SINGLE SPACED with NUMBERED PAGES and SECTION BREAKS including numbered responses to correspond to questions on the application. If you want to provide a Table of Contents with your proposal (optional), it should precede Part I: Identifying Information.
NYS OMRDD
OPTIONS FOR PEOPLE THROUGH SERVICES (NYS-OPTS)
APPLICATION FOR PARTICIPATION
PART II: EXECUTIVE SUMMARY
In this 1 page stand-alone document briefly describe what you hope to do and achieve by providing this/these service(s). Please include the number of people to be served in this proposal.
NYS OMRDD
OPTIONS FOR PEOPLE THROUGH SERVICES (NYS-OPTS)
APPLICATION FOR PARTICIPATION
PART III: PROGRAM NARRATIVE
1. Describe the services/supports/opportunities you propose to offer.
2. The services/supports/opportunities to be offered will be
___ On-Going ___ On-Going and involve Phase-In ___ Time-Limited
3. Is this a new service that you currently do not provide, or is this an expansion of an existing service?
___ Expansion ___ New Service, please describe how this proposal relates to your agency’s long-term direction.
4. Describe the relationship of this proposal to the NYS-OPTS Guiding Principles (presented at the end of this Part). Address the first Guiding Principle plus as many as are relevant, with a minimum of three in total answered. Please put the corresponding guiding principle number next to each one addressed.
PROPOSED PARTICIPANT DESCRIPTION, VALUES OUTCOMES AND BENEFITS
5. Answer for each participant for proposals serving 15 or fewer people. For proposals that serve more than 15 individuals, answer for a representative sample of at least 15 people. Provide sufficient detail to show the principles of person-centered planning are being applied.
a. Who is the individual to be served,
b. What are the individualized valued outcomes, and
c. What are the individualized benefits expected from the proposal.
6. Describe the process used to involve consumers, families/advocates and others in planning this program.
7. How will you continue to involve consumers and families throughout the duration of this project other than through the Pilot Evaluation Committee (e.g. Staff training, forums for ongoing discussion etc.)?
8. Are there any OMRDD funded services currently being provided to the participants identified for this proposal that will be impacted?
____ Yes ____ Some, But Not All ____ No
Please note, if Yes or Some, But Not All is checked, the Applicant will be contacted by the Budget Unit to provide specific information. The discussion will focus on identifying the impact that this proposal will have on existing services and their associated funding. Guidance Document (A) will provide a preview of the types of questions that will be posed.
9. Provide a schedule of when services will be given. If this proposal is for a service(s) you currently do not provide, also include relationship of this service(s) to other services the individual receives during the day.
REGULATION, POLICY OR GUIDELINE VARIANCES
10. Will this service require a variance with existing regulation, policy or guidelines?
____Yes (If yes please explain) ____No
PROPERTY QUESTIONS
11. Does this proposal require a new or existing physical site that is owned or leased by the agency?
______ Yes ______ No
If Yes, provide a general description of the site you will be seeking or using and please complete the attached property questionnaire.
If No, describe where the service(s) will be provided.
12. Are there any special features that the physical site will need to accommodate the participants to be served through this proposal?
13. If funding is needed to acquire and/or renovate/construct the physical site, do you know how the capital costs will be financed? ______ Yes ______ No
If Yes, please explain.
REQUIRED PROJECT WORK PLAN WITH BENCHMARK ACTIVITIES AND DATES
14. Provide a project work plan with benchmark activities and proposed dates, for implementation, and property as applicable.
STAFFING INFORMATION
15. Provide a staffing schedule/plan (with titles and FTEs). If this proposal is for a service you currently do not provide or different then you normally provide, also include detailed information as to how the service will be delivered by staff (e.g. a typical day in the life of …).
16. Is there anything innovative about the way the staff/personnel will be deployed?
______ Yes ______ No
If Yes, please explain.
NYS OMRDD
OPTIONS FOR PEOPLE THROUGH SERVICES (NYS-OPTS)
APPLICATION FOR PARTICIPATION
NYS OPTS GUIDING PRINCIPLES*
1. Maximize opportunities for individual choice through person-centered planning;
2. Advance independence, inclusion and individual and family responsibility throughout the system;
3. Create funding mechanisms that strengthen capacity to deliver individualized services;
4. Preserve oversight systems to ensure the highest quality of services for all individuals;
5. Assure that all providers promote the health, safety and protection of individuals through compliance with the highest standards of operation;
6. Improve access to needed services and supports for eligible individuals;
7. Enhance flexibility within the services system;
8. Promote user-friendly efficient and effective operations; and
9. Encourage continued participation and open communication among all those involved in the system
* Not in priority order
NYS OMRDD
OPTIONS FOR PEOPLE THROUGH SERVICES (NYS-OPTS)
APPLICATION FOR PARTICIPATION
PART IV: EVALUATION
Overview: Each agency conducting an OPTS Pilot Project must conduct a self-evaluation. One important purpose of evaluation is to encourage participants, families, and providers to think carefully about the outcomes they want to achieve and decide on the objective indicators that will define success. The evaluation should answer the question: “What evidence will tell us that our project/service is successful?”
There are two major classes of outcomes: personal and organizational. All projects must define and measure personal outcomes. Many projects will choose to identify organizational outcomes.
The original OPTS process required agencies to include an evaluation design in their application. In the streamlined application process, this is no longer required. The OPTS oversight committees believe a more effective strategy is for OMRDD staff to provide technical assistance while the proposal is being reviewed and modified, prior to finalizing the evaluation design. Therefore, the only requirement at this stage of the application is to describe key personal and organizational outcomes of your project (i.e., what you hope to achieve).
Once your project is under review, OMRDD staff will contact you to discuss your evaluation design and suggest strategies to further define outcomes, measure them, and collect data. In requiring OPTS self-evaluation, OMRDD’s overriding objectives are (a) to keep the evaluation process simple, and (b) to help everyone involved learn if the project is achieving its intended goals.
E1. Application Requirement for Personal Outcomes (required): Define in objective terms between three and six critical outcomes that you expect for the participants in the OPTS service. For each outcome, provide one or more indicators you believe would be a valid way to measure that outcome. The following are some examples of the types of outcomes often used in OPTS proposals:
Example A - Personal Outcome: Increased community inclusion.
Indicators for A: Increase in the number and diversity of places a person goes in the average week; decrease in the size of groups participating in community experiences; increase in percent of community places chosen specifically by the person.
Example B - Personal Outcome: Increased work-related skills.
Indicators for B: Improvements in personal hygiene; improvements in motivation to work; increase in time on tasks; improved cooperation and communication with supervisor and co-workers. (Note: OMRDD staff can recommend instruments that can capture these outcomes.)
Example C - Personal Outcome: Increased independence at home.
Indicators for C: Reduced dependence upon staff to do household chores; increased skills in cooking, cleaning, laundry; increased capacity to be at home safely, unsupervised; increased knowledge about safety precautions, emergency procedures, use of phone; increased capacity to travel in neighborhood.
E2. Application Requirement for Organizational Outcomes (optional): Define in objective terms one or more changes or improvements in your agency that will result from your delivery of the new OPTS service. For each change or improvement, provide one or more indicators you believe would be a valid way to measure that result.
Example D - Organizational Outcome: Reduced turnover of direct support staff.
Indicators for D: Reduction in percent of direct support staff leaving during first year of OPTS service.
Example E - Organizational Outcome: Increased job satisfaction.
Indicators for E: For employees in the OPTS program, increased levels of satisfaction expressed in key areas: supervision, enjoyment of work, feelings about co-workers, attitudes toward agency; reduced unscheduled leave; reduced turnover. (Note: OMRDD staff can recommend job satisfaction instruments that measure these factors.)
E3. OPTS agencies must measure consumer satisfaction, minimally at six, twelve, and 24 months after service implementation. You will learn more about this during the technical assistance phase.
E4. OPTS agencies must establish a Pilot Evaluation Committee with required membership and specific responsibilities. Please refer to the OMRDD OPTS Evaluation Guidelines (attached) for a more detailed description of Pilot Evaluation Committee requirements.
When OMRDD calls your designated evaluation contact (please list who this person is and their phone number to be indicated on Part I of the application), OMRDD staff will discuss specific approaches, evaluation designs, and data gathering tools that can measure the outcome indicators you have defined in E1 and E2. OMRDD staff will also discuss who will collect data, how, and when, and the Interim Reporting Questions you will be required to respond to at six, twelve, and 24 months.
Contact us: OMRDD staff are available to answer questions and help you think through your evaluation at any phase. Please contact Christine Muller or Allen A. Schwartz, Ph.D. at (518) 474-4904 for additional information or assistance.
NYS OMRDD
OPTIONS FOR PEOPLE THROUGH SERVICES (NYS OPTS)
APPLICATION FOR PARTICIPATION
PART V: BUDGET and ROSTER
Completing the Options for People through Services (OPTS) Budget Template
1. The OPTS budget template is in the form of a Microsoft Excel spreadsheet file named NYS OPTS BUDGET TEMPLATE.XLS.
2. Assistance in completing the budget template can be obtained by writing or calling:
Bill Britton
(518) 402-4107
William.Britton @ omr.state.ny.us
3. The template file is to be opened using Microsoft’s Excel software. The spreadsheet cells that do not require inputs and the spreadsheet’s format are protected. DO NOT ATTEMPT TO DISABLE THE PROTECTION, ALTER THE FORMAT, CHANGE FORMULAS, ADD OR DELETE ROWS OR COLUMNS, OR MAKE ANY OTHER CHANGE TO THE FILE. IF YOU BELIEVE A CHANGE IS REQUIRED, CALL Bill BRITTON AT THE BUREAU OF RATE SETTING AT (518) 402-4107. SUBMITTED BUDGETS THAT CONTAIN ANY CHANGES TO THE FORMAT WILL NOT BE ACCEPTED AND A NEW SUBMISSION WILL BE REQUIRED.
4. A separate budget template must be completed for each program type (viz., Supervised IRA, Supportive IRA, Day Habilitation, At-Home Res Hab, Respite, etc.) proposed to become part of OPTS. All sites within a single program type, which are proposed for participation in OPTS, must have a separate budget template completed for each site. An aggregated budget may be submitted as long as a separate budget template for each site has been completed. Hourly Respite and Free-Standing Respite, should not be combined into a single OPTS budget. If a service fits more than one OPTS service type, a separate budget must be completed (e.g. Supervised IRA & Comprehensive Supervised IRA).
5. IRAs may be configured as “supervised” or “supportive”. Supervised sites are defined as those sites where the staff is on site or proximately available at all times when the persons are present. Supportive sites are defined as those sites where the staff typically is not onsite nor proximately available at all times when the persons are present. Separate OPTS budgets must be completed for supervised and supportive sites.
6. To complete the budget template, first enter the required general information in Cells B11 through B24. Use the program type code listing that begins in Cell E11 to determine the program type code that goes in Cell B16. Use the “Program Description” button to see an explanation of the OPTS Service Types. If a proposal involves different OPTS Service Types, a separate budget must be submitted for each service type. Some OPTS Service Types have been left blank for future use.
7. Next complete the Site Information Section of the template, beginning in Cell J11. Note the instructions at the top of the section. When inputting the number of consumers for non-respite programs, do not include any (TUBs) Temporary Use Beds (i.e., respite beds). TUBs beds should be included only as part of a Respite budget.
8. If, as part of joining OPTS, the program type is to convert (viz., ICF or CR to IRA, Day Treatment to Day Habilitation) enter Y in Column Q. Enter the former program type in Column R. Supervised and supportive programs may not be combined in the same price.
9. If the program will receive funding from the U.S. Dept. of Housing and Urban Development (HUD), enter Y in Column S. For new sites, following a review of the initial budget, OMRDD will request the provider to submit HUD Form 92264.
10. Next complete the Participant Information Section of the template. A list of participant names is required* as part of the budget and must include the names of all participants that will be served under the submitted budget. Begin entering the required consumer information in Cell AC11. A single certified IRA may not serve more than 14 consumers. For each consumer, the TABS ID from OMRDD’s Tracking and Billing System (TABS) must be provided. If that number is unknown, please contact your local DDSO or the NYC Regional Office for assistance. If that number still can not be determined, leave that cell blank; however, the TABS ID must be determined prior to the fee’s effective date or billings against the fee will not be permitted. For any proposal which includes a Willowbrook class member, enter a “Y” in the Willowbrook Class field on the Participant Listing and complete page 5. The class member’s name will automatically appear on the Willowbrook Information sheet from the Participant Listing. The Date of Birth must also be entered for each consumer in 00/00/00 format. The consumers CIN # should be entered if known. If this proposal will create backfill opportunities, enter the name of the potential backfill and TABS ID. If there are to be more than 75 participants in the OPTS budget for the particular program type, contact Rate Setting for assistance.
* In certain situations, the list of participants is not required, but these proposals will require a letter of support from the local DDSO identifying that a need exists for the proposed service and the proposal must specifically state that individuals referred from the DDSO will be served if the proposal is developed.
11. Next complete the Consumer Verification Section of the template. Provide a contact person for consumer verification starting in Cell AM11 along with the relationship to the participant and the phone number of the contact person. To expedite this process, also provide the best time when the contact person may be reached and alternative phone #. Indicate the current site address for each consumer in Column AR or the anticipated address if a relocation/downsizing.
12. Next complete the Willowbrook Information Section of the template if a “Y” was entered in the Willowbrook Class field on the Participant Listing. The class member’s name will automatically appear on the Willowbrook Information sheet from the Participant Listing. The other four fields in the Willowbrook Information sheet are REQUIRED for each class member. A dropdown box has been provided for the Relationship to Class Member field. Once you have entered the required information, you can scroll back to the Participant Listing to enter additional information.
13. Once the site and consumer information sections of the template have been completed, begin entering the budget information in Cell C30.
14. The PS (personal service) dollar amounts and FTEs (full time equivalents) for each expense category must be entered in the designated boxes. FTEs may represent fifty-two 35.0, 37.5, or 40.0 hour weeks and may contain up to three decimal places. Salaried PS and Contracted PS must be entered separately in the cells provided. Whenever Contracted PS is requested in a budget, details must be provided in the designated cells or the Contracted PS may be disallowed without follow-up. The various PS categories are defined as follows:
Direct Care PS – Includes all individuals engaged in non-discipline specific services that involve training in ADL (Activities of Daily Living) skills, provision of personal care, the promotion of habilitation, or the supervision thereof (excluding the Site Director).
Support PS – Includes all individuals involved in food service, maintenance, cleaning, repair, transportation, security, etc., or the supervision thereof.
Clinical PS – Includes the titles; Case Manager, Counselor - Rehabilitation, DDS/QMRP, EMT, Nurse, Nurse’s Aide, Psychiatrist, Physician, Physician’s Assistant, Psychologist, Certified Social Worker, Therapist, Dietician, Intake Screener, Family Counselor, Staff Trainer, Pharmacist, etc. Service Coordination is not allowable in an OPTS fee and is reimbursed apart from the OPTS fee.
15. Fringe benefit dollars attributable to salaried Direct Care, Support, and Clinical PS should be entered in Cell C40.
16. When completing the Site NPS (Non-Personal-Service) portion of the template, remember that vehicle rental, vehicle depreciation, and interest on vehicle-related loans are not considered to be Site NPS and should be included under Property & Equipment. (However, as noted below, Property & Equipment amounts should be included in the budget only for new sites.) Vehicle insurance, maintenance, gas, tires, etc. are part of Site NPS and should be included under “Transportation”. BE SURE TO JUSTIFY ALL SITE NON-PERSONAL SERVICE COSTS WHICH EXCEED HISTORICAL EXPENDITURES.
17. Program Administration PS includes the following titles; Program or Site Director, Assistant Program Director, Office Worker (at program site), Program Researcher, Staff Trainer (of program staff), and other Program Administration Staff. Agency Administration PS includes that portion of the salaries applicable to the price of the following titles; Executive Director, Assistant Executive Director, Controller, and other Agency Administration Staff. Administrative Fringe Benefits and Administrative NPS should be allocated between Program and Agency Administration, as appropriate. Administrative Property & Equipment should be allocated between Agency and Program Administration NPS.
18. Do not include any Property & Equipment items for existing sites. OMRDD already has that information. For new sites, do not include those items that require “Prior Property Approval” (i.e., items pertaining to acquisition, construction or renovation, and pre-operational start-up costs; including rent, depreciation, mortgage interest, start-up, state-paid items, etc.) .
19. The property and equipment lines provided should be sufficient to capture all non-PPA Property & Equipment costs (for new sites only). Detail Equipment and Vehicle items, with dollar amounts, in the boxes provided. Equipment and Vehicles are depreciated over four years. Enter only the annual depreciation expense (i.e., one quarter of the purchase price).
20. Real estate taxes are allowable only for the first year the program site is open. It is expected that the provider will apply for a real estate tax exemption from the local government. If extraordinary circumstances require that real estate taxes be paid subsequent to the first year the program site is open, the provider should contact OMRDD for instructions on obtaining reimbursement thereof.
21. If a dollar amount is included in one of the “Other” categories for NPS or Property & Equipment, the component items and dollar breakdown must be specified in the boxes provided. If that detail is omitted, the “Other” items in the budget will be disallowed until follow-up by OMRDD. This policy also applies to the Equipment and Vehicle items budgeted under Property & Equipment, as well as Contracted PS.
22. Consult the CFR (Consolidated Fiscal Report) manual for details regarding staffing, NPS, and administration that are not contained herein.
23. All budgeted values are subject to OMRDD review and approval. If the provider feels that a budgeted item (including FTE levels) will be deemed excessive by OMRDD, the provider may attach justification supporting the proposed level of expenditures for that item. That justification should be separate from the budget spreadsheet (i.e., not a part of the spreadsheet file) and should be transmitted electronically to OMRDD along with the OPTS budget and application. The provision of justification by the provider does not guarantee that the item in question will not be reduced or disallowed by OMRDD.
ADDITIONAL INFORMATION WHICH MAY NEED TO BE PROVIDED:
1. Describe the funding needs which, per the budget instructions #18, are not identified on the budget template (property related costs, start-up, etc.).
2. Identify other sources of funding, and expected amount, you expect to access for this proposal.
3. For OPTS Service Type Supplemental Group Day Habilitation or OPTS Service Type Supplemental Individual Day Habilitation, identify where each proposed participant resides. When one of these services is delivered to individuals who are residents of an ICF/DD, Supervised IRA, or Supervised CR, the residential provider will be expected to reimburse the Supplemental Day provider for the service.
4. Please note that additional information may be requested in the form of ISPs, behavior plans/data, or other information to support staffing requests which exceed normal thresholds.
Guidance Document (A)
BUDGET UNIT DISCUSSION POINTS
Please keep in mind that the following questions have been developed in order to assist the Budget Unit in identifying whether:
a) new funding is required to support each opportunity that is being proposed under NYSOPTS, or if
b) in the case of a conversion or downsizing, whether the current funding will be reinvested into the NYSOPTS proposal, or if a backfill will occur.
------------------------------------------------------------------------------------------------------------
Will the service that will be impacted now be provided under OPTS if this proposal is approved? ____ No ____ Yes
IF NO -- Then it is assumed the OPTS proposal will provide additional services to individuals.
IF YES -- Is your agency the service provider for the service that will be impacted?
____ No for ____ # of individuals
____ Yes for ____ # of individuals (i., ii. or both must be answered)
____ i. Funding follows participants and will be available to reinvest into the OPTS proposal. Identify funding resource (e.g., Price ID, Provider ID, Contract number, Operating Certificate number, etc.) by individual and amount of funding that will be reinvested into OPTS. Provide as a separate attachment.
____ ii. Funding is available from current services to serve new individuals (backfill). Are the new individuals on the agency’s mandatory waitlist (e.g., NYS-CARES, Aging Out, etc.)?
____ No ____ Yes ____ Don’t Know
Property Questionnaire
OMRDD
PROPERTY QUESTIONNAIRE FOR NYS-OPTS PROPOSALS
If existing or new property, or a combination of both, is required to provide the services identified in your NYS-OPTS proposal and it is your expectation that you will receive reimbursement for property costs, please provide in detail all available property information including estimated costs and financing information, if known. For proposals that involve multiple sites, please provide information for each site.
• For existing property:
Address__________________________________________________
Current use(s): ____________________________________________
Current use reimbursed by OMRDD? Y/N_____
Will site be abandoned / sold? Y/N _____ Converted to another use? Y/N____
What portion of the property will be dedicated for OPTS use?
_____% _____ sq. ft.
Will renovations be required for proposed OPTS services? Y/N ____
Are completion of renovations necessary prior to implementing OPTS services?
Y/N____
How will renovations be financed? _______________________________
Please submit before and after floor plans for downsized sites
Please submit before and after floor plan with PPA if site is to be renovated.
• For new property
Has property been identified? Y/N ____ Location? ___________________
What portion of the property will be dedicated for OPTS use? ____% ____ sq.ft.
Please submit floor plan or conceptual layout with PPA.
OPTS Evaluation Guideline
OMRDD OPTS Evaluation Guidelines
All NYS OPTS pilot project proposals are required to include an evaluation. This document identifies the elements that are related to each project’s processes and outcomes that must be addressed in the evaluation.
Required Evaluation Elements
All pilot project evaluation descriptions and submissions must include:
1. A description of the actual service that is being provided in the pilot project. The project description must state what the pilot service is intended to achieve, and what limitation, obstacle, or problem it is designed to resolve.
2. A description of the methods to be used in the evaluation. These may include quantitative and qualitative methods, but must be specific in their design and implementation, and provide objective evidence of outcomes and benefits. Agencies are encouraged to consider using pre- and post- measures or a comparison group to assess the benefits of a pilot service. (Note: some examples of acceptable approaches are provided in Addendum A to these guidelines.) Agencies should note that the information to be collected need not be extensive as long as it is sufficiently specific to allow key questions about outcomes to be answered. Such questions should focus on the benefits of, or “valued added” by, the service being piloted.
3. A description of the resources needed and available to carry out the evaluation successfully. This should include information about who will conduct interviews or collect data, and how the agency will protect consumer and family identity in order to encourage complete honesty. In addition, the agency must indicate how its data gathering strategies will insure that all individuals who receive piloted services, regardless of their preferred method of communication, will be included.
4. A description of how the agency intends to address concerns that are raised through the ongoing consumer feedback process. All pilot projects will be required to conduct repeated measures of customer satisfaction that are part of the evaluation, so that all pilot projects can report on this outcome. An initial report on customer satisfaction will be due at six months, and then at least annually thereafter.
5. An identification of the specific objective outcomes of the service or support being piloted, and that are being measured or assessed. These may include:
• personal outcomes for consumers, parents, siblings or other family
members, or community members and others who may be impacted.
• organizational outcomes, organizational changes, new administrative
systems, staff recruitment, training, skills, attitudes, changes in mission or
strategic plan consistent with OPTS guiding principles, or streamlining
and efficiencies.
Pilot projects that measure personal outcomes need not necessarily measure organizational outcomes. However, projects that focus on organizational outcomes must also assess personal outcomes.
6. A timeline for the evaluation process that indicates when the evaluation and data gathering will begin, midpoints for data collection and feedback, and an end point. As stated above, an initial report on customer satisfaction is due at six months. Interim reports on findings are encouraged to assist pilot agencies in evaluating progress. Such reports, if produced, should be submitted to OMRDD according to the timeline.
Pilot Evaluation Committees
Each pilot agency will be required to establish a Pilot Evaluation Committee.
This Committee will be responsible for designing, conducting, and documenting the results of the pilot project evaluation, and assuring its rigor, accuracy, and meaningfulness. They should be administratively positioned to endure objectivity in their operating procedures and findings.
Agencies may wish to form evaluation consortia separate from or associated with their DDSO, to pool resources and share expertise. Such consortia may substitute for individual agency Pilot Evaluation Committees, with the approval of OMRDD. Provider associations may sponsor such consortia.
Pilot Evaluation Committees must include as partners:
• consumer members (e.g., self-advocate)
• parent or family members
• staff members
• a member who is a psychologist, program evaluator, or similarly
skilled person
• a member with a quality assurance background
These committees may seek guidance and technical support from the Evaluation Oversight Workgroup. The Evaluation Oversight Workgroup will be a statewide group established by OMRDD to support the activities of the Pilot Evaluation Committees at each OPTS pilot agency, evaluation consortia, and DDSO evaluation staff. The Evaluation Oversight Workgroup will include consumers and family members, providers, and OMRDD staff with technical knowledge about program evaluation.
Questions regarding this document may be addressed to:
Dr. Allen Schwartz
OMRDD
44 Holland Avenue
Albany, New York 12229-0001
(518) 474-4904
Examples of Acceptable Evaluation Approaches
Example 1. An agency is conducting a pilot project that involves an integrated
funding approach for supporting consumers in various activities in the
community. Consumers enrolled in the pilot were formerly in a center-based day
habilitation program, but were not happy with their weekly activities.
In order to evaluate the effectiveness of the new service, the agency has developed a customer satisfaction survey that is first implemented before the introduction of the pilot. It asks the participants about various aspects of their current activities (while they are still at the day hab site). Objective data are also collected by staff during this phase to document some of the activities consumers are engaged in, and their locations.
After the first six months of implementing the new integrated funding for community activities, the customer satisfaction survey is readministered, and activity data are again collected. The pre-pilot and post-pilot findings are compared to see if specific outcomes were achieved for participating individuals, and whether higher levels of satisfaction are reported.
Example 2. A pilot project is proposed that will offer person centered planning to
students transitioning from school into adult work, volunteering, community
service, and continuing education. Individuals offered this service have
expressed a desire for activities beyond traditional program offerings in their
area. The agency identifies 10 young adults who wish to participate.
The agency’s evaluation will use a “portfolio approach” by which each of the
10 pilot participants will have a personal profile developed over time that describes their life and how it is changing. A person centered plan is developed for each of the 10 people in the new pilot group, and individualized community supports for their personal goals and activities are funded through the pilot.
After six months, the portfolios of the 10 participants are updated with information
about current activities, relationships, community connections, and
accomplishments. Staff write descriptions that focus on the degree to which
personal goals are being achieved, how lives have changed, and how the
portfolios may differ from the stories of similar consumers who have opted for
traditional services. Interviews with the consumers are also conducted to get their
views on how things are going.
Example 3. A project proposes to enhance access to family supports through
the use of a new intake and service coordination system. The agency will
evaluate the success of this service by looking at customer satisfaction and
response time.
The agency decides to use a focus group method to evaluate customer satisfaction. It conducts two focus groups: one with randomly selected service coordinators and one with randomly selected family members who use the agency’s services. The focus groups are organized, and findings summarized, to identify the strengths and weaknesses of the current intake and service coordination system and the changes that parents and SCs would like to see in service access and delivery. The new systems are then implemented.
After six months, focus groups are conducted with different, randomly selected SCs and family members and similar information is generated. The agency analyzes the results of the two focus groups to determine if the improvements in service access, response time, and delivery were achieved, and the degree to which continued improvements are needed.
|
|
|
