Loading...
Cultural Arts Council/V\I.A/\1\1 BE H BUDGET AND PERFORMANCE IMPROVEMENT Internal Audit Division INTERNAL AUDIT REPORT TO: VIA: FROM: Jorge M. Gonzalez, City Manager .I{)..., Kathie G. Brooks, Budget and Performancg Improvement DirectorP)/ James J. Sutter, Internal Auditor4r May 10, 2010 / v DATE: AUDIT: PERIOD: Miami Beach Cultural Arts Council Audit October 1, 2008 to September 30, 2009 This regularly scheduled audit of the Miami Beach Cultural Arts Council (CAC) covering the 2008/09 fiscal year was performed in compliance with the annual audit requirements specified by City Code Section 2-64. INTRODUCTION On March 5, 1997, the City Commission passed City Ordinance No. 97-3075 thereby creating the Miami Beach Cultural Arts Council (CAC) to develop, coordinate, and promote the performing and visual arts in Miami Beach for the enjoyment, education, cultural enrichment, and benefit of both residents and visitors. Since that time, the City has awarded more than $6,500,000 in cultural art grants, supporting thousands of performances, exhibits and other cultural activities in Miami Beach. On June 3, 1998 the City Commission approved the Miami Beach Cultural Arts Council Master Plan through Resolution No. 98-22760. The Master Plan process began in October of 1997 and was the result of a series of public town meetings held in cultural venues throughout Miami Beach. At these sessions, council members heard directly from members of the City's cultural community and from other municipal agencies and boards with a stake in the City's cultural future. The Master Plan included the program areas of Cultural Grants Program, Joint Marketing, Cultural Facilities, Advocacy and Planning, and Revenue Development; each with individual objectives. In fiscal year 2004/05, the City Commission authorized expanding the allocation of the 50% of the 1% resort tax previously equally distributed to North Beach, Middle Beach and South Beach, to include funding for the arts. This funding is distributed in the year after it is collected. This commitment of funding for arts and culture provided a new permanent funding source that sustains cultural programming long term. As a result, the 1% resort tax revenues collected in FY 2007/08 totaling $819,420.03 were appropriated to each of North Beach, Middle Beach, South Beach and the Arts in FY 2008/09. In an effort to address the needs of the arts community, CAC staff in conjunction with the CAC Board annually reviews each of the grants programs (currently Cultural and Junior Anchors; Cultural Presenters; North Beach Initiative/Cultural Ignition; Arts & Cultural Education; Cultural Heritage; Artistic Disciplines; and Cultural Tourism) to determine the type of programs which will benefit the , community. A summary of each grant program follows: • Cultural Anchors provide grants to the major preeminent arts institutions physically based in Miami Beach, whose primary mission is year round artistic and cultural programming that contributes significantly to the City's cultural life. Organizations must have minimum We we committed to ptovicling excellent public setvice one/ _~ofety to oil who live, wotk, oncl ploy in out vibront, tropicol, historic community. Internal Audit Report Miami Beach Cultural Arts Council Audit May 10,2010 organizational budget levels of $750,000 and the monies received may be used towards operational expenses in association with the annual cultural programming described in the application. In addition, for the second consecutive fiscal year there are now "Junior" Cultural Anchors which meet all the same criteria as Cultural Anchors except they have organizational budget levels below $750,000. • Cultural Presenters provide grants to organizations whose primary mission is to produce and present cultural and artistic productions. • North Beach Initiative earmarks funds for proposed projects at the Byron Carlyle Theatre, North Beach Band Shell and other North Beach locations (north of 63rd Street) for the purpose of increasing the amount of cultural activities in Miami Beach's North Beach community. Meanwhile, Cultural Ignition provides funds for organizations never before funded by the City thereby encouraging new cultural organizations to participate in the grants process. • Arts & Cultural Education provides grants to cultural organizations whose primary mission is to provide support for rich and challenging arts learning opportunities for children and adults in Miami Beach schools and neighborhoods. Grants may be used for workshops, lectures/demonstrations, and other projects designed to increase skills and awareness of the arts. • Cultural Heritage provides project specific grants for arts programming to not-for-profit 501 c(3) Miami Beach based institutions devoted to promoting and protecting the diverse heritage, traditions and culture of Miami Beach. Grants may be used for presentation of visual and performing arts, arts programs and workshops, and other projects which emphasize the artistic experience and are accessible to a broad audience. • Artistic Disciplines funding is limited to arts and cultural organizations whose primary mission is to create and present work in one of the following artistic disciplines; Dance, Music Theater, Film and Visual Arts. Proposed projects should be for work that is new or has never before been presented in Miami Beach. • Cultural Tourism are grant funds jointly provided by the CAC and Miami Beach Visitor and Convention Authority (VCA) to support major cultural arts performances, festivals, and events which attract a significant number of tourists to the City. The VCA provided $60,000 and the CAC gave $40,000 to fund this program during the 2008/09 fiscal year. Suggestions and ideas are received from grantees, grants' panelists, members of the CAC and other granting organizations through orientation meetings held at the beginning of each grant period, consultations with grantees, public application and proposal reviews, public meetings held at the end of the review cycle, the Grants Task Force, and through staff, so that the programs can continually be improved. Program application periods and relevant deadlines are announced in January via press releases, electronic mail, on its website and in area newspapers. Program guidelines, applications and instructions are created in Microsoft Word format and made available on the website in both English and Spanish. To be eligible to apply, organizations must be non-profit, 501 c(3) cultural organizations and cultural departments within an institution of higher learning (college or university) with tax-exempt status under subsections of 501 c(s) wishing to present or produce cultural and artistic events in the City of Miami Beach. Furthermore, any applying organization that satisfies one of the following criteria is not entitled to receive funding: • applied through a fiscal agent Page 2 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10,2010 • received prior CAC grant funding and did not comply with the Grant Agreement and/or other requirements • did not include the City of Miami Beach logo and required credit line in all publications related to the grant • received funding from the VCA during the same fiscal year. A total of $970,000 was requested in grant monies by fifty organizations for the 2008/09 fiscal year. Each grant applicant must follow an extensive multi-level public review process which includes a general orientation for all applicants and required consultations with CAC staff to determine eligibility and appropriate programs. Applicants must meet all application requirements, including deadlines, or be disqualified. Applications were scored by the CAC at a series of publicly noticed panel meetings. Applicants should attend their panel meeting where they will be permitted to respond to panelists' questions. A score will be assigned to the applicant and must exceed 80 out of 100 to be eligible to receive funding. Thirty-eight organizations exceeded or met the designated minimum score required for funding. One of the eleven Cultural Anchor and Junior Anchor organizations, Miami Design Preservation League or MDPL, scored 76.45, below the minimum score of 80. However, the City Administration and CAC recommended awarding MDPL $15,000 for the 2008/09 fiscal year which was approved by the Commission. As a result, the City Commission, through the passage of Resolution No. 2008-26906, awarded grants totaling $627,169.33 to forty-three eligible organizations for the 2008/09 fiscal year. The largest award of $27,910.75 was given to the New World Symphony, while the lowest amount of $6,908.59 was given to the Center for Folk & Community Art. All of these authorized grant recipients receive half their funding upon submittal and approval of the required documents with the remaining half upon completion from the City. Resolution No. 2008-26906 also included four programs (XIV International Ballet Festival, 13th Brazilian Film Festival of Miami, 11th Annual Miami Gay & Lesbian Film Festival in Miami Beach and the 4th Annual Women's International Film & Arts Festival) whereby the CAC partnered with the Miami Beach Visitor and Convention Authority or VCA for Cultural Tourism grants that support cultural events with documented tourism benefits. The City's agreed upon payment of $40,000 was made to the VCA through check number 2524 dated 12/02/08. These authorized grant recipients receive all their funding from the VCA upon submittal and approval of the required documents after the project's completion. The CAC budget does not include a contribution towards the City's Cultural Endowment for the 2008/09 fiscal year due to budget constraints. However, the endowment's 09/30/08 balance held in a restricted cash account increased by $256,050.09 to $1,476,050.09 from the prior fiscal year's ending balance of $1,220,000 due to the interest earned by the endowment. Under Resolution No. 99-23228, the City Commission approved the appropriation of $220,000 from the CAC budget to provide matching funds in the form of a non-interest bearing loan to fund a portion of the Colony Theatre Improvements. Since the City was able to obtain the maximum grant award from the State's Cultural Arts Facilities Grant Program, the loan has been reduced to $165,000. The City will repay the loan to the Cultural Arts Council from the 70% recapture of the Page 3 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10, 2010 Colony Theatre Cafe revenues. As of September 2009, the total amount in the Colony Theater Trust account was $143,139.82 with no payments made during the past four fiscal years. OVERALL OPINION The Cultural Arts Council (CAC) has properly followed the guidelines established by the Miami Beach City Commission as to their purpose in the City. It has established a master plan for the arts, and developed and established criteria and guidelines for review and funding for qualified not-for- profit organizations. The active CAC is comprised of eleven Miami Beach residents and business owners who routinely meet once a month to plan the organizations' future. Program files were well organized for each tested vendor. Documentation was present confirming that funded programs were properly staged and written final reports were received before grant payments were authorized for reimbursement. Finally, grantee payments were made timely and were recorded correctly in the City's Financial System. However, the following findings were identified which are in need of corrective action: • One grantee, the Jewish Museum of Florida, submitted incomplete cancelled checks, invoices, etc. by $7 43.62 to support the receipt of the full amount of the grant. This was the only incorrect payment noted and represented .12% ($743.62/$627,169.33) of all monies paid to grantees. Upon notification, the grantee submitted additional documentation to satisfy the City's requirements and to earn full reimbursement. • Grantees are submitting either incomplete or late documentation (IRS letters, IRS form 990s, certified financial statements and/or completed final reports) but are still receiving the second City payment in the next fiscal year. • The $143,139.82 balance remains unnecessarily in the Colony Theatre Trust Account 601- 7000-229220 to pay back a loan that was not made. • The Friends of the Bass Museum's second grant reimbursement payment of $10,316.63 was incorrectly split between two general ledger accounts when the corresponding debit should have been only to 140-6080-000349. As the result of our verification of departmental measures detailed in finding #5, we noted that there were inaccuracies in some of the measures reported in the City's Environmental Scan, performance management software and adopted budget book. These measures should be changed to reflect the data supported by the department's records. PURPOSE To determine whether the Cultural Arts Council complied with tested requirements set forth by the City Commission during the 2008/09 fiscal year, which includes the timely distribution of approved grant funds in the correct amount to qualified eligible not-for-profit organizations; properly reported all grant payments in the City's Financial System; and accurately reported data in the City's Environmental Scan, performance management software and adopted budget book. Page 4 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10, 2010 SCOPE 1. Confirm that the Cultural Arts Council has developed, established and adhered to criteria and guidelines for review and funding of qualified not-for-profit arts organizations. 2. Confirm that the Cultural Arts Council consists of eleven members, who are appointed for three-year terms and no more than six consecutive years, are residents or own businesses on Miami Beach. 3. Confirm that Cultural Arts Council meetings are held once a month (excluding August) and that one public hearing before the City Commission is held annually. 4. Confirm that grant payments were correctly recorded in the City's Financial System. 5. Select a sample of grants awarded, ensuring that all grant categories are represented, and confirm that funds paid for grants were for approved programs, meeting the criteria set forth by the Cultural Arts Council. 6. Confirm that selected departmental measurement data reported in the City's Environmental Scan, performance management system and adopted budget book is accurately calculated and properly supported by source documentation. FINDINGS, RECOMMENDATIONS AND MANAGEMENT RESPONSES 1. Finding -Submitted Supporting Documentation Does Not Satisfy Grant Reimbursement Amount Forty-three grantees were awarded monies totaling $627,169.33 for the 2008/09 fiscal year by the City Commission's ratification of Resolution No. 2008-26906. These authorized grant recipients are to receive half their funding upon submittal and approval of the required documents with the remaining half upon completion. The Grants and Operations Administrator approves the submitted documentation for sufficiency before a Request for Direct Payment is prepared authorizing the City's Finance Department to issue a check to the grantee. Fifteen of the forty-three grantees or 34.88% and representing 35.22% of the dollars disbursed were randomly selected for review with at least one non profit corporation chosen from each of the designated categories (Cultural Anchors, Cultural Presenters, etc.). In doing so, it was discovered that the Jewish Museum of Florida's submitted documentation was not sufficient to justify the received full grant reimbursement for the following reasons: • The summary listing of advertising expenses showed check #22433 as totaling $300.00 when in actuality it equaled $115.30 for a $184.70 difference. • The summary listing of advertising expenses had two duplicate listings for check #22311 for $595.00. Review of the cancelled check and supporting invoice confirmed that the check was for $595.00. • The advertising expenses submitted totaled $23,037.08 or $36.08 more than the grant reimbursement amount of $23,001.00. As a result, the grantee was reimbursed by $743.62 more than proper based on the supporting documentation provided. Page 5 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10,2010 Recommendation(s) The Grants and Operations Administrator should more closely scrutinize the supporting documentation (cancelled checks, vendor invoices, etc.) to ensure that they are complete and warrant the issuing of the second City reimbursement grant check in full. Any differences noted should then be forwarded to the grantee thereby allowing them sufficient time to investigate and possibly submit other qualified expenditures. This opportunity was granted to the Jewish Museum of Florida and they promptly provided additional advertising expenses to justify receipt of the full second grant reimbursement payment. Management's Response The Tourism & Cultural Development Department agrees with the recommendation to more closely scrutinize the supporting documentation provided. A policy is already in place to contact the organization regarding inaccuracies in supporting documentation and allowing them to submit other qualified expenditures. 2. Finding -Grantees Submitted Incomplete or Late Documentation but still Receive Full Reimbursement Prior to receiving any monies, a grant agreement must be completed and signed by both the City and the participating 501 (c)3 entity. This standard document requires the submittal of a final report within 45 days of the project's completion date; proof of the City of Miami Beach logo and credit line must be included in all project publications and advertisements; proof of performances such as programs, brochures and flyers; etc. The Grants and Operations Administrator monitors these non profit organizations and collects and organizes all received documentation in designated folders. Internal Audit reviewed these folders for sufficiency and completeness and found the following deficiencies: a. Five of the fifteen files reviewed or 33.33% did not contain an Internal Revenue Service letter dated after 01/01/06 giving the grantee 501 (c)3 tax exempt status as specified in the grant application. However, Internal Audit's review of the State of Florida Division of Corporation's website properly found that all were active Florida non profit corporations. b. CAC is relatively lax in requiring each grantee to submit either their last completed Internal Revenue Service form 990 or certified financial statements as these items should be done annually unless an extension is requested and granted. All grantees complied with CAC's terms but several were outdated with the oldest being submitted by Seraphic Fire covering the fiscal year ending 12/31/05. c. Grantees are required to submit their final report no later than 45 days after the project end or 11/15/09 for projects ending on 09/30/09 to receive the second half of the grant award. Although CAC does not time/date stamp these documents when received, the grantee does notarize, sign and date the last page of these final reports prior to submittal. Therefore, Internal Audit conservatively used this date to measure when the final reports were received by the City. In doing so, it was noted that three grantees (Italian Film Festival, Culture in the City and Seraphic Fire) were received late but the second grant reimbursement checks Page 6 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10,2010 totaling $22,536.66 were still disbursed in the next fiscal year. Furthermore, three other grantees (Miami Dade Community College, Florida Grand Opera and Wallenstein Musical Organization Inc.) had not provided the Grants and Operations Administrator with all the documentation necessary to have their final checks issued totaling $22,718.83 prior to the conclusion of this audit. Yet, all of the aforementioned grantees are expected to receive full funding per inquiries. Subsequent inquiries with Cultural Arts Division staff found that they are flexible on this due date. Furthermore, they spend time continually reminding the grantees that deadlines have passed and work with them to help ensure that they receive full funding, whenever possible. Recommendation(s) The implementation of the following recommendations should help alleviate the aforementioned documentation shortcomings: a & b. Cultural Arts Division staff should more closely scrutinize submitted documentation to ensure that all requirements are satisfied. The usage of a checklist for each grantee should help ensure all grant applicants should have complied with all the terms listed in the grant application before they are considered for funding. c. The date that grantee documents are received should be indicated by the newly purchased time/date stamp machine to help verify that the grantee complied with the agreed upon requirements and is eligible for future funding. Finally, the timely submittal of the grantees should allow Cultural Affairs Division staff sufficient time to review the report for completeness and for the Finance Department to issue the second and final check during the same fiscal year that the monies were appropriated by the City Commission. Management's Response The Tourism & Cultural Development Department responds as follows to the above finding: a. Agree, more scrutiny is necessary and has been applied strictly in the two fiscal years since. b. Policy in place asks only for the most recent IRS 990; it is not date-specific. Staff does not feel it is their responsibility to police non-profit organizations with regard to taxes filed. Miami-Dade County Division of Cultural Affairs also asks for "most recent 990;" however, if it is more than two years old the applicant is told that the panel will be informed of such. c. Miami Dade Community College (MDCC), not selected for audit, provided their Final Report in accordance with the deadline. Upon review of their report, they were informed of their lack of necessary documentation and were asked to provide it. After receiving this documentation, MDCC received their 2nd-half payment. Florida Grand Opera, not selected for audit, submitted a revised project proposal after receiving their 1 81-half payment. This revision precluded them from receiving any further funding from the CAC. Page 7 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May10,2010 Walenstein Musical Organization, not selected for audit, provided their Final Report in accordance with the deadline. Upon review of their report, they were informed of the lack of necessary documentation and were asked to provide it. This documentation was not received in full or in a timely manner. As a result, a reduced final payment was issued less those expenses unaccounted for. The CAC is committed to fully funding their portion of awarded cultural projects. It establishes relationships with cultural organizations for the purpose of fulfilling the City's cultural needs now and into the future. In many cases it is not possible to issue the final grant payment within the same fiscal year, especially when project completion date coincides with the end date of that fiscal year. We would like to note that, due to increased staff diligence, the number of late submissions have decreased drastically. 3. Finding-Colony Theatre Cafe!Segafredo Loan Trust Account Monies not yet Transferred to a Capital Reserve Fund Under Resolution No. 99-23228, the City Commission approved the appropriation of $220,000 from the CAC budget to provide matching funds in the form of a non-interest bearing loan to fund a portion of the Colony Theatre Improvements. Since the City was able to obtain the maximum grant award from the State's Cultural Arts Facilities Grant Program, the loan was subsequently reduced to $165,000. The City was to repay the loan to the CAC from the 70% recapture of the Colony Theatre Cafe/Segafredo rental revenues. Monies received from the rental revenues were placed into the Colony Theatre Trust Account 601-7000-229220 through 09/30/05 as the balance grew to $143,139.82. However, no additional monies were put into this trust account during the 2006/07, 2007/08 and 2008/09 fiscal years despite continuing to collect rental revenues and not yet reaching the $165,000 desired balance. We could not locate any evidence that the City (CAC) loaned $165,000 to the Colony Theatre Cafe. Consequently, the monies should not be held in a trust account to pay back a loan that was not made. This finding was also present in last year's audit report whereby discussions were held between the Tourism & Cultural Development Director and City Administration where it was agreed that these funds would be placed in a capital reserve fund for future capital needs of the Colony Theatre but no such action has been taken to date. Recommendation(s) The trust account should be closed and the $143,139.82 balance should be used in the manner recommended by the City Administration and subsequently authorized by the City Commission. Management Response This was an oversight on Tourism & Cultural Development Department's part. A recommendation for the trust account balance is included in the department's FY 2010/11 budget request. Page 8 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10, 2010 4. Finding -Incorrect General Ledger Account Used for Friends of the Bass Museum $10,316.63 Second Half Grant Reimbursement All grantee payments during previous audits have originated from general ledger account number 140-6080-000349 entitled CAC Other Contractual Services. This practice held true for the current audit except that the Friends of the Bass Museum's second payment of $10,316.63 which was split as follows: $3,579.42 debit to general ledger account number 140-6080-00345 (CAC Program Operations) and the remaining $6,737.21 was debited to the Other Contractual Services account. Inquiries were made as to the reasons for this deviation by Tourism & Cultural Development Department who responded that it was a clerical error in need of correction. Recommendation(s) The Finance Department should prepare a journal entry to correct the aforementioned Friends of the Bass Museum's split payment so that all $10,316.63 is debited from CAC's Other Contractual Services account. Tourism & Cultural Development staff should be more careful in the future to ensure that the proper general ledger accounts are impacted for grant reimbursements. Management Response We regret this bookkeeping error and will contact the Finance Department to prepare a correcting journal entry. 5. Finding-Review of Departmental Measures The Cultural Affairs Division measures both the number and amounts of grants awarded by the Commission, as well as the actual number of grants issued (output measures). These numbers may differ as the number of grants issued is usually not available until the end of the year since grantees may withdraw or be denied funding due to not fulfilling grant requirements. These departmental measures are reported in either the City's Environmental Scan or the performance management software. We reviewed the basis for each measure and compared the reported amounts to the actual supporting documentation. For the two measures reported annually in the City's 05/01/09 Environmental Scan we noted the following: Meanwhile, results from the audit of the amount of cultural grants issued (paid out), as reported in the performance management software were as follows: Page 9 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10, 2010 # of grants Issued Fiscal Year · · · · CAC ·Audited Difference 2003/04 47 47 0 2004/05 49 45 4 2005/06 47 45 2 2006/07 49 49 0 2007/08 36 36 0 2008/09 43 43 0 A review of the performance management software found that the differences listed above for the 2004/05 and 2005/06 fiscal years were caused by changes during the current fiscal year as last year's audit found no differences. After completing our review of the above measures for fiscal year 2008/09, we determined the following assessment categories suitable as defined in Exhibit 1 attached: . Source. .· ·.· Narhe of Measure ·· ·.·· Assessment Category Environmental Scan # of grants Awarded Certified Environmental Scan Grant Award Amounts Certified Performance Management # of grants Issued Certified with Qualifications Software Finally, it was discovered that the performance measure entitled "# {number} of cultural grants issued" for the 2005/06, 2006/07 and 2007/08 fiscal years reported in the City's adopted budget book is incorrect. Each of the aforementioned fiscal years' figures is overstated by two and was confirmed in discussions with Cultural Affairs personnel. Recommendation(s) The identified incorrect measures for the number of grants issued in the City's performance measurement software should be corrected. Finally, the City's budget book for the 2009/10 fiscal year should be revised to have the# of cultural grants issued equal those figures listed above in the column labeled "audited". Management's Response The # of Grants Issued will be changed to agree with the Audited column findings. EXIT CONFERENCE An exit conference was held on March 23, 2010 and the participants included Max Sklar (Tourism Development Director), Gary Farmer (Cultural Affair Program Manager), Mary Heaton (Grants & Operations Administrator), Georgina Echert (Assistant Finance Director), James Sutter (Internal Auditor), and Mark Coolidge (Senior Auditor). All present were in agreement with the findings and recommendations discussed and management responses were submitted shortly thereafter for inclusion in this report. Page 10 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10, 2010 JJS:MC:mc Audit performed by Senior Auditor Mark Coolidge F:lobpii$AUD\INTERNAL AUDIT FILES\DOC09-10\REPORTS-FINAL\CAC 2009.doc cc Hilda Fernandez, Assistant City Manager Patricia Walker, Chief Financial Officer Max Sklar, Cultural Arts and Tourism Development Director Gary Farmer, Cultural Affairs Program Manager Georgina Echert, Assistant Finance Director Jose Cruz, Budget Officer Page 11 of 12 Internal Audit Report Miami Beach Cultural Arts Council Audit May 10, 2010 EXHIBIT: 1 -Assessment Categories for Performance Measures Verification .· Asse$sment Category ': . ' : :· . Criteria ·: ' ... ' : ' ·.··' .· . If reported performance is 100% accurate and if it appears that Certified controls are in place to ensure accuracy for collecting and reporting performance data. Measurement data is supported by source documents. This category is assigned under either one of two conditions: 1. Reported performance is within +/-3%, but the controls over data collection and reporting are not adequate to ensure Certified with Qualifications continued accuracy. 2. The department's calculation of actual performance deviates from the measure definition, but was still within an acceptable range. Documentation is unavailable or incomplete and controls are not Factors Prevented Certification adequate to ensure accuracy. This category is also assigned when there is a deviation from the measure definition and the reviewer cannot determine the correct measure result. Inaccurate Actual performance is not within 3% of reported performance, or there is a greater than 5% error in the sample of documents tested. EXHIBIT: 2 -Additional References for Performance Management Software's Description of Measure 1) Measure Name: 2) Measure Type: 3) Measure Description: 4) Measure Frequency: 5) Data Sources: 6) Calculation Methodology: Should list the name of the measure for which data is being collected and reported. Should recognize whether the measure is an "Input", "Output", or "Outcome" measure. One must consider whether results are measured through input (Ex: Customer surveys), output (Ex: Number of contracts completed without change orders), or outcome (Ex: %of projects substantially completed or in beneficial use within 120 days). Should describe the measure. Some measures are self explanatory and some may not; however, a description should always be included (Ex: The measure considers the% of change in the value of the change orders for a specific category divided by the original project cost). Describes how often performance data is reported (Ex: Quarterly). Should list sources used to collect performance data (Ex: System Software Names, Report Names, Schedules, etc.) Should explain how data reported was calculated (Ex: change orders for that quarter divided by the original total projects costs, projects substantially completed or in beneficial use within 120 days of the contract milestone date divided by the total number of projects completed during the same quarter). Page12of12