94-21143 Reso
"
.
.
. ,
.
RESOLUTION NO. 94-21143
A RESOLUTION OF THE MAYOR AND CITY
COMMISSION OF THE CITY OF MIAMI BEACH,
FLORIDA, AUTHORIZING THE MAYOR AND CITY CLERK
TO EXECUTE THE PROFESSIONAL SERVICES
AGREEMENT BElWEEN THE CITY OF MIAMI BEACH
AND DR. DAVID SANTISTEBAN, FOR JOB ANALYSIS AND
DEVELOPMENT AND VALIDATION OF PERMANENT
SELECTION PROCEDURES FOR FIREFIGHTER I AND
POLICE OFFICER/TRAINEE IN CONSIDERATION FORAN
AMOUNT NOT TO EXCEED $93,500, INCLUDING ALL
OUT-OF-POCKET AND OTHER EXPENSES INCURRED BY
THE CONSULTANT, AND FURTHER AUTHORIZING THE
CITY MANAGER TO EXERCISE THE OPTION INCLUDED
IN THE CONSULTANT'S PROPOSAL TO DEVELOP AND
ADMINISTER AN ORIENTATION/TEST PREPARATION
PROGRAM FOR AN ADDITIONAL SUM NOT TO EXCEED
$10,000 INCLUDING ALL OUT-OF-POCKET AND OTHER
EXPENSES INCURRED BY THE CONSULTANT.
Whereas, on June 4, 1993, the Federal District Court entered the final Consent
Decree, between the Department of Justice (DOl) and the City of Miami Beach, which
prohibits continued use of the City's previous written tests for Firefighter I and Police
Officer/Trainee and allows the City to develop alternative validated selection procedures
for these entry-level classifications; and
Whereas, the Consent Decree requires that new written tests for Firefighter I and
Police Officer/Trainee must be developed in conjunction with the DOJ and provides that
the City will contract with recognized experts and that the method of test validation used
by such experts shall be approved by the DOJ; and
Whereas, the DOl had no objections to the City's Request For Proposal and the
proposal and methodology submitted by Dr. David Santisteban (Consultant); and
Whereas, the City retains the ultimate responsibility for the development and
administration of lawful written examinations for entry-level Firefighter I and Police
Officer/Trainees; and
Whereas, under the Consent Decree, any test development by the City shall include
the preparation of written job analyses for entry-level Firefighter I and Police Officer!
Trainee examinations; and
Whereas, the Consultant is considered to be an expert and has extensive experience
in conducting job analyses and developing and validating civil service written tests, oral
boards, and physical ability tests (both entry-level and promotional) for uniformed positions,
including Fire and Police; and
Whereas, the City, through a competitive bidding process, entered into contracts with
Dr. David Santisteban as low bidder, on June 1, 1990, for written test development for the
Sergeant of Police classification; and, on October 10, 1990, for the Lieutenant of Police
classification, and the City entered into contracts with Dr. Santisteban, on October 12, 1992,
for job analysis and test development and validation for the Sergeant of Police and
Lieutenant of Police classifications, and, on July 28, 1993, for job analysis and test
development and validation for the Firefighter II, Fire Lieutenant, and Fire Captain
classifications; and, Dr. Santisteban is therefore familiar with the City of Miami Beach's
procedures; and
..
Whereas, the performance of a job analysis and test development and validation will
be in accordance with the Consent Decree and will help insure the administration of fair
job-related select procedures, to selection of a diverse group of the most qualified
Firefighters I and Police Officer/Trainees and help protect the City against litigation under
Title VII of the Civil Rights Act; and
Whereas, funding for this contract is available from the Police Department and Fire
Department Professional Services budgets.
NOW, THEREFORE, BE IT DULY RESOLVED BY THE MAYOR AND CITY
COMMISSION OF THE CITY OF MIAMI BEACH, FLORIDA, that the Mayor and the
City Commission hereby authorizes the Mayor and City Clerk to execute the attached
Professional Services Agreement between the City and Dr. David Santisteban, in
consideration for an amount not to exceed $93,500, including all out-of-pocket and other
expenses incurred by the Consultant, and further authorizes the City Manager to exercise
the option included in the Consultant's Proposal to develop and administer an
orientation/test preparation program for an additional sum not to exceed $10,000 including
all out-of-pocket and other expenses incurred by the Consultant.
PASSED AND ADOPTED this 20th day of
April
, 1994
Attest:
~uL.~L~
City Clerk, Richard E. Brown
RMC:TCA:GPL:cp
HR DEPT DEV: 04/11/94
CommMemo2:Resoff&P.San
~~
',--
CITY OF
MIAMI
BEACH
CITY HALL 1700 CONVENTION CENTER DRIVE IAMI BEACH FLORIDA 33139
OFFICE OF THE CITY MANAGER TELEPHONE: (305) 173.7010
FAX: (305) 173-7712
COMMISSION MEMORANDUM NO. ~
FROM:
Mayor Seymour Gelber and
Members of the CIty Commlsston
Roger M. Ca-""'-- n AI j f'J A ~
City Manager ~~~
DATE:
April 20, 1994
TO:
SUBJECT:
PROFESSIONAL SERVICES AGREEMENT WITH DR. DAVID SANTISTEBAN FOR JOB
ANALYSIS AND DEVELOPMENT AND VALIDATION OF PERMANENT SELECTION
PROCEDURES FOR FIREFIGHTER I AND POLICE OFFICER{fRAINEE
ADMINISTRATION RECOMMENDATION:
It is recommended that thC Mayor and the City Commission authorize the Mayor and City Clerk to execute the
attached Professional Services Agreement with Dr. David Santisteban for Job Analysis and Development and
Validation of Permanent Selection Procedures for f'trertgbter I and Police Officer {Trainee.
CONTRACf AMOUNT AND FUNDING:
F"1Xed fee of $93,500.00.
Funds are available from the f'tre Department's and Police Department's Professional Services budgets.
BACKGROUND:
On ]une 4, 1993. the court entered the final Consent Decree between the City of Miami Beach and the United
States Department of Justice (DO]). The Consent Decree prohibits continued use of the City's previous written
tests for F"trefighter I and Police Officer fTrainee and allows the City to develop alternative validated selection
procedures for these entry level classifications. Under the terms of tbe Consent Decree, it is required tbat new
written tests for F"trefigbter I and Police OfficerfTrainee be developed in conjunction with the DOJ and that tbe
City contract with recognized experts. The experts selected and the method of validation used by such experts
must be approved by the DO].
Outside legal counsel and the DOJ reviewed and approved the Request For Proposals (RFP) before it was issued
by the City. Notice of RFP was sent to 24 individuals and agencies which perform job analysis and test
development and validation. There was one proposal submitted in response to the RFP, that of Dr. David
Santisteban.
Dr. Santisteban's proposal was reviewed by the DO]. The DO] had no objections to the use of Dr. Santisteban,
his proposal, and the methodologies for job analysis, test development, and test validation.
Through competitive bidding procedures, the City has previously entered into Agreements with Dr. David
Santisteban as low bidder. On June 1, 1990, the City contracted with Dr. Santisteban for the Sergeant of Police
classification and, on October 10, 1990, for the Lieutenant of Police classification. These two contracts included
the development of written tests only and did not include job analysis or test validation. On October 12, 1992.
the City entered into a subsequent Agreement with Dr. Santisteban again for the Sergeant and Lieutenant of
Police classifications. This project was much wider in scope than the first one. The project was started from
saatch and conducted in accordance with the criteria established under Title VII of the Civil Rights Act and the
EEO guidelines. Under this Agreement, the Consultant conducted thorough job- analyses, developed the two
written tests based upon the results of the job analyses, and validated the two written tests. The last contract
the City entered into with Dr. Santisteban was on July 28, 1993, for the classifications of Firefighter n, Fire
Lieutenant, and rtre Captain. This project also included the essential factors of job analyses and test validation 9
as wen as the development of written and oral tests.
AGENDA
ITEM
R-1-f
OATE~
~
coMMISSlON MEMO
PROFESSIONAL SERVICES AGREEMENT
P.2
It is required by the Consent Decree that any test development by tbe City shall include the preparation of
written job analyses for entry-level F"trefigbter I and Police Officer (Trainee. The City and the DOJ entered into
the Consent Decree because the DOJ alleged that the written examinations previously used by the City did nO(
meet the criteria established under Title \11 of the Civil Rights Act and the EEOC Uniform Guidelines on
Employee Selection Procedures.
ANALYSIS:
A. InSTORY OF TEST DEVELOPMENT RFPS
RFLl128-8919O. Police Ser~ant
Dr. David Santisteban
$7,850
Burroughs, Wooten & Assoc.
$7,640 + $5,000 for use of Job Analysis
software + S40/each for more than 20
challenges + $2SO/hr for court testimony
+ travel
Inti. Assn. of Chiefs of Police
$11,000 + $3SO/day for expert testimony +
travel
Cooperative Personnel Services
S13,918 + $67/br for litigation support +
S26/hr for clerical
O'Leary, Brokaw & Associates, Ine.
$20,574 witb 2% discount for prompt
payment
RFLI 12-90/91. Police Lieutenant
Or. David Santisteban
$7,850 - sole bidder
RFLI 33-93/94 rllcfii!tter I and Police Officer
Dr. David Santisteban
$93,500." sole bidder
B. InSTORY OF PROJEcrs BY OR. SANTISTEBAN FOR THE CITY OF MIAMI BEACH
.IMIe ctASSIFlCA TION(S) WSI SERVICES RENDERED
June 01. 1990 Sergeant of Police $ 7,850 Develop Written Test
(RFLI 128-89/90)
October 10, 1990 Lieutenant of Police $ 7,850 Develop Written Test
(RFLI 12-90/91)
Odober 12, 1992 Sergeant of Police $55,800 2 Job Analyses
Lieutenant of Police Develop 2 Written Tests
(rate Reference 8-92/93) Validate 2 Written Tests
July 28, 1993 rlfefigbter n $69,000 3 Job Analyses
rlee Lieutenant Develop 2 Written Tests
rlee Captain Develop 2 Oral Board Tests
Validate all 5 tests
April 20, 1994 rllefJghter I $93,500 2 Job Analyses
Police Officer (Trainee Develop 2 Written Tests
(RFP 33-93/94) Develop 2 Physical Ability Tests
Develop 2 Oral Board Tests
Develop 2 Interview Selection Procedures
Validate all 8 selection procedures
Develop 2 ADA compliant Job Specifications
10
4
..
COMMISSION MEMO
PROFESSIONAL SERVICES AGREEMENT
Page 3
C. CONSENT DECREE
The attached Agreement between the City and Dr. Santisteban includes job analysis and test development and
validation to which tbe DOJ bad no objection and whicb will be conducted in accordance with the provisions of
the Consent Decree. The work to be completed under this Agreement will help insure the Adminisuation of
fair job-related selection procedures to select a diverse group of the most qualilied Firelighters I and Police
Officer rrrainees, and help protect the City against litigation under Federal law, especially Title VII of the Civil
Rights Act. All work will be performed in accordance will all Federal. state, and local laws, regulations, and
guidelines. includin& but not limited to, the EEO Uniform Guidelines 00 Employee Selection Procedures and
the Americans with ~isabilities Act (ADA).
D. EXPERIENCE OF DR. SANT1STEBAN
Dr. Santisteban is considered to be an expert in this field and has extensive experience conducting job analyses
and developing and validating civil service written tests, oral boards. and physical ability tests for entry level
uniformed PoUce and rue classifications. This consultant bas also appeared as an expert witness and been
successful in litigation locaUy in this field, regarding botb his work and as expert witness for the work of others.
.
As listed under Section B above, Dr. Santisteban has completed significant projects for the City of Miami Beach
in the past. This consultant has extensive experience in this field and considerable experience with the City of
Miami Beach.
E. ORlENTATION(I'EST DEVELOPMENT PROGRAM
In the Consultant's Proposal, there is an optional project component of developing and administering an
Orientationrrcst Development Program. This component is not required by the Consent Decree, however. it
is recommended that the Mayor and the Commission authorize the City Manager to exercise tbe option of
extending the Agreement to include this option.
As stated in the Consultant's proposal, .Given the City's concern about increasing the diversity of the work force,
the nature of the Consent Decree, and the possibility that the different ethnic/gender groups may have very
different backgrounds with respect to exposure to testing programs (be it wrinen, oral, or performance) it may
be advisable to develop an orientation program in order to provide information to prospective applicants about
the entire selection process, the requirements of the position, and the do's and don'ts of test taking, including
written, oral. and performance tCSU..
CONCLUSIONz
Dr. Oavid SaDtisteban has extensive experience in job analysis and test development and validation for entry level
Pollce and Fue classifications. Further, Dr. Santisteban has previously performed these services for the City of
Miami Beach for promotional FIre and Police classifications. Dr. Santisteban was the low bidder or the sole
bidder for the last three Requests For Proposals. Therefore, it is recommended that the Mayor and tbe City
Commission authorize the Mayor and the City Clerk to execute tbe attached Professional Services Agreement
between the City and Dr. Santisteban. It is also recommended that the Mayor and City Commission authorize:
the City Manager to exercise the option included in the Consultant's Proposal for an orientation/test preparation
program.
RMC:TCA:GPL:cp
Attachment
11
NATIONAL UNION FIRE INSURANCE CO. OF PITTSBURGH, PA.
THIS IS A CLAIMS MADE POLlCY - PLEASE READ CAREFULLY
11/22Z93 PSYCHOLOGISTS PROFESSIO~AL LIABILITY POLICY
. , } Jo.Jo.Jo. R;::"j~"'AL Jo.Jo.oh '
NOTICE: A LOWER LIMIT OF LIABILITY APPLIES TO JUDGMENTS OR $E'rTC.EMENTSWHEN THERE ARE ALLEGATIONS OF SEXUAL MISCONDUCT
(SEE THE SPECIAL PROVISION .SEXUAL MISCONDUCT" IN THE POLICY)
DECLARATIONS
ACCOUNT NO: FL-SAND950-0 0020341
ITEM 1.(b) ADDITIONAL NAMED INSUREDS:
POLICY NO: PSY-5116044
ITEM 1.(a) NAME AND ADDRESS OF INSURED:
DAVID SANTISTEBAN, PH.D.
95 HERRICK WAY
SUITE 523
CORAL GABLES, FL 33134
TYPE OF ORG:
INDIVIDUAL
ITEM 2. ADDITIONAL INSUREDS:
ITEM 3. . POLICY PERIOD:
FROM: 12/01/93 TO: 12/01/94
12:01 A.M. STANDARD TIME AT THE ADDRESS OF THE INSURED AS STATED HEREIN:
$ 3,000.000
EACH WRONGFUL ACT OR SERIES OF CONTINUOUS. REPEAT
OR INTERRELATED WRONGFUL ACTS OR OCCURRENCE
AGGREGATE
EACH WRONGFUL ACT OR SERIES OF CONTINUOUS. REPEAT
OR INTERRELATED WRONGFUL ACTS OR OCCURRENCE
ITEM 4.
LIMITS OF LIABILITY:
$ 1.000.000
ITEM 5.
DEDUCTIBLE
$ NON E
ITEM 6.
PREMIUM SCHEDULE:
en
Oi
:>
w ITEM 7.
a:
0 ITEM 8.
<(
Q.
<(
CLASSIFICATION NUMBER ANNUAL PREMIUM
PSYCHOLOGISTS 1 439.00 439.00
RETROACTIVE DATE: 12/01/92 TOTAL PREMIUM: 439.00
EXTENDED REPORTING PERIOD
ADDITIONAL PREMIUM (If exercised): $
7 b8. 00
ITEM 9. POLICY FORMS AND ENDORSEMENTS
ATTACHED TO THIS POLICY:
FORM ~ 52349 -8-91 FORM = 53131 8-92 FJ~~ 35
6/+l..,Jlc IC I\I,,",T t. ~1I I PRJ::MllJMHAS BEEN PAID.
.
, . . .
J
\
CITY OF MIAMI BEACH
PROFESSIONAL SERVICES AGREEMENT
BETWEEN THE CITY OF MIAMI BEACH (CITY)
AND DR. DAVID SANTISTEBAN (CONSULTANT)
FOR JOB ANALYSIS, SELECTION PROCEDURES DEVELOPMENT, AND
VALIDATION
FOR FIREFIGHTER I AND POLICE OFFICER/TRAINEE CLASSIFICATIONS
THIS AGREEMENT, made this 20th day of
April
in the
year of 1994, by and between the CITY OF MIAMI BEACH, a Florida
municipal corporation, hereinafter called the "City", which term
shall include its officials, successors, legal representatives, and
assigns, and DR. DAVID SANTISTEBAN, an individual, hereinafter
called the "Consultant" for consultant services as stated herein.
RFP NO. 33-93/94
-1-
/
.
.
Agreement:
city Manager:
Consultant:
Final Acceptance:
Fixed Fee:
Project
Coordinator:
SECTION 1
DEFINITIONS
This written Agreement between the city and
the Consultant.
"city Manager" means the Chief Administrative
Officer of the City.
For the purposes of this Agreement, Consultant
shall be
deemed to be
independent
an
contractor, and not an agent or employee of
the city.
"Final Acceptance" means notice from the city
to the Consultant that the Consultant's
Services are complete as provided in section
4.8 of this Agreement.
Fixed amount paid to the Consultant to allow
for his costs and margin of profit.
An individual designated by the city Manager
to coordinate, direct and review on behalf of
the City all technical matters involved in the
Scope of Work.
-2-
.
.
Risk Manager:
The Risk Manager of the City, with offices at
1700 Convention Center Drive, Third Floor,
Miami Beach, Florida 33139.
services:
All services, work and actions by the
Consultant performed pursuant to or undertaken
under this Agreement described in section 2.
Termination:
Termination of Consultant Services as provided
. . f ,
ln Sectlon 4.9 of thlS Agreement.
Task:
A discrete portion of the Scope of Services to
be
accomplished by the Consultant,
as
described in section 2 herein, if directed and
authorized.
SECTION 2
SCOPE OF WORK AND SERVICES REOUIRED
The Scope of Work for this proj ect to be performed by the
Consultant is set forth in the RFP 33-93/94 and the Consultant's
Proposal which is attached as Exhibit A. All terms of the RFP are
hereby incorporated by reference.
-3-
.
.
SECTION 3
COMPENSATION
3.1 LUMP SUM FIXED FEE
Consultant shall be compensated for the Services
performed herein on a lump sum fixed fee basis of Ninety-three
Thousand Five Hundred Dollars ($93,500.00) to include all out-of-
pocket and other expenses incurred by the Consultant for Job
Analysis: written Test, Oral Board, Physical Ability Test, and
Selection Interview Development and Validation: Administration of
Oral Board and Selection Interview to 15 applicants each: and
Development of Job Specifications for Firefighter I and police
Officer/Trainee. The city Manager may exercise the option included
in the Consultant's Proposal to develop and administer an
orientation/test preparation program for an additional sum not to
exceed Ten Thousand Dollars ($10,000) to include all out-of-pocket
and other expenses incurred by the Consultant.
3.2 METHOD OF PAYMENT
3.2.1 Monthlv Payment
Payment shall be made to the Consultant monthly pursuant
to invoices submitted by the Consultant which detail percentage of
completion of each task. Invoices shall be accompanied by a
narrative progress report which supports the invoices, and shall
contain a statement that the items set forth therein are true and
correct and in accordance with the Agreement. payments of such
invoices shall be made within 30 days of receipt by city.
-4-
,
.
SECTION 4
GENERAL PROVISIONS
4.1 RESPONSIBILITY OF THE CONSULTANT
with respect to the performance of the services, the
Consultant shall exercise that degree of skill, care, efficiency,
and diligence normally exercised by recognized professionals with
respect to the performance of comparable services. In its
performance of the Services, the Consultant shall comply with all
applicable laws, ordinances, and guidelines including but not
limited to applicable regulations of the city, county, State,
Federal Government, ADA, EEO Regulations and Guidelines.
Additionally, the Consultant shall provide such information as may
be requested by the Department of Justice and directed by the city
under the terms of Consent Decree as entered in United States v.
citv of Miami Beach, Case No. 91-2926-CIV-NESBITT.
4.2
PUBLIC ENTITY CRIMES
state of Florida Form PUR 7068, Sworn Statement under
Section 287.133 (3) (a) Florida Statute on Public Entity Crimes,
Exhibit "C".
4.3 PROJECT MANAGEMENT
The Consultant shall appoint a qualified individual
acceptable to the City to serve as Project Manager for the Services
who shall be fully responsible for the day-to-day activities under
this Agreement and who shall serve as the primary contact for the
city's project Coordinator.
-5-
.
.
.
,
4.4 TIME OF COMPLETION
The services to be rendered by the Consultant shall be
commenced upon receipt of a written Notice to Proceed from the city
subsequent to the execution of this Agreement, and Consultant shall
adhere to the completion schedule as outlined in the Consultant's
Proposal attached hereto as Exhibit "A".
A reasonable extension of time shall be granted in the
event the work of the Consultant is delayed or prevented by the
City or by any circumstances beyond the reasonable control of the
Consultant, including weather conditions or acts of God which
render performance of the Consultant's duties impracticable.
Such extensions of time shall not be a basis for any
claim by the Consultant for additional compensation, unless an
extension is based on a delay caused solely by the City and is in
excess of sixty (60) days.
4.5 NOTICE TO PROCEED
Unless directed by the city otherwise, the Consultant
shall proceed with the work only upon issuance of a Notice to
Proceed by the City.
4.6 OWNERSHIP OF DOCUMENTS AND EQUIPMENT
All documents, including but not limited to test and test
scoring data or programs stored electronically, prepared by the
Consultant pursuant to this Agreement, are related exclusively to
the Services described herein. They are intended or represented to
be suitable for reuse by the city.
-6-
I
.
"
4.7
INDEMNIFICATION
Consultant agrees to indemnify and hold harmless, the
City of Miami Beach and its officers, employees, and agents, from
and against any and all actions, claims, liabilities, losses, and
expenses, including, but not limited to, attorneys' fees, for
personal, economic, or bodily injury, wrongful death, loss of or
damage to property, in law or in equity, which may arise or be
alleged to have arisen from the negligent acts or omission or other
wrongful conduct of the consultant, employees, or agents in
connection with the Consultant's performance of Services pursuant
to this Agreement; and to that extent, the Consultant shall pay all
such claims and losses and shall pay all such costs and judgements
which may issue from any lawsuit arising from such claims and
losses, and shall pay all costs and attorneys fees expended by the
City in the defense of such claims and losses, including appeals.
The parties agree that one percent (1%) of the total Compensation
to the Consultant for performance of this Agreement is the specific
consideration from the City to the Consultant for the Consultant's
Indemnity Agreement.
The Consultant's obligation under this article shall not
include the obligation to indemnify the City of Miami Beach and its
officers, employees, and agents, from and against any actions or
claims which arise or are alleged to have arisen from negligent
acts or omissions or other wrongful conduct of the City and its
officers, employees, and agents. The parties each agree to give
the other party prompt notice of any claim coming to its knowledge
that in any way directly or indirectly affects the other party.
-7-
.
.
.
,
4.8
INSURANCE REOUIREMENTS
The Consultant shall not commence any work pursuant to
this Agreement until all insurance required under this section has
been obtained and such insurance has been approved by the City's
Risk Manager. The Consultant shall maintain and carry in full
force during the term of this Agreement and throughout the duration
of this project the following insurance:
1.
Consultant
Prof.essional
Liability
in
the
amount
of
$1,000,000.00. For a claims made policy, the Consultant agrees
to carry five (5) years tail coverage after work is completed,
or maintain a comparable policy for five (5) years, provided
that such comparable policy shall include coverage for prior
acts effective from the date of execution of this Agreement.
A certified copy of the Consultant's (and any subconsultants')
Insurance Policy must be filed and approved by the Risk
Manager prior to commencement.
2. Workers Compensation & Employers Liability as required per
Florida statutes.
3. Thirty (30) days written notice of cancellation or substantial
modification in the insurance coverages must be given to the
City by the Consultant and his insurance company.
4. The insurance must be furnished by insurance companies
authorized to do business in the state of Florida and approved
by the City's Risk Manager.
5. Original certificates of insurance for the above coverages
must be submitted to the City's Risk Manager for approval
prior to any work commencing. These certificates will be kept
on file in the office of the Risk Manager, 3rd Floor, city
-8-
"
Hall.
6. The Consultant is responsible for obtaining and submitting all
insurance certificates for their consultants.
All insurance policies must be issued by companies authorized to do
business under the laws of the state of Florida. The companies must
be rated no less than "B+" as to management and not less than
"Class VI" as to strength by the latest edition of Best's Insurance
Guide, published by A.M. Best Company, Oldwick, New Jersey, or its
equivalent, subject to the approval of the City's Risk Manager.
Compliance with the foregoing requirements shall not relieve the
Consultant of the liabilities and obligations under this section or
under any other portion of this Agreement, and the City shall have
the right to obtain from the Consultant specimen copies of the
insurance policies in the event that submitted certificates of
insurance are inadequate to ascertain compliance with required
coverages.
4.8.1 Endorsements
All of Consultant's certificates, above, shall contain
endorsements providing that written notice shall be given to the
City at least thirty (30) days prior to termination, cancellation
or reduction in coverage in the policy.
-9-
,
,
4.8.2 Certificates
Unless directed by the city otherwise, the Consultant
shall not commence any services pursuant to this Agreement until
the City has received and approved, in writing, certificates of
insurance showing that the requirements of this section (in its
entirety) have been met and provided for.
4.9 FINAL ACCEPTANCE
When the Consultant's services have been completed, the
Consultant shall so advise the City in writing. Final Acceptance
shall not constitute a waiver or abandonment of any rights to
remedies available to the City under any other section of this
Agreement.
4.10
TERMINATION, SUSPENSION AND SANCTIONS
4.10.1 Termination for Default
If through any cause within the reasonable control of the
Consultant, the Consultant shall fail to fulfill in a timely
manner, or otherwise violate any of the covenants, agreements, or
stipulations material to this Agreement, the City shall thereupon
have the right to terminate the Services then remaining to be
performed by giving written notice to the Consultant of such
termination which shall become effective upon receipt by the
Consultant of the written termination notice.
In that event, all finished and unfinished documents,
data, studies, surveys, drawings, maps, models, photographs,
reports and other work products prepared by the Consultant and its
-10-
..
subcontractors shall be properly delivered to the city and the city
shall compensate the consultant in accordance with section 3 for
all services performed by the Consultant prior to Termination.
Notwithstanding the above, the Consultant shall not be
relieved of liability to the City for damages sustained by the city
by virtue of any breach of the Agreement by the Consultant and the
City may reasonably withhold payments to the Consultant for the
purposes of set off until such time as the exact amount of damages
due the city from the Consultant is determined.
4.10.2 Termination for convenience of city
The city may, for its convenience, terminate the Services
then remaining to be performed at any time by giving written notice
to Consultant of such termination, which shall become effective
seven (7) days following receipt by Consultant of the written
termination notice. In that event, all finished or unfinished
documents and other materials as described in section 2 shall be
properly delivered to the city. If the Agreement is terminated by
the city as provided in this section, the city shall compensate the
Consultant in accordance with section 2 for all Services actually
performed by the Consultant and reasonable direct costs of
Consultant for assembling and delivering to City all documents.
Such payments shall be the total extent of the city's liability to
the Consultant upon a Termination as provided for in this Section.
-11-
"
4.10.3 Termination for Insolvencv
The city also reserves the right to terminate the
remaining services to be performed in the event the Consultant is
placed either in voluntary or involuntary bankruptcy or makes an
assignment for the benefit of creditors. In such event, the right
and obligations for the parties shall be the same as provided for
in section 4.10.2.
Sanctions for NoncomDliance with Nondiscrimination
provisions
In the event of the Consultant's noncompliance with the
nondiscrimination provisions of this Agreement, the City shall
impose such Agreement Sanctions as the City or the state of Florida
may determine to be appropriate, including but not limited to
withholding of payments to the Consultant under the Agreement until
the Consultant complies and/or cancellation, termination or
suspension of the services, in whole or in part. In the event the
City cancels or terminates the Services pursuant to this section
the rights and obligations of the parties shall be the same as
provided in section 4.10.2.
4.10.4
4.10.5 Chanqes and Additions
Each such change shall be directed by a written Notice
signed by the duly authorized representatives of the Consultant.
Said Notices shall provide an equitable adjustment in the time of
performance, a reallocation of the task budget and, if applicable,
any provision of this Agreement which is affected by said Notice.
The City shall not reimburse the Consultant for the cost of
-12-
.
,
"
preparing Agreement change documents, written Notices to Proceed, '
or other documentation in this regard.
4.11 ASSIGNMENT. TRANSFER OR SUBCONTRACTING
The Consultant shall not subcontract, assign, or transfer
his/her rights or obligations under this Agreement without the
written consent of the city. When applicable and upon receipt of
such consent in writing, the Consultant shall cause the names of
the individuals or consulting firms responsible for the major
portions of each separate specialty of the work to be inserted into
the pertinent documents or data. The Consultant shall include in
such subcontracts the appropriate versions of the sections of this
Agreement as are necessary to carry out the intent of this
Agreement, as instructed by the City.
4.12
EQUAL EMPLOYMENT OPPORTUNITY
In connection with the performance of this Agreement, the
Consultant shall not discriminate against any employee or applicant
for employment because of age, ancestry, citizenship or intending
citizenship status, color, disability, gender, marital status,
national origin, place of birth, race, religion, or sexual
orientation. The Consultant shall take affirmative action to
ensure that applicants are employed and that employees are treated
during their employment without regard to their age, ancestry,
citizenship or intending citizenship status, color, disability,
gender, marital status, national origin, place of birth, race,
religion, or sexual orientation. Such action shall include, but not
be limited to the following: employment, upgrading, demotion, or
-13-
,
.
"
termination; recruitment or recruitment advertising; layoff or
termination; rates of pay, or other forms of compensation; and
selection for training, including apprenticeship.
4.13
CONFLICT OF INTEREST
The Consultant agrees to adhere to and be governed by the
Metropolitan Dade County Conflict of Interest Ordinance (No. 72-
82), as amended; and by the city of Miami Beach Charter and Code,
which are incorporated by reference herein as if fully set forth
herein, in connection with the Agreement conditions hereunder.
The Consultant covenants that it presently has no
interest and shall not acquire any interest, direct or indirectly
which should conflict in any manner or degree with the performance
of the services. The Consultant further covenants that in the
performance of this Agreement, no person having any such interest
shall knowingly be employed by the Consultant. No member of or
delegate to the congress of the United states shall be admitted to
any share or part of this Agreement or to any benefits arising
therefrom.
4.14 PATENT RIGHTS: COPYRIGHTS: CONFIDENTIAL FINDINGS
Any patentable result arising out of this Agreement, as
well as all information, design specifications, processes, data and
findings, shall be made available to the city for public use.
No reports, other documents, articles or devices produced
in whole or in part under this Agreement shall be the subject of
-14-
.
I
~
any application for copyright or patent by or on behalf of the
Consultant or its employees or subcontractors.
4.15
NOTICES
All communications relating to the day-to-day activities
shall be exchanged between the project Manager appointed by
Consultant and the project Coordinator designated by the City. The
Consultant's project Manager and the city's Project Coordinator
shall be designated promptly upon commencement of the Services.
All other notices and communications in writing required
or permitted hereunder may be delivered personally to the
representatives of the Consultant and the City listed below or may
be mailed by registered mail, postage prepaid (or airmailed if
addressed to an address outside of the city of dispatch) .
Until changed by notice in writing, all such notices and
communications shall be addressed as follows:
TO CONSULTANT:
David santisteban, Ph.D.
The Alahambra West
95 Merrick Way- suite 523
Coral Gables, Florida 33134
(305) 445-2471
TO CITY:
city Manager
city of Miami Beach
City Hall, 4th Floor
1700 Convention Center Drive
Miami Beach, FL 33139
(305) 673-7010
-15-
.
\
~
WITH COPIES TO:
Office of the city Attorney
city of Miami Beach
1700 Convention Center Drive
Miami Beach, FL 33139
Notices hereunder shall be effective:
If delivered personally, on delivery; if mailed to an
address in the city of dispatch, on the day following the
date mailed; and if mailed to an address outside the city
of dispatch on the seventh day following the date mailed.
4.16
LITIGATION JURISDICTION
Any litigation between the parties, arising of, or in
connection with this Agreement, shall be initiated either in the
court system of the state of Florida or the united states District
Court for the Southern District of Florida.
4.17
ENTIRETY OF AGREEMENT
This writing and the Scope of Services embody the entire
Agreement and understanding between the parties hereto, and there
are no other agreements and understandings, oral or written with
reference to the subject matter hereof that are not merged herein
and superseded hereby.
The Scope of Services are hereby
incorporated by reference into this Agreement to the extent that
the terms and conditions contained in the Scope of Services are
consistent with the Agreement. To the extent that any term in the
Scope of Services is inconsistent with this Agreement, this
-16-
,
.
. ,
Agreement including the Scope of Services shall prevail.
No alteration, change, or modification of the terms of
this Agreement shall be valid unless amended in writing, signed by
both parties hereto, and approved by the city commission of the
city of Miami Beach.
This Agreement, regardless of where executed, shall be
governed by and construed according to the laws of the State of
Florida.
4.18 LIMITATION OF CITY'S LIABILITY
The city desires to enter into this Agreement only if in
so doing the City can place a limit on the city's liability for any
cause of action for money damages due to an alleged breach by the
city of this Agreement, so that the city's liability for any such
breach never exceeds the sum of Ninety-Three Thousand Five Hundred
Dollars and no/100 ($93,500.00). Consultant hereby expresses
his/her willingness to enter into this Agreement with Consultant's
recovery from the city for any damage action for breach of contract
to be limited to a maximum amount of $93,500.00 less the amount of
all funds actually paid by the city to Consultant pursuant to this
Agreement.
Accordingly, and notwithstanding any other term or
condition of this Agreement, Consultant hereby agrees that the city
shall not be liable to the Consultant for damages in an amount in
excess of $93,500.00 which amount shall be reduced by the amount
actually paid by the city to consultant pursuant to this Agreement,
-17-
for any action or claim for breach of contract arising out of the
performance or non-performance of any obligations imposed upon the
city by this Agreement. Nothing contained in this section or
elsewhere in this Agreement is in any way intended to be a waiver
of the limitation placed upon city I s liability as set forth in
Florida statutes, section 768.28.
4.19
ARBITRATION
Any controversy or claim for money damages arising out of
or relating to this Agreement, or the breach hereof, shall be
settled by arbitration in accordance with the commercial
Arbitration Rules of the American Arbitration Association, and the
arbitration award shall be final and binding upon the parties
hereto and subject to no appeal, and shall deal with the question
of the costs of arbitration and all matters related thereto. In
that regard, the parties shall mutually select one arbitrator, but
to the extent the parties cannot agree upon the arbitrator, then
the American Arbitration Association shall appoint one. Judgement
upon the award rendered may be entered into any court having
jurisdiction, or application may be made to such court for an order
of enforcement. Any controversy or claim other than a controversy
or claim for money damages arising out of or relating to this
Agreement, or the breach hereof, including any controversy or claim
relating to the right to specific performance shall be settled by
litigation and not arbitration.
-18-
,
,
IN WITNESS WHEREOF, the parties hereto have caused this
Agreement to be executed by their appropriate officials, as of the
date first entered above.
FOR CITY:
ATTEST:
By:jG~ C.~
City Clerk, Richard Brown
FOR CONSULTANT:
WITNESS:
~~ ~oej~
Human Resource Department
GPL:cp
Contract #2:\RFP93-94
DAVID SANTISTEBAN, Ph.D.
By:
-19-
EXHIBIT A
,
~ i .
DAVID SANTISTEBAN. PH. D.
PSYCHOLOGIST
and
Proposal to Conduct Job Analyses,
Selection Procedure Development,
Validation for Firefighter I and police Officer/Trainee
February, 1994
THE ALHAMBRA WEST. 95 MERRICK WAY. SUITE 523 e CORAL GABLES. FLORIDA 33134 · (305) 445-2471
. ,
TABLE OF CONTENTS
I.
Firm
II.
Experience of Firm
III.
Project Plan
A. Job Analyses
B. Written Tests
C. Oral Boards
D. physical Ability Tests
E. Selection Interviews
F. Revise Job Specifications
G. Pilot Test Preparation/Orientation
H. General Services
IV.
Documentation and Follow-Up
V.
Project Security
VI.
Litigation Experience
VII.
Insurance
VIII.
Affirmative Action
IX.
Diversification of Income
X.
Project Schedule
XI.
Cost Breakdown
Exhibits
1. Curriculum Vitae
2. Work Sample
3. Litigation Experience
4. Liability Insurance Certificate
2
Page
3
3
5
8
8
9
10
10
Program 1 1
1 1
12
14
15
15
15
16
16
17
I ,
I. FIRM
This proposal is being submitted by an independent consultant,
Dr. David Santisteban, who will be responsible for the development
and implementation of the complete project. As will be seen in
subsequent sections, Dr. Santisteban has extensive experience with
the scope of work being addressed in this proposal. His personal
involvement ensures the quality of work from proposal development
to successful project conclusion. Dr. Santisteban's Curriculum
vitae is included as Exhibit 1.
II. EXPERIENCE OF FIRM FOR PROPOSED PROJECT
The Proposer has extensive experience in conducting job
analyses and developing and validating civil service written tests,
oral boards, and physical ability tests (both entry level and
promotional) for uniformed positions, including Fire, police, and
Correctional Officers. The methodology employed has been primarily
that of Content Validity with extensive input from departments'
Subject Matter Experts in all phases of the project (i.e., task
analysis, test content areas, passing points) in order to insure
job relevancy of tests and to link cut-off scores to performance
criteria:
Major clients in comparable projects include Metropolitan Dade
County, City of Coral Gables, and City of Miami Beach. More
specifically, the Proposer has been responsible for the development
of police and fire entry level and promotional exams. Validation
activities have included task analyses to determine the areas to
be assessed, development and administration of the different
components of the assessment process (e. g., wri tten test, oral
boards, physical ability tests), and the relative weight to be
assigned to each.
The Proposer is currently in the process of completing two
major projects for the City of Miami Beach which included task
analyses and written test development for the police promotional
series (Sergeant and Lieutenant) and task analyses, written tests,
and oral boards for the Fire promotional series (Firefighter II,
Lieutenant, Captain).
The Proposer has also recently comp 1 eted task analyses for
the Metropolitan Dade County entry level classifications of police
Officer and Firefighter. Work with these classifications have also
included the development of entry level written tests, oral boards,
and physical capability tests.
3
;,}~ .J<
",h".
~.
I
,
A. Metropolitan Dade County
1 . For the 1 ast ten years, Dr., Sant i steban has served as
the in-house consu 1 tant in the area of job anal ys is, test
development,. and val idation. In this capacity, he has
been responsible for the development of entry level anq
promotional tests for over 100 job classifications,
including the uniformed positions (Fire, police,
Corrections).
2. Contact persons:
Mr. Donald Allen
Director, Personnel Services Division
Metro Dade Government Center
111 NW 1st Street, suite 2020
Miami, Florida 33131
(305) 375-2670
Mr. Lee Kraftchick
Assistant County Attorney
Metro Dade Government Center
111 NW 1st Street, suite 2800
Miami, Florida 33131
(305) 375-5725
B. CITY OF CORAL GABLES
1 . For the last seven years the Proposer has provided test
development and validation services to the Coral Gableg
Fire and police Departments. These services hav~
included job analyses and development and validation Of
promotional multiple-choice written tests based on all
established reading list.
2. Contact person:
Alice Smith, Personnel Analyst
Employee Relations Department
2327 Salzedo street
Coral Gables, Florida 33134
(305) 442-6472
C. CITY OF MIAMI BEACH
1. The Proposer has developed written tests for the Police
and fi re promotional series. In the Fi re Department
these tests were complimented by an oral board. The
multiple-choice tests were based on a promotional reading
list. The content and relative weight of the areas
tested were based on extensive job analyses which
included surveys, panel of subject matter experts, field
observations, and interviews.
2. Contact person:
Gail Poe-liu
Examinations Supervisor
City of Miami Beach
City Hall
1700 Convention Center Drive
Miami Beach, Florida 33139
(305) 673-7520
III. PROJECT PLAN
The methodology being proposed is based on a Content Validity
model of test development which calls for extensive documentation
of major Knowledges, Skills, and Abilities (KSAs) through a job
analysis and adequate sampling of such KSAs through the selection
instruments. The methodological procedures to be followed for each
task included in the proposed scope of work are described below.
All aspects of the project will be completed in a manner which
complies with the terms of the Consent Decree and which meets with
the approval of the Department of Justice and the Court.
Task A - Job Analvsis
The job analysis methodology to be used in this project will
rely heavily on the use of questionnaire methods and input from
Subject Matter Experts (SMEs). The methodology conforms to
professional and legal standards as outlined in the American
Psychological Association's (APA) Standards on Education and
Psychological Tests, the APA's Division of
Industrial/Organizational Psychology Principles on the Validation
and Use of Personal Selection Procedures, and the Uniform
Guidelines on Employee Selection. The requirements of the American
with Disabilities Act will also be incorporated into the
methodology.
A. Develooment of Initial Survey Questionnaires:
A panel of SME's will be selected to provide detailed
information on the classification through the use of an open-ended
5
, .
Position Description Questionnaire. This instrument will collect
information on key functions, tasks, abilities, know1edges and
situations relevant for effective performance on the job.
Individual responses will be summarized in order to identify areas
most common to the position. In addition, panel members wi 11
review tasks, know1edges, and abilities inventories from previous
job analyses and existing job specifications in order to determine
their relevance to the classification.
Panel members will be selected on the basis of:
1) familiarity with the position (i.e., incumbents
and their immediate supervisors).
2) representation of the various geographic locations (i.e.,
stations) served by the department.
3) ethnic and gender representation of the work force.
B. Develooment and Distribution of Job Analysis Questionnaire:
A final questionnaire will be developed based on the input
from the SME Panel. The questionnaire will consist of several
parts:
Part 1 - RatinQs of Imoortance and Freauency of Tasks
Information obtained from the open ended questionnaires and
input from the SME Panel will be used to generate a list of tasks
that could be related to successful performance on the job.
Respondents wi 11 be asked to rate each task on a Likert-type seal e
according to its importance and frequency of occurrence.
Questionnaire responses will
tasks considered important for
pos it ion. Tasks wi th ratings of
from further analyses.
be analyzed to determine those
successful performance in the
low importance will be dropped
Part 2 - CateQorization of Tasks into Duty Areas
Existing literature in the field and job analysis information
for the position will be used to identify categories of activities
that would seem to describe the majority of tasks considered
important by the respondents. These categori es or .. Duty Areas"
will be generated based on the content of the tasks. These Duty
Areas will be reviewed by the SME Panel.
In order to support the integrity and relevance of the Duty
Areas, respondents will be asked to categorize each of the
important tasks according to the relevant duty areas. This linkage
will provide an index of the relative weight or importance of each
Duty Area.
6
, .
Parts 3 and 4 - ImDortance RatinQs of KnowledQes and Abilities
Respondents will be asked to rate the importance of abilities,
knowledges that might be required in order to
perform each of the duty areas previously identified.
abilities, skills, and knowledges will be drawn from
analyses and will be reviewed and updated by the SME
skills and
successfully
The lists of
previous job
Pane 1 .
The linkage between knowledges, skills, and abilities and the
respective duty areas will provide two important indices:
a) the relative weights to be assigned to instruments
measuring knowledges (job domain measured by written
test), those measuring abilities (job domain measured by
oral boards), and those measuring skills (physical
capability tests).
b) the re 1 at i ve we i ght to be ass i gned to each know 1 edge,
skill, and ability areas.
Part 5 - Trainabilitv RatinQs of KSAs
Respondents wi 11 rate each KSA accord i ng to how eas i 1 y it
could be enhanced on the job or through a formal training program.
Selection device development will focus on measuring KSAs that the
candidates are expected to possess prior to any formal or on the
job training or areas where the candidates can demonstrate their
ability to learn required material (such as in the police and fire
academi es) .
Part 6 - Identification of Essential Functions
In addition to the ratings indicated above, respondents will
be asked to provide input on the essential functions of the job and
work condition requirements following current ADA guidelines.
The final questionnaire will be distributed to a large
representative sample of incumbents. The sampling strategy to be
used will depend on the actual breakdown of the work force, based
on such factors as ethnicity, gender, work locations, and job
functions. This information will be obtained from the city once the
contract is awarded. The overriding guidelines to be followed
include representation of ethnic and gender groups at each step of
the process and use of incumbents and immediate supervisors who are
familiar with the generic functions of the position.
It is recommended that the job analysis activities be
presented to the incumbents as an important task for the respective
department (police or Fire). Therefore, cover letters, traces, and
follow up activities need to take place with the collaboration of
the department's administration. By the same token, the
7
. '
respondents need to be assured that the confidentiality of their
responses will be safeguarded and that the information obtained
will be reported only as group data without identifying
individuals.
Task B - Develooment of Written Tests
The methodology being proposed is based on a Content Validity
model of test development which calls for adequate sampling of the
ability to learn the major KSAs required of the position.
The samp 1 i ng of content areas wi 11 be based on the actual
content of the major KSAs. Items wi 11 be wri tten in order to
discriminate between different levels of performance on the
re 1 evant areas. Profess i ona 11 y accepted methods of item
construction will be followed in order to insure such desirable
psychometric properties as good level of difficulty, clear
discrimination among test takers, freedom from ethnic and gender
bias.
The above procedure will insure that all items are at an
appropriate level of difficulty for the position, that all items
are linked to specific KSAs, and that redundancy and confounding
are avoided.
Task C - Job Simulation Oral Boards
Job Simulation Oral Boards are being recommended as a cost-
effect i ve a 1 ternat i ve to assessment center methodology. The job
simulation boards are simpler to administer than assessment
centers, and except for the role playing type of simulations, they
provide a wide range of scenarios that can be used to assess major
ability areas. This is especially the case with entry level
positions such as police Officer/Trainee and Firefighter.
As indicated above, the Proposer has recently completed the
development and administration of job simulation oral boards for
the classifications of Fire Lieutenant and Fire Captain in the City
of Miami Beach. The s i mu 1 at ions i nc 1 uded emp 1 oyee counse 1 i ng,
dealing with cultural diversity in the work force, fireground
tactics, and community relations. The instruments developed were
well received by the panel of raters and allowed for the smooth and
efficient processing of the candidates.
The oral test selection device will be developed on the basis
of job analysis results. This device will measure the major KSAs
or behavioral dimensions required to satisfactorily perform the
duties of the position. The items will also reflect the relative
importance of the areas and will seek to discriminate among
anticipated levels of job performance.
8
, '
The instrument will have a standardized and detailed procedure
for administration, presentation of items, rating of responses, and
scoring. It will be administered by a panel of at least three
individuals with appropriate ethnic and gender representation.
Training will be provided in order to standardize the
administration and rating process. The raters will also be
selected on the basis of previous experience with oral boards and
wi 11 ingness to train in a structured assessment process. The
training of raters will consist of an explanation of the instrument
and guidelines for administration and rating to ensure consistency
both within and between raters. Raters must be able to evaluate
candidates based on observed behaviors or responses and
substantiate their ratings with step by step documentation. The
panel will do their ratings independently and the ratings will be
aggregated statistically.
The Job Simulation Oral Boards will consist of job related
scenarios developed on the basis of critical incidents provided by
SMEs. Critical incidents provide information on effective and
i neffect i ve performance on the target pos i t i on thus 1 ink i ng the
candidate's response to an anticipated performance level.
Information is also provided on the level of difficulty of the
scenarios thus allowing for a broad pool of items that can be
balanced and mixed so that candidates are administered parallel
forms of the instrument if needed.
The candidates will be given a brief orientation with respect
to time frame, general nature of questions/scenarios, and
parameters of the interview (e.g., follow-up questions).
Task 0 - physical Ability Tests
The Proposer will develop physical ability tests (PAT) as needed
for both classifications based upon the results of the job
analysis. The PAT will consist of work sample type of simulations
which are content valid and are administered and scored in a
standard format. In order to establish passing scores, a pilot
study may be conducted wi th a samp 1 e of job incumbents in each
classification. One of the major issues in the utilization of PATs
in fire departments has to do with their impact on female
applicants as well as the swimming component's impact on Blacks.
These issues will be addressed in the pilot study.
The PAT will be developed on the basis of the state-of-the-
art methodology in this field. The Proposer has recently completed
a development and validation project for a physical capability test
for firefighters. The methodology used proved successful in
providing a product that was highly relevant to the position while
at the same time mitigating the possible adverse impact problems
noted above.
9
Task E - Job Interview Selection Device
1. Development of a Job Interview Selection Device:
The Proposer will develop, facilitate and monitor
administration of, analyze, interpret results, and validate a job
interview selection device and process based upon the job analysis
in order to compliment the oral boards.
In differentiating it from the Job Simulation Oral Board, the
Selection Interview will focus more on the past work history of the
candidate, how job experience links to the position being sought,
and the candidate's ability to interact with the interview panel,
including overall presentation and communication skills. The goal
is to make th is step in the se 1 ect i on process more targeted at
achieving a match between the applicant and the department.
The i terns to be used in th is instrument will be 1 inked to
specific KSAs or behavioral dimensions identified through the job
analysis and will reflect the relative importance or criticality
to successful job performance. The desired responses to the items
and standards by which candidates will be evaluated and scored will
also be provided.
This instrument could also be administered by an interview
panel with the appropriate ethnic/gender representation. However,
in contrast to the Oral Boards, the panel could have representation
of department personnel. Again, the idea is to make this step in
the process more position-specific with departmental staff being
able to provide input into the selection process.
Training will be provided for these raters along with practice
exercises where ratings can be discussed and standardization can
be achieved.
2. Review Job Interview Selection Device Prepared by City:
The Proposer wi 11 review and recommend any improvements or
modifications to the job interview selection device and process
submitted by the City. The review will be based upon the job
analysis results, and will include an evaluation of
standards/criteria by which applicants would be evaluated.
Task F - Revise Job SDecifications
The current job specifications for the police Officer/Trainee
and Firefighter classifications will be updated based on the
results of the job analyses.
The results will include, but not be limited to, information
on: nature of work; examples of work performed; knowledges, skills,
and abilities; work condition requirements, supervision received
10
. ' .
and supervision exercised. Essential job junctions will be noted
as per ADA. A SME panel will provide feedback on the final draft
of the revised job specification.
Task G - Pilot Test PreDarat;on/Or;entat;on Program
Given the City's concern about increasing the diversity of the
work force, the nature of the Consent Decree, and the possibility
that the different ethnic/gender groups may have very different
backgrounds with respect to exposure to testing programs (be it
written, oral, or performance) it may be advisable to develop an
orientation program in order to provide information to prospective
applicants about the entire selection process, the requirements of
the pos it ion, and the do's and don'ts of test-tak i ng, i ncl udi ng
written, oral, and performance tests.
There are a coup 1 e of mode 1 s that cou 1 d be used in th is
orientation/test preparation program. One model could include a
general one-time orientation/test preparation session for all
applicants who meet the minimum requirements. An alternative model
could have more in depth sessions as applicants progress through
the selection process in order to help them become more familiar
with the next test that they will be taking. Again, the extent of
the program could vary from a general overview of the process to
a more detailed preparation course which could include sessions on
how to interview and a videotape of the physical ability test.
This orientation/preparation program would take into account
the different socio-cultural backgrounds of the applicant pool and
would represent a very proactive way of "level ing the playing
field" for motivated but possibly disadvantaged minority
applicants.
Task H - General Services
In addition to the scope of work enumerated above, the Proposer
wi 11 :
1. Prepare and deliver such presentations and participate
in such discussions, proceedings, as may be requested to
comply with Department of Justice and court requests and
requirements.
2. Remain available on an as-needed basis to answer any
questions or clarify or interpret the results of any
individual's results.
3. Provide supplemental written material and training to
City personnel to faci 1 itate the use of the selection
device results and reports.
1 1
, ' .
4. Defend and/or represent the City and testify on the
City's behalf if any claims or allegations are made
regarding the services provided including, but not
limited to, the validity of the test battery or
evaluation methods or results.
5. Maintain all raw test results and notes with respect to
each individual evaluated for the time period set forth
in Florida records retention or other applicable laws or
as specified by the City, whichever is greater.
6 .
Research and investigate to insure that all
procedures, results, and interpretations are
reliable, cross-cultural, and legally defensible
purposes used.
tests,
valid,
for the
IV - DOCUMENTATION AND FOLLOW-UP
Technical Reoorts:
The process involved in each of the above tasks (e.g., Job
Analysis, Written Tests, Oral Boards, Selection Interview) will be
documented through comprehensive technical reports, including
methodologies, rationales, procedures, and results. Summary and
descriptive statistics will be provided for the candidates'
performance on all selection instruments. An applicant flow chart
will be developed to include each step in the selection process and
the outcome by ethnic and gender groups. In addition, ethnic, age,
and gende r group compar i sons will be conducted on the bas is of
passing rates and average scores. Recommendations will be made for
future reduction of any adverse impact, as needed.
The psychometric properties of the instruments
identified, including but not limited to, reliability,
tendencies, and difficulty and discrimination indices.
wi 1 1 be
central
A sample technical report (certain sections only) for a
uniformed promotional classification is included in Exhibit 2.
Resoonse to Item Challenges:
For each of the se 1 ect ion instruments, the Proposer wi 11
review candidates' challenges and will provide a written response
with justifications for upholding or denying such challenges.
Determining Passing Score:
The methodology for determining the appropriate passing score
for each of the procedures will be based on ratings provided by
SMEs in terms of what they consider to be minimum proficiency
levels.
The Angoff Method will be used to establish a cut-off for the
12
. .
written test. This procedure is well recommended both from a legal
and professional standpoint, and includes the following steps:
a) a panel of Subject Matter Experts is selected based on
security concerns, fami 1 iarity with the position, and
ethnic/gender representation. These are usually Captains
or above who have already gone through the Civil Service
testing process;
b) concurrent with test administration, the panel meets to
provide ratings on each test item. A rating of relevancy
is obtained to document the appropriateness of each test
item to the position. A rating of item difficulty is
obtained to link the passing score to an expected
performance level.
c) The level of difficulty ratings are tabulated and a
projected passing score is established. The projected
passing score is then used as a guideline for
establishing the actual cut-off by taking into account
the psychometric properties of the test, including level
of difficulty and the distribution of scores.
DeveloD and Recommend Method of Final Selection and Any Other
Process:
Based on the results of the above-named tasks, the Proposer will:
1. Prepare written recommendations as to the method(s) of
referring qualified candidates from the Eligible List to
the Appointing Authority for interview, consideration and
se 1 ect ion. Recommendat ions w j 11 i nc 1 ude number of
candidates to be referred, method of selection (e.g.,
banding of score), and appropriate supporting rationale
and criteria.
2. Prepare and prov ide wr i tten recommendation of methodo logy
and procedures for making final selections from among
qualified candidates.
3. Prepare written recommendations on any additional
selection steps that may be needed, such a background
investigation, drug use history, polygraph, psychological
and/or psychiatric evaluations. The Proposer will make
these recommendations based not only on the results of
the present study, but also on the state of the art
literature in the field.
A means of combining the scores wi 1 1 also be provided as
applicable. There are a couple of alternatives which will be
discussed with the City, including rank ordering and banding. For
instance, it may be possible to use certain instruments which are
13
. .
known to have adverse impact when used for rank-ordering (e.g.,
written test, physical ability test) on a pass/fail basis and then
band the scores on the Oral Board into 3 or 4 categories from Least
Qualified to Most Qualified, giving the department the flexibility
to interview from the top categories down. The Selection Interview
could then be used to make the final determination on hiring.
Insuring Quality Control:
Professionally accepted methods of item construction will be
followed in order to insure such desirable psychometric properties
as good level of difficulty, clear discrimination among test
takers, and freedom from ethnic and gender bias. The above
procedures will insure that all items are at an appropriate level
of difficulty for the position, that all items are linked to
spec if i c KSAs, and that redundancy and confoundi ng are avoi ded.
The Proposer will be primarily responsible for developing all items
and supervising typing in order to insure the quality of the
product in terms of clerical and grammatical errors.
Freedom from ethnic and gender bias will also be insured by
a conscientious review of the material. The Proposer follows the
legal and professional standards of the field, especially as
exemplified in the Uniform Guidelines on Employee Selection
Procedures and the American Psychological Association's Division
of Industrial/Organizational Psychology Principles on the
Validation and Use of Personnel Selection Procedures. This ensures
adherence to well established professional standards and case law.
Potential Problem Areas:
One of the potential problem areas identified in this project
is the amount of information that needs to be obtained from SMEs
(incumbents and their supervisors). The collaboration of the
respective departments will therefore be critical in expediting the
distribution and collection of survey instruments.
Coordination with Other Projects:
If the City joins with other local government agencies in a
separate contract for parts of th is project, the Proposer wi 11
coordinate with the consultant of the multi-agency project to
ensure a legally defensible, technically sound final product.
v. PROJECT SECURITY
The Proposer will be personally responsible for safeguarding
the security of the selection instruments and any related
materials. All related materials will be kept at the Proposer's
office and will not be shared with anyone except those individuals
designated by the City as having security clearance.
14
The selection instruments will be developed specifically for
this classification and will not be used for any other
classification or department either now or in the future. Since the
Proposer resides in the local area, it will be easy to hand carry
materials to and from the City. Bonded messenger service will also
be used as needed.
The Proposer will, in collaboration with the City, develop a
schedule for material retention, storage, and destruction.
Materials that are no longer needed will be shredded. The Proposer
is well aware of the importance of safeguarding the security and
credibility of the project.
VI. LITIGATION EXPERIENCE
The Proposer has served as an expert witness in the area of
test development and validation, including job analysis
methodology, in a number of arbitration hearings and court cases
(see Curriculum Vitae, Exhibit 1). Exhibit 3, includes
documentation of the Proposer's successful defense of test
development and validation practices in expert testimony provided
in Federal Court. As can be seen by excerpts from the Judge's
opinion, the Court was favorably impressed by the Proposer's
testimony and ruled in favor of his testing and validation
practices.
The Proposer recently assisted the City of Jacksonville in
preparing the defense on an age-discrimination case against the
pol ice department's promotional procedures which included a written
test and an assessment center. The ass i stance prov i ded i nc 1 uded
data analysis, literature review, and written expert testimony.
VII. INSURANCE
The Proposer carries professional liability insurance of
$1,000,000 per occurrence and $3,000,000 aggregate (see Exhibit 4,
for Liability Insurance Certificate).
VIII. AFFIRMATIVE ACTION
The Proposer will ensure that there will be no
discrimination as to race, sex, color, creed, or national origin
in regard to obligations, work, or services performed under the
terms of this contract.
IX. DIVERSIFICATION OF INCOME
It is projected that this contract will comprise approximately
50-60 percent of the Proposer's income.
15
x.
PROJECT SCHEDULE:
concurrently.
Activity
Both
classifications
to
be
done
Weeks into Proiect
A. Job Analysis
1 .
2 .
3.
4.
5 .
6 .
Prepare preliminary list
of tasks and KSAs based on
existing information.
Select SME panel and larger
sample of incumbents and
supervisors.
Finalize survey questlonnaires.
Complete survey of large sample
of incumbents.
Complete analyslS of data.
Complete prellminary report
for lnstrument development.
B. Wntten Test
2 .
3.
1 .
Complete final draft based on
job analysis results (100 ltems).
Respond to challenges, if applicable.
Final scoring and eligible register.
C. Ora 1 Board
1 .
Obtain critical incldents from
SMEs.
Finalize instrument format and
scOrlng procedure.
Train Raters and admlnister
instruments
Respond to challenges, if applicable.
Final scoring and eligible register.
2.
3 .
4.
5.
D. Physical Ability Test
1 .
2.
3.
4.
Obtain work samples from SMEs.
Finalize instrument format.
Conduct pilot study.
Finalize scoring procedure.
E. Selection Interview
1 .
Finalize instrument format and
scoring procedure.
Train raters and administer instruments.
Respond to challenges, if applicable.
Final scoring and register of
participants' overall results.
2.
3.
4.
16
02
03
04
08
10
12
14
16
17
10
15
18
20
21
12
16
20
22
20
24
26
28
F. Revise Job Specifications
1 .
Complete first draft of revised
job specification.
Obtain feedback from SMEs.
Complete final draft.
20
22
24
2.
3.
XI. COST BREAKDOWN
A. Job Analysis
B. Written Test
C. Oral Board*
D. physical Ability Test
E. Selection Interview*
1. Develop Instrument
2. Review Instrument
F. Job Specification
G. Pilot Program**
Cost:
police Fire
$ 10,500. $ 10,500.
$ 7,975. $ 7,975.
$ 11,200. $ 11,200.
$ 7,850. $ 10,850.
$ 6,250. $ 6,250.
$ 2,240. $ 2,240.
$ 1,475 $ 1 ,475.
""
Activity:
Total
$41,240.-$45,250. $44,240.-$48,250.
*Includes the time involved in facilitating and monitoring
administration of instruments to an initial group of
candidates (15 per procedure). Additional charges for
monitoring continued administration of instruments will be
made at the rate of $300 to $500 per day depending upon the
Proposer's level of involvement in the process.
**The optional Test Preparation/orientation program would cost
from $3,000 to $5,000 depending upon the extent of the program
and the involvement of City staff in the delivery of services.
17
RBQUBST FOR PROPOSALS NO. 33-93/94
CITY 0' MIAMI BEACH
To Be Opened on 2/3/94
at 2:00 P.M.
PRO P 0 S A L
For providing the services of job analysis, job specifications development,
selection device development, and validating each component and entire selection
process for Firefighter I and police Officer/Trainee classification for the City
of Miami Beach, per foregoing specifications:
Lump Sum Fee
$
gs, '1;'0 - '9'iS-oO
;
Any additional fees not included in lump sum fee (describe):
Exnert ''';i tness
12S.()O /~
.
$
$
$
Amend:i1ent No.3:
GE RECEIPT 01" ADDENDUM (11" NECESSARY) ,
BIDDERS ~ SIGN
Amendment No.1:
Amendment No.2:
ANY LETTERS. ATTACHMENTS, OR ADDITIONAL INFORMATION TO BE CONSIDERED PART OF THE
PROPOSAL MUST BE SUBMITTED WITH SIX (6) ADDITIONAL COPIES.
SUBMITTED BY:
TELEPHONE:
execute this proposal and
COMPANY:
SIGNED:
NAME/TITLE
ADDRESS:
CITY/STATB:
ZIP:
RFP NO:
DATE:
33-93/94
1/6/94
CITY OF MIAMI BEACH
42
t
'\
gBCT .1RATI:ON
TO:
Roger M. Carlton
City Manager
City of Miami Beach, Florida
.,~
Submitted this ~ day of
F&. e~v~
, 1994
The undersigned, as bidder/proposer (herein used in the masculine, singular,
irrespective of actual gender and number) declares that the only persons
interested in this proposal are named herein; that no other person has any
interest in this proposal or in the contract to which this proposal pertains;
that this proposal is made without connection or arrangement with any other
person and that this proposal is in every respect fair and made in good faith,
without collusion or fraud.
The proposer further declares that he/she has complied in every respect with all
of the Inscructions to Proposers issued prior to the opening of proposals, and
that he/she has satisfied himself fully relative to all matters and conditions
w~th respect to the general condition of the contract to which the proposal
pertains.
The proposer proposes and agrees, if this proposal is accepted, to execute an
appropr~ace City of Miami Beach document for the purpose of establishing a formal
contractual relationship becween him, and the City of Miami Beach, Florida, for
the performance of all requirements to which the proposals pertains.
The proposer states that the proposal is based
documencs idencified by the following number:
RFP NO. 33-93/94
proposal
?)Avd) :;A N,l Srr,I3AN,{J~ ,0-
PRINTED NAME OF SIGNATURE
TITLE (IF CORPORATION)
RFP NO:
DATE:
33-93/94
1/6/94
CITY OF MIAMI BEACH
43
..
SWORN STATEMENT PURSUANT TO SECTION 287.133(3)(a),
FLORIDA STATUTES, ON PUBLIC ENTITY CRIMES
THIS FORM MUST BE SIGNED AND SWORN TO IN THE PRESENCE OF A NOTARY PUBLIC OR OTHER OFFICIAL AUTHORIZED TO
ADMINISTER OATHS.
1.
This sworn statement is submitted to
by j)AV Ib '. SAI\JTI STL~~ ~
(print individual's name and title)
for <; t.Lt=:
(print name of entity submitting sworn statement)
whose bus i ness address is q r- fY\ r..R R- ( c...K..
Ca'L~ G r~ fJ-. j ~ J
WA y/ #r2.:~
3'3/'3y
and (if applicable) its Federal Employer Identification Number (FEIN) is
-
the individual signing this sworn
. )
2. I understand that a "public entity crime" as defined in Paragraph 287.133(1)(g), Florida Statutes.
means a violation of any state or federal law by a person with respect to and directly related to the
transaction of business with any public entity or with an agency or political subdivision of any other
state or of the United States, including, but not limited to, any bid or contract for goods or services
to be provided to any public entity or an agency or political subdivision of any other state or of the
United States and involving antitrust, fraud, theft, bribery, collusion, racketeering, conspiracy, or
material misrepresentation.
3. I understand that "convicted" or "conviction" as defined in Paragraph 287.133(1)(b), Florida Statutes,
means a finding of guilt or a conviction of a public entity crime, with or without an adjudication of
gui 1 t, in any federal or state trial court of record relating to charges brought by indictment or
information after July 1, 1989, as a result of a jury verdict, non-jury trial, or entry of a plea of
guilty or nolo contendere.
4. understand that an "affiliate" as defined in Paragraph 287.133(1)(a), Florida Statutes means:
1. A predecessor or successor of a person convicted of a public entity crime; or
2. An entity under the control of any natural person who is active in the management of the
entity and who has been convicted of a public entity crime. The term "affiliate" includes those
officers, directors, executives, partners, shareholders, employees, members, and agents who
are act i ve in the management of an aff i l i ate. The ownersh i p by one person of shares
constituting a controlling interest in another person, or a poOling of equipment or income
among persons when not for fair market value under an arm's length agreement, shall be a prima
facie case that one person controls another person. A person who knowingly enters into a joint
venture with a person who has been convicted of a public entity crime in Florida during the
preceding 36 months shall be considered an affiliate.
5. I understand that a "person" as defined in Paragraph 287.133(1)(e), Florida Statutes. means any natural
person or entity organized under the laws of any state or of the United States with the legal power to
enter into a binding contract and which bids or applies to bid on contracts for the provision of goods
or services let by a public entity, or which otherwise transacts or applies to transact business with
a publ ic entity. The term "person" includes those officers, directors, executives, partners,
shareholders, employees, members, and agents who are active in management of an entity,
RFP NO:
DATE:
33-93/94
1/6/94
CITY OF MIAMI BEACH
44
6.
Based on information and belief, the statement which I hIve marked is true
submitting this sworn statement. (indicate which statement applies.)
L
in relation to the entity
Neither the entity submitting this sworn statement, nor any of its officers, directors,
executives, partners, shareholders, ~loyees, menbers, or agents who are active in the
management of the entity, nor any affiliate of the entity have been charged with and convicted
of a public entity crime subsequent to July 1, 1989.
The entity submitting this sworn statement, or one or more of the officers, directors,
executives, partners, shareholders, ~loyees, rnerars, or agents who are active in the
management of the entity, or an affiliate of the entity has been charged with and convicted
of a public entity crime subsequent to July 1, 1989.
The entity submitting this sworn statement, or one or more of its officers, directors,
executives, partners, shareholders, ~loyees rnerars, or agents who are active in the
management of the entity, or an affiliate of the entity has been charged with and convicted
of a public entity crime subsequent to July 1, 1989. However, there has been a subsequent
proceeding before a Hearing Officer of the State of Florida, Division of Aaninistrative
Hearings and the Final Order entered by the Hearing Officer determined that it was not in the
public interest to place the entity submitting this sworn statement on the convicted vendor
list. (attach a copy of the final order)
I UNDERSTAND THAT THE SUBMISSION OF THIS FORM TO THE CONTRACTING OFFICER FOR THE PUBLIC ENTITY IDENTIFIED IN
PARAGRAPH 1 (ONE) ABOVE IS FOR THAT PUBLIC ENTITY ONLY AND, THAT THIS FORM IS VALID THROUGH DECEMBER 31 OF THE
CALENDAR YEAR IN ~HICH IT IS FILED. I ALSO UNDERSTAND THAT I AM REQUIRED TO FORM THE PUBLIC EN ITY PRIOR TO
ENTERING INTO A CONTRACT IN EXCESS OF THE THRESHOLD AMOUNT PROVIDED IN SEC ON 287. 17 FOR
CATEGORY TWO OF ANY CHANGE IN THE INFORMATION CONTAINED IN THIS F
STATE OF
/0)
c
J..7 U_/_
COUNTY OF
PERSONALLY APPEARED BEFORE ME, the undersigned authority,
!}./,D 5",JIi ~~r-)
in tne space provided above on this J -~
affixed his/her signature
My C~ission E)(cire~~ ,....., OF n.O\{\!)A.
....OT .\In Pl1\:':fL. ~. .. L
\1\ ('O"'lI~";I(:~ LXi';l{;-::' ....... 13 1-'"
KO''[>f.D 11110:1 :'<1l'T.\K\ rLIIUl: l:\Vli.\\lU~
i
/
FORM PUR 7068 (Rev. 06/05/91)
RFP NO:
DATE:
33-93/94
1/6/94
CITY OF MIAMI BEACH
45
Exhibit 1
Curriculum Vitae
\
DAVID SANTISTEBAN. PH, D.
PSYCHOLOGIST
Education:
Doctor of Philosophy (Clinical/Community .Psychology),
University of South Florida, Tampa, Florida, 1978.
Languages:
English and Spanish
License: I
Florida Psychology License No. PY3119
Academic Appointments:
Adjunct Ass~stant Professor, Department of psychiatry,
University of Miami School of Medicine, 1984-1985.
Assistant Professor, Department of psychiatry, University of
Miami School of Medicine, 1975-1984.
Invited Lecturer, Miami Institute of Psychology/Caribbean
Center for Advanced Graduate Studies, 1979-1981.
Consultantships:
Office for Substance Abuse Prevention, U.S. Department of
Health and Human Services, Washington, D.C., 1990.
City of Miami Beach, Personnel Department, 1990-Present.
Southern WindS Hospital, Hialeah, Florida, 1989-Present.
Metropolitan Dade County, Employee Relations Department,
Miami, Florida, 1982-Present.
City of Coral Gables, Employee Relations Department, Coral
Gables, Florida, 1986-Present.
American Psychological Association, Policy and Planning Task
Force, Washington, D.C., 1988.
National Institute of Mental Health, Washington, D.C.,
1978-1986.
City of Miami, Department of Human Resources, Miami, Florida,
1984-1986.
Institute of Comparative and Social Studies, Washington,
D.C., 1979-1983.
Southeast Florida Academy of Fire Science, Miami Dade
Community College, Miami, Florida, 1985.
National Institute on Drug Abuse, Washington, D.C.,
1979-1983.
Federal Correction Institute, Miami, Florida, 1979-1982.
Driving Under the Influence Countermeasures Program, Miami,
Florida, 1979-1982.
Administration on Aging, Department of Health, Education and
Welfare, Washington, D.C., 1979-1981.
Office of Refugee Resettlement, Washington, D.C., 1980.
National Hispanic Committee for the Implementation of Reports
and Recommendations to the Presidential Commission on
Mental Health, Washington, D.C., 1978.
THE ALHAMBRA \NEST. 95 MERRICK WAY. SUITE 523 · CORAL GABLES, FLORIDA 33134 · (305) 445-2471
\
Professional Vitae
David Santisteban, Ph. D.
page 2
Forensic Experience:
United States District Court, Southern District of Florida
Circuit Court of the Eleventh Judicial Circuit in and for
Dade County, Florida
American Arbitration Association, Miami Regional Office,
Dade County, Florida
State of Florida, Division of Administrative Hearings
Superior Court of New Jersey, Essex County, New Jersey
Forensic Evaluator Training, Department of Health and
Rehabilitative Services.
Professional Associations:
Past Vice President of the National Hispanic Psychological
Association
Editorial Board, Hispanic Journal of Behavioral Sciences,
University of California, Los Angeles
American Psychological Association:
Division of Psychology and the Law
Division of Industrial/Organizational Psychology
Division of Ethnic Minority Affairs
Division of Consumer Psychology
Florida Psychological Association
Interamerican Society of Psychology
Selected Publications: *
Santisteban, D., Santisteban, D.A. Psychological testing of
minorities: Its possible uses and misuses in the legal
system. Florida General Practice Journal, Vol. XIII,
No.3, 1988.
Santisteban, D., Santisteban, D.A. Informed consent and
limits of confidentiality, in Trimble, J. and Bolek, C.
(Eds.), Conducting Cross-cultural Substance Abuse
Research: Emerging Strategies and Methods, in press.
Santisteban, D., Harmon, L., Dorsett, H., Harris, M. Job
analysis of a firefighter position: A multi-dimensional
model, in preparation.
Santisteban, D., Harmon, L., Dorsett, H. The Situational
Interview as a selection procedure, in preparation.
Santisteban, David. Cultural issues in the translation and
adaptation of evaluation measures. In Blackwell, B.L.,
and Cartwright, L.K., (Eds.), Program Consultation with
Human Service Programs: A Clinical Perspective. Oakland,
California: Third Party Publishing Company, 1988.
Szapocznik, J., Santisteban, David, Rio, A., Perez-Vidal,
A., Santisteban, Daniel, & Kurtinez, W.M. Family
effectiveness training: An intervention to prevent
problem behaviors in Hispanic adolescents. Hispanic
Journal of Behavioral Sciences, February, 1'(1), 4-27
'989.
,
. -
Professional Vitae
David Santisteban, Ph. O.
page 3
Szapocznik, J., Santisteban, Daniel, Rio, A., Santisteban,
David, & Kurtinez. W.M. Repeated testing and its impact
on clinical outcome: Serious confound or negligible
effect? Under editorial review, Journal of Consulting
and Clinical Psychology.
Szapocznik, J., Santisteban, 0., Rio, A., Perez-Vidal, A.,
Kurtinez, W., & Hervis, O. Bicultural effectiveness
training, Hispanic Journal of Behavioral Sciences,
December 1986, (4), 303-330.
Szapocznik, J., Santisteban, 0., Rio, A., Perez-Vidal, A.,
and Kurtinez, W. Family effectiveness training for
Hispanic families: strategic structural systems
intervention for the prevention of drug abuse. In
Lefley, H.P., and Pedersen, P.B. (Eds.), Cross Cultural
Training for Mental Health Professionals. Springfield,
Illinois: Charles C. Thomas, 1985.
Szapocznik, J., Santisteban, D., Kurtinez, W., Perez-Vidal,
A., Hervis, O. Bicultural effectiveness training: A
treatment intervention for enhancing intercultural
adjustment in Cuban-American families. Hispanic Journal
of Behavioral Sciences, Vol. 6, 1985, No.4, 317-344.
Rodriguez, A., Skotko, V., Santisteban, D., The experience
of coping in the camps. In Szapocznik, J., Cohen, R.C.,
and Hernandez, R. (Eds.), Coping with Adolescent
Refugees. New York: praeger Publishers, 1985.
szapocznik J., Wray, S., Cohen, R.C., Sussex, J., and
Santisteban, D. Caribbean mental health priorities:
Recommendations of the caribbean conference on
health/mental health models. Bulletin of the Pan
American Health Organization, 1984, 18, (4), 408-411.
Bernal, G., Bernal, M.E., Martinez, A.C., Olmedo, E.L.,
Santisteban, D. Hispanic mental health curriculum for
psychology. In Chunn, J., Dunston, P., Ross-Sheriff, F.
(Eds.), Mental Health and People of Color: Curriculum
Development and Change. Washington, D.C.: Howard
University Press, 1983.
Santisteban, D., and Szapocznik, J., Substance abuse
disorders among Hispanics: A focus on prevention. In
Becerra, R.M., Karno, M., Escobar, J.I. (Eds.), Mental
Health and Hispanic Americans, Grune and stratton,1982.
szapocznik, J., Santisteban, C., Kurtinez, W., King, 0., and
Spencer, F. Life enhancement counseling: A
psychological model of services for cuban elders. In
Jones, E.E., Korchin, S.J. (Eds.), Minority, Mental
Health, N.Y.: Holt, Rinehart, & Winston, 1982.
Szapocznik, J., Santi steban , D., Kurtinez, W. Research on
innovative treatment modalities for Hispanics. Research
Bulletin, Fordham University, New York, 1979.
* Additional publications upon request.
Exhibit 2
Work Sample
."
*" COAl F t"b&AJT/t(J-
KL ~Ro DueL D
JOB ANALYSIS REPORT FOR
THE POSITION OF POLICE CAPTAIN
-
February. 1992
Prep&red by: David Santi.teban Ph D
Con.ulting p.ychoioli~t'
1
IIV F () R. IV\. -A T1 f) j....J - - Nt' r 7'P p Eo.
()J 177'+ 0 v r Co M s~;Vr .f) F 11f~
-
,A u 7ilCJR .
'.
.
TABLE OF CONTENTS
b.u
AC~NOWLEDGEMENTS.................................. i
I. I.TRODUCTIO....................................... 1
Ii. . Pu r po. e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
B. Ba ckaroulld. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
II. METHODOLOGY AND RESULTS........................... 2
A. Panel Of Subject Matter Ezpert................. 2
B. Development .nd Diatribution of ~ob Analyai.
Qu e 8 t ion n air e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
C. Weilhting of Duty Are.......................... 19
D. Relative Importance of Manalerial Abilitie. and
Performance Di.en.ion.......................... 20
E. Relative Importance of Technical ~novledle. .nd
Job Relevant Source Material................... 26
F. Relative Iaportance of Ability v. Technical
En ow 1 e die. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
31
37
G. Trainability of ~novledle. .nd Abilitie........
III.
D I S CUS S I OK. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
T.ble of Con~en~. Con~inued:
EAu
IV. APPBNDICES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendiz A - Job An.ly.i. Open-ended Que.~ionn.ire
Pora. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Appendiz B - Job An.lyai. Que.~ionn.ire.......... 49
Appendiz C
I.por~.nce a.~in&. of Abili~ie. by
Du~y Ar............................. 86
Appendiz D- C.~eaoriz.~ion of Abili~i.. in~o
Perfor..nce Di.en.ion............... 90
Appendiz I - Import.nce a.~ina. of Enovl.dae. by
Duty Are............................ 93
LIST OF TAILBS...................................
T.ble 1 - Demoar.phic Infor..~ion................ 4
Table 2 - Duty Are............................... 6
T.ble 3 - ael.~ive Sianific.nce of Du~y Are...... 21
T.ble 4
ael.~ive I.por~.nce of Abili~ie........
22
T.ble S - A..e.amen~ Cen~er Perform.nce Di.en.ion
Definition............................. 24
T.ble 6 - Performance Dimenaion Weiah~........... 28
T.ble 7 - Rel.~ive I.por~.nce Weilh~. of
Technical Enovledae.................... 29
Table 8 - Rela~ive I.por~ance of a..dina
Source................................. 32
T.ble 9 - Tr.in.bili~y a.~ina. of Abili~ie. .nd
Enovledle.............................. 33
.
. ,
I. INTRODUCTION
A. PurpoBe
The purpose of this report is to describe activities
related to the development of an updated jOb analysis for
the position of Police Captain in the
1 The primary focus of these activiti.s was to
review ta.ks considered important based on findinl' from the
earlier job analy.is conducted in 1984 and 1988 and to
determine to what extent these tasks are important for
success in the Job today.
In addition, several analy..s were conducted that 10
beyond the identification of important ta.ks. The following
pages will describe reaearch in these areaa:
1.
Clu.tering
areas of
repreaent
throughout
of important taska into "Duty
primary responsibility. The
major functions performed
the Department.
Areas" or
Duty Areas
by Captains
2, Ratings of the importance of a variety of abilities
as they may be required to perform these Duty Areas.
3. Ratings of the importance of a variety of technical
knowledges aa they may be required to perform the
Duty Areas.
4. Ratings of the extent to which important knowledges
and abilities may be enhanced on the job or throulh
formalized training.
5. Demographic information on the sample of incumbents
and immediate superiors who completed the job
analysis questionnaires,
The primary purpose of the.e analy.e. was to identify
important technical knowledaes and performance dimensions
(clusters of abilities) that relate to effective performance
as a Police Captain, a. well as key .ituations faced by
incumbents on the Job. Thi. information will .erve a. the
basis for the development of a written technical knowledge
exam and a.se.sment center .timulation..
1
..
B. B.ac:k&round
The earlier job analyses for the tarlet position were
conducted in 1984 and 1988, At that time the job analysis
interview questionnaires and other job relevant materials
were reviewed with the loal of identifyinl the .pecific
ta.ks, knowledles, and abilities required for successful
performance as a Captain within the
Department. The ability statements were then clustered baaed
on similarity of activity, and catelorized into performance
dimen.iona. From these job analy.is data, simulation
exercise. and technical knowledae te.ts were developed to
accurately reflect the duties, tasks, and responsibilities
encountered by per.onnel in the taraet position.
The current job analy.is was conducted to determine what,
if any, significant chanaes have occurred in the Job as it
related to the tasks and key performance dimensions
currently performed by Captain. within the
Department.
II. METHODOLOGY AND RESULTS
The jOb analysis methodoloay used in the project relied
heavily on the use of the questionnaire method and input
from subject matter experts throuahout. The methodoloay
conforms to prOfessional and leaal standards as outlined in
the American Plycholoaical Association's (APA) Standards on
Education and Psycholosical Tests, the APA'. Divi.ion of
Industrial/Oraanizational PsychololY Principles on the
Validation and Use of Personnel Selection Procedures (1980),
and the Uniform Guidelines on Employee Selection (1978).
The various project phases or steps are described in this
section.
The most typical Captain'. position is at the district
level. Each district has an Executive Captain that reports
directly to the Di.trict Commander, a Police Major. The
Captain, in turn, typically has... Police Lieutenants under
his/her direct supervision, with a total of approximately
L subordinate. under his/her command. At the time
this study was conducted, there were _ individuals in the
Department holdina the rank of Police Captain.
A. P.anel of Subiec:t M.atter Experts (SME'.'
A panel of SME's was .elected to provide detailed
information on the position throulh the use of an open-ended
Position Description Questionnaire (see Appendix A).
2
~
The instrument was uaed to obtain information on key
functions, tasks, abilities, knowledges and aituations
relevant for effective performance on the job. Individual
responses were aummarized in order to identify areas moat
common to the position. In addition, panel members reviewed
tasks, knowledies, and abilitiea inventoriea aa well aa
descriptiona of duty .reaa, .a determined from the previous
job analyses in order to determine their relevance for the
poaition today.
Panel members were aelected baaed on:
familiarity with the poaition (1. e., incumbents and
their immediate aupervisora).
representation of
where Captains are
the varioua leolraphic locations
placed in the organization,
ethnic and lender representation.
The composition of the panel in terms of rank, lender,
ethnicity, and tenure in their respective position was as
follows: four Captains and five Majors; four non-Hispanic
white males, two non-Hispanic white females, one Black male
and two Hispanic males. The averale tenure as a Captain waa
3 years 6 months; the averale tenure with the Department was
22 year..
B. Development and Distribution of job Analysis
Questionnaire
A final questionnaire
from the SHE Panel. The
parts, described below
questionnaire).
.was developed based on
questionnaire consisted
(see Appendix B for
the input
of several
copy of
The questionnaires were distributed to 28 individuals (16
incumbents and 12 immediate aupervisors, Police Majors). Of
these, 23 completed questionnaires were returned. The
demographic breakdown of the respondents is shown in Table 1
below (information was not available for one respondent).
3
. .
.
TABLE 1
DEMOGRAPHIC INFORMATION
Tot.l
population
Number in
Samplll
Percent of
Total
police M.jora
police C.pt.ina
Over.ll
12
16
28
10
15
25
83%
94%
89~
Aver.ae Tenure .a . C.pt.in: 8 yra. 4 moa.
Aver.ge Tenure with the Dep.rtment: 23 yra. 4 moa.
Part 1 _ Ratin&s of Importance and Frequency of Tasks
Information obtained from the open ended queationn.irea .nd
input from the SME P.nel w.a uaed to gener.te . liat of 62
tasks that could be related to aucceaaful perform.nce on the
job. Respondenta were .aked to rate each talk on a 1-7
Likert-type acale according to it. importance and frequency
of occurrence (aee Appendix B).
Questionnaire responses were analyzed to determine tho.e
t.ak. considered important for aucceaaful performance in the
position. Thi. waa baaed on the ratinga of taak importance
provided by the reapondent.. Ta.ka with a rating of leaa
than 4.0 (Important for Effective Performance) were to be
dropped from further analy.ea. All 62 taak., however
achieved ratings of 4.0 or better.
Part: 2. CateiOri~ation of Tasks into Duty Arll.s
previoua job analyae. h.d identified 10 cateloriea of
.ctivities that would aeem to deacribe the majority of taaka
considered import.nt by the respondenta. The.e cateaoriea,
or "Duty Are.... were gener.ted baaed on the content of the
t.ska, and are liated in Appendix B. Theae Duty Area. were
reviewed and updated by the current SME Panel and and
eleventh .re. w.. .dded to t.ke into account aeveral taaka
which h.d not been c.tegorized before under .ny of the
exiltins areal.
4
..
In order to support the intearity and relevance of the
Duty Areas, respondents were aaked to cateaorize each of the
62 important tasks accordina to the 11 Duty Areas.
In order for a task atatement to be considered related to
a duty area more than on-half (51%) agreement among
re.pondenta was set as the minimum criterion. Of the 62
taak., 54 were cateaorized in thil way, indicatina the Duty
Areas are meaninaful deacriptions of the taska oriainally
rated as important by incumbents. In addition, only five of
the 54 talka were placed in more than one catelory,
indicating the varioua Duty Areas repreaent distinct facets
of the Captain'S jOb.
Table 2 lista the Duty Areaa and taaks that were related
to each area. The mean importance and frequency ratings of
each task are shown, as well as the average mean importance
and frequency ratings for all tasks belonging to each Duty
Area. The lame data is provided for the eight
"miscellaneoua" tasks that failed to meet the 51~ rule for
inclusion in a Duty Area. Mi.cellaneous tasks were not
necessarily rated any lea a important than talks related to
the Duty Areaa. They were taaks that did not meet the
minimum criterion for inclusion in a Duty Area.
5
T ASK Ii
30
31
35
42
44
TABLE 2
..
AREAS OF PRIMARY RESPONSIBILITY
(DUTY AREAS) FOR POLICE CAPTAIN
I. Community Relations
To attend Citizen Advisory Committee meetinlsj to represent
the Department to various community groupSj to addresl civic
orlanizati~~s on matters of mutual concern.
TASK STATEMENT
Interacts with members of the
community for the purpose of
establishing and maintaining
positive relationl (e.g., Citizen
Advisory Committees).
Addresses civic organizational
groups.
Personally meets with concerned
concerned citizen groups.
Personally responds to public
criticism concerning
departmental activities.
Responds to community requests
to initiate program (e.a., crime-
watch) .
Overall mean
6
IMPORTANCE
RATING
FREQUENCY
RATING
5.79
5.54
5.00
4.25
5.54
4.46
5.42
4.21
5.25
3.71
5.40
4.43
"
Parts 3 and 4
Abilities
Importance Ratinas of Knowledaes and
Respondents were asked to rate the importance of 39
abilitiea and 40 knowledaes that milht be required in order
to successfully perform each of the Duty Areas previously
identified. The list. of abilitiel and knowledge. were
drawn from previoul job analyses and were reviewed and
updated by the current SME Panel. Each ability and
knowledge was rated on a 1-5 Likert-type Icale.
Part 5 - Trainability Ratin&8 of Knowled&ea and Abilities
Respondents rated each knowledge and ability, alain on a
1-5 acale, according to how easily it could be enhanced on
the job or through a formal training program. Written test
and assessment center development will focus on mea.urina
knowledges and abilities that would not be easily acquired
in a short period of time in the target job. Also, this
information may be used in the preparation of career
development training programs.
C. Weiihtin& of the Dut'V Ares.
The varioua Duty Areas
comprised of tasks that vary
and frequency of occurrence.
established to determine the
Areas and miscellaneous tasks
pertormed by Captains are
according to their importance
The following procedure was
relative "weight" of the Duty
associated with the job.
1) The mean importance and frequency of occurrence ratings
of all tasks associated with a Duty Area were
determined. (Miscellaneous tasks were treated as a
separate Duty Area). This procedure yielded a lingle
importance rating and frequency rating for each Duty Area.
2) The cros.-product of the importance and frequency ratings
tor each Duty Area was obtained.
3) The cross-product. were lummed.
4) The cro.s-productl for each Duty Area was divided by the
total to obtain the percent weight for each area.
Table 3 shows the relative weishts of each Duty Area and
miscellaneous talks.
19
D. Relative Importance
Performance Dimensions
of
Manaaerial
Abilities
and
As mentioned earlier, questionnaire re.pondents were
asked to indicate the importance of each of 39 abilities a.
they may be required to perform the Duty Areas identified
previously. Since an ability may be extremely important to
perform one activity, but irrelevant tor performing another,
abilities were linked independently to each Duty Area. The
mean importance ratings tor these abilitiel al they related
to each Duty Area are Ihown in the matrix in Appendix C.
1. :r~~~dure for determinin~ the relativlI
_b___tv toward on the 10b ~erformance
determine the importance of an ability
Areas, the following procedure was used:
weiaht
In
of lIach
order to
all Duty
across
a)
Multiplied the importance ratinl of
~imeB the percent weight of the
being rated under. (If an ability
of less than "3" for any Duty Area
less than important, and wa. not
analysis).
an ability
Duty Area it was
received a ratin&
it was considered
included in the
b) Summed the cross-products to obtain a .um across
Duty Areas for each ability.
c) Summed the cross-product. for all abilities.
d) Divided the sum of crosa-product. for each ability
by the total sum of cros.-products for each ability
by the total sum of cross-product I tor all
abilities. This procedure yielded a ~ weight for
each ability.
The individual ability weights are contained in Table 4.
2. Procedure for clusterina abili~ieB into PerformanclI
Dimensions
Previous job analyses had clustered each of the abilities
rationally, ba.ed on similarity of content, into broader
"performance dimen.ions" that could serve al convenient
labels for measuring similar abilities. The original job
analysia included nine such abilities and a tenth one
(Control and FollOW up) was added in the most recent task
analysis. However, reports obtained form the utilization
of these dimensionl have indicated the desirability of
reducing the number of dimensions to the original nine in
order to maintain their discretionary power and to
increase the efficiency of the behavioral measures. The
nine performance dimenaions, defined in terms of the
abilities that comprise them are listed in Table 5. A
mAtrix ahowing each ability and the dimension(s) it is
related to is contained in Appendix D.
20
"
3. Procedure for weishtins Performance Dimensions
Havina determined the percent weight of each ability, and
after establishing clusters of abilities labeled as
performance dimensions, it was now posaible to derive
relative weights for each dimenaion. The procedure u.ed
wa. as followa:
a) Summed the ability weightl for each dimenaion, and
divided by the number of abilities that define each
dimenaion to obtain the raw weilht for the dimension.
b) Summed the raw weilhtl for the nine dimensions.
c) Divided the raw weight for each dimension by the lum
of raw weight. in order to determine the relative
weight of each dimension.
The dimension weights are contained in Table 6. Weight.
for the dimensions derived from previous Captains job
analyses are shown for comparison purposes and indicate
the consi.tency of the relative weights of the dimensiona.
E.
Relative
Relevant
Importance of Technical
Source Materials
Knowled&es
and
.Job
A rating procedure similar to that used for abilitiel
was used to determine the importance of a variety of
knowledge areas that might be important in order to
perform each of the Duty Area.. The mean importance
ratings for each knowledge as it is related to the Duty
Areas ia shown in Appendix E.
The relative weights for each knowledge
determined in the .ame way as for abilities.
importance of each knowledge area for
performance on the job i. shown in Table 7.
area were
The percent
successful
Linking of Knowledae Areas to Source Material A
panel of subject matter experts was used to link each of
the knowledae areal to reading materiall that would be
used as the basi. for development of the mUltiple-Choice
technical knowledae exam. In addition, the linkages were
reviewed by the Job analyst and 11 telt development
specialist both of whom were familiar with the content
area of both the technical knowledge. and the reading
lource.. The relative percent importance ot the ditterent
reading aources are shown in Table 8.
26
.,
F. Relative Importance of Ability V8. Technical Knowllld&e
Having performed the computation. neceslary to
determine the relative importance of each ability and
knowledge, a limilar procedure wal used to determine the
relative importance of managerial ability v.. technical
knowledge. The following procedure was Uled:
1) Summed the cross-product. for all abilitie..
2) Summed the cross-products for all technical
knowledge..
3) Added the two .ums to obtain a grand total.
4) Divided the sum for abilities by the total, to
obtain the relative percent weight of managerial
abilitiea. The remainder was the percent weight
for technical knowledge. These figures are Ihown
in Table 6.
27
",
"
G. Train~bility of Knowled&es and Abilities
The Uniform Guidelines on Employee Selection Procedures
(1978), states that applicants for a position should not be
eliminated based on qualifications that could be acquired
during a brief orientation period in the target jOb. In
order to minimize the poslibility of evaluating .pplicants
based on easily learned abilities or knowledges, respondents
were asked to indicate the extent to which each of the
abilities and knowledges could be enhanced on the Job or
through a formal traininl prolram.
Respondent. rated each knowledse and ability on a 1-5
point Likert-type scale. Hilher ratings indicated
respondents felt the abilitylknowledge could be easily
learned in a ahort period of time. Lower ratings indicated
the ability/knowledge was relatively difficult to acquire.
The Guidelinel do not define Mbrief orientation period".
After discussion with testing profeasionals and Departmental
personnel, however, it leemed that a rea.onable period of
time would be no longer than three months. As a result, the
acale point equivalent to a rating of "4" (see questionnaire
in Appendix B) was defined aa the point above which an item
would be considered readily learned in a short periOd of
time. The definition that was used incorporates the
following key elements:
Can be learned within a period of 3 months.
- Can be learned and demonstrated effectively.
_ Can be learned through training and lor on the job
performance.
- Can be learned re&ardless of prior levell of ability
or knowledge in the area. "
It is important to note, however, that even if an
ability/knowledge may be learned on the job, it is Itill
important for an individual to demonstrate this knowledge or
ability immediately upon assuming the duties of the
position. Therefore, knowledges or abilities that obtained
rAtings of "4" or above would be de-emphasized in the
development of testing materiall, but not neceslarily ruled
out completely.
Table 9 show. the mean trainability ratings of each
ability and knowledge. Alone would expect, knowledges
received higher ratings than abilitiel, indicatins that
knowledges can be more readily acquired on the job or
through training than managerial ability. These findings
were consistent with the results of the previous job
analysis.
3 1
III.
Discussion
The results of this update of the task analysis for the
Captain's clas.ification indicatel a areat deal of
consistency with the analyse. conducted previously.
The vast majority of the ta.ks, knowledges, and abilities
considered important in 1984 and 1988, are atill viewed aa
important today. Furthermore, the relative importance of the
performance dimensionl which capture the primary areas of the
poaition are very con,i,tent with previoul reaulta. The
relative importance of man_aerial abilitiel in compariaon to
technical knowledges also remained conltant.
The reliability of the results obtained confirm the
appropriateneas of the methodololY beinl used to as.es. the
knowledaea, .kills, and abilities required of the Captain's
classification. The multi-level approach to data latherina
with extensive input from Subject Matter Expertl has proven
successful not only in helping to define the requirements of
the position, but allo in serving aa a auide to the
development of promotional measures which are job relevant
and can help to target the most important characteristics for
successful on the job performance.
Now that the job analysis methodology has been
successfully established in terms of providing consi.tent and
reliable results, future task analysel can begin to explore
some modifications that can alli.t in developina information
not only for promotions, but alao for trainina and
prOfessional development. By the .ame token, we can explore
making some minor modifications in the methodology that would
allow us to analyze the knowledges, abilities, and
performance dimension. from different vantage points in an
effort to further refine our testing instruments.
37
"
Exhibit 3
Litigation Experience
A. Court Findings
B. Commendations
r "
MEMORANDUM
103.01-14
FROM
Lee Kraftchick
Assistant County Attorney
DATE
March 30, 1993
TO
Grace Poley
Personnel Director
peightal v. Fire Dept.
Affirmative Action suit
I wanted to thank you for making the services of
Dr. Santisteban available in connection with the County's defense
of the Fire Department's Affirmative Action Plan in Peiqhtal v.
Metropolitan Dade County. The court recently issued a decision
upholding the affirmative action plan in which it placed
significant reliance upon Dr. santisteban's statistical analysis.
David's assistance in preparing this case and his testimony
at trial were, as usual, of the highest quality. It is
reassuring to know that a professional with David's experience
and background is available when the County needs to defend its
recruitment practices.
Thank you again for allowing David to participate in the
defense of this case.
~'~
' ,
/.' .
v," L
~
cc:
Don Allen
David Santisteban
,'~'
L . t t t
ASS1S an Coufi y
I
1 fshht]tz1 '
~
Attorney
LK: mm
~v
Diredor' $ Office
APR 1 3 1993
PERSONNEL DEPARTMENT
~1
r l,
MEMORANDUM
10.... 01.14
TO
Grace Poley
Personnel Director
)
fl/
2, 1990
Inc.
FROM
Lee Kraftchick
Assistant County Attorney
Enclosed is a copy of a recent decision we received in
federal court concerning the Police Department.s assessment
center for lieutenants. The opinion provides a good description
of the current state of the law applicable to employment testing
and you may find it helpful in analyzing the County. s current
selection procedures and in developing new procedures for the
future.
You will note that the opinion characterizes the testimony
of Dr. David Santisteban as particularly persuasive.
Dr. Santisteban was instrumental in preparing this case for
trial and provided exceptionally clear expert testimony. Be
deserves to be highly commended for his efforts. Wi thout his
help, I doubt the County would have been so successful in this
lawsuit.
Another member of your staff,
to be commended. . . was most helpful
and in suggesting different approaches
of evidence. Ber cooperation with our
for information was deeply appreciated.
also deserves
in preparing documents
to the presentation
last minute requests
If you have any questions about the Court · s decision or
its implications about the County.s current selection procedures,
please feel free to contact me at your convenience.
Assistant Co
LX :mm
Enclosure
cc: David Santisteban
Director's Office
JAN 051990
PERSONNEL DEPARTMENT
r
MEMORANDUM
:OM:
Graclel la poley, Director
~Aso::artment
uardo Gonzalez
Deputy Director
Metro-Dade Pol Ice Department
DATE: January 8, 1990
SUBJECT: commendation -
Or. David Santlsteban
I have Just
Judge Lenore
Lawsuit" and
mance of Or.
completed a review
Nesbitt In what we
wanted to commend
Santlsteban.
decision rendered by
commonly refer to as -the POC
to you the exceptional perfor-
There Is no doubt, when one reviews the document, that his
expertise and courtroom testimony contributed significantly to
the County's abl I Ity to preval I In this case.
I have had many occasions to deal with David on a variety of
testing Issues and have always found him to be dl I Igent In his
support for systems that can withstand court challenges. ThIs
dl I Igence paid off and he Is to be commended for his profes-
sional Ism and dedication.
EG/mcr
... t · Ofl-
. 4r~"'.:)f S Ice
JAN 16 1990
PE~SONNEL DEP,\RTMENT
.,." ( ..
,'" f.... \v..~
'S~\;':~~") ..
\j':,.j
t.').~ '2.' "ro'"'
~
"
....--
".~ -.. i _
~-'::-,. ~~:..~~:
. . ~r:~:~~t.:;~~ ...
'. '.~::rz
Blacks from being promoted. '!'he County reconstructed each of the
eligible lists for the years 1983 through 1986 to dete~ne what
would have happened if assessment center .cor.. had not been u.ed.
The .Lmulated eliqible li.t. con.i.t of all of the component. of
the promotional proces. except a.."!J...ment center .cor.., J....a.tL.., they
includ. written te.t .cor.., .eniority point., veteran'. point.,
and point. earned tor ..rvice .. a ...ter .ergeant. The County'.
an.ly.i. inrltc.~e. t.h.t l' ......m.nt~ n.nt"..r .nore. were entirely
eliminated fram the ..Jection prnne.., the number of ftlaek
promotion. from 1983 to 1908 would have been ...ctly the .... ..
with the assessment center. Three Black candidates were pro.ot~
using the ass.ssment center and three Black candidates would have
been promoted had the assessment center not been usech
The Plaintiffs did not refute the conclusions of the County's
analysis of the effect of the assessment center, but did pre.ent
evidence throuqh an expert, Dr. pra.er, that there va. a
statistically significant difference between the mean rank scores
of Black ana other candidates at the assessment center. Using a
-t-test- procedure, Dr. Praser concluded that there was a
IItatlstically .ignificant difference between assessment center rank
score means of Blacks and the rank score meana of Whites and
Inspanics cambined. Dr. Fraser vas unable to state whether this
statistical difference in mean rank scores actually had an effect
on the number of promotions awarded to Black candidates.
'!'he Defendants' expert, Dr. Santisteban, testified that th~
Plaintiffs' analysis violated. several basic assumptions underlying ~
22
j.
~. :,...;~:.:~~
..;., .~...~:.'":
the t-test.
~e t-test assumes that the samples under
consideration are nor.mally distributed, that each of the
observations is independent of the other observations, and that the
samples involve interval, as opposed to ordinal (ranked l numbers.
'1'he most fundamental pro~lem with the Plaintiffs' statistical
analysis, according to Dr. Santisteban, is its erroneous assumption
that each of the observations in the samples is independent of each
of the other observations. Specifically, the Plaintiffs' analysis
erroneously assumes that the 34 measures included. in the Black
sample and the 204 measures included in the White sample were
measures of distinct individuals 0
In fact, since 1983 only 18
individual alack candidates and 123 individual White candidates
have gone through the assessment centerJ the 18 Blacks went through
the center on 34 occasions, while the 123 Whites went through the
center on 204 occasions 00
In effect, the figures used in the
Plaintiffs' analysis inflated the sample sizes by counting repeat
candidates as separate candidates. According to Dr. Santisteban,
the t-test assumes indepenaent observations, and' observations are
not ~dependent when they involve repeated. measures or scores from
.
the same individual- Dr. Santisteban, in the Court's view,
successfully rebutted Dr. Praser's testJJDony-
Further, Dr. S~tisteban stated that violation of the
assumption of independence of observations can be avoided by
redesigning the statistical analysis to account for the effect of
repeat performance. Dr. Santisteban conducted such an analysis by
counting only the first time performance of each candidate. By
23
I.
....1J}.
using first time performance only, Dr. Santisteban explained that
he was able to avoid the practice effect associated with IlUltiple
test taking and to comply with the requirements of independence.
Dr. Santisteban' s analysis concluded that there was no
statistically significant. difference among the means of Black,
White and Hispanic candidates at the assessment center for either
mean scores or mean rank sc'ores. 'the Court finds Dr. Santisteban' s
analysis persuasive.
Dr. Santiateban also conducted a statistical analysis to
determine if the promotion rates for Black assessment center
candidates was significantly different from the rate for other
ethnic groups. Using a · chi-square · analysis, Dr. Santisteban
concluded that the difference between the Black promotion rates and
the rates for. Hispanics and Whites was not statistically
significant at the .05 level. S
S Dr. Fraser complained that the chi-square should been
performed using two groups (Blacks and Whites) instead of three
(Blacks, Whites and EU.spanics), but the results are the same under
either analys is. Dr. Santisteban explained that whether the
comparison is of two groups or three, the same result is obtained I
there is no statistically significant difference between the
promotion rate of Blacks and the promotion rate of other
candidates. In rebuttal, Dr. Fraser suggested. that the County
should have used a .proportional analysis. instead of a chi-square,
but the statistical literature describes the twO tests as
essentially identical vhen two groups are being campared. Using
the proportional analysis, Dr. Fraser concluded. that the rate of
Black promotio'ns vas significantly different from that of Whites,
but his analysis failed to include all of the promotions that have
actually been awarded. EU.s analysis included promotions only up
to August 1988, vhen the more recent promotions of October 1988 are
included, the difference in promotion rates is no longer
statistically significant using either the chi-square or the
proportional .analysis. The fact that the analysis can be affected
by the promotion of just one Black candidate between August and
October 1988 demonstrates that, whatever statistical difference
24
/.
.. .~~~4--'
.'-~- . . ~'.'
. "~r:.~f74P::"
."'. "#6lf!:.' ~
:-. ~ .
:....,..~,~
. ..:t1.~{-~f~
_ [c 10urts are. generally less competent than employers to
restructure business practices, the judiciary should proceed vith
care before mandating that an employer must adopt a plaintiff'S
alternate selection or hiring practice.- wards Cove, 109 S. Ct.
at 2127.
1. Proof of Adverse Imoact
The Plaintiffs have identified the specific employment
practice that is allegedly responsible for statistical disparities
in this case as the assessment center port~on'of the lieutenants'
promotional procedure.
~e plaintitfs'" allegation that the
assessment center has had a disparate impact is predicated upon Dr.
Fraser's statistical analyse.s comparing assessment center mean rank
scores of Black candidates to the mean rank scores of Whites and
Hispanic candidates combined. Usinq a t-test procedure, Dr. Fraser
concluded that there was a statistically significant difference
between ~ssessment center mean rank scores of the two groups.
The Plaintiffs' analysis suffers from many of the statistical
'.
flaws id~ntified by the Supreme Court in ~atson.
All the
Defendants' expert, Dr. Santisteban,' explained, the plaintiffs'
analysis used misleadinq sample sizes by assuming that each of the
scores in the sample vas independent of each of the other scores.
In effect, the. figures used in Dr. Fraser's analysis inflated the
sample sizes by countinq repeaters as separate candidates. -rhe
analysis violated the assumption of independence of observations.
required to properly conduct the statistical tests. See ,owers v.
Alabama Deo't of Education, 854 F.2d 1285, 1296 n.18 (11th Cir.
29
( .
. ". ..
=-~~::. ~)~~.-:.
.',~.~,." - .. ~:..~~
. .- -:-;.., ,. "'1:.7.-.
. ';<:"."t~lf':.: .
(a statistical analysis can be -skewed [if it is] based on
on the scores given particular
persons-)I Smith v. Conrail, 49 F.E.P. Cases 266 (Mich. 1988) (-The
statistics must reflect the number of individuals who took the
challenged test rather than the number of attempts that were aade
by those individuals.-)
':he County conducted. analyses designed to avoid the effeet of
counting the same person two or more times by using only the first
time scores of each candidate. First time scores represent the
purest lIleas~e of a candidate's perfoJ:ma!1ce because ~ey..ar.e. no~
contaminated by the possible effeet of practice that arises fram
repeated attempts. The results of the County's analyses indicate
that there 1s no statistically significant difference between the
two groups. The results of the County's analyses suggest that Dr.
Praser's finding that there is a statistically significant
. .
difference between the. mean rank scores of Black and White
candidates may be due to an artificial increase in sample size, to
the confoundingeffeet of repeat performance, or to the individual
characteristics of the 18 persons involved, and flot to any ethical
bias in the test.
Kere important, the plaintiffs' statistical analysis does not
meet the supreme Court's J:8qU1reII8nt that the plaintiffS prove
causation. 'rhe fact that there are differences between the ..an
rank scores of Black and White candidates does not prove that any
disparate number of Black candidates were excluded. from the
promotional process.
'n1e relevant inquiry is not whether the
30
l .
.". "'~'" '.
....-'".: . . ~".'#
-. ..'". I..',.
'-, :. '.'i";:?..':-~. .
. .~:....''':''
".'
. .:..:.....~~:
j
the assessment center as would have been promoted without it, it
is difficult to see how the Plaintiffs' statistical analysis, even
assuming it was properly conducted, proves that the lack of more
Black promotions was caused by the assessment center. Without
proof of causation, the Plaintiffs cannot meet their burden of
establishing a prima facie case of disparate impact.
The lack of any substantial adverse impact froll the a88es8.entl
center is confiJ:med by Dr. Santisteban' 8 chi-square analysi8. '1'h~
chi-square procedure is a statistical method for examining whether
differences in promotion rates are a8sociated vith ethnic
differences. See Black Law Enforcement Officers v. Citv of Akron,
40 F.E.P. Cases 322 (R.D. Ohio 1986), ~ff'd in oart and rev'd in
part, 824 F.2d 475 (6th Cir. 1987).
Accord powers v. Alabama
Deot. of Education, 854 F.2d 1285 (11th Cir. 1988) (discussing with
approval the use of the chi-square procedure by a 'litle VII
plaintiff).
The chi-square procedure e8tabli8he8 an expected
selection rate, then detexmines whether the difference between the
expected 8election rate and the actual 8election rate i8
8tati8tically 8ignificant. Small deviations are tl!xpected to occur
(lS a result of chance, but at 80me point the deviation become8 80
large that chanCe alone can be ruled out.
Ifhe level of
8ignificimCe i8 generally 8et at .05, equivalent to a chanCe of 1
in 20. Dr. santisteban concluded that the chi-8quare analysi8 he
conducted revealed no 8ignificant difference among the promotional
rates of Black, White, and Hispanic candidate..
In 8Upport of their allegation of a 8ubstantial
32
.. ~. ,/0
,
. .
r
f .
~
Exhibit 4
Liability Insurance Certificate