HD8 - House Joint Resolution 586 (2011): Requesting the Bureau of Insurance within the State Corporation Commission to Collect Certain Demographic and Other Information from Agent Licensing Examination Candidates


Executive Summary:
Summary of Legislation

2011 House Joint Resolution (HJR) 586 requested the Bureau of Insurance (Bureau) within the State Corporation Commission to collect certain demographic and other information from producer licensing examination candidates for the period of July 1, 2011 through July 1, 2015. The Bureau was requested to post information collected biannually on the website of the State Corporation Commission and to continue examination and review of the data collected from July 1, 2008 (pursuant to 2008 HJR 196) through July 1, 2015. The Bureau was also requested to continue changes to the licensing examination process to ensure that licensing examinations are fair, unbiased and designed to test entry-level knowledge regardless of the race, gender or educational background of the candidate. The Bureau was asked to submit to the Governor and General Assembly an executive summary and report of its findings and recommendations for publication no later than the first day of the 2012 Regular Session.

This Virginia Insurance Demographic Report July 1, 2011, through October 31, 2011 was developed for the Bureau by its contracted insurance licensing examination provider (Pearson VUE). The report summarizes statistical information related to all insurance examinations administered in Virginia by Pearson VUE between July 1, 2011, and October 31, 2011.

House Joint Resolution (HJR) 196 from the 2008 General Assembly requested the Bureau to collect certain demographic information from agent licensing examination candidates for the period of July 1, 2008 through September 30, 2008. Data collection continued for two years, and reports of collected test and demographic data information were posted from July 1, 2008 through June 30, 2011 on the Bureau website at http://www.scc.virginia.gov/boi/resp.aspx.

Data collection pursuant to 2008 HJR 196 continued for two years. Reports of collected test and demographic data information were posted biannually on the Bureau's homepage through the website of the State Corporation Commission.

Virginia Demographics Reports:

Jan. 1, 2009 - June 30, 2009
July 1, 2008 - June 30, 2009
July 1, 2008 - Dec. 31, 2009
July 1, 2008 - June 30, 2010
July 1, 2010 - June 30, 2011

The Bureau also submitted a report to the 2009 General Assembly with an executive summary and the information collected, which was published as RD 28, "Report of the State Corporation Commission, Bureau of Insurance on Certain Demographic and Other Information From Agent Licensing Examination Candidates for the Period of July 1, 2008 through September 30, 2008."

Recent Changes to the Virginia Insurance Licensing Examination Process

Demographic data collected from July 1, 2008 through June 30, 2010, was evaluated by Pearson Vue on behalf of the Bureau. Subsequent to the review, the passing standards for the Virginia specific section for the Life, Health, Life & Health, Title and Property & Casualty examinations were reviewed and modified by a committee of subject matter experts from the insurance industry and the Bureau. As a result, new passing standards for the Virginia examinations went into effect in November, 2010.

In April, 2011, the Bureau introduced new retest rules for Virginia Insurance Licensing examination candidates. Candidates are required to pass both the general and state specific sections of the Virginia examination. If a candidate should fail any section of the examination, he may retake, under the new retest rules, the failed section of the examination only. Prior to April, 2011, candidates were required to retake and pass the entire two-part examination to receive a passing score on the Virginia examination.

Scope/Method of Work

On July 1, 2008, Pearson VUE added the demographic data questionnaire to the Virginia Insurance Licensing examination. The questionnaire allowed Pearson VUE to collect information on the applicant's gender, ethnicity, education, English-language ability, native language, and age, along with the candidate's method of preparation for the examination. Candidates were advised that the requested information concerning a candidate's pre-licensing study course was required for the examination. In addition, candidates were notified that demographic information would be collected for statistical reporting purposes from those applicants willing to provide the demographic information. Candidates were advised, however, that refusal to disclose such demographic information would not affect the result of the applicant's examination.

During this reporting period (July 1, 2011 through October 31, 2011), candidates were advised that they were not required to respond to the demographic questions. However, all demographic questions were answered by more than 99% of candidates. At the request of the Bureau, Pearson VUE psychometricians used the information collected from candidates to calculate pass rates, mean and standard deviation of the scaled scores, mean item proportion answered correctly (p value), and mean point biserial correlations (pBis) for candidates in each demographic category.

The information contained in Appendices A through F summarizes statistical information related to all insurance examinations administered in Virginia by Pearson VUE between July 1, 2011, and October 31, 2011, the last quarter of data available for analysis prior to the deadline of this report. Demographic survey results for all candidates are broken down by examination level.

The Bureau has also placed the demographic survey results on the Bureau website at http://www.scc.virginia.gov/boi/resp.aspx.

• July 1, 2011 - October 31, 2011

Recommendations

Pearson VUE cautioned the Bureau against interpreting differences in mean scores without evaluating the statistical significance of the differences. The Bureau recommends against interpretation of statistics when candidate volumes are low (e.g. typically fewer than 50 candidates) since interpretation of statistics based on small sample sizes may be misleading.

Note: comparisons between pass rates referred to in this report and pass rates referred to in 2009 RD 28 will be impacted by the implementation of the new passing standards in November, 2010 and retest rules beginning April, 2011.