Peer Assessor Information

Thank you for agreeing to serve as a member of an Arts Nova Scotia (ANS) Peer Assessment Committee (PAC).

Nationally and internationally, peer review is a well-established standard for assessing achievement and merit in the creative and scientific worlds. It is a decision-making process used in medical and scientific research, in law, and in education.

Arts Nova Scotia, like all other arts councils in Canada, uses a peer review process to evaluate applications to its granting programs. The people who serve as assessors are crucial in deciding who is awarded grants; they provide ANS with independent and informed opinions and encourage accountability within the arts community.

The purpose of this guide is to provide important information about your role and responsibilities in the assessment process, considerations as you review the applications, and, ultimately, make your recommendations.

Serving as an assessor requires commitment to a time-consuming and often challenging process. Arts Nova Scotia appreciates your involvement and contribution. If you have any questions or comments, please contact the Program Officer responsible for the program you are assessing.

About Arts Nova Scotia

Arts Nova Scotia is the provincial funding agency that provides support to professional artists and arts organizations, arts education programs and a number of arts awards and prizes. You can read more about our organization here.

Policies

In addition to our mission and values, it's important for assessors to have a familiarity with our policies, including:

Who is involved in a peer assessment?

Assessors

Assessors are professional artists and other arts professionals who:
•   represent a diversity of artistic and cultural practices and expertise;
•   represent the diversity of the province;
•   bring vision, open-mindedness and generosity of spirit to their deliberations;
•   provide fair and objective analysis, based on assessment criteria;
•   treat applications equitably, without prejudice
•   can work collaboratively.
•   are willing to express their views while respecting and listening to the views of others.

Eligibility

Assessors are professional artists or arts administrators with an ongoing connection to the Nova Scotian arts community.

If an assessor cannot be found in Nova Scotia for a particular PAC, artists from other Atlantic Canadian provinces may be asked to participate.

Recommended assessors do not have to be former grant recipients.

When possible, ANS tries to limit participation of individuals in PACs to once every two years.

Composition

A new and independent PAC is appointed for each program at each grant deadline. The PAC is normally comprised of three eligible individuals., in select some programs, committees are made up of five members.

Peer Assessment Committees are designed to have expertise in the disciplines and skills needed to review the applications under assessment. While recognizing that no one committee can include all the varied demographics of the province, Arts Nova Scotia strives for a balance over time that is sensitive to and reflects the diversity of regions, genders, equity, generations, cultures, styles and skills in artistic practices in Nova Scotia.

Peer Assessment Committees are not randomly selected but are chosen after considerable thought and research by the program officers. Officers accumulate information about potential assessors through ongoing, regular contact with their communities. Arts professionals from all cultural communities and regions of the province are encouraged to submit names of potential jurors and advisors, including their own. Added to this list on an ongoing basis are recommendations from applicants, assessors, and other professionals in the field.

Anyone interested in serving as an assessor, can put their name on the list by signing up here.

Compensation

Assessors will be compensated for the reading of applications and for their time during the peer assessment meetings. We follow the CARFAC recommendations for full day and half day flat rates. Per the CARFAC guidelines, these flat rates are inclusive of the preparatory work of reviewing the applications. While the CARFAC recommendations do not include reading fees, we include these on top of the flat rates in recognition of the value of artists' time.

Flat Rates:

$362/half day (up to 4 hours)

$638/full day (more than 4 hours)

Reading Fees:

The reading fee will be dependent on the complexity of the application ranging from $6-$10 per application.

Other expenses:

At this time, our PAC meetings will take place virtually. If meeting in person, lunch and refreshments will be provided, and a per diem provided for additional meals as required by travel schedules. Accommodation (if required) will be arranged by ANS and travel expenses will be reimbursed upon the presentation of receipts and completion of the non-employee expense form provided by ANS staff.

Program Officer

The Program Officer ensures that assessors understand the materials provided, the assessment process, and addresses any concerns that may arise for assessors. The Program Officer chairs the PAC meeting, facilitates discussion, and ensures that all competing applications receive equal and fair assessment according to the established assessment criteria. The officer does not have a vote and may not express any opinion about the merit of an application. They will, however, act as a resource person, push the committee members for clarification, and contextualize the application if required by providing factual information. The Program Officer will answer questions where they can, when appropriate.

The Program Officer keeps the discussion on time, ensures that discussion is recorded, and oversees the scoring process. They ensure that there are no direct conflicts of interest and that indirect conflicts of interest are declared ensuring appropriate procedures are followed. They ensure that the assessors are reminded of ANS priorities, program priorities, budget context, as well as managing the available budget, ensuring the results have unanimous support from the committee, and the final grant decisions are signed off.

Admin Support

Program Officers work with a team of support from the Grants Management Team in the Department of Communities, Culture and Heritage. Their role is to intake applications, prepare applications for the PAC, assist with meeting logistics such as preparing scoresheets, note-taking and technical support. They record PAC decisions into a database and prepare sign-off sheets. For in-person meetings, support is provided to arrange accommodations and catering in the meeting.

Observers

From time to time, Arts Nova Scotia will permit observers during the assessment process. An observer could be the ANS Director, another program officer, an officer in training, an Arts Nova Scotia board member, or guests from peer Arts Councils.

The PAC will be notified at the start of the session and on the day in question if an observer is to be present in the room. PAC members will have the opportunity to discuss with the Program Officer should any questions arise. In order to minimize disruption of the assessment process, observers will not sit at the assessment table, speak, or in any way communicate with the assessors beyond their initial introduction.

Preparing for the Meeting

Peer Assessors are sent all materials prior to the meeting containing the program guidelines and assessment criteria as well as application and support material for each applicant. It is the responsibility of the assessor to become familiar with the program, guidelines, assessment criteria, and make notes about each application in the package prior to arrival. All support materials (images, video, audio, written) are included in the package and must also be reviewed in advance.

It is important that assessors provide a fair, unbiased assessment of each application based on the following criteria – the artistic merit of the project, the artistic activity, and the ability of the individual or organization to carry out the project. While these are the criteria used to score, we want you to also be aware that other issues for consideration can arise that are designed to broaden the scope of the final outcome.

A score sheet for each application is included for you to make notes and make preliminary scores which we will use as a tool for ranking applicants. All preliminary scores should be sent to the Program Assistant the day before the meeting day.

Receiving Your Files

A member of our admin team will send you a download package with all of the applications using our Secure File Transfer Service. Keep an eye on your email for a message with the subject "Secure Message is Waiting."

TIPS

Once you receive notification that the files have been sent to you, you have 14 days before the download link expires, so be sure to download your files as soon as you can.

You will be prompted to change your password when you first login to the Secure File Transfer Service. Be sure to remember this password in case you need to log in again.

After the Meeting

All materials remain the property of ANS and must be returned/deleted at the conclusion of the meetings. ANS will inform candidates of the PAC’s decisions. Upon request, the Program Officer will communicate feedback to the applicants, but will not attribute comments to a specific PAC member. Names of PAC members will be available in alphabetical order in the Annual Report and not attributed to specific committees.

Joining the Meeting

For now, all Arts Nova Scotia Peer Assessment Meetings meet using Microsoft Teams. We recommend you download the Teams app prior to the meeting. You have an option to join using your browser, but the app is much more user-friendly and functional.

Click here for a tip sheet on connecting to Microsoft Teams.

During the Meeting

Peer Assessors are responsible for bringing all notes and application materials to the committee meetings. The Program Officer will then facilitate discussion by seeking equal engagement from all members of the PAC for each applicant. If a point has been made by one assessor, they will ask that members indicate whether they agree or not rather than repeat this point. Officers will ensure that discussions remain cordial and focused and will halt any discussion that is inappropriate to the competition (e.g. anecdotes, what-the applicant-should-have-done). Notes will be taken for feedback and will be compiled in an aggregate manner to not attribute comments to a specific assessor.

Please feel free to ask the Program Officer any questions you may have. Quite often questions raised by one peer assessor are on the minds of the others. Applications will be rescored when discussion concludes for that file. Once final scores have been consolidated, the program assistant will create a ranking which will be used to help the PAC make final decisions on who is awarded grants.

Assessors are expected to read through Arts Nova Scotia policies and consider key questions as they are reading through applications.

After the Meeting

Confidentiality

It is extremely important that confidentiality be maintained in your role as a peer assessor as per the Peer Assessment Committee Confidentiality Agreement and Policy. PAC members must maintain the highest degree of confidentiality regarding the assessment process, the identity of fellow committee member, and applicants to the program. Discussions of PAC proceedings will be limited to other members of the committee and Arts Nova Scotia. It is in your interest to avoid being subject to any kind of influence prior to the meetings or any issues afterwards.

Applicants will be informed of the results by email after the meeting and the names of successful applicants will be listed on Arts Nova Scotia website at the end of the fiscal year. Names of PAC members will be available in alphabetical order in the Annual Report and not attributed to specific committees.

Click here to view the Confidentiality Agreement.

Conflict of Interest

When composing the peer assessment committees, great care is taken to avoid conflict of interest situations, however, it is recognized that the Nova Scotia arts community is small and inter-related and all connections may not be immediately apparent. If you are involved with the application or may benefit, or might appear to benefit from a requested grant, you may be in conflict of interest. Conflicts can be real, apparent or perceived. Please discuss any conflicts with the Program Officer in advance of the meeting and be prepared to declare any conflicts at the meeting. Members in a conflict of interest position will be excluded from the discussion and scoring of the particular application(s). Questionable involvement of a PAC member shall be resolved in favour of the integrity of the process and Arts Nova Scotia’s credibility in the community.

Assessor Dismissal

While it is a rare occurrence, there are circumstances when an assessor will be dismissed from their responsibilities:

Has a direct conflict of interest that precludes participation.
Has not read all the applications and materials, and reviewed all artistic examples, e.g. audio/visual or digital examples and/or documents, made notes about each application based on the assessment criteria and pre-scored each application before the first day of the assessment panel.
Is unable to attend all the required days of the assessment meeting.
Obstructs or disrupts the assessment process or is disrespectful to staff, assessors or applicants.
Discriminates against applicants, assessors or staff.

Depending on the circumstance of dismissal, the assessor may be paid a stipend to reflect the work up to the dismissal. The Director of Arts Nova Scotia in discussion with the Program Officer will determine fair compensation.