Welcome to BJCPExams! BJCPExams handles the complete life-cycle of an Exam from the Grader and Exam Director point of view - from the time that the Exam is received from the Exam Administrator (via scans from AHA), through final creation of the RTPs for publishing to the examinees. To get started:

• Go to My Info / Personal. Create a JPEG image file of your signature and upload it. The signature will automatically be pasted into the RTP - no need to copy/paste for each one. Also, change your password from the one that BJCPExams initially assigned to you. Verify your contact information (name, BJCP Rank and ID, Email, Phone) - and if they need to be updated, log in to https://bjcp.org to change them.
• Next, go to My Info / My Exams. This is your main page to determine which exams are assigned to you, and what parts of the exam you need to deal with to move the exam grading along to completion. Each yellow-highlighted box that has a question mark or red X in it needs to be addressed. Your goal is to have every yellow-highlighted box contain a green checkmark to indicate that that task has been completed.
• To contact the other graders and directors for exams, go to My Info/Exam Contacts. You'll probably already be in contact with the other people, but this is the place to go if you don't have the contact info that you need.
• Each grader has a highest level for which they can be used - 2nd Grader (2G), Lead Grader (LG), Associate Director (AD), or Exam Director (ED). If you can be used as a Director, you will see an Exam Director section in the menu, with links to the various tasks that you will need to perform. All levels will have the Grader section in the menu, as well as the Wrapup section to do the final RTP creation (which can be done by any level). In general, grader scores can only be changed by that grader, but True/False and BJCP Program essay question answers can be assigned to any level (typically done by the ED, but can be delegated).

As an Exam Director (ED) or Associate Director (AD), there are three main functions that you need to perform in BJCPExams from when you receive the initial scans and completed exam notes from the Exam Administrator (via scans that the AHA does for us):

• Initial creation of the exam in BJCPExams, along with filling in the exam and question details and, for Judging (tasting) exams, entry of the scoresheet scores from each examinee and proctor, along with the proctor consensus scores
• Monitoring all exams for progress
• Reviewing and finalizing the score information and RTP that is returned to the examinees

Here are more detailed instructions for these tasks:

• Add/Edit Exam (under Exam Directors on the menu) allows the ED or AD to create an exam in BJCPExams and specify it's parameters and who will be working on the exam. Start by entering the Exam ID (YYMMSSCC typically - year, month, state, and city code). Next, choose each of the four graders who will be working on this exam - the ED, the AD, the LG, and the 2G. Select a name, then click the appropriate button. If you are adding someone who is not yet set up in BJCPExams, use the Add A BJCP Grader section - this will allow you to choose from all currently active National or higher BJCP judges, and will fill in all information needed to set the judge up as a grader. Specify the highest job function that this judge is allowed to perform (you can always change that later).
  
  Next, fill in the Exam details - the date that the exam was held and the exam location (e.g., Brighton, CO - the city and state, country if not USA). Choose what type of exam this was - BJCPExams currently supports Legacy Mead (combined essay and judging), Beer Judging, Beer Essay, and the new Mead Judging. Cider Judging will be added when it is released. Choose the number of examinee seats that were allocated for the exam (typically 12 or 24), then click Save Exam.
  
  The ED can go back in later to modify some of these settings (particularly the staff names) - but the number of seats and type of exam can not be changed once the exam has started to be graded. The staff names can be changed if different graders are to be used, or if the ED or AD change. Note that whoever created the exam initially will always be able to edit these settings - so if you give away your ED status for a particular exam, you can get it back.
  
  Once the exam has been created in BJCPExams, the exam questions need to be set up. Click on Exam Director / Exam Questions to enter the list of True/False question IDs (if this exam has the T/F questions), and also the CHOICE values for each essay question, as appropriate (this is where you specify which styles are to be compared and contrasted for Beer Essay exams, for example). For Tasting/Judging exams, provide a sentence that describes each sample, as given by the Exam Administrator - this should contain the SubStyle (e.g., English Barleywine), notes from the Administrator about the sample (homebrew/commercial, defects, perfect example, out-of- style, etc.), and the approximate score that the Administrator expects. After entering the appropriate information, click Save Questions.
  
  Finally, for Judging (Tasting) exams, go to Exam Director / Scoresheets to enter the scoresheet scores. Get the Proctor and Proctor Consensus scores from the Administrator report sheet, and get the scoresheet scores for each examinee from their scanned PDF file.
• Monitoring Exam progress - use the My Info / My Exams menu option to see all exams that you have been assigned to (in any capacity). This will show you which parts have not been completed - the highlighted yellow ones are ones that you can change. For AD or ED, you should make sure that all sections to the left of the grader inputs have been completed, including the Essay, True/False, and Judging/Tasting question lists, the examinee BJCP Program answers (if appropriate), and the examinee T/F answers (if appropriate). Optionally, you may delegate entry of these last two to the AD, LG, or 2G.
• Finalizing and publishing the completed exam - once the graders have finished grading the exam and have come to a consensus and have written the RTP notes, your My Info / My Exams section will show green checkmarks for 2G and LG grading, as well as for the RTP Notes and the LG Consensus. Once these are done, the AD can review the RTPs and compare with the exam PDFs to determine whether the LG Consensus from the Lead Grader needs any adjustment. When the AD is happy with the numbers, the AD Consensus can be entered - this does not need to agree exactly with the LG Consensus, but if more than a few points off (particularly if crossing a Judge Rank level), it should be discussed with the Graders. When the AD has completed the AD Consensus, the ED can do the same - reviewing as needed, and then entering the ED Consensus numbers. At this point, the exam is complete, the RTP will reflect the ED Consensus numbers, and can be published.

As a Grader, there are several functions that you need to perform in BJCPExams from when the ED assigns you to the exam, until you've completed the RTPs and consensus scores:

• Monitoring your exams for status - so you know what needs to be addressed
• Review the text of the questions to verify that the ED has set them up correctly
• For any essay True/False or BJCP Program questions, enter or verify the answers given by examinees (these can be updated by ED, AD, LG, or 2G - there is only one copy of them - not one per grader)
• For any Judging (tasting) questions, verify that the Exam Administrator notes have been entered - these are the notes by the person who administered the exam about each sample - what the style is, what the potential issues are (the sample was in the wrong category, is defective, is perfect example, etc), and the expected score (according to the Administrator)
• Go through the process of evaluating the scores for each question (essay or tasting) for each examinee, using either paper notes, or using the My Notes feature in BJCPExams, and assign grading scores for each
• Creating the RTP notes for each question and examinee, as appropriate
• Coming to a consensus score with the other grader
• Telling the AD/ED that you have completed the scoring, consensus, and RTP creation

Here are more detailed instructions for these tasks:

• Verify that the exam is ready to be graded (and that the ED has completed the setup). Start by looking at My Info / My Exams to make sure that everything to the left of the 2G and LG columns has green checkmarks. The ED or AD may assign you to evaluate the BJCP Program or True/False answers and enter them - this is done in the Grader / Scoring section. These two essay sections can be assigned to (and modified by) the ED, AD, LG, or 2G, so the ED may delegate these responsibilities.
  
  Next, check that all of the question text is configured. Under Grader / Scoring, choose your exam, and then drop-down the Pick. For each essay question, you should see the text of the question - and there should be no CHOICE values - each of the CHOICE values (for example, the list of substyles to compare and contrast) should have been specified by the ED. For each judging/tasting question, you should see the substyle of the sample listed.
  
  If you see all of these correctly, you're ready to get started on the grading.
• Doing the Grading - use the Grader / Scoring section to review any essay True/False and BJCP Program questions and enter their results. Then, for each Essay or Tasting/Judging question, use your own methods to determine the appropriate score for each sub-part of each question. NOTE - All scores are on a 100-point percentage scale which makes it easy to know how you're scoring each sub-part on a judge ranking scale. You can take notes on paper if you like, or you can write your notes of your thoughts as you were assigning scores by clicking the My Notes button next to each question. These notes are only for you - they are not visible to the other grader or to the directors. These notes are VERY handy to be able to quickly defend/support your scoring when coming to consensus with the other grader - and also can be used to paraphrase or copy/paste when you are creating the RTP notes for that question. Assign scores for each sub-part of each question for each examinee. You can enter the scores one examinee at a time, or you can enter them one question at a time - or you can freely switch back and forth at any time, as needed - all of the scores come from the same database table.
• Create the RTP notes - After you have scored a question, use your paper or My Notes notes to create the text that will be displayed on the RTP to provide feedback to the examinee. The RTP Notes are visible to both graders and both directors, and any of the graders or directors can edit the RTP notes. Typically, the LG is responsible for producing the RTP notes, but one or several RTP notes should be delegated to the 2G for experience purposes. Since the My Notes section for each grader is only visible to that grader, one grader may write the RTP Notes, and the other grader (by agreement) may tweak the RTP Notes based on his/her own My Notes about that question.
• After you have completed your grading of the exam, check with the other grader (you can see their status by looking at My Info / My Exams). When you have both completed the initial grading, share your scores (you can see these on the Grader / Consensus page) and discuss any significant differences and adjust scores as needed to get to a consensus. The LG is responsible for assigning the LG Consensus score - the consensus arrived at by the two graders.
• After consensus scores have been reached between the graders, either grader can use their own notes to create the RTP text, both on a per-question basis (as needed) and for the overall Wrapup / RTP Comments (the introduction, the recommended reading, the conclusion, etc.). BJCPExams creates the scoring graphs on the RTP for you - no need to do anything with that, other than eyeballing them to make sure they make sense, and that you have addressed the graph levels in the comments.
• Finally, when all consensus and RTP notes have been completed, use the Wrapup / Generate RTPs section to look at the RTP for each examinee - make sure that everything makes sense, and that the appropriate notes have been recorded. When the RTPs have been verified, contact the AD and ED to let them know that the graders have finished grading the exam and created the RTPs - the AD and ED can then go in to look at the RTPs and assign their own Consensus scores.

Add a BJCP grader: (type last name)

    Job Function2nd Grader Lead GraderAssoc DirectorExam Director         Update highest job function Email?

Grader Dates:Date that exam became available to Graders (YYYYMMDD):
 Date that Cover Letter was emailed to Graders (YYYYMMDD):  

Tour of BJCPExams Features