13.E. Setting Up Teacher Observation Rubrics

13.E. How to Set Up an Observation Rubric

Setting up an Observation Rubric

Setting up a teacher observation rubric/form within eDoctrina is a task that can only be completed by a user at the "Administrator” level (school or district). There are many controls that need to be understood, which means it is important to reach out the the eDoctrina team before taking on this task to ensure a high quality rubric/form is built in your district.

The first step is to navigate to the Observation Rubrics option in the Settings menu as this is where the shell of the rubric/form is managed.

This page will list all of the observation rubrics that are linked/assigned to the district in accordance with the selected filters.

The button will exist for any user that has permissions to create new observation rubrics. But please make note that some rubrics entail a very complicated score calculation methodology and it is a good practice to contact the eDoctrina team for gidance in configuration of such rubrics.
The will be filters to show which rubrics are linked to specific schools and school years. Just rememeber that a single observation rubric can be linked to one or many schools and/or years.
The edit button will be available for any observation rubric that the signed-in user has access to. For some rubrics, this will not be available because an eDoctrina team member may have configured the rubric and this rubric should not be able to be edited by some other user. If you would like to make changes to one of these rubrics, then please contact the eDoctrina team.
The copy button will be available for any observation rubric that the signed in user has access to. Please be careful when using this tool as copying a rubric will also copy all rubric assignments and may lead to duplicate rubrics displaying. If you are looking to move a rubric into the next school year, you are probably looking to add another rubric assignment, not another rubric.
The Hide from teachers checkbox is available to ensure that teachers cannot view observations within eDoctrina. This control is currently only applicable to the electronic access, so you should keep in mind that email notifications still may apply.

NOTE: If eDoctrina has configured this rubric for the district, the edit pencil may be unavailable. In this scenario, any changes that need to be made will have to be done by an eDoctrina team member. The copy button will be available to all users with access to this page, but it is strongly recommended to not copy the rubric unless there are significant change to the rubric. In any scenario, it is best to reach out to the eDoctrina team to receive some basic training on this before changing any rubric configurations or settings.

Creating a New Observation Rubric

To create a new rubric, begin by selecting the Create button.

A pop-up window will appear that requires a name to be entered for the rubric. This name will be how this rubric displays on many printed forms and in the rubric filters throughout eDoctrina. Some examples of this name may be "Teacher Observation Rubric" or "Goal Setting Form".

The next step would be to designate an assignment for the rubric. If you are unsure about how the rubric will be assigned and would like to start building the staff expectations within the rubric, just click save and navigate over to the staff expectations page to start building. There are some qualities that will be determined, such as categories (i.e. announced), in the staff expectation configuration, which means that it is sometimes better to leave the configuration of the rubric assignments until after the rubric starts to take form in the staff expectations.

If the rubric assignment is ready to be made, click on the Create button to generate another pop-up where many of the details of the observation rubric assignment will be determined.

General

Name: The name of the rubric assignment will be how the rubric will display on the Assignments layout view of the OBSeRVE Dashboard. Most rubrics will only need one assignment, but there are some scenarios where more than one assignment needs to be created for a specific rubric. Some examples of this would be if there are different minimum observation requirements for the rubric (ex: Non-tenured received 2 announced observation and Tenured recieves only 1). There may also be unique print settings for each school or different electronic signature statements. If there are any differences in the settings within this pop-up, a unique assignment must be made and the name is the best way to differentiate these differences.
Year: The year selection determines which year the rubric will be available in on the OBSeRVE Dashboard.
District (if applicable): This selection will only be available to users who have accounts in multiple districts. This occassion is rare and this selection is mostly not applicable to district users.
School: If an observation rubric is to only be available in one school within the district, the school indication should be made here. If the rubric is to be available across the entire district, please keep the School pull-down as "--All schools within district--"
Locked: Selecting this assignment setting prevents any new observations from being created within the assignment and also prevents any future updates to be made on observations that have already been conducted. This setting is best described as an "ice down" for the specific rubric assignment.
Show Points: Many observation rubrics/forms are linked to related point values as the observer selects the associated ratings. If the associated point values are to be displayed on the observation form and the printouts, this setting should be selected. If the points are to be hidden, keep this setting unselected.
Sign with button only: This setting applies to observation rubrics where electronic signatures are enabled. By enabling this setting, an electronic signature can be applied without having to enter the signed in users Username or Password. It is suggested that this only be used for non-performance related observation rubrics as electronic signatures are important for data validity.
Collapse by Default: Each observation form is designed with a specific parent-child relationship among the staff expectations that have been entered. Enabling this setting in the rubric assignment will load the observation form initially by displaying only the first subcategory (child) underneath the main parent. This is a great setting for observation forms that have a few main categories and each of these categories will be rated in separate instances. The recommendation is to keep this setting unselected until the staff expectations are built so the collapse feature can be better understood.
Show not previously rated message: Depending on the configuration of the staff expectations, there may be a default rating selected that makes it difficult for an observer to recognize which expectations have been previously rated. Enabling this setting will display a message underneath the rating of the staff expectation indicating the the rating has not been previously rated.
View assignments by observer: This setting is to be used for calibration training as observations performed by other observers will not count towards the minimum requirements that the signed in observer will see. In other words, if there are two minimum required observations, each observer will be able to complete the two observations without the observations of other observers counting towards this minimum requirement. In application, one would set a minimum of one observation for this rubric assignment, enable this feature, add a calibration video, then ask the observers to fill out the observation form in accordance. This will allow for comparable data and great conversations about calibration.
Rubric Code: The rubric code is applicable to the "Auto Assign Rubric" function within eDoctrina. Assigning a rubric to each staff member is an essential step to being successful using the OBSeRVE module. If it is desired to manage rubric assignments through automated imports, each staff member should have an associated rubric code imported in column 13 of the staff import file. Maintaining rubric assignments in this manner will automatically update rubric assignments as long as the assigned rubric code for the staff member matches the rubric code in the rubric. There are many configuration steps required to get this set up properly, so it is recommended to reach out to an eDoctrina team member if this sounds like a good fit for your district.
Ignore unobserved expectations: Selecting this setting will not consider any unrated staff expectations in the total calculation for the rubric or category. This is a common configuration setting as many rubrics do not require that every staff expectation is rated to complete the observation event. If this setting is not enabled, unrated expectations will be counted as 0 and be averaged in accordingly.
Hide Notes and Attachments Column Enabling this option will remove the Notes & Attachments column within the observation form. The Notes & Attachments column is used when there are mini-discussion boards and dropboxes for a specific staff expectation. Not all rubrics use this option and it is a good practice to hide this column to optimize space on the observation form. This is unrelated to the main discussion board and attachments as these are configured for the main parent staff expectation.

OBSeRVE report

Within group summary type: This is the first setting related to how the main parent staff expectations and their associated ratings should be calculated. By selecting average within the group, the highest level children (underneath the main parent)and their selected/calculated ratings will be averaged to arrive at a single value for each main parent, respectively. Selecting sum will add all of the highest level children together to arrive at a single total value.
Across group summary type: This is the second setting related to how the main parent staff expectations and their associated ratings should be calculated. This setting is only applicable if there exists two main parents with calculated values associated with them. As each main parent is expected to result in a single value, selecting average will take the average between all of the main parent values within the observation rubric, while selecting sum will simply add the calculated values for all of the main parents.
Calculation type:
- Based on average scores: Selecting this calculation type will consider all observations within each category (if applicable) and average the scores for each expectation to produce a total score.
- Based on most recent scores: this calculation type is applicable if and only if there has been a minimum number on observations set for a rubric or category within a specific rubric. For example, if there is a minimum of 2 observations required and, for some reason, 3 observations occur the most recent 2 observations will be considered into the calculated score for the respective rubric or category. This most common use case for this is if a teacher requests an extra observation beyond the minimum requirements and only the most recent are to be considered. This is a very specific setting, therefore the recommended setting is to use "Based on average scores" for the calculation type.
- Based on average scores & Based on most recent scores: This setting exists in cases where there may be a combined way of determining the total score for a rubric or category. Choosing this option will display both the average and most recent calculated values on the observation reports.
- Custom Calculations: If a specific calculation type cannot be achieved using the existing functionality of eDoctrina, we will build specific calculation methods to accommodate for this. It is recommended to speak with an eDoctrina team member to conduct a feasibility analysis. This option should not be explored unless all existing options do not work for the nature of the rubric calculation methodology.

OBSeRVE report: Default print options

For more information on OBSeRVE reports, please visit our help guide on the OBSeRVE Report Dashboard.

Always use default print options: Default print settings can be selected in this section and every print out that is generated will follow these defaults. If it is preferred to keep all printouts consistent, it is recommended to always use default prints settings to keep all observation reports uniform. If this checkbox is not selected, the user who is generating the report will be able to modify the generated printout according to the available selections. It should also be noted that if checked, the the other options will be preselected by default so do not ignore the other default print options.
Include header with report scope: The report scope is a top-level section of the observation report that summarizes the selected filters upon generation of the report. The report scope will assist users in generating a report in eDoctrina that is similar to another. Most districts prefer to omit the report scope as it does not contain any information sensitive data.
Include all observation ratings: The observation ratings are the values that are selected for each staff expectation. This option provides the opportunity to hide or display these ratings.
Include script: The scripting area is used for many different purposes. The option to include the information that has been entered into the main script is available here.
Include suppporting information: If there has been supporting information that has been entered for a specific staff expectations, the option to display them on the print out exists here.
Include score summary table: Depending on the configuration of the district's rubric, it may be beneficial to include the Score Summary table in the report, which will display the weighted score for the observee's observation events and the number of observations that contribute to this score. Also, if the Final Rating has been selected, this will also display in this table.
Include discussion board: The discussion board can be used for many different purposes. If desired, the discussion board comments can be included on the print-out. The name of the user, the date the comment was posted, and the comment will display.
Include notes: If an observation rubric is designed to allow notes to be posted for each staff expectation, the option to include them is available.
Include attachments/links: If there has been any hyperlinks or attachments linked to the observation or a specific staff expectation, enabling this setting will include them on the printout.
Include Calculations: Including calculations will display basic calculations and how an observee performs for each staff expectation for each category(if applicable).
Include weighted points: This option does not display unless the calculations page has been also enabled. If an observation rubric is set up to apply special weighting to observation scores, displaying the weighted points will show the product respective observation score and its weight (ex: 4.0 X 90% = 3.6).
Include unweighted points: This option does not display unless the calculations page has been also enabled. The unweighted points option will display the observation score for each category(if applicable). The calculation method varies, but the calculated score for each category is displayed.
Include totals: The totals will display the current calculated score for each observee according to the rubrics calculation method.
Observer/Date/Time location: The most concise way of displaying the observer and the date/time of any observation is to include it in the header of the observation. This is the best option to choose when a single observer is conducting an observation. In other cases where there are more than one person contributing to an observation or evaluation, it can be beneficial to change this setting to "In row" as each selected rating will then show the person who selected it and when they selected it.
Orientation: Setting the default orientation helps to pick the optimal format when printing observations. Some forms benefit from printing in the wider landscape orientation, but the default for many rubrics is portrait because it provides more vertical space per page.

Assignment Requirements

The assignment requirements section of the rubric assignment determines the minimum and maximum number of observation events that must occur for the specific rubric assignment. It is possible to set minimums and maximums for the main parent staff expectation as well as any categories that have been set up within the rubric's staff expectations. A sample configuration may look like the below image.

NOTE: In this setup, there is a minimum of 2 announced observations (no maximum) and a minimum of 1 unannounced observation (no maximum).

TE Export Configuration

If your district is utilizing the Teacher Effectiveness System (where Student Learning Objectives and Professional Practice scores are combined), configuring the TE Export section allows scores determined on the OBSeRVE Dashboard to be sent to the TE System. The information and data that is transferred to the TE System varies by district and is usually linked to custom reports, so it is recommended to speak with an eDoctrina team member to ensure the results are as expected. Here is an example of a rubric assignment that has been configured to only send the Final Rating to the TE System.

Electronic Signature Statement

The electronic signature statement is only applicable to observation rubrics/forms that have electronic signatures enabled. The entered electronic signature statement will display on the the pop-up window where both the observer and observee apply their electronic signature.

eDoctrina has conveniently made this a WYSIWYG text editor so many variations of an electronic signature can be be displayed upon user signature.

Report Header & Footer

It is possible to enter any additional text to be printed in the OBSeRVE report header and/or footer. Although these fields are usually used for aesthetic purposes, they allow for extra information to be included on the observation forms. Many districts use the report header to add their school logo, while others will use the report footer to include an additional signature line.

Configuring the Staff Expectations for an Observation Rubric

After the observation rubric has been added to the district database (by following the steps above), it will now be possible build the staff expectations within the rubric. A Staff Expectation can be best described as the individual lines within the rubric form where ratings will be selected, supporting data will be entered, or just be added for a placeholder. These staff expectations are highly customizable. The different features available for staff expectations will described below in this document.

To begin building the staff expectations, start by navigating to the Staff Expectations option located in the SETTINGS menu.

To create or modify a specific observation rubric, simply choose the desired rubric from the pull-down menu. It is possible that an observation rubric to be linked to one or many years (as controlled by the rubric assignment). The Observation Rubric pull-down menu will be populated with all rubrics that are linked to the selected year. It is also possible for a created rubric to not be linked to a year. In these cases, choose "--any--" in the year filer to open up the search.

Although the filters are some of the most simple in eDoctrina, the complexity of the staff expectations page lies within the actual staff expectations. Before diving into the multitude of configuration options, it is important to understand the structure of how staff expectations are built.

Staff expectations are designed with high consideration to the parent-child relationship that is setup. To help to understand this we shall refer to the highest level staff expectations as the parent. The staff expectations one level down will be referred to as children. The next level will be referred to as the grandchildren, and so on. The expand and collapse each of the different levels, click on the plus sign and minus sign , respectively.

NOTE: If the edit pencil does not exist for a staff expectation, this means that it was created by the eDoctrina team.
If changes need to be made for a staff expectation without an edit pencil, please reach out to eDoctrina at
[email protected].

The highest level parent staff expectation defines the name of the of the observation event as it displays throughout eDoctrina. Most importantly, it is the title of the resulting individual staff report that will be generated. The main parents will also be a filtering option for Rubric Category on the OBSeRVE Dashboard. So choose these wisely as sometimes it is better to keep these separate and other times the rubric should be configured all underneath one main parent expectation.

. . . .

The main parent also includes some vital configuration options:

1. Enabling the Discussion Board for an Observation Form

The primary discussion board for observations is configured at the main parent of the rubric configuration. Set the option to Enabled (Yearly) if all discussion comments should be available for all observations for each teacher throughout the year OR set the option to Enabled (Observation) if each discussion board should be unique to each observation. There are some configuration options also available for email notifications on discussion boards.

2. Enabling Attachments for an Observation Form

The primary attachments.dropbox for observations is configured at the main parent of the rubric configuration. Set the option to Enabled (Yearly) if all attachments should be available for all observations for each teacher throughout the year OR set the option to Enabled (Observation) if each attachment should be unique to each observation. If this option is changed and there are already attachments made, the changes will not be applied retroactively.

3. Enabling Electronic Signatures for an Observation Form

Turning on the ability to electronically sign observations is as easy as checking the Electronic Signatures report checkbox for any of the main parent staff expectations. If enabled, end-users will find a signature option located in or around the observation they are looking to sign. Notice how you can only toggle this option on or off for a main parent staff expectation. If there are occasions where there exists some observation type within the main parent that does not require electronic signaturea(but others do require it), then do not worry because there is an option to disable electronic signatures at the observation type level if the main parent happens to be enabled for electronic signature. This will be explained later in this document. We should also note that there are some configuration options for electronic signatures that also exist within district settings.

Explaining the different Rating Types and their Utility

Each staff expectation can be configured using different rating types to modify how they behave in the observation form. It is important to understand each of these as each option can have a significant impact on the calculation methodology and general behavior of the observation rubric.

Observation Type

The observation type staff expectation may be the most improtant one to configure because this allows us to create different categories of observations within a main parent expectation. So if there is a scenario where we configure a rubric that has two categories, "Announced" and "Unannounced", then we can create a staff expectation with the Observation Type selected and this enables us to configure these different categories, their associated behaviors, and apply a weighting schema for accurate calculation methods. There can only be one Observation TYpe staff expectation per each main parent.

Scripting Area

Show Average

Show Category Average

Show Category Sum

Final

--none--

When “Insert” (plus button) is clicked, a pop-up menu will prompt you to enter the name of a Teacher Expectation. (Again, ensure you are editing the correct rubric in the drop-down menu at the top.)

Enter the expectation in the “Name” text box with the red asterisk. Note: eDoctrina lists expectations alphabetically, so if the rubric being entered doesn’t contain numbered or lettered expectations, a sorting system should be added. For example, in order for “Students Engaged” to come before “Rules and Procedures,” either a letter or number must be added (ex. “A. Students Engaged” and “B. Rules and Procedures”).

Then “Domain 1: Within Classroom Expectations” would not have a parent itself, “1.A. Classroom management” would have a parent of “Domain 1: Within Classroom Expectations,” etc.

In this example in the screen shot below, "2.0 In-Class Observation" might be one of the parent expectations. Nothing would therefore be selected in the "Parent" drop-down menu.

Selecting insert again would allow me to create a child of this parent expectation ("2.1 Students appear to learn"):

Once entered, this is what parent and child expectations look like from the Staff Expectations screen:

Note that in the screen shot above, Standard 2 has not been expanded as indicated by the blue plus button next to its name.

Any staff expectation may be deleted by clicking on the trash can to the right; however, child expectations must be deleted first:

Entering Ratings

Ratings for this expectation may be added by clicking "Insert" in the Ratings section.

This pop-up will appear:

A name for the rating is required, such as "Developing" or "Exemplary". It is recommended that the first item in the list be “Not Observed,” as this will be the default option. Users can enter narrative information that clarifies what the particular rating looks like in the Description box. These details help the observer select the most appropriate rating. For example, for an expectation of “Content,” the rating of “Distinguished” is described as “Teacher displays extensive knowledge of the discipline and how it relates to others.” “Points,” is optional. If your school/district is assigning point values to particular ratings, that number would be entered here. If points aren’t being used, that field can be left empty.

Please note that not all expectations require ratings. In the example used above, “Domain 1. Within Classroom Expectations” and even the next level, “1.A. Classroom management” are just headings. It’s not until the next level down, “1.A.1. Students engaged” and “1.A.2. Rules and procedures posted” that ratings are necessary. Knowing which skills will be rated/not rated is much easier if users have a good, working knowledge of the rubric prior to entering it.

Selecting Observation Type

Another optional step is manipulating the “Type” drop-down menu (see below).

Clicking the edit pencil next to the Staff Expectation brings you back to this pop-up, where a Rating Type can be identified:

Most expectations can be left with the default (any/no observation type ). However, if selected, this option allows users to weigh different types of observations. For example, a district uses calculations to weigh both their formal (70%) and informal (30%) observations. In this case, a user would select “Observation Type” in the “Rating Type” drop-down and then insert two ratings in the “Ratings” section below: “Formal” with points equal to 70 and “Informal” with points equal to 30.

In addition, users have the option enter the number of observations required for each observation type under the "# Required" column. This works in conjunction with the "Required" feature on the OBSeRVE Dashboard. (For more information, please refer to the OBSeRVE Report Dashboard help guide.)

The "Send schedule Email" option helps in managing whether a user will receive an email notification once their observation is complete. There are 3 choices to select here: 1. Per user selection: This will allow Observers the choice of whether they want to send an email or not. This is determined when Scheduling an Observation: 2. Always: This option will always send and email notification when an Observation is scheduled. You will not see the "Send to Observee" check box if this option is chosen. 3. Never: This option will never send and email notification when an Observation is scheduled. You will not see the "Send to Observee" check box if this option is chosen.

Observation Type: Final

In most cases, you must create a Staff Expectation that is assigned the Observation Type "Final".

When an observation is started for a teacher under this Final Staff Expectation, all prior ratings will feed into this screen (per the calculation type selected), and the observer will be able to confirm the final rating by selecting the appropriate score. (If the APPR report will be used, this "final" number will then flow into the APPR report.)

Once finished, click “Save.”

Please note that observation rubrics can be complicated, especially if point values and weighted scores are required. In these cases, don’t hesitate to contact eDoctrina’s Technical Support team at 1 (888) 537-3348 or [email protected].