Guide to Judging: Judging Engineering Notebooks – VIQC

Overview: The Engineering Notebook

REC Foundation programs help students develop life skills that they may use in their academic and professional future. Documenting work in an Engineering Notebook is a widely used engineering and design industry practice. By following the Engineering Design Process and documenting that process in an Engineering Notebook, students practice project management, time management, brainstorming, effective interpersonal and written communication skills.

The Engineering Design Process is iterative: Students identify and define a problem, brainstorm ideas to solve the problem, test their design ideas, and continue to refine their design until a satisfactory solution is reached. Students will encounter obstacles, successes, and setbacks as they work through the Engineering Design Process. All of these should be documented by the students in their Engineering Notebook.

Below is an example graphic outlining the steps of the Engineering Design Process:

In REC Foundation programs, the Engineering Notebook is required for the Excellence, Design, and Innovate Awards, but is not a requirement for other awards. Submitting a notebook is not required for a team to receive an in-person interview, and all teams at an event must be given the opportunity to be interviewed.

Teams may use the notebook available from VEX Robotics, or they may purchase a different form of physical notebook. Teams may also use any one of various computer applications or cloud-based services available for digitally creating and maintaining a Digital Engineering Notebook. Please see the section on Remote Judging for more information on Digital Engineering Notebook submissions. Regardless of the format, all notebooks are evaluated by the Judges according to the same award criteria and rubric.

Engineering Notebooks should contain these elements:

Team number on the cover/beginning of document
Errors crossed out using a single line (so errors can be seen)
Unedited entries
All pages intact; no pages or parts of pages removed
Each page/entry chronologically numbered and dated
Each page/entry signed or initialed by a student author
Team meeting notes as they relate to the design process
Permanently affixed pictures, CAD drawings, documents, examples of code, or other material relevant to the design process (in the case of physical notebooks, tape is acceptable, but glue is preferred)

Outstanding Engineering Notebooks should contain these additional elements:

Table of contents
Entries are dated with the names of contributing students included
Notebook begins with the first team meeting
Descriptions, sketches, and pictures of design concepts and the design process
Observations and thoughts of team members about their design and their design process
Records of tests, test results, and evaluations of specific designs or design concepts
Project management practices including their use of personnel, financial, and time resources
Notes and observations from competitions to consider in the next design iteration
Descriptions of programming concepts, programming improvements, or significant programming modifications
Enough detail that a person unfamiliar with the team’s work would be able to follow the logic used by the team to develop their design, and recreate the robot design using only the Engineering Notebook

Note: If the Engineering Notebook is written in a language that is not common for the region, it is the team’s responsibility to provide the original language version along with a translated copy, if any Judges fluent in the original language are not available. This should be brought to the Event Partner’s attention as early as possible so they can inform the Judge Advisor.

Note: Different teams may submit notebooks with varying levels of sophistication and beautification. For example, some teams may have brief sketches in pen, others may have colorized illustrations or CAD/electronic drawings. Judges should be cognizant of evaluating the content of notebooks, not the level of beautification. It is possible for many different types of notebook and different communication styles to present relevant content explaining the design process.

Notebook Submission Format

The choice of judging format for the event rests with the Event Partner. Detailed information about judging should be found on the event page on RobotEvents. All teams at the event must submit their notebooks in the same format, regardless of its native format. A team with a physical engineering notebook will need to upload a link to a digital copy via RobotEvents, or conversely, a team with a digital engineering notebook may be asked to print it out prior to the event.

Irrespective of whether the notebook is submitted digitally or in-person (physical notebook), teams are responsible for their notebook’s formatting, presentation, and ensuring all materials are properly organized, including numbering and/or dating pages.

Engineering Notebook Judging Process

Step 1 – Sorting the Notebooks

Judges perform a quick scan of all the Engineering Notebooks and divide them into two categories: Developing and Fully Developed.

Developing Engineering Notebooks contain little detail, will have few drawings, and will not be a complete record of the design process. To save Judges’ time, the Engineering Notebook Rubric will not be completed for these teams. However, all Engineering Notebooks should be retained until the end of judging deliberations.

If it is unclear whether a notebook should be categorized as Developing or Fully Developed, either another Judge can help make that determination, or the notebook should be given the benefit of the doubt and scored using the rubric.

Fully Developed Engineering Notebooks contain great detail, and will include detailed drawings, tests and test results, solutions to problems the team encountered, and will be a complete record of the design process. Notebook attributes for Fully Developed notebooks will be scored as Emerging, Proficient, and Expert on the Engineering Notebook Rubric. Only Fully Developed Notebooks should be considered for any awards requiring a notebook. The absolute minimum for a notebook to be considered “Fully Developed” would be the first four criteria of the rubric, outlining the initial design process of a single iteration.

Step 2 – Completing the Engineering Notebook Rubric

Fully Developed notebooks will be scored and ranked using the Engineering Notebook Rubric. Roughly the top 10 or top 20% of Fully Developed notebooks, (whichever is greater) will be in consideration for the Design, Excellence and Innovate awards. They may be initially ranked according to their rubric scores, then be re-ranked according to further qualitative evaluation by Judges.

Judges should review the notebook to identify the proficiency level of the student entries for each of the Engineering Notebook Rubric criteria. There will likely not be enough time to do a page-by-page reading of every notebook. Judges should focus on the entries associated with the Rubric criteria and proficiency level to determine the scores for each Fully Developed notebook. It is recommended that at least two judges score each Fully Developed notebook, and the first few notebook scores be discussed so that judges can “calibrate” scores so as to be consistent across the event. Additional judges may review the top scoring notebooks and interview those teams to support final ranking of the notebooks.

Note: The Engineering Notebook Rubric is a tool for initial notebook evaluations to determine eligibility for the Design and Innovate awards through quantitative comparison. The final determination of those award winners are done through further qualitative deliberation among judges.

Continue to the next section, Guide to Judging: Team Interviews